2024-09-05T06:24:15.637 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T06:24:15.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T06:24:15.752 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7889968' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1743 sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 sleep_before_teardown: 0 subset: 459/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0CdzpDBEYEy6TdTrofB0hDfLxsqOH96UaDH9oTiSYNinH25cwSaLFyIuvQjOORA0AcrPsJOBb+S6n5vdeZ/uw= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgZtRcUDSqvQP5amZdDEXyDwLNB6De3Xc9NiWr68ulTB5vhqwBDYrMZAw7NflKgG9bNTSStSbJ3Xw1oUVXaa0Q= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF7l3O4hy1JoHq0yPQ9SCK89+PEBLrsMgNDcLGQKSKlHOkL0P4sRiRmjl832mCyXKpoOVozSYTHdkYh9uWEA1Z0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-04_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T06:24:15.752 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa; will attempt to use it 2024-09-05T06:24:15.753 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks 2024-09-05T06:24:15.753 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T06:24:15.754 INFO:teuthology.task.internal:Checking packages... 2024-09-05T06:24:15.776 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4' 2024-09-05T06:24:15.776 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T06:24:15.776 INFO:teuthology.packaging:ref: None 2024-09-05T06:24:15.776 INFO:teuthology.packaging:tag: None 2024-09-05T06:24:15.776 INFO:teuthology.packaging:branch: squid 2024-09-05T06:24:15.776 INFO:teuthology.packaging:sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:24:15.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-05T06:24:15.972 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-259.g91c8a7e5 2024-09-05T06:24:15.974 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T06:24:15.980 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T06:24:15.980 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T06:24:15.987 INFO:teuthology.task.internal:Saving configuration 2024-09-05T06:24:16.009 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T06:24:16.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T06:24:16.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 06:19:02.362028', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0CdzpDBEYEy6TdTrofB0hDfLxsqOH96UaDH9oTiSYNinH25cwSaLFyIuvQjOORA0AcrPsJOBb+S6n5vdeZ/uw='} 2024-09-05T06:24:16.059 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 06:19:02.360230', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgZtRcUDSqvQP5amZdDEXyDwLNB6De3Xc9NiWr68ulTB5vhqwBDYrMZAw7NflKgG9bNTSStSbJ3Xw1oUVXaa0Q='} 2024-09-05T06:24:16.077 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 06:19:02.363557', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF7l3O4hy1JoHq0yPQ9SCK89+PEBLrsMgNDcLGQKSKlHOkL0P4sRiRmjl832mCyXKpoOVozSYTHdkYh9uWEA1Z0='} 2024-09-05T06:24:16.078 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T06:24:16.084 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-05T06:24:16.084 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-05T06:24:16.084 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-05T06:24:16.084 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T06:24:16.218 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4ceb90bbe0>, signals=[15]) 2024-09-05T06:24:16.218 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T06:24:16.225 INFO:teuthology.task.internal:Opening connections... 2024-09-05T06:24:16.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:24:16.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:24:16.303 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-05T06:24:16.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:24:16.378 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-05T06:24:16.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:24:16.456 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T06:24:16.463 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-09-05T06:24:16.482 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-09-05T06:24:16.482 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-09-05T06:24:16.540 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T06:24:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T06:24:16.542 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-09-05T06:24:16.571 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-05T06:24:16.591 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-05T06:24:16.591 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-05T06:24:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-09-05T06:24:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-09-05T06:24:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-09-05T06:24:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-09-05T06:24:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T06:24:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T06:24:16.650 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-05T06:24:16.670 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-05T06:24:16.689 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-05T06:24:16.689 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-05T06:24:16.747 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-05T06:24:16.748 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T06:24:16.748 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-05T06:24:16.748 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T06:24:16.748 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T06:24:16.748 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T06:24:16.748 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-05T06:24:16.770 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T06:24:16.781 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T06:24:16.788 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T06:24:16.788 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-09-05T06:24:16.790 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-05T06:24:16.793 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-05T06:24:16.810 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T06:24:16.818 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T06:24:16.818 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-09-05T06:24:16.848 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-05T06:24:16.850 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-05T06:24:16.956 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T06:24:17.028 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T06:24:17.042 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-05T06:24:17.043 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-05T06:24:17.043 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T06:24:17.044 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T06:24:17.044 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T06:24:17.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:24:17.061 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-09-05T06:24:17.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:24:17.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:24:17.063 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-05T06:24:17.063 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-05T06:24:17.120 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-503.el9.x86_64 2024-09-05T06:24:17.120 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-503.el9.x86_64 2024-09-05T06:24:17.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-09-05T06:24:17.122 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-05T06:24:17.122 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-503.el9.x86_64 2024-09-05T06:24:17.123 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-05T06:24:17.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-05T06:24:17.123 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-503.el9.x86_64 2024-09-05T06:24:17.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-09-05T06:24:19.379 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-05T06:24:19.388 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-05T06:24:21.173 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-05T06:24:21.252 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.2 MB 00:02 2024-09-05T06:24:22.580 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-05T06:24:24.111 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-05T06:24:27.815 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-05T06:24:29.304 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:02 2024-09-05T06:24:29.790 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-05T06:24:30.903 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-05T06:24:30.921 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-05T06:24:31.759 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-05T06:24:36.442 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-05T06:24:37.239 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-05T06:24:38.399 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T06:24:38.399 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T06:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T06:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T06:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T06:24:38.602 DEBUG:teuthology.orchestra.run.smithi086:> echo no | sudo yum reinstall kernel || true 2024-09-05T06:24:39.137 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 06:24:36 AM UTC. 2024-09-05T06:24:39.212 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T06:24:39.212 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T06:24:39.277 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T06:24:39.288 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-05T06:24:39.289 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-05T06:24:39.289 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T06:24:39.290 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:39.291 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:39.291 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 1.9 M 2024-09-05T06:24:39.291 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-09-05T06:24:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:Operation aborted. 2024-09-05T06:24:39.343 INFO:teuthology.orchestra.run.smithi086.stdout:Is this ok [y/N]: 2024-09-05T06:24:39.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum reinstall -y kernel || true 2024-09-05T06:24:39.400 DEBUG:teuthology.orchestra.run.smithi096:> echo no | sudo yum reinstall kernel || true 2024-09-05T06:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 06:24:36 AM UTC. 2024-09-05T06:24:39.940 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:02 ago on Thu 05 Sep 2024 06:24:37 AM UTC. 2024-09-05T06:24:40.002 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T06:24:40.004 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:40.004 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:40.004 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:40.004 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-09-05T06:24:40.004 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 1.9 M 2024-09-05T06:24:40.005 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-09-05T06:24:40.006 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T06:24:40.077 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:40.089 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 1.9 M 2024-09-05T06:24:40.090 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-09-05T06:24:40.090 INFO:teuthology.orchestra.run.smithi096.stderr:Operation aborted. 2024-09-05T06:24:40.145 INFO:teuthology.orchestra.run.smithi096.stdout:Is this ok [y/N]: 2024-09-05T06:24:40.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum reinstall -y kernel || true 2024-09-05T06:24:40.386 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-503.el9.x86_64.rpm 7.0 MB/s | 1.9 MB 00:00 2024-09-05T06:24:40.386 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:24:40.386 INFO:teuthology.orchestra.run.smithi086.stdout:Total 4.9 MB/s | 1.9 MB 00:00 2024-09-05T06:24:40.414 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T06:24:40.653 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T06:24:40.654 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T06:24:40.678 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 06:24:37 AM UTC. 2024-09-05T06:24:40.684 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T06:24:40.684 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T06:24:40.804 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-05T06:24:40.805 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.805 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:40.805 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 1.9 M 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-09-05T06:24:40.806 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-05T06:24:40.851 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T06:24:40.955 INFO:teuthology.orchestra.run.smithi086.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:40.955 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:41.665 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-503.el9.x86_64.rpm 3.1 MB/s | 1.9 MB 00:00 2024-09-05T06:24:41.666 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:24:41.666 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.2 MB/s | 1.9 MB 00:00 2024-09-05T06:24:41.695 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-05T06:24:41.940 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-05T06:24:41.940 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-05T06:24:41.971 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-05T06:24:41.971 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-05T06:24:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-05T06:24:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalled: 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T06:24:42.592 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T06:24:42.690 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T06:24:42.691 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T06:24:42.691 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T06:24:42.691 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-05T06:24:42.691 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T06:24:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-09-05T06:24:42.745 DEBUG:teuthology.parallel:result is None 2024-09-05T06:24:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalled: 2024-09-05T06:24:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T06:24:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:24:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-05T06:24:43.853 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T06:24:43.912 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T06:24:43.912 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T06:24:43.912 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T06:24:43.912 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-05T06:24:43.912 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T06:24:43.942 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-09-05T06:24:43.971 DEBUG:teuthology.parallel:result is None 2024-09-05T06:24:46.258 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 363 kB/s | 6.5 MB 00:18 2024-09-05T06:24:47.791 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-05T06:24:48.612 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-05T06:24:54.051 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-05T06:24:56.041 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T06:24:56.041 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T06:24:56.120 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-05T06:24:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-05T06:24:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-05T06:24:56.226 DEBUG:teuthology.orchestra.run.smithi115:> echo no | sudo yum reinstall kernel || true 2024-09-05T06:24:56.761 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:02 ago on Thu 05 Sep 2024 06:24:54 AM UTC. 2024-09-05T06:24:56.896 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-05T06:24:56.901 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:56.901 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:56.901 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout:Reinstalling: 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 1.9 M 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 0 2024-09-05T06:24:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:Operation aborted. 2024-09-05T06:24:56.956 INFO:teuthology.orchestra.run.smithi115.stdout:Is this ok [y/N]: 2024-09-05T06:24:56.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum reinstall -y kernel || true 2024-09-05T06:24:57.491 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 06:24:54 AM UTC. 2024-09-05T06:24:57.616 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout:Reinstalling: 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:57.617 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-05T06:24:57.618 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-05T06:24:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:57.618 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 1.9 M 2024-09-05T06:24:57.618 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 0 2024-09-05T06:24:57.618 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-05T06:24:58.103 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-503.el9.x86_64.rpm 4.0 MB/s | 1.9 MB 00:00 2024-09-05T06:24:58.103 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:24:58.103 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.9 MB/s | 1.9 MB 00:00 2024-09-05T06:24:58.131 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-05T06:24:58.363 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-05T06:24:58.363 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-05T06:24:58.393 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-05T06:24:58.393 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-05T06:24:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-05T06:24:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout:Reinstalled: 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:24:59.923 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-05T06:25:00.136 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T06:25:00.236 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T06:25:00.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T06:25:00.236 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T06:25:00.237 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-05T06:25:00.237 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T06:25:00.266 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-05T06:25:00.285 DEBUG:teuthology.parallel:result is None 2024-09-05T06:25:00.286 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T06:25:00.294 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T06:25:00.295 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T06:25:00.298 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T06:25:00.301 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T06:25:00.340 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T06:25:00.347 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T06:25:00.358 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T06:25:00.358 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T06:25:00.361 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T06:25:00.363 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T06:25:00.424 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T06:25:00.431 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T06:25:00.431 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T06:25:00.441 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T06:25:00.443 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T06:25:00.492 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.493 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.499 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.504 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.504 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.509 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T06:25:00.510 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T06:25:00.517 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T06:25:00.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T06:25:00.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T06:25:00.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T06:25:00.546 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T06:25:00.556 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T06:25:00.556 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T06:25:00.588 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T06:25:00.590 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T06:25:00.605 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:00.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:00.731 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:00.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:00.855 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:25:00.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T06:25:00.921 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:00.965 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:01.013 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:01.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:01.146 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:25:01.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T06:25:01.214 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:01.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T06:25:01.310 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:01.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T06:25:01.432 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:25:01.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T06:25:01.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-09-05T06:25:01.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-05T06:25:01.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-05T06:25:01.540 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T06:25:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T06:25:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T06:25:02.028 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T06:25:02.036 INFO:teuthology.task.internal:Starting timer... 2024-09-05T06:25:02.036 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T06:25:02.067 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T06:25:02.075 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-05T06:25:02.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-09-05T06:25:02.116 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-09-05T06:25:02.118 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-09-05T06:25:02.158 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-09-05T06:25:02.160 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-05T06:25:02.208 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-05T06:25:02.210 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T06:25:02.210 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-09-05T06:25:02.235 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-09-05T06:25:02.235 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-09-05T06:25:02.255 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-09-05T06:25:02.256 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-05T06:25:02.282 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-05T06:25:02.282 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2024-09-05T06:25:02.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T06:25:02.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:25:02.310 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T06:25:02.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:25:02.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T06:25:02.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T06:25:02.363 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T06:25:02.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-09-05T06:25:02.397 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-09-05T06:25:02.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-05T06:25:02.466 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T06:25:02.476 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-05T06:25:02.494 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-05T06:25:02.601 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T06:25:02.618 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T06:25:02.619 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T06:28:43.651 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2024-09-05T06:28:43.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-05T06:28:43.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:28:43.730 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-09-05T06:28:43.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-05T06:28:43.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-05T06:28:43.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:28:43.897 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-05T06:28:43.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-05T06:28:43.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-05T06:28:43.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T06:28:44.061 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-05T06:28:44.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-05T06:28:44.148 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T06:28:44.163 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T06:28:44.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T06:28:44.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T06:28:44.168 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T06:28:44.168 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T06:28:44.171 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T06:28:44.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T06:28:44.194 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T06:28:44.200 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T06:28:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T06:28:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T06:28:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T06:28:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-09-05T06:28:44.236 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T06:28:44.240 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-09-05T06:28:44.244 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-09-05T06:28:44.252 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-09-05T06:28:44.258 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T06:28:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-05T06:28:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T06:28:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T06:28:44.274 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-05T06:28:44.280 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T06:28:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T06:28:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T06:28:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.336 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-05T06:28:44.339 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T06:28:44.340 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-05T06:28:44.340 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.340 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.340 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.341 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.358 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-05T06:28:44.361 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T06:28:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-05T06:28:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T06:28:44.362 INFO:teuthology.run_tasks:Running task install... 2024-09-05T06:28:44.371 DEBUG:teuthology.task.install:project ceph 2024-09-05T06:28:44.371 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'}} 2024-09-05T06:28:44.371 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'} 2024-09-05T06:28:44.371 INFO:teuthology.task.install:Using flavor: default 2024-09-05T06:28:44.377 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T06:28:44.377 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-05T06:28:44.377 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T06:28:44.378 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:28:44.380 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T06:28:44.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:28:44.381 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T06:28:44.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:28:44.529 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T06:28:44.529 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T06:28:44.536 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T06:28:44.536 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T06:28:44.547 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T06:28:44.547 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T06:28:44.641 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T06:28:44.641 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:28:44.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T06:28:44.648 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T06:28:44.648 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:28:44.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T06:28:44.656 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T06:28:44.657 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:28:44.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T06:28:44.667 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T06:28:44.667 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T06:28:44.674 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T06:28:44.674 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T06:28:44.682 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T06:28:44.682 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T06:28:44.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T06:28:44.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T06:28:44.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T06:28:44.808 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T06:28:44.815 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T06:28:44.824 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T06:28:44.872 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-09-05T06:28:44.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-09-05T06:28:44.880 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-05T06:28:44.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-05T06:28:44.888 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-09-05T06:28:44.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-09-05T06:28:45.097 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-09-05T06:28:45.104 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-05T06:28:45.111 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-09-05T06:28:45.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T06:28:45.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T06:28:45.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T06:28:45.665 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 328 kB/s | 84 kB 00:00 2024-09-05T06:28:45.670 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-05T06:28:45.670 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-05T06:28:45.941 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-05T06:28:45.954 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-05T06:28:45.954 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-05T06:28:46.216 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T06:28:46.221 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-05T06:28:46.230 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T06:28:46.813 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-05T06:28:48.125 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-05T06:28:48.446 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.2 MB 00:02 2024-09-05T06:28:48.664 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-05T06:28:50.118 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-05T06:28:53.198 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-05T06:28:54.352 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-05T06:28:54.874 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-05T06:28:55.686 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-05T06:28:55.844 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-05T06:28:56.723 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-05T06:29:01.108 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-05T06:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-05T06:29:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:03.397 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T06:29:03.397 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T06:29:03.398 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T06:29:03.399 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T06:29:03.401 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T06:29:03.403 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T06:29:03.404 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T06:29:03.466 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T06:29:03.475 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T06:29:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T06:29:03.477 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T06:29:03.478 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T06:29:03.479 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T06:29:03.480 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T06:29:03.481 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T06:29:03.482 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T06:29:03.483 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T06:29:03.484 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T06:29:03.485 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T06:29:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T06:29:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout:Install 148 Packages 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 6 Packages 2024-09-05T06:29:03.488 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:03.489 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 222 M 2024-09-05T06:29:03.489 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-05T06:29:04.076 INFO:teuthology.orchestra.run.smithi115.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-05T06:29:04.268 INFO:teuthology.orchestra.run.smithi115.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 4.6 MB/s | 896 kB 00:00 2024-09-05T06:29:04.280 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:04.281 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:04.293 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T06:29:04.293 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T06:29:04.294 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T06:29:04.295 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T06:29:04.297 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T06:29:04.299 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T06:29:04.300 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T06:29:04.335 INFO:teuthology.orchestra.run.smithi115.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 2.1 MB/s | 145 kB 00:00 2024-09-05T06:29:04.362 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-05T06:29:04.370 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-05T06:29:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T06:29:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T06:29:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T06:29:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T06:29:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T06:29:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T06:29:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T06:29:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T06:29:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T06:29:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T06:29:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T06:29:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T06:29:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T06:29:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T06:29:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T06:29:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T06:29:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T06:29:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 6 Packages 2024-09-05T06:29:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:04.385 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 222 M 2024-09-05T06:29:04.385 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-05T06:29:04.535 INFO:teuthology.orchestra.run.smithi115.stdout:(4/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-05T06:29:04.661 INFO:teuthology.orchestra.run.smithi115.stdout:(5/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-05T06:29:04.880 INFO:teuthology.orchestra.run.smithi096.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-05T06:29:04.903 INFO:teuthology.orchestra.run.smithi115.stdout:(6/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 22 MB/s | 21 MB 00:00 2024-09-05T06:29:05.028 INFO:teuthology.orchestra.run.smithi115.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-05T06:29:05.103 INFO:teuthology.orchestra.run.smithi115.stdout:(8/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 4.7 MB/s | 5.4 MB 00:01 2024-09-05T06:29:05.153 INFO:teuthology.orchestra.run.smithi115.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 505 kB/s | 25 kB 00:00 2024-09-05T06:29:05.405 INFO:teuthology.orchestra.run.smithi096.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 896 kB 00:00 2024-09-05T06:29:05.539 INFO:teuthology.orchestra.run.smithi096.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 1.1 MB/s | 145 kB 00:00 2024-09-05T06:29:05.695 INFO:teuthology.orchestra.run.smithi115.stdout:(10/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 21 MB/s | 17 MB 00:00 2024-09-05T06:29:05.781 INFO:teuthology.orchestra.run.smithi096.stdout:(4/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-09-05T06:29:05.804 INFO:teuthology.orchestra.run.smithi115.stdout:(11/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 14 MB/s | 11 MB 00:00 2024-09-05T06:29:05.829 INFO:teuthology.orchestra.run.smithi115.stdout:(12/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 1.6 MB/s | 224 kB 00:00 2024-09-05T06:29:05.846 INFO:teuthology.orchestra.run.smithi115.stdout:(13/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 649 kB/s | 27 kB 00:00 2024-09-05T06:29:05.863 INFO:teuthology.orchestra.run.smithi115.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 942 kB/s | 32 kB 00:00 2024-09-05T06:29:05.913 INFO:teuthology.orchestra.run.smithi115.stdout:(15/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 3.2 MB/s | 163 kB 00:00 2024-09-05T06:29:05.938 INFO:teuthology.orchestra.run.smithi115.stdout:(16/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-05T06:29:05.955 INFO:teuthology.orchestra.run.smithi115.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 3.0 MB/s | 127 kB 00:00 2024-09-05T06:29:05.964 INFO:teuthology.orchestra.run.smithi096.stdout:(5/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 8.3 MB/s | 1.5 MB 00:00 2024-09-05T06:29:05.989 INFO:teuthology.orchestra.run.smithi115.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 9.9 MB/s | 506 kB 00:00 2024-09-05T06:29:06.056 INFO:teuthology.orchestra.run.smithi115.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 678 kB/s | 45 kB 00:00 2024-09-05T06:29:06.123 INFO:teuthology.orchestra.run.smithi115.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 2.0 MB/s | 136 kB 00:00 2024-09-05T06:29:06.198 INFO:teuthology.orchestra.run.smithi115.stdout:(21/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 22 MB/s | 5.3 MB 00:00 2024-09-05T06:29:06.240 INFO:teuthology.orchestra.run.smithi115.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 1.3 MB/s | 162 kB 00:00 2024-09-05T06:29:06.264 INFO:teuthology.orchestra.run.smithi096.stdout:(6/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-09-05T06:29:06.265 INFO:teuthology.orchestra.run.smithi115.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 4.7 MB/s | 321 kB 00:00 2024-09-05T06:29:06.290 INFO:teuthology.orchestra.run.smithi115.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 5.8 MB/s | 301 kB 00:00 2024-09-05T06:29:06.316 INFO:teuthology.orchestra.run.smithi115.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.9 MB/s | 99 kB 00:00 2024-09-05T06:29:06.341 INFO:teuthology.orchestra.run.smithi115.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T06:29:06.408 INFO:teuthology.orchestra.run.smithi115.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-05T06:29:06.484 INFO:teuthology.orchestra.run.smithi115.stdout:(28/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-05T06:29:06.509 INFO:teuthology.orchestra.run.smithi115.stdout:(29/154): ceph-grafana-dashboards-19.1.1-259.g9 316 kB/s | 32 kB 00:00 2024-09-05T06:29:06.512 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:16 2024-09-05T06:29:06.534 INFO:teuthology.orchestra.run.smithi115.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.8 MB/s | 146 kB 00:00 2024-09-05T06:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 8.1 MB/s | 4.7 MB 00:00 2024-09-05T06:29:06.743 INFO:teuthology.orchestra.run.smithi115.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 15 MB/s | 3.6 MB 00:00 2024-09-05T06:29:06.810 INFO:teuthology.orchestra.run.smithi115.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 3.7 MB/s | 249 kB 00:00 2024-09-05T06:29:06.902 INFO:teuthology.orchestra.run.smithi115.stdout:(33/154): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-05T06:29:06.927 INFO:teuthology.orchestra.run.smithi115.stdout:(34/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 421 kB/s | 49 kB 00:00 2024-09-05T06:29:06.952 INFO:teuthology.orchestra.run.smithi115.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 327 kB/s | 16 kB 00:00 2024-09-05T06:29:06.977 INFO:teuthology.orchestra.run.smithi115.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 5.6 MB/s | 285 kB 00:00 2024-09-05T06:29:07.011 INFO:teuthology.orchestra.run.smithi115.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 13 MB/s | 765 kB 00:00 2024-09-05T06:29:07.036 INFO:teuthology.orchestra.run.smithi115.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 310 kB/s | 18 kB 00:00 2024-09-05T06:29:07.178 INFO:teuthology.orchestra.run.smithi115.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-05T06:29:07.262 INFO:teuthology.orchestra.run.smithi115.stdout:(40/154): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-05T06:29:07.329 INFO:teuthology.orchestra.run.smithi115.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-05T06:29:07.354 INFO:teuthology.orchestra.run.smithi115.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-05T06:29:07.405 INFO:teuthology.orchestra.run.smithi115.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-05T06:29:07.438 INFO:teuthology.orchestra.run.smithi115.stdout:(44/154): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-05T06:29:07.463 INFO:teuthology.orchestra.run.smithi115.stdout:(45/154): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-05T06:29:07.489 INFO:teuthology.orchestra.run.smithi115.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-05T06:29:07.522 INFO:teuthology.orchestra.run.smithi115.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-05T06:29:07.556 INFO:teuthology.orchestra.run.smithi096.stdout:(8/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9. 11 MB/s | 11 MB 00:01 2024-09-05T06:29:07.607 INFO:teuthology.orchestra.run.smithi096.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 505 kB/s | 25 kB 00:00 2024-09-05T06:29:07.665 INFO:teuthology.orchestra.run.smithi115.stdout:(48/154): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-09-05T06:29:07.689 INFO:teuthology.orchestra.run.smithi115.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-05T06:29:07.748 INFO:teuthology.orchestra.run.smithi115.stdout:(50/154): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-05T06:29:07.773 INFO:teuthology.orchestra.run.smithi115.stdout:(51/154): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 107 kB 00:00 2024-09-05T06:29:07.816 INFO:teuthology.orchestra.run.smithi115.stdout:(52/154): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-05T06:29:07.841 INFO:teuthology.orchestra.run.smithi115.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-05T06:29:07.908 INFO:teuthology.orchestra.run.smithi115.stdout:(54/154): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-09-05T06:29:07.967 INFO:teuthology.orchestra.run.smithi115.stdout:(55/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-05T06:29:07.991 INFO:teuthology.orchestra.run.smithi115.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-05T06:29:08.051 INFO:teuthology.orchestra.run.smithi115.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-05T06:29:08.076 INFO:teuthology.orchestra.run.smithi115.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 539 kB/s | 45 kB 00:00 2024-09-05T06:29:08.117 INFO:teuthology.orchestra.run.smithi115.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-05T06:29:08.160 INFO:teuthology.orchestra.run.smithi115.stdout:(60/154): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-05T06:29:08.174 INFO:teuthology.orchestra.run.smithi096.stdout:(10/154): ceph-common-19.1.1-259.g91c8a7e5.el9. 6.2 MB/s | 21 MB 00:03 2024-09-05T06:29:08.185 INFO:teuthology.orchestra.run.smithi115.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-05T06:29:08.224 INFO:teuthology.orchestra.run.smithi096.stdout:(11/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 4.4 MB/s | 224 kB 00:00 2024-09-05T06:29:08.235 INFO:teuthology.orchestra.run.smithi115.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-05T06:29:08.260 INFO:teuthology.orchestra.run.smithi115.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-05T06:29:08.266 INFO:teuthology.orchestra.run.smithi096.stdout:(12/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 650 kB/s | 27 kB 00:00 2024-09-05T06:29:08.294 INFO:teuthology.orchestra.run.smithi115.stdout:(64/154): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-05T06:29:08.299 INFO:teuthology.orchestra.run.smithi096.stdout:(13/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 949 kB/s | 32 kB 00:00 2024-09-05T06:29:08.319 INFO:teuthology.orchestra.run.smithi115.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-05T06:29:08.417 INFO:teuthology.orchestra.run.smithi096.stdout:(14/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 6.3 MB/s | 752 kB 00:00 2024-09-05T06:29:08.445 INFO:teuthology.orchestra.run.smithi115.stdout:(66/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-05T06:29:08.467 INFO:teuthology.orchestra.run.smithi096.stdout:(15/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 3.2 MB/s | 163 kB 00:00 2024-09-05T06:29:08.517 INFO:teuthology.orchestra.run.smithi096.stdout:(16/154): librados-devel-19.1.1-259.g91c8a7e5.e 2.5 MB/s | 127 kB 00:00 2024-09-05T06:29:08.520 INFO:teuthology.orchestra.run.smithi115.stdout:(67/154): openblas-openmp-0.3.26-2.el9.x86_64.r 22 MB/s | 4.9 MB 00:00 2024-09-05T06:29:08.562 INFO:teuthology.orchestra.run.smithi115.stdout:(68/154): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-05T06:29:08.609 INFO:teuthology.orchestra.run.smithi096.stdout:(17/154): libradosstriper1-19.1.1-259.g91c8a7e5 5.4 MB/s | 506 kB 00:00 2024-09-05T06:29:08.612 INFO:teuthology.orchestra.run.smithi115.stdout:(69/154): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-05T06:29:08.637 INFO:teuthology.orchestra.run.smithi115.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-05T06:29:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:(71/154): postgresql-server-13.16-1.el9.x86_64. 41 MB/s | 5.8 MB 00:00 2024-09-05T06:29:08.788 INFO:teuthology.orchestra.run.smithi115.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-09-05T06:29:08.830 INFO:teuthology.orchestra.run.smithi115.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-05T06:29:08.939 INFO:teuthology.orchestra.run.smithi115.stdout:(74/154): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-05T06:29:08.964 INFO:teuthology.orchestra.run.smithi115.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 458 kB/s | 61 kB 00:00 2024-09-05T06:29:08.998 INFO:teuthology.orchestra.run.smithi115.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-05T06:29:09.031 INFO:teuthology.orchestra.run.smithi115.stdout:(77/154): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-05T06:29:09.065 INFO:teuthology.orchestra.run.smithi115.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-09-05T06:29:09.099 INFO:teuthology.orchestra.run.smithi115.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-09-05T06:29:09.126 INFO:teuthology.orchestra.run.smithi115.stdout:(80/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 172 kB 00:00 2024-09-05T06:29:09.169 INFO:teuthology.orchestra.run.smithi096.stdout:(18/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 9.6 MB/s | 5.3 MB 00:00 2024-09-05T06:29:09.176 INFO:teuthology.orchestra.run.smithi115.stdout:(81/154): python3-markupsafe-1.1.1-12.el9.x86_6 450 kB/s | 35 kB 00:00 2024-09-05T06:29:09.218 INFO:teuthology.orchestra.run.smithi096.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 909 kB/s | 45 kB 00:00 2024-09-05T06:29:09.268 INFO:teuthology.orchestra.run.smithi115.stdout:(82/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-05T06:29:09.302 INFO:teuthology.orchestra.run.smithi115.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 250 kB 00:00 2024-09-05T06:29:09.327 INFO:teuthology.orchestra.run.smithi115.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-05T06:29:09.360 INFO:teuthology.orchestra.run.smithi115.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-05T06:29:09.393 INFO:teuthology.orchestra.run.smithi096.stdout:(20/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 5.4 MB/s | 17 MB 00:03 2024-09-05T06:29:09.394 INFO:teuthology.orchestra.run.smithi115.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-09-05T06:29:09.427 INFO:teuthology.orchestra.run.smithi096.stdout:(21/154): python3-ceph-common-19.1.1-259.g91c8a 650 kB/s | 136 kB 00:00 2024-09-05T06:29:09.436 INFO:teuthology.orchestra.run.smithi115.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-05T06:29:09.452 INFO:teuthology.orchestra.run.smithi096.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 2.7 MB/s | 162 kB 00:00 2024-09-05T06:29:09.461 INFO:teuthology.orchestra.run.smithi115.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-05T06:29:09.494 INFO:teuthology.orchestra.run.smithi096.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 4.7 MB/s | 321 kB 00:00 2024-09-05T06:29:09.511 INFO:teuthology.orchestra.run.smithi115.stdout:(89/154): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-05T06:29:09.537 INFO:teuthology.orchestra.run.smithi115.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-05T06:29:09.544 INFO:teuthology.orchestra.run.smithi096.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 3.2 MB/s | 301 kB 00:00 2024-09-05T06:29:09.570 INFO:teuthology.orchestra.run.smithi096.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.3 MB/s | 99 kB 00:00 2024-09-05T06:29:09.579 INFO:teuthology.orchestra.run.smithi115.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 803 kB/s | 54 kB 00:00 2024-09-05T06:29:09.595 INFO:teuthology.orchestra.run.smithi096.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T06:29:09.646 INFO:teuthology.orchestra.run.smithi096.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-05T06:29:09.646 INFO:teuthology.orchestra.run.smithi115.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-05T06:29:09.696 INFO:teuthology.orchestra.run.smithi096.stdout:(28/154): ceph-grafana-dashboards-19.1.1-259.g9 636 kB/s | 32 kB 00:00 2024-09-05T06:29:09.705 INFO:teuthology.orchestra.run.smithi115.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-05T06:29:09.747 INFO:teuthology.orchestra.run.smithi096.stdout:(29/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.9 MB/s | 146 kB 00:00 2024-09-05T06:29:09.872 INFO:teuthology.orchestra.run.smithi115.stdout:(94/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-09-05T06:29:09.896 INFO:teuthology.orchestra.run.smithi115.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-05T06:29:09.930 INFO:teuthology.orchestra.run.smithi115.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-05T06:29:09.938 INFO:teuthology.orchestra.run.smithi096.stdout:(30/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 8.4 MB/s | 3.1 MB 00:00 2024-09-05T06:29:09.955 INFO:teuthology.orchestra.run.smithi115.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-05T06:29:10.247 INFO:teuthology.orchestra.run.smithi096.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 7.1 MB/s | 3.6 MB 00:00 2024-09-05T06:29:10.314 INFO:teuthology.orchestra.run.smithi096.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 3.7 MB/s | 249 kB 00:00 2024-09-05T06:29:10.364 INFO:teuthology.orchestra.run.smithi096.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 990 kB/s | 49 kB 00:00 2024-09-05T06:29:10.381 INFO:teuthology.orchestra.run.smithi115.stdout:(98/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 9.3 MB/s | 49 MB 00:05 2024-09-05T06:29:10.408 INFO:teuthology.orchestra.run.smithi115.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 128 kB/s | 58 kB 00:00 2024-09-05T06:29:10.414 INFO:teuthology.orchestra.run.smithi096.stdout:(34/154): ceph-prometheus-alerts-19.1.1-259.g91 330 kB/s | 16 kB 00:00 2024-09-05T06:29:10.431 INFO:teuthology.orchestra.run.smithi115.stdout:(100/154): socat-1.7.4.1-6.el9.x86_64.rpm 607 kB/s | 304 kB 00:00 2024-09-05T06:29:10.474 INFO:teuthology.orchestra.run.smithi115.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-09-05T06:29:10.481 INFO:teuthology.orchestra.run.smithi096.stdout:(35/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 4.2 MB/s | 285 kB 00:00 2024-09-05T06:29:10.498 INFO:teuthology.orchestra.run.smithi115.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-05T06:29:10.549 INFO:teuthology.orchestra.run.smithi115.stdout:(103/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 572 kB/s | 96 kB 00:00 2024-09-05T06:29:10.574 INFO:teuthology.orchestra.run.smithi115.stdout:(104/154): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-05T06:29:10.607 INFO:teuthology.orchestra.run.smithi096.stdout:(36/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-05T06:29:10.624 INFO:teuthology.orchestra.run.smithi115.stdout:(105/154): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-05T06:29:10.650 INFO:teuthology.orchestra.run.smithi115.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-05T06:29:10.657 INFO:teuthology.orchestra.run.smithi096.stdout:(37/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 367 kB/s | 18 kB 00:00 2024-09-05T06:29:10.675 INFO:teuthology.orchestra.run.smithi115.stdout:(107/154): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-05T06:29:10.700 INFO:teuthology.orchestra.run.smithi115.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T06:29:10.725 INFO:teuthology.orchestra.run.smithi115.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-05T06:29:10.749 INFO:teuthology.orchestra.run.smithi096.stdout:(38/154): ceph-mgr-diskprediction-local-19.1.1- 9.1 MB/s | 7.4 MB 00:00 2024-09-05T06:29:10.767 INFO:teuthology.orchestra.run.smithi115.stdout:(110/154): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-05T06:29:10.793 INFO:teuthology.orchestra.run.smithi115.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-05T06:29:10.843 INFO:teuthology.orchestra.run.smithi115.stdout:(112/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 19 MB/s | 839 kB 00:00 2024-09-05T06:29:10.868 INFO:teuthology.orchestra.run.smithi115.stdout:(113/154): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-05T06:29:10.899 INFO:teuthology.orchestra.run.smithi096.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 330 kB/s | 80 kB 00:00 2024-09-05T06:29:10.902 INFO:teuthology.orchestra.run.smithi115.stdout:(114/154): python3-asyncssh-2.13.2-5.el9.noarch 9.1 MB/s | 548 kB 00:00 2024-09-05T06:29:10.927 INFO:teuthology.orchestra.run.smithi115.stdout:(115/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-05T06:29:10.952 INFO:teuthology.orchestra.run.smithi115.stdout:(116/154): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-05T06:29:10.955 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-05T06:29:10.978 INFO:teuthology.orchestra.run.smithi115.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-05T06:29:10.983 INFO:teuthology.orchestra.run.smithi096.stdout:(40/154): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-05T06:29:11.003 INFO:teuthology.orchestra.run.smithi115.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-05T06:29:11.017 INFO:teuthology.orchestra.run.smithi096.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-05T06:29:11.037 INFO:teuthology.orchestra.run.smithi115.stdout:(119/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-05T06:29:11.042 INFO:teuthology.orchestra.run.smithi096.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-05T06:29:11.062 INFO:teuthology.orchestra.run.smithi115.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-05T06:29:11.093 INFO:teuthology.orchestra.run.smithi096.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-05T06:29:11.095 INFO:teuthology.orchestra.run.smithi115.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-05T06:29:11.121 INFO:teuthology.orchestra.run.smithi115.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-05T06:29:11.126 INFO:teuthology.orchestra.run.smithi096.stdout:(44/154): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-05T06:29:11.138 INFO:teuthology.orchestra.run.smithi115.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 650 kB/s | 11 kB 00:00 2024-09-05T06:29:11.151 INFO:teuthology.orchestra.run.smithi096.stdout:(45/154): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-05T06:29:11.177 INFO:teuthology.orchestra.run.smithi096.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-05T06:29:11.196 INFO:teuthology.orchestra.run.smithi115.stdout:(124/154): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-09-05T06:29:11.213 INFO:teuthology.orchestra.run.smithi115.stdout:(125/154): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-09-05T06:29:11.219 INFO:teuthology.orchestra.run.smithi096.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-05T06:29:11.239 INFO:teuthology.orchestra.run.smithi115.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 555 kB/s | 23 kB 00:00 2024-09-05T06:29:11.264 INFO:teuthology.orchestra.run.smithi115.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-05T06:29:11.289 INFO:teuthology.orchestra.run.smithi115.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-05T06:29:11.315 INFO:teuthology.orchestra.run.smithi115.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T06:29:11.348 INFO:teuthology.orchestra.run.smithi115.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-05T06:29:11.361 INFO:teuthology.orchestra.run.smithi096.stdout:(48/154): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-09-05T06:29:11.373 INFO:teuthology.orchestra.run.smithi115.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T06:29:11.386 INFO:teuthology.orchestra.run.smithi096.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-05T06:29:11.407 INFO:teuthology.orchestra.run.smithi115.stdout:(132/154): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-05T06:29:11.449 INFO:teuthology.orchestra.run.smithi115.stdout:(133/154): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-05T06:29:11.474 INFO:teuthology.orchestra.run.smithi115.stdout:(134/154): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-09-05T06:29:11.499 INFO:teuthology.orchestra.run.smithi115.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-05T06:29:11.516 INFO:teuthology.orchestra.run.smithi115.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-05T06:29:11.533 INFO:teuthology.orchestra.run.smithi115.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 1.8 MB/s | 31 kB 00:00 2024-09-05T06:29:11.550 INFO:teuthology.orchestra.run.smithi115.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-05T06:29:11.575 INFO:teuthology.orchestra.run.smithi115.stdout:(139/154): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-05T06:29:11.601 INFO:teuthology.orchestra.run.smithi115.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-05T06:29:11.643 INFO:teuthology.orchestra.run.smithi115.stdout:(141/154): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-09-05T06:29:11.677 INFO:teuthology.orchestra.run.smithi115.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 738 kB/s | 74 kB 00:00 2024-09-05T06:29:11.693 INFO:teuthology.orchestra.run.smithi115.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T06:29:11.719 INFO:teuthology.orchestra.run.smithi115.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-05T06:29:11.744 INFO:teuthology.orchestra.run.smithi115.stdout:(145/154): python3-webob-1.8.8-2.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-09-05T06:29:11.769 INFO:teuthology.orchestra.run.smithi115.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T06:29:11.795 INFO:teuthology.orchestra.run.smithi115.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-05T06:29:11.828 INFO:teuthology.orchestra.run.smithi096.stdout:(50/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 12 MB/s | 49 MB 00:04 2024-09-05T06:29:11.845 INFO:teuthology.orchestra.run.smithi115.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-05T06:29:11.853 INFO:teuthology.orchestra.run.smithi096.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 46 kB/s | 22 kB 00:00 2024-09-05T06:29:11.879 INFO:teuthology.orchestra.run.smithi096.stdout:(52/154): boost-program-options-1.75.0-8.el9.x8 206 kB/s | 107 kB 00:00 2024-09-05T06:29:11.912 INFO:teuthology.orchestra.run.smithi096.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-05T06:29:11.996 INFO:teuthology.orchestra.run.smithi096.stdout:(54/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-09-05T06:29:12.021 INFO:teuthology.orchestra.run.smithi096.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-05T06:29:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:(56/154): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-09-05T06:29:12.072 INFO:teuthology.orchestra.run.smithi096.stdout:(57/154): libaio-devel-0.3.111-13.el9.x86_64.rp 147 kB/s | 11 kB 00:00 2024-09-05T06:29:12.078 INFO:teuthology.orchestra.run.smithi115.stdout:(149/154): librados2-19.1.1-259.g91c8a7e5.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-05T06:29:12.097 INFO:teuthology.orchestra.run.smithi096.stdout:(58/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-05T06:29:12.122 INFO:teuthology.orchestra.run.smithi096.stdout:(59/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-05T06:29:12.154 INFO:teuthology.orchestra.run.smithi115.stdout:(150/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-05T06:29:12.173 INFO:teuthology.orchestra.run.smithi096.stdout:(60/154): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-05T06:29:12.204 INFO:teuthology.orchestra.run.smithi115.stdout:(151/154): NetworkManager-1.48.10-1.el9.x86_64. 6.5 MB/s | 2.3 MB 00:00 2024-09-05T06:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout:(61/154): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-05T06:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout:(62/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 224 kB/s | 24 kB 00:00 2024-09-05T06:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-05T06:29:12.262 INFO:teuthology.orchestra.run.smithi115.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-09-05T06:29:12.288 INFO:teuthology.orchestra.run.smithi115.stdout:(153/154): NetworkManager-team-1.48.10-1.el9.x8 303 kB/s | 40 kB 00:00 2024-09-05T06:29:12.299 INFO:teuthology.orchestra.run.smithi096.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-05T06:29:12.324 INFO:teuthology.orchestra.run.smithi096.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-09-05T06:29:12.349 INFO:teuthology.orchestra.run.smithi096.stdout:(66/154): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-05T06:29:12.363 INFO:teuthology.orchestra.run.smithi115.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 6.3 MB/s | 1.8 MB 00:00 2024-09-05T06:29:12.368 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:29:12.369 INFO:teuthology.orchestra.run.smithi115.stdout:Total 25 MB/s | 222 MB 00:08 2024-09-05T06:29:12.441 INFO:teuthology.orchestra.run.smithi096.stdout:(67/154): postgresql-13.16-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-05T06:29:12.508 INFO:teuthology.orchestra.run.smithi096.stdout:(68/154): openblas-openmp-0.3.26-2.el9.x86_64.r 24 MB/s | 4.9 MB 00:00 2024-09-05T06:29:12.567 INFO:teuthology.orchestra.run.smithi096.stdout:(69/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-05T06:29:12.592 INFO:teuthology.orchestra.run.smithi096.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-05T06:29:12.634 INFO:teuthology.orchestra.run.smithi096.stdout:(71/154): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-05T06:29:12.676 INFO:teuthology.orchestra.run.smithi096.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-05T06:29:12.702 INFO:teuthology.orchestra.run.smithi096.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-05T06:29:12.818 INFO:teuthology.orchestra.run.smithi096.stdout:(74/154): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-05T06:29:12.847 INFO:teuthology.orchestra.run.smithi096.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 420 kB/s | 61 kB 00:00 2024-09-05T06:29:12.889 INFO:teuthology.orchestra.run.smithi096.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 3.4 MB/s | 245 kB 00:00 2024-09-05T06:29:12.923 INFO:teuthology.orchestra.run.smithi096.stdout:(77/154): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-05T06:29:12.999 INFO:teuthology.orchestra.run.smithi096.stdout:(78/154): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-09-05T06:29:13.024 INFO:teuthology.orchestra.run.smithi096.stdout:(79/154): python3-jsonpatch-1.21-16.el9.noarch. 191 kB/s | 26 kB 00:00 2024-09-05T06:29:13.049 INFO:teuthology.orchestra.run.smithi096.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-05T06:29:13.074 INFO:teuthology.orchestra.run.smithi096.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-05T06:29:13.124 INFO:teuthology.orchestra.run.smithi096.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-09-05T06:29:13.150 INFO:teuthology.orchestra.run.smithi096.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-05T06:29:13.192 INFO:teuthology.orchestra.run.smithi096.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-05T06:29:13.259 INFO:teuthology.orchestra.run.smithi096.stdout:(85/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-05T06:29:13.284 INFO:teuthology.orchestra.run.smithi096.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-05T06:29:13.318 INFO:teuthology.orchestra.run.smithi096.stdout:(87/154): python3-packaging-20.9-5.el9.noarch.r 461 kB/s | 77 kB 00:00 2024-09-05T06:29:13.343 INFO:teuthology.orchestra.run.smithi096.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-05T06:29:13.368 INFO:teuthology.orchestra.run.smithi096.stdout:(89/154): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-05T06:29:13.394 INFO:teuthology.orchestra.run.smithi096.stdout:(90/154): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-05T06:29:13.411 INFO:teuthology.orchestra.run.smithi096.stdout:(91/154): python3-pytz-2021.1-5.el9.noarch.rpm 757 kB/s | 51 kB 00:00 2024-09-05T06:29:13.428 INFO:teuthology.orchestra.run.smithi096.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 910 kB/s | 54 kB 00:00 2024-09-05T06:29:13.461 INFO:teuthology.orchestra.run.smithi096.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-05T06:29:13.495 INFO:teuthology.orchestra.run.smithi096.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-09-05T06:29:13.497 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-05T06:29:13.520 INFO:teuthology.orchestra.run.smithi096.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-05T06:29:13.554 INFO:teuthology.orchestra.run.smithi096.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-05T06:29:13.571 INFO:teuthology.orchestra.run.smithi096.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-05T06:29:13.630 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-05T06:29:13.631 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-05T06:29:13.746 INFO:teuthology.orchestra.run.smithi096.stdout:(98/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-09-05T06:29:13.763 INFO:teuthology.orchestra.run.smithi096.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-09-05T06:29:13.788 INFO:teuthology.orchestra.run.smithi096.stdout:(100/154): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-05T06:29:13.814 INFO:teuthology.orchestra.run.smithi096.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-05T06:29:13.839 INFO:teuthology.orchestra.run.smithi096.stdout:(102/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 842 kB/s | 64 kB 00:00 2024-09-05T06:29:13.864 INFO:teuthology.orchestra.run.smithi096.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-05T06:29:13.889 INFO:teuthology.orchestra.run.smithi096.stdout:(104/154): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-09-05T06:29:13.931 INFO:teuthology.orchestra.run.smithi096.stdout:(105/154): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-09-05T06:29:13.982 INFO:teuthology.orchestra.run.smithi096.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 417 kB/s | 19 kB 00:00 2024-09-05T06:29:14.007 INFO:teuthology.orchestra.run.smithi096.stdout:(107/154): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-05T06:29:14.032 INFO:teuthology.orchestra.run.smithi096.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T06:29:14.057 INFO:teuthology.orchestra.run.smithi096.stdout:(109/154): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-09-05T06:29:14.083 INFO:teuthology.orchestra.run.smithi096.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-05T06:29:14.108 INFO:teuthology.orchestra.run.smithi096.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-05T06:29:14.133 INFO:teuthology.orchestra.run.smithi096.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-05T06:29:14.184 INFO:teuthology.orchestra.run.smithi096.stdout:(113/154): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-05T06:29:14.209 INFO:teuthology.orchestra.run.smithi096.stdout:(114/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-05T06:29:14.234 INFO:teuthology.orchestra.run.smithi096.stdout:(115/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-09-05T06:29:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 8.9 kB/s | 19 kB 00:02 2024-09-05T06:29:14.260 INFO:teuthology.orchestra.run.smithi096.stdout:(116/154): python3-cachetools-4.2.4-1.el9.noarc 637 kB/s | 32 kB 00:00 2024-09-05T06:29:14.285 INFO:teuthology.orchestra.run.smithi096.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-05T06:29:14.310 INFO:teuthology.orchestra.run.smithi096.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-05T06:29:14.344 INFO:teuthology.orchestra.run.smithi096.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T06:29:14.378 INFO:teuthology.orchestra.run.smithi096.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-05T06:29:14.411 INFO:teuthology.orchestra.run.smithi096.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-05T06:29:14.445 INFO:teuthology.orchestra.run.smithi096.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-05T06:29:14.470 INFO:teuthology.orchestra.run.smithi096.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-05T06:29:14.495 INFO:teuthology.orchestra.run.smithi096.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-05T06:29:14.521 INFO:teuthology.orchestra.run.smithi096.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-05T06:29:14.546 INFO:teuthology.orchestra.run.smithi096.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-05T06:29:14.571 INFO:teuthology.orchestra.run.smithi096.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-05T06:29:14.630 INFO:teuthology.orchestra.run.smithi096.stdout:(128/154): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-09-05T06:29:14.655 INFO:teuthology.orchestra.run.smithi096.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T06:29:14.681 INFO:teuthology.orchestra.run.smithi096.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T06:29:14.723 INFO:teuthology.orchestra.run.smithi096.stdout:(131/154): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-09-05T06:29:14.748 INFO:teuthology.orchestra.run.smithi096.stdout:(132/154): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-09-05T06:29:14.773 INFO:teuthology.orchestra.run.smithi096.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-05T06:29:14.798 INFO:teuthology.orchestra.run.smithi096.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-05T06:29:14.832 INFO:teuthology.orchestra.run.smithi096.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-05T06:29:14.866 INFO:teuthology.orchestra.run.smithi096.stdout:(136/154): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-05T06:29:14.882 INFO:teuthology.orchestra.run.smithi096.stdout:(137/154): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-09-05T06:29:14.900 INFO:teuthology.orchestra.run.smithi096.stdout:(138/154): python3-repoze-lru-0.7-16.el9.noarch 456 kB/s | 31 kB 00:00 2024-09-05T06:29:14.917 INFO:teuthology.orchestra.run.smithi096.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-05T06:29:14.933 INFO:teuthology.orchestra.run.smithi096.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-05T06:29:14.959 INFO:teuthology.orchestra.run.smithi096.stdout:(141/154): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-05T06:29:14.984 INFO:teuthology.orchestra.run.smithi096.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-05T06:29:15.009 INFO:teuthology.orchestra.run.smithi096.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T06:29:15.043 INFO:teuthology.orchestra.run.smithi096.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-05T06:29:15.069 INFO:teuthology.orchestra.run.smithi096.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 339 kB/s | 20 kB 00:00 2024-09-05T06:29:15.094 INFO:teuthology.orchestra.run.smithi096.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T06:29:15.119 INFO:teuthology.orchestra.run.smithi096.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-05T06:29:15.170 INFO:teuthology.orchestra.run.smithi096.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-05T06:29:15.201 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-05T06:29:15.398 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-05T06:29:15.399 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-05T06:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout:(149/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-05T06:29:15.495 INFO:teuthology.orchestra.run.smithi096.stdout:(150/154): NetworkManager-1.48.10-1.el9.x86_64. 7.1 MB/s | 2.3 MB 00:00 2024-09-05T06:29:15.555 INFO:teuthology.orchestra.run.smithi096.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 684 kB/s | 40 kB 00:00 2024-09-05T06:29:15.612 INFO:teuthology.orchestra.run.smithi096.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-09-05T06:29:15.688 INFO:teuthology.orchestra.run.smithi096.stdout:(153/154): librados2-19.1.1-259.g91c8a7e5.el9.x 5.8 MB/s | 3.4 MB 00:00 2024-09-05T06:29:15.729 INFO:teuthology.orchestra.run.smithi096.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 6.3 MB/s | 1.8 MB 00:00 2024-09-05T06:29:15.736 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:29:15.736 INFO:teuthology.orchestra.run.smithi096.stdout:Total 20 MB/s | 222 MB 00:11 2024-09-05T06:29:16.914 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-05T06:29:17.046 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-05T06:29:17.046 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-05T06:29:18.800 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-05T06:29:18.800 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-05T06:29:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-05T06:29:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-05T06:29:21.544 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T06:29:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T06:29:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T06:29:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T06:29:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:29:22.563 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T06:29:22.589 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T06:29:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T06:29:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T06:29:22.819 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:22.820 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T06:29:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:22.833 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T06:29:22.834 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T06:29:22.835 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T06:29:22.836 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T06:29:22.838 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T06:29:22.841 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T06:29:22.842 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T06:29:22.905 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T06:29:22.914 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T06:29:22.915 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T06:29:22.916 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-09-05T06:29:22.917 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T06:29:22.918 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T06:29:22.919 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T06:29:22.920 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T06:29:22.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T06:29:22.922 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T06:29:22.923 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T06:29:22.924 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T06:29:22.925 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T06:29:22.926 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================== 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 6 Packages 2024-09-05T06:29:22.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:29:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:22.928 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 222 M 2024-09-05T06:29:22.928 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T06:29:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:23.251 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:23.402 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:23.466 INFO:teuthology.orchestra.run.smithi086.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-05T06:29:23.521 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:23.581 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:23.667 INFO:teuthology.orchestra.run.smithi086.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 4.4 MB/s | 896 kB 00:00 2024-09-05T06:29:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:23.717 INFO:teuthology.orchestra.run.smithi086.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 2.9 MB/s | 145 kB 00:00 2024-09-05T06:29:23.766 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:23.942 INFO:teuthology.orchestra.run.smithi086.stdout:(4/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-09-05T06:29:23.985 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:23.992 INFO:teuthology.orchestra.run.smithi086.stdout:(5/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 8.7 MB/s | 2.4 MB 00:00 2024-09-05T06:29:24.085 INFO:teuthology.orchestra.run.smithi086.stdout:(6/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-05T06:29:24.097 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T06:29:24.215 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T06:29:24.246 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T06:29:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T06:29:24.368 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T06:29:24.401 INFO:teuthology.orchestra.run.smithi086.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-05T06:29:24.416 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T06:29:24.458 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T06:29:24.508 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T06:29:24.540 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T06:29:24.568 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T06:29:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T06:29:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T06:29:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T06:29:24.765 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T06:29:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 14 MB/s | 21 MB 00:01 2024-09-05T06:29:24.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T06:29:24.875 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T06:29:24.885 INFO:teuthology.orchestra.run.smithi086.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 608 kB/s | 25 kB 00:00 2024-09-05T06:29:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:29:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T06:29:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-05T06:29:25.018 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T06:29:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T06:29:25.062 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T06:29:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T06:29:25.077 INFO:teuthology.orchestra.run.smithi086.stdout:(10/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 17 MB/s | 17 MB 00:00 2024-09-05T06:29:25.116 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T06:29:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T06:29:25.202 INFO:teuthology.orchestra.run.smithi086.stdout:(11/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 13 MB/s | 11 MB 00:00 2024-09-05T06:29:25.227 INFO:teuthology.orchestra.run.smithi086.stdout:(12/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 1.5 MB/s | 224 kB 00:00 2024-09-05T06:29:25.253 INFO:teuthology.orchestra.run.smithi086.stdout:(13/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 540 kB/s | 27 kB 00:00 2024-09-05T06:29:25.278 INFO:teuthology.orchestra.run.smithi086.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 631 kB/s | 32 kB 00:00 2024-09-05T06:29:25.278 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T06:29:25.320 INFO:teuthology.orchestra.run.smithi086.stdout:(15/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 11 MB/s | 752 kB 00:00 2024-09-05T06:29:25.331 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T06:29:25.345 INFO:teuthology.orchestra.run.smithi086.stdout:(16/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 2.4 MB/s | 163 kB 00:00 2024-09-05T06:29:25.371 INFO:teuthology.orchestra.run.smithi086.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 2.5 MB/s | 127 kB 00:00 2024-09-05T06:29:25.396 INFO:teuthology.orchestra.run.smithi086.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 9.9 MB/s | 506 kB 00:00 2024-09-05T06:29:25.446 INFO:teuthology.orchestra.run.smithi086.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 902 kB/s | 45 kB 00:00 2024-09-05T06:29:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T06:29:25.530 INFO:teuthology.orchestra.run.smithi086.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 1.6 MB/s | 136 kB 00:00 2024-09-05T06:29:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:25.647 INFO:teuthology.orchestra.run.smithi086.stdout:(21/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-05T06:29:25.672 INFO:teuthology.orchestra.run.smithi086.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 1.1 MB/s | 162 kB 00:00 2024-09-05T06:29:25.697 INFO:teuthology.orchestra.run.smithi086.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 6.3 MB/s | 321 kB 00:00 2024-09-05T06:29:25.705 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:25.722 INFO:teuthology.orchestra.run.smithi086.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 5.9 MB/s | 301 kB 00:00 2024-09-05T06:29:25.739 INFO:teuthology.orchestra.run.smithi086.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 2.3 MB/s | 99 kB 00:00 2024-09-05T06:29:25.742 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:25.773 INFO:teuthology.orchestra.run.smithi086.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T06:29:25.834 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:25.865 INFO:teuthology.orchestra.run.smithi086.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-05T06:29:25.867 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T06:29:25.884 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:29:25.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T06:29:25.924 INFO:teuthology.orchestra.run.smithi086.stdout:(28/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-05T06:29:25.939 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T06:29:25.949 INFO:teuthology.orchestra.run.smithi086.stdout:(29/154): ceph-grafana-dashboards-19.1.1-259.g9 381 kB/s | 32 kB 00:00 2024-09-05T06:29:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T06:29:25.974 INFO:teuthology.orchestra.run.smithi086.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.8 MB/s | 146 kB 00:00 2024-09-05T06:29:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T06:29:26.031 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T06:29:26.069 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T06:29:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T06:29:26.106 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T06:29:26.133 INFO:teuthology.orchestra.run.smithi086.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 19 MB/s | 3.6 MB 00:00 2024-09-05T06:29:26.161 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T06:29:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T06:29:26.189 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:26.200 INFO:teuthology.orchestra.run.smithi086.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 3.7 MB/s | 249 kB 00:00 2024-09-05T06:29:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T06:29:26.258 INFO:teuthology.orchestra.run.smithi086.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 846 kB/s | 49 kB 00:00 2024-09-05T06:29:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T06:29:26.277 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:26.299 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:26.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T06:29:26.317 INFO:teuthology.orchestra.run.smithi086.stdout:(34/154): ceph-prometheus-alerts-19.1.1-259.g91 283 kB/s | 16 kB 00:00 2024-09-05T06:29:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T06:29:26.362 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T06:29:26.385 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:26.406 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:26.417 INFO:teuthology.orchestra.run.smithi086.stdout:(35/154): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-05T06:29:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T06:29:26.451 INFO:teuthology.orchestra.run.smithi086.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 2.1 MB/s | 285 kB 00:00 2024-09-05T06:29:26.470 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T06:29:26.493 INFO:teuthology.orchestra.run.smithi086.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 10 MB/s | 765 kB 00:00 2024-09-05T06:29:26.509 INFO:teuthology.orchestra.run.smithi086.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 311 kB/s | 18 kB 00:00 2024-09-05T06:29:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:26.602 INFO:teuthology.orchestra.run.smithi086.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-05T06:29:26.610 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T06:29:26.652 INFO:teuthology.orchestra.run.smithi086.stdout:(40/154): libquadmath-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T06:29:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T06:29:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T06:29:26.696 INFO:teuthology.orchestra.run.smithi086.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-05T06:29:26.719 INFO:teuthology.orchestra.run.smithi086.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-05T06:29:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:26.761 INFO:teuthology.orchestra.run.smithi086.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-05T06:29:26.803 INFO:teuthology.orchestra.run.smithi086.stdout:(44/154): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-05T06:29:26.856 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:26.900 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:27.005 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T06:29:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T06:29:27.076 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:27.086 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T06:29:27.111 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T06:29:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T06:29:27.218 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T06:29:27.245 INFO:teuthology.orchestra.run.smithi086.stdout:(45/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 21 MB/s | 49 MB 00:02 2024-09-05T06:29:27.271 INFO:teuthology.orchestra.run.smithi086.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 267 kB/s | 125 kB 00:00 2024-09-05T06:29:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:27.304 INFO:teuthology.orchestra.run.smithi086.stdout:(47/154): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-09-05T06:29:27.406 INFO:teuthology.orchestra.run.smithi086.stdout:(48/154): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-05T06:29:27.408 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T06:29:27.488 INFO:teuthology.orchestra.run.smithi086.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-05T06:29:27.513 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T06:29:27.530 INFO:teuthology.orchestra.run.smithi086.stdout:(50/154): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-09-05T06:29:27.541 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T06:29:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T06:29:27.555 INFO:teuthology.orchestra.run.smithi086.stdout:(51/154): boost-program-options-1.75.0-8.el9.x8 710 kB/s | 107 kB 00:00 2024-09-05T06:29:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T06:29:27.585 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T06:29:27.589 INFO:teuthology.orchestra.run.smithi086.stdout:(52/154): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-05T06:29:27.613 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T06:29:27.622 INFO:teuthology.orchestra.run.smithi086.stdout:(53/154): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-05T06:29:27.647 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T06:29:27.648 INFO:teuthology.orchestra.run.smithi086.stdout:(54/154): flexiblas-3.0.4-8.el9.x86_64.rpm 354 kB/s | 33 kB 00:00 2024-09-05T06:29:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T06:29:27.673 INFO:teuthology.orchestra.run.smithi086.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-05T06:29:27.691 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T06:29:27.698 INFO:teuthology.orchestra.run.smithi086.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-05T06:29:27.709 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T06:29:27.723 INFO:teuthology.orchestra.run.smithi086.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-05T06:29:27.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T06:29:27.748 INFO:teuthology.orchestra.run.smithi086.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-05T06:29:27.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T06:29:27.762 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T06:29:27.782 INFO:teuthology.orchestra.run.smithi086.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-05T06:29:27.792 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T06:29:27.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T06:29:27.826 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T06:29:27.832 INFO:teuthology.orchestra.run.smithi086.stdout:(60/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-05T06:29:27.835 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T06:29:27.857 INFO:teuthology.orchestra.run.smithi086.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-05T06:29:27.868 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T06:29:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T06:29:27.883 INFO:teuthology.orchestra.run.smithi086.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-05T06:29:27.910 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T06:29:27.917 INFO:teuthology.orchestra.run.smithi086.stdout:(63/154): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-05T06:29:27.940 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T06:29:27.941 INFO:teuthology.orchestra.run.smithi086.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-05T06:29:27.944 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T06:29:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:(65/154): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-05T06:29:27.992 INFO:teuthology.orchestra.run.smithi086.stdout:(66/154): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-05T06:29:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T06:29:28.025 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T06:29:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T06:29:28.075 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T06:29:28.076 INFO:teuthology.orchestra.run.smithi086.stdout:(67/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-05T06:29:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T06:29:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T06:29:28.118 INFO:teuthology.orchestra.run.smithi086.stdout:(68/154): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-05T06:29:28.159 INFO:teuthology.orchestra.run.smithi086.stdout:(69/154): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-05T06:29:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T06:29:28.185 INFO:teuthology.orchestra.run.smithi086.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-05T06:29:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T06:29:28.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T06:29:28.246 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:29:28.266 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T06:29:28.269 INFO:teuthology.orchestra.run.smithi086.stdout:(71/154): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-05T06:29:28.275 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T06:29:28.309 INFO:teuthology.orchestra.run.smithi086.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-05T06:29:28.334 INFO:teuthology.orchestra.run.smithi086.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-05T06:29:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T06:29:28.384 INFO:teuthology.orchestra.run.smithi086.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-05T06:29:28.390 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T06:29:28.460 INFO:teuthology.orchestra.run.smithi086.stdout:(75/154): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-05T06:29:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T06:29:28.485 INFO:teuthology.orchestra.run.smithi086.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-05T06:29:28.510 INFO:teuthology.orchestra.run.smithi086.stdout:(77/154): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-05T06:29:28.535 INFO:teuthology.orchestra.run.smithi086.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-05T06:29:28.552 INFO:teuthology.orchestra.run.smithi086.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-09-05T06:29:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:(80/154): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-05T06:29:28.594 INFO:teuthology.orchestra.run.smithi086.stdout:(81/154): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-09-05T06:29:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T06:29:28.670 INFO:teuthology.orchestra.run.smithi086.stdout:(82/154): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-05T06:29:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T06:29:28.695 INFO:teuthology.orchestra.run.smithi086.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-05T06:29:28.720 INFO:teuthology.orchestra.run.smithi086.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-05T06:29:28.745 INFO:teuthology.orchestra.run.smithi086.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-05T06:29:28.770 INFO:teuthology.orchestra.run.smithi086.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-05T06:29:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-05T06:29:28.821 INFO:teuthology.orchestra.run.smithi086.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-05T06:29:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:(89/154): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-05T06:29:28.872 INFO:teuthology.orchestra.run.smithi086.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-05T06:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-05T06:29:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:(92/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-05T06:29:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-05T06:29:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T06:29:29.023 INFO:teuthology.orchestra.run.smithi086.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-05T06:29:29.048 INFO:teuthology.orchestra.run.smithi086.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-05T06:29:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T06:29:29.073 INFO:teuthology.orchestra.run.smithi086.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-05T06:29:29.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T06:29:29.099 INFO:teuthology.orchestra.run.smithi086.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-05T06:29:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-05T06:29:29.149 INFO:teuthology.orchestra.run.smithi086.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-05T06:29:29.166 INFO:teuthology.orchestra.run.smithi086.stdout:(100/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-05T06:29:29.191 INFO:teuthology.orchestra.run.smithi086.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-05T06:29:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T06:29:29.216 INFO:teuthology.orchestra.run.smithi086.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-05T06:29:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T06:29:29.242 INFO:teuthology.orchestra.run.smithi086.stdout:(103/154): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-05T06:29:29.244 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T06:29:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T06:29:29.292 INFO:teuthology.orchestra.run.smithi086.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T06:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T06:29:29.325 INFO:teuthology.orchestra.run.smithi086.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-05T06:29:29.337 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T06:29:29.351 INFO:teuthology.orchestra.run.smithi086.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-05T06:29:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T06:29:29.388 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T06:29:29.409 INFO:teuthology.orchestra.run.smithi086.stdout:(107/154): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-09-05T06:29:29.427 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T06:29:29.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T06:29:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T06:29:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T06:29:29.577 INFO:teuthology.orchestra.run.smithi086.stdout:(108/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-05T06:29:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T06:29:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T06:29:29.610 INFO:teuthology.orchestra.run.smithi086.stdout:(109/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-09-05T06:29:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T06:29:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T06:29:29.643 INFO:teuthology.orchestra.run.smithi086.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 751 kB/s | 49 kB 00:00 2024-09-05T06:29:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T06:29:29.669 INFO:teuthology.orchestra.run.smithi086.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-05T06:29:29.694 INFO:teuthology.orchestra.run.smithi086.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-05T06:29:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T06:29:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T06:29:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T06:29:29.761 INFO:teuthology.orchestra.run.smithi086.stdout:(113/154): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-05T06:29:29.786 INFO:teuthology.orchestra.run.smithi086.stdout:(114/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-05T06:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T06:29:29.811 INFO:teuthology.orchestra.run.smithi086.stdout:(115/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-09-05T06:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T06:29:29.837 INFO:teuthology.orchestra.run.smithi086.stdout:(116/154): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-05T06:29:29.879 INFO:teuthology.orchestra.run.smithi086.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 211 kB/s | 14 kB 00:00 2024-09-05T06:29:29.904 INFO:teuthology.orchestra.run.smithi086.stdout:(118/154): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-05T06:29:29.929 INFO:teuthology.orchestra.run.smithi086.stdout:(119/154): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-09-05T06:29:29.954 INFO:teuthology.orchestra.run.smithi086.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-05T06:29:29.969 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T06:29:29.988 INFO:teuthology.orchestra.run.smithi086.stdout:(121/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-05T06:29:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T06:29:30.013 INFO:teuthology.orchestra.run.smithi086.stdout:(122/154): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-05T06:29:30.030 INFO:teuthology.orchestra.run.smithi086.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 644 kB/s | 11 kB 00:00 2024-09-05T06:29:30.051 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T06:29:30.055 INFO:teuthology.orchestra.run.smithi086.stdout:(124/154): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-05T06:29:30.072 INFO:teuthology.orchestra.run.smithi086.stdout:(125/154): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-05T06:29:30.098 INFO:teuthology.orchestra.run.smithi086.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 555 kB/s | 23 kB 00:00 2024-09-05T06:29:30.123 INFO:teuthology.orchestra.run.smithi086.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-05T06:29:30.148 INFO:teuthology.orchestra.run.smithi086.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-05T06:29:30.153 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T06:29:30.174 INFO:teuthology.orchestra.run.smithi086.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T06:29:30.199 INFO:teuthology.orchestra.run.smithi086.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T06:29:30.200 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T06:29:30.224 INFO:teuthology.orchestra.run.smithi086.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T06:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T06:29:30.266 INFO:teuthology.orchestra.run.smithi086.stdout:(132/154): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-05T06:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T06:29:30.291 INFO:teuthology.orchestra.run.smithi086.stdout:(133/154): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-05T06:29:30.308 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T06:29:30.316 INFO:teuthology.orchestra.run.smithi086.stdout:(134/154): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-05T06:29:30.342 INFO:teuthology.orchestra.run.smithi086.stdout:(135/154): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-05T06:29:30.359 INFO:teuthology.orchestra.run.smithi086.stdout:(136/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-05T06:29:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T06:29:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T06:29:30.384 INFO:teuthology.orchestra.run.smithi086.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 737 kB/s | 31 kB 00:00 2024-09-05T06:29:30.409 INFO:teuthology.orchestra.run.smithi086.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-05T06:29:30.434 INFO:teuthology.orchestra.run.smithi086.stdout:(139/154): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-05T06:29:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-05T06:29:30.485 INFO:teuthology.orchestra.run.smithi086.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-05T06:29:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T06:29:30.516 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T06:29:30.535 INFO:teuthology.orchestra.run.smithi086.stdout:(142/154): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-09-05T06:29:30.546 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T06:29:30.560 INFO:teuthology.orchestra.run.smithi086.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-05T06:29:30.574 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T06:29:30.586 INFO:teuthology.orchestra.run.smithi086.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-05T06:29:30.608 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T06:29:30.611 INFO:teuthology.orchestra.run.smithi086.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-05T06:29:30.636 INFO:teuthology.orchestra.run.smithi086.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T06:29:30.661 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T06:29:30.670 INFO:teuthology.orchestra.run.smithi086.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-05T06:29:30.707 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T06:29:30.720 INFO:teuthology.orchestra.run.smithi086.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T06:29:30.739 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T06:29:30.769 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T06:29:30.803 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T06:29:30.837 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T06:29:30.878 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T06:29:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T06:29:30.995 INFO:teuthology.orchestra.run.smithi086.stdout:(149/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-05T06:29:31.046 INFO:teuthology.orchestra.run.smithi086.stdout:(150/154): librados2-19.1.1-259.g91c8a7e5.el9.x 8.4 MB/s | 3.4 MB 00:00 2024-09-05T06:29:31.104 INFO:teuthology.orchestra.run.smithi086.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 698 kB/s | 40 kB 00:00 2024-09-05T06:29:31.179 INFO:teuthology.orchestra.run.smithi086.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 3.2 MB/s | 248 kB 00:00 2024-09-05T06:29:31.333 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T06:29:31.363 INFO:teuthology.orchestra.run.smithi086.stdout:(153/154): NetworkManager-1.48.10-1.el9.x86_64. 3.6 MB/s | 2.3 MB 00:00 2024-09-05T06:29:31.395 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T06:29:31.441 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T06:29:31.454 INFO:teuthology.orchestra.run.smithi086.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 3.9 MB/s | 1.8 MB 00:00 2024-09-05T06:29:31.461 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:29:31.461 INFO:teuthology.orchestra.run.smithi086.stdout:Total 26 MB/s | 222 MB 00:08 2024-09-05T06:29:31.548 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T06:29:31.609 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T06:29:31.759 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T06:29:31.843 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T06:29:31.870 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T06:29:31.903 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T06:29:32.215 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T06:29:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:32.341 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:32.346 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T06:29:32.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T06:29:32.440 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T06:29:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T06:29:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T06:29:32.600 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T06:29:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T06:29:32.655 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T06:29:32.657 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T06:29:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T06:29:32.788 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T06:29:32.788 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T06:29:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T06:29:32.977 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T06:29:33.007 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T06:29:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T06:29:33.093 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T06:29:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:33.182 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:33.204 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T06:29:33.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T06:29:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T06:29:34.538 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T06:29:34.539 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T06:29:35.129 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T06:29:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T06:29:35.449 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T06:29:35.467 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:35.490 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T06:29:35.501 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:35.798 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T06:29:35.845 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:35.966 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T06:29:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T06:29:36.046 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:36.089 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:36.183 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T06:29:36.183 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T06:29:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:36.216 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:29:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T06:29:36.514 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T06:29:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:39.021 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:39.701 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:39.701 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T06:29:39.701 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T06:29:39.701 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:39.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:29:40.579 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T06:29:40.649 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T06:29:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T06:29:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T06:29:41.064 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T06:29:41.217 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:41.406 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T06:29:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:29:41.567 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:29:41.597 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T06:29:41.622 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T06:29:41.776 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T06:29:41.819 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T06:29:41.848 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:41.952 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:29:41.982 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T06:29:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T06:29:42.301 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T06:29:42.580 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:42.640 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T06:29:42.747 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:42.816 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:43.046 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T06:29:43.153 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T06:29:43.251 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T06:29:43.279 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T06:29:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T06:29:43.371 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T06:29:43.415 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T06:29:43.446 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T06:29:43.490 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T06:29:43.522 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T06:29:43.551 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T06:29:43.621 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T06:29:43.673 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T06:29:43.703 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T06:29:43.738 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T06:29:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T06:29:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T06:29:43.892 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:29:43.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T06:29:43.993 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T06:29:44.035 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T06:29:44.113 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T06:29:44.254 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T06:29:44.314 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T06:29:44.855 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T06:29:44.947 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T06:29:44.982 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T06:29:45.016 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T06:29:45.046 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T06:29:45.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T06:29:45.136 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T06:29:45.209 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T06:29:45.247 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T06:29:45.283 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T06:29:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T06:29:45.366 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T06:29:45.409 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T06:29:45.452 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T06:29:45.592 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T06:29:45.636 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T06:29:45.678 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T06:29:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T06:29:46.016 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T06:29:46.111 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T06:29:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T06:29:46.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T06:29:46.253 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T06:29:46.414 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T06:29:46.615 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T06:29:46.651 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T06:29:46.685 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T06:29:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T06:29:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T06:29:46.835 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T06:29:46.869 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T06:29:46.904 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T06:29:46.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T06:29:46.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T06:29:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T06:29:47.102 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T06:29:47.143 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T06:29:47.183 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T06:29:47.287 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T06:29:47.352 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T06:29:48.089 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T06:29:48.139 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T06:29:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T06:29:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:29:48.179 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-05T06:29:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:48.273 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T06:29:48.327 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T06:29:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:29:48.372 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T06:29:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:29:48.375 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T06:29:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:48.462 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T06:29:48.500 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T06:29:48.683 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T06:29:48.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T06:29:48.791 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T06:29:49.452 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T06:29:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T06:29:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:29:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:29:49.898 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:49.898 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T06:29:49.898 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:29:49.899 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:29:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:29:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:50.581 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:29:50.604 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:29:50.605 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:50.605 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T06:29:50.605 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:29:50.605 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:29:50.605 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:50.712 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T06:29:50.768 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T06:29:50.824 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T06:29:50.828 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T06:29:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T06:29:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T06:29:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T06:29:51.020 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T06:29:51.066 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T06:29:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T06:29:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T06:29:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T06:29:51.332 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:51.373 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T06:29:51.407 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T06:29:51.424 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T06:29:51.437 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T06:29:51.456 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T06:29:51.472 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-09-05T06:29:51.476 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-09-05T06:29:51.477 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-09-05T06:29:51.477 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-09-05T06:29:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:51.492 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T06:29:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:29:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:29:51.663 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:51.663 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T06:29:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:29:51.945 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T06:29:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:29:51.976 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T06:29:52.031 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T06:29:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T06:29:52.061 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T06:29:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:29:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:29:53.157 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:29:53.165 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:29:53.199 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:29:53.472 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:29:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:29:53.943 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:53.943 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T06:29:53.943 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:29:53.943 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:29:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T06:29:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T06:29:54.115 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:29:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:29:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T06:29:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:29:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T06:29:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T06:29:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T06:29:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:29:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:29:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:29:54.530 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:29:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T06:29:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T06:29:54.675 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T06:29:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T06:29:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T06:29:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T06:29:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T06:29:54.860 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T06:29:54.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:54.977 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T06:29:54.977 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T06:29:54.977 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T06:29:54.978 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:29:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:29:55.242 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:29:55.270 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:29:55.301 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T06:29:55.363 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T06:29:55.394 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:29:55.597 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T06:29:55.629 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:29:55.653 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:55.697 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:29:55.719 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:29:55.720 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:55.720 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T06:29:55.720 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:55.764 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:29:55.787 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:29:55.787 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:55.787 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T06:29:55.787 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:29:55.788 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:29:55.788 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:29:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:29:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:29:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:29:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:29:57.732 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T06:29:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:29:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:57.832 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:29:57.844 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:29:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T06:29:57.997 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T06:29:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T06:29:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:29:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:29:59.041 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:29:59.042 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:59.042 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T06:29:59.042 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:59.086 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:29:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:29:59.412 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:29:59.435 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:29:59.435 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:29:59.435 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T06:29:59.435 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:29:59.436 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:29:59.436 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:30:00.643 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T06:30:00.684 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T06:30:00.725 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T06:30:00.758 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T06:30:00.784 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T06:30:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T06:30:01.023 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T06:30:01.053 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T06:30:01.091 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T06:30:01.138 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T06:30:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:02.667 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T06:30:02.741 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T06:30:02.781 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T06:30:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T06:30:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T06:30:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T06:30:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T06:30:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:03.306 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:03.307 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T06:30:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T06:30:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T06:30:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T06:30:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T06:30:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T06:30:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T06:30:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T06:30:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T06:30:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T06:30:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:05.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T06:30:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T06:30:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T06:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T06:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T06:30:06.185 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T06:30:06.185 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:06.270 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T06:30:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:06.480 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:06.480 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-09-05T06:30:06.996 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-09-05T06:30:06.997 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-09-05T06:30:06.997 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-09-05T06:30:06.997 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:30:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T06:30:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T06:30:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:08.651 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:30:08.679 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T06:30:08.679 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:08.679 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T06:30:08.679 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:30:08.679 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T06:30:08.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:08.901 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:30:08.925 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T06:30:08.925 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:08.925 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T06:30:08.926 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:30:08.926 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T06:30:08.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T06:30:09.025 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T06:30:09.026 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T06:30:09.027 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T06:30:09.028 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T06:30:09.029 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T06:30:09.030 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T06:30:09.031 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T06:30:09.032 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T06:30:09.033 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T06:30:09.034 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T06:30:09.035 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T06:30:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:09.037 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T06:30:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:30:09.384 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T06:30:09.385 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:09.385 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T06:30:09.385 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:30:09.385 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T06:30:09.385 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:09.507 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T06:30:09.563 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T06:30:09.620 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T06:30:09.805 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T06:30:09.852 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T06:30:09.930 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T06:30:10.018 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T06:30:10.158 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T06:30:10.202 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T06:30:10.234 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T06:30:10.270 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T06:30:10.715 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:30:10.743 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T06:30:10.774 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T06:30:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T06:30:10.866 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:11.839 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.840 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.841 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T06:30:11.842 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.843 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T06:30:11.844 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T06:30:11.845 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T06:30:11.846 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T06:30:11.847 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T06:30:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T06:30:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T06:30:11.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T06:30:11.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T06:30:11.852 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T06:30:11.853 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T06:30:11.853 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T06:30:11.853 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:30:11.853 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-05T06:30:11.906 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T06:30:11.943 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:30:12.241 DEBUG:teuthology.parallel:result is None 2024-09-05T06:30:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:12.415 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T06:30:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T06:30:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T06:30:12.418 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T06:30:12.419 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T06:30:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T06:30:12.421 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T06:30:12.421 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T06:30:12.421 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T06:30:12.421 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T06:30:12.422 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T06:30:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T06:30:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T06:30:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T06:30:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T06:30:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T06:30:12.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T06:30:12.425 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T06:30:12.425 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T06:30:12.425 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T06:30:12.425 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T06:30:12.425 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T06:30:12.426 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T06:30:12.427 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T06:30:12.428 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T06:30:12.429 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T06:30:12.430 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T06:30:12.432 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T06:30:12.433 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T06:30:12.434 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:12.435 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T06:30:12.910 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T06:30:12.935 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:30:13.037 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T06:30:13.196 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T06:30:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T06:30:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:30:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T06:30:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T06:30:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T06:30:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T06:30:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T06:30:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:30:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T06:30:14.505 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:14.506 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T06:30:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T06:30:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:30:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T06:30:14.899 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T06:30:14.899 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T06:30:14.899 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:30:14.899 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T06:30:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.014 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T06:30:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T06:30:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T06:30:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T06:30:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T06:30:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T06:30:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T06:30:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T06:30:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T06:30:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T06:30:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T06:30:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T06:30:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T06:30:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:30:15.029 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-05T06:30:15.410 DEBUG:teuthology.parallel:result is None 2024-09-05T06:30:19.462 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T06:30:19.503 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T06:30:19.553 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T06:30:19.584 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T06:30:19.612 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T06:30:19.796 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T06:30:19.842 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T06:30:19.872 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T06:30:19.901 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T06:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T06:30:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:20.586 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:20.607 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T06:30:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T06:30:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T06:30:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T06:30:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T06:30:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T06:30:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T06:30:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T06:30:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T06:30:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T06:30:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:27.750 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T06:30:27.750 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T06:30:27.750 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T06:30:27.750 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T06:30:27.750 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T06:30:27.751 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T06:30:27.752 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T06:30:27.764 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T06:30:27.765 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T06:30:27.766 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T06:30:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T06:30:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T06:30:27.769 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T06:30:27.770 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T06:30:27.771 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T06:30:27.772 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T06:30:27.773 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T06:30:27.774 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T06:30:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T06:30:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T06:30:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T06:30:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T06:30:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T06:30:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T06:30:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T06:30:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T06:30:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T06:30:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T06:30:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T06:30:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:30:30.422 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T06:30:30.797 DEBUG:teuthology.parallel:result is None 2024-09-05T06:30:30.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:30.944 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T06:30:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:30.967 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:30.967 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T06:30:30.969 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:31.106 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T06:30:31.133 INFO:teuthology.orchestra.run.smithi096.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:31.133 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:31.133 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T06:30:31.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:31.287 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T06:30:31.313 INFO:teuthology.orchestra.run.smithi115.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:31.314 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T06:30:31.314 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T06:30:31.315 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T06:30:31.316 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:30:31.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T06:30:31.345 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:30:31.345 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T06:30:31.378 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:30:31.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T06:30:31.409 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T06:30:31.409 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:30:31.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T06:30:31.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T06:30:31.502 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:30:31.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T06:30:31.531 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T06:30:31.598 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:30:31.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T06:30:31.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T06:30:31.696 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T06:30:31.697 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:30:31.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T06:30:31.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T06:30:31.791 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:30:31.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T06:30:31.823 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T06:30:31.890 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:30:31.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T06:30:31.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T06:30:31.985 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T06:30:31.986 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:30:31.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T06:30:32.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T06:30:32.079 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:30:32.080 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T06:30:32.105 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T06:30:32.169 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:30:32.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T06:30:32.196 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T06:30:32.263 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-05T06:30:32.362 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-05T06:30:32.362 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:32.363 INFO:tasks.cephadm:Cluster fsid is 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:30:32.363 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-05T06:30:32.363 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.115'} 2024-09-05T06:30:32.363 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-09-05T06:30:32.363 INFO:tasks.cephadm:First mgr is x 2024-09-05T06:30:32.363 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-05T06:30:32.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-09-05T06:30:32.393 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-09-05T06:30:32.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-09-05T06:30:32.449 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-05T06:30:32.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:32.591 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56000/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-259-g91c8a7e5', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-259.g91c8a7e5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-04 16:53:37.418684', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-05T06:30:32.729 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:30:32.731 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T06:30:32.731 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T06:30:32.731 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.136 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 06:30 /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.137 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.517 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 06:30 /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.517 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.906 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 06:30 /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.906 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.926 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.946 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T06:30:33.982 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 on all hosts... 2024-09-05T06:30:33.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T06:30:33.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T06:30:33.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T06:30:35.474 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T06:30:35.536 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T06:30:35.568 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T06:31:02.077 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T06:31:02.078 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:31:02.414 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-05T06:31:02.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T06:31:02.414 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T06:31:02.415 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-09-05T06:31:02.415 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T06:31:02.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-05T06:31:02.415 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-05T06:31:07.416 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-05T06:31:07.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-09-05T06:31:07.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-09-05T06:31:07.501 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-09-05T06:31:07.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-09-05T06:31:07.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-09-05T06:31:07.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-09-05T06:31:07.616 INFO:tasks.cephadm:Writing seed config... 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-05T06:31:07.618 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-05T06:31:07.619 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-05T06:31:07.620 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-05T06:31:07.621 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-05T06:31:07.622 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:31:07.622 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-05T06:31:07.640 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5a055634-6b50-11ef-bcd6-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-05T06:31:07.640 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service 2024-09-05T06:31:07.682 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x.service 2024-09-05T06:31:07.725 INFO:tasks.cephadm:Bootstrapping... 2024-09-05T06:31:07.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 -v bootstrap --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-05T06:31:07.928 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T06:31:07.928 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', '-v', 'bootstrap', '--fsid', '5a055634-6b50-11ef-bcd6-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-09-05T06:31:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-05T06:31:07.929 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-09-05T06:31:07.952 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 5.2.2 2024-09-05T06:31:07.952 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-09-05T06:31:07.952 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-09-05T06:31:07.960 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T06:31:07.960 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T06:31:07.966 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T06:31:07.966 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:31:07.973 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-09-05T06:31:07.979 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-09-05T06:31:07.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-09-05T06:31:07.979 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-09-05T06:31:08.002 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 5.2.2 2024-09-05T06:31:08.002 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-05T06:31:08.002 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-09-05T06:31:08.002 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-09-05T06:31:08.008 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T06:31:08.009 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T06:31:08.015 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T06:31:08.015 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:31:08.021 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 139680433431552 on /run/cephadm/5a055634-6b50-11ef-bcd6-c7b262605968.lock 2024-09-05T06:31:08.028 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 139680433431552 acquired on /run/cephadm/5a055634-6b50-11ef-bcd6-c7b262605968.lock 2024-09-05T06:31:08.029 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-09-05T06:31:08.029 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-09-05T06:31:08.029 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-09-05T06:31:08.032 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.86 metric 100 2024-09-05T06:31:08.032 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-09-05T06:31:08.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-05T06:31:08.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-05T06:31:08.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-05T06:31:08.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-05T06:31:08.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-05T06:31:08.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T06:31:08.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-05T06:31:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link noprefixroute 2024-09-05T06:31:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T06:31:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-09-05T06:31:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-09-05T06:31:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-05T06:31:08.039 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-05T06:31:08.039 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T06:31:08.420 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:3f43494a24a2d1fe0d791eb19b05391e70ab9f54611bc9deae80cf7c3fb2dba9 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T06:31:08.421 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-05T06:31:10.420 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T06:31:10.420 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T06:31:10.420 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-09-05T06:31:19.426 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-09-05T06:31:19.426 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-09-05T06:31:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQA4UNlm3hlDAhAAN2MAgPXwcJJxu4TWqwQzag== 2024-09-05T06:31:21.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQA4UNlmA2wtNRAAFq9xdTOcK67DZ7zpkkf0gA== 2024-09-05T06:31:22.478 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQA5UNlmRg15NRAAQ16xrxkbaUXnX2zaCNrk4w== 2024-09-05T06:31:22.479 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T06:31:23.530 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:31:23.531 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-09-05T06:31:24.548 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-09-05T06:31:24.785 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-05T06:31:24.926 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-05T06:31:25.095 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target. 2024-09-05T06:31:25.096 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target. 2024-09-05T06:31:25.874 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a 2024-09-05T06:31:25.874 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service: Unit ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service not loaded. 2024-09-05T06:31:26.143 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target.wants/ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968@.service. 2024-09-05T06:31:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 podman[30560]: 2024-09-05 06:31:26.386155111 +0000 UTC m=+0.115529712 container create 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T06:31:26.677 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T06:31:26.677 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-09-05T06:31:26.684 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-09-05T06:31:26.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 podman[30560]: 2024-09-05 06:31:26.597080954 +0000 UTC m=+0.326455518 container init 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T06:31:26.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 podman[30560]: 2024-09-05 06:31:26.600156388 +0000 UTC m=+0.329530944 container start 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T06:31:26.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 ceph-mon[30580]: mkfs 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:26.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 ceph-mon[30580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T06:31:26.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 bash[30560]: 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 2024-09-05T06:31:26.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:26 smithi086 systemd[1]: Started Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:31:27.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.539618s) 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-09-05T06:31:27.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.108+0000 7f6048f1a640 1 Processor -- start 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.108+0000 7f6048f1a640 1 -- start start 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.108+0000 7f6048f1a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.108+0000 7f6048f1a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6044107e30 con 0x7f6044107460 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.109+0000 7f604259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.109+0000 7f604259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50992/0 (socket says 172.21.15.86:50992) 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.109+0000 7f604259c640 1 -- 172.21.15.86:0/3641845348 learned_addr learned my addr 172.21.15.86:0/3641845348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:27.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.109+0000 7f604259c640 1 -- 172.21.15.86:0/3641845348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6044108610 con 0x7f6044107460 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f604259c640 1 --2- 172.21.15.86:0/3641845348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6034009820 tx=0x7f603402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ae822f0912619f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6041d9b640 1 -- 172.21.15.86:0/3641845348 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034037070 con 0x7f6044107460 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6041d9b640 1 -- 172.21.15.86:0/3641845348 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6034032440 con 0x7f6044107460 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6041d9b640 1 -- 172.21.15.86:0/3641845348 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034032740 con 0x7f6044107460 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6048f1a640 1 -- 172.21.15.86:0/3641845348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 msgr2=0x7f6044107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6048f1a640 1 --2- 172.21.15.86:0/3641845348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6034009820 tx=0x7f603402a270 comp rx=0 tx=0).stop 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6048f1a640 1 -- 172.21.15.86:0/3641845348 shutdown_connections 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6048f1a640 1 --2- 172.21.15.86:0/3641845348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f6044107860 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.111+0000 7f6048f1a640 1 -- 172.21.15.86:0/3641845348 >> 172.21.15.86:0/3641845348 conn(0x7f6044102c50 msgr2=0x7f6044105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.112+0000 7f6048f1a640 1 -- 172.21.15.86:0/3641845348 shutdown_connections 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.112+0000 7f6048f1a640 1 -- 172.21.15.86:0/3641845348 wait complete. 2024-09-05T06:31:27.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.112+0000 7f6048f1a640 1 Processor -- start 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.112+0000 7f6048f1a640 1 -- start start 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f6048f1a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f6048f1a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6044108bb0 con 0x7f6044107460 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f604259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f604259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51006/0 (socket says 172.21.15.86:51006) 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f604259c640 1 -- 172.21.15.86:0/815741757 learned_addr learned my addr 172.21.15.86:0/815741757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f604259c640 1 -- 172.21.15.86:0/815741757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419b0d0 con 0x7f6044107460 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f604259c640 1 --2- 172.21.15.86:0/815741757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f60340090b0 tx=0x7f60340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034041070 con 0x7f6044107460 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603403c7a0 con 0x7f6044107460 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034037040 con 0x7f6044107460 2024-09-05T06:31:27.765 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419b360 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.113+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604419e050 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.114+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6034044070 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.114+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f603403d3f0 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.114+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044109200 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.117+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6034046450 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.174+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f604419e5a0 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.174+0000 7f603bfff640 1 -- 172.21.15.86:0/815741757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6034046630 con 0x7f6044107460 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 msgr2=0x7f604419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 --2- 172.21.15.86:0/815741757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f60340090b0 tx=0x7f60340046c0 comp rx=0 tx=0).stop 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 shutdown_connections 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 --2- 172.21.15.86:0/815741757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6044107460 0x7f604419ab90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 >> 172.21.15.86:0/815741757 conn(0x7f6044102c50 msgr2=0x7f604418ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 shutdown_connections 2024-09-05T06:31:27.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:27.175+0000 7f6048f1a640 1 -- 172.21.15.86:0/815741757 wait complete. 2024-09-05T06:31:27.767 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-09-05T06:31:27.767 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-09-05T06:31:27.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: monmap epoch 1 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: last_changed 2024-09-05T06:31:22.792336+0000 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: min_mon_release 19 (squid) 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: election_strategy: 1 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: fsmap 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: mgrmap e1: no daemons active 2024-09-05T06:31:27.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:27 smithi086 ceph-mon[30580]: from='client.? 172.21.15.86:0/815741757' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-05T06:31:28.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:28 smithi086 ceph-mon[30580]: from='client.? 172.21.15.86:0/545373509' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T06:31:28.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:28 smithi086 ceph-mon[30580]: from='client.? 172.21.15.86:0/545373509' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-05T06:31:28.792 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:28.792 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T06:31:28.793 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.150+0000 7fc799ad9640 1 Processor -- start 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.150+0000 7fc799ad9640 1 -- start start 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.150+0000 7fc799ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.150+0000 7fc799ad9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc794107e30 con 0x7fc794107460 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.151+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.151+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51010/0 (socket says 172.21.15.86:51010) 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.151+0000 7fc7937fe640 1 -- 172.21.15.86:0/643136024 learned_addr learned my addr 172.21.15.86:0/643136024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.151+0000 7fc7937fe640 1 -- 172.21.15.86:0/643136024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc794108610 con 0x7fc794107460 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc7937fe640 1 --2- 172.21.15.86:0/643136024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc784009820 tx=0x7fc78402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a804acfa75e0e972 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc7927fc640 1 -- 172.21.15.86:0/643136024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784037070 con 0x7fc794107460 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc7927fc640 1 -- 172.21.15.86:0/643136024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc784032440 con 0x7fc794107460 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc7927fc640 1 -- 172.21.15.86:0/643136024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784032740 con 0x7fc794107460 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 -- 172.21.15.86:0/643136024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 msgr2=0x7fc794107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 --2- 172.21.15.86:0/643136024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc784009820 tx=0x7fc78402a270 comp rx=0 tx=0).stop 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 -- 172.21.15.86:0/643136024 shutdown_connections 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 --2- 172.21.15.86:0/643136024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc794107860 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 -- 172.21.15.86:0/643136024 >> 172.21.15.86:0/643136024 conn(0x7fc794102c50 msgr2=0x7fc794105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 -- 172.21.15.86:0/643136024 shutdown_connections 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.152+0000 7fc799ad9640 1 -- 172.21.15.86:0/643136024 wait complete. 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc799ad9640 1 Processor -- start 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc799ad9640 1 -- start start 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc799ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:28.795 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc799ad9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc794108bb0 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc7937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51014/0 (socket says 172.21.15.86:51014) 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.153+0000 7fc7937fe640 1 -- 172.21.15.86:0/545373509 learned_addr learned my addr 172.21.15.86:0/545373509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc7937fe640 1 -- 172.21.15.86:0/545373509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79407e0d0 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc7937fe640 1 --2- 172.21.15.86:0/545373509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7840090b0 tx=0x7fc7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784041070 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78403c7a0 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79407c2c0 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc784037040 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79407c800 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc784044070 con 0x7fc794107460 2024-09-05T06:31:28.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.154+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc78403d3f0 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.155+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc794109200 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.157+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc784046450 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.214+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc79407cb00 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.217+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fc784046630 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc790ff9640 1 -- 172.21.15.86:0/545373509 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc7840584b0 con 0x7fc794107460 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 msgr2=0x7fc79407db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc799ad9640 1 --2- 172.21.15.86:0/545373509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7840090b0 tx=0x7fc7840046c0 comp rx=0 tx=0).stop 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 shutdown_connections 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc799ad9640 1 --2- 172.21.15.86:0/545373509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc794107460 0x7fc79407db90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.218+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 >> 172.21.15.86:0/545373509 conn(0x7fc794102c50 msgr2=0x7fc79418ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.219+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 shutdown_connections 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:28.219+0000 7fc799ad9640 1 -- 172.21.15.86:0/545373509 wait complete. 2024-09-05T06:31:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-09-05T06:31:29.857 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:29 smithi086 ceph-mon[30580]: from='client.? 172.21.15.86:0/822371374' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.187+0000 7f11fad84640 1 Processor -- start 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.187+0000 7f11fad84640 1 -- start start 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.187+0000 7f11fad84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.187+0000 7f11fad84640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4106cb0 con 0x7f11f41062e0 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.188+0000 7f11f8b20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.188+0000 7f11f8b20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51022/0 (socket says 172.21.15.86:51022) 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.188+0000 7f11f8b20640 1 -- 172.21.15.86:0/362856213 learned_addr learned my addr 172.21.15.86:0/362856213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.188+0000 7f11f8b20640 1 -- 172.21.15.86:0/362856213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41074e0 con 0x7f11f41062e0 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11f8b20640 1 --2- 172.21.15.86:0/362856213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009e80 tx=0x7f11e402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb31546f3a7af878 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11f37fe640 1 -- 172.21.15.86:0/362856213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e4036070 con 0x7f11f41062e0 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11f37fe640 1 -- 172.21.15.86:0/362856213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f11e4031890 con 0x7f11f41062e0 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11f37fe640 1 -- 172.21.15.86:0/362856213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e4007980 con 0x7f11f41062e0 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 -- 172.21.15.86:0/362856213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 msgr2=0x7f11f41066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 --2- 172.21.15.86:0/362856213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009e80 tx=0x7f11e402a7a0 comp rx=0 tx=0).stop 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 -- 172.21.15.86:0/362856213 shutdown_connections 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 --2- 172.21.15.86:0/362856213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f41066e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 -- 172.21.15.86:0/362856213 >> 172.21.15.86:0/362856213 conn(0x7f11f4101a90 msgr2=0x7f11f4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 -- 172.21.15.86:0/362856213 shutdown_connections 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.189+0000 7f11fad84640 1 -- 172.21.15.86:0/362856213 wait complete. 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11fad84640 1 Processor -- start 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11fad84640 1 -- start start 2024-09-05T06:31:29.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11fad84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11fad84640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410a890 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11f8b20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11f8b20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51034/0 (socket says 172.21.15.86:51034) 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11f8b20640 1 -- 172.21.15.86:0/822371374 learned_addr learned my addr 172.21.15.86:0/822371374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.190+0000 7f11f8b20640 1 -- 172.21.15.86:0/822371374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f419bc20 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f8b20640 1 --2- 172.21.15.86:0/822371374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007bb0 tx=0x7f11e4031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e4041020 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f11e403c900 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f419beb0 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e4036040 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f419eba0 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11e4044020 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.191+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11e4046980 con 0x7f11f41062e0 2024-09-05T06:31:29.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.192+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f410aa10 con 0x7f11f41062e0 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.194+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f11e404b020 con 0x7f11f41062e0 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.251+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f11f402d7c0 con 0x7f11f41062e0 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.251+0000 7f11f1ffb640 1 -- 172.21.15.86:0/822371374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f11e4046400 con 0x7f11f41062e0 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 msgr2=0x7f11f419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 --2- 172.21.15.86:0/822371374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007bb0 tx=0x7f11e4031c70 comp rx=0 tx=0).stop 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 shutdown_connections 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 --2- 172.21.15.86:0/822371374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f41062e0 0x7f11f419b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 >> 172.21.15.86:0/822371374 conn(0x7f11f4101a90 msgr2=0x7f11f4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 shutdown_connections 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:29.252+0000 7f11fad84640 1 -- 172.21.15.86:0/822371374 wait complete. 2024-09-05T06:31:29.868 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-09-05T06:31:30.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:29 smithi086 systemd[1]: Stopping Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:31:30.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a[30576]: 2024-09-05T06:31:30.144+0000 7f7ce1197640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T06:31:30.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a[30576]: 2024-09-05T06:31:30.144+0000 7f7ce1197640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-05T06:31:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 podman[30824]: 2024-09-05 06:31:30.247224921 +0000 UTC m=+0.217706203 container died 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T06:31:30.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 podman[30824]: 2024-09-05 06:31:30.585807951 +0000 UTC m=+0.556289220 container cleanup 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:31:30.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 bash[30824]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a 2024-09-05T06:31:31.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 podman[30836]: 2024-09-05 06:31:30.761926744 +0000 UTC m=+0.515659118 container remove 1e343b74bc478e11fea236c6050d1452b1ecdc94573ed21b044f8872a0d8d8a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T06:31:31.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-05T06:31:31.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 systemd[1]: Stopped Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:31:31.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:30 smithi086 systemd[1]: Starting Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:31:31.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 podman[30925]: 2024-09-05 06:31:31.097229915 +0000 UTC m=+0.015233179 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:31:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 podman[30925]: 2024-09-05 06:31:31.209325865 +0000 UTC m=+0.127329130 container create 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T06:31:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 podman[30925]: 2024-09-05 06:31:31.707000876 +0000 UTC m=+0.625004141 container init 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=) 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 podman[30925]: 2024-09-05 06:31:31.710493285 +0000 UTC m=+0.628496550 container start 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e) 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable), process ceph-mon, pid 2 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: pidfile_write: ignore empty --pid-file 2024-09-05T06:31:31.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: load: jerasure load: lrc 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: RocksDB version: 7.9.2 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Git sha 0 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Compile date 2024-09-04 15:32:03 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: DB SUMMARY 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: DB Session ID: C03HP0GXG3NRG4KM4VAC 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: CURRENT file: CURRENT 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104124 ; 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.error_if_exists: 0 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.create_if_missing: 0 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.paranoid_checks: 1 2024-09-05T06:31:31.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.env: 0x5605b7eae940 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.info_log: 0x5605b94f36e0 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T06:31:31.918 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.statistics: (nil) 2024-09-05T06:31:31.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.use_fsync: 0 2024-09-05T06:31:31.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_log_file_size: 0 2024-09-05T06:31:31.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T06:31:31.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_fallocate: 1 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.use_direct_reads: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.db_log_dir: 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.wal_dir: 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T06:31:31.920 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.write_buffer_manager: 0x5605b9502dc0 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T06:31:31.921 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.unordered_write: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.row_cache: None 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.wal_filter: None 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.two_write_queues: 0 2024-09-05T06:31:31.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.wal_compression: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.atomic_flush: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.log_readahead_size: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T06:31:31.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_background_jobs: 2 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_background_compactions: -1 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_subcompactions: 1 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_open_files: -1 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T06:31:31.924 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_background_flushes: -1 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Compression algorithms supported: 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kZSTD supported: 0 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kXpressCompression supported: 0 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kBZip2Compression supported: 0 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kLZ4Compression supported: 1 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kZlibCompression supported: 1 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: kSnappyCompression supported: 1 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-05T06:31:31.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.merge_operator: 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_filter: None 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5605b94f3320) 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-09-05T06:31:31.926 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x5605b94e71f0 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-09-05T06:31:31.927 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-09-05T06:31:31.928 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression: NoCompression 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.num_levels: 7 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T06:31:31.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T06:31:31.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T06:31:31.931 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T06:31:31.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.inplace_update_support: 0 2024-09-05T06:31:31.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.bloom_locality: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.max_successive_merges: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.ttl: 2592000 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T06:31:31.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enable_blob_files: false 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.min_blob_size: 0 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ad98e159-50ff-49c0-9856-bda117c8d6b3 2024-09-05T06:31:31.935 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517891737638, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517891738677, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99367, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97261, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14024, "raw_average_key_size": 49, "raw_value_size": 89595, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725517891, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ad98e159-50ff-49c0-9856-bda117c8d6b3", "db_session_id": "C03HP0GXG3NRG4KM4VAC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517891738779, "job": 1, "event": "recovery_finished"} 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5605b9514e00 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: DB pointer 0x5605b9610000 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T06:31:31.936 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 98.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 98.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.937 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T06:31:31.938 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x5605b94e71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???) e1 preinit fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).mds e1 new map 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).mds e1 print_map 2024-09-05T06:31:31.939 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: btime 2024-09-05T06:31:26:636593+0000 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 bash[30925]: 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 systemd[1]: Started Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:31:31.940 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: monmap epoch 1 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: last_changed 2024-09-05T06:31:22.792336+0000 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: election_strategy: 1 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: fsmap 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T06:31:31.941 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:31 smithi086 ceph-mon[30943]: mgrmap e1: no daemons active 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.256+0000 7f4b8a064640 1 Processor -- start 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.256+0000 7f4b8a064640 1 -- start start 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.256+0000 7f4b8a064640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.256+0000 7f4b8a064640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84107e30 con 0x7f4b84107460 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.257+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.257+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51042/0 (socket says 172.21.15.86:51042) 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.257+0000 7f4b837fe640 1 -- 172.21.15.86:0/2991580937 learned_addr learned my addr 172.21.15.86:0/2991580937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.257+0000 7f4b837fe640 1 -- 172.21.15.86:0/2991580937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b84108610 con 0x7f4b84107460 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.259+0000 7f4b837fe640 1 --2- 172.21.15.86:0/2991580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009820 tx=0x7f4b7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b41efa8fedee8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:32.921 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.259+0000 7f4b82ffd640 1 -- 172.21.15.86:0/2991580937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b74037070 con 0x7f4b84107460 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.259+0000 7f4b82ffd640 1 -- 172.21.15.86:0/2991580937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4b740328d0 con 0x7f4b84107460 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.259+0000 7f4b82ffd640 1 -- 172.21.15.86:0/2991580937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b740307d0 con 0x7f4b84107460 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 -- 172.21.15.86:0/2991580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 msgr2=0x7f4b84107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 --2- 172.21.15.86:0/2991580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009820 tx=0x7f4b7402a270 comp rx=0 tx=0).stop 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 -- 172.21.15.86:0/2991580937 shutdown_connections 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 --2- 172.21.15.86:0/2991580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b84107860 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 -- 172.21.15.86:0/2991580937 >> 172.21.15.86:0/2991580937 conn(0x7f4b84102c50 msgr2=0x7f4b84105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 -- 172.21.15.86:0/2991580937 shutdown_connections 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 -- 172.21.15.86:0/2991580937 wait complete. 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.260+0000 7f4b8a064640 1 Processor -- start 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b8a064640 1 -- start start 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b8a064640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:32.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b8a064640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8410b9c0 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51048/0 (socket says 172.21.15.86:51048) 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b837fe640 1 -- 172.21.15.86:0/565204289 learned_addr learned my addr 172.21.15.86:0/565204289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b837fe640 1 -- 172.21.15.86:0/565204289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8419b8c0 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b837fe640 1 --2- 172.21.15.86:0/565204289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009940 tx=0x7f4b74032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b74042070 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8419bb50 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4b74030c80 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b74037040 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.261+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8419e840 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.262+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b74045070 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.262+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b74047a40 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.262+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8410bb40 con 0x7f4b84107460 2024-09-05T06:31:32.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.265+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4b7403c700 con 0x7f4b84107460 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.322+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f4b841078e0 con 0x7f4b84107460 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.324+0000 7f4b817fa640 1 -- 172.21.15.86:0/565204289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4b7403c8e0 con 0x7f4b84107460 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 msgr2=0x7f4b8419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 --2- 172.21.15.86:0/565204289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009940 tx=0x7f4b74032e10 comp rx=0 tx=0).stop 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 shutdown_connections 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 --2- 172.21.15.86:0/565204289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b84107460 0x7f4b8419b380 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 >> 172.21.15.86:0/565204289 conn(0x7f4b84102c50 msgr2=0x7f4b84103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 shutdown_connections 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:32.325+0000 7f4b8a064640 1 -- 172.21.15.86:0/565204289 wait complete. 2024-09-05T06:31:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-05T06:31:32.937 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-05T06:31:32.938 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-09-05T06:31:32.938 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-05T06:31:32.938 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-05T06:31:33.181 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T06:31:33.181 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x.service: Unit ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x.service not loaded. 2024-09-05T06:31:33.329 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968.target.wants/ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5a055634-6b50-11ef-bcd6-c7b262605968@.service. 2024-09-05T06:31:33.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/565204289' entity='client.admin' 2024-09-05T06:31:33.839 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T06:31:33.840 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-09-05T06:31:33.847 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T06:31:33.847 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:31:33.847 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-09-05T06:31:33.847 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T06:31:33.855 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T06:31:33.855 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-09-05T06:31:33.863 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-09-05T06:31:34.086 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.085+0000 7f66605bf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:31:34.360 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.154+0000 7f66605bf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:31:34.360 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.306+0000 7f66605bf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:31:34.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1943090422' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T06:31:34.690 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.453+0000 7f66605bf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:31:34.941 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.772+0000 7f66605bf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:31:34.941 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:34.832+0000 7f66605bf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T06:31:34.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:34.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T06:31:34.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T06:31:34.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T06:31:34.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T06:31:34.954 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:34.955 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T06:31:26:636593+0000", 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:34.956 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T06:31:26.637243+0000", 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.291+0000 7fb67e5ba640 1 Processor -- start 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.291+0000 7fb67e5ba640 1 -- start start 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.291+0000 7fb67e5ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.291+0000 7fb67e5ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780757a0 con 0x7fb678105c40 2024-09-05T06:31:34.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.292+0000 7fb677fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.292+0000 7fb677fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33142/0 (socket says 172.21.15.86:33142) 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.292+0000 7fb677fff640 1 -- 172.21.15.86:0/2098431240 learned_addr learned my addr 172.21.15.86:0/2098431240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.292+0000 7fb677fff640 1 -- 172.21.15.86:0/2098431240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb678106580 con 0x7fb678105c40 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.292+0000 7fb677fff640 1 --2- 172.21.15.86:0/2098431240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb668009820 tx=0x7fb66802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a2a8e19d73f35e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb676ffd640 1 -- 172.21.15.86:0/2098431240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb668037070 con 0x7fb678105c40 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb676ffd640 1 -- 172.21.15.86:0/2098431240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6680328d0 con 0x7fb678105c40 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb676ffd640 1 -- 172.21.15.86:0/2098431240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6680307d0 con 0x7fb678105c40 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 -- 172.21.15.86:0/2098431240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 msgr2=0x7fb678106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 --2- 172.21.15.86:0/2098431240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb668009820 tx=0x7fb66802a270 comp rx=0 tx=0).stop 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 -- 172.21.15.86:0/2098431240 shutdown_connections 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 --2- 172.21.15.86:0/2098431240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb678106040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 -- 172.21.15.86:0/2098431240 >> 172.21.15.86:0/2098431240 conn(0x7fb678101a90 msgr2=0x7fb678103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.293+0000 7fb67e5ba640 1 -- 172.21.15.86:0/2098431240 shutdown_connections 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.294+0000 7fb67e5ba640 1 -- 172.21.15.86:0/2098431240 wait complete. 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.294+0000 7fb67e5ba640 1 Processor -- start 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.294+0000 7fb67e5ba640 1 -- start start 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.294+0000 7fb67e5ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.294+0000 7fb67e5ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb678107620 con 0x7fb678105c40 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb677fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb677fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33144/0 (socket says 172.21.15.86:33144) 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb677fff640 1 -- 172.21.15.86:0/1943090422 learned_addr learned my addr 172.21.15.86:0/1943090422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb677fff640 1 -- 172.21.15.86:0/1943090422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67819dda0 con 0x7fb678105c40 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb677fff640 1 --2- 172.21.15.86:0/1943090422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb668032e10 tx=0x7fb668032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb668042070 con 0x7fb678105c40 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67819e030 con 0x7fb678105c40 2024-09-05T06:31:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb668030c80 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb668037040 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.295+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6781a0d20 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.296+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb668045070 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.296+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6680463b0 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.297+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb678106040 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.299+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb668047440 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.360+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb6781970c0 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.360+0000 7fb6757fa640 1 -- 172.21.15.86:0/1943090422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb668047620 con 0x7fb678105c40 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 msgr2=0x7fb67819d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 --2- 172.21.15.86:0/1943090422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb668032e10 tx=0x7fb668032e40 comp rx=0 tx=0).stop 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 shutdown_connections 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 --2- 172.21.15.86:0/1943090422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb678105c40 0x7fb67819d860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:34.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 >> 172.21.15.86:0/1943090422 conn(0x7fb678101a90 msgr2=0x7fb678194290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:34.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 shutdown_connections 2024-09-05T06:31:34.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:34.361+0000 7fb67e5ba640 1 -- 172.21.15.86:0/1943090422 wait complete. 2024-09-05T06:31:34.961 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-09-05T06:31:35.236 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:35.022+0000 7f66605bf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:31:36.236 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:35.897+0000 7f66605bf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:31:36.237 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:35.962+0000 7f66605bf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:31:36.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:36.673+0000 7f66605bf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:31:37.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:36.787+0000 7f66605bf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:31:37.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:36.927+0000 7f66605bf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:31:37.076 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:36.991+0000 7f66605bf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:31:37.335 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.144+0000 7f66605bf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:31:37.335 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.215+0000 7f66605bf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:31:37.335 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.277+0000 7f66605bf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:31:37.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/348310626' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T06:31:37.663 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.442+0000 7f66605bf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:31:37.663 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.513+0000 7f66605bf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:31:37.663 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.580+0000 7f66605bf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:31:37.663 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.662+0000 7f66605bf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:31:37.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.723+0000 7f66605bf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:31:37.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:37.784+0000 7f66605bf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:31:38.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:38.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-09-05T06:31:38.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T06:31:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T06:31:38.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:38.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T06:31:38.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T06:31:38.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T06:31:38.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T06:31:26:636593+0000", 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T06:31:38.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T06:31:26.637243+0000", 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:31:38.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.411+0000 7f471f9bf640 1 Processor -- start 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.411+0000 7f471f9bf640 1 -- start start 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471f9bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471f9bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4718106cb0 con 0x7f47181062e0 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471d75b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471d75b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33152/0 (socket says 172.21.15.86:33152) 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471d75b640 1 -- 172.21.15.86:0/2346360430 learned_addr learned my addr 172.21.15.86:0/2346360430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.412+0000 7f471d75b640 1 -- 172.21.15.86:0/2346360430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181074e0 con 0x7f47181062e0 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.413+0000 7f471d75b640 1 --2- 172.21.15.86:0/2346360430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4714009820 tx=0x7f471402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbd42e811f3c7df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.413+0000 7f470ffff640 1 -- 172.21.15.86:0/2346360430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4714037070 con 0x7f47181062e0 2024-09-05T06:31:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.413+0000 7f470ffff640 1 -- 172.21.15.86:0/2346360430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f47140328d0 con 0x7f47181062e0 2024-09-05T06:31:38.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.413+0000 7f470ffff640 1 -- 172.21.15.86:0/2346360430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47140307d0 con 0x7f47181062e0 2024-09-05T06:31:38.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- 172.21.15.86:0/2346360430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 msgr2=0x7f47181066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:38.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 --2- 172.21.15.86:0/2346360430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4714009820 tx=0x7f471402a270 comp rx=0 tx=0).stop 2024-09-05T06:31:38.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- 172.21.15.86:0/2346360430 shutdown_connections 2024-09-05T06:31:38.101 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 --2- 172.21.15.86:0/2346360430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f47181066e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- 172.21.15.86:0/2346360430 >> 172.21.15.86:0/2346360430 conn(0x7f4718101a90 msgr2=0x7f4718103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- 172.21.15.86:0/2346360430 shutdown_connections 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- 172.21.15.86:0/2346360430 wait complete. 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 Processor -- start 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- start start 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.414+0000 7f471f9bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f471810a890 con 0x7f47181062e0 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471d75b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471d75b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33168/0 (socket says 172.21.15.86:33168) 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471d75b640 1 -- 172.21.15.86:0/348310626 learned_addr learned my addr 172.21.15.86:0/348310626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:38.102 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471d75b640 1 -- 172.21.15.86:0/348310626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471819e3f0 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471d75b640 1 --2- 172.21.15.86:0/348310626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4714009940 tx=0x7f4714032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4714042070 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471819e5c0 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4714030c80 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4714037040 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471819e960 con 0x7f47181062e0 2024-09-05T06:31:38.103 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4714045070 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.415+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4714047a40 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.416+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4718106760 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.418+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f471403c700 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.479+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f471802da20 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.479+0000 7f470e7fc640 1 -- 172.21.15.86:0/348310626 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f471403c8e0 con 0x7f47181062e0 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 msgr2=0x7f4718075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 --2- 172.21.15.86:0/348310626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4714009940 tx=0x7f4714032e10 comp rx=0 tx=0).stop 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 shutdown_connections 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 --2- 172.21.15.86:0/348310626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181062e0 0x7f4718075340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 >> 172.21.15.86:0/348310626 conn(0x7f4718101a90 msgr2=0x7f4718102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:38.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 shutdown_connections 2024-09-05T06:31:38.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:37.480+0000 7f471f9bf640 1 -- 172.21.15.86:0/348310626 wait complete. 2024-09-05T06:31:38.105 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-09-05T06:31:38.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:38.575+0000 7f66605bf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: Activating manager daemon x 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: mgrmap e2: x(active, starting, since 0.00317474s) 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: Manager daemon x is now available 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:31:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:31:38.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' 2024-09-05T06:31:38.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' 2024-09-05T06:31:38.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:38 smithi086 ceph-mon[30943]: from='mgr.14100 172.21.15.86:0/2461107716' entity='mgr.x' 2024-09-05T06:31:40.732 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:40 smithi086 ceph-mon[30943]: mgrmap e3: x(active, since 1.0068s) 2024-09-05T06:31:41.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:41.291 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:41.292 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T06:31:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T06:31:41.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T06:31:41.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T06:31:41.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T06:31:41.301 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T06:31:26:636593+0000", 2024-09-05T06:31:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T06:31:41.303 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T06:31:26.637243+0000", 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.486+0000 7f4d614b5640 1 Processor -- start 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.486+0000 7f4d614b5640 1 -- start start 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.486+0000 7f4d614b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.486+0000 7f4d614b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c107e70 con 0x7f4d5c1074a0 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.487+0000 7f4d5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.487+0000 7f4d5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33212/0 (socket says 172.21.15.86:33212) 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.487+0000 7f4d5affd640 1 -- 172.21.15.86:0/2859471251 learned_addr learned my addr 172.21.15.86:0/2859471251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.487+0000 7f4d5affd640 1 -- 172.21.15.86:0/2859471251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c108600 con 0x7f4d5c1074a0 2024-09-05T06:31:41.304 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d5affd640 1 --2- 172.21.15.86:0/2859471251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009820 tx=0x7f4d4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfabea1d00abf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d59ffb640 1 -- 172.21.15.86:0/2859471251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d44037070 con 0x7f4d5c1074a0 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d59ffb640 1 -- 172.21.15.86:0/2859471251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4d440328d0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d59ffb640 1 -- 172.21.15.86:0/2859471251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d440307d0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d614b5640 1 -- 172.21.15.86:0/2859471251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 msgr2=0x7f4d5c1078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.488+0000 7f4d614b5640 1 --2- 172.21.15.86:0/2859471251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009820 tx=0x7f4d4402a270 comp rx=0 tx=0).stop 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 -- 172.21.15.86:0/2859471251 shutdown_connections 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 --2- 172.21.15.86:0/2859471251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c1078a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 -- 172.21.15.86:0/2859471251 >> 172.21.15.86:0/2859471251 conn(0x7f4d5c102c50 msgr2=0x7f4d5c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 -- 172.21.15.86:0/2859471251 shutdown_connections 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 -- 172.21.15.86:0/2859471251 wait complete. 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 Processor -- start 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.489+0000 7f4d614b5640 1 -- start start 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d614b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:41.305 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d614b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c10b9b0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33226/0 (socket says 172.21.15.86:33226) 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d5affd640 1 -- 172.21.15.86:0/4025344618 learned_addr learned my addr 172.21.15.86:0/4025344618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d5affd640 1 -- 172.21.15.86:0/4025344618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c19b8b0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d5affd640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d440097f0 tx=0x7f4d44030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d44037040 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4d44032bb0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c19bb40 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4403a390 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.490+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c19e830 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.492+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f4d4403b3f0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.492+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c10bb30 con 0x7f4d5c1074a0 2024-09-05T06:31:41.306 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.492+0000 7f4d53fff640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 0x7f4d4803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.492+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d4406bef0 con 0x7f4d5c1074a0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.492+0000 7f4d5a7fc640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 0x7f4d4803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.493+0000 7f4d5a7fc640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 0x7f4d4803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0098c0 tx=0x7f4d4c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.497+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d44040210 con 0x7f4d5c1074a0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.585+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4d44038d40 con 0x7f4d5c1074a0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.732+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d5c19ee00 con 0x7f4d5c1074a0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.732+0000 7f4d53fff640 1 -- 172.21.15.86:0/4025344618 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f4d4406ec30 con 0x7f4d5c1074a0 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 msgr2=0x7f4d4803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 0x7f4d4803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0098c0 tx=0x7f4d4c006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 msgr2=0x7f4d5c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d440097f0 tx=0x7f4d44030d00 comp rx=0 tx=0).stop 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 shutdown_connections 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f4d48038e10 0x7f4d4803b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:41.307 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 --2- 172.21.15.86:0/4025344618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5c1074a0 0x7f4d5c19b370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:41.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 >> 172.21.15.86:0/4025344618 conn(0x7f4d5c102c50 msgr2=0x7f4d5c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:41.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 shutdown_connections 2024-09-05T06:31:41.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:40.735+0000 7f4d614b5640 1 -- 172.21.15.86:0/4025344618 wait complete. 2024-09-05T06:31:41.308 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-09-05T06:31:41.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:41 smithi086 ceph-mon[30943]: mgrmap e4: x(active, since 2s) 2024-09-05T06:31:41.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:41 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4025344618' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T06:31:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T06:31:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-09-05T06:31:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T06:31:42.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6ee1aea640 1 Processor -- start 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6ee1aea640 1 -- start start 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6ee1aea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6ee1aea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc07be50 con 0x7f6edc107f10 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33238/0 (socket says 172.21.15.86:33238) 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.662+0000 7f6edb7fe640 1 -- 172.21.15.86:0/3715058746 learned_addr learned my addr 172.21.15.86:0/3715058746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.663+0000 7f6edb7fe640 1 -- 172.21.15.86:0/3715058746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc10a840 con 0x7f6edc107f10 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6edb7fe640 1 --2- 172.21.15.86:0/3715058746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009820 tx=0x7f6ecc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f95cd96a9bf0ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6edaffd640 1 -- 172.21.15.86:0/3715058746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ecc037070 con 0x7f6edc107f10 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6edaffd640 1 -- 172.21.15.86:0/3715058746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6ecc0328d0 con 0x7f6edc107f10 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6edaffd640 1 -- 172.21.15.86:0/3715058746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ecc0307d0 con 0x7f6edc107f10 2024-09-05T06:31:42.445 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3715058746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 msgr2=0x7f6edc10a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.664+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3715058746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009820 tx=0x7f6ecc02a270 comp rx=0 tx=0).stop 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3715058746 shutdown_connections 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3715058746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc10a300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3715058746 >> 172.21.15.86:0/3715058746 conn(0x7f6edc101ab0 msgr2=0x7f6edc103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3715058746 shutdown_connections 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3715058746 wait complete. 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 Processor -- start 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- start start 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.665+0000 7f6ee1aea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc10b7c0 con 0x7f6edc107f10 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33248/0 (socket says 172.21.15.86:33248) 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6edb7fe640 1 -- 172.21.15.86:0/3029511610 learned_addr learned my addr 172.21.15.86:0/3029511610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:42.446 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6edb7fe640 1 -- 172.21.15.86:0/3029511610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc19b8e0 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6edb7fe640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097f0 tx=0x7f6ecc030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ecc037040 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6ecc032bb0 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ecc03a390 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc19bb70 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.666+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc19e860 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.667+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6ecc03a580 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.667+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc108370 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.668+0000 7f6ed97fa640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 0x7f6ec803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.668+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ecc06c1f0 con 0x7f6edc107f10 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.668+0000 7f6ed3fff640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 0x7f6ec803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.668+0000 7f6ed3fff640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 0x7f6ec803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098c0 tx=0x7f6ec4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.672+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ecc032dd0 con 0x7f6edc107f10 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.841+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f6edc1a0060 con 0x7f6edc107f10 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.842+0000 7f6ed97fa640 1 -- 172.21.15.86:0/3029511610 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f6ecc0317a0 con 0x7f6edc107f10 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 msgr2=0x7f6ec803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 0x7f6ec803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098c0 tx=0x7f6ec4006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 msgr2=0x7f6edc19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097f0 tx=0x7f6ecc030d00 comp rx=0 tx=0).stop 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 shutdown_connections 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6ec8038f70 0x7f6ec803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 --2- 172.21.15.86:0/3029511610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc107f10 0x7f6edc19b3a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 >> 172.21.15.86:0/3029511610 conn(0x7f6edc101ab0 msgr2=0x7f6edc102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 shutdown_connections 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:41.844+0000 7f6ee1aea640 1 -- 172.21.15.86:0/3029511610 wait complete. 2024-09-05T06:31:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-09-05T06:31:42.812 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:42 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3029511610' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T06:31:43.882 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/507101701' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-05T06:31:43.883 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-05T06:31:43.883 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-05T06:31:43.883 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:43.734+0000 7fd020126040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:31:43.883 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:43.790+0000 7fd020126040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:31:44.153 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:43.938+0000 7fd020126040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:31:44.153 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.005+0000 7fd020126040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:31:44.153 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.153+0000 7fd020126040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:31:44.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e71e57640 1 Processor -- start 2024-09-05T06:31:44.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e71e57640 1 -- start start 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e71e57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e71e57640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c107e30 con 0x7f6e6c107460 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47442/0 (socket says 172.21.15.86:47442) 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.891+0000 7f6e6b7fe640 1 -- 172.21.15.86:0/3924513267 learned_addr learned my addr 172.21.15.86:0/3924513267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.892+0000 7f6e6b7fe640 1 -- 172.21.15.86:0/3924513267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c108610 con 0x7f6e6c107460 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.892+0000 7f6e6b7fe640 1 --2- 172.21.15.86:0/3924513267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009820 tx=0x7f6e5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb085849f5a0687 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.892+0000 7f6e6affd640 1 -- 172.21.15.86:0/3924513267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c037070 con 0x7f6e6c107460 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e6affd640 1 -- 172.21.15.86:0/3924513267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6e5c0328d0 con 0x7f6e6c107460 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e6affd640 1 -- 172.21.15.86:0/3924513267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c0307d0 con 0x7f6e6c107460 2024-09-05T06:31:44.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 -- 172.21.15.86:0/3924513267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 msgr2=0x7f6e6c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 --2- 172.21.15.86:0/3924513267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009820 tx=0x7f6e5c02a270 comp rx=0 tx=0).stop 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 -- 172.21.15.86:0/3924513267 shutdown_connections 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 --2- 172.21.15.86:0/3924513267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c107860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 -- 172.21.15.86:0/3924513267 >> 172.21.15.86:0/3924513267 conn(0x7f6e6c102c50 msgr2=0x7f6e6c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 -- 172.21.15.86:0/3924513267 shutdown_connections 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.893+0000 7f6e71e57640 1 -- 172.21.15.86:0/3924513267 wait complete. 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e71e57640 1 Processor -- start 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e71e57640 1 -- start start 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e71e57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e71e57640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c10b9c0 con 0x7f6e6c107460 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47448/0 (socket says 172.21.15.86:47448) 2024-09-05T06:31:44.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e6b7fe640 1 -- 172.21.15.86:0/507101701 learned_addr learned my addr 172.21.15.86:0/507101701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.894+0000 7f6e6b7fe640 1 -- 172.21.15.86:0/507101701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c19b8c0 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e6b7fe640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009940 tx=0x7f6e5c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c042070 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6e5c030c80 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c19bb50 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e5c037040 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.895+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c19e840 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6e5c045070 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c10bb40 con 0x7f6e6c107460 2024-09-05T06:31:44.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e697fa640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 0x7f6e5803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e5c06bf60 con 0x7f6e6c107460 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e63fff640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 0x7f6e5803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.896+0000 7f6e63fff640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 0x7f6e5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e540098c0 tx=0x7f6e54006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:42.901+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e5c045360 con 0x7f6e6c107460 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.106+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6e6c19eed0 con 0x7f6e6c107460 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.587+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6e5c046dd0 con 0x7f6e6c107460 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.589+0000 7f6e697fa640 1 -- 172.21.15.86:0/507101701 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6e5c03cac0 con 0x7f6e6c107460 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 msgr2=0x7f6e5803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 0x7f6e5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e540098c0 tx=0x7f6e54006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:44.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 msgr2=0x7f6e6c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009940 tx=0x7f6e5c032e10 comp rx=0 tx=0).stop 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 shutdown_connections 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f6e58038f70 0x7f6e5803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 --2- 172.21.15.86:0/507101701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e6c107460 0x7f6e6c19b380 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.590+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 >> 172.21.15.86:0/507101701 conn(0x7f6e6c102c50 msgr2=0x7f6e6c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.591+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 shutdown_connections 2024-09-05T06:31:44.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:43.591+0000 7f6e71e57640 1 -- 172.21.15.86:0/507101701 wait complete. 2024-09-05T06:31:44.420 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.303+0000 7fd020126040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:31:44.689 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/507101701' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-05T06:31:44.689 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-mon[30943]: mgrmap e5: x(active, since 5s) 2024-09-05T06:31:44.690 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.627+0000 7fd020126040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:31:44.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.689+0000 7fd020126040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:31:44.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:44.880+0000 7fd020126040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T06:31:45.439 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.621+0000 7f89dbbd7640 1 Processor -- start 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.621+0000 7f89dbbd7640 1 -- start start 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.621+0000 7f89dbbd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.621+0000 7f89dbbd7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4106cf0 con 0x7f89d4106320 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.622+0000 7f89d9973640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.622+0000 7f89d9973640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47484/0 (socket says 172.21.15.86:47484) 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.622+0000 7f89d9973640 1 -- 172.21.15.86:0/1245757217 learned_addr learned my addr 172.21.15.86:0/1245757217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.622+0000 7f89d9973640 1 -- 172.21.15.86:0/1245757217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4107520 con 0x7f89d4106320 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.623+0000 7f89d9973640 1 --2- 172.21.15.86:0/1245757217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009820 tx=0x7f89d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4cb64c6f773ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.623+0000 7f89d8971640 1 -- 172.21.15.86:0/1245757217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0037070 con 0x7f89d4106320 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.623+0000 7f89d8971640 1 -- 172.21.15.86:0/1245757217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f89d00328d0 con 0x7f89d4106320 2024-09-05T06:31:45.440 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.623+0000 7f89d8971640 1 -- 172.21.15.86:0/1245757217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d00307d0 con 0x7f89d4106320 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- 172.21.15.86:0/1245757217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 msgr2=0x7f89d4106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/1245757217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009820 tx=0x7f89d002a270 comp rx=0 tx=0).stop 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- 172.21.15.86:0/1245757217 shutdown_connections 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/1245757217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d4106720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- 172.21.15.86:0/1245757217 >> 172.21.15.86:0/1245757217 conn(0x7f89d4101ab0 msgr2=0x7f89d4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- 172.21.15.86:0/1245757217 shutdown_connections 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- 172.21.15.86:0/1245757217 wait complete. 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 Processor -- start 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- start start 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.624+0000 7f89dbbd7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410a8d0 con 0x7f89d4106320 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89d9973640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:45.441 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89d9973640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47492/0 (socket says 172.21.15.86:47492) 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89d9973640 1 -- 172.21.15.86:0/3805383344 learned_addr learned my addr 172.21.15.86:0/3805383344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89d9973640 1 -- 172.21.15.86:0/3805383344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d419e130 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89d9973640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f89d00097f0 tx=0x7f89d0030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0037040 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d419e300 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f89d0032bb0 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d003a390 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.625+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d419e6a0 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f89d003a5d0 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89c6ffd640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 0x7f89c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89d003b730 con 0x7f89d4106320 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89d9172640 1 -- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 msgr2=0x7f89c003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89d9172640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 0x7f89c003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:31:45.442 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.626+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d41067a0 con 0x7f89d4106320 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.631+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89d0040210 con 0x7f89d4106320 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.827+0000 7f89d9172640 1 -- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 msgr2=0x7f89c003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.827+0000 7f89d9172640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 0x7f89c003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.848+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f89d419ee10 con 0x7f89d4106320 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.848+0000 7f89c6ffd640 1 -- 172.21.15.86:0/3805383344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f89d0031560 con 0x7f89d4106320 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 msgr2=0x7f89c003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 0x7f89c003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 msgr2=0x7f89d407ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f89d00097f0 tx=0x7f89d0030d00 comp rx=0 tx=0).stop 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 shutdown_connections 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7f89c0038f70 0x7f89c003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 --2- 172.21.15.86:0/3805383344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4106320 0x7f89d407ba70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:45.443 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 >> 172.21.15.86:0/3805383344 conn(0x7f89d4101ab0 msgr2=0x7f89d41025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:45.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 shutdown_connections 2024-09-05T06:31:45.444 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:44.850+0000 7f89dbbd7640 1 -- 172.21.15.86:0/3805383344 wait complete. 2024-09-05T06:31:45.444 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-09-05T06:31:45.444 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-09-05T06:31:45.706 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:45 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3805383344' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T06:31:45.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:45.776+0000 7fd020126040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:31:45.987 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:45.846+0000 7fd020126040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:31:46.810 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:46.551+0000 7fd020126040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:31:46.810 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:46.667+0000 7fd020126040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:31:47.085 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:46.810+0000 7fd020126040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:31:47.085 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:46.871+0000 7fd020126040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:31:47.085 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.018+0000 7fd020126040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:31:47.085 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.084+0000 7fd020126040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:31:47.364 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.145+0000 7fd020126040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:31:47.365 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.300+0000 7fd020126040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:31:47.365 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.364+0000 7fd020126040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:31:47.628 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.430+0000 7fd020126040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:31:47.628 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.512+0000 7fd020126040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:31:47.628 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.570+0000 7fd020126040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:31:47.629 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:47 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:47.628+0000 7fd020126040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:31:48.736 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:48.410+0000 7fd020126040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: Active manager daemon x restarted 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: Activating manager daemon x 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: osdmap e2: 0 total, 0 up, 0 in 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: mgrmap e6: x(active, starting, since 0.0032081s) 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: Manager daemon x is now available 2024-09-05T06:31:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:48.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:48.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:31:48.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:31:48.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:31:48.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:48 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:31:49.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:49 smithi086 ceph-mon[30943]: Found migration_current of "None". Setting to last migration. 2024-09-05T06:31:49.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:49 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:49.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:49 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:49 smithi086 ceph-mon[30943]: mgrmap e7: x(active, since 1.00608s) 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa4bdd5640 1 Processor -- start 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa4bdd5640 1 -- start start 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa4bdd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa4bdd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa44075740 con 0x7faa44107eb0 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa49b71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa49b71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47500/0 (socket says 172.21.15.86:47500) 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.896+0000 7faa49b71640 1 -- 172.21.15.86:0/3947458051 learned_addr learned my addr 172.21.15.86:0/3947458051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.897+0000 7faa49b71640 1 -- 172.21.15.86:0/3947458051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4410a7e0 con 0x7faa44107eb0 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.897+0000 7faa49b71640 1 --2- 172.21.15.86:0/3947458051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa40009820 tx=0x7faa4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9f2cc145ea61ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.897+0000 7faa48b6f640 1 -- 172.21.15.86:0/3947458051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40037070 con 0x7faa44107eb0 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.897+0000 7faa48b6f640 1 -- 172.21.15.86:0/3947458051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa400328d0 con 0x7faa44107eb0 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.897+0000 7faa48b6f640 1 -- 172.21.15.86:0/3947458051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa400307d0 con 0x7faa44107eb0 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 -- 172.21.15.86:0/3947458051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 msgr2=0x7faa4410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/3947458051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa40009820 tx=0x7faa4002a270 comp rx=0 tx=0).stop 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 -- 172.21.15.86:0/3947458051 shutdown_connections 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/3947458051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa4410a2a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 -- 172.21.15.86:0/3947458051 >> 172.21.15.86:0/3947458051 conn(0x7faa44101a90 msgr2=0x7faa44103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 -- 172.21.15.86:0/3947458051 shutdown_connections 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.898+0000 7faa4bdd5640 1 -- 172.21.15.86:0/3947458051 wait complete. 2024-09-05T06:31:50.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa4bdd5640 1 Processor -- start 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa4bdd5640 1 -- start start 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa4bdd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa4bdd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa4410b7c0 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa49b71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa49b71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47506/0 (socket says 172.21.15.86:47506) 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa49b71640 1 -- 172.21.15.86:0/4085212129 learned_addr learned my addr 172.21.15.86:0/4085212129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa49b71640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa44107c70 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.899+0000 7faa49b71640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa400097f0 tx=0x7faa40030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.900+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40037040 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.900+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa44105e60 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.900+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa40032bb0 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.900+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa4003a390 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.900+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa441063a0 con 0x7faa44107eb0 2024-09-05T06:31:50.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7faa4003b3f0 con 0x7faa44107eb0 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa3affd640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa3003bb40 con 0x7faa30038f70 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa4006c190 con 0x7faa44107eb0 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa49370640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 msgr2=0x7faa3003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:45.901+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:46.102+0000 7faa49370640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 msgr2=0x7faa3003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:46.102+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:46.503+0000 7faa49370640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 msgr2=0x7faa3003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:46.503+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:47.304+0000 7faa49370640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 msgr2=0x7faa3003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/164214999 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:47.304+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T06:31:50.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:48.413+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7faa4006ad10 con 0x7faa44107eb0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:48.414+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 msgr2=0x7faa3003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:48.414+0000 7faa3affd640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.414+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7faa40049cc0 con 0x7faa44107eb0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.414+0000 7faa3affd640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 0x7faa3003ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.414+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa3003bb40 con 0x7faa3003c8a0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.414+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 0x7faa3003ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.415+0000 7faa49370640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 0x7faa3003ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa34003980 tx=0x7faa34006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.416+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7faa3003bb40 con 0x7faa3003c8a0 2024-09-05T06:31:50.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7faa44108310 con 0x7faa3003c8a0 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa3affd640 1 -- 172.21.15.86:0/4085212129 <== mgr.14118 v2:172.21.15.86:6800/1793567579 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7faa44108310 con 0x7faa3003c8a0 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 msgr2=0x7faa3003ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 0x7faa3003ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa34003980 tx=0x7faa34006e10 comp rx=0 tx=0).stop 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 msgr2=0x7faa44107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa400097f0 tx=0x7faa40030d00 comp rx=0 tx=0).stop 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 shutdown_connections 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7faa3003c8a0 0x7faa3003ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:6800/164214999,v1:172.21.15.86:6801/164214999] conn(0x7faa30038f70 0x7faa3003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 --2- 172.21.15.86:0/4085212129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa44107eb0 0x7faa44107730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 >> 172.21.15.86:0/4085212129 conn(0x7faa44101a90 msgr2=0x7faa441027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 shutdown_connections 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:49.423+0000 7faa4bdd5640 1 -- 172.21.15.86:0/4085212129 wait complete. 2024-09-05T06:31:50.036 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-09-05T06:31:50.037 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-09-05T06:31:50.340 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:50.028+0000 7fcf8cc8c640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:06:31:50] ENGINE Error in HTTPServer.serve 2024-09-05T06:31:50.340 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-05T06:31:50.341 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:31:50.342 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-05T06:31:50.342 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:31:50.342 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-05T06:31:50.342 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:31:50.342 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:31:50.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-mon[30943]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T06:31:50.606 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-mon[30943]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T06:31:50.606 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:50 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.416+0000 7f11cc8a3640 1 Processor -- start 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.416+0000 7f11cc8a3640 1 -- start start 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.416+0000 7f11cc8a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.416+0000 7f11cc8a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8107db0 con 0x7f11c81073e0 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.417+0000 7f11c659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.417+0000 7f11c659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47596/0 (socket says 172.21.15.86:47596) 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.417+0000 7f11c659c640 1 -- 172.21.15.86:0/958905443 learned_addr learned my addr 172.21.15.86:0/958905443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.417+0000 7f11c659c640 1 -- 172.21.15.86:0/958905443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81085e0 con 0x7f11c81073e0 2024-09-05T06:31:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11c659c640 1 --2- 172.21.15.86:0/958905443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009820 tx=0x7f11b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1422cb232001f76c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11c5d9b640 1 -- 172.21.15.86:0/958905443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11b8037070 con 0x7f11c81073e0 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11c5d9b640 1 -- 172.21.15.86:0/958905443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f11b80328d0 con 0x7f11c81073e0 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11c5d9b640 1 -- 172.21.15.86:0/958905443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11b80307d0 con 0x7f11c81073e0 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11cc8a3640 1 -- 172.21.15.86:0/958905443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 msgr2=0x7f11c81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/958905443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009820 tx=0x7f11b802a270 comp rx=0 tx=0).stop 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11cc8a3640 1 -- 172.21.15.86:0/958905443 shutdown_connections 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.418+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/958905443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c81077e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 -- 172.21.15.86:0/958905443 >> 172.21.15.86:0/958905443 conn(0x7f11c8102b50 msgr2=0x7f11c8104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 -- 172.21.15.86:0/958905443 shutdown_connections 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 -- 172.21.15.86:0/958905443 wait complete. 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 Processor -- start 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 -- start start 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:51.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11cc8a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11c810b990 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.419+0000 7f11c659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11c659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47602/0 (socket says 172.21.15.86:47602) 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11c659c640 1 -- 172.21.15.86:0/1982876480 learned_addr learned my addr 172.21.15.86:0/1982876480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11c659c640 1 -- 172.21.15.86:0/1982876480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c819b8e0 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11c659c640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11b8037040 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f11b8032bb0 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11b803a390 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c819bb70 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.420+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c819e860 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.421+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f11b803b3f0 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.421+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c810bb10 con 0x7f11c81073e0 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.421+0000 7f11bf7fe640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 0x7f11b403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.421+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11b806c020 con 0x7f11c81073e0 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.421+0000 7f11bffff640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 0x7f11b403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.422+0000 7f11bffff640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 0x7f11b403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.426+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11b8049c80 con 0x7f11c81073e0 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.468+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f11b803a580 con 0x7f11c81073e0 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.604+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f11c8103dc0 con 0x7f11b4038e60 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.610+0000 7f11bf7fe640 1 -- 172.21.15.86:0/1982876480 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11c8103dc0 con 0x7f11b4038e60 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 msgr2=0x7f11b403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 0x7f11b403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b0006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 msgr2=0x7f11c819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b8030d00 comp rx=0 tx=0).stop 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 shutdown_connections 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f11b4038e60 0x7f11b403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 --2- 172.21.15.86:0/1982876480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11c81073e0 0x7f11c819b3a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:51.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 >> 172.21.15.86:0/1982876480 conn(0x7f11c8102b50 msgr2=0x7f11c81036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:51.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 shutdown_connections 2024-09-05T06:31:51.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:50.612+0000 7f11cc8a3640 1 -- 172.21.15.86:0/1982876480 wait complete. 2024-09-05T06:31:51.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: [05/Sep/2024:06:31:49] ENGINE Bus STARTING 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: [05/Sep/2024:06:31:50] ENGINE Serving on https://172.21.15.86:7150 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: [05/Sep/2024:06:31:50] ENGINE Error in HTTPServer.serve 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: Traceback (most recent call last): 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._connections.run(self.expiration_interval) 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._run(expiration_interval) 2024-09-05T06:31:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:31:51.740 INFO:journalctl@ceph.mon.a.smithi086.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:31:51.740 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:31:51.740 INFO:journalctl@ceph.mon.a.smithi086.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:31:51.740 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: s = self.context.wrap_socket( 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: return self.sslsocket_class._create( 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: self.do_handshake() 2024-09-05T06:31:51.741 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._sslobj.do_handshake() 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: [05/Sep/2024:06:31:50] ENGINE Serving on http://172.21.15.86:8765 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: [05/Sep/2024:06:31:50] ENGINE Bus STARTED 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: mgrmap e8: x(active, since 2s) 2024-09-05T06:31:51.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:31:51.743 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:51.743 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:51 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:31:52.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3ede03640 1 Processor -- start 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3ede03640 1 -- start start 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3ede03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3ede03640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8107e70 con 0x7fc3e81074a0 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47618/0 (socket says 172.21.15.86:47618) 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3e77fe640 1 -- 172.21.15.86:0/2095838386 learned_addr learned my addr 172.21.15.86:0/2095838386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.581+0000 7fc3e77fe640 1 -- 172.21.15.86:0/2095838386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e8108600 con 0x7fc3e81074a0 2024-09-05T06:31:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.582+0000 7fc3e77fe640 1 --2- 172.21.15.86:0/2095838386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009820 tx=0x7fc3d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1777ad410640b183 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.582+0000 7fc3e67fc640 1 -- 172.21.15.86:0/2095838386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8037070 con 0x7fc3e81074a0 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.582+0000 7fc3e67fc640 1 -- 172.21.15.86:0/2095838386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc3d80328d0 con 0x7fc3e81074a0 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.582+0000 7fc3e67fc640 1 -- 172.21.15.86:0/2095838386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d80307d0 con 0x7fc3e81074a0 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 -- 172.21.15.86:0/2095838386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 msgr2=0x7fc3e81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 --2- 172.21.15.86:0/2095838386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009820 tx=0x7fc3d802a270 comp rx=0 tx=0).stop 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 -- 172.21.15.86:0/2095838386 shutdown_connections 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 --2- 172.21.15.86:0/2095838386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e81078a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 -- 172.21.15.86:0/2095838386 >> 172.21.15.86:0/2095838386 conn(0x7fc3e8102c50 msgr2=0x7fc3e8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 -- 172.21.15.86:0/2095838386 shutdown_connections 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 -- 172.21.15.86:0/2095838386 wait complete. 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.583+0000 7fc3ede03640 1 Processor -- start 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3ede03640 1 -- start start 2024-09-05T06:31:52.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3ede03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3ede03640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810b9b0 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47626/0 (socket says 172.21.15.86:47626) 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e77fe640 1 -- 172.21.15.86:0/3506864775 learned_addr learned my addr 172.21.15.86:0/3506864775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e77fe640 1 -- 172.21.15.86:0/3506864775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819db00 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e77fe640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009940 tx=0x7fc3d8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8042070 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc3d8030c80 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819dd90 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8037040 con 0x7fc3e81074a0 2024-09-05T06:31:52.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.584+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a0a80 con 0x7fc3e81074a0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.585+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc3d8045070 con 0x7fc3e81074a0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.585+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e810bb30 con 0x7fc3e81074a0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.585+0000 7fc3e4ff9640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 0x7fc3d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.586+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc3d806c060 con 0x7fc3e81074a0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.586+0000 7fc3e6ffd640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 0x7fc3d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.586+0000 7fc3e6ffd640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 0x7fc3d403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098c0 tx=0x7fc3d0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.590+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3d803c6b0 con 0x7fc3e81074a0 2024-09-05T06:31:52.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.769+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc3e8103e00 con 0x7fc3d4038f70 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.769+0000 7fc3e4ff9640 1 -- 172.21.15.86:0/3506864775 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc3e8103e00 con 0x7fc3d4038f70 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 msgr2=0x7fc3d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 0x7fc3d403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098c0 tx=0x7fc3d0006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 msgr2=0x7fc3e819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009940 tx=0x7fc3d8032e10 comp rx=0 tx=0).stop 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 shutdown_connections 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fc3d4038f70 0x7fc3d403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 --2- 172.21.15.86:0/3506864775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e81074a0 0x7fc3e819d5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 >> 172.21.15.86:0/3506864775 conn(0x7fc3e8102c50 msgr2=0x7fc3e81036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 shutdown_connections 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:51.772+0000 7fc3ede03640 1 -- 172.21.15.86:0/3506864775 wait complete. 2024-09-05T06:31:52.338 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-09-05T06:31:52.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:52 smithi086 ceph-mon[30943]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:31:53.631 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Generating public/private rsa key pair. 2024-09-05T06:31:53.631 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Your identification has been saved in /tmp/tmpi2bu_alc/key 2024-09-05T06:31:53.631 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Your public key has been saved in /tmp/tmpi2bu_alc/key.pub 2024-09-05T06:31:53.631 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: The key fingerprint is: 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: SHA256:Adoscl0KxF4P4Z//yEw+ZPIeQmkhe+EXsOMK2nBxK7g ceph-5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: The key's randomart image is: 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: +---[RSA 3072]----+ 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | oo o.o | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | .Bo+ o | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | ..=.Oo* . | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | +.+ B.B . | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | o + o S . | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | * o =.oo | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | E . . .=+ | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | *o+ | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: | .*.. | 2024-09-05T06:31:53.632 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:31:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: +----[SHA256]-----+ 2024-09-05T06:31:53.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.728+0000 7f95bc69f640 1 Processor -- start 2024-09-05T06:31:53.864 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.728+0000 7f95bc69f640 1 -- start start 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95bc69f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95bc69f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4106cb0 con 0x7f95b41062e0 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95ba43b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95ba43b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60262/0 (socket says 172.21.15.86:60262) 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95ba43b640 1 -- 172.21.15.86:0/1852197453 learned_addr learned my addr 172.21.15.86:0/1852197453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.729+0000 7f95ba43b640 1 -- 172.21.15.86:0/1852197453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41074e0 con 0x7f95b41062e0 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95ba43b640 1 --2- 172.21.15.86:0/1852197453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009820 tx=0x7f95a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e908954040dea5ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:53.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95b9439640 1 -- 172.21.15.86:0/1852197453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8037070 con 0x7f95b41062e0 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95b9439640 1 -- 172.21.15.86:0/1852197453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f95a80328d0 con 0x7f95b41062e0 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95b9439640 1 -- 172.21.15.86:0/1852197453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a80307d0 con 0x7f95b41062e0 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95bc69f640 1 -- 172.21.15.86:0/1852197453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 msgr2=0x7f95b41066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95bc69f640 1 --2- 172.21.15.86:0/1852197453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009820 tx=0x7f95a802a270 comp rx=0 tx=0).stop 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95bc69f640 1 -- 172.21.15.86:0/1852197453 shutdown_connections 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95bc69f640 1 --2- 172.21.15.86:0/1852197453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b41066e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.730+0000 7f95bc69f640 1 -- 172.21.15.86:0/1852197453 >> 172.21.15.86:0/1852197453 conn(0x7f95b4101a90 msgr2=0x7f95b4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 -- 172.21.15.86:0/1852197453 shutdown_connections 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 -- 172.21.15.86:0/1852197453 wait complete. 2024-09-05T06:31:53.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 Processor -- start 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 -- start start 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95bc69f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b410a890 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95ba43b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95ba43b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60274/0 (socket says 172.21.15.86:60274) 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.731+0000 7f95ba43b640 1 -- 172.21.15.86:0/3670592833 learned_addr learned my addr 172.21.15.86:0/3670592833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95ba43b640 1 -- 172.21.15.86:0/3670592833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b419bc20 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95ba43b640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009940 tx=0x7f95a8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8042070 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f95a8030c80 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8037040 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b419beb0 con 0x7f95b41062e0 2024-09-05T06:31:53.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.732+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b419eba0 con 0x7f95b41062e0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f95a8045070 con 0x7f95b41062e0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b410aa10 con 0x7f95b41062e0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95a77fe640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 0x7f95a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95a806be90 con 0x7f95b41062e0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95b9c3a640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 0x7f95a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.733+0000 7f95b9c3a640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 0x7f95a003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95b00098c0 tx=0x7f95b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.738+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95a8045360 con 0x7f95b41062e0 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:52.905+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f95b4062630 con 0x7f95a0038f70 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.348+0000 7f95a77fe640 1 -- 172.21.15.86:0/3670592833 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95b4062630 con 0x7f95a0038f70 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 msgr2=0x7f95a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 0x7f95a003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95b00098c0 tx=0x7f95b0006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:53.868 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 msgr2=0x7f95b419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009940 tx=0x7f95a8032e10 comp rx=0 tx=0).stop 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 shutdown_connections 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f95a0038f70 0x7f95a003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 --2- 172.21.15.86:0/3670592833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b41062e0 0x7f95b419b6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 >> 172.21.15.86:0/3670592833 conn(0x7f95b4101a90 msgr2=0x7f95b4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 shutdown_connections 2024-09-05T06:31:53.869 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:53.350+0000 7f95bc69f640 1 -- 172.21.15.86:0/3670592833 wait complete. 2024-09-05T06:31:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:54 smithi086 ceph-mon[30943]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:31:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:54 smithi086 ceph-mon[30943]: Generating ssh key... 2024-09-05T06:31:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:54 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:54 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:31:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5bdc8c640 1 Processor -- start 2024-09-05T06:31:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5bdc8c640 1 -- start start 2024-09-05T06:31:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5bdc8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5bdc8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b807c840 con 0x7fd5b807de00 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60288/0 (socket says 172.21.15.86:60288) 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5b77fe640 1 -- 172.21.15.86:0/2477798762 learned_addr learned my addr 172.21.15.86:0/2477798762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.233+0000 7fd5b77fe640 1 -- 172.21.15.86:0/2477798762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b807c9c0 con 0x7fd5b807de00 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.234+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/2477798762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009820 tx=0x7fd5a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa74b3c61dbe5dec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.234+0000 7fd5b67fc640 1 -- 172.21.15.86:0/2477798762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a8037070 con 0x7fd5b807de00 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.234+0000 7fd5b67fc640 1 -- 172.21.15.86:0/2477798762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd5a80328d0 con 0x7fd5b807de00 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.234+0000 7fd5b67fc640 1 -- 172.21.15.86:0/2477798762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a80307d0 con 0x7fd5b807de00 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/2477798762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 msgr2=0x7fd5b807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/2477798762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009820 tx=0x7fd5a802a270 comp rx=0 tx=0).stop 2024-09-05T06:31:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/2477798762 shutdown_connections 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/2477798762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b807c300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/2477798762 >> 172.21.15.86:0/2477798762 conn(0x7fd5b8102c50 msgr2=0x7fd5b8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/2477798762 shutdown_connections 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/2477798762 wait complete. 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 Processor -- start 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.235+0000 7fd5bdc8c640 1 -- start start 2024-09-05T06:31:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5bdc8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5bdc8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b810da10 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60304/0 (socket says 172.21.15.86:60304) 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b77fe640 1 -- 172.21.15.86:0/931134674 learned_addr learned my addr 172.21.15.86:0/931134674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b77fe640 1 -- 172.21.15.86:0/931134674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b819db20 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b77fe640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8032e10 tx=0x7fd5a8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a8042070 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd5a8030c80 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b819ddb0 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a8037040 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.236+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b81a0aa0 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.237+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd5a8045070 con 0x7fd5b807de00 2024-09-05T06:31:55.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.237+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b807c380 con 0x7fd5b807de00 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.237+0000 7fd5b4ff9640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 0x7fd5a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.237+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5a806bfe0 con 0x7fd5b807de00 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.237+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 0x7fd5a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.238+0000 7fd5b6ffd640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 0x7fd5a003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098c0 tx=0x7fd5ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.242+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5a8045360 con 0x7fd5b807de00 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.411+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd5b8103ec0 con 0x7fd5a0038f70 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.411+0000 7fd5b4ff9640 1 -- 172.21.15.86:0/931134674 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd5b8103ec0 con 0x7fd5a0038f70 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 msgr2=0x7fd5a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 0x7fd5a003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098c0 tx=0x7fd5ac006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 msgr2=0x7fd5b819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8032e10 tx=0x7fd5a8032e40 comp rx=0 tx=0).stop 2024-09-05T06:31:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 shutdown_connections 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fd5a0038f70 0x7fd5a003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 --2- 172.21.15.86:0/931134674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b807de00 0x7fd5b819d5e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.413+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 >> 172.21.15.86:0/931134674 conn(0x7fd5b8102c50 msgr2=0x7fd5b81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.414+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 shutdown_connections 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:54.414+0000 7fd5bdc8c640 1 -- 172.21.15.86:0/931134674 wait complete. 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-09-05T06:31:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-09-05T06:31:55.593 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:55 smithi086 ceph-mon[30943]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:31:56.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:56 smithi086 ceph-mon[30943]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:31:57.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:57 smithi086 ceph-mon[30943]: Deploying cephadm binary to smithi086 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.411+0000 7fa621a18640 1 Processor -- start 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.411+0000 7fa621a18640 1 -- start start 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa621a18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa621a18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0757a0 con 0x7fa61c105c40 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60310/0 (socket says 172.21.15.86:60310) 2024-09-05T06:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa61affd640 1 -- 172.21.15.86:0/1655138607 learned_addr learned my addr 172.21.15.86:0/1655138607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.412+0000 7fa61affd640 1 -- 172.21.15.86:0/1655138607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c108590 con 0x7fa61c105c40 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa61affd640 1 --2- 172.21.15.86:0/1655138607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009e80 tx=0x7fa60c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154fa5b9ead46d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa619ffb640 1 -- 172.21.15.86:0/1655138607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa60c036070 con 0x7fa61c105c40 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa619ffb640 1 -- 172.21.15.86:0/1655138607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa60c031890 con 0x7fa61c105c40 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa619ffb640 1 -- 172.21.15.86:0/1655138607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa60c007980 con 0x7fa61c105c40 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 -- 172.21.15.86:0/1655138607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 msgr2=0x7fa61c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 --2- 172.21.15.86:0/1655138607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009e80 tx=0x7fa60c02a7a0 comp rx=0 tx=0).stop 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 -- 172.21.15.86:0/1655138607 shutdown_connections 2024-09-05T06:31:59.296 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 --2- 172.21.15.86:0/1655138607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c108050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 -- 172.21.15.86:0/1655138607 >> 172.21.15.86:0/1655138607 conn(0x7fa61c101a90 msgr2=0x7fa61c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 -- 172.21.15.86:0/1655138607 shutdown_connections 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.413+0000 7fa621a18640 1 -- 172.21.15.86:0/1655138607 wait complete. 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa621a18640 1 Processor -- start 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa621a18640 1 -- start start 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa621a18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa621a18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c109630 con 0x7fa61c105c40 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa61affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60318/0 (socket says 172.21.15.86:60318) 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.414+0000 7fa61affd640 1 -- 172.21.15.86:0/1426075512 learned_addr learned my addr 172.21.15.86:0/1426075512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:31:59.297 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa61affd640 1 -- 172.21.15.86:0/1426075512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a21e0 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa61affd640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009680 tx=0x7fa60c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa60c041020 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa60c03c900 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a2470 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa60c036040 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.415+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a5160 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.416+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa60c044020 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.416+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c106040 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.416+0000 7fa620a16640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 0x7fa60803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.416+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa60c06d480 con 0x7fa61c105c40 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.416+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 0x7fa60803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.417+0000 7fa61a7fc640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 0x7fa60803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098c0 tx=0x7fa604006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:31:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.421+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa60c04b020 con 0x7fa61c105c40 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:55.592+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) -- 0x7fa61c10a610 con 0x7fa608038f70 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.739+0000 7fa620a16640 1 -- 172.21.15.86:0/1426075512 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa61c10a610 con 0x7fa608038f70 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 msgr2=0x7fa60803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 0x7fa60803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098c0 tx=0x7fa604006db0 comp rx=0 tx=0).stop 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 msgr2=0x7fa61c1a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009680 tx=0x7fa60c031c70 comp rx=0 tx=0).stop 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 shutdown_connections 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fa608038f70 0x7fa60803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 --2- 172.21.15.86:0/1426075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61c105c40 0x7fa61c1a1ca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 >> 172.21.15.86:0/1426075512 conn(0x7fa61c101a90 msgr2=0x7fa61c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 shutdown_connections 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:58.742+0000 7fa621a18640 1 -- 172.21.15.86:0/1426075512 wait complete. 2024-09-05T06:31:59.299 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-09-05T06:31:59.959 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:59 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:31:59.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:59 smithi086 ceph-mon[30943]: Added host smithi086 2024-09-05T06:31:59.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:31:59 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.772+0000 7fb1b9330640 1 Processor -- start 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.772+0000 7fb1b9330640 1 -- start start 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.772+0000 7fb1b9330640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.773+0000 7fb1b9330640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4075760 con 0x7fb1b4107e90 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.773+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.773+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60332/0 (socket says 172.21.15.86:60332) 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.773+0000 7fb1b2ffd640 1 -- 172.21.15.86:0/4290495252 learned_addr learned my addr 172.21.15.86:0/4290495252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.773+0000 7fb1b2ffd640 1 -- 172.21.15.86:0/4290495252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b410a7c0 con 0x7fb1b4107e90 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b2ffd640 1 --2- 172.21.15.86:0/4290495252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009e80 tx=0x7fb1a402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d92879c67d8f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:00.555 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b1ffb640 1 -- 172.21.15.86:0/4290495252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a4036070 con 0x7fb1b4107e90 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b1ffb640 1 -- 172.21.15.86:0/4290495252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb1a4031890 con 0x7fb1b4107e90 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b1ffb640 1 -- 172.21.15.86:0/4290495252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a4007980 con 0x7fb1b4107e90 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 -- 172.21.15.86:0/4290495252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 msgr2=0x7fb1b410a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4290495252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009e80 tx=0x7fb1a402a7a0 comp rx=0 tx=0).stop 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 -- 172.21.15.86:0/4290495252 shutdown_connections 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4290495252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b410a280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 -- 172.21.15.86:0/4290495252 >> 172.21.15.86:0/4290495252 conn(0x7fb1b4101a90 msgr2=0x7fb1b4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 -- 172.21.15.86:0/4290495252 shutdown_connections 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.774+0000 7fb1b9330640 1 -- 172.21.15.86:0/4290495252 wait complete. 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b9330640 1 Processor -- start 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b9330640 1 -- start start 2024-09-05T06:32:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b9330640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b9330640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b410b7a0 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60342/0 (socket says 172.21.15.86:60342) 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.775+0000 7fb1b2ffd640 1 -- 172.21.15.86:0/4143871307 learned_addr learned my addr 172.21.15.86:0/4143871307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1b2ffd640 1 -- 172.21.15.86:0/4143871307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b4107c00 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1b2ffd640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009680 tx=0x7fb1a4031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a4041020 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb1a403c900 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b4105e40 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b4106380 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.776+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1a4036040 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.777+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb1a4044020 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.777+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1940052c0 con 0x7fb1b4107e90 2024-09-05T06:32:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.778+0000 7fb1abfff640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 0x7fb1a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.778+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb1a406d490 con 0x7fb1b4107e90 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.778+0000 7fb1b27fc640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 0x7fb1a003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.778+0000 7fb1b27fc640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 0x7fb1a003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098c0 tx=0x7fb19c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.782+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1a404b020 con 0x7fb1b4107e90 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.959+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb194000bf0 con 0x7fb1a0038fc0 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.964+0000 7fb1abfff640 1 -- 172.21.15.86:0/4143871307 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb194000bf0 con 0x7fb1a0038fc0 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 msgr2=0x7fb1a003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 0x7fb1a003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098c0 tx=0x7fb19c006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 msgr2=0x7fb1b41076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009680 tx=0x7fb1a4031c70 comp rx=0 tx=0).stop 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 shutdown_connections 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fb1a0038fc0 0x7fb1a003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:00.558 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.966+0000 7fb1b9330640 1 --2- 172.21.15.86:0/4143871307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b4107e90 0x7fb1b41076c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:00.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.967+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 >> 172.21.15.86:0/4143871307 conn(0x7fb1b4101a90 msgr2=0x7fb1b41027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:00.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.967+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 shutdown_connections 2024-09-05T06:32:00.559 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:31:59.967+0000 7fb1b9330640 1 -- 172.21.15.86:0/4143871307 wait complete. 2024-09-05T06:32:00.559 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-09-05T06:32:01.125 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:00 smithi086 ceph-mon[30943]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:01.125 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:00 smithi086 ceph-mon[30943]: Saving service mon spec with placement count:5 2024-09-05T06:32:01.125 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:00 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f624101c640 1 Processor -- start 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f624101c640 1 -- start start 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f624101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f624101c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f623c107e70 con 0x7f623c1074a0 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:01.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60356/0 (socket says 172.21.15.86:60356) 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f623ad9d640 1 -- 172.21.15.86:0/1946800896 learned_addr learned my addr 172.21.15.86:0/1946800896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.971+0000 7f623ad9d640 1 -- 172.21.15.86:0/1946800896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c108600 con 0x7f623c1074a0 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.972+0000 7f623ad9d640 1 --2- 172.21.15.86:0/1946800896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6224009820 tx=0x7f622402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec381c5dfec0119c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.972+0000 7f6239d9b640 1 -- 172.21.15.86:0/1946800896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6224037070 con 0x7f623c1074a0 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.972+0000 7f6239d9b640 1 -- 172.21.15.86:0/1946800896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f62240328d0 con 0x7f623c1074a0 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.972+0000 7f6239d9b640 1 -- 172.21.15.86:0/1946800896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62240307d0 con 0x7f623c1074a0 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 -- 172.21.15.86:0/1946800896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 msgr2=0x7f623c1078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 --2- 172.21.15.86:0/1946800896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6224009820 tx=0x7f622402a270 comp rx=0 tx=0).stop 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 -- 172.21.15.86:0/1946800896 shutdown_connections 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 --2- 172.21.15.86:0/1946800896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c1078a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 -- 172.21.15.86:0/1946800896 >> 172.21.15.86:0/1946800896 conn(0x7f623c102c50 msgr2=0x7f623c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 -- 172.21.15.86:0/1946800896 shutdown_connections 2024-09-05T06:32:01.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 -- 172.21.15.86:0/1946800896 wait complete. 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.973+0000 7f624101c640 1 Processor -- start 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f624101c640 1 -- start start 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f624101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f624101c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10b9b0 con 0x7f623c1074a0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60358/0 (socket says 172.21.15.86:60358) 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f623ad9d640 1 -- 172.21.15.86:0/1633051223 learned_addr learned my addr 172.21.15.86:0/1633051223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f623ad9d640 1 -- 172.21.15.86:0/1633051223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c19b8b0 con 0x7f623c1074a0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f623ad9d640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f62240097f0 tx=0x7f6224030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6224037040 con 0x7f623c1074a0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6224032bb0 con 0x7f623c1074a0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c19bb40 con 0x7f623c1074a0 2024-09-05T06:32:01.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622403a390 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.974+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c19e830 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.976+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c10bb30 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.976+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f622403b3f0 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.976+0000 7f6233fff640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 0x7f622803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.976+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f622406c290 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.976+0000 7f623a59c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 0x7f622803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.977+0000 7f623a59c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 0x7f622803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098c0 tx=0x7f622c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:00.981+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6224040210 con 0x7f623c1074a0 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.163+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f623c103e90 con 0x7f6228038f70 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.171+0000 7f6233fff640 1 -- 172.21.15.86:0/1633051223 <== mgr.14118 v2:172.21.15.86:6800/1793567579 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f623c103e90 con 0x7f6228038f70 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 msgr2=0x7f622803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 0x7f622803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098c0 tx=0x7f622c006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:01.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 msgr2=0x7f623c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f62240097f0 tx=0x7f6224030d00 comp rx=0 tx=0).stop 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 shutdown_connections 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f6228038f70 0x7f622803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 --2- 172.21.15.86:0/1633051223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c1074a0 0x7f623c19b370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 >> 172.21.15.86:0/1633051223 conn(0x7f623c102c50 msgr2=0x7f623c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 shutdown_connections 2024-09-05T06:32:01.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:01.173+0000 7f624101c640 1 -- 172.21.15.86:0/1633051223 wait complete. 2024-09-05T06:32:02.426 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:02 smithi086 ceph-mon[30943]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:02.426 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:02 smithi086 ceph-mon[30943]: Saving service mgr spec with placement count:2 2024-09-05T06:32:02.426 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:02 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:32:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:03 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/506985345' entity='client.admin' 2024-09-05T06:32:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:03 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:32:03.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d8b03b640 1 Processor -- start 2024-09-05T06:32:03.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d8b03b640 1 -- start start 2024-09-05T06:32:03.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d8b03b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d8b03b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8407be50 con 0x7f2d84107f00 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d88dd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d88dd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60362/0 (socket says 172.21.15.86:60362) 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.281+0000 7f2d88dd7640 1 -- 172.21.15.86:0/741100935 learned_addr learned my addr 172.21.15.86:0/741100935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.282+0000 7f2d88dd7640 1 -- 172.21.15.86:0/741100935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8410a830 con 0x7f2d84107f00 2024-09-05T06:32:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.282+0000 7f2d88dd7640 1 --2- 172.21.15.86:0/741100935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009820 tx=0x7f2d7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb0271359229f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d837fe640 1 -- 172.21.15.86:0/741100935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74037070 con 0x7f2d84107f00 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d837fe640 1 -- 172.21.15.86:0/741100935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2d740328d0 con 0x7f2d84107f00 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d837fe640 1 -- 172.21.15.86:0/741100935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d740307d0 con 0x7f2d84107f00 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 -- 172.21.15.86:0/741100935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 msgr2=0x7f2d8410a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/741100935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009820 tx=0x7f2d7402a270 comp rx=0 tx=0).stop 2024-09-05T06:32:03.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 -- 172.21.15.86:0/741100935 shutdown_connections 2024-09-05T06:32:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/741100935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8410a2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 -- 172.21.15.86:0/741100935 >> 172.21.15.86:0/741100935 conn(0x7f2d84101ba0 msgr2=0x7f2d84103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 -- 172.21.15.86:0/741100935 shutdown_connections 2024-09-05T06:32:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.283+0000 7f2d8b03b640 1 -- 172.21.15.86:0/741100935 wait complete. 2024-09-05T06:32:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d8b03b640 1 Processor -- start 2024-09-05T06:32:03.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d8b03b640 1 -- start start 2024-09-05T06:32:03.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d8b03b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:03.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d8b03b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8410b7b0 con 0x7f2d84107f00 2024-09-05T06:32:03.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d88dd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:03.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d88dd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60370/0 (socket says 172.21.15.86:60370) 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.284+0000 7f2d88dd7640 1 -- 172.21.15.86:0/506985345 learned_addr learned my addr 172.21.15.86:0/506985345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d88dd7640 1 -- 172.21.15.86:0/506985345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8419b8c0 con 0x7f2d84107f00 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d88dd7640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d74032e10 tx=0x7f2d74032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74042070 con 0x7f2d84107f00 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2d74030c80 con 0x7f2d84107f00 2024-09-05T06:32:03.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8419bb50 con 0x7f2d84107f00 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74037040 con 0x7f2d84107f00 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.285+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8419e840 con 0x7f2d84107f00 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.286+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2d74045070 con 0x7f2d84107f00 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.286+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84108360 con 0x7f2d84107f00 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.286+0000 7f2d81ffb640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 0x7f2d6c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:03.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.286+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d7406c010 con 0x7f2d84107f00 2024-09-05T06:32:03.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.286+0000 7f2d83fff640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 0x7f2d6c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:03.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.287+0000 7f2d83fff640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 0x7f2d6c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098c0 tx=0x7f2d78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:03.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.291+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d7403b330 con 0x7f2d84107f00 2024-09-05T06:32:03.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.460+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2d8419eec0 con 0x7f2d84107f00 2024-09-05T06:32:03.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.465+0000 7f2d81ffb640 1 -- 172.21.15.86:0/506985345 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f2d74046da0 con 0x7f2d84107f00 2024-09-05T06:32:03.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 msgr2=0x7f2d6c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:03.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 0x7f2d6c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098c0 tx=0x7f2d78006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:03.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 msgr2=0x7f2d8419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:03.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d74032e10 tx=0x7f2d74032e40 comp rx=0 tx=0).stop 2024-09-05T06:32:03.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 shutdown_connections 2024-09-05T06:32:03.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2d6c038fc0 0x7f2d6c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:03.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.467+0000 7f2d8b03b640 1 --2- 172.21.15.86:0/506985345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d84107f00 0x7f2d8419b380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:03.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.468+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 >> 172.21.15.86:0/506985345 conn(0x7f2d84101ba0 msgr2=0x7f2d841026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:03.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.468+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 shutdown_connections 2024-09-05T06:32:03.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:02.468+0000 7f2d8b03b640 1 -- 172.21.15.86:0/506985345 wait complete. 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.125+0000 7fca40297640 1 Processor -- start 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.125+0000 7fca40297640 1 -- start start 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.125+0000 7fca40297640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.126+0000 7fca40297640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca3807be90 con 0x7fca38107e80 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.126+0000 7fca3e033640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:04.891 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.126+0000 7fca3e033640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43734/0 (socket says 172.21.15.86:43734) 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.126+0000 7fca3e033640 1 -- 172.21.15.86:0/2037804255 learned_addr learned my addr 172.21.15.86:0/2037804255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.126+0000 7fca3e033640 1 -- 172.21.15.86:0/2037804255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3810a7b0 con 0x7fca38107e80 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.127+0000 7fca3e033640 1 --2- 172.21.15.86:0/2037804255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca34009820 tx=0x7fca3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a298e24d3f807f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.127+0000 7fca3d031640 1 -- 172.21.15.86:0/2037804255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca34037070 con 0x7fca38107e80 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.127+0000 7fca3d031640 1 -- 172.21.15.86:0/2037804255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca340328f0 con 0x7fca38107e80 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.127+0000 7fca3d031640 1 -- 172.21.15.86:0/2037804255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca34030740 con 0x7fca38107e80 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- 172.21.15.86:0/2037804255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 msgr2=0x7fca3810a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 --2- 172.21.15.86:0/2037804255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca34009820 tx=0x7fca3402a270 comp rx=0 tx=0).stop 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- 172.21.15.86:0/2037804255 shutdown_connections 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 --2- 172.21.15.86:0/2037804255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3810a270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- 172.21.15.86:0/2037804255 >> 172.21.15.86:0/2037804255 conn(0x7fca38101a60 msgr2=0x7fca38103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- 172.21.15.86:0/2037804255 shutdown_connections 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- 172.21.15.86:0/2037804255 wait complete. 2024-09-05T06:32:04.892 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 Processor -- start 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.128+0000 7fca40297640 1 -- start start 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca40297640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca40297640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca3810b7d0 con 0x7fca38107e80 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca3e033640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca3e033640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43748/0 (socket says 172.21.15.86:43748) 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca3e033640 1 -- 172.21.15.86:0/2439654646 learned_addr learned my addr 172.21.15.86:0/2439654646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca3e033640 1 -- 172.21.15.86:0/2439654646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3819db00 con 0x7fca38107e80 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca3e033640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca340097f0 tx=0x7fca34030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca34042070 con 0x7fca38107e80 2024-09-05T06:32:04.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca34030d80 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3819dd90 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.129+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca34037040 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.130+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca381a0a80 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.130+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fca34045070 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.131+0000 7fca2f7fe640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 0x7fca2403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.131+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca3406c000 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.131+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca381082e0 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.131+0000 7fca3d832640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 0x7fca2403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.131+0000 7fca3d832640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 0x7fca2403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca280098c0 tx=0x7fca28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.137+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca34045360 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.323+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fca381a1020 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.325+0000 7fca2f7fe640 1 -- 172.21.15.86:0/2439654646 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fca3402ed90 con 0x7fca38107e80 2024-09-05T06:32:04.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 msgr2=0x7fca2403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 0x7fca2403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca280098c0 tx=0x7fca28006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 msgr2=0x7fca3819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca340097f0 tx=0x7fca34030c70 comp rx=0 tx=0).stop 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 shutdown_connections 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7fca24038fc0 0x7fca2403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 --2- 172.21.15.86:0/2439654646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca38107e80 0x7fca3819d5c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 >> 172.21.15.86:0/2439654646 conn(0x7fca38101a60 msgr2=0x7fca381025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 shutdown_connections 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:04.328+0000 7fca40297640 1 -- 172.21.15.86:0/2439654646 wait complete. 2024-09-05T06:32:04.895 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-09-05T06:32:05.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:05 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:32:05.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:05 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2439654646' entity='client.admin' 2024-09-05T06:32:06.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1086437664' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-05T06:32:06.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-mon[30943]: from='mgr.14118 172.21.15.86:0/3594471994' entity='mgr.x' 2024-09-05T06:32:06.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-05T06:32:06.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-05T06:32:06.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.341+0000 7fb0bcdf8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:32:06.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.397+0000 7fb0bcdf8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:32:06.907 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.549+0000 7fb0bcdf8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:32:06.907 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.615+0000 7fb0bcdf8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:32:06.908 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.765+0000 7fb0bcdf8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:32:06.908 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:06.907+0000 7fb0bcdf8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:32:06.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f843b858640 1 Processor -- start 2024-09-05T06:32:06.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f843b858640 1 -- start start 2024-09-05T06:32:06.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f843b858640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:06.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f843b858640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8434107230 con 0x7f8434106860 2024-09-05T06:32:06.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f84395f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:06.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f84395f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43758/0 (socket says 172.21.15.86:43758) 2024-09-05T06:32:06.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f84395f4640 1 -- 172.21.15.86:0/1215822401 learned_addr learned my addr 172.21.15.86:0/1215822401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.323+0000 7f84395f4640 1 -- 172.21.15.86:0/1215822401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434107580 con 0x7f8434106860 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.324+0000 7f84395f4640 1 --2- 172.21.15.86:0/1215822401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8428009820 tx=0x7f842802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7ab0efe2056b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.324+0000 7f8427fff640 1 -- 172.21.15.86:0/1215822401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428037070 con 0x7f8434106860 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.324+0000 7f8427fff640 1 -- 172.21.15.86:0/1215822401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84280328f0 con 0x7f8434106860 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.324+0000 7f8427fff640 1 -- 172.21.15.86:0/1215822401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428030740 con 0x7f8434106860 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- 172.21.15.86:0/1215822401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 msgr2=0x7f8434106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:06.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 --2- 172.21.15.86:0/1215822401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8428009820 tx=0x7f842802a270 comp rx=0 tx=0).stop 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- 172.21.15.86:0/1215822401 shutdown_connections 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 --2- 172.21.15.86:0/1215822401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f8434106c60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- 172.21.15.86:0/1215822401 >> 172.21.15.86:0/1215822401 conn(0x7f8434101330 msgr2=0x7f8434103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- 172.21.15.86:0/1215822401 shutdown_connections 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- 172.21.15.86:0/1215822401 wait complete. 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 Processor -- start 2024-09-05T06:32:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.325+0000 7f843b858640 1 -- start start 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f843b858640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f843b858640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f843410a930 con 0x7f8434106860 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84395f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84395f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43770/0 (socket says 172.21.15.86:43770) 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84395f4640 1 -- 172.21.15.86:0/1086437664 learned_addr learned my addr 172.21.15.86:0/1086437664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84395f4640 1 -- 172.21.15.86:0/1086437664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843419b970 con 0x7f8434106860 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84395f4640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8428009940 tx=0x7f8428030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:06.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428042070 con 0x7f8434106860 2024-09-05T06:32:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8428030e30 con 0x7f8434106860 2024-09-05T06:32:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843419bc00 con 0x7f8434106860 2024-09-05T06:32:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428037040 con 0x7f8434106860 2024-09-05T06:32:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.326+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843419e8f0 con 0x7f8434106860 2024-09-05T06:32:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.327+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8428045070 con 0x7f8434106860 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.327+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8434106ce0 con 0x7f8434106860 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.327+0000 7f84267fc640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 0x7f842003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.327+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f842806c000 con 0x7f8434106860 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.328+0000 7f8438df3640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 0x7f842003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.328+0000 7f8438df3640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 0x7f842003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84300098c0 tx=0x7f8430006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:06.949 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.332+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f842804b170 con 0x7f8434106860 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:05.590+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f843402d9a0 con 0x7f8434106860 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.197+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f842802fd90 con 0x7f8434106860 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.198+0000 7f84267fc640 1 -- 172.21.15.86:0/1086437664 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f842803d950 con 0x7f8434106860 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 msgr2=0x7f842003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 0x7f842003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84300098c0 tx=0x7f8430006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 msgr2=0x7f843419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:06.950 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8428009940 tx=0x7f8428030cc0 comp rx=0 tx=0).stop 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 shutdown_connections 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f8420038f70 0x7f842003b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 --2- 172.21.15.86:0/1086437664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8434106860 0x7f843419b430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 >> 172.21.15.86:0/1086437664 conn(0x7f8434101330 msgr2=0x7f8434102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 shutdown_connections 2024-09-05T06:32:06.951 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:06.200+0000 7f843b858640 1 -- 172.21.15.86:0/1086437664 wait complete. 2024-09-05T06:32:07.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:07 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1086437664' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-05T06:32:07.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:07 smithi086 ceph-mon[30943]: mgrmap e9: x(active, since 17s) 2024-09-05T06:32:07.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:07.230+0000 7fb0bcdf8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:32:07.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:07.291+0000 7fb0bcdf8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:32:07.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:07.486+0000 7fb0bcdf8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f22313e0640 1 Processor -- start 2024-09-05T06:32:08.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f22313e0640 1 -- start start 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f22313e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f22313e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f222c106ce0 con 0x7f222c106310 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f222affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f222affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43790/0 (socket says 172.21.15.86:43790) 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.367+0000 7f222affd640 1 -- 172.21.15.86:0/3400596017 learned_addr learned my addr 172.21.15.86:0/3400596017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.368+0000 7f222affd640 1 -- 172.21.15.86:0/3400596017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c107510 con 0x7f222c106310 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.368+0000 7f222affd640 1 --2- 172.21.15.86:0/3400596017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cda1f67f521b87f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.368+0000 7f222a7fc640 1 -- 172.21.15.86:0/3400596017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c038070 con 0x7f222c106310 2024-09-05T06:32:08.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.368+0000 7f222a7fc640 1 -- 172.21.15.86:0/3400596017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221c031af0 con 0x7f222c106310 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.368+0000 7f222a7fc640 1 -- 172.21.15.86:0/3400596017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c033b90 con 0x7f222c106310 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 -- 172.21.15.86:0/3400596017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 msgr2=0x7f222c106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 --2- 172.21.15.86:0/3400596017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02c480 comp rx=0 tx=0).stop 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 -- 172.21.15.86:0/3400596017 shutdown_connections 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 --2- 172.21.15.86:0/3400596017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c106710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 -- 172.21.15.86:0/3400596017 >> 172.21.15.86:0/3400596017 conn(0x7f222c101ba0 msgr2=0x7f222c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 -- 172.21.15.86:0/3400596017 shutdown_connections 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.369+0000 7f22313e0640 1 -- 172.21.15.86:0/3400596017 wait complete. 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f22313e0640 1 Processor -- start 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f22313e0640 1 -- start start 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f22313e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f22313e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f222c07b170 con 0x7f222c106310 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f222affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:08.210 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f222affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43802/0 (socket says 172.21.15.86:43802) 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f222affd640 1 -- 172.21.15.86:0/3225428486 learned_addr learned my addr 172.21.15.86:0/3225428486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.370+0000 7f222affd640 1 -- 172.21.15.86:0/3225428486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c199690 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f222affd640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f221c033720 tx=0x7f221c031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c038050 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221c042070 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c199920 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c03d9c0 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.371+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c19c610 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f221c03dbb0 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c106710 con 0x7f222c106310 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f2228ff9640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 0x7f221803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f2223fff640 1 -- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 msgr2=0x7f221803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:08.211 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f221c06eb20 con 0x7f222c106310 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.372+0000 7f2223fff640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 0x7f221803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.377+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f221c03b300 con 0x7f222c106310 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.573+0000 7f2223fff640 1 -- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 msgr2=0x7f221803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.573+0000 7f2223fff640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 0x7f221803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.597+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f222c19c7e0 con 0x7f222c106310 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.598+0000 7f2228ff9640 1 -- 172.21.15.86:0/3225428486 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f222c19c7e0 con 0x7f222c106310 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 msgr2=0x7f221803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 0x7f221803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 msgr2=0x7f222c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f221c033720 tx=0x7f221c031880 comp rx=0 tx=0).stop 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 shutdown_connections 2024-09-05T06:32:08.212 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f2218039020 0x7f221803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 --2- 172.21.15.86:0/3225428486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222c106310 0x7f222c199150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 >> 172.21.15.86:0/3225428486 conn(0x7f222c101ba0 msgr2=0x7f222c18f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 shutdown_connections 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:07.600+0000 7f22313e0640 1 -- 172.21.15.86:0/3225428486 wait complete. 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-09-05T06:32:08.213 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-09-05T06:32:08.325 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:08 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3225428486' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T06:32:08.607 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:08.399+0000 7fb0bcdf8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:32:08.607 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:08.469+0000 7fb0bcdf8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:32:09.435 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.181+0000 7fb0bcdf8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:32:09.435 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.295+0000 7fb0bcdf8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:32:09.710 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.435+0000 7fb0bcdf8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:32:09.710 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.495+0000 7fb0bcdf8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:32:09.711 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.642+0000 7fb0bcdf8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:32:09.711 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.710+0000 7fb0bcdf8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:32:09.986 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.770+0000 7fb0bcdf8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:32:09.987 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.925+0000 7fb0bcdf8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:32:10.254 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:09.989+0000 7fb0bcdf8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:32:10.254 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:10.055+0000 7fb0bcdf8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:32:10.254 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:10.137+0000 7fb0bcdf8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:32:10.254 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:10.195+0000 7fb0bcdf8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:32:10.254 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:10.254+0000 7fb0bcdf8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:32:11.486 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:11.077+0000 7fb0bcdf8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: Active manager daemon x restarted 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: Activating manager daemon x 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: osdmap e3: 0 total, 0 up, 0 in 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: mgrmap e10: x(active, starting, since 0.00349407s) 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:32:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: Manager daemon x is now available 2024-09-05T06:32:11.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:11.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:32:11.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:32:13.202 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.606+0000 7f58e8975640 1 Processor -- start 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e8975640 1 -- start start 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e8975640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e8975640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4075740 con 0x7f58e4107eb0 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43804/0 (socket says 172.21.15.86:43804) 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e259c640 1 -- 172.21.15.86:0/253587522 learned_addr learned my addr 172.21.15.86:0/253587522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.607+0000 7f58e259c640 1 -- 172.21.15.86:0/253587522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e410a7e0 con 0x7f58e4107eb0 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.608+0000 7f58e259c640 1 --2- 172.21.15.86:0/253587522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009820 tx=0x7f58d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7598f453433f1b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.608+0000 7f58e159a640 1 -- 172.21.15.86:0/253587522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d4037070 con 0x7f58e4107eb0 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.608+0000 7f58e159a640 1 -- 172.21.15.86:0/253587522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d40328f0 con 0x7f58e4107eb0 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.608+0000 7f58e159a640 1 -- 172.21.15.86:0/253587522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d4030740 con 0x7f58e4107eb0 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 -- 172.21.15.86:0/253587522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 msgr2=0x7f58e410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 --2- 172.21.15.86:0/253587522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009820 tx=0x7f58d402a270 comp rx=0 tx=0).stop 2024-09-05T06:32:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 -- 172.21.15.86:0/253587522 shutdown_connections 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 --2- 172.21.15.86:0/253587522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e410a2a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 -- 172.21.15.86:0/253587522 >> 172.21.15.86:0/253587522 conn(0x7f58e4101a90 msgr2=0x7f58e4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 -- 172.21.15.86:0/253587522 shutdown_connections 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.609+0000 7f58e8975640 1 -- 172.21.15.86:0/253587522 wait complete. 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e8975640 1 Processor -- start 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e8975640 1 -- start start 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e8975640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e8975640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410b800 con 0x7f58e4107eb0 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43812/0 (socket says 172.21.15.86:43812) 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e259c640 1 -- 172.21.15.86:0/3146068224 learned_addr learned my addr 172.21.15.86:0/3146068224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.610+0000 7f58e259c640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e4107c70 con 0x7f58e4107eb0 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58e259c640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097f0 tx=0x7f58d4030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d4042070 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e4105e60 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41063a0 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d4030d80 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.611+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d4037040 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f58d4045070 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58d37fe640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58cc03bbf0 con 0x7f58cc039020 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58d406c170 con 0x7f58e4107eb0 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58e1d9b640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 msgr2=0x7f58cc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.613+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.814+0000 7f58e1d9b640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 msgr2=0x7f58cc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:08.814+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:09.215+0000 7f58e1d9b640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 msgr2=0x7f58cc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:09.215+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:10.016+0000 7f58e1d9b640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 msgr2=0x7f58cc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1793567579 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:10.016+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:11.080+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f58d406b090 con 0x7f58e4107eb0 2024-09-05T06:32:13.207 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:11.080+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 msgr2=0x7f58cc03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:11.080+0000 7f58d37fe640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.081+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f58d403c660 con 0x7f58e4107eb0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.082+0000 7f58d37fe640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 0x7f58cc03eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.082+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58cc03bbf0 con 0x7f58cc03cae0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.082+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 0x7f58cc03eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.083+0000 7f58e1d9b640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 0x7f58cc03eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58d8003980 tx=0x7f58d80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.083+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f58cc03bbf0 con 0x7f58cc03cae0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f58c40025a0 con 0x7f58cc03cae0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58d37fe640 1 -- 172.21.15.86:0/3146068224 <== mgr.14150 v2:172.21.15.86:6800/3708924769 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f58c40025a0 con 0x7f58cc03cae0 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 msgr2=0x7f58cc03eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 0x7f58cc03eed0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58d8003980 tx=0x7f58d80092b0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 msgr2=0x7f58e4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:13.208 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097f0 tx=0x7f58d4030c70 comp rx=0 tx=0).stop 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 shutdown_connections 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58cc03cae0 0x7f58cc03eed0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:6800/1793567579,v1:172.21.15.86:6801/1793567579] conn(0x7f58cc039020 0x7f58cc03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 --2- 172.21.15.86:0/3146068224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4107eb0 0x7f58e4107730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 >> 172.21.15.86:0/3146068224 conn(0x7f58e4101a90 msgr2=0x7f58e41027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 shutdown_connections 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:12.096+0000 7f58e8975640 1 -- 172.21.15.86:0/3146068224 wait complete. 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-09-05T06:32:13.209 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-09-05T06:32:13.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-mon[30943]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T06:32:13.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-mon[30943]: mgrmap e11: x(active, since 1.00568s) 2024-09-05T06:32:13.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-mon[30943]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T06:32:13.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:14.237 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:13.913+0000 7fb0280cd640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:06:32:13] ENGINE Error in HTTPServer.serve 2024-09-05T06:32:14.237 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:32:14.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-05T06:32:14.239 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:32:14.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-05T06:32:14.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:32:14.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-05T06:32:14.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:32:14.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:32:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: mgrmap e12: x(active, since 2s) 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: [05/Sep/2024:06:32:13] ENGINE Bus STARTING 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:14.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: [05/Sep/2024:06:32:13] ENGINE Serving on https://172.21.15.86:7150 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: [05/Sep/2024:06:32:13] ENGINE Error in HTTPServer.serve 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: Traceback (most recent call last): 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._connections.run(self.expiration_interval) 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._run(expiration_interval) 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:32:14.663 INFO:journalctl@ceph.mon.a.smithi086.stdout: s = self.context.wrap_socket( 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: return self.sslsocket_class._create( 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: self.do_handshake() 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._sslobj.do_handshake() 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: [05/Sep/2024:06:32:14] ENGINE Serving on http://172.21.15.86:8765 2024-09-05T06:32:14.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:14 smithi086 ceph-mon[30943]: [05/Sep/2024:06:32:14] ENGINE Bus STARTED 2024-09-05T06:32:14.936 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-05T06:32:14.936 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.643+0000 7f4892fc0640 1 Processor -- start 2024-09-05T06:32:14.936 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.643+0000 7f4892fc0640 1 -- start start 2024-09-05T06:32:14.936 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.643+0000 7f4892fc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.643+0000 7f4892fc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f488c106cb0 con 0x7f488c1062e0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.644+0000 7f4890d5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.644+0000 7f4890d5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44172/0 (socket says 172.21.15.86:44172) 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.644+0000 7f4890d5c640 1 -- 172.21.15.86:0/522067806 learned_addr learned my addr 172.21.15.86:0/522067806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.644+0000 7f4890d5c640 1 -- 172.21.15.86:0/522067806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c1074e0 con 0x7f488c1062e0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.645+0000 7f4890d5c640 1 --2- 172.21.15.86:0/522067806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f487c009820 tx=0x7f487c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde62f3a2eb86cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.645+0000 7f488b7fe640 1 -- 172.21.15.86:0/522067806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c037070 con 0x7f488c1062e0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.645+0000 7f488b7fe640 1 -- 172.21.15.86:0/522067806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f487c0328f0 con 0x7f488c1062e0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.645+0000 7f488b7fe640 1 -- 172.21.15.86:0/522067806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c030740 con 0x7f488c1062e0 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 -- 172.21.15.86:0/522067806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 msgr2=0x7f488c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 --2- 172.21.15.86:0/522067806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f487c009820 tx=0x7f487c02a270 comp rx=0 tx=0).stop 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 -- 172.21.15.86:0/522067806 shutdown_connections 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 --2- 172.21.15.86:0/522067806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c1066e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 -- 172.21.15.86:0/522067806 >> 172.21.15.86:0/522067806 conn(0x7f488c101a90 msgr2=0x7f488c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 -- 172.21.15.86:0/522067806 shutdown_connections 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.646+0000 7f4892fc0640 1 -- 172.21.15.86:0/522067806 wait complete. 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4892fc0640 1 Processor -- start 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4892fc0640 1 -- start start 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4892fc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4892fc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10a890 con 0x7f488c1062e0 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4890d5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4890d5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44186/0 (socket says 172.21.15.86:44186) 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4890d5c640 1 -- 172.21.15.86:0/4208716208 learned_addr learned my addr 172.21.15.86:0/4208716208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4890d5c640 1 -- 172.21.15.86:0/4208716208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c191a50 con 0x7f488c1062e0 2024-09-05T06:32:14.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.647+0000 7f4890d5c640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f487c0097f0 tx=0x7f487c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.648+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c042070 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.648+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f487c030d80 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.648+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c18fc40 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.648+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c037040 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.648+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c190180 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.649+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c10aa10 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.650+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f487c045070 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.650+0000 7f4889ffb640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 0x7f487403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.650+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f487c06c200 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.650+0000 7f488bfff640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 0x7f487403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.651+0000 7f488bfff640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 0x7f487403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48800098c0 tx=0x7f4880006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.655+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f487c045360 con 0x7f488c1062e0 2024-09-05T06:32:14.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.842+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f488c194fa0 con 0x7f4874038fc0 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.912+0000 7f4889ffb640 1 -- 172.21.15.86:0/4208716208 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f488c194fa0 con 0x7f4874038fc0 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 msgr2=0x7f487403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 0x7f487403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48800098c0 tx=0x7f4880006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 msgr2=0x7f488c191510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f487c0097f0 tx=0x7f487c030c70 comp rx=0 tx=0).stop 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 shutdown_connections 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4874038fc0 0x7f487403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 --2- 172.21.15.86:0/4208716208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f488c1062e0 0x7f488c191510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 >> 172.21.15.86:0/4208716208 conn(0x7f488c101a90 msgr2=0x7f488c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 shutdown_connections 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:13.915+0000 7f4892fc0640 1 -- 172.21.15.86:0/4208716208 wait complete. 2024-09-05T06:32:14.940 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SztRZsh.e/4WGb7TvGGucu8okOLpphIPKczvf1..M7dh48txhYq/S", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725517935, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.304+0000 7f491863c640 1 Processor -- start 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.304+0000 7f491863c640 1 -- start start 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f491863c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f491863c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4910107e70 con 0x7f49101074a0 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f49163d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f49163d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44196/0 (socket says 172.21.15.86:44196) 2024-09-05T06:32:16.355 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f49163d8640 1 -- 172.21.15.86:0/1216261809 learned_addr learned my addr 172.21.15.86:0/1216261809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.305+0000 7f49163d8640 1 -- 172.21.15.86:0/1216261809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910108600 con 0x7f49101074a0 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.306+0000 7f49163d8640 1 --2- 172.21.15.86:0/1216261809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f490c009820 tx=0x7f490c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69022ee3f8a1eddd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.306+0000 7f49153d6640 1 -- 172.21.15.86:0/1216261809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c037070 con 0x7f49101074a0 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.306+0000 7f49153d6640 1 -- 172.21.15.86:0/1216261809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f490c0328f0 con 0x7f49101074a0 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.306+0000 7f49153d6640 1 -- 172.21.15.86:0/1216261809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c030740 con 0x7f49101074a0 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- 172.21.15.86:0/1216261809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 msgr2=0x7f49101078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 --2- 172.21.15.86:0/1216261809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f490c009820 tx=0x7f490c02a270 comp rx=0 tx=0).stop 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- 172.21.15.86:0/1216261809 shutdown_connections 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 --2- 172.21.15.86:0/1216261809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f49101078a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:16.356 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- 172.21.15.86:0/1216261809 >> 172.21.15.86:0/1216261809 conn(0x7f4910102c50 msgr2=0x7f4910105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- 172.21.15.86:0/1216261809 shutdown_connections 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- 172.21.15.86:0/1216261809 wait complete. 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 Processor -- start 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- start start 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.307+0000 7f491863c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f491010b9b0 con 0x7f49101074a0 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49163d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49163d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44210/0 (socket says 172.21.15.86:44210) 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49163d8640 1 -- 172.21.15.86:0/2759371231 learned_addr learned my addr 172.21.15.86:0/2759371231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49163d8640 1 -- 172.21.15.86:0/2759371231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491019b930 con 0x7f49101074a0 2024-09-05T06:32:16.357 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49163d8640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097f0 tx=0x7f490c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c042070 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f490c030d80 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c037040 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491019bbc0 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.308+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491019e8b0 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.309+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f490c045070 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.309+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491010bb30 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.309+0000 7f49077fe640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 0x7f48fc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.310+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f490c06c140 con 0x7f49101074a0 2024-09-05T06:32:16.358 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.310+0000 7f4915bd7640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 0x7f48fc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.310+0000 7f4915bd7640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 0x7f48fc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49000098c0 tx=0x7f4900006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.314+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f490c02ed70 con 0x7f49101074a0 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.494+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f491019ea80 con 0x7f48fc038fc0 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.715+0000 7f49077fe640 1 -- 172.21.15.86:0/2759371231 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f491019ea80 con 0x7f48fc038fc0 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 msgr2=0x7f48fc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 0x7f48fc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49000098c0 tx=0x7f4900006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 msgr2=0x7f491019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097f0 tx=0x7f490c030c70 comp rx=0 tx=0).stop 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 shutdown_connections 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48fc038fc0 0x7f48fc03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:16.359 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 --2- 172.21.15.86:0/2759371231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49101074a0 0x7f491019b3f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:16.360 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 >> 172.21.15.86:0/2759371231 conn(0x7f4910102c50 msgr2=0x7f4910103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:16.360 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 shutdown_connections 2024-09-05T06:32:16.360 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:15.717+0000 7f491863c640 1 -- 172.21.15.86:0/2759371231 wait complete. 2024-09-05T06:32:16.360 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-09-05T06:32:16.922 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:16 smithi086 ceph-mon[30943]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:16.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-09-05T06:32:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.745+0000 7f25cdbc0640 1 Processor -- start 2024-09-05T06:32:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.745+0000 7f25cdbc0640 1 -- start start 2024-09-05T06:32:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.745+0000 7f25cdbc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.745+0000 7f25cdbc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8075720 con 0x7f25c8107ed0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.746+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.746+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44222/0 (socket says 172.21.15.86:44222) 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.746+0000 7f25c77fe640 1 -- 172.21.15.86:0/3386654426 learned_addr learned my addr 172.21.15.86:0/3386654426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.746+0000 7f25c77fe640 1 -- 172.21.15.86:0/3386654426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c810a800 con 0x7f25c8107ed0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.746+0000 7f25c77fe640 1 --2- 172.21.15.86:0/3386654426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009820 tx=0x7f25b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab3c4dff5b85573 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25c6ffd640 1 -- 172.21.15.86:0/3386654426 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25b8037070 con 0x7f25c8107ed0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25c6ffd640 1 -- 172.21.15.86:0/3386654426 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25b80328f0 con 0x7f25c8107ed0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25c6ffd640 1 -- 172.21.15.86:0/3386654426 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25b8030740 con 0x7f25c8107ed0 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3386654426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 msgr2=0x7f25c810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3386654426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009820 tx=0x7f25b802a270 comp rx=0 tx=0).stop 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3386654426 shutdown_connections 2024-09-05T06:32:17.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3386654426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c810a2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3386654426 >> 172.21.15.86:0/3386654426 conn(0x7f25c8101a90 msgr2=0x7f25c8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3386654426 shutdown_connections 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.747+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3386654426 wait complete. 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25cdbc0640 1 Processor -- start 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25cdbc0640 1 -- start start 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25cdbc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25cdbc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810b7c0 con 0x7f25c8107ed0 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44238/0 (socket says 172.21.15.86:44238) 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.748+0000 7f25c77fe640 1 -- 172.21.15.86:0/3074612601 learned_addr learned my addr 172.21.15.86:0/3074612601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25c77fe640 1 -- 172.21.15.86:0/3074612601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c819bbf0 con 0x7f25c8107ed0 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25c77fe640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25b8030cc0 tx=0x7f25b8030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:17.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25b8042070 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25b8030e60 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c819be80 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25b8037040 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.749+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c819eb70 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f25b8045070 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25c57fa640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 0x7f25b403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25b806c140 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25beffd640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 0x7f25b403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c8108330 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.750+0000 7f25beffd640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 0x7f25b403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25b00098c0 tx=0x7f25b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.755+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25b8045520 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.923+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f25c8075ad0 con 0x7f25c8107ed0 2024-09-05T06:32:17.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.923+0000 7f25c57fa640 1 -- 172.21.15.86:0/3074612601 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f25b803add0 con 0x7f25c8107ed0 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 msgr2=0x7f25b403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 0x7f25b403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25b00098c0 tx=0x7f25b0006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 msgr2=0x7f25c819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25b8030cc0 tx=0x7f25b8030cf0 comp rx=0 tx=0).stop 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 shutdown_connections 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25b4039020 0x7f25b403b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 --2- 172.21.15.86:0/3074612601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25c8107ed0 0x7f25c819b6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 >> 172.21.15.86:0/3074612601 conn(0x7f25c8101a90 msgr2=0x7f25c81027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 shutdown_connections 2024-09-05T06:32:17.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:16.925+0000 7f25cdbc0640 1 -- 172.21.15.86:0/3074612601 wait complete. 2024-09-05T06:32:17.535 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T06:32:17.535 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-09-05T06:32:17.543 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T06:32:17.543 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-09-05T06:32:17.543 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-09-05T06:32:17.543 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086:8443/ 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout: Password: f0ea4spvaw 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:17.545 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config directory 2024-09-05T06:32:17.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3074612601' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-05T06:32:18.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:18 smithi086 ceph-mon[30943]: mgrmap e13: x(active, since 6s) 2024-09-05T06:32:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2684944653' entity='client.admin' 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f3640b0f640 1 Processor -- start 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f3640b0f640 1 -- start start 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f3640b0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f3640b0f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f363c107e30 con 0x7f363c107460 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f363a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f363a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44254/0 (socket says 172.21.15.86:44254) 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.934+0000 7f363a59c640 1 -- 172.21.15.86:0/1353916476 learned_addr learned my addr 172.21.15.86:0/1353916476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.935+0000 7f363a59c640 1 -- 172.21.15.86:0/1353916476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c108610 con 0x7f363c107460 2024-09-05T06:32:18.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.935+0000 7f363a59c640 1 --2- 172.21.15.86:0/1353916476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f362c009820 tx=0x7f362c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e864af54b2d6ac70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.935+0000 7f363959a640 1 -- 172.21.15.86:0/1353916476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c037070 con 0x7f363c107460 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f363959a640 1 -- 172.21.15.86:0/1353916476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c0328f0 con 0x7f363c107460 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f363959a640 1 -- 172.21.15.86:0/1353916476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c030740 con 0x7f363c107460 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 -- 172.21.15.86:0/1353916476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 msgr2=0x7f363c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 --2- 172.21.15.86:0/1353916476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f362c009820 tx=0x7f362c02a270 comp rx=0 tx=0).stop 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 -- 172.21.15.86:0/1353916476 shutdown_connections 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 --2- 172.21.15.86:0/1353916476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c107860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 -- 172.21.15.86:0/1353916476 >> 172.21.15.86:0/1353916476 conn(0x7f363c102c50 msgr2=0x7f363c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 -- 172.21.15.86:0/1353916476 shutdown_connections 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.936+0000 7f3640b0f640 1 -- 172.21.15.86:0/1353916476 wait complete. 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f3640b0f640 1 Processor -- start 2024-09-05T06:32:18.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f3640b0f640 1 -- start start 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f3640b0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f3640b0f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10b9c0 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f363a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f363a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44268/0 (socket says 172.21.15.86:44268) 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f363a59c640 1 -- 172.21.15.86:0/2684944653 learned_addr learned my addr 172.21.15.86:0/2684944653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.937+0000 7f363a59c640 1 -- 172.21.15.86:0/2684944653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c19b910 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f363a59c640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f362c009940 tx=0x7f362c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c042070 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c030e30 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c037040 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c19bba0 con 0x7f363c107460 2024-09-05T06:32:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.938+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c19e890 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f362c045070 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f362b7fe640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 0x7f362403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f362c06c3b0 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c10bb40 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f3639d9b640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 0x7f362403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.939+0000 7f3639d9b640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 0x7f362403b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36300098c0 tx=0x7f3630006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:17.944+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f362c045360 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.196+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f363c19ea60 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.199+0000 7f362b7fe640 1 -- 172.21.15.86:0/2684944653 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f362c03c650 con 0x7f363c107460 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 msgr2=0x7f362403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:18.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 0x7f362403b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36300098c0 tx=0x7f3630006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 msgr2=0x7f363c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f362c009940 tx=0x7f362c030cc0 comp rx=0 tx=0).stop 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 shutdown_connections 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3624039060 0x7f362403b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 --2- 172.21.15.86:0/2684944653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363c107460 0x7f363c19b3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 >> 172.21.15.86:0/2684944653 conn(0x7f363c102c50 msgr2=0x7f363c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 shutdown_connections 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-09-05T06:32:18.201+0000 7f3640b0f640 1 -- 172.21.15.86:0/2684944653 wait complete. 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.771 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:18.772 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-09-05T06:32:18.807 INFO:tasks.cephadm:Fetching config... 2024-09-05T06:32:18.807 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:32:18.807 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-05T06:32:18.826 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-05T06:32:18.826 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:32:18.826 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-05T06:32:18.882 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-05T06:32:18.883 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:32:18.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-05T06:32:18.948 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-05T06:32:18.948 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:32:18.948 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-05T06:32:19.003 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-05T06:32:19.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T06:32:19.078 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:19.089 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T06:32:19.131 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:19.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T06:32:19.185 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrIuPM6r73hvYoEuKXnDkkEjMtGKPCGWF13nlqMD1qZ6eZaCAJurC28jK24i0++9vsdwZnWZ3SJclnV1CO5sWTd/O0SoU4cu9DmM5ihhr8iHrF1nliOGlPL8IPheD9AvG/MpSJ8PIDtpeg8w+DnjBTolHRctYKWQAquHCYGa9xYsFH2d/GLRuvxNWEuVMwskURBpltWeyXU2NRlvNWnJYwDu/ohlosifNBHsRVvh2e2/Bu3MHEof1TsRopl9tBi+ve68KyK+j92tjwEG9s6v4J3ZL3yo+xwnpIfZ/+nk+3b7qsH+prVdN4Mehvm5s5HH5DYKtC25R4HO0DWCBAQ+lHu9Q4ce5OrMTuWzXlJI9Z5Uxaah0RW9HzcbnFmMQLC69o/psBzZamJIHBfnaLdzN26Y0aPPk1s92ynvSnF0YswOXaoMRtDUkFpeSRhUCVvoF7btOidkNOj00PfvU/Tir0LBESewzXSCp3vVakkhspWaZBj+vkdogRuCJZhIHVVvU= ceph-5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:19.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-05T06:32:19.445 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 -- 172.21.15.86:0/349365729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 msgr2=0x7f0474104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 --2- 172.21.15.86:0/349365729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474104490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0464009880 tx=0x7f046402a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 -- 172.21.15.86:0/349365729 shutdown_connections 2024-09-05T06:32:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 --2- 172.21.15.86:0/349365729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474104490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 -- 172.21.15.86:0/349365729 >> 172.21.15.86:0/349365729 conn(0x7f04740fbc80 msgr2=0x7f04740fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 -- 172.21.15.86:0/349365729 shutdown_connections 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.926+0000 7f0478dfc640 1 -- 172.21.15.86:0/349365729 wait complete. 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f0478dfc640 1 Processor -- start 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f0478dfc640 1 -- start start 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f0478dfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f0478dfc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0474107560 con 0x7f04741020a0 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f047259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f047259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44284/0 (socket says 172.21.15.86:44284) 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.927+0000 7f047259c640 1 -- 172.21.15.86:0/1388290539 learned_addr learned my addr 172.21.15.86:0/1388290539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f047259c640 1 -- 172.21.15.86:0/1388290539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0474197200 con 0x7f04741020a0 2024-09-05T06:32:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f047259c640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0464005ab0 tx=0x7f04640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046403f070 con 0x7f04741020a0 2024-09-05T06:32:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741974f0 con 0x7f04741020a0 2024-09-05T06:32:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0464003aa0 con 0x7f04741020a0 2024-09-05T06:32:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0464035030 con 0x7f04741020a0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.928+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047419a180 con 0x7f04741020a0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.929+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0464042070 con 0x7f04741020a0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.929+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474107ce0 con 0x7f04741020a0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.929+0000 7f046b7fe640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 0x7f046003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.929+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f046406cf30 con 0x7f04741020a0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.929+0000 7f046bfff640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 0x7f046003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.930+0000 7f046bfff640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 0x7f046003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f045c0098c0 tx=0x7f045c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:19.934+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0464042360 con 0x7f04741020a0 2024-09-05T06:32:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.102+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f047402d9a0 con 0x7f04741020a0 2024-09-05T06:32:20.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.104+0000 7f046b7fe640 1 -- 172.21.15.86:0/1388290539 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f046402f300 con 0x7f04741020a0 2024-09-05T06:32:20.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 msgr2=0x7f046003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:20.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 0x7f046003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f045c0098c0 tx=0x7f045c006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:20.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 msgr2=0x7f0474196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0464005ab0 tx=0x7f04640048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 shutdown_connections 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0460039060 0x7f046003b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 --2- 172.21.15.86:0/1388290539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04741020a0 0x7f0474196cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 >> 172.21.15.86:0/1388290539 conn(0x7f04740fbc80 msgr2=0x7f04740fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 shutdown_connections 2024-09-05T06:32:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:20.106+0000 7f0478dfc640 1 -- 172.21.15.86:0/1388290539 wait complete. 2024-09-05T06:32:21.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1388290539' entity='client.admin' 2024-09-05T06:32:21.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:21.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:21.198 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:32:21.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:21.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:21.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:21.224 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-05T06:32:21.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-05T06:32:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:21.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 -- 172.21.15.86:0/3218526953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 msgr2=0x7fc9f8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:21.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 --2- 172.21.15.86:0/3218526953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009880 tx=0x7fc9f402a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:21.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 -- 172.21.15.86:0/3218526953 shutdown_connections 2024-09-05T06:32:21.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 --2- 172.21.15.86:0/3218526953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 -- 172.21.15.86:0/3218526953 >> 172.21.15.86:0/3218526953 conn(0x7fc9f80fbc80 msgr2=0x7fc9f80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 -- 172.21.15.86:0/3218526953 shutdown_connections 2024-09-05T06:32:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.894+0000 7fc9ff742640 1 -- 172.21.15.86:0/3218526953 wait complete. 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9ff742640 1 Processor -- start 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9ff742640 1 -- start start 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9ff742640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9ff742640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81064d0 con 0x7fc9f8100510 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9fd4de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9fd4de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44302/0 (socket says 172.21.15.86:44302) 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9fd4de640 1 -- 172.21.15.86:0/4253915424 learned_addr learned my addr 172.21.15.86:0/4253915424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.895+0000 7fc9fd4de640 1 -- 172.21.15.86:0/4253915424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f8196170 con 0x7fc9f8100510 2024-09-05T06:32:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9fd4de640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005ab0 tx=0x7fc9f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f403f070 con 0x7fc9f8100510 2024-09-05T06:32:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f8196400 con 0x7fc9f8100510 2024-09-05T06:32:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9f4003aa0 con 0x7fc9f8100510 2024-09-05T06:32:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4035030 con 0x7fc9f8100510 2024-09-05T06:32:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.896+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f81990f0 con 0x7fc9f8100510 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc9f4042070 con 0x7fc9f8100510 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9ee7fc640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 0x7fc9e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f8106c50 con 0x7fc9f8100510 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9f406cc90 con 0x7fc9f8100510 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9fccdd640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 0x7fc9e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.897+0000 7fc9fccdd640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 0x7fc9e403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098c0 tx=0x7fc9e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:21.902+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9f40372d0 con 0x7fc9f8100510 2024-09-05T06:32:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.070+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc9f802d830 con 0x7fc9e4039060 2024-09-05T06:32:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.073+0000 7fc9ee7fc640 1 -- 172.21.15.86:0/4253915424 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9f802d830 con 0x7fc9e4039060 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 msgr2=0x7fc9e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 0x7fc9e403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098c0 tx=0x7fc9e8006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 msgr2=0x7fc9f8195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005ab0 tx=0x7fc9f40048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 shutdown_connections 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc9e4039060 0x7fc9e403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 --2- 172.21.15.86:0/4253915424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f8100510 0x7fc9f8195c30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 >> 172.21.15.86:0/4253915424 conn(0x7fc9f80fbc80 msgr2=0x7fc9f80fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 shutdown_connections 2024-09-05T06:32:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:22.076+0000 7fc9ff742640 1 -- 172.21.15.86:0/4253915424 wait complete. 2024-09-05T06:32:22.837 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-09-05T06:32:22.837 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:32:22.837 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-09-05T06:32:22.857 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:32:22.857 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:22.915 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-09-05T06:32:22.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch host add smithi096 2024-09-05T06:32:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: Updating smithi086:/etc/ceph/ceph.conf 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 -- 172.21.15.86:0/3125043477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 msgr2=0x7f6de8102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 --2- 172.21.15.86:0/3125043477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8102bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009880 tx=0x7f6de002a670 comp rx=0 tx=0).stop 2024-09-05T06:32:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 -- 172.21.15.86:0/3125043477 shutdown_connections 2024-09-05T06:32:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 --2- 172.21.15.86:0/3125043477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8102bd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 -- 172.21.15.86:0/3125043477 >> 172.21.15.86:0/3125043477 conn(0x7f6de80fc1b0 msgr2=0x7f6de80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 -- 172.21.15.86:0/3125043477 shutdown_connections 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.574+0000 7f6decec1640 1 -- 172.21.15.86:0/3125043477 wait complete. 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6decec1640 1 Processor -- start 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6decec1640 1 -- start start 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6decec1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6decec1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6de810bb70 con 0x7f6de81027f0 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6de659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6de659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53942/0 (socket says 172.21.15.86:53942) 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.575+0000 7f6de659c640 1 -- 172.21.15.86:0/1719836478 learned_addr learned my addr 172.21.15.86:0/1719836478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6de659c640 1 -- 172.21.15.86:0/1719836478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de8102340 con 0x7f6de81027f0 2024-09-05T06:32:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6de659c640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009230 tx=0x7f6de00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de003f070 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de8100990 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de0004d10 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de0035030 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.576+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de8100e50 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.577+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6de0042070 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.577+0000 7f6dd77fe640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 0x7f6dd003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.577+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de806a710 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.577+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6de006cd70 con 0x7f6de81027f0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.577+0000 7f6de5d9b640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 0x7f6dd003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.578+0000 7f6de5d9b640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 0x7f6dd003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0098c0 tx=0x7f6ddc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.582+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de0039350 con 0x7f6de81027f0 2024-09-05T06:32:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:23.750+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7f6de8101980 con 0x7f6dd0039060 2024-09-05T06:32:24.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:24 smithi086 ceph-mon[30943]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:24.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:24 smithi086 ceph-mon[30943]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:24.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:24 smithi086 ceph-mon[30943]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:32:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:25 smithi086 ceph-mon[30943]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:26.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:26 smithi086 ceph-mon[30943]: Deploying cephadm binary to smithi096 2024-09-05T06:32:26.944 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-09-05T06:32:26.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.944+0000 7f6dd77fe640 1 -- 172.21.15.86:0/1719836478 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6de8101980 con 0x7f6dd0039060 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 msgr2=0x7f6dd003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 0x7f6dd003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0098c0 tx=0x7f6ddc006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 msgr2=0x7f6de8100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009230 tx=0x7f6de00048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 shutdown_connections 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6dd0039060 0x7f6dd003b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 --2- 172.21.15.86:0/1719836478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de81027f0 0x7f6de8100450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 >> 172.21.15.86:0/1719836478 conn(0x7f6de80fc1b0 msgr2=0x7f6de81055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 shutdown_connections 2024-09-05T06:32:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:26.946+0000 7f6decec1640 1 -- 172.21.15.86:0/1719836478 wait complete. 2024-09-05T06:32:27.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T06:32:27.785 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:28.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:28.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:27 smithi086 ceph-mon[30943]: Added host smithi096 2024-09-05T06:32:28.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:28.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:28.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/3016710989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 msgr2=0x7fc8a4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/3016710989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc894009880 tx=0x7fc89402a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/3016710989 shutdown_connections 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/3016710989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/3016710989 >> 172.21.15.86:0/3016710989 conn(0x7fc8a40fd290 msgr2=0x7fc8a40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/3016710989 shutdown_connections 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.264+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/3016710989 wait complete. 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8aa2b2640 1 Processor -- start 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8aa2b2640 1 -- start start 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8aa2b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8aa2b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410bb40 con 0x7fc8a4107660 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53950/0 (socket says 172.21.15.86:53950) 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.265+0000 7fc8a3fff640 1 -- 172.21.15.86:0/1036263157 learned_addr learned my addr 172.21.15.86:0/1036263157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8a3fff640 1 -- 172.21.15.86:0/1036263157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a4197580 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8a3fff640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc894009230 tx=0x7fc8940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc89403f070 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc894003aa0 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a4197810 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc894035030 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.266+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a419b910 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.267+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc894042070 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.267+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a4102cd0 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.267+0000 7fc8a17fa640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 0x7fc88c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.267+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc89406cf50 con 0x7fc8a4107660 2024-09-05T06:32:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.267+0000 7fc8a37fe640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 0x7fc88c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.268+0000 7fc8a37fe640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 0x7fc88c03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc8980098c0 tx=0x7fc898006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.272+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8940433b0 con 0x7fc8a4107660 2024-09-05T06:32:28.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.457+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8a41983a0 con 0x7fc88c0390c0 2024-09-05T06:32:28.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.458+0000 7fc8a17fa640 1 -- 172.21.15.86:0/1036263157 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc8a41983a0 con 0x7fc88c0390c0 2024-09-05T06:32:28.458 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:28.458 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 msgr2=0x7fc88c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 0x7fc88c03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc8980098c0 tx=0x7fc898006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 msgr2=0x7fc8a4197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc894009230 tx=0x7fc8940048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 shutdown_connections 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc88c0390c0 0x7fc88c03b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 --2- 172.21.15.86:0/1036263157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4107660 0x7fc8a4197040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 >> 172.21.15.86:0/1036263157 conn(0x7fc8a40fd290 msgr2=0x7fc8a40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 shutdown_connections 2024-09-05T06:32:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:28.460+0000 7fc8aa2b2640 1 -- 172.21.15.86:0/1036263157 wait complete. 2024-09-05T06:32:29.097 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2024-09-05T06:32:29.097 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:32:29.098 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2024-09-05T06:32:29.117 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:32:29.117 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:29.175 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2024-09-05T06:32:29.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch host add smithi115 2024-09-05T06:32:29.413 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:29.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:29 smithi086 ceph-mon[30943]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:32:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 -- 172.21.15.86:0/2684158229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 msgr2=0x7faebc102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 --2- 172.21.15.86:0/2684158229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc102b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faeac009c10 tx=0x7faeac02aa10 comp rx=0 tx=0).stop 2024-09-05T06:32:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 -- 172.21.15.86:0/2684158229 shutdown_connections 2024-09-05T06:32:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 --2- 172.21.15.86:0/2684158229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc102b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 -- 172.21.15.86:0/2684158229 >> 172.21.15.86:0/2684158229 conn(0x7faebc077310 msgr2=0x7faebc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 -- 172.21.15.86:0/2684158229 shutdown_connections 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.863+0000 7faec17f9640 1 -- 172.21.15.86:0/2684158229 wait complete. 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faec17f9640 1 Processor -- start 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faec17f9640 1 -- start start 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faec17f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faec17f9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faebc10bb30 con 0x7faebc102730 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faebaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faebaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53966/0 (socket says 172.21.15.86:53966) 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faebaffd640 1 -- 172.21.15.86:0/1959284304 learned_addr learned my addr 172.21.15.86:0/1959284304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.864+0000 7faebaffd640 1 -- 172.21.15.86:0/1959284304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc197650 con 0x7faebc102730 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faebaffd640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeac005dd0 tx=0x7faeac02f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac034070 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeac02feb0 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faebc1978e0 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac004360 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.865+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faebc197ce0 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faeac037020 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faeb3fff640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 0x7faea803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faebc069f90 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faeac06cfd0 con 0x7faebc102730 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faeba7fc640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 0x7faea803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.866+0000 7faeba7fc640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 0x7faea803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faea40098c0 tx=0x7faea4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:29.871+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faeac03f070 con 0x7faebc102730 2024-09-05T06:32:30.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:30.038+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) -- 0x7faebc106be0 con 0x7faea8039060 2024-09-05T06:32:30.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:30 smithi086 ceph-mon[30943]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:32.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: Deploying cephadm binary to smithi115 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:33.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.254+0000 7faeb3fff640 1 -- 172.21.15.86:0/1959284304 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7faebc106be0 con 0x7faea8039060 2024-09-05T06:32:33.255 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi115' with addr '172.21.15.115' 2024-09-05T06:32:33.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 msgr2=0x7faea803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 0x7faea803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faea40098c0 tx=0x7faea4006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 msgr2=0x7faebc197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeac005dd0 tx=0x7faeac02f980 comp rx=0 tx=0).stop 2024-09-05T06:32:33.257 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:33 smithi086 ceph-mon[30943]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:33.257 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:33 smithi086 ceph-mon[30943]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 shutdown_connections 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7faea8039060 0x7faea803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 --2- 172.21.15.86:0/1959284304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc102730 0x7faebc197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:33.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 >> 172.21.15.86:0/1959284304 conn(0x7faebc077310 msgr2=0x7faebc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:33.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 shutdown_connections 2024-09-05T06:32:33.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:33.257+0000 7faec17f9640 1 -- 172.21.15.86:0/1959284304 wait complete. 2024-09-05T06:32:33.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T06:32:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:34 smithi086 ceph-mon[30943]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 -- 172.21.15.86:0/1613891637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 msgr2=0x7f9260106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 --2- 172.21.15.86:0/1613891637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9250009880 tx=0x7f925002a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 -- 172.21.15.86:0/1613891637 shutdown_connections 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 --2- 172.21.15.86:0/1613891637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 -- 172.21.15.86:0/1613891637 >> 172.21.15.86:0/1613891637 conn(0x7f92600fc1b0 msgr2=0x7f92600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 -- 172.21.15.86:0/1613891637 shutdown_connections 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.661+0000 7f9265783640 1 -- 172.21.15.86:0/1613891637 wait complete. 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f9265783640 1 Processor -- start 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f9265783640 1 -- start start 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f9265783640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f9265783640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f926010aa40 con 0x7f9260106560 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f925effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f925effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60776/0 (socket says 172.21.15.86:60776) 2024-09-05T06:32:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.662+0000 7f925effd640 1 -- 172.21.15.86:0/3307952510 learned_addr learned my addr 172.21.15.86:0/3307952510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f925effd640 1 -- 172.21.15.86:0/3307952510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9260196460 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f925effd640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9250009230 tx=0x7f92500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925003f070 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9250004d10 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601966f0 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9250035030 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.663+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926019a7f0 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9250042070 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9260101bd0 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f9247fff640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 0x7f924803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f925006cee0 con 0x7f9260106560 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f925e7fc640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 0x7f924803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.664+0000 7f925e7fc640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 0x7f924803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f92540098c0 tx=0x7f9254006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:34.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.669+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9250039350 con 0x7f9260106560 2024-09-05T06:32:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.836+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92601069c0 con 0x7f9248039060 2024-09-05T06:32:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.837+0000 7f9247fff640 1 -- 172.21.15.86:0/3307952510 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f92601069c0 con 0x7f9248039060 2024-09-05T06:32:34.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:32:34.838 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 msgr2=0x7f924803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 0x7f924803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f92540098c0 tx=0x7f9254006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 msgr2=0x7f9260195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9250009230 tx=0x7f92500048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 shutdown_connections 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9248039060 0x7f924803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 --2- 172.21.15.86:0/3307952510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9260106560 0x7f9260195f20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 >> 172.21.15.86:0/3307952510 conn(0x7f92600fc1b0 msgr2=0x7f92600fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 shutdown_connections 2024-09-05T06:32:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:34.840+0000 7f9265783640 1 -- 172.21.15.86:0/3307952510 wait complete. 2024-09-05T06:32:35.433 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:35 smithi086 ceph-mon[30943]: Added host smithi115 2024-09-05T06:32:35.433 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:35.463 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-05T06:32:35.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd crush tunables default 2024-09-05T06:32:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 -- 172.21.15.86:0/3840142526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 msgr2=0x7fb844106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3840142526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb834009880 tx=0x7fb83402a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 -- 172.21.15.86:0/3840142526 shutdown_connections 2024-09-05T06:32:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3840142526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 -- 172.21.15.86:0/3840142526 >> 172.21.15.86:0/3840142526 conn(0x7fb8440fc1b0 msgr2=0x7fb8440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 -- 172.21.15.86:0/3840142526 shutdown_connections 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.124+0000 7fb848d2e640 1 -- 172.21.15.86:0/3840142526 wait complete. 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb848d2e640 1 Processor -- start 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb848d2e640 1 -- start start 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb848d2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb848d2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb84410aa40 con 0x7fb844106560 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb84259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb84259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60788/0 (socket says 172.21.15.86:60788) 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.125+0000 7fb84259c640 1 -- 172.21.15.86:0/3362561893 learned_addr learned my addr 172.21.15.86:0/3362561893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb84259c640 1 -- 172.21.15.86:0/3362561893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844197560 con 0x7fb844106560 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb84259c640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb834009230 tx=0x7fb8340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb83403f070 con 0x7fb844106560 2024-09-05T06:32:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441977f0 con 0x7fb844106560 2024-09-05T06:32:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb834004d10 con 0x7fb844106560 2024-09-05T06:32:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb834035030 con 0x7fb844106560 2024-09-05T06:32:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.126+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84419b8f0 con 0x7fb844106560 2024-09-05T06:32:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.127+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb834042070 con 0x7fb844106560 2024-09-05T06:32:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.127+0000 7fb83b7fe640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 0x7fb83003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.127+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb844101bd0 con 0x7fb844106560 2024-09-05T06:32:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.127+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb83406ccc0 con 0x7fb844106560 2024-09-05T06:32:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.127+0000 7fb83bfff640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 0x7fb83003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.128+0000 7fb83bfff640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 0x7fb83003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0098c0 tx=0x7fb82c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:36.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.132+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb834039350 con 0x7fb844106560 2024-09-05T06:32:36.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:36.306+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb844198380 con 0x7fb844106560 2024-09-05T06:32:36.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:36 smithi086 ceph-mon[30943]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:32:36.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:36 smithi086 ceph-mon[30943]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.256+0000 7fb83b7fe640 1 -- 172.21.15.86:0/3362561893 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb83402f300 con 0x7fb844106560 2024-09-05T06:32:37.257 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 msgr2=0x7fb83003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 0x7fb83003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0098c0 tx=0x7fb82c006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 msgr2=0x7fb844197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb834009230 tx=0x7fb8340048f0 comp rx=0 tx=0).stop 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 shutdown_connections 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb830039060 0x7fb83003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 --2- 172.21.15.86:0/3362561893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb844106560 0x7fb844197020 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 >> 172.21.15.86:0/3362561893 conn(0x7fb8440fc1b0 msgr2=0x7fb844069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 shutdown_connections 2024-09-05T06:32:37.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:37.259+0000 7fb848d2e640 1 -- 172.21.15.86:0/3362561893 wait complete. 2024-09-05T06:32:37.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3362561893' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T06:32:38.003 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-09-05T06:32:38.003 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-09-05T06:32:38.003 INFO:tasks.cephadm:Adding mon.c on smithi115 2024-09-05T06:32:38.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c' 2024-09-05T06:32:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:38.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 -- 172.21.15.115:0/1197058363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 msgr2=0x7f42ec102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:38.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 --2- 172.21.15.115:0/1197058363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec102b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009810 tx=0x7f42dc02a540 comp rx=0 tx=0).stop 2024-09-05T06:32:38.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 -- 172.21.15.115:0/1197058363 shutdown_connections 2024-09-05T06:32:38.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 --2- 172.21.15.115:0/1197058363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec102b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:38.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 -- 172.21.15.115:0/1197058363 >> 172.21.15.115:0/1197058363 conn(0x7f42ec077310 msgr2=0x7f42ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:38.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 -- 172.21.15.115:0/1197058363 shutdown_connections 2024-09-05T06:32:38.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.707+0000 7f42f191e640 1 -- 172.21.15.115:0/1197058363 wait complete. 2024-09-05T06:32:38.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.708+0000 7f42f191e640 1 Processor -- start 2024-09-05T06:32:38.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.708+0000 7f42f191e640 1 -- start start 2024-09-05T06:32:38.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.708+0000 7f42f191e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:38.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.708+0000 7f42f191e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec10bb30 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.115:41396/0 (socket says 172.21.15.115:41396) 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42eaffd640 1 -- 172.21.15.115:0/1084094469 learned_addr learned my addr 172.21.15.115:0/1084094469 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42eaffd640 1 -- 172.21.15.115:0/1084094469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec197580 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42eaffd640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42dc030b00 tx=0x7f42dc030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc040070 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec197810 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42dc0318b0 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc003de0 con 0x7f42ec102730 2024-09-05T06:32:38.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.709+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec197c70 con 0x7f42ec102730 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.710+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f42dc039040 con 0x7f42ec102730 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.710+0000 7f42f091c640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 0x7f42d403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.710+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f42dc06e4e0 con 0x7f42ec102730 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.711+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec069f90 con 0x7f42ec102730 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.711+0000 7f42ea7fc640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 0x7f42d403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:38.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.711+0000 7f42ea7fc640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 0x7f42d403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42e00098c0 tx=0x7f42e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:38.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.716+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42dc04dbf0 con 0x7f42ec102730 2024-09-05T06:32:38.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3362561893' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:38 smithi086 ceph-mon[30943]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:32:38.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.899+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}) -- 0x7f42ec103bc0 con 0x7f42d4039060 2024-09-05T06:32:38.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.904+0000 7f42f091c640 1 -- 172.21.15.115:0/1084094469 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f42ec103bc0 con 0x7f42d4039060 2024-09-05T06:32:38.904 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mon update... 2024-09-05T06:32:38.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 msgr2=0x7f42d403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:38.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 0x7f42d403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42e00098c0 tx=0x7f42e0006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:38.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 msgr2=0x7f42ec197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f42dc030b00 tx=0x7f42dc030b30 comp rx=0 tx=0).stop 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 shutdown_connections 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f42d4039060 0x7f42d403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 --2- 172.21.15.115:0/1084094469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42ec102730 0x7f42ec197040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 >> 172.21.15.115:0/1084094469 conn(0x7f42ec077310 msgr2=0x7f42ec1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.906+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 shutdown_connections 2024-09-05T06:32:38.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:38.907+0000 7f42f191e640 1 -- 172.21.15.115:0/1084094469 wait complete. 2024-09-05T06:32:39.444 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.b.service 2024-09-05T06:32:39.447 DEBUG:teuthology.orchestra.run.smithi115:mon.c> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.c.service 2024-09-05T06:32:39.449 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T06:32:39.450 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c;count:3 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: Deploying daemon mon.c on smithi115 2024-09-05T06:32:39.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:39 smithi086 ceph-mon[30943]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:40.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 -- 172.21.15.115:0/2310799000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 msgr2=0x7f7894106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:40.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 --2- 172.21.15.115:0/2310799000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f788c009c10 tx=0x7f788c02aa10 comp rx=0 tx=0).stop 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 -- 172.21.15.115:0/2310799000 shutdown_connections 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 --2- 172.21.15.115:0/2310799000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 -- 172.21.15.115:0/2310799000 >> 172.21.15.115:0/2310799000 conn(0x7f7894077310 msgr2=0x7f78940fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 -- 172.21.15.115:0/2310799000 shutdown_connections 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.586+0000 7f7899d9b640 1 -- 172.21.15.115:0/2310799000 wait complete. 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f7899d9b640 1 Processor -- start 2024-09-05T06:32:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f7899d9b640 1 -- start start 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f7899d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f7899d9b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f789410aa20 con 0x7f7894106540 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f78937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f78937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.115:41402/0 (socket says 172.21.15.115:41402) 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.587+0000 7f78937fe640 1 -- 172.21.15.115:0/3556543500 learned_addr learned my addr 172.21.15.115:0/3556543500 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:32:40.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f78937fe640 1 -- 172.21.15.115:0/3556543500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7894196490 con 0x7f7894106540 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f78937fe640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f788c005dd0 tx=0x7f788c02f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c034070 con 0x7f7894106540 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7894196720 con 0x7f7894106540 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f788c02feb0 con 0x7f7894106540 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c004360 con 0x7f7894106540 2024-09-05T06:32:40.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.588+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7894196ba0 con 0x7f7894106540 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.589+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f788c037020 con 0x7f7894106540 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.589+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7894101bb0 con 0x7f7894106540 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.589+0000 7f7890ff9640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 0x7f788003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.589+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f788c06cf40 con 0x7f7894106540 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.590+0000 7f7892ffd640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 0x7f788003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:40.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.590+0000 7f7892ffd640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 0x7f788003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098c0 tx=0x7f787c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:40.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.594+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f788c03f070 con 0x7f7894106540 2024-09-05T06:32:40.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.832+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f789419a880 con 0x7f7894106540 2024-09-05T06:32:40.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.832+0000 7f7890ff9640 1 -- 172.21.15.115:0/3556543500 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f788c02c040 con 0x7f7894106540 2024-09-05T06:32:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:32:40.833 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","modified":"2024-09-05T06:31:22.792336Z","created":"2024-09-05T06:31:22.792336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T06:32:40.833 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2024-09-05T06:32:40.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 msgr2=0x7f788003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:40.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 0x7f788003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f787c0098c0 tx=0x7f787c006db0 comp rx=0 tx=0).stop 2024-09-05T06:32:40.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 msgr2=0x7f7894195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:40.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f788c005dd0 tx=0x7f788c02f980 comp rx=0 tx=0).stop 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 shutdown_connections 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7880039060 0x7f788003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 --2- 172.21.15.115:0/3556543500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7894106540 0x7f7894195f50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 >> 172.21.15.115:0/3556543500 conn(0x7f7894077310 msgr2=0x7f78940fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 shutdown_connections 2024-09-05T06:32:40.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:40.834+0000 7f7899d9b640 1 -- 172.21.15.115:0/3556543500 wait complete. 2024-09-05T06:32:41.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:40 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/3556543500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:42.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:41 smithi086 ceph-mon[30943]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:43.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 podman[34772]: 2024-09-05 06:32:43.388770571 +0000 UTC m=+0.114591227 container create 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-05T06:32:43.865 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T06:32:43.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T06:32:43.889 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 podman[34772]: 2024-09-05 06:32:43.615972326 +0000 UTC m=+0.341792982 container init 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T06:32:43.889 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 podman[34772]: 2024-09-05 06:32:43.619292058 +0000 UTC m=+0.345112723 container start 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T06:32:43.889 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T06:32:43.889 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable), process ceph-mon, pid 2 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: pidfile_write: ignore empty --pid-file 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: load: jerasure load: lrc 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: RocksDB version: 7.9.2 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Git sha 0 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Compile date 2024-09-04 15:32:03 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: DB SUMMARY 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: DB Session ID: CT9FQGRPAQNHCDVKCN8P 2024-09-05T06:32:43.890 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: CURRENT file: CURRENT 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.error_if_exists: 0 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.create_if_missing: 0 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.paranoid_checks: 1 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.env: 0x5608207b4940 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.info_log: 0x56082138f6e0 2024-09-05T06:32:43.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T06:32:43.893 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.statistics: (nil) 2024-09-05T06:32:43.893 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.use_fsync: 0 2024-09-05T06:32:43.893 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_log_file_size: 0 2024-09-05T06:32:43.893 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T06:32:43.893 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_fallocate: 1 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.use_direct_reads: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.db_log_dir: 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.wal_dir: 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T06:32:43.894 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.write_buffer_manager: 0x56082139edc0 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T06:32:43.895 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.unordered_write: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.row_cache: None 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.wal_filter: None 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.two_write_queues: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.wal_compression: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.atomic_flush: 0 2024-09-05T06:32:43.896 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.log_readahead_size: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_background_jobs: 2 2024-09-05T06:32:43.897 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_background_compactions: -1 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_subcompactions: 1 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_open_files: -1 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T06:32:43.898 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_background_flushes: -1 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Compression algorithms supported: 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kZSTD supported: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kXpressCompression supported: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kBZip2Compression supported: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kLZ4Compression supported: 1 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kZlibCompression supported: 1 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: kSnappyCompression supported: 1 2024-09-05T06:32:43.899 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.merge_operator: 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_filter: None 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56082138f320) 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-09-05T06:32:43.900 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_type: 0 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_index_type: 0 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_shortening: 1 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: checksum: 4 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: no_block_cache: 0 2024-09-05T06:32:43.901 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache: 0x5608213831f0 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_options: 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: capacity : 536870912 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_shard_bits : 4 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: strict_capacity_limit : 0 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_compressed: (nil) 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: persistent_cache: (nil) 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size: 4096 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size_deviation: 10 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_restart_interval: 16 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_block_restart_interval: 1 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: metadata_block_size: 4096 2024-09-05T06:32:43.902 INFO:journalctl@ceph.mon.c.smithi115.stdout: partition_filters: 0 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: use_delta_encoding: 1 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: filter_policy: bloomfilter 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: whole_key_filtering: 1 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: verify_compression: 0 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: format_version: 5 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_index_compression: 1 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_align: 0 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: max_auto_readahead_size: 262144 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: prepopulate_block_cache: 0 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: initial_auto_readahead_size: 8192 2024-09-05T06:32:43.903 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression: NoCompression 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.num_levels: 7 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T06:32:43.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T06:32:43.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T06:32:43.906 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T06:32:43.907 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.inplace_update_support: 0 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T06:32:43.908 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.bloom_locality: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.max_successive_merges: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.ttl: 2592000 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enable_blob_files: false 2024-09-05T06:32:43.909 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.min_blob_size: 0 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: afb52822-65a1-43cd-8f04-6aeb75f388e3 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517963645795, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-05T06:32:43.910 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517963646216, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725517963, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "afb52822-65a1-43cd-8f04-6aeb75f388e3", "db_session_id": "CT9FQGRPAQNHCDVKCN8P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725517963646309, "job": 1, "event": "recovery_finished"} 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5608213b0e00 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: DB pointer 0x5608214ac000 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T06:32:43.911 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** DB Stats ** 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T06:32:43.912 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T06:32:43.913 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: Block cache BinnedLRUCache@0x5608213831f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: using public_addr v2:172.21.15.115:0/0 -> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: starting mon.c rank -1 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(???) e0 preinit fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).mds e1 new map 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-05T06:32:43.914 INFO:journalctl@ceph.mon.c.smithi115.stdout: e1 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: btime 2024-09-05T06:31:26:636593+0000 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: legacy client fscid: -1 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout: No filesystems configured 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:32:43.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3362561893' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3362561893' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:32:43.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c;count:3 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:43.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: Deploying daemon mon.c on smithi115 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/3556543500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:43.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 ceph-mon[34791]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T06:32:43.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 bash[34772]: 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 2024-09-05T06:32:43.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:43 smithi115 systemd[1]: Started Ceph mon.c for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:32:44.121 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/2779942080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 msgr2=0x7f3db0007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/2779942080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3db0007bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4101170 tx=0x7f3dc002bb10 comp rx=0 tx=0).stop 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/2779942080 shutdown_connections 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/2779942080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3db0007bd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/2779942080 >> 172.21.15.115:0/2779942080 conn(0x7f3dc40fb9a0 msgr2=0x7f3dc40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:48.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/2779942080 shutdown_connections 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.960+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/2779942080 wait complete. 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.961+0000 7f3dcc6e6640 1 Processor -- start 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.961+0000 7f3dcc6e6640 1 -- start start 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dcc6e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3dc4194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dcc6e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dcc6e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc4107430 con 0x7f3db0005740 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dcc6e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41072b0 con 0x7f3dc4195510 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60728/0 (socket says 172.21.15.115:60728) 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 -- 172.21.15.115:0/1043414243 learned_addr learned my addr 172.21.15.115:0/1043414243 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dca482640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3dc4194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 -- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 msgr2=0x7f3dc41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T06:32:48.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 -- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 msgr2=0x7f3dc41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T06:32:48.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T06:32:48.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dc9c81640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:48.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dca482640 1 -- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 msgr2=0x7f3dc41998a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:48.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dca482640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:48.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.962+0000 7f3dca482640 1 -- 172.21.15.115:0/1043414243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc4195b30 con 0x7f3db0005740 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dca482640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3dc4194fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ec10 tx=0x7f3dc0037630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3dc0005050 con 0x7f3db0005740 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dc0004660 con 0x7f3db0005740 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc4199de0 con 0x7f3db0005740 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3dc003f5e0 con 0x7f3db0005740 2024-09-05T06:32:48.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.963+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc419a2f0 con 0x7f3db0005740 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.964+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3dc003f780 con 0x7f3db0005740 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.964+0000 7f3dbb7fe640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 0x7f3da803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.964+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc4101420 con 0x7f3db0005740 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.964+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3dc006e430 con 0x7f3db0005740 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.964+0000 7f3dc9c81640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 0x7f3da803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:48.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.965+0000 7f3dc9c81640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 0x7f3da803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009100 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:48.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:48.971+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3dc0033030 con 0x7f3db0005740 2024-09-05T06:32:48.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:48.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: mon.a calling monitor election 2024-09-05T06:32:48.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: mon.c calling monitor election 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: monmap epoch 2 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: last_changed 2024-09-05T06:32:43.670863+0000 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-05T06:32:48.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: election_strategy: 1 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: fsmap 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: mgrmap e13: x(active, since 37s) 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:32:48.989 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: mon.a calling monitor election 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: mon.c calling monitor election 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: monmap epoch 2 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: last_changed 2024-09-05T06:32:43.670863+0000 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: min_mon_release 19 (squid) 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: election_strategy: 1 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: fsmap 2024-09-05T06:32:49.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: mgrmap e13: x(active, since 37s) 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:32:49.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:32:49.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.207+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3dc418eba0 con 0x7f3db0005740 2024-09-05T06:32:49.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.207+0000 7f3dbb7fe640 1 -- 172.21.15.115:0/1043414243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3dc002dc40 con 0x7f3db0005740 2024-09-05T06:32:49.209 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:32:49.209 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","modified":"2024-09-05T06:32:43.670863Z","created":"2024-09-05T06:31:22.792336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T06:32:49.209 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2024-09-05T06:32:49.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 msgr2=0x7f3da803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:49.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 0x7f3da803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009100 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 msgr2=0x7f3dc4194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3dc4194fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ec10 tx=0x7f3dc0037630 comp rx=0 tx=0).stop 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 shutdown_connections 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3da8038e50 0x7f3da803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dc4195510 0x7f3dc41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 --2- 172.21.15.115:0/1043414243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db0005740 0x7f3dc4194fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 >> 172.21.15.115:0/1043414243 conn(0x7f3dc40fb9a0 msgr2=0x7f3dc40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 shutdown_connections 2024-09-05T06:32:49.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:49.210+0000 7f3dcc6e6640 1 -- 172.21.15.115:0/1043414243 wait complete. 2024-09-05T06:32:50.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:49 smithi115 ceph-mon[34791]: Deploying daemon mon.b on smithi096 2024-09-05T06:32:50.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:49 smithi115 ceph-mon[34791]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:50.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:49 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/1043414243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:50.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:50.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:49 smithi086 ceph-mon[30943]: Deploying daemon mon.b on smithi096 2024-09-05T06:32:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:49 smithi086 ceph-mon[30943]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:49 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/1043414243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:50.780 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T06:32:50.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T06:32:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.443+0000 7f34c0384640 1 -- 172.21.15.115:0/2541089600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b8106630 msgr2=0x7f34b8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.443+0000 7f34c0384640 1 --2- 172.21.15.115:0/2541089600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b8106630 0x7f34b8106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009880 tx=0x7f34b402a5d0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- 172.21.15.115:0/2541089600 shutdown_connections 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 --2- 172.21.15.115:0/2541089600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b8106630 0x7f34b8106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- 172.21.15.115:0/2541089600 >> 172.21.15.115:0/2541089600 conn(0x7f34b80fc220 msgr2=0x7f34b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- 172.21.15.115:0/2541089600 shutdown_connections 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- 172.21.15.115:0/2541089600 wait complete. 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 Processor -- start 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- start start 2024-09-05T06:32:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 0x7f34b8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34b810ab10 con 0x7f34b81985a0 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.444+0000 7f34c0384640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34b810a990 con 0x7f34b8106630 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52328/0 (socket says 172.21.15.115:52328) 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 -- 172.21.15.115:0/1433729109 learned_addr learned my addr 172.21.15.115:0/1433729109 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34bd91f640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 0x7f34b8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 -- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 msgr2=0x7f34b8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 -- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 msgr2=0x7f34b8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34be120640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34bd91f640 1 -- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 msgr2=0x7f34b8198060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34bd91f640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34bd91f640 1 -- 172.21.15.115:0/1433729109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b81a10d0 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.445+0000 7f34bd91f640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 0x7f34b8198a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b400 tx=0x7f34a800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34a8011630 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b81a12a0 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34a8011c80 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34a8010d10 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b81a1800 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f34a8018020 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.446+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b8101ca0 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.447+0000 7f34af7fe640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 0x7f34a403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.447+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f34a804bc90 con 0x7f34b81985a0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.447+0000 7f34be120640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 0x7f34a403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:51.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.447+0000 7f34be120640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 0x7f34a403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34b402abc0 tx=0x7f34b402aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:51.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.452+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34a801f020 con 0x7f34b81985a0 2024-09-05T06:32:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.682+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34b81a1b40 con 0x7f34b81985a0 2024-09-05T06:32:51.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.682+0000 7f34af7fe640 1 -- 172.21.15.115:0/1433729109 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f34a8018310 con 0x7f34b81985a0 2024-09-05T06:32:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:32:51.683 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","modified":"2024-09-05T06:32:43.670863Z","created":"2024-09-05T06:31:22.792336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T06:32:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2024-09-05T06:32:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 msgr2=0x7f34a403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 0x7f34a403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34b402abc0 tx=0x7f34b402aad0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 msgr2=0x7f34b8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 0x7f34b8198a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b400 tx=0x7f34a800b8c0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 shutdown_connections 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f34a4038e00 0x7f34a403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b81985a0 0x7f34b8198a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 --2- 172.21.15.115:0/1433729109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34b8106630 0x7f34b8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 >> 172.21.15.115:0/1433729109 conn(0x7f34b80fc220 msgr2=0x7f34b80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 shutdown_connections 2024-09-05T06:32:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:51.684+0000 7f34c0384640 1 -- 172.21.15.115:0/1433729109 wait complete. 2024-09-05T06:32:52.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:52 smithi115 ceph-mon[34791]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:52 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/1433729109' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:52.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:52 smithi086 ceph-mon[30943]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:52 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/1433729109' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:52 smithi096 systemd[1]: Started Ceph mon.b for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:32:53.235 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T06:32:53.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T06:32:53.480 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:32:57.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 -- 172.21.15.115:0/836306981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778003530 msgr2=0x7fd7780059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:57.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 --2- 172.21.15.115:0/836306981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778003530 0x7fd7780059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7901016a0 tx=0x7fd78002c3f0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 -- 172.21.15.115:0/836306981 shutdown_connections 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 --2- 172.21.15.115:0/836306981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778003530 0x7fd7780059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 -- 172.21.15.115:0/836306981 >> 172.21.15.115:0/836306981 conn(0x7fd7900fc220 msgr2=0x7fd7900fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 -- 172.21.15.115:0/836306981 shutdown_connections 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.630+0000 7fd794ae7640 1 -- 172.21.15.115:0/836306981 wait complete. 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.631+0000 7fd794ae7640 1 Processor -- start 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.631+0000 7fd794ae7640 1 -- start start 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.631+0000 7fd794ae7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.631+0000 7fd794ae7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.631+0000 7fd794ae7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 0x7fd79019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:57.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd794ae7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd790108520 con 0x7fd79019e270 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd794ae7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901083a0 con 0x7fd7901999a0 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd794ae7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901086a0 con 0x7fd778003530 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52334/0 (socket says 172.21.15.115:52334) 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 -- 172.21.15.115:0/20597254 learned_addr learned my addr 172.21.15.115:0/20597254 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 0x7fd79019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 msgr2=0x7fd790199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 msgr2=0x7fd790199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T06:32:57.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T06:32:57.634 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: mon.a calling monitor election 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: mon.c calling monitor election 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T06:32:57.635 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: monmap epoch 3 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: min_mon_release 19 (squid) 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: election_strategy: 1 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: fsmap 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: mgrmap e13: x(active, since 46s) 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.636 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.637 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78dd9b640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:57.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78e59c640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:57.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78dd9b640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 msgr2=0x7fd79019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78dd9b640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 msgr2=0x7fd79019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78dd9b640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78dd9b640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 msgr2=0x7fd790199460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 msgr2=0x7fd79019dd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 -- 172.21.15.115:0/20597254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7901a5fe0 con 0x7fd79019e270 2024-09-05T06:32:57.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.632+0000 7fd78ed9d640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 0x7fd79019e720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd78800bd50 tx=0x7fd78800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.633+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788019070 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.633+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7880148f0 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.633+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7901a6210 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.633+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788012910 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.633+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a67a0 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.634+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd78801c070 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.634+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7901095d0 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.634+0000 7fd77f7fe640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 0x7fd77003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.634+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd78804be00 con 0x7fd79019e270 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.634+0000 7fd78e59c640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 0x7fd77003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:57.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.635+0000 7fd78e59c640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 0x7fd77003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd78002c960 tx=0x7fd78002c8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:57.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.639+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd788007290 con 0x7fd79019e270 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: mon.a calling monitor election 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: mon.c calling monitor election 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: monmap epoch 3 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: election_strategy: 1 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:32:57.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: fsmap 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: mgrmap e13: x(active, since 46s) 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:57.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.884+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd79002d7c0 con 0x7fd79019e270 2024-09-05T06:32:57.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.885+0000 7fd77f7fe640 1 -- 172.21.15.115:0/20597254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fd78804e020 con 0x7fd79019e270 2024-09-05T06:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":3,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","modified":"2024-09-05T06:32:52.319130Z","created":"2024-09-05T06:31:22.792336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T06:32:57.885 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 3 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 msgr2=0x7fd77003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 0x7fd77003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd78002c960 tx=0x7fd78002c8f0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 msgr2=0x7fd79019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 0x7fd79019e720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd78800bd50 tx=0x7fd78800be40 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 shutdown_connections 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79019e270 0x7fd79019e720 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7700394f0 0x7fd77003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7901999a0 0x7fd79019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 --2- 172.21.15.115:0/20597254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd778003530 0x7fd790199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:57.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 >> 172.21.15.115:0/20597254 conn(0x7fd7900fc220 msgr2=0x7fd79006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:57.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 shutdown_connections 2024-09-05T06:32:57.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:32:57.887+0000 7fd794ae7640 1 -- 172.21.15.115:0/20597254 wait complete. 2024-09-05T06:32:58.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:58 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/20597254' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:32:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:58 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/20597254' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:32:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.237 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-05T06:32:59.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph config generate-minimal-conf 2024-09-05T06:32:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: mon.a calling monitor election 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: mon.c calling monitor election 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: monmap epoch 3 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: min_mon_release 19 (squid) 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: election_strategy: 1 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:32:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: fsmap 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: mgrmap e13: x(active, since 46s) 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: overall HEALTH_OK 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/20597254' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T06:32:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:32:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:32:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 -- 172.21.15.86:0/134023524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641027d0 msgr2=0x7fc264102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 --2- 172.21.15.86:0/134023524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641027d0 0x7fc264102bb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc260009810 tx=0x7fc26002a540 comp rx=0 tx=0).stop 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 -- 172.21.15.86:0/134023524 shutdown_connections 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 --2- 172.21.15.86:0/134023524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641027d0 0x7fc264102bb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 -- 172.21.15.86:0/134023524 >> 172.21.15.86:0/134023524 conn(0x7fc264077310 msgr2=0x7fc2640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 -- 172.21.15.86:0/134023524 shutdown_connections 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.969+0000 7fc26c793640 1 -- 172.21.15.86:0/134023524 wait complete. 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 Processor -- start 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 -- start start 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2641027d0 0x7fc264100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 0x7fc264100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410bae0 con 0x7fc2641a3500 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410b960 con 0x7fc2641027d0 2024-09-05T06:32:59.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26c793640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410bc60 con 0x7fc264100970 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39468/0 (socket says 172.21.15.86:39468) 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 -- 172.21.15.86:0/2744759951 learned_addr learned my addr 172.21.15.86:0/2744759951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26a52f640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2641027d0 0x7fc264100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc269d2e640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 0x7fc264100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 -- 172.21.15.86:0/2744759951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 msgr2=0x7fc264100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 0x7fc264100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 -- 172.21.15.86:0/2744759951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2641027d0 msgr2=0x7fc264100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.970+0000 7fc26ad30640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2641027d0 0x7fc264100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc26ad30640 1 -- 172.21.15.86:0/2744759951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641a5fc0 con 0x7fc2641a3500 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc269d2e640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 0x7fc264100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc26ad30640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ba50 tx=0x7fc25c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c019070 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc25c00cbe0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2641a62b0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0075d0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.971+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641a67a0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc25c01c070 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc25b7fe640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 0x7fc25003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc25c04bde0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc26a52f640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 0x7fc25003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc264069ef0 con 0x7fc2641a3500 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.972+0000 7fc26a52f640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 0x7fc25003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc260005e60 tx=0x7fc260005870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:32:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:32:59.977+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc25c0112a0 con 0x7fc2641a3500 2024-09-05T06:33:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.157+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc264101a70 con 0x7fc2641a3500 2024-09-05T06:33:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.157+0000 7fc25b7fe640 1 -- 172.21.15.86:0/2744759951 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fc25c00cd90 con 0x7fc2641a3500 2024-09-05T06:33:00.158 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:33:00.158 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-09-05T06:33:00.158 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:33:00.158 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 msgr2=0x7fc25003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 0x7fc25003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc260005e60 tx=0x7fc260005870 comp rx=0 tx=0).stop 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 msgr2=0x7fc2641a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ba50 tx=0x7fc25c00bf10 comp rx=0 tx=0).stop 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 shutdown_connections 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2641a3500 0x7fc2641a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc264100970 0x7fc264100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc2500394f0 0x7fc25003b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 --2- 172.21.15.86:0/2744759951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2641027d0 0x7fc264100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:00.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 >> 172.21.15.86:0/2744759951 conn(0x7fc264077310 msgr2=0x7fc264103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:00.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 shutdown_connections 2024-09-05T06:33:00.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:00.160+0000 7fc26c793640 1 -- 172.21.15.86:0/2744759951 wait complete. 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mon.b calling monitor election 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mon.b calling monitor election 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mon.c calling monitor election 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mon.a calling monitor election 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T06:33:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: monmap epoch 3 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: election_strategy: 1 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: fsmap 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: mgrmap e13: x(active, since 48s) 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:00 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2744759951' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mon.b calling monitor election 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mon.b calling monitor election 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mon.c calling monitor election 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mon.a calling monitor election 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: monmap epoch 3 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:33:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: min_mon_release 19 (squid) 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: election_strategy: 1 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: fsmap 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: mgrmap e13: x(active, since 48s) 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: overall HEALTH_OK 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:00.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:00.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:00 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2744759951' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mon.b calling monitor election 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mon.b calling monitor election 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mon.c calling monitor election 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mon.a calling monitor election 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: monmap epoch 3 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: last_changed 2024-09-05T06:32:52.319130+0000 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: created 2024-09-05T06:31:22.792336+0000 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: min_mon_release 19 (squid) 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: election_strategy: 1 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: 1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-05T06:33:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: fsmap 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: mgrmap e13: x(active, since 48s) 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:00 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2744759951' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:00.802 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-05T06:33:00.802 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:33:00.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T06:33:00.829 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:33:00.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:33:00.894 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:33:00.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T06:33:00.929 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:33:00.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:33:00.997 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:33:00.998 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T06:33:01.029 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:33:01.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:33:01.098 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-09-05T06:33:01.098 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-09-05T06:33:01.098 INFO:tasks.cephadm:Adding mgr.z on smithi115 2024-09-05T06:33:01.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch apply mgr '3;smithi086=x;smithi096=y;smithi115=z' 2024-09-05T06:33:01.393 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:33:01.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:01 smithi115 ceph-mon[34791]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T06:33:01.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:01 smithi115 ceph-mon[34791]: Reconfiguring daemon mon.a on smithi086 2024-09-05T06:33:01.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:01 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:33:01.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:01 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:01.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:01 smithi086 ceph-mon[30943]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T06:33:01.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:01 smithi086 ceph-mon[30943]: Reconfiguring daemon mon.a on smithi086 2024-09-05T06:33:01.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:01 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:33:01.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:01 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:01 smithi096 ceph-mon[34468]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T06:33:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:01 smithi096 ceph-mon[34468]: Reconfiguring daemon mon.a on smithi086 2024-09-05T06:33:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:01 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:33:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:01 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:01.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 -- 172.21.15.115:0/2037414526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c01005f0 msgr2=0x7fd5c01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:01.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2037414526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c01005f0 0x7fd5c01009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009880 tx=0x7fd5bc02a5d0 comp rx=0 tx=0).stop 2024-09-05T06:33:01.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 -- 172.21.15.115:0/2037414526 shutdown_connections 2024-09-05T06:33:01.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2037414526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c01005f0 0x7fd5c01009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:01.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 -- 172.21.15.115:0/2037414526 >> 172.21.15.115:0/2037414526 conn(0x7fd5c00fbda0 msgr2=0x7fd5c00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 -- 172.21.15.115:0/2037414526 shutdown_connections 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.846+0000 7fd5c8759640 1 -- 172.21.15.115:0/2037414526 wait complete. 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 Processor -- start 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 -- start start 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c01005f0 0x7fd5c0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c0197480 0x7fd5c019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01076b0 con 0x7fd5c0197480 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0107530 con 0x7fd5c01005f0 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c8759640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0107830 con 0x7fd5c0197900 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c6cf6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c6cf6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47678/0 (socket says 172.21.15.115:47678) 2024-09-05T06:33:01.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c6cf6640 1 -- 172.21.15.115:0/2475558701 learned_addr learned my addr 172.21.15.115:0/2475558701 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c64f5640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c01005f0 0x7fd5c0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.847+0000 7fd5c5cf4640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c0197480 0x7fd5c019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 -- 172.21.15.115:0/2475558701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c01005f0 msgr2=0x7fd5c0196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c01005f0 0x7fd5c0196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 -- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c0197480 msgr2=0x7fd5c019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c0197480 0x7fd5c019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 -- 172.21.15.115:0/2475558701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c019f340 con 0x7fd5c0197900 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c6cf6640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ba50 tx=0x7fd5b800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8019070 con 0x7fd5c0197900 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c019f630 con 0x7fd5c0197900 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b800cbe0 con 0x7fd5c0197900 2024-09-05T06:33:01.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80075d0 con 0x7fd5c0197900 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.848+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c019fba0 con 0x7fd5c0197900 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.849+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd5b801c070 con 0x7fd5c0197900 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.849+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c0190470 con 0x7fd5c0197900 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.849+0000 7fd5b37fe640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 0x7fd5ac03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.849+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd5b804ba60 con 0x7fd5c0197900 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.849+0000 7fd5c64f5640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 0x7fd5ac03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.850+0000 7fd5c64f5640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 0x7fd5ac03b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc02ab40 tx=0x7fd5bc02aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:01.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:01.854+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5b80133a0 con 0x7fd5c0197900 2024-09-05T06:33:02.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.033+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi096=y;smithi115=z", "target": ["mon-mgr", ""]}) -- 0x7fd5c00626c0 con 0x7fd5ac0394f0 2024-09-05T06:33:02.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.038+0000 7fd5b37fe640 1 -- 172.21.15.115:0/2475558701 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd5c00626c0 con 0x7fd5ac0394f0 2024-09-05T06:33:02.038 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mgr update... 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 msgr2=0x7fd5ac03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 0x7fd5ac03b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc02ab40 tx=0x7fd5bc02aad0 comp rx=0 tx=0).stop 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 msgr2=0x7fd5c019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ba50 tx=0x7fd5b800bf10 comp rx=0 tx=0).stop 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 shutdown_connections 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c0197900 0x7fd5c019ea80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c0197480 0x7fd5c019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5ac0394f0 0x7fd5ac03b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:02.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 --2- 172.21.15.115:0/2475558701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c01005f0 0x7fd5c0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:02.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 >> 172.21.15.115:0/2475558701 conn(0x7fd5c00fbda0 msgr2=0x7fd5c00fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:02.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 shutdown_connections 2024-09-05T06:33:02.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:02.040+0000 7fd5c8759640 1 -- 172.21.15.115:0/2475558701 wait complete. 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.437 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:02.437 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.438 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.438 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:02.438 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:02.438 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:02.438 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.569 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.y.service 2024-09-05T06:33:02.571 DEBUG:teuthology.orchestra.run.smithi115:mgr.z> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.z.service 2024-09-05T06:33:02.574 INFO:tasks.cephadm:Deploying OSDs... 2024-09-05T06:33:02.574 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:33:02.574 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T06:33:02.594 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T06:33:02.595 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-09-05T06:33:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T06:33:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:02.653 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-05T06:33:02.653 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:02.653 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:02.654 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 06:32:19.647318210 +0000 2024-09-05T06:33:02.654 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 06:28:20.974043890 +0000 2024-09-05T06:33:02.654 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 06:28:20.974043890 +0000 2024-09-05T06:33:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 06:28:20.974043890 +0000 2024-09-05T06:33:02.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T06:33:02.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:02.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:02.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:02.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:02.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T06:33:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T06:33:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000292619 s, 1.7 MB/s 2024-09-05T06:33:02.724 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T06:33:02.781 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-09-05T06:33:02.838 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T06:33:02.838 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 06:32:19.672317664 +0000 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 06:28:21.310036456 +0000 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 06:28:21.310036456 +0000 2024-09-05T06:33:02.839 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 06:28:21.310036456 +0000 2024-09-05T06:33:02.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T06:33:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T06:33:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T06:33:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000208895 s, 2.5 MB/s 2024-09-05T06:33:02.905 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T06:33:02.962 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 06:32:19.700317053 +0000 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 06:28:21.637029221 +0000 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 06:28:21.637029221 +0000 2024-09-05T06:33:03.020 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 06:28:21.637029221 +0000 2024-09-05T06:33:03.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T06:33:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T06:33:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T06:33:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000209683 s, 2.4 MB/s 2024-09-05T06:33:03.085 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T06:33:03.142 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 06:32:19.727316464 +0000 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 06:28:21.979021655 +0000 2024-09-05T06:33:03.197 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 06:28:21.979021655 +0000 2024-09-05T06:33:03.198 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 06:28:21.979021655 +0000 2024-09-05T06:33:03.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T06:33:03.261 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T06:33:03.261 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T06:33:03.261 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000247779 s, 2.1 MB/s 2024-09-05T06:33:03.262 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T06:33:03.319 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-05T06:33:03.319 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T06:33:03.336 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T06:33:03.336 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-05T06:33:03.392 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T06:33:03.392 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-05 06:32:51.795054355 +0000 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-05 06:28:24.139544821 +0000 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-05 06:28:24.139544821 +0000 2024-09-05T06:33:03.393 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-05 06:28:24.139544821 +0000 2024-09-05T06:33:03.393 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T06:33:03.455 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: Reconfiguring mon.b (monmap changed)... 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: Reconfiguring daemon mon.b on smithi096 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi096=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: Saving service mgr spec with placement smithi086=x;smithi096=y;smithi115=z;count:3 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:03.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:03.457 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-05T06:33:03.458 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-05T06:33:03.458 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000265271 s, 1.9 MB/s 2024-09-05T06:33:03.459 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T06:33:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: Reconfiguring mon.b (monmap changed)... 2024-09-05T06:33:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: Reconfiguring daemon mon.b on smithi096 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi096=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: Saving service mgr spec with placement smithi086=x;smithi096=y;smithi115=z;count:3 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:03.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: Reconfiguring mon.b (monmap changed)... 2024-09-05T06:33:03.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: Reconfiguring daemon mon.b on smithi096 2024-09-05T06:33:03.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi096=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:03.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: Saving service mgr spec with placement smithi086=x;smithi096=y;smithi115=z;count:3 2024-09-05T06:33:03.512 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.512 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:03.512 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T06:33:03.512 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T06:33:03.512 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:03.515 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-05T06:33:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T06:33:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.570 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-05 06:32:51.796054332 +0000 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-05 06:28:24.460537053 +0000 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-05 06:28:24.460537053 +0000 2024-09-05T06:33:03.571 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-05 06:28:24.460537053 +0000 2024-09-05T06:33:03.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T06:33:03.635 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-05T06:33:03.635 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-05T06:33:03.635 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000195964 s, 2.6 MB/s 2024-09-05T06:33:03.636 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T06:33:03.692 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-05T06:33:03.748 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T06:33:03.748 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.748 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-05T06:33:03.748 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-05 06:32:51.796054332 +0000 2024-09-05T06:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-05 06:28:24.782529261 +0000 2024-09-05T06:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-05 06:28:24.782529261 +0000 2024-09-05T06:33:03.749 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-05 06:28:24.782529261 +0000 2024-09-05T06:33:03.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T06:33:03.811 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-05T06:33:03.811 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-05T06:33:03.811 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000253598 s, 2.0 MB/s 2024-09-05T06:33:03.813 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T06:33:03.869 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-05T06:33:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-05 06:32:51.796054332 +0000 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-05 06:28:25.120521081 +0000 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-05 06:28:25.120521081 +0000 2024-09-05T06:33:03.926 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-05 06:28:25.120521081 +0000 2024-09-05T06:33:03.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T06:33:03.989 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-05T06:33:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-05T06:33:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000219438 s, 2.3 MB/s 2024-09-05T06:33:03.991 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T06:33:04.047 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-05T06:33:04.047 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T06:33:04.065 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T06:33:04.065 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-05T06:33:04.121 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T06:33:04.121 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:04.121 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-05T06:33:04.121 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:04.121 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:04.122 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-05 06:32:43.092013162 +0000 2024-09-05T06:33:04.122 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-05 06:28:19.558618554 +0000 2024-09-05T06:33:04.122 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-05 06:28:19.558618554 +0000 2024-09-05T06:33:04.122 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-05 06:28:19.558618554 +0000 2024-09-05T06:33:04.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T06:33:04.186 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-05T06:33:04.187 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-05T06:33:04.187 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000195785 s, 2.6 MB/s 2024-09-05T06:33:04.188 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T06:33:04.245 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-05T06:33:04.300 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T06:33:04.300 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:04.300 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T06:33:04.300 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:04.301 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:04.301 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-05 06:32:43.092013162 +0000 2024-09-05T06:33:04.301 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-05 06:28:19.891612166 +0000 2024-09-05T06:33:04.301 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-05 06:28:19.891612166 +0000 2024-09-05T06:33:04.301 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-05 06:28:19.891612166 +0000 2024-09-05T06:33:04.301 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T06:33:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-05T06:33:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-05T06:33:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000198715 s, 2.6 MB/s 2024-09-05T06:33:04.367 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T06:33:04.424 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-05 06:32:43.092013162 +0000 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-05 06:28:20.202606201 +0000 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-05 06:28:20.202606201 +0000 2024-09-05T06:33:04.480 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-05 06:28:20.202606201 +0000 2024-09-05T06:33:04.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T06:33:04.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: Reconfiguring mon.c (monmap changed)... 2024-09-05T06:33:04.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: Reconfiguring daemon mon.c on smithi115 2024-09-05T06:33:04.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:04.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:04.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:04.543 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.543 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:04.543 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:04.543 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:04.543 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.545 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-05T06:33:04.545 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-05T06:33:04.545 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000197555 s, 2.6 MB/s 2024-09-05T06:33:04.546 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T06:33:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: Reconfiguring mon.c (monmap changed)... 2024-09-05T06:33:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: Reconfiguring daemon mon.c on smithi115 2024-09-05T06:33:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:04.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:04.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:04.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.601 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-05 06:32:43.093013143 +0000 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-05 06:28:20.534599832 +0000 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-05 06:28:20.534599832 +0000 2024-09-05T06:33:04.656 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-05 06:28:20.534599832 +0000 2024-09-05T06:33:04.657 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T06:33:04.721 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-05T06:33:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-05T06:33:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000204851 s, 2.5 MB/s 2024-09-05T06:33:04.723 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: Reconfiguring mon.c (monmap changed)... 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: Reconfiguring daemon mon.c on smithi115 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:04.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:04.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:04.779 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-09-05T06:33:04.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T06:33:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:33:05.402 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:05 smithi086 ceph-mon[30943]: Deploying daemon mgr.z on smithi115 2024-09-05T06:33:05.537 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:05 smithi115 ceph-mon[34791]: Deploying daemon mgr.z on smithi115 2024-09-05T06:33:05.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:05 smithi096 ceph-mon[34468]: Deploying daemon mgr.z on smithi115 2024-09-05T06:33:06.183 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:05 smithi115 systemd[1]: Starting Ceph mgr.z for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:33:06.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:06 smithi086 ceph-mon[30943]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-mon[34791]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:06.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 podman[36581]: 2024-09-05 06:33:06.089001355 +0000 UTC m=+0.015875228 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:33:06.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 podman[36581]: 2024-09-05 06:33:06.19001536 +0000 UTC m=+0.116889213 container create dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T06:33:06.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 podman[36581]: 2024-09-05 06:33:06.409137636 +0000 UTC m=+0.336011490 container init dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T06:33:06.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 podman[36581]: 2024-09-05 06:33:06.41231141 +0000 UTC m=+0.339185280 container start dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-05T06:33:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:06 smithi096 ceph-mon[34468]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-mgr[36601]: -- 172.21.15.115:0/4206779772 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c980f30e0 con 0x562c980d0800 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 bash[36581]: dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 systemd[1]: Started Ceph mgr.z for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:06.542+0000 7f372659a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:06.599+0000 7f372659a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:33:06.745 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:06.744+0000 7f372659a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:33:07.097 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:06.811+0000 7f372659a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:33:07.098 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:06.957+0000 7f372659a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:33:07.098 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:07.097+0000 7f372659a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:33:07.369 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:33:07.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-09-05T06:33:07.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: Deploying daemon mgr.y on smithi096 2024-09-05T06:33:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:07 smithi096 ceph-mon[34468]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:07.636 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:33:07.682 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:07.411+0000 7f372659a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:33:07.682 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:07.471+0000 7f372659a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:07.655+0000 7f372659a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: Deploying daemon mgr.y on smithi096 2024-09-05T06:33:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:07 smithi115 ceph-mon[34791]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:07.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:07.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:07.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T06:33:07.871 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:07.871 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:07.871 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: Deploying daemon mgr.y on smithi096 2024-09-05T06:33:07.871 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:07 smithi086 ceph-mon[30943]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:08.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.076+0000 7f958a279640 1 -- 172.21.15.86:0/2427199611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584105030 msgr2=0x7f9584107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.076+0000 7f958a279640 1 --2- 172.21.15.86:0/2427199611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584105030 0x7f9584107420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9578009900 tx=0x7f957802a410 comp rx=0 tx=0).stop 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 -- 172.21.15.86:0/2427199611 shutdown_connections 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 --2- 172.21.15.86:0/2427199611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f9584109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 --2- 172.21.15.86:0/2427199611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584105030 0x7f9584107420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 --2- 172.21.15.86:0/2427199611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584102700 0x7f9584104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 -- 172.21.15.86:0/2427199611 >> 172.21.15.86:0/2427199611 conn(0x7f95840fc280 msgr2=0x7f95840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 -- 172.21.15.86:0/2427199611 shutdown_connections 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 -- 172.21.15.86:0/2427199611 wait complete. 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 Processor -- start 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.077+0000 7f958a279640 1 -- start start 2024-09-05T06:33:08.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584105030 0x7f9584198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f958419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f958410d700 con 0x7f9584102700 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f958410d580 con 0x7f9584105030 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f958410d880 con 0x7f9584107960 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37100/0 (socket says 172.21.15.86:37100) 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 -- 172.21.15.86:0/3318422943 learned_addr learned my addr 172.21.15.86:0/3318422943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9588816640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f958419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f95837fe640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584105030 0x7f9584198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 -- 172.21.15.86:0/3318422943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 msgr2=0x7f958419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f958419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:08.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 -- 172.21.15.86:0/3318422943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584105030 msgr2=0x7f9584198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584105030 0x7f9584198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 -- 172.21.15.86:0/3318422943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958419fa70 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9588816640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f958419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f9583fff640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f957400d950 tx=0x7f957400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574017070 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9574009c30 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958419fd60 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.078+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957400f620 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.079+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95841a0200 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.079+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f957400f7c0 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.079+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95840fdd70 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.080+0000 7f95817fa640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 0x7f956c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.080+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f957404bfa0 con 0x7f9584102700 2024-09-05T06:33:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.080+0000 7f95837fe640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 0x7f956c03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:08.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.080+0000 7f95837fe640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 0x7f956c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9578004770 tx=0x7f9578000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:08.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.084+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9574009de0 con 0x7f9584102700 2024-09-05T06:33:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:08.258+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9584062630 con 0x7f956c0392c0 2024-09-05T06:33:08.486 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 systemd[1]: Starting Ceph mgr.y for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:33:08.737 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-mon[34468]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:08.737 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:08.737 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:08.737 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:08.738 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 podman[35814]: 2024-09-05 06:33:08.388826648 +0000 UTC m=+0.015571675 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:33:08.738 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 podman[35814]: 2024-09-05 06:33:08.49122765 +0000 UTC m=+0.117972705 container create fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-05T06:33:08.738 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 podman[35814]: 2024-09-05 06:33:08.735286194 +0000 UTC m=+0.362031215 container init fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T06:33:08.738 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 podman[35814]: 2024-09-05 06:33:08.738150426 +0000 UTC m=+0.364895439 container start fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T06:33:08.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:08 smithi086 ceph-mon[30943]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:08.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:08 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:08.855 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:08 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:08.856 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:08 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:08.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-mon[34791]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:08.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:08.933 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:08.533+0000 7f372659a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:33:08.933 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:08.601+0000 7f372659a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:33:09.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-mgr[35833]: -- 172.21.15.96:0/771735814 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fce86450e0 con 0x55fce8623000 2024-09-05T06:33:09.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 bash[35814]: fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 2024-09-05T06:33:09.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 systemd[1]: Started Ceph mgr.y for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:33:09.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:08.860+0000 7f7427439040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:33:09.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:08 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:08.922+0000 7f7427439040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:33:09.457 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.079+0000 7f7427439040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:33:09.457 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.148+0000 7f7427439040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:33:09.457 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.305+0000 7f7427439040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:33:09.458 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.456+0000 7f7427439040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:33:09.630 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.347+0000 7f372659a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:33:09.630 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.485+0000 7f372659a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:33:09.630 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.630+0000 7f372659a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:33:09.906 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.696+0000 7f372659a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:33:09.907 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.845+0000 7f372659a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:33:10.039 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.779+0000 7f7427439040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:33:10.039 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:09.840+0000 7f7427439040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:33:10.039 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.039 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:10.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:09 smithi096 ceph-mon[34468]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:10.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-mon[34791]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.914+0000 7f372659a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:09.983+0000 7f372659a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:33:10.183 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.145+0000 7f372659a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:09 smithi086 ceph-mon[30943]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:10.306 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:10.038+0000 7f7427439040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:33:10.477 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.210+0000 7f372659a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:33:10.477 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.276+0000 7f372659a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:33:10.477 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.359+0000 7f372659a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:33:10.477 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.418+0000 7f372659a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:33:10.477 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:10.477+0000 7f372659a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: Reconfiguring daemon mgr.x on smithi086 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]: dispatch 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]': finished 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T06:33:11.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:11.270 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.271 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: Reconfiguring daemon mgr.x on smithi086 2024-09-05T06:33:11.272 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]: dispatch 2024-09-05T06:33:11.272 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]': finished 2024-09-05T06:33:11.272 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T06:33:11.272 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:10.919+0000 7f7427439040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:10.987+0000 7f7427439040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: Reconfiguring daemon mgr.x on smithi086 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]: dispatch 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/864356132' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f"}]': finished 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T06:33:11.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:11.682 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:33:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:33:11.270+0000 7f372659a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:33:11.953 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:11.698+0000 7f7427439040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:33:11.954 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:11.813+0000 7f7427439040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:33:11.954 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:11.953+0000 7f7427439040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:33:12.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:12.132+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f9574023870 con 0x7f9584102700 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.015+0000 7f7427439040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.164+0000 7f7427439040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.232+0000 7f7427439040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2699689044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: Standby manager daemon z started 2024-09-05T06:33:12.233 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:33:12.234 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:12.234 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:33:12.234 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:12.428 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:12.428 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2699689044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: Standby manager daemon z started 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:33:12.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:12 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2699689044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: Standby manager daemon z started 2024-09-05T06:33:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:33:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:33:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:12 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/3728828683' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:12.511 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.291+0000 7f7427439040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:33:12.511 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.446+0000 7f7427439040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:33:12.512 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.511+0000 7f7427439040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:33:12.776 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.576+0000 7f7427439040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:33:12.776 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.658+0000 7f7427439040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:33:12.776 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.717+0000 7f7427439040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:33:12.776 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:12.775+0000 7f7427439040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:33:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:13 smithi115 ceph-mon[34791]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-05T06:33:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:33:13.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:13 smithi086 ceph-mon[30943]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-05T06:33:13.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:33:13.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:13 smithi096 ceph-mon[34468]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-05T06:33:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:33:14.057 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:33:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:33:13.557+0000 7f7427439040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:33:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:14.144+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9574023350 con 0x7f9584102700 2024-09-05T06:33:14.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: Standby manager daemon y started 2024-09-05T06:33:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:33:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:33:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:14 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:14.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: Standby manager daemon y started 2024-09-05T06:33:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:33:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:33:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:14 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: Standby manager daemon y started 2024-09-05T06:33:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:33:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:33:14.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:33:14.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:14 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3184578249' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:33:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:15 smithi086 ceph-mon[30943]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-05T06:33:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:33:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:15 smithi115 ceph-mon[34791]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-05T06:33:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:33:15.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:15 smithi096 ceph-mon[34468]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-05T06:33:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:33:16.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:16 smithi086 ceph-mon[30943]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:16.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:16 smithi115 ceph-mon[34791]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:16 smithi096 ceph-mon[34468]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:18.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:18 smithi086 ceph-mon[30943]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:18.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:18 smithi115 ceph-mon[34791]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:18 smithi096 ceph-mon[34468]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:19.459 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.460 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:19.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:19.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:19.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.615 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:19.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:19.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:19.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:19.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:20.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:20 smithi086 ceph-mon[30943]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:20.423 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T06:33:20.423 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:20 smithi096 ceph-mon[34468]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T06:33:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:20.565 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:20 smithi115 ceph-mon[34791]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:20.565 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T06:33:20.565 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:21.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:21 smithi086 ceph-mon[30943]: Deploying daemon osd.0 on smithi086 2024-09-05T06:33:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:21 smithi096 ceph-mon[34468]: Deploying daemon osd.0 on smithi086 2024-09-05T06:33:21.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:21 smithi115 ceph-mon[34791]: Deploying daemon osd.0 on smithi086 2024-09-05T06:33:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:22 smithi086 ceph-mon[30943]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:22.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:22 smithi096 ceph-mon[34468]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:22 smithi115 ceph-mon[34791]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:24.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:24 smithi086 ceph-mon[30943]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:24 smithi096 ceph-mon[34468]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:24 smithi115 ceph-mon[34791]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:25.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:26 smithi086 ceph-mon[30943]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:26.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:26 smithi096 ceph-mon[34468]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:26 smithi115 ceph-mon[34791]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:28 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:28 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:28.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:28 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.097+0000 7f95817fa640 1 -- 172.21.15.86:0/3318422943 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9584062630 con 0x7f956c0392c0 2024-09-05T06:33:29.098 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-09-05T06:33:29.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 msgr2=0x7f956c03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:29.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 0x7f956c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9578004770 tx=0x7f9578000bc0 comp rx=0 tx=0).stop 2024-09-05T06:33:29.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 msgr2=0x7f9584197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f957400d950 tx=0x7f957400de10 comp rx=0 tx=0).stop 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 shutdown_connections 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9584107960 0x7f958419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f956c0392c0 0x7f956c03b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584105030 0x7f9584198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 --2- 172.21.15.86:0/3318422943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584102700 0x7f9584197e40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 >> 172.21.15.86:0/3318422943 conn(0x7f95840fc280 msgr2=0x7f95841030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 shutdown_connections 2024-09-05T06:33:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:33:29.099+0000 7f958a279640 1 -- 172.21.15.86:0/3318422943 wait complete. 2024-09-05T06:33:29.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:29.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:29.781 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.0.service 2024-09-05T06:33:29.784 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-09-05T06:33:29.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T06:33:30.021 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:33:30.421 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:30 smithi096 ceph-mon[34468]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:30.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:30 smithi115 ceph-mon[34791]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:30.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:30 smithi086 ceph-mon[30943]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:31.453 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:31 smithi096 ceph-mon[34468]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T06:33:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:31 smithi086 ceph-mon[30943]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T06:33:31.487 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:33:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T06:33:31.127+0000 7f64adf60640 -1 osd.0 0 log_to_monitors true 2024-09-05T06:33:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:31 smithi115 ceph-mon[34791]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T06:33:32.284 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:33:32.310 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-09-05T06:33:32.478 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T06:33:32.255+0000 7f64a9ee3640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T06:33:32.478 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T06:33:32.258+0000 7f64a5d0b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:33:32.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-mon[30943]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:32.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-mon[30943]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T06:33:32.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-mon[30943]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T06:33:32.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-mon[30943]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:33:32.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:32.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:32 smithi096 ceph-mon[34468]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:32 smithi096 ceph-mon[34468]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T06:33:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:32 smithi096 ceph-mon[34468]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T06:33:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:32 smithi096 ceph-mon[34468]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:33:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:32.564 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:33:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:32 smithi115 ceph-mon[34791]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:32 smithi115 ceph-mon[34791]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T06:33:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:32 smithi115 ceph-mon[34791]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T06:33:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:32 smithi115 ceph-mon[34791]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:33:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- 172.21.15.96:0/2057382688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb490100b00 msgr2=0x7fb490100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 --2- 172.21.15.96:0/2057382688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb490100b00 0x7fb490100f00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb480009900 tx=0x7fb48002a410 comp rx=0 tx=0).stop 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- 172.21.15.96:0/2057382688 shutdown_connections 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 --2- 172.21.15.96:0/2057382688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4901026c0 0x7fb490108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 --2- 172.21.15.96:0/2057382688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 0x7fb490102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 --2- 172.21.15.96:0/2057382688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb490100b00 0x7fb490100f00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- 172.21.15.96:0/2057382688 >> 172.21.15.96:0/2057382688 conn(0x7fb4900fc270 msgr2=0x7fb4900fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- 172.21.15.96:0/2057382688 shutdown_connections 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- 172.21.15.96:0/2057382688 wait complete. 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 Processor -- start 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.995+0000 7fb495fa7640 1 -- start start 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb490100b00 0x7fb49006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 0x7fb49019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010c690 con 0x7fb490101d00 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010c510 con 0x7fb4901026c0 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010c810 con 0x7fb490100b00 2024-09-05T06:33:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34434/0 (socket says 172.21.15.96:34434) 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 -- 172.21.15.96:0/668064105 learned_addr learned my addr 172.21.15.96:0/668064105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb487fff640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 0x7fb49019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48f7fe640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb490100b00 0x7fb49006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 -- 172.21.15.96:0/668064105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb490100b00 msgr2=0x7fb49006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb490100b00 0x7fb49006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 -- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 msgr2=0x7fb49019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 0x7fb49019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 -- 172.21.15.96:0/668064105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49019fbb0 con 0x7fb4901026c0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48ffff640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb48800b970 tx=0x7fb48800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488019070 con 0x7fb4901026c0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49019fe40 con 0x7fb4901026c0 2024-09-05T06:33:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb488014440 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4880075d0 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.996+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901a0420 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.998+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb490108090 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.999+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb48801c070 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.999+0000 7fb48d7fa640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 0x7fb47c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.999+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fb4880cca10 con 0x7fb4901026c0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:32.999+0000 7fb48f7fe640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 0x7fb47c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:33.000+0000 7fb48f7fe640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 0x7fb47c0a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:33.004+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb48809aeb0 con 0x7fb4901026c0 2024-09-05T06:33:33.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:33.184+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb490062630 con 0x7fb47c0a2af0 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:33.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:33.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:33.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:33.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:33.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:33.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:33.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi086 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 17088M 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] boot 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T06:33:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi086 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 17088M 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] boot 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T06:33:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi086 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 17088M 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: osd.0 [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] boot 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T06:33:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/1893511893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]': finished 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T06:33:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T06:33:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/1893511893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]': finished 2024-09-05T06:33:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T06:33:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:35.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/1893511893' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]: dispatch 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2892f95-df5c-4d96-955a-d3d741302bc2"}]': finished 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T06:33:35.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:36.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:36 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/3082441261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:36.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:36 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/3082441261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:36.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:36 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/3082441261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:33:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:37 smithi096 ceph-mon[34468]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:37 smithi115 ceph-mon[34791]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:37.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:37 smithi086 ceph-mon[30943]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:40 smithi096 ceph-mon[34468]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:40.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:40 smithi115 ceph-mon[34791]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:40.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:40 smithi086 ceph-mon[30943]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:42.412 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:42 smithi096 ceph-mon[34468]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:42.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:42 smithi115 ceph-mon[34791]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:42.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:42 smithi086 ceph-mon[30943]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:44.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:44 smithi096 ceph-mon[34468]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:44.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:44 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T06:33:44.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:44 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:44.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:44 smithi115 ceph-mon[34791]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:44.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:44 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T06:33:44.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:44 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:44 smithi086 ceph-mon[30943]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:44 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T06:33:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:44 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:45.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:45 smithi115 ceph-mon[34791]: Deploying daemon osd.1 on smithi096 2024-09-05T06:33:45.435 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:45 smithi096 ceph-mon[34468]: Deploying daemon osd.1 on smithi096 2024-09-05T06:33:45.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:45 smithi086 ceph-mon[30943]: Deploying daemon osd.1 on smithi096 2024-09-05T06:33:46.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:46 smithi096 ceph-mon[34468]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:46.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:46 smithi115 ceph-mon[34791]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:46.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:46 smithi086 ceph-mon[30943]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:48.289 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:48 smithi096 ceph-mon[34468]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:48.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:48 smithi115 ceph-mon[34791]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:48.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:48 smithi086 ceph-mon[30943]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:49.296 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:49.297 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:49.297 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:50.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:50 smithi096 ceph-mon[34468]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:50.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:50 smithi115 ceph-mon[34791]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:50.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:50 smithi086 ceph-mon[30943]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:52.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.108+0000 7fb48d7fa640 1 -- 172.21.15.96:0/668064105 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb490062630 con 0x7fb47c0a2af0 2024-09-05T06:33:52.109 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-09-05T06:33:52.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 msgr2=0x7fb47c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:52.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 0x7fb47c0a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800314a0 comp rx=0 tx=0).stop 2024-09-05T06:33:52.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 msgr2=0x7fb49019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb48800b970 tx=0x7fb48800be30 comp rx=0 tx=0).stop 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 shutdown_connections 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb47c0a2af0 0x7fb47c0a4fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4901026c0 0x7fb49019f2a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490101d00 0x7fb49019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 --2- 172.21.15.96:0/668064105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb490100b00 0x7fb49006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 >> 172.21.15.96:0/668064105 conn(0x7fb4900fc270 msgr2=0x7fb490105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 shutdown_connections 2024-09-05T06:33:52.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:33:52.111+0000 7fb495fa7640 1 -- 172.21.15.96:0/668064105 wait complete. 2024-09-05T06:33:52.333 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.333 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.334 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:52.334 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:52.334 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.334 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:52 smithi096 ceph-mon[34468]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:52 smithi115 ceph-mon[34791]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:52 smithi086 ceph-mon[30943]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:52.723 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.1.service 2024-09-05T06:33:52.726 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_4... 2024-09-05T06:33:52.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T06:33:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:33:53.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:53.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:53.283 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:53.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:53.383 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:53.383 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:53.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:33:53.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:53.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:54.157 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:54 smithi096 ceph-mon[34468]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:54.158 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:33:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T06:33:54.141+0000 7f90501d8640 -1 osd.1 0 log_to_monitors true 2024-09-05T06:33:54.303 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:54 smithi115 ceph-mon[34791]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:54 smithi086 ceph-mon[30943]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:55.384 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:55 smithi115 ceph-mon[34791]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.384 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:55 smithi115 ceph-mon[34791]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.386 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:33:55.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-09-05T06:33:55.442 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:55 smithi096 ceph-mon[34468]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.442 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:55 smithi096 ceph-mon[34468]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:55 smithi086 ceph-mon[30943]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:55 smithi086 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T06:33:55.659 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 -- 172.21.15.115:0/2101874392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b4102ea0 msgr2=0x7f65b4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 --2- 172.21.15.115:0/2101874392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b4102ea0 0x7f65b4103320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f659c009930 tx=0x7f659c02a420 comp rx=0 tx=0).stop 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 -- 172.21.15.115:0/2101874392 shutdown_connections 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 --2- 172.21.15.115:0/2101874392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 0x7f65b410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 --2- 172.21.15.115:0/2101874392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b4102ea0 0x7f65b4103320 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 --2- 172.21.15.115:0/2101874392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b40ffec0 0x7f65b41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:56.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.108+0000 7f65b9cce640 1 -- 172.21.15.115:0/2101874392 >> 172.21.15.115:0/2101874392 conn(0x7f65b40fba10 msgr2=0x7f65b40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- 172.21.15.115:0/2101874392 shutdown_connections 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- 172.21.15.115:0/2101874392 wait complete. 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 Processor -- start 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- start start 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 0x7f65b4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 0x7f65b419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d7a0 con 0x7f65b4103860 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d620 con 0x7f65b4102ea0 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.109+0000 7f65b9cce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65b410d920 con 0x7f65b40ffec0 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40928/0 (socket says 172.21.15.115:40928) 2024-09-05T06:33:56.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 0x7f65b4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 -- 172.21.15.115:0/1866144019 learned_addr learned my addr 172.21.15.115:0/1866144019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b3fff640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 0x7f65b419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 -- 172.21.15.115:0/1866144019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 msgr2=0x7f65b4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 0x7f65b4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 -- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 msgr2=0x7f65b419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 0x7f65b419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 -- 172.21.15.115:0/1866144019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b419fb20 con 0x7f65b40ffec0 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65abfff640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 0x7f65b4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:33:56.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.110+0000 7f65b37fe640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65a400e9c0 tx=0x7f65a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.112+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400cd80 con 0x7f65b40ffec0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.112+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a40035f0 con 0x7f65b40ffec0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.113+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b419fe10 con 0x7f65b40ffec0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.113+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4010690 con 0x7f65b40ffec0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.113+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41a0260 con 0x7f65b40ffec0 2024-09-05T06:33:56.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.114+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4109220 con 0x7f65b40ffec0 2024-09-05T06:33:56.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.115+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f65a4004030 con 0x7f65b40ffec0 2024-09-05T06:33:56.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.115+0000 7f65b17fa640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 0x7f65a00a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:33:56.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.116+0000 7f65abfff640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 0x7f65a00a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:33:56.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.116+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f65a40cdb60 con 0x7f65b40ffec0 2024-09-05T06:33:56.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.116+0000 7f65abfff640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 0x7f65a00a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f659c0314e0 tx=0x7f659c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:33:56.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.120+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65a409c430 con 0x7f65b40ffec0 2024-09-05T06:33:56.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:33:56.300+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f65b4062630 con 0x7f65a00a2ba0 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:56.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:56.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:56.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T06:33:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:56.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:56.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:56.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.557 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T06:33:56.110+0000 7f904c15b640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T06:33:56.557 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T06:33:56.115+0000 7f9047f83640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:33:56.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:33:57.368 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi096 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:57.369 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi096 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi096 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:33:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] boot 2024-09-05T06:33:58.320 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T06:33:58.321 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:58.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] boot 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T06:33:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:58.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: osd.1 [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] boot 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T06:33:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:33:59.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T06:33:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/2192822382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]': finished 2024-09-05T06:33:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T06:33:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:33:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:33:59.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T06:33:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/2192822382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]': finished 2024-09-05T06:33:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T06:33:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:33:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:33:59.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T06:33:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/2192822382' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]: dispatch 2024-09-05T06:33:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec111fa-9256-44e0-b091-da1f005c82d7"}]': finished 2024-09-05T06:33:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T06:33:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:33:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:00.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:00 smithi115 ceph-mon[34791]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:00.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:00 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/154655072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:00.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:00 smithi086 ceph-mon[30943]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:00 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/154655072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:00.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:00 smithi096 ceph-mon[34468]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:00 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/154655072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:02.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:02 smithi096 ceph-mon[34468]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:02.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:02 smithi115 ceph-mon[34791]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:02.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:02 smithi086 ceph-mon[30943]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:04.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:04 smithi115 ceph-mon[34791]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:04 smithi086 ceph-mon[30943]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:04 smithi096 ceph-mon[34468]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:06.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:06 smithi115 ceph-mon[34791]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:06.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:06 smithi086 ceph-mon[30943]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:06 smithi096 ceph-mon[34468]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:07.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T06:34:07.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:07 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:07.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T06:34:07.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:07 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T06:34:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:07 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:08.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:08 smithi115 ceph-mon[34791]: Deploying daemon osd.2 on smithi115 2024-09-05T06:34:08.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:08 smithi115 ceph-mon[34791]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:08.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:08 smithi086 ceph-mon[30943]: Deploying daemon osd.2 on smithi115 2024-09-05T06:34:08.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:08 smithi086 ceph-mon[30943]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:08.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:08 smithi096 ceph-mon[34468]: Deploying daemon osd.2 on smithi115 2024-09-05T06:34:08.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:08 smithi096 ceph-mon[34468]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:10 smithi115 ceph-mon[34791]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:10.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:10 smithi086 ceph-mon[30943]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:10 smithi096 ceph-mon[34468]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:12 smithi115 ceph-mon[34791]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:12.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:12 smithi086 ceph-mon[30943]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:12.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:12 smithi096 ceph-mon[34468]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:12.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:14.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:14.229 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:14.229 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:14.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:14.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:14.561 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.099+0000 7f65b17fa640 1 -- 172.21.15.115:0/1866144019 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65b4062630 con 0x7f65a00a2ba0 2024-09-05T06:34:16.099 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2024-09-05T06:34:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 msgr2=0x7f65a00a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 0x7f65a00a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f659c0314e0 tx=0x7f659c005a20 comp rx=0 tx=0).stop 2024-09-05T06:34:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 msgr2=0x7f65b4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65a400e9c0 tx=0x7f65a400ee80 comp rx=0 tx=0).stop 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 shutdown_connections 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b4103860 0x7f65b419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f65a00a2ba0 0x7f65a00a5060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b4102ea0 0x7f65b4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 --2- 172.21.15.115:0/1866144019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65b40ffec0 0x7f65b4197ef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 >> 172.21.15.115:0/1866144019 conn(0x7f65b40fba10 msgr2=0x7f65b41063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 shutdown_connections 2024-09-05T06:34:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:34:16.101+0000 7f65b9cce640 1 -- 172.21.15.115:0/1866144019 wait complete. 2024-09-05T06:34:16.420 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:16 smithi115 ceph-mon[34791]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:16.420 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:16.421 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.421 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:16 smithi086 ceph-mon[30943]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:16 smithi096 ceph-mon[34468]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:16.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:16.747 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.2.service 2024-09-05T06:34:16.750 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-09-05T06:34:16.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T06:34:17.006 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:34:18.114 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T06:34:17.804+0000 7fad6d9e9640 -1 osd.2 0 log_to_monitors true 2024-09-05T06:34:18.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:18 smithi115 ceph-mon[34791]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:18.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:18 smithi115 ceph-mon[34791]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:18.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:18 smithi115 ceph-mon[34791]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:18.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:18 smithi086 ceph-mon[30943]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:18.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:18 smithi086 ceph-mon[30943]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:18.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:18 smithi086 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:18 smithi096 ceph-mon[34468]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:18 smithi096 ceph-mon[34468]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:18 smithi096 ceph-mon[34468]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T06:34:19.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:19 smithi086 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T06:34:19.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:19 smithi086 ceph-mon[30943]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T06:34:19.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:19.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:19 smithi086 ceph-mon[30943]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:19 smithi086 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-mon[34791]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T06:34:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-mon[34791]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T06:34:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-mon[34791]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-mon[34791]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T06:34:19.178+0000 7fad6996c640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T06:34:19.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T06:34:19.182+0000 7fad65794640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:34:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:19 smithi096 ceph-mon[34468]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T06:34:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:19 smithi096 ceph-mon[34468]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T06:34:19.458 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:19.459 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:19 smithi096 ceph-mon[34468]: from='osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.459 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:19 smithi096 ceph-mon[34468]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:34:19.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:34:19.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-09-05T06:34:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:20.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:20.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:20 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 -- 172.21.15.86:0/770013415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 msgr2=0x7f4f0c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/770013415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c102150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009900 tx=0x7f4f0002a410 comp rx=0 tx=0).stop 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 -- 172.21.15.86:0/770013415 shutdown_connections 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/770013415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 0x7f4f0c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/770013415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c102150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/770013415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 -- 172.21.15.86:0/770013415 >> 172.21.15.86:0/770013415 conn(0x7f4f0c0fc280 msgr2=0x7f4f0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 -- 172.21.15.86:0/770013415 shutdown_connections 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 -- 172.21.15.86:0/770013415 wait complete. 2024-09-05T06:34:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.340+0000 7f4f11f6c640 1 Processor -- start 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 -- start start 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 0x7f4f0c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c10c6b0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c10c530 con 0x7f4f0c100ad0 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f11f6c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c10c830 con 0x7f4f0c102690 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34354/0 (socket says 172.21.15.86:34354) 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 -- 172.21.15.86:0/2103502350 learned_addr learned my addr 172.21.15.86:0/2103502350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0bfff640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 0x7f4f0c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 -- 172.21.15.86:0/2103502350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 msgr2=0x7f4f0c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 0x7f4f0c1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 -- 172.21.15.86:0/2103502350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 msgr2=0x7f4f0c104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 -- 172.21.15.86:0/2103502350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c103eb0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0b7fe640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:34:20.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.341+0000 7f4f0affd640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f000314e0 tx=0x7f4f00003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.342+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0003f070 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.342+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c1a1ff0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.342+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f00031c40 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.342+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00035030 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.342+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c1a2470 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.343+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c1a26a0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.344+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f4f00042070 con 0x7f4f0c101cd0 2024-09-05T06:34:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.345+0000 7f4f08ff9640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 0x7f4ef40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.345+0000 7f4f0b7fe640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 0x7f4ef40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.345+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f4f000ee2e0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.345+0000 7f4f0b7fe640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 0x7f4ef40a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c101b30 tx=0x7f4efc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:34:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.349+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f000bc7d0 con 0x7f4f0c101cd0 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:20 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:20.528+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f0c0fe010 con 0x7f4ef40a2af0 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:20.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:20 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi115 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] boot 2024-09-05T06:34:21.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T06:34:21.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:21.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:21.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:21.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:21.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi115 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] boot 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi115 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: osd.2 [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] boot 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:21.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:21.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:22 smithi086 ceph-mon[30943]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:22.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:22 smithi086 ceph-mon[30943]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:22.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:22 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:22.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:22 smithi086 ceph-mon[30943]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T06:34:22.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:22 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:22 smithi115 ceph-mon[34791]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:22 smithi115 ceph-mon[34791]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:22 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:22 smithi115 ceph-mon[34791]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T06:34:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:22 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:22 smithi096 ceph-mon[34468]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:22 smithi096 ceph-mon[34468]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T06:34:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:22 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:22 smithi096 ceph-mon[34468]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T06:34:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:22 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T06:34:22.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:22.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:22.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46508]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:22.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46508]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:23.269 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:23.269 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:23.269 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46534]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:23.269 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 06:34:22 smithi086 sudo[46534]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:23.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:23.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T06:34:23.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]: dispatch 2024-09-05T06:34:23.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]': finished 2024-09-05T06:34:23.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T06:34:23.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:23.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:23.521 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:23 smithi086 ceph-mon[30943]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:23.521 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:23.521 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:23.522 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42588]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:23.522 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42588]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]: dispatch 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]': finished 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:23.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:23 smithi096 ceph-mon[34468]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T06:34:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T06:34:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]: dispatch 2024-09-05T06:34:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/345490071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d41eff29-18b3-48ac-8d2b-5ef2945f4144"}]': finished 2024-09-05T06:34:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T06:34:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:23 smithi115 ceph-mon[34791]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:23.806 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:23.807 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:23.807 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42591]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:23.807 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 06:34:23 smithi096 sudo[42591]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:24.029+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f000b8740 con 0x7f4f0c101cd0 2024-09-05T06:34:24.049 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:23 smithi115 sudo[43615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:24.049 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:23 smithi115 sudo[43615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:24.050 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:23 smithi115 sudo[43615]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:24.050 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:23 smithi115 sudo[43615]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:24.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:24.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43618]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:24.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43618]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46611]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:24.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46611]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:24.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46611]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:24.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46611]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/306772846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:24.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:24.831 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 ceph-mon[34791]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43621]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43621]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43621]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:24.832 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:24 smithi115 sudo[43621]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46617]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46617]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 sudo[46617]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/306772846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 ceph-mon[30943]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:24.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:24 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/306772846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:24 smithi096 ceph-mon[34468]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T06:34:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:24 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 sudo[43624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:25.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 sudo[43624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:25.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 sudo[43624]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:25.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 sudo[43624]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:25.563 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42594]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T06:34:25.563 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42594]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:25.563 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42594]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:25.563 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42594]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:25 smithi115 ceph-mon[34791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:25.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:25.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:25.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:25.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:25.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:25 smithi086 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 sudo[42597]: pam_unix(sudo:session): session closed for user root 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:26.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:26.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:25 smithi096 ceph-mon[34468]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T06:34:28.408 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:28 smithi086 ceph-mon[30943]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:28.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:28 smithi115 ceph-mon[34791]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:28.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:28 smithi096 ceph-mon[34468]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:30.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:30 smithi086 ceph-mon[30943]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-05T06:34:30.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:30 smithi115 ceph-mon[34791]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-05T06:34:30.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:30 smithi096 ceph-mon[34468]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-09-05T06:34:32.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:32 smithi115 ceph-mon[34791]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-05T06:34:32.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T06:34:32.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:32.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:32 smithi086 ceph-mon[30943]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-05T06:34:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T06:34:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:32.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:32 smithi096 ceph-mon[34468]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-09-05T06:34:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T06:34:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:33 smithi115 ceph-mon[34791]: Deploying daemon osd.3 on smithi086 2024-09-05T06:34:33.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:33 smithi086 ceph-mon[30943]: Deploying daemon osd.3 on smithi086 2024-09-05T06:34:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:33 smithi096 ceph-mon[34468]: Deploying daemon osd.3 on smithi086 2024-09-05T06:34:34.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:34 smithi115 ceph-mon[34791]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-05T06:34:34.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:34 smithi086 ceph-mon[30943]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-05T06:34:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:34 smithi096 ceph-mon[34468]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-05T06:34:36.304 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:36 smithi086 ceph-mon[30943]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-05T06:34:36.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:36 smithi115 ceph-mon[34791]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-05T06:34:36.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:36 smithi096 ceph-mon[34468]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-05T06:34:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:37.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:37.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:37.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:37.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:38.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:38 smithi115 ceph-mon[34791]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:38.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:38 smithi086 ceph-mon[30943]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:38.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:38 smithi096 ceph-mon[34468]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:40.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:40 smithi086 ceph-mon[30943]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:40.483 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.484 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:40 smithi096 ceph-mon[34468]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:40 smithi115 ceph-mon[34791]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-05T06:34:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.820+0000 7f4f08ff9640 1 -- 172.21.15.86:0/2103502350 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f0c0fe010 con 0x7f4ef40a2af0 2024-09-05T06:34:40.820 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-09-05T06:34:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 msgr2=0x7f4ef40a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 0x7f4ef40a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c101b30 tx=0x7f4efc005f90 comp rx=0 tx=0).stop 2024-09-05T06:34:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 msgr2=0x7f4f0c1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f000314e0 tx=0x7f4f00003c10 comp rx=0 tx=0).stop 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 shutdown_connections 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f0c102690 0x7f4f0c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c101cd0 0x7f4f0c1030b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ef40a2af0 0x7f4ef40a4fb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 --2- 172.21.15.86:0/2103502350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f0c100ad0 0x7f4f0c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.822+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 >> 172.21.15.86:0/2103502350 conn(0x7f4f0c0fc280 msgr2=0x7f4f0c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.823+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 shutdown_connections 2024-09-05T06:34:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:34:40.823+0000 7f4f11f6c640 1 -- 172.21.15.86:0/2103502350 wait complete. 2024-09-05T06:34:41.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:41.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:41.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:41.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.447 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.3.service 2024-09-05T06:34:41.450 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-09-05T06:34:41.451 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.480 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:34:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:41.703 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:34:42.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:42 smithi086 ceph-mon[30943]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:42 smithi096 ceph-mon[34468]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:42 smithi115 ceph-mon[34791]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:43.482 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 06:34:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T06:34:43.258+0000 7ffac02ec640 -1 osd.3 0 log_to_monitors true 2024-09-05T06:34:44.112 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:34:44.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-09-05T06:34:44.409 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:44 smithi096 ceph-mon[34468]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:44.410 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:44 smithi096 ceph-mon[34468]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T06:34:44.429 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:34:44.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:44 smithi086 ceph-mon[30943]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:44 smithi086 ceph-mon[30943]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T06:34:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:44 smithi115 ceph-mon[34791]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:44 smithi115 ceph-mon[34791]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 -- 172.21.15.96:0/3247838790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 msgr2=0x7f8204104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 --2- 172.21.15.96:0/3247838790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204104a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8200009900 tx=0x7f820002a410 comp rx=0 tx=0).stop 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 -- 172.21.15.96:0/3247838790 shutdown_connections 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 --2- 172.21.15.96:0/3247838790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82041078c0 0x7f8204109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 --2- 172.21.15.96:0/3247838790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8204104f90 0x7f8204107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 --2- 172.21.15.96:0/3247838790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204104a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.897+0000 7f820bb22640 1 -- 172.21.15.96:0/3247838790 >> 172.21.15.96:0/3247838790 conn(0x7f82040fc280 msgr2=0x7f82040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- 172.21.15.96:0/3247838790 shutdown_connections 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- 172.21.15.96:0/3247838790 wait complete. 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 Processor -- start 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- start start 2024-09-05T06:34:44.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8204104f90 0x7f8204100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f820410d740 con 0x7f8204102660 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f820410d5c0 con 0x7f82041078c0 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.898+0000 7f820bb22640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f820410d8c0 con 0x7f8204104f90 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f820a0bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.96:58222/0 (socket says 172.21.15.96:58222) 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f820a0bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37698/0 (socket says 172.21.15.96:37698) 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 -- 172.21.15.96:0/190397830 learned_addr learned my addr 172.21.15.96:0/190397830 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-05T06:34:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82090bd640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8204104f90 0x7f8204100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 -- 172.21.15.96:0/190397830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8204104f90 msgr2=0x7f8204100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8204104f90 0x7f8204100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 -- 172.21.15.96:0/190397830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 msgr2=0x7f8204100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 -- 172.21.15.96:0/190397830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8204101410 con 0x7f8204102660 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f820a0bf640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f82098be640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8200005940 tx=0x7f82000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:34:44.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820003f030 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8200003cb0 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200035070 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041a1f80 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.899+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041a2320 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.900+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82040fdd70 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.901+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8200042030 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.902+0000 7f81faffd640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 0x7f81f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.903+0000 7f82090bd640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 0x7f81f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.903+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3373+0+0 (secure 0 0 0) 0x7f82000ee4d0 con 0x7f8204102660 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.903+0000 7f82090bd640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 0x7f81f00a5040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81f40095f0 tx=0x7f81f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:34:44.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:44.906+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82000bc090 con 0x7f8204102660 2024-09-05T06:34:45.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:34:45.087+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8204062630 con 0x7f81f00a2b80 2024-09-05T06:34:45.422 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.423 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:45.424 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:45.424 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:45.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:45.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:45.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:34:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:34:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:46.204 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T06:34:45.874+0000 7ffabc26f640 -1 osd.3 0 waiting for initial osdmap 2024-09-05T06:34:46.205 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 06:34:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T06:34:45.878+0000 7ffab408f640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi086 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 8544M 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' 2024-09-05T06:34:46.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.510 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi086 2024-09-05T06:34:46.510 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 8544M 2024-09-05T06:34:46.510 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:46.510 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' 2024-09-05T06:34:46.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi086 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 8544M 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107]' entity='osd.3' 2024-09-05T06:34:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] boot 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:47 smithi086 ceph-mon[30943]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T06:34:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:34:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:34:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] boot 2024-09-05T06:34:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T06:34:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:47 smithi096 ceph-mon[34468]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T06:34:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:34:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:34:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: osd.3 [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] boot 2024-09-05T06:34:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T06:34:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:34:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:47 smithi115 ceph-mon[34791]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/3987393744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]': finished 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/3251743455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/3987393744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]': finished 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/3251743455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T06:34:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/3987393744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]: dispatch 2024-09-05T06:34:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83c0b669-bfcd-4de2-8349-2eb03e0f28fb"}]': finished 2024-09-05T06:34:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: osdmap e26: 5 total, 4 up, 5 in 2024-09-05T06:34:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/3251743455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:34:50.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:49 smithi096 ceph-mon[34468]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T06:34:50.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:50.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:49 smithi096 ceph-mon[34468]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:50.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:49 smithi115 ceph-mon[34791]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T06:34:50.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:50.185 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:49 smithi115 ceph-mon[34791]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:50.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:49 smithi086 ceph-mon[30943]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T06:34:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:49 smithi086 ceph-mon[30943]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:51.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:50 smithi096 ceph-mon[34468]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T06:34:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:50 smithi096 ceph-mon[34468]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T06:34:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:50 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:51.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:50 smithi115 ceph-mon[34791]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T06:34:51.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:50 smithi115 ceph-mon[34791]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T06:34:51.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:50 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:50 smithi086 ceph-mon[30943]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T06:34:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:50 smithi086 ceph-mon[30943]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T06:34:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:50 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:34:52.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:51 smithi096 ceph-mon[34468]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:52.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:51 smithi115 ceph-mon[34791]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:52.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:51 smithi086 ceph-mon[30943]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:54.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:54 smithi115 ceph-mon[34791]: pgmap v99: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:54.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:54 smithi096 ceph-mon[34468]: pgmap v99: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:54.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:54 smithi086 ceph-mon[30943]: pgmap v99: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T06:34:55.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:55 smithi096 ceph-mon[34468]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T06:34:55.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:55 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T06:34:55.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:55 smithi086 ceph-mon[30943]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T06:34:55.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:55 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T06:34:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:55 smithi115 ceph-mon[34791]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T06:34:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:55 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T06:34:56.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:56 smithi096 ceph-mon[34468]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-05T06:34:56.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T06:34:56.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:56.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:56 smithi086 ceph-mon[30943]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-05T06:34:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T06:34:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:56 smithi115 ceph-mon[34791]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-05T06:34:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T06:34:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:34:57.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:57 smithi086 ceph-mon[30943]: Deploying daemon osd.4 on smithi096 2024-09-05T06:34:57.553 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:57 smithi096 ceph-mon[34468]: Deploying daemon osd.4 on smithi096 2024-09-05T06:34:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:57 smithi115 ceph-mon[34791]: Deploying daemon osd.4 on smithi096 2024-09-05T06:34:58.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:34:58 smithi086 ceph-mon[30943]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:34:58.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:34:58 smithi096 ceph-mon[34468]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:34:58.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:34:58 smithi115 ceph-mon[34791]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:35:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:00 smithi086 ceph-mon[30943]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:35:00.503 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:00 smithi096 ceph-mon[34468]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:35:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:00 smithi115 ceph-mon[34791]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:35:02.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:02 smithi086 ceph-mon[30943]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:35:02.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:02.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:02.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:02 smithi096 ceph-mon[34468]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:35:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:02 smithi115 ceph-mon[34791]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:35:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:04 smithi096 ceph-mon[34468]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:04 smithi115 ceph-mon[34791]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:04.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:04 smithi086 ceph-mon[30943]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:05.540 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:05.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:05.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.580+0000 7f81faffd640 1 -- 172.21.15.96:0/190397830 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8204062630 con 0x7f81f00a2b80 2024-09-05T06:35:05.581 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-09-05T06:35:05.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 msgr2=0x7f81f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:05.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 0x7f81f00a5040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81f40095f0 tx=0x7f81f4005f90 comp rx=0 tx=0).stop 2024-09-05T06:35:05.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 msgr2=0x7f8204101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:05.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8200005940 tx=0x7f82000042f0 comp rx=0 tx=0).stop 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 shutdown_connections 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f81f00a2b80 0x7f81f00a5040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82041078c0 0x7f8204100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8204104f90 0x7f8204100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 --2- 172.21.15.96:0/190397830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204102660 0x7f8204101fc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 >> 172.21.15.96:0/190397830 conn(0x7f82040fc280 msgr2=0x7f8204103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 shutdown_connections 2024-09-05T06:35:05.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:05.583+0000 7f820bb22640 1 -- 172.21.15.96:0/190397830 wait complete. 2024-09-05T06:35:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.254 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.4.service 2024-09-05T06:35:06.257 INFO:tasks.cephadm:Deploying osd.5 on smithi115 with /dev/vg_nvme/lv_3... 2024-09-05T06:35:06.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T06:35:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:35:06.538 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:06 smithi096 ceph-mon[34468]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:06.538 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:06.538 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.538 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.629 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:06 smithi115 ceph-mon[34791]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:06.630 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:06.630 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.630 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:06 smithi086 ceph-mon[30943]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:07.751 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 06:35:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T06:35:07.454+0000 7f26c4605640 -1 osd.4 0 log_to_monitors true 2024-09-05T06:35:08.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:08 smithi115 ceph-mon[34791]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:08.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:08 smithi115 ceph-mon[34791]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:08 smithi115 ceph-mon[34791]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:08 smithi096 ceph-mon[34468]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:08.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:08 smithi096 ceph-mon[34468]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:08 smithi096 ceph-mon[34468]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:08 smithi086 ceph-mon[30943]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:08 smithi086 ceph-mon[30943]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:08 smithi086 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T06:35:08.987 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:35:09.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-09-05T06:35:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:35:09.544 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T06:35:09.544 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:09.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T06:35:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T06:35:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 -- 172.21.15.115:0/1371690761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 msgr2=0x7f871010a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 --2- 172.21.15.115:0/1371690761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871010a060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8708009ca0 tx=0x7f870802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 -- 172.21.15.115:0/1371690761 shutdown_connections 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 --2- 172.21.15.115:0/1371690761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871010a060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 --2- 172.21.15.115:0/1371690761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710102e10 0x7f8710103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 --2- 172.21.15.115:0/1371690761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8710101c10 0x7f8710102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.736+0000 7f8717860640 1 -- 172.21.15.115:0/1371690761 >> 172.21.15.115:0/1371690761 conn(0x7f87100fd3c0 msgr2=0x7f87100ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- 172.21.15.115:0/1371690761 shutdown_connections 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- 172.21.15.115:0/1371690761 wait complete. 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 Processor -- start 2024-09-05T06:35:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- start start 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710101c10 0x7f8710197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8710102e10 0x7f8710198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f871010d740 con 0x7f8710102e10 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f871010d5c0 con 0x7f8710101c10 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8717860640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f871010d8c0 con 0x7f87101037d0 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8715dfd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8715dfd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41498/0 (socket says 172.21.15.115:41498) 2024-09-05T06:35:09.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f8715dfd640 1 -- 172.21.15.115:0/1853331533 learned_addr learned my addr 172.21.15.115:0/1853331533 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.737+0000 7f87155fc640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710101c10 0x7f8710197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 -- 172.21.15.115:0/1853331533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710101c10 msgr2=0x7f8710197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710101c10 0x7f8710197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 -- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8710102e10 msgr2=0x7f8710198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8710102e10 0x7f8710198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 -- 172.21.15.115:0/1853331533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871019fa20 con 0x7f87101037d0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8715dfd640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8708031970 tx=0x7f8708003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708004040 con 0x7f87101037d0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871019fcb0 con 0x7f87101037d0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8708004690 con 0x7f87101037d0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708030c60 con 0x7f87101037d0 2024-09-05T06:35:09.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.738+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87101a01f0 con 0x7f87101037d0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.739+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87101090a0 con 0x7f87101037d0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.740+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f87080041e0 con 0x7f87101037d0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.741+0000 7f87067fc640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 0x7f86fc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.741+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f87080ef550 con 0x7f87101037d0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.741+0000 7f87155fc640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 0x7f86fc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.743+0000 7f87155fc640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 0x7f86fc0a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8710102c70 tx=0x7f870c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:09.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.745+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87080bd640 con 0x7f87101037d0 2024-09-05T06:35:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:09.920+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8710062630 con 0x7f86fc0a2bb0 2024-09-05T06:35:10.293 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T06:35:09.974+0000 7f26c0588640 -1 osd.4 0 waiting for initial osdmap 2024-09-05T06:35:10.293 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 06:35:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T06:35:09.978+0000 7f26b83a8640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi096 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.545 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:10.546 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:10.546 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:10.546 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:10 smithi115 ceph-mon[34791]: from='osd.4 ' entity='osd.4' 2024-09-05T06:35:10.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:10.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi096 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:10 smithi096 ceph-mon[34468]: from='osd.4 ' entity='osd.4' 2024-09-05T06:35:10.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi096 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:10 smithi086 ceph-mon[30943]: from='osd.4 ' entity='osd.4' 2024-09-05T06:35:11.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:35:11.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:35:11.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:11.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.541 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] boot 2024-09-05T06:35:11.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T06:35:11.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.542 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:11 smithi115 ceph-mon[34791]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:11.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] boot 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:11 smithi096 ceph-mon[34468]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: osd.4 [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] boot 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:35:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:11 smithi086 ceph-mon[30943]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T06:35:13.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/239039183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]': finished 2024-09-05T06:35:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: osdmap e32: 6 total, 5 up, 6 in 2024-09-05T06:35:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/2902236933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:13.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/239039183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]': finished 2024-09-05T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: osdmap e32: 6 total, 5 up, 6 in 2024-09-05T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/2902236933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:13.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/239039183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]: dispatch 2024-09-05T06:35:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6110ec1a-05cb-42e3-9e2f-8442e1e2abf2"}]': finished 2024-09-05T06:35:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: osdmap e32: 6 total, 5 up, 6 in 2024-09-05T06:35:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/2902236933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:14.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:13 smithi096 ceph-mon[34468]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:14.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:13 smithi115 ceph-mon[34791]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:14.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:13 smithi086 ceph-mon[30943]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:16.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:16 smithi115 ceph-mon[34791]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:16.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:16 smithi086 ceph-mon[30943]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:16.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:16 smithi096 ceph-mon[34468]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:18.430 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:18 smithi115 ceph-mon[34791]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:18.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:18 smithi086 ceph-mon[30943]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:18 smithi096 ceph-mon[34468]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:20 smithi115 ceph-mon[34791]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:20.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:20 smithi086 ceph-mon[30943]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:20 smithi096 ceph-mon[34468]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:21.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T06:35:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:21.496 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T06:35:21.496 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T06:35:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:22.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:22 smithi115 ceph-mon[34791]: Deploying daemon osd.5 on smithi115 2024-09-05T06:35:22.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:22 smithi115 ceph-mon[34791]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:22 smithi086 ceph-mon[30943]: Deploying daemon osd.5 on smithi115 2024-09-05T06:35:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:22 smithi086 ceph-mon[30943]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:22.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:22 smithi096 ceph-mon[34468]: Deploying daemon osd.5 on smithi115 2024-09-05T06:35:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:22 smithi096 ceph-mon[34468]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:24.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:24 smithi086 ceph-mon[30943]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:24.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:24 smithi115 ceph-mon[34791]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:24.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:24 smithi096 ceph-mon[34468]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:26 smithi086 ceph-mon[30943]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:26 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:26 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:26 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:26.510 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:26 smithi115 ceph-mon[34791]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:26.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:26 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:26.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:26 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:26.511 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:26 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:26 smithi096 ceph-mon[34468]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:26 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:26 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:26 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:28.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:28 smithi086 ceph-mon[30943]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:28.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:28 smithi096 ceph-mon[34468]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:28 smithi115 ceph-mon[34791]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:29 smithi096 ceph-mon[34468]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.816 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:29 smithi115 ceph-mon[34791]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.855+0000 7f87067fc640 1 -- 172.21.15.115:0/1853331533 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8710062630 con 0x7f86fc0a2bb0 2024-09-05T06:35:29.856 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 5 on host 'smithi115' 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 msgr2=0x7f86fc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 0x7f86fc0a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8710102c70 tx=0x7f870c00b390 comp rx=0 tx=0).stop 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 msgr2=0x7f871019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8708031970 tx=0x7f8708003ba0 comp rx=0 tx=0).stop 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 shutdown_connections 2024-09-05T06:35:29.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87101037d0 0x7f871019f3b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8710102e10 0x7f8710198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f86fc0a2bb0 0x7f86fc0a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 --2- 172.21.15.115:0/1853331533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8710101c10 0x7f8710197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 >> 172.21.15.115:0/1853331533 conn(0x7f87100fd3c0 msgr2=0x7f8710106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 shutdown_connections 2024-09-05T06:35:29.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:35:29.857+0000 7f8717860640 1 -- 172.21.15.115:0/1853331533 wait complete. 2024-09-05T06:35:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:29.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:29 smithi086 ceph-mon[30943]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:30.554 DEBUG:teuthology.orchestra.run.smithi115:osd.5> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.5.service 2024-09-05T06:35:30.556 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-09-05T06:35:30.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T06:35:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:35:30.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:30.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:30.818 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:30.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:30.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:30.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:31.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:31.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:31.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:31.857 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:31 smithi096 ceph-mon[34468]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:31.860 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:31 smithi115 ceph-mon[34791]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:31.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:31 smithi086 ceph-mon[30943]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:32.183 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 06:35:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T06:35:31.946+0000 7f4595b8b640 -1 osd.5 0 log_to_monitors true 2024-09-05T06:35:32.622 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:32 smithi115 ceph-mon[34791]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:32.622 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:32 smithi115 ceph-mon[34791]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:32.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:32 smithi086 ceph-mon[30943]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:32.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:32 smithi086 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:33.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:32 smithi096 ceph-mon[34468]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:33.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:32 smithi096 ceph-mon[34468]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T06:35:33.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:35:33.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-09-05T06:35:33.842 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T06:35:33.842 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T06:35:33.842 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:33.842 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:33.842 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:33.843 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:33.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:33.935 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T06:35:33.591+0000 7f4591b0e640 -1 osd.5 0 waiting for initial osdmap 2024-09-05T06:35:33.935 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 06:35:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T06:35:33.597+0000 7f458d936640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:34.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:34.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:34.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/3690873003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c105750 msgr2=0x7f5c9c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:34.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/3690873003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c105750 0x7f5c9c105c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009ca0 tx=0x7f5c9402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/3690873003 shutdown_connections 2024-09-05T06:35:34.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/3690873003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c105750 0x7f5c9c105c00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/3690873003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 0x7f5c9c105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/3690873003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c100450 0x7f5c9c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/3690873003 >> 172.21.15.86:0/3690873003 conn(0x7f5c9c0fc280 msgr2=0x7f5c9c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/3690873003 shutdown_connections 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/3690873003 wait complete. 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 Processor -- start 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.329+0000 7f5ca3bd1640 1 -- start start 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 0x7f5c9c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c105750 0x7f5c9c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c113d50 con 0x7f5c9c100450 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c113bd0 con 0x7f5c9c105750 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c113ed0 con 0x7f5c9c102da0 2024-09-05T06:35:34.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37942/0 (socket says 172.21.15.86:37942) 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 -- 172.21.15.86:0/742919551 learned_addr learned my addr 172.21.15.86:0/742919551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca216e640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c105750 0x7f5c9c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca116c640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 0x7f5c9c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 -- 172.21.15.86:0/742919551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 msgr2=0x7f5c9c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 0x7f5c9c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 -- 172.21.15.86:0/742919551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c105750 msgr2=0x7f5c9c1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c105750 0x7f5c9c1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 -- 172.21.15.86:0/742919551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1a5fb0 con 0x7f5c9c100450 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca196d640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c920 tx=0x7f5c9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98007c70 con 0x7f5c9c100450 2024-09-05T06:35:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c1a62a0 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c98007e10 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98018820 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.330+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a6830 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.332+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c0fdd70 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.333+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c980189c0 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.334+0000 7f5c92ffd640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 0x7f5c880a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.334+0000 7f5ca116c640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 0x7f5c880a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.334+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f5c980ce170 con 0x7f5c9c100450 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.334+0000 7f5ca116c640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 0x7f5c880a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c19f8a0 tx=0x7f5c8c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.338+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c98014030 con 0x7f5c9c100450 2024-09-05T06:35:34.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:34.517+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5c9c062630 con 0x7f5c880a2bb0 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi115 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.927 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] boot 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: osdmap e35: 6 total, 6 up, 6 in 2024-09-05T06:35:34.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi115 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] boot 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: osdmap e35: 6 total, 6 up, 6 in 2024-09-05T06:35:34.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi115 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: osd.5 [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] boot 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: osdmap e35: 6 total, 6 up, 6 in 2024-09-05T06:35:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:35:35.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:35 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:35:35.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:35 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:35:35.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:35 smithi086 ceph-mon[30943]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:35.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:35 smithi086 ceph-mon[30943]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:35.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:35 smithi086 ceph-mon[30943]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T06:35:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:35 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:35:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:35 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:35:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:35 smithi115 ceph-mon[34791]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:35 smithi115 ceph-mon[34791]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:35 smithi115 ceph-mon[34791]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T06:35:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:35 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:35:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:35 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:35:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:35 smithi096 ceph-mon[34468]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:35:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:35 smithi096 ceph-mon[34468]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:35 smithi096 ceph-mon[34468]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]: dispatch 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]': finished 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:37.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2461114636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]: dispatch 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]': finished 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:37 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2461114636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]: dispatch 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1511019191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637d7ec0-a913-4e18-92a2-9503fedd4a6b"}]': finished 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:38.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:38.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:37 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2461114636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:35:40.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:40 smithi115 ceph-mon[34791]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T06:35:40.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:40 smithi086 ceph-mon[30943]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T06:35:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:40 smithi096 ceph-mon[34468]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T06:35:42.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:42 smithi086 ceph-mon[30943]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-05T06:35:42.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:42 smithi115 ceph-mon[34791]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-05T06:35:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:42 smithi096 ceph-mon[34468]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-05T06:35:44.312 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:44 smithi086 ceph-mon[30943]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-05T06:35:44.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:44 smithi115 ceph-mon[34791]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-05T06:35:44.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:44 smithi096 ceph-mon[34468]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-05T06:35:45.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T06:35:45.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:45.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T06:35:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T06:35:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:46.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:46 smithi086 ceph-mon[30943]: Deploying daemon osd.6 on smithi086 2024-09-05T06:35:46.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:46 smithi086 ceph-mon[30943]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:35:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:46 smithi096 ceph-mon[34468]: Deploying daemon osd.6 on smithi086 2024-09-05T06:35:46.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:46 smithi096 ceph-mon[34468]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:35:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:46 smithi115 ceph-mon[34791]: Deploying daemon osd.6 on smithi086 2024-09-05T06:35:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:46 smithi115 ceph-mon[34791]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T06:35:48.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:48 smithi086 ceph-mon[30943]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-05T06:35:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:48 smithi096 ceph-mon[34468]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-05T06:35:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:48 smithi115 ceph-mon[34791]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-05T06:35:50.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:50 smithi086 ceph-mon[30943]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:50.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:50 smithi115 ceph-mon[34791]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:50 smithi096 ceph-mon[34468]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:35:51.328 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:51.328 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:51.328 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:51.497 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:51.497 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:51.497 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:52 smithi086 ceph-mon[30943]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:52.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:52 smithi096 ceph-mon[34468]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:52 smithi115 ceph-mon[34791]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.175+0000 7f5c92ffd640 1 -- 172.21.15.86:0/742919551 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c9c062630 con 0x7f5c880a2bb0 2024-09-05T06:35:54.176 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-09-05T06:35:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 msgr2=0x7f5c880a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 0x7f5c880a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c19f8a0 tx=0x7f5c8c008040 comp rx=0 tx=0).stop 2024-09-05T06:35:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 msgr2=0x7f5c9c19e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c920 tx=0x7f5c9800cde0 comp rx=0 tx=0).stop 2024-09-05T06:35:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 shutdown_connections 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c880a2bb0 0x7f5c880a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c9c105750 0x7f5c9c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c9c102da0 0x7f5c9c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 --2- 172.21.15.86:0/742919551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c100450 0x7f5c9c19e380 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 >> 172.21.15.86:0/742919551 conn(0x7f5c9c0fc280 msgr2=0x7f5c9c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.177+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 shutdown_connections 2024-09-05T06:35:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:35:54.178+0000 7f5ca3bd1640 1 -- 172.21.15.86:0/742919551 wait complete. 2024-09-05T06:35:54.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:54 smithi086 ceph-mon[30943]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:54 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:54 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:54 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:54 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:54 smithi096 ceph-mon[34468]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:54 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:54 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:54 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:54 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:54 smithi115 ceph-mon[34791]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:54 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:54 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:54 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:54 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:55.109 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.6.service 2024-09-05T06:35:55.111 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-09-05T06:35:55.112 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T06:35:55.363 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:35:56.206 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 06:35:55 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T06:35:55.755+0000 7f2fcd274640 -1 osd.6 0 log_to_monitors true 2024-09-05T06:35:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:56 smithi086 ceph-mon[30943]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:56 smithi086 ceph-mon[30943]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T06:35:56.507 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:56 smithi096 ceph-mon[34468]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:56.507 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:56 smithi096 ceph-mon[34468]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T06:35:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:56 smithi115 ceph-mon[34791]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:56 smithi115 ceph-mon[34791]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T06:35:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T06:35:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.727 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T06:35:57.728 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:57.987 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T06:35:57.870+0000 7f2fc9a0b640 -1 osd.6 0 waiting for initial osdmap 2024-09-05T06:35:57.987 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 06:35:57 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T06:35:57.877+0000 7f2fc1017640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:35:57.988 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:35:58.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-09-05T06:35:58.269 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:58.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620]' entity='osd.6' 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:35:58.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:58.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 -- 172.21.15.96:0/4013179840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 msgr2=0x7f2188102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:58.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 --2- 172.21.15.96:0/4013179840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188102190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2174009900 tx=0x7f217402a410 comp rx=0 tx=0).stop 2024-09-05T06:35:58.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 -- 172.21.15.96:0/4013179840 shutdown_connections 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 --2- 172.21.15.96:0/4013179840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 0x7f2188108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 --2- 172.21.15.96:0/4013179840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188102190 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 --2- 172.21.15.96:0/4013179840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 0x7f2188100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 -- 172.21.15.96:0/4013179840 >> 172.21.15.96:0/4013179840 conn(0x7f21880fc2a0 msgr2=0x7f21880fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 -- 172.21.15.96:0/4013179840 shutdown_connections 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.737+0000 7f218f19b640 1 -- 172.21.15.96:0/4013179840 wait complete. 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 Processor -- start 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 -- start start 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 0x7f2188197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:58.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 0x7f218819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f218810d770 con 0x7f21881026d0 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f218810d5f0 con 0x7f2188101d10 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218f19b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f218810d8f0 con 0x7f2188100b10 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f2187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f2187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44116/0 (socket says 172.21.15.96:44116) 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f2187fff640 1 -- 172.21.15.96:0/2163354861 learned_addr learned my addr 172.21.15.96:0/2163354861 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218d738640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 0x7f218819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.738+0000 7f218cf37640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 0x7f2188197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 -- 172.21.15.96:0/2163354861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 msgr2=0x7f2188197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 0x7f2188197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 -- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 msgr2=0x7f218819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:35:58.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 0x7f218819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 -- 172.21.15.96:0/2163354861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218819fa00 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2187fff640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2174005940 tx=0x7f2174004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217403f030 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2174003cb0 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174035070 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218819fc90 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.739+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881a01c0 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.740+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2188107f60 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.741+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2174042030 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.742+0000 7f2185ffb640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 0x7f21780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.742+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4685+0+0 (secure 0 0 0) 0x7f21740ee910 con 0x7f2188101d10 2024-09-05T06:35:58.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.742+0000 7f218cf37640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 0x7f21780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:35:58.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.743+0000 7f218cf37640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 0x7f21780a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f217c0058c0 tx=0x7f217c00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:35:58.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.746+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21740bc6a0 con 0x7f2188101d10 2024-09-05T06:35:58.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:35:58.924+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2188062630 con 0x7f21780a2c30 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi086 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 5696M 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] boot 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: osdmap e42: 7 total, 7 up, 7 in 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:35:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:59.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi086 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 5696M 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] boot 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: osdmap e42: 7 total, 7 up, 7 in 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:35:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi086 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 5696M 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: osd.6 [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] boot 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: osdmap e42: 7 total, 7 up, 7 in 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:35:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:35:59.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:35:59.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:35:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:00.370 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:00 smithi096 ceph-mon[34468]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:00.370 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:00 smithi096 ceph-mon[34468]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:36:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:00 smithi115 ceph-mon[34791]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:00 smithi115 ceph-mon[34791]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:36:00.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:00 smithi086 ceph-mon[30943]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:00.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:00 smithi086 ceph-mon[30943]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-05T06:36:01.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:01 smithi096 ceph-mon[34468]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T06:36:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:01 smithi115 ceph-mon[34791]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T06:36:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:01 smithi086 ceph-mon[30943]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T06:36:02.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:02.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/2323827251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]': finished 2024-09-05T06:36:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: osdmap e44: 8 total, 7 up, 8 in 2024-09-05T06:36:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:02 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/56422920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/2323827251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]': finished 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: osdmap e44: 8 total, 7 up, 8 in 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:02 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/56422920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:02.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/2323827251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]: dispatch 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76945c8d-4539-4690-a020-b9af497ee44e"}]': finished 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: osdmap e44: 8 total, 7 up, 8 in 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:02 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/56422920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:04 smithi096 ceph-mon[34468]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:04 smithi115 ceph-mon[34791]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:04.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:04 smithi086 ceph-mon[30943]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:06.491 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:06 smithi096 ceph-mon[34468]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:06 smithi115 ceph-mon[34791]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:06.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:06 smithi086 ceph-mon[30943]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:08.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:08 smithi096 ceph-mon[34468]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:08 smithi115 ceph-mon[34791]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:08.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:08 smithi086 ceph-mon[30943]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:10.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:10 smithi096 ceph-mon[34468]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:10.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T06:36:10.537 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:10 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:10 smithi115 ceph-mon[34791]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T06:36:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:10 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:10 smithi086 ceph-mon[30943]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T06:36:10.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:10 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:11.468 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:11 smithi096 ceph-mon[34468]: Deploying daemon osd.7 on smithi096 2024-09-05T06:36:11.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:11 smithi115 ceph-mon[34791]: Deploying daemon osd.7 on smithi096 2024-09-05T06:36:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:11 smithi086 ceph-mon[30943]: Deploying daemon osd.7 on smithi096 2024-09-05T06:36:12.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:12 smithi096 ceph-mon[34468]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:12 smithi115 ceph-mon[34791]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:12.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:12 smithi086 ceph-mon[30943]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:14 smithi096 ceph-mon[34468]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:14.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:14 smithi086 ceph-mon[30943]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:14.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:14 smithi115 ceph-mon[34791]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:16 smithi096 ceph-mon[34468]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:16 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:16 smithi115 ceph-mon[34791]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:16 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:16.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:16 smithi086 ceph-mon[30943]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:16 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.277+0000 7f2185ffb640 1 -- 172.21.15.96:0/2163354861 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2188062630 con 0x7f21780a2c30 2024-09-05T06:36:19.278 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-09-05T06:36:19.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.279+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 msgr2=0x7f21780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:19.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.279+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 0x7f21780a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f217c0058c0 tx=0x7f217c00a4c0 comp rx=0 tx=0).stop 2024-09-05T06:36:19.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.279+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 msgr2=0x7f2188198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:19.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.279+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2174005940 tx=0x7f2174004430 comp rx=0 tx=0).stop 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 shutdown_connections 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21881026d0 0x7f218819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21780a2c30 0x7f21780a50f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101d10 0x7f2188198310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 --2- 172.21.15.96:0/2163354861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2188100b10 0x7f2188197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 >> 172.21.15.96:0/2163354861 conn(0x7f21880fc2a0 msgr2=0x7f2188105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 shutdown_connections 2024-09-05T06:36:19.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:36:19.280+0000 7f218f19b640 1 -- 172.21.15.96:0/2163354861 wait complete. 2024-09-05T06:36:19.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:19.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:19 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:19 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:19 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:19.998 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.7.service 2024-09-05T06:36:20.000 INFO:tasks.cephadm:Deploying osd.8 on smithi115 with /dev/vg_nvme/lv_2... 2024-09-05T06:36:20.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T06:36:20.254 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:36:20.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:20 smithi096 ceph-mon[34468]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:20.622 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:20 smithi115 ceph-mon[34791]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:20.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:20 smithi086 ceph-mon[30943]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:21.419 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:21 smithi096 ceph-mon[34468]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:21.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:21 smithi096 ceph-mon[34468]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:21.420 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 06:36:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T06:36:21.279+0000 7fd5df075640 -1 osd.7 0 log_to_monitors true 2024-09-05T06:36:21.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:21 smithi115 ceph-mon[34791]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:21.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:21 smithi115 ceph-mon[34791]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:21 smithi086 ceph-mon[30943]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:21 smithi086 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:22.380 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:22 smithi115 ceph-mon[34791]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.669 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:22.670 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T06:36:22.670 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.670 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T06:36:22.670 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:22.670 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:22 smithi096 ceph-mon[34468]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T06:36:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: from='osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T06:36:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:22 smithi086 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:36:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:36:22.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-09-05T06:36:23.185 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:36:23.306 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T06:36:23.009+0000 7fd5daff8640 -1 osd.7 0 waiting for initial osdmap 2024-09-05T06:36:23.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T06:36:23.018+0000 7fd5d2e18640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='osd.7 ' entity='osd.7' 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi096 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.573 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] boot 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: osdmap e47: 8 total, 8 up, 8 in 2024-09-05T06:36:23.574 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:23 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 -- 172.21.15.115:0/3313997892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 msgr2=0x7f9438105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 --2- 172.21.15.115:0/3313997892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f9438105250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9428009900 tx=0x7f942802a410 comp rx=0 tx=0).stop 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 -- 172.21.15.115:0/3313997892 shutdown_connections 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 --2- 172.21.15.115:0/3313997892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f9438105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 --2- 172.21.15.115:0/3313997892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f9438105250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 --2- 172.21.15.115:0/3313997892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 0x7f94381028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 -- 172.21.15.115:0/3313997892 >> 172.21.15.115:0/3313997892 conn(0x7f94380fc2a0 msgr2=0x7f94380fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 -- 172.21.15.115:0/3313997892 shutdown_connections 2024-09-05T06:36:23.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.655+0000 7f943f67d640 1 -- 172.21.15.115:0/3313997892 wait complete. 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 Processor -- start 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 -- start start 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 0x7f943819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f94381a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9438113d50 con 0x7f9438105790 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9438113bd0 con 0x7f9438100490 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943f67d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9438113ed0 con 0x7f9438102de0 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42380/0 (socket says 172.21.15.115:42380) 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 -- 172.21.15.115:0/3541380223 learned_addr learned my addr 172.21.15.115:0/3541380223 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943d419640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 0x7f943819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943dc1a640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f94381a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:23.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 -- 172.21.15.115:0/3541380223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 msgr2=0x7f943819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 0x7f943819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 -- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 msgr2=0x7f94381a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.656+0000 7f943cc18640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f94381a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f943cc18640 1 -- 172.21.15.115:0/3541380223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94381a6070 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f943dc1a640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f94381a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f943cc18640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9428004770 tx=0x7f94280042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942803f030 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94381a6300 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9428003cb0 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428035070 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.657+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94381a6830 con 0x7f9438102de0 2024-09-05T06:36:23.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.658+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94380fdd90 con 0x7f9438102de0 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.659+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9428042030 con 0x7f9438102de0 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.660+0000 7f942e7fc640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 0x7f94240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.660+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5430+0+0 (secure 0 0 0) 0x7f94280ef180 con 0x7f9438102de0 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.660+0000 7f943d419640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 0x7f94240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.660+0000 7f943d419640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 0x7f94240a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9434005fd0 tx=0x7f943400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:36:23.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.663+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94280bcc00 con 0x7f9438102de0 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='osd.7 ' entity='osd.7' 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi096 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] boot 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: osdmap e47: 8 total, 8 up, 8 in 2024-09-05T06:36:23.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:23 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='osd.7 ' entity='osd.7' 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi096 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: osd.7 [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] boot 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: osdmap e47: 8 total, 8 up, 8 in 2024-09-05T06:36:23.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:23 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:36:23.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:23.842+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9438062630 con 0x7f94240a2bb0 2024-09-05T06:36:24.583 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:24.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:24 smithi115 ceph-mon[34791]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T06:36:24.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:24 smithi086 ceph-mon[30943]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T06:36:24.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:24 smithi096 ceph-mon[34468]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T06:36:25.523 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:25 smithi115 ceph-mon[34791]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:25.523 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:25 smithi115 ceph-mon[34791]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T06:36:25.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:25 smithi086 ceph-mon[30943]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:25.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:25 smithi086 ceph-mon[30943]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T06:36:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:25 smithi096 ceph-mon[34468]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:25 smithi096 ceph-mon[34468]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T06:36:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:26 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/1943195946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:26 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:26 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]': finished 2024-09-05T06:36:26.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:26 smithi115 ceph-mon[34791]: osdmap e50: 9 total, 8 up, 9 in 2024-09-05T06:36:26.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:26 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:26 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/1943195946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:26 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:26 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]': finished 2024-09-05T06:36:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:26 smithi086 ceph-mon[30943]: osdmap e50: 9 total, 8 up, 9 in 2024-09-05T06:36:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:26 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:26 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/1943195946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:26 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]: dispatch 2024-09-05T06:36:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:26 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "608c50ef-1ed5-407d-8375-a0eff5fa26f0"}]': finished 2024-09-05T06:36:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:26 smithi096 ceph-mon[34468]: osdmap e50: 9 total, 8 up, 9 in 2024-09-05T06:36:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:26 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:27 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/2774365665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:27 smithi115 ceph-mon[34791]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:27 smithi115 ceph-mon[34791]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T06:36:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:27 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/2774365665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:27 smithi086 ceph-mon[30943]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:27 smithi086 ceph-mon[30943]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T06:36:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:27 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/2774365665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:27 smithi096 ceph-mon[34468]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:27 smithi096 ceph-mon[34468]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T06:36:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:30 smithi086 ceph-mon[30943]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:30.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:30 smithi096 ceph-mon[34468]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:30 smithi115 ceph-mon[34791]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:32.401 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:32 smithi115 ceph-mon[34791]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:32.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:32 smithi086 ceph-mon[30943]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:32 smithi096 ceph-mon[34468]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:34.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:34 smithi115 ceph-mon[34791]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:34.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:34 smithi086 ceph-mon[30943]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:34 smithi096 ceph-mon[34468]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:35.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T06:36:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:35.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T06:36:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:35.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T06:36:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:36.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:36 smithi086 ceph-mon[30943]: Deploying daemon osd.8 on smithi115 2024-09-05T06:36:36.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:36 smithi086 ceph-mon[30943]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:36.535 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:36 smithi115 ceph-mon[34791]: Deploying daemon osd.8 on smithi115 2024-09-05T06:36:36.535 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:36 smithi115 ceph-mon[34791]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:36.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:36 smithi096 ceph-mon[34468]: Deploying daemon osd.8 on smithi115 2024-09-05T06:36:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:36 smithi096 ceph-mon[34468]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:38.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:38 smithi115 ceph-mon[34791]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:38.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:38 smithi086 ceph-mon[30943]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:38.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:38 smithi096 ceph-mon[34468]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:40 smithi086 ceph-mon[30943]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:40.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:40 smithi096 ceph-mon[34468]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:40.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:40 smithi115 ceph-mon[34791]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:40.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:40.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:40.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:42.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:42 smithi086 ceph-mon[30943]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:42.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:42 smithi115 ceph-mon[34791]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:42 smithi096 ceph-mon[34468]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:43.917 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 8 on host 'smithi115' 2024-09-05T06:36:43.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.916+0000 7f942e7fc640 1 -- 172.21.15.115:0/3541380223 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9438062630 con 0x7f94240a2bb0 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 msgr2=0x7f94240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 0x7f94240a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9434005fd0 tx=0x7f943400b390 comp rx=0 tx=0).stop 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 msgr2=0x7f943819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9428004770 tx=0x7f94280042f0 comp rx=0 tx=0).stop 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 shutdown_connections 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9438105790 0x7f94381a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9438102de0 0x7f943819e980 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:43.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94240a2bb0 0x7f94240a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:43.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 --2- 172.21.15.115:0/3541380223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9438100490 0x7f943819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:43.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 >> 172.21.15.115:0/3541380223 conn(0x7f94380fc2a0 msgr2=0x7f9438100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:36:43.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 shutdown_connections 2024-09-05T06:36:43.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:36:43.919+0000 7f943f67d640 1 -- 172.21.15.115:0/3541380223 wait complete. 2024-09-05T06:36:44.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:43 smithi115 ceph-mon[34791]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:44.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:44.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:44.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:43 smithi086 ceph-mon[30943]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:44.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:43 smithi096 ceph-mon[34468]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:44.597 DEBUG:teuthology.orchestra.run.smithi115:osd.8> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.8.service 2024-09-05T06:36:44.599 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-09-05T06:36:44.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T06:36:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:36:45.146 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:44 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:45.146 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:44 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.146 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:44 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.221 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:44 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:45.222 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:44 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.222 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:44 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:44 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:36:45.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:44 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:44 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:45.898 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 06:36:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T06:36:45.677+0000 7f0aa803f640 -1 osd.8 0 log_to_monitors true 2024-09-05T06:36:46.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:45 smithi115 ceph-mon[34791]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:46.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:45 smithi115 ceph-mon[34791]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:45 smithi115 ceph-mon[34791]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:46.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:45 smithi086 ceph-mon[30943]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:46.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:45 smithi086 ceph-mon[30943]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:46.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:45 smithi086 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:45 smithi096 ceph-mon[34468]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:45 smithi096 ceph-mon[34468]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:45 smithi096 ceph-mon[34468]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T06:36:47.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:46 smithi086 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T06:36:47.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:46 smithi086 ceph-mon[30943]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T06:36:47.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:47.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:46 smithi086 ceph-mon[30943]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:46 smithi086 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:46 smithi096 ceph-mon[34468]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T06:36:47.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:46 smithi096 ceph-mon[34468]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T06:36:47.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:47.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:46 smithi096 ceph-mon[34468]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:46 smithi096 ceph-mon[34468]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:46 smithi115 ceph-mon[34791]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T06:36:47.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:46 smithi115 ceph-mon[34791]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T06:36:47.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:47.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:46 smithi115 ceph-mon[34791]: from='osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:46 smithi115 ceph-mon[34791]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:36:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:36:47.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-09-05T06:36:47.905 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:36:47.925 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T06:36:47.588+0000 7f0aa47d6640 -1 osd.8 0 waiting for initial osdmap 2024-09-05T06:36:47.925 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T06:36:47.597+0000 7f0a9bde2640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:36:48.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:36:48.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T06:36:48.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:48.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.180 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='osd.8 ' entity='osd.8' 2024-09-05T06:36:48.180 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] boot 2024-09-05T06:36:48.180 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: osdmap e54: 9 total, 9 up, 9 in 2024-09-05T06:36:48.180 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:47 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:36:48.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='osd.8 ' entity='osd.8' 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] boot 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: osdmap e54: 9 total, 9 up, 9 in 2024-09-05T06:36:48.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:47 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='osd.8 ' entity='osd.8' 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: osd.8 [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] boot 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: osdmap e54: 9 total, 9 up, 9 in 2024-09-05T06:36:48.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:47 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:36:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 -- 172.21.15.86:0/767328636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878101c00 msgr2=0x7f2878102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 --2- 172.21.15.86:0/767328636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878101c00 0x7f2878102000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2868009900 tx=0x7f286802a410 comp rx=0 tx=0).stop 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 -- 172.21.15.86:0/767328636 shutdown_connections 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 --2- 172.21.15.86:0/767328636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 0x7f287810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 --2- 172.21.15.86:0/767328636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878102e00 0x7f2878103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 --2- 172.21.15.86:0/767328636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878101c00 0x7f2878102000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 -- 172.21.15.86:0/767328636 >> 172.21.15.86:0/767328636 conn(0x7f28780fd390 msgr2=0x7f28780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 -- 172.21.15.86:0/767328636 shutdown_connections 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.371+0000 7f288062b640 1 -- 172.21.15.86:0/767328636 wait complete. 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 Processor -- start 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 -- start start 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878101c00 0x7f287819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 0x7f28781a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f287810d730 con 0x7f2878102e00 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f287810d5b0 con 0x7f28781037c0 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f288062b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f287810d8b0 con 0x7f2878101c00 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34630/0 (socket says 172.21.15.86:34630) 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 -- 172.21.15.86:0/3079078103 learned_addr learned my addr 172.21.15.86:0/3079078103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287e3c7640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878101c00 0x7f287819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287ebc8640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 0x7f28781a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 -- 172.21.15.86:0/3079078103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878101c00 msgr2=0x7f287819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878101c00 0x7f287819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 -- 172.21.15.86:0/3079078103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 msgr2=0x7f28781a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 0x7f28781a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:36:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.372+0000 7f287dbc6640 1 -- 172.21.15.86:0/3079078103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28781a1c60 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f287dbc6640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f287400d770 tx=0x7f287400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874017070 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28740054d0 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781a1f50 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874004cf0 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.373+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781a2400 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.374+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28781a2630 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.375+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2874004e90 con 0x7f2878102e00 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.375+0000 7f286f7fe640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 0x7f28640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.376+0000 7f287e3c7640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 0x7f28640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:36:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.376+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5862+0+0 (secure 0 0 0) 0x7f28740cdfd0 con 0x7f2878102e00 2024-09-05T06:36:48.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.376+0000 7f287e3c7640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 0x7f28640a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2868004770 tx=0x7f2868000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:36:48.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.380+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f287409b270 con 0x7f2878102e00 2024-09-05T06:36:48.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:36:48.558+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2878109d90 con 0x7f28640a2c30 2024-09-05T06:36:49.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:36:49.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi115 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:49.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:48 smithi115 ceph-mon[34791]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T06:36:49.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:36:49.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi115 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:49.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:48 smithi086 ceph-mon[30943]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi115 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:36:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:48 smithi096 ceph-mon[34468]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T06:36:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:49 smithi086 ceph-mon[30943]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:49 smithi086 ceph-mon[30943]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:49 smithi086 ceph-mon[30943]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T06:36:50.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:49 smithi115 ceph-mon[34791]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:50.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:49 smithi115 ceph-mon[34791]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:50.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:49 smithi115 ceph-mon[34791]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T06:36:50.208 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:49 smithi096 ceph-mon[34468]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:36:50.208 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:49 smithi096 ceph-mon[34468]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:50.208 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:49 smithi096 ceph-mon[34468]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]: dispatch 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]': finished 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:36:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1997779202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]: dispatch 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]': finished 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:36:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1997779202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:52.316 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]: dispatch 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/986482778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "311dbd75-2e56-4242-b1b4-536719722c62"}]': finished 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:36:52.317 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1997779202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:36:54.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:54 smithi086 ceph-mon[30943]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:54.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:54 smithi096 ceph-mon[34468]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:54 smithi115 ceph-mon[34791]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:36:56.431 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:56 smithi086 ceph-mon[30943]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:36:56.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:56 smithi096 ceph-mon[34468]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:36:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:56 smithi115 ceph-mon[34791]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:36:58.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:36:58 smithi086 ceph-mon[30943]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T06:36:58.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:36:58 smithi096 ceph-mon[34468]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T06:36:58.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:36:58 smithi115 ceph-mon[34791]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T06:37:00.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:00 smithi086 ceph-mon[30943]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T06:37:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T06:37:00.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:00 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:00.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:00 smithi096 ceph-mon[34468]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T06:37:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T06:37:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:00 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:00 smithi115 ceph-mon[34791]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T06:37:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T06:37:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:00 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:01.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:01 smithi086 ceph-mon[30943]: Deploying daemon osd.9 on smithi086 2024-09-05T06:37:01.495 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:01 smithi096 ceph-mon[34468]: Deploying daemon osd.9 on smithi086 2024-09-05T06:37:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:01 smithi115 ceph-mon[34791]: Deploying daemon osd.9 on smithi086 2024-09-05T06:37:02.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:02 smithi086 ceph-mon[30943]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:37:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:02 smithi096 ceph-mon[34468]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:37:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:02 smithi115 ceph-mon[34791]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T06:37:04.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:04 smithi086 ceph-mon[30943]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:04 smithi096 ceph-mon[34468]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:04 smithi115 ceph-mon[34791]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:06.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:06 smithi086 ceph-mon[30943]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:06.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:06.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:06.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:06 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:06 smithi096 ceph-mon[34468]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:06.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:06.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:06 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:06 smithi115 ceph-mon[34791]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:06 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:08.426 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:08 smithi086 ceph-mon[30943]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:08.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:08 smithi096 ceph-mon[34468]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:08 smithi115 ceph-mon[34791]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:09.391 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-09-05T06:37:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.390+0000 7f286f7fe640 1 -- 172.21.15.86:0/3079078103 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2878109d90 con 0x7f28640a2c30 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 msgr2=0x7f28640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 0x7f28640a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2868004770 tx=0x7f2868000bc0 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 msgr2=0x7f287819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f287400d770 tx=0x7f287400dc30 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 shutdown_connections 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28640a2c30 0x7f28640a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28781037c0 0x7f28781a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2878102e00 0x7f287819a570 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 --2- 172.21.15.86:0/3079078103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2878101c00 0x7f287819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 >> 172.21.15.86:0/3079078103 conn(0x7f28780fd390 msgr2=0x7f2878108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:37:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 shutdown_connections 2024-09-05T06:37:09.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:37:09.393+0000 7f288062b640 1 -- 172.21.15.86:0/3079078103 wait complete. 2024-09-05T06:37:09.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:09.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:09.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:09 smithi086 ceph-mon[30943]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:09.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:09.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:09 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.012 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.9.service 2024-09-05T06:37:10.014 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-09-05T06:37:10.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T06:37:10.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:09 smithi096 ceph-mon[34468]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:10.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.044 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:09 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:09 smithi115 ceph-mon[34791]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:09 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:10.268 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:37:12.198 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 06:37:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T06:37:11.999+0000 7fc9fe75a640 -1 osd.9 0 log_to_monitors true 2024-09-05T06:37:12.291 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:12 smithi096 ceph-mon[34468]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:12.292 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:12 smithi096 ceph-mon[34468]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T06:37:12.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:12 smithi086 ceph-mon[30943]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:12 smithi086 ceph-mon[30943]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T06:37:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:12 smithi115 ceph-mon[34791]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:12 smithi115 ceph-mon[34791]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T06:37:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-05T06:37:13.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-09-05T06:37:13.426 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T06:37:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-mon[30943]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T06:37:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-mon[30943]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T06:37:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-mon[30943]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:37:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-mon[30943]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:37:13.487 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T06:37:13.201+0000 7fc9fa6dd640 -1 osd.9 0 waiting for initial osdmap 2024-09-05T06:37:13.487 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 06:37:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T06:37:13.210+0000 7fc9f6505640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:37:13.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:13 smithi096 ceph-mon[34468]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T06:37:13.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:13 smithi096 ceph-mon[34468]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T06:37:13.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:13 smithi096 ceph-mon[34468]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:37:13.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:13.547 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:13 smithi096 ceph-mon[34468]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:37:13.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:13 smithi115 ceph-mon[34791]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T06:37:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:13 smithi115 ceph-mon[34791]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T06:37:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:13 smithi115 ceph-mon[34791]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-09-05T06:37:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:13 smithi115 ceph-mon[34791]: from='osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-09-05T06:37:13.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 -- 172.21.15.96:0/1747844607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028100ad0 msgr2=0x7f6028100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:13.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 --2- 172.21.15.96:0/1747844607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028100ad0 0x7f6028100ed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6018009900 tx=0x7f601802a410 comp rx=0 tx=0).stop 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 -- 172.21.15.96:0/1747844607 shutdown_connections 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 --2- 172.21.15.96:0/1747844607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028102690 0x7f6028108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 --2- 172.21.15.96:0/1747844607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028101cd0 0x7f6028102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 --2- 172.21.15.96:0/1747844607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028100ad0 0x7f6028100ed0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 -- 172.21.15.96:0/1747844607 >> 172.21.15.96:0/1747844607 conn(0x7f60280fc280 msgr2=0x7f60280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 -- 172.21.15.96:0/1747844607 shutdown_connections 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.936+0000 7f602e84f640 1 -- 172.21.15.96:0/1747844607 wait complete. 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 Processor -- start 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 -- start start 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028100ad0 0x7f6028104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028101cd0 0x7f60281030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:13.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f602810c610 con 0x7f6028100ad0 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f602810c490 con 0x7f6028102690 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602e84f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f602810c790 con 0x7f6028101cd0 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59780/0 (socket says 172.21.15.96:59780) 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 -- 172.21.15.96:0/4000832931 learned_addr learned my addr 172.21.15.96:0/4000832931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f6027fff640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028100ad0 0x7f6028104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f60277fe640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028101cd0 0x7f60281030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 -- 172.21.15.96:0/4000832931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028101cd0 msgr2=0x7f60281030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028101cd0 0x7f60281030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 -- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028100ad0 msgr2=0x7f6028104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:13.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028100ad0 0x7f6028104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.937+0000 7f602cdec640 1 -- 172.21.15.96:0/4000832931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028103eb0 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f602cdec640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f602000eaa0 tx=0x7f602000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000cdb0 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a1fe0 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6020005020 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200116d0 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.938+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a2490 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.939+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60281a2720 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.940+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f60200051d0 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.941+0000 7f60257fa640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 0x7f60100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.941+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7f60200ce820 con 0x7f6028102690 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.941+0000 7f6027fff640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 0x7f60100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.941+0000 7f6027fff640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 0x7f60100a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6018004770 tx=0x7f6018000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:37:13.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:13.945+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6020016030 con 0x7f6028102690 2024-09-05T06:37:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:14.120+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f60281a28f0 con 0x7f60100a2b80 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:14.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:14.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:14 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.522 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:14 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:14.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:14 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi086 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 4272M 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] boot 2024-09-05T06:37:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: osdmap e61: 10 total, 10 up, 10 in 2024-09-05T06:37:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:15.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi086 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 4272M 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] boot 2024-09-05T06:37:15.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: osdmap e61: 10 total, 10 up, 10 in 2024-09-05T06:37:15.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:37:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi086 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 4272M 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: osd.9 [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] boot 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: osdmap e61: 10 total, 10 up, 10 in 2024-09-05T06:37:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:37:16.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:16 smithi086 ceph-mon[30943]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:16 smithi086 ceph-mon[30943]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T06:37:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:16 smithi096 ceph-mon[34468]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:16 smithi096 ceph-mon[34468]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T06:37:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:16 smithi115 ceph-mon[34791]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-05T06:37:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:16 smithi115 ceph-mon[34791]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T06:37:17.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/3998432128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]': finished 2024-09-05T06:37:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: osdmap e63: 11 total, 10 up, 11 in 2024-09-05T06:37:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.96:0/1543941076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/3998432128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]': finished 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: osdmap e63: 11 total, 10 up, 11 in 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.96:0/1543941076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/3998432128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]: dispatch 2024-09-05T06:37:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9e00a49-77d5-4256-8ec1-9f43c99a1326"}]': finished 2024-09-05T06:37:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: osdmap e63: 11 total, 10 up, 11 in 2024-09-05T06:37:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.96:0/1543941076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:18.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:18 smithi086 ceph-mon[30943]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:18 smithi096 ceph-mon[34468]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:18 smithi115 ceph-mon[34791]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:20.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:20 smithi115 ceph-mon[34791]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:20.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:20 smithi086 ceph-mon[30943]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:20.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:20 smithi096 ceph-mon[34468]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:22.411 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:22 smithi096 ceph-mon[34468]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:22 smithi086 ceph-mon[30943]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:22 smithi115 ceph-mon[34791]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:24.472 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:24 smithi096 ceph-mon[34468]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:24.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:24 smithi086 ceph-mon[30943]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:24 smithi115 ceph-mon[34791]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:25.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T06:37:25.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T06:37:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T06:37:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:26.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:26 smithi096 ceph-mon[34468]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:26 smithi096 ceph-mon[34468]: Deploying daemon osd.10 on smithi096 2024-09-05T06:37:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:26 smithi115 ceph-mon[34791]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:26.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:26 smithi115 ceph-mon[34791]: Deploying daemon osd.10 on smithi096 2024-09-05T06:37:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:26 smithi086 ceph-mon[30943]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:26 smithi086 ceph-mon[30943]: Deploying daemon osd.10 on smithi096 2024-09-05T06:37:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:28 smithi096 ceph-mon[34468]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:28 smithi115 ceph-mon[34791]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:28.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:28 smithi086 ceph-mon[30943]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:30.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:30 smithi096 ceph-mon[34468]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:30 smithi115 ceph-mon[34791]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:30.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:30 smithi086 ceph-mon[30943]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:31.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:31 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:31.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:31 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:31.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:31 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:31 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:31 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:31.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:31 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:31.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:31 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:31 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:31 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:32.511 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:32 smithi096 ceph-mon[34468]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:32 smithi115 ceph-mon[34791]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:32.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:32 smithi086 ceph-mon[30943]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:34.543 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-09-05T06:37:34.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.543+0000 7f60257fa640 1 -- 172.21.15.96:0/4000832931 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f60281a28f0 con 0x7f60100a2b80 2024-09-05T06:37:34.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:34.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:34.545 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:34.546 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:34 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 msgr2=0x7f60100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 0x7f60100a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6018004770 tx=0x7f6018000bc0 comp rx=0 tx=0).stop 2024-09-05T06:37:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 msgr2=0x7f60281035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f602000eaa0 tx=0x7f602000ef60 comp rx=0 tx=0).stop 2024-09-05T06:37:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 shutdown_connections 2024-09-05T06:37:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60100a2b80 0x7f60100a5040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6028102690 0x7f60281035f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6028101cd0 0x7f60281030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 --2- 172.21.15.96:0/4000832931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6028100ad0 0x7f6028104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 >> 172.21.15.96:0/4000832931 conn(0x7f60280fc280 msgr2=0x7f6028069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:37:34.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 shutdown_connections 2024-09-05T06:37:34.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-05T06:37:34.545+0000 7f602e84f640 1 -- 172.21.15.96:0/4000832931 wait complete. 2024-09-05T06:37:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:34.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:34 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:34.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:34 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.271 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.10.service 2024-09-05T06:37:35.273 INFO:tasks.cephadm:Deploying osd.11 on smithi115 with /dev/vg_nvme/lv_1... 2024-09-05T06:37:35.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T06:37:35.530 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:37:35.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:35.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.652 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:35.652 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.652 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:36.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:36 smithi096 ceph-mon[34468]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:36.672 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:36 smithi115 ceph-mon[34791]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:36 smithi086 ceph-mon[30943]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:36.860 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 06:37:36 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T06:37:36.648+0000 7fe60a00a640 -1 osd.10 0 log_to_monitors true 2024-09-05T06:37:37.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:37 smithi115 ceph-mon[34791]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:37.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:37 smithi115 ceph-mon[34791]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:37.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:37 smithi096 ceph-mon[34468]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:37 smithi096 ceph-mon[34468]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:37.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:37 smithi086 ceph-mon[30943]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:37 smithi086 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T06:37:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-05T06:37:38.424 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-09-05T06:37:38.556 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T06:37:38.276+0000 7fe605f8d640 -1 osd.10 0 waiting for initial osdmap 2024-09-05T06:37:38.557 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T06:37:38.285+0000 7fe601db5640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:38.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:38 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.670 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:37:38.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:38.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:38.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:38.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:38 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:38.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:37:38.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:38 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 -- 172.21.15.115:0/1363607657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c1026d0 msgr2=0x7f2a6c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 --2- 172.21.15.115:0/1363607657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c108f60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009ca0 tx=0x7f2a6402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 -- 172.21.15.115:0/1363607657 shutdown_connections 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 --2- 172.21.15.115:0/1363607657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c108f60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 --2- 172.21.15.115:0/1363607657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c101d10 0x7f2a6c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 --2- 172.21.15.115:0/1363607657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 0x7f2a6c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 -- 172.21.15.115:0/1363607657 >> 172.21.15.115:0/1363607657 conn(0x7f2a6c0fc2a0 msgr2=0x7f2a6c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 -- 172.21.15.115:0/1363607657 shutdown_connections 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.148+0000 7f2a746ed640 1 -- 172.21.15.115:0/1363607657 wait complete. 2024-09-05T06:37:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 Processor -- start 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 -- start start 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 0x7f2a6c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c10d7d0 con 0x7f2a6c1026d0 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c10d650 con 0x7f2a6c100b10 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a746ed640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c10d950 con 0x7f2a6c101d10 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41706/0 (socket says 172.21.15.115:41706) 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 -- 172.21.15.115:0/3085128370 learned_addr learned my addr 172.21.15.115:0/3085128370 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a72489640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 0x7f2a6c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a72c8a640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 -- 172.21.15.115:0/3085128370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 msgr2=0x7f2a6c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 0x7f2a6c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 -- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 msgr2=0x7f2a6c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 -- 172.21.15.115:0/3085128370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c19faa0 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a72c8a640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.149+0000 7f2a71c88640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d950 tx=0x7f2a5c00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.150+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.150+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a5c00de80 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.150+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00f620 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.150+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c19fd90 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.150+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a0220 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.151+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c107f10 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.152+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a5c009c30 con 0x7f2a6c101d10 2024-09-05T06:37:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.152+0000 7f2a637fe640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 0x7f2a580a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:37:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.152+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7f2a5c0ce480 con 0x7f2a6c101d10 2024-09-05T06:37:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.153+0000 7f2a72489640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 0x7f2a580a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:37:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.153+0000 7f2a72489640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 0x7f2a580a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c101b70 tx=0x7f2a68009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:37:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.156+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a5c09bb50 con 0x7f2a6c101d10 2024-09-05T06:37:39.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:39.333+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2a6c062630 con 0x7f2a580a2bb0 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi096 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] boot 2024-09-05T06:37:39.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: osdmap e66: 11 total, 11 up, 11 in 2024-09-05T06:37:39.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:39 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi096 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] boot 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: osdmap e66: 11 total, 11 up, 11 in 2024-09-05T06:37:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:39 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi096 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: osd.10 [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] boot 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: osdmap e66: 11 total, 11 up, 11 in 2024-09-05T06:37:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:39 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:37:40.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:40 smithi096 ceph-mon[34468]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T06:37:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:40 smithi115 ceph-mon[34791]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T06:37:40.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:40 smithi086 ceph-mon[30943]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T06:37:41.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:41 smithi115 ceph-mon[34791]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:41.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:41 smithi115 ceph-mon[34791]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T06:37:41.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:41 smithi096 ceph-mon[34468]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:41 smithi096 ceph-mon[34468]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T06:37:41.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:41 smithi086 ceph-mon[30943]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:37:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:41 smithi086 ceph-mon[30943]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T06:37:42.425 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:42.425 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T06:37:42.425 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/2965715232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.425 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.425 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]': finished 2024-09-05T06:37:42.426 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T06:37:42.426 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:42 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:37:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/2965715232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]': finished 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T06:37:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:42 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:37:42.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/2965715232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]: dispatch 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfda802c-c8b3-4230-81b0-48a3a331b1cf"}]': finished 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: osdmap e70: 12 total, 11 up, 12 in 2024-09-05T06:37:42.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:42 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:37:43.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:43 smithi096 ceph-mon[34468]: from='client.? 172.21.15.115:0/2931477180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.115:0/2931477180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:43.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:43 smithi086 ceph-mon[30943]: from='client.? 172.21.15.115:0/2931477180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T06:37:44.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:44 smithi096 ceph-mon[34468]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:44.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:44 smithi086 ceph-mon[30943]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:44 smithi115 ceph-mon[34791]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:46 smithi096 ceph-mon[34468]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T06:37:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:46 smithi115 ceph-mon[34791]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T06:37:46.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:46 smithi086 ceph-mon[30943]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T06:37:48.530 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:48 smithi115 ceph-mon[34791]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:37:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:48 smithi096 ceph-mon[34468]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:37:48.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:48 smithi086 ceph-mon[30943]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T06:37:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:50 smithi096 ceph-mon[34468]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T06:37:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:50 smithi115 ceph-mon[34791]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T06:37:50.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:50 smithi086 ceph-mon[30943]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T06:37:51.362 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T06:37:51.362 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:51 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:51.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T06:37:51.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:51 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T06:37:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:51 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:37:52.528 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:52 smithi115 ceph-mon[34791]: Deploying daemon osd.11 on smithi115 2024-09-05T06:37:52.528 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:52 smithi115 ceph-mon[34791]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T06:37:52.586 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:52 smithi096 ceph-mon[34468]: Deploying daemon osd.11 on smithi115 2024-09-05T06:37:52.586 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:52 smithi096 ceph-mon[34468]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T06:37:52.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:52 smithi086 ceph-mon[30943]: Deploying daemon osd.11 on smithi115 2024-09-05T06:37:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:52 smithi086 ceph-mon[30943]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T06:37:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:54 smithi115 ceph-mon[34791]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:37:54.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:54 smithi086 ceph-mon[30943]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:37:54.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:54 smithi096 ceph-mon[34468]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T06:37:56.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:56 smithi115 ceph-mon[34791]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:56.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:56.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:56.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:56 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:56.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:56 smithi086 ceph-mon[30943]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:56.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:56.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:56.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:56 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:56.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:56 smithi096 ceph-mon[34468]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T06:37:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:37:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:56 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:37:57.495 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:57 smithi115 ceph-mon[34791]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:57.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:57 smithi086 ceph-mon[30943]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:57 smithi096 ceph-mon[34468]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:37:59.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.647+0000 7f2a637fe640 1 -- 172.21.15.115:0/3085128370 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2a6c062630 con 0x7f2a580a2bb0 2024-09-05T06:37:59.649 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 11 on host 'smithi115' 2024-09-05T06:37:59.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 msgr2=0x7f2a580a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:59.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 0x7f2a580a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c101b70 tx=0x7f2a68009110 comp rx=0 tx=0).stop 2024-09-05T06:37:59.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 msgr2=0x7f2a6c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d950 tx=0x7f2a5c00dd50 comp rx=0 tx=0).stop 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 shutdown_connections 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a6c1026d0 0x7f2a6c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a6c101d10 0x7f2a6c1983b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a580a2bb0 0x7f2a580a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 --2- 172.21.15.115:0/3085128370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a6c100b10 0x7f2a6c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 >> 172.21.15.115:0/3085128370 conn(0x7f2a6c0fc2a0 msgr2=0x7f2a6c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 shutdown_connections 2024-09-05T06:37:59.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:37:59.650+0000 7f2a746ed640 1 -- 172.21.15.115:0/3085128370 wait complete. 2024-09-05T06:38:00.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:37:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:37:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.289 DEBUG:teuthology.orchestra.run.smithi115:osd.11> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.11.service 2024-09-05T06:38:00.291 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-05T06:38:00.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T06:38:00.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:37:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:00.554 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:01.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1856702733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 msgr2=0x7fbe64108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1856702733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe64108f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009ca0 tx=0x7fbe5c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1856702733 shutdown_connections 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1856702733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe64108f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1856702733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64101cd0 0x7fbe64102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.044+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1856702733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64100ad0 0x7fbe64100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1856702733 >> 172.21.15.86:0/1856702733 conn(0x7fbe640fc280 msgr2=0x7fbe640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1856702733 shutdown_connections 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1856702733 wait complete. 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 Processor -- start 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- start start 2024-09-05T06:38:01.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 0x7fbe64197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 0x7fbe641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6410c600 con 0x7fbe64102690 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6410c480 con 0x7fbe64100ad0 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6c43c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6410c780 con 0x7fbe64101cd0 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6a9d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6a9d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51378/0 (socket says 172.21.15.86:51378) 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe6a9d9640 1 -- 172.21.15.86:0/1072939734 learned_addr learned my addr 172.21.15.86:0/1072939734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.045+0000 7fbe699d7640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 0x7fbe641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a1d8640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 0x7fbe64197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 -- 172.21.15.86:0/1072939734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 msgr2=0x7fbe641982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:01.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 0x7fbe641982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 -- 172.21.15.86:0/1072939734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 msgr2=0x7fbe64197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 0x7fbe64197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 -- 172.21.15.86:0/1072939734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6419f9d0 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe699d7640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 0x7fbe641982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a1d8640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 0x7fbe64197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6a9d9640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c031970 tx=0x7fbe5c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004040 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6419fc60 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe5c004690 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c030c60 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.046+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641a01a0 con 0x7fbe64102690 2024-09-05T06:38:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.048+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe641a0490 con 0x7fbe64102690 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.049+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe5c0041e0 con 0x7fbe64102690 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.049+0000 7fbe5b7fe640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 0x7fbe500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.050+0000 7fbe6a1d8640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 0x7fbe500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.050+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7fbe5c0eff30 con 0x7fbe64102690 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.050+0000 7fbe6a1d8640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 0x7fbe500a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe64101b30 tx=0x7fbe60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.054+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe5c0bd470 con 0x7fbe64102690 2024-09-05T06:38:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.228+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbe641a0660 con 0x7fbe64102690 2024-09-05T06:38:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.229+0000 7fbe5b7fe640 1 -- 172.21.15.86:0/1072939734 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fbe5c0bcd70 con 0x7fbe64102690 2024-09-05T06:38:01.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:01.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.231+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 msgr2=0x7fbe500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 0x7fbe500a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe64101b30 tx=0x7fbe60005f90 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 msgr2=0x7fbe6419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c031970 tx=0x7fbe5c003ba0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 shutdown_connections 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe64102690 0x7fbe6419f360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe64101cd0 0x7fbe641982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fbe500a2bb0 0x7fbe500a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 --2- 172.21.15.86:0/1072939734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe64100ad0 0x7fbe64197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 >> 172.21.15.86:0/1072939734 conn(0x7fbe640fc280 msgr2=0x7fbe640fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 shutdown_connections 2024-09-05T06:38:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:01.232+0000 7fbe6c43c640 1 -- 172.21.15.86:0/1072939734 wait complete. 2024-09-05T06:38:02.004 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1725518259,"num_in_osds":12,"osd_in_since":1725518261,"num_remapped_pgs":0} 2024-09-05T06:38:02.183 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 06:38:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T06:38:01.831+0000 7f3f2de3b640 -1 osd.11 0 log_to_monitors true 2024-09-05T06:38:02.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:02 smithi086 ceph-mon[30943]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:02.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:02 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1072939734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:02.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:02 smithi086 ceph-mon[30943]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:02.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:02 smithi086 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:02.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:02 smithi115 ceph-mon[34791]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:02.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:02 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1072939734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:02.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:02 smithi115 ceph-mon[34791]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:02.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:02 smithi115 ceph-mon[34791]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:02.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:02 smithi096 ceph-mon[34468]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:02 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1072939734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:02 smithi096 ceph-mon[34468]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:02 smithi096 ceph-mon[34468]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T06:38:03.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T06:38:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:03.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:03 smithi086 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T06:38:03.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:03 smithi086 ceph-mon[30943]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T06:38:03.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:03 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:03.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:03 smithi086 ceph-mon[30943]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:03 smithi086 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:03 smithi096 ceph-mon[34468]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T06:38:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:03 smithi096 ceph-mon[34468]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T06:38:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:03 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:03 smithi096 ceph-mon[34468]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:03 smithi096 ceph-mon[34468]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-mon[34791]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T06:38:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-mon[34791]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T06:38:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-mon[34791]: from='osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-mon[34791]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-05T06:38:03.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- 172.21.15.86:0/1844822877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 msgr2=0x7f9440100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:03.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 --2- 172.21.15.86:0/1844822877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440100ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9430009900 tx=0x7f943002a410 comp rx=0 tx=0).stop 2024-09-05T06:38:03.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- 172.21.15.86:0/1844822877 shutdown_connections 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 --2- 172.21.15.86:0/1844822877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440102690 0x7f9440108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 --2- 172.21.15.86:0/1844822877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440101cd0 0x7f9440102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 --2- 172.21.15.86:0/1844822877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440100ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- 172.21.15.86:0/1844822877 >> 172.21.15.86:0/1844822877 conn(0x7f94400fc280 msgr2=0x7f94400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- 172.21.15.86:0/1844822877 shutdown_connections 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- 172.21.15.86:0/1844822877 wait complete. 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 Processor -- start 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.801+0000 7f94464c5640 1 -- start start 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440101cd0 0x7f94401030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 0x7f94401035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f944010c680 con 0x7f9440100ad0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f944010c500 con 0x7f9440102690 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f944010c800 con 0x7f9440101cd0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59528/0 (socket says 172.21.15.86:59528) 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 -- 172.21.15.86:0/3891884474 learned_addr learned my addr 172.21.15.86:0/3891884474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943f7fe640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440101cd0 0x7f94401030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f9444a62640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 0x7f94401035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 -- 172.21.15.86:0/3891884474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440101cd0 msgr2=0x7f94401030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440101cd0 0x7f94401030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 -- 172.21.15.86:0/3891884474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 msgr2=0x7f94401035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 0x7f94401035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 -- 172.21.15.86:0/3891884474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9440103eb0 con 0x7f9440100ad0 2024-09-05T06:38:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f9444a62640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 0x7f94401035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943ffff640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94300314e0 tx=0x7f9430031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943003f030 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9430003aa0 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94401a2090 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.802+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430035070 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.803+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94401a25c0 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.803+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94401a27f0 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.805+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9430042030 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.806+0000 7f943d7fa640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.806+0000 7f943f7fe640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.806+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7f94300ef3e0 con 0x7f9440100ad0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.806+0000 7f943f7fe640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 0x7f94280a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f94401047e0 tx=0x7f9434009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.810+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94300bc900 con 0x7f9440100ad0 2024-09-05T06:38:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.978+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f94401a29c0 con 0x7f9440100ad0 2024-09-05T06:38:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.978+0000 7f943d7fa640 1 -- 172.21.15.86:0/3891884474 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f94300bc200 con 0x7f9440100ad0 2024-09-05T06:38:03.979 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 msgr2=0x7f94280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 0x7f94280a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f94401047e0 tx=0x7f9434009110 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 msgr2=0x7f9440104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94300314e0 tx=0x7f9430031a30 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 shutdown_connections 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9440102690 0x7f94401035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9440101cd0 0x7f94401030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 --2- 172.21.15.86:0/3891884474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9440100ad0 0x7f9440104a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.980+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 >> 172.21.15.86:0/3891884474 conn(0x7f94400fc280 msgr2=0x7f944006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.981+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 shutdown_connections 2024-09-05T06:38:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:03.981+0000 7f94464c5640 1 -- 172.21.15.86:0/3891884474 wait complete. 2024-09-05T06:38:04.182 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T06:38:03.890+0000 7f3f2a5d2640 -1 osd.11 0 waiting for initial osdmap 2024-09-05T06:38:04.182 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 06:38:03 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T06:38:03.901+0000 7f3f21bde640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: Detected new or changed devices on smithi115 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='osd.11 ' entity='osd.11' 2024-09-05T06:38:04.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:04 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3891884474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: Detected new or changed devices on smithi115 2024-09-05T06:38:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='osd.11 ' entity='osd.11' 2024-09-05T06:38:04.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:04 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3891884474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:04.677 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1725518259,"num_in_osds":12,"osd_in_since":1725518261,"num_remapped_pgs":0} 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: Detected new or changed devices on smithi115 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='osd.11 ' entity='osd.11' 2024-09-05T06:38:04.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:04 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3891884474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:05.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: purged_snaps scrub starts 2024-09-05T06:38:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: purged_snaps scrub ok 2024-09-05T06:38:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] boot 2024-09-05T06:38:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: osdmap e73: 12 total, 12 up, 12 in 2024-09-05T06:38:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:05 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: purged_snaps scrub starts 2024-09-05T06:38:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: purged_snaps scrub ok 2024-09-05T06:38:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] boot 2024-09-05T06:38:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: osdmap e73: 12 total, 12 up, 12 in 2024-09-05T06:38:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:05 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T06:38:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: purged_snaps scrub starts 2024-09-05T06:38:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: purged_snaps scrub ok 2024-09-05T06:38:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: osd.11 [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] boot 2024-09-05T06:38:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: osdmap e73: 12 total, 12 up, 12 in 2024-09-05T06:38:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:05 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:38:05.922 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:06.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:06 smithi086 ceph-mon[30943]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:06.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 -- 172.21.15.86:0/1582718766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 msgr2=0x7fe770109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1582718766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770109d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe768009ca0 tx=0x7fe76802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 -- 172.21.15.86:0/1582718766 shutdown_connections 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1582718766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770109d50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1582718766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 0x7fe7701073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1582718766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 -- 172.21.15.86:0/1582718766 >> 172.21.15.86:0/1582718766 conn(0x7fe7700fc2a0 msgr2=0x7fe7700fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 -- 172.21.15.86:0/1582718766 shutdown_connections 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 -- 172.21.15.86:0/1582718766 wait complete. 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.367+0000 7fe7760cf640 1 Processor -- start 2024-09-05T06:38:06.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 -- start start 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 0x7fe770100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010d770 con 0x7fe770107900 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010d5f0 con 0x7fe770104fd0 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe7760cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010d8f0 con 0x7fe7701026a0 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59538/0 (socket says 172.21.15.86:59538) 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 -- 172.21.15.86:0/1752125171 learned_addr learned my addr 172.21.15.86:0/1752125171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76effd640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 0x7fe770100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76f7fe640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 -- 172.21.15.86:0/1752125171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 msgr2=0x7fe770102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:06.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 -- 172.21.15.86:0/1752125171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 msgr2=0x7fe770100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 0x7fe770100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 -- 172.21.15.86:0/1752125171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770101450 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76f7fe640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.368+0000 7fe76ffff640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe768009dc0 tx=0x7fe768003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.369+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004040 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.369+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7701a1ed0 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.369+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe768004690 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.369+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76802f4c0 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.369+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7701a23d0 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.370+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7700fdd90 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.371+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe768030650 con 0x7fe770107900 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.371+0000 7fe76cff9640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 0x7fe7580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.371+0000 7fe76f7fe640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 0x7fe7580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.372+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fe7680f03a0 con 0x7fe770107900 2024-09-05T06:38:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.372+0000 7fe76f7fe640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 0x7fe7580a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7600058c0 tx=0x7fe760005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.375+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe76804d3a0 con 0x7fe770107900 2024-09-05T06:38:06.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.541+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe77002d970 con 0x7fe770107900 2024-09-05T06:38:06.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.541+0000 7fe76cff9640 1 -- 172.21.15.86:0/1752125171 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fe7680bd7a0 con 0x7fe770107900 2024-09-05T06:38:06.542 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:06.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.543+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 msgr2=0x7fe7580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.543+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 0x7fe7580a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7600058c0 tx=0x7fe760005d30 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.543+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 msgr2=0x7fe770100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.543+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe768009dc0 tx=0x7fe768003ba0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 shutdown_connections 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770107900 0x7fe770100b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe7580a2b80 0x7fe7580a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe770104fd0 0x7fe770100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 --2- 172.21.15.86:0/1752125171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7701026a0 0x7fe770102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 >> 172.21.15.86:0/1752125171 conn(0x7fe7700fc2a0 msgr2=0x7fe770102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:06.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 shutdown_connections 2024-09-05T06:38:06.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:06.544+0000 7fe7760cf640 1 -- 172.21.15.86:0/1752125171 wait complete. 2024-09-05T06:38:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:06 smithi096 ceph-mon[34468]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:06 smithi115 ceph-mon[34791]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-05T06:38:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1725518284,"num_in_osds":12,"osd_in_since":1725518261,"num_remapped_pgs":0} 2024-09-05T06:38:07.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:38:07.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:07 smithi086 ceph-mon[30943]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T06:38:07.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:07 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1752125171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:07.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:07 smithi096 ceph-mon[34468]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T06:38:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:07 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1752125171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:07 smithi115 ceph-mon[34791]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T06:38:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:07 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1752125171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T06:38:08.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 -- 172.21.15.86:0/220852465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 msgr2=0x7f0128103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 --2- 172.21.15.86:0/220852465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128103180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0118009900 tx=0x7f011802a480 comp rx=0 tx=0).stop 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 -- 172.21.15.86:0/220852465 shutdown_connections 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 --2- 172.21.15.86:0/220852465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 --2- 172.21.15.86:0/220852465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 0x7f012810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 --2- 172.21.15.86:0/220852465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128103180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 -- 172.21.15.86:0/220852465 >> 172.21.15.86:0/220852465 conn(0x7f01280fc7a0 msgr2=0x7f01280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 -- 172.21.15.86:0/220852465 shutdown_connections 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.000+0000 7f012f8b5640 1 -- 172.21.15.86:0/220852465 wait complete. 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 Processor -- start 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 -- start start 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 0x7f0128198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111e10 con 0x7f0128102da0 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111c90 con 0x7f012810c180 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012f8b5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111f90 con 0x7f01281036c0 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59554/0 (socket says 172.21.15.86:59554) 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 -- 172.21.15.86:0/1470285523 learned_addr learned my addr 172.21.15.86:0/1470285523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012de52640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012ce50640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 0x7f0128198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 -- 172.21.15.86:0/1470285523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 msgr2=0x7f0128198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 0x7f0128198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 -- 172.21.15.86:0/1470285523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 msgr2=0x7f012819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 -- 172.21.15.86:0/1470285523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012819d2b0 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012de52640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.001+0000 7f012d651640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0118006010 tx=0x7f0118004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.002+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011803f030 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.002+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012819d540 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.002+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01180035f0 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.002+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118035070 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.002+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281a4e80 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.003+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0128103180 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.005+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0118003b10 con 0x7f0128102da0 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.005+0000 7f011e7fc640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 0x7f01140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.005+0000 7f012ce50640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 0x7f01140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:08.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.005+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f01180ef9e0 con 0x7f0128102da0 2024-09-05T06:38:08.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.006+0000 7f012ce50640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 0x7f01140a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0124005e10 tx=0x7f0124005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:08.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.009+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01180bcde0 con 0x7f0128102da0 2024-09-05T06:38:08.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.177+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0128107220 con 0x7f0128102da0 2024-09-05T06:38:08.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.178+0000 7f011e7fc640 1 -- 172.21.15.86:0/1470285523 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18431 (secure 0 0 0) 0x7f01180bc6e0 con 0x7f0128102da0 2024-09-05T06:38:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:08.178 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:38:06.229274+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 msgr2=0x7f01140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 0x7f01140a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0124005e10 tx=0x7f0124005da0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 msgr2=0x7f0128198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0118006010 tx=0x7f0118004270 comp rx=0 tx=0).stop 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 shutdown_connections 2024-09-05T06:38:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f01140a2b80 0x7f01140a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f012810c180 0x7f012819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01281036c0 0x7f0128198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 --2- 172.21.15.86:0/1470285523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0128102da0 0x7f0128198300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 >> 172.21.15.86:0/1470285523 conn(0x7f01280fc7a0 msgr2=0x7f01280fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 shutdown_connections 2024-09-05T06:38:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:08.180+0000 7f012f8b5640 1 -- 172.21.15.86:0/1470285523 wait complete. 2024-09-05T06:38:08.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:08 smithi086 ceph-mon[30943]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:08.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:08 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1470285523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:38:08.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:08 smithi096 ceph-mon[34468]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:08.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:08 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1470285523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:38:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:08 smithi115 ceph-mon[34791]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:08 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1470285523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:38:08.946 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T06:34:21.377500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T06:38:08.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-05T06:38:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/1403727721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e581036c0 msgr2=0x7f9e5810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/1403727721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e581036c0 0x7f9e5810bbb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009900 tx=0x7f9e4c02a480 comp rx=0 tx=0).stop 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/1403727721 shutdown_connections 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/1403727721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e5810c180 0x7f9e5810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/1403727721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e581036c0 0x7f9e5810bbb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/1403727721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e58102da0 0x7f9e58103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/1403727721 >> 172.21.15.86:0/1403727721 conn(0x7f9e580fc7a0 msgr2=0x7f9e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/1403727721 shutdown_connections 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.679+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/1403727721 wait complete. 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 Processor -- start 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 -- start start 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e581036c0 0x7f9e5819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c180 0x7f9e5819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111e50 con 0x7f9e58102da0 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111cd0 con 0x7f9e5810c180 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e5d2a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111fd0 con 0x7f9e581036c0 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59578/0 (socket says 172.21.15.86:59578) 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 -- 172.21.15.86:0/3507890366 learned_addr learned my addr 172.21.15.86:0/3507890366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 -- 172.21.15.86:0/3507890366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e581036c0 msgr2=0x7f9e5819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e581036c0 0x7f9e5819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 -- 172.21.15.86:0/3507890366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c180 msgr2=0x7f9e5819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c180 0x7f9e5819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.680+0000 7f9e56ffd640 1 -- 172.21.15.86:0/3507890366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5819f4f0 con 0x7f9e58102da0 2024-09-05T06:38:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e56ffd640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cce0 tx=0x7f9e5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000f650 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5819f7e0 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e5000fca0 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50018b90 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.681+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a70c0 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.682+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e58103180 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.683+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e5000f7f0 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.684+0000 7f9e3ffff640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 0x7f9e400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.684+0000 7f9e567fc640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 0x7f9e400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.684+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f9e500cf110 con 0x7f9e58102da0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.684+0000 7f9e567fc640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 0x7f9e400a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0096c0 tx=0x7f9e4c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.688+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e50014030 con 0x7f9e58102da0 2024-09-05T06:38:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.853+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e581a7470 con 0x7f9e58102da0 2024-09-05T06:38:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.854+0000 7f9e3ffff640 1 -- 172.21.15.86:0/3507890366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f9e5009c510 con 0x7f9e58102da0 2024-09-05T06:38:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 msgr2=0x7f9e400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 0x7f9e400a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0096c0 tx=0x7f9e4c031400 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 msgr2=0x7f9e5819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cce0 tx=0x7f9e5000a620 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 shutdown_connections 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e400a2b80 0x7f9e400a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c180 0x7f9e5819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e581036c0 0x7f9e5819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 --2- 172.21.15.86:0/3507890366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58102da0 0x7f9e5819a540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 >> 172.21.15.86:0/3507890366 conn(0x7f9e580fc7a0 msgr2=0x7f9e58101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 shutdown_connections 2024-09-05T06:38:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:09.856+0000 7f9e5d2a9640 1 -- 172.21.15.86:0/3507890366 wait complete. 2024-09-05T06:38:10.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:10 smithi086 ceph-mon[30943]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:10.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3507890366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.c on smithi115 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.f on smithi115 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.i on smithi115 2024-09-05T06:38:10.553 INFO:tasks.cephadm:Adding mds.l on smithi115 2024-09-05T06:38:10.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l' 2024-09-05T06:38:10.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:10 smithi096 ceph-mon[34468]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3507890366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:38:10.582 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:10 smithi115 ceph-mon[34791]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:10.582 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3507890366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:38:10.811 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T06:38:11.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- 172.21.15.115:0/2715274593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530107960 msgr2=0x7f2530109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 --2- 172.21.15.115:0/2715274593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530107960 0x7f2530109d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2528009ca0 tx=0x7f252802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- 172.21.15.115:0/2715274593 shutdown_connections 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 --2- 172.21.15.115:0/2715274593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530107960 0x7f2530109d50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 --2- 172.21.15.115:0/2715274593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530105030 0x7f2530107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 --2- 172.21.15.115:0/2715274593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530102700 0x7f2530104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- 172.21.15.115:0/2715274593 >> 172.21.15.115:0/2715274593 conn(0x7f25300fc280 msgr2=0x7f25300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- 172.21.15.115:0/2715274593 shutdown_connections 2024-09-05T06:38:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- 172.21.15.115:0/2715274593 wait complete. 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 Processor -- start 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.303+0000 7f2535d09640 1 -- start start 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 0x7f2530102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530107960 0x7f2530100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f253010d710 con 0x7f2530102700 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f253010d590 con 0x7f2530107960 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f2535d09640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f253010d890 con 0x7f2530105030 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33660/0 (socket says 172.21.15.115:33660) 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 -- 172.21.15.115:0/3011879025 learned_addr learned my addr 172.21.15.115:0/3011879025 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252f7fe640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 0x7f2530102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252ffff640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530107960 0x7f2530100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 -- 172.21.15.115:0/3011879025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530107960 msgr2=0x7f2530100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530107960 0x7f2530100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 -- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 msgr2=0x7f2530102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 0x7f2530102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 -- 172.21.15.115:0/3011879025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25301014b0 con 0x7f2530105030 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252f7fe640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 0x7f2530102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:38:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252effd640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.304+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252400cd80 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.305+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25240054d0 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.305+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524010430 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.305+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25301a1f90 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.305+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25301a23f0 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.306+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25300fdd70 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.308+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2524004fb0 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.308+0000 7f252cff9640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.308+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f25240cf1b0 con 0x7f2530105030 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.309+0000 7f252f7fe640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.309+0000 7f252f7fe640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 0x7f25180a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f252000bfd0 tx=0x7f252000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.311+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25240541b0 con 0x7f2530105030 2024-09-05T06:38:11.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.494+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}) -- 0x7f2530062730 con 0x7f25180a2b80 2024-09-05T06:38:11.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.503+0000 7f252cff9640 1 -- 172.21.15.115:0/3011879025 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2530062730 con 0x7f25180a2b80 2024-09-05T06:38:11.503 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mds.all update... 2024-09-05T06:38:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 msgr2=0x7f25180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 0x7f25180a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f252000bfd0 tx=0x7f252000b390 comp rx=0 tx=0).stop 2024-09-05T06:38:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 msgr2=0x7f25301006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ee80 comp rx=0 tx=0).stop 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 shutdown_connections 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2530107960 0x7f2530100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2530105030 0x7f25301006b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 --2- 172.21.15.115:0/3011879025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2530102700 0x7f2530102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.505+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 >> 172.21.15.115:0/3011879025 conn(0x7f25300fc280 msgr2=0x7f25301031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.506+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 shutdown_connections 2024-09-05T06:38:11.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-05T06:38:11.506+0000 7f2535d09640 1 -- 172.21.15.115:0/3011879025 wait complete. 2024-09-05T06:38:12.294 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.a.service 2024-09-05T06:38:12.296 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.d.service 2024-09-05T06:38:12.339 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.g.service 2024-09-05T06:38:12.383 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.j.service 2024-09-05T06:38:12.426 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.b.service 2024-09-05T06:38:12.428 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.e.service 2024-09-05T06:38:12.471 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.h.service 2024-09-05T06:38:12.514 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.k.service 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:12.533 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.534 DEBUG:teuthology.orchestra.run.smithi115:mds.c> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.c.service 2024-09-05T06:38:12.536 DEBUG:teuthology.orchestra.run.smithi115:mds.f> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.f.service 2024-09-05T06:38:12.556 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:12.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.558 DEBUG:teuthology.orchestra.run.smithi115:mds.i> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.i.service 2024-09-05T06:38:12.600 DEBUG:teuthology.orchestra.run.smithi115:mds.l> sudo journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.l.service 2024-09-05T06:38:12.643 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-05T06:38:12.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:12.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:12.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:13.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 -- 172.21.15.86:0/1192549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100e30 msgr2=0x7f4afc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 --2- 172.21.15.86:0/1192549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100e30 0x7f4afc104e80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009900 tx=0x7f4aec02a410 comp rx=0 tx=0).stop 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 -- 172.21.15.86:0/1192549595 shutdown_connections 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 --2- 172.21.15.86:0/1192549595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 --2- 172.21.15.86:0/1192549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100e30 0x7f4afc104e80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 --2- 172.21.15.86:0/1192549595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100480 0x7f4afc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 -- 172.21.15.86:0/1192549595 >> 172.21.15.86:0/1192549595 conn(0x7f4afc077210 msgr2=0x7f4afc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 -- 172.21.15.86:0/1192549595 shutdown_connections 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.367+0000 7f4b0442c640 1 -- 172.21.15.86:0/1192549595 wait complete. 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 Processor -- start 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 -- start start 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:13.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 0x7f4afc1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10fc90 con 0x7f4afc100480 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10fb10 con 0x7f4afc1055b0 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b0442c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10fe10 con 0x7f4afc100e30 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b021c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b021c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52260/0 (socket says 172.21.15.86:52260) 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.368+0000 7f4b021c8640 1 -- 172.21.15.86:0/3070685700 learned_addr learned my addr 172.21.15.86:0/3070685700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b019c7640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 0x7f4afc1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b029c9640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 -- 172.21.15.86:0/3070685700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 msgr2=0x7f4afc1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 0x7f4afc1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 -- 172.21.15.86:0/3070685700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 msgr2=0x7f4afc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:13.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 -- 172.21.15.86:0/3070685700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc19b160 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b019c7640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 0x7f4afc1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b029c9640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b021c8640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4af80078f0 tx=0x7f4af8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8019020 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc19b450 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af80054d0 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1a2d30 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.369+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800e830 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.370+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc100860 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.371+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4af800e9d0 con 0x7f4afc100480 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.372+0000 7f4af37fe640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 0x7f4ae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.372+0000 7f4b019c7640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 0x7f4ae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:13.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.372+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4af80d0260 con 0x7f4afc100480 2024-09-05T06:38:13.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.372+0000 7f4b019c7640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 0x7f4ae80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000c00 tx=0x7f4aec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:13.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.376+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4af8023020 con 0x7f4afc100480 2024-09-05T06:38:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:13 smithi096 ceph-mon[34468]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:38:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:13 smithi096 ceph-mon[34468]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-09-05T06:38:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:13 smithi096 ceph-mon[34468]: Deploying daemon mds.l on smithi115 2024-09-05T06:38:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.576+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4afc02d7c0 con 0x7f4afc100480 2024-09-05T06:38:13.576 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:13 smithi086 ceph-mon[30943]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:38:13.576 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:13 smithi086 ceph-mon[30943]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-09-05T06:38:13.576 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:13 smithi086 ceph-mon[30943]: Deploying daemon mds.l on smithi115 2024-09-05T06:38:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.576+0000 7f4af37fe640 1 -- 172.21.15.86:0/3070685700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f4af801b2b0 con 0x7f4afc100480 2024-09-05T06:38:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:13.577 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"btime":"2024-09-05T06:31:26:636593+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-05T06:38:13.577 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 msgr2=0x7f4ae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 0x7f4ae80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000c00 tx=0x7f4aec005a20 comp rx=0 tx=0).stop 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 msgr2=0x7f4afc1961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4af80078f0 tx=0x7f4af8007db0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 shutdown_connections 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4ae80a2b80 0x7f4ae80a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4afc1055b0 0x7f4afc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4afc100e30 0x7f4afc1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 --2- 172.21.15.86:0/3070685700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4afc100480 0x7f4afc1961b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 >> 172.21.15.86:0/3070685700 conn(0x7f4afc077210 msgr2=0x7f4afc103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 shutdown_connections 2024-09-05T06:38:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:13.579+0000 7f4b0442c640 1 -- 172.21.15.86:0/3070685700 wait complete. 2024-09-05T06:38:13.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:13 smithi115 ceph-mon[34791]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:38:13.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:13 smithi115 ceph-mon[34791]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-09-05T06:38:13.595 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:13 smithi115 ceph-mon[34791]: Deploying daemon mds.l on smithi115 2024-09-05T06:38:14.182 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 systemd[1]: Starting Ceph mds.l for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:14.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:14 smithi086 ceph-mon[30943]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:14 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3070685700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:14 smithi096 ceph-mon[34468]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:14 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3070685700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:14.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:14 smithi115 ceph-mon[34791]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:14.559 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:14 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3070685700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:14.559 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 podman[63982]: 2024-09-05 06:38:14.181992767 +0000 UTC m=+0.015376071 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:14.559 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 podman[63982]: 2024-09-05 06:38:14.307536021 +0000 UTC m=+0.140919310 container create 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:14.559 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 podman[63982]: 2024-09-05 06:38:14.560187805 +0000 UTC m=+0.393571093 container init 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T06:38:14.933 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 podman[63982]: 2024-09-05 06:38:14.562994265 +0000 UTC m=+0.396377564 container start 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T06:38:14.933 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 ceph-mds[64002]: -- 172.21.15.115:0/3979100330 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7e8c79860 con 0x55e7e8c57400 2024-09-05T06:38:14.933 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: starting mds.l at 2024-09-05T06:38:14.933 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 bash[63982]: 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 2024-09-05T06:38:14.933 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 06:38:14 smithi115 systemd[1]: Started Ceph mds.l for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:boot 2024-09-05T06:38:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: fsmap 1 up:standby 2024-09-05T06:38:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:38:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: Deploying daemon mds.k on smithi096 2024-09-05T06:38:15.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:15 smithi086 ceph-mon[30943]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:boot 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: fsmap 1 up:standby 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: Deploying daemon mds.k on smithi096 2024-09-05T06:38:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:15 smithi096 ceph-mon[34468]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:boot 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: fsmap 1 up:standby 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: Deploying daemon mds.k on smithi096 2024-09-05T06:38:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:15 smithi115 ceph-mon[34791]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:16.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:16.807 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:16 smithi096 systemd[1]: Starting Ceph mds.k for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 -- 172.21.15.86:0/436542638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 msgr2=0x7f626c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 --2- 172.21.15.86:0/436542638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c10eaa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6264009ca0 tx=0x7f626402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 -- 172.21.15.86:0/436542638 shutdown_connections 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 --2- 172.21.15.86:0/436542638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c10eaa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 --2- 172.21.15.86:0/436542638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 --2- 172.21.15.86:0/436542638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 0x7f626c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 -- 172.21.15.86:0/436542638 >> 172.21.15.86:0/436542638 conn(0x7f626c0fd8b0 msgr2=0x7f626c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 -- 172.21.15.86:0/436542638 shutdown_connections 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.977+0000 7f62708d2640 1 -- 172.21.15.86:0/436542638 wait complete. 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 Processor -- start 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 -- start start 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 0x7f626c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111e30 con 0x7f626c106f20 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111cb0 con 0x7f626c101df0 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f62708d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111fb0 con 0x7f626c1027a0 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f626ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f626ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52272/0 (socket says 172.21.15.86:52272) 2024-09-05T06:38:16.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f626ad9d640 1 -- 172.21.15.86:0/2564838875 learned_addr learned my addr 172.21.15.86:0/2564838875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.978+0000 7f626a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 0x7f626c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f6269d9b640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 -- 172.21.15.86:0/2564838875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 msgr2=0x7f626c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 -- 172.21.15.86:0/2564838875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 msgr2=0x7f626c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 0x7f626c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 -- 172.21.15.86:0/2564838875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c19d270 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f6269d9b640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f626ad9d640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6264009dc0 tx=0x7f6264032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264003cf0 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62640044f0 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c19d500 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264030bf0 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.979+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a4e40 con 0x7f626c106f20 2024-09-05T06:38:16.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.980+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c1021d0 con 0x7f626c106f20 2024-09-05T06:38:16.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.981+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6264004040 con 0x7f626c106f20 2024-09-05T06:38:16.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.982+0000 7f62637fe640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 0x7f62580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:16.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.982+0000 7f626a59c640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 0x7f62580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:16.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.982+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f62640f0180 con 0x7f626c106f20 2024-09-05T06:38:16.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.983+0000 7f626a59c640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 0x7f62580a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6254005d00 tx=0x7f625400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:16.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:16.986+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62640bd580 con 0x7f626c106f20 2024-09-05T06:38:17.175 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:16 smithi096 podman[63143]: 2024-09-05 06:38:16.793841218 +0000 UTC m=+0.015773493 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:17.175 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:16 smithi096 podman[63143]: 2024-09-05 06:38:16.906555245 +0000 UTC m=+0.128487506 container create 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-91c8a7e, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T06:38:17.175 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 podman[63143]: 2024-09-05 06:38:17.176234463 +0000 UTC m=+0.398166725 container init 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T06:38:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.184+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f626c109bd0 con 0x7f626c106f20 2024-09-05T06:38:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.185+0000 7f62637fe640 1 -- 172.21.15.86:0/2564838875 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f62640bce80 con 0x7f626c106f20 2024-09-05T06:38:17.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:17.185 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"btime":"2024-09-05T06:38:14:590669+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-05T06:38:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 msgr2=0x7f62580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 0x7f62580a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6254005d00 tx=0x7f625400b410 comp rx=0 tx=0).stop 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 msgr2=0x7f626c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6264009dc0 tx=0x7f6264032ff0 comp rx=0 tx=0).stop 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 shutdown_connections 2024-09-05T06:38:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c106f20 0x7f626c19cb90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c1027a0 0x7f626c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62580a2c00 0x7f62580a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 --2- 172.21.15.86:0/2564838875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626c101df0 0x7f626c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 >> 172.21.15.86:0/2564838875 conn(0x7f626c0fd8b0 msgr2=0x7f626c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 shutdown_connections 2024-09-05T06:38:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:17.187+0000 7f62708d2640 1 -- 172.21.15.86:0/2564838875 wait complete. 2024-09-05T06:38:17.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2564838875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2564838875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 podman[63143]: 2024-09-05 06:38:17.17932202 +0000 UTC m=+0.401254289 container start 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 ceph-mds[63162]: -- 172.21.15.96:0/933557988 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea4f7db860 con 0x55ea4f7b9400 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: starting mds.k at 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 bash[63143]: 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f 2024-09-05T06:38:17.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 06:38:17 smithi096 systemd[1]: Started Ceph mds.k for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2564838875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:boot 2024-09-05T06:38:18.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: fsmap 2 up:standby 2024-09-05T06:38:18.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:18 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:38:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:boot 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: fsmap 2 up:standby 2024-09-05T06:38:18.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:18 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:38:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:boot 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: fsmap 2 up:standby 2024-09-05T06:38:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:18 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:38:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:19 smithi086 ceph-mon[30943]: Deploying daemon mds.j on smithi086 2024-09-05T06:38:19.541 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:19 smithi096 ceph-mon[34468]: Deploying daemon mds.j on smithi086 2024-09-05T06:38:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:19 smithi115 ceph-mon[34791]: Deploying daemon mds.j on smithi086 2024-09-05T06:38:19.883 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:19 smithi086 systemd[1]: Starting Ceph mds.j for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:20.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:20.174 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:19 smithi086 podman[68093]: 2024-09-05 06:38:19.785926162 +0000 UTC m=+0.015695304 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:20.174 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:19 smithi086 podman[68093]: 2024-09-05 06:38:19.924916847 +0000 UTC m=+0.154685988 container create 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:20.174 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 podman[68093]: 2024-09-05 06:38:20.174898044 +0000 UTC m=+0.404667196 container init 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T06:38:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:20.453 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:20 smithi086 ceph-mon[30943]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:20.453 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 podman[68093]: 2024-09-05 06:38:20.179277901 +0000 UTC m=+0.409047051 container start 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T06:38:20.453 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 ceph-mds[68131]: -- 172.21.15.86:0/2556572288 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b12e257860 con 0x55b12e235000 2024-09-05T06:38:20.454 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j[68127]: starting mds.j at 2024-09-05T06:38:20.454 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 bash[68093]: 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 2024-09-05T06:38:20.454 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 06:38:20 smithi086 systemd[1]: Started Ceph mds.j for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:20 smithi096 ceph-mon[34468]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:20.584 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:20 smithi115 ceph-mon[34791]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:20.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3431158672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 msgr2=0x7f8eb410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:20.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3431158672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb410ec40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009930 tx=0x7f8eac02a420 comp rx=0 tx=0).stop 2024-09-05T06:38:20.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3431158672 shutdown_connections 2024-09-05T06:38:20.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3431158672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb410ec40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3431158672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eb41036e0 0x7f8eb4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3431158672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 0x7f8eb4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3431158672 >> 172.21.15.86:0/3431158672 conn(0x7f8eb40fc7d0 msgr2=0x7f8eb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3431158672 shutdown_connections 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3431158672 wait complete. 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.837+0000 7f8ebbdb1640 1 Processor -- start 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 -- start start 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 0x7f8eb4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eb41036e0 0x7f8eb41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111f50 con 0x7f8eb41082a0 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111dd0 con 0x7f8eb41036e0 2024-09-05T06:38:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41120d0 con 0x7f8eb4102d30 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52322/0 (socket says 172.21.15.86:52322) 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 -- 172.21.15.86:0/3370701250 learned_addr learned my addr 172.21.15.86:0/3370701250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eb9b4d640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 0x7f8eb4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 -- 172.21.15.86:0/3370701250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 msgr2=0x7f8eb4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 0x7f8eb4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 -- 172.21.15.86:0/3370701250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eb41036e0 msgr2=0x7f8eb41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eb41036e0 0x7f8eb41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 -- 172.21.15.86:0/3370701250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb419d420 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eba34e640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005940 tx=0x7f8eac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac03f030 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8eac003aa0 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb419d6b0 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.838+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac035070 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.839+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41a4ff0 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.839+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb4103110 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.841+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8eac042030 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.841+0000 7f8eb2ffd640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 0x7f8e9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.841+0000 7f8eb9b4d640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 0x7f8e9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.841+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f8eac0ef7d0 con 0x7f8eb41082a0 2024-09-05T06:38:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.842+0000 7f8eb9b4d640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 0x7f8e9c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005fd0 tx=0x7f8ea4009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:20.844+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eac0bcbd0 con 0x7f8eb41082a0 2024-09-05T06:38:21.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.057+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb410a2b0 con 0x7f8eb41082a0 2024-09-05T06:38:21.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.058+0000 7f8eb2ffd640 1 -- 172.21.15.86:0/3370701250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f8eac0bc4d0 con 0x7f8eb41082a0 2024-09-05T06:38:21.058 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:21.058 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"btime":"2024-09-05T06:38:20:245394+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T06:38:21.058 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 msgr2=0x7f8e9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 0x7f8e9c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005fd0 tx=0x7f8ea4009870 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 msgr2=0x7f8eb419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005940 tx=0x7f8eac004270 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 shutdown_connections 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb41082a0 0x7f8eb419cd40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8e9c0a2b80 0x7f8e9c0a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eb41036e0 0x7f8eb41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 --2- 172.21.15.86:0/3370701250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8eb4102d30 0x7f8eb4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 >> 172.21.15.86:0/3370701250 conn(0x7f8eb40fc7d0 msgr2=0x7f8eb4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 shutdown_connections 2024-09-05T06:38:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:21.060+0000 7f8ebbdb1640 1 -- 172.21.15.86:0/3370701250 wait complete. 2024-09-05T06:38:21.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6834/3431663557,v1:172.21.15.86:6835/3431663557] up:boot 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: fsmap 3 up:standby 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:21 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3370701250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6834/3431663557,v1:172.21.15.86:6835/3431663557] up:boot 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: fsmap 3 up:standby 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:21.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:21.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:21 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3370701250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6834/3431663557,v1:172.21.15.86:6835/3431663557] up:boot 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: fsmap 3 up:standby 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:21.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:21.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:21.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:21.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:21 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3370701250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:22 smithi086 ceph-mon[30943]: Deploying daemon mds.d on smithi086 2024-09-05T06:38:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:22 smithi086 ceph-mon[30943]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:22 smithi096 ceph-mon[34468]: Deploying daemon mds.d on smithi086 2024-09-05T06:38:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:22 smithi096 ceph-mon[34468]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:22 smithi115 ceph-mon[34791]: Deploying daemon mds.d on smithi086 2024-09-05T06:38:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:22 smithi115 ceph-mon[34791]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:23.488 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 systemd[1]: Starting Ceph mds.d for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:23.790 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 podman[68506]: 2024-09-05 06:38:23.42500422 +0000 UTC m=+0.016041509 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:23.790 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 podman[68506]: 2024-09-05 06:38:23.536517068 +0000 UTC m=+0.127554342 container create f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T06:38:23.790 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 podman[68506]: 2024-09-05 06:38:23.763895075 +0000 UTC m=+0.354932352 container init f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2) 2024-09-05T06:38:23.790 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 podman[68506]: 2024-09-05 06:38:23.766852657 +0000 UTC m=+0.357889940 container start f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-05T06:38:23.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:24.174 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:24.236 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 ceph-mds[68525]: -- 172.21.15.86:0/1187325464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ebfa6b860 con 0x562ebfa49000 2024-09-05T06:38:24.236 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d[68521]: starting mds.d at 2024-09-05T06:38:24.237 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 bash[68506]: f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b 2024-09-05T06:38:24.237 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 06:38:23 smithi086 systemd[1]: Started Ceph mds.d for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:24 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:24.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:24.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:24.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:24.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:24 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:24.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3866512397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c106a80 msgr2=0x7f8e0c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:24.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3866512397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c106a80 0x7f8e0c106e60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009930 tx=0x7f8dfc02a420 comp rx=0 tx=0).stop 2024-09-05T06:38:24.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3866512397 shutdown_connections 2024-09-05T06:38:24.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3866512397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c101420 0x7f8e0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3866512397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 0x7f8e0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3866512397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c106a80 0x7f8e0c106e60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3866512397 >> 172.21.15.86:0/3866512397 conn(0x7f8e0c0fc7a0 msgr2=0x7f8e0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3866512397 shutdown_connections 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.639+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3866512397 wait complete. 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 Processor -- start 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 -- start start 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 0x7f8e0c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:24.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 0x7f8e0c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c110d30 con 0x7f8e0c101420 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c110bb0 con 0x7f8e0c100a80 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e10a2f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c110eb0 con 0x7f8e0c106a80 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49170/0 (socket says 172.21.15.86:49170) 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 -- 172.21.15.86:0/3090253362 learned_addr learned my addr 172.21.15.86:0/3090253362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e0a59c640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 0x7f8e0c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e0ad9d640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 0x7f8e0c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 -- 172.21.15.86:0/3090253362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 msgr2=0x7f8e0c19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 0x7f8e0c19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 -- 172.21.15.86:0/3090253362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 msgr2=0x7f8e0c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 0x7f8e0c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 -- 172.21.15.86:0/3090253362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c19c210 con 0x7f8e0c101420 2024-09-05T06:38:24.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e0ad9d640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 0x7f8e0c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.640+0000 7f8e09d9b640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8df400e9f0 tx=0x7f8df400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.641+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df400cd80 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.641+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df40108e0 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.641+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4018920 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.641+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c19c500 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.641+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1a3de0 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.642+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c100ee0 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.643+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8df4010a90 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.644+0000 7f8e037fe640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 0x7f8df80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.644+0000 7f8e0a59c640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 0x7f8df80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.644+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f8df40cf460 con 0x7f8e0c101420 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.644+0000 7f8e0a59c640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 0x7f8df80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009900 tx=0x7f8dfc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:24.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.647+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8df409c800 con 0x7f8e0c101420 2024-09-05T06:38:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:24 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:24.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.846+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e0c02d7c0 con 0x7f8e0c101420 2024-09-05T06:38:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.846+0000 7f8e037fe640 1 -- 172.21.15.86:0/3090253362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f8df409c100 con 0x7f8e0c101420 2024-09-05T06:38:24.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:24.848 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"btime":"2024-09-05T06:38:24:254139+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T06:38:24.848 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 5 2024-09-05T06:38:24.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 msgr2=0x7f8df80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:24.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 0x7f8df80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009900 tx=0x7f8dfc000bc0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 msgr2=0x7f8e0c1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:24.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8df400e9f0 tx=0x7f8df400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 shutdown_connections 2024-09-05T06:38:24.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e0c106a80 0x7f8e0c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c101420 0x7f8e0c1977a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f8df80a2b80 0x7f8df80a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 --2- 172.21.15.86:0/3090253362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0c100a80 0x7f8e0c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 >> 172.21.15.86:0/3090253362 conn(0x7f8e0c0fc7a0 msgr2=0x7f8e0c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 shutdown_connections 2024-09-05T06:38:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:24.849+0000 7f8e10a2f640 1 -- 172.21.15.86:0/3090253362 wait complete. 2024-09-05T06:38:25.450 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:25 smithi096 ceph-mon[34468]: Deploying daemon mds.e on smithi096 2024-09-05T06:38:25.450 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:25 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6836/78876235,v1:172.21.15.86:6837/78876235] up:boot 2024-09-05T06:38:25.451 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:25 smithi096 ceph-mon[34468]: fsmap 4 up:standby 2024-09-05T06:38:25.451 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:25 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:38:25.451 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:25 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3090253362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:25 smithi086 ceph-mon[30943]: Deploying daemon mds.e on smithi096 2024-09-05T06:38:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:25 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6836/78876235,v1:172.21.15.86:6837/78876235] up:boot 2024-09-05T06:38:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:25 smithi086 ceph-mon[30943]: fsmap 4 up:standby 2024-09-05T06:38:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:25 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:38:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:25 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3090253362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:25 smithi115 ceph-mon[34791]: Deploying daemon mds.e on smithi096 2024-09-05T06:38:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:25 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6836/78876235,v1:172.21.15.86:6837/78876235] up:boot 2024-09-05T06:38:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:25 smithi115 ceph-mon[34791]: fsmap 4 up:standby 2024-09-05T06:38:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:25 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:38:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:25 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3090253362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:26.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:25 smithi096 systemd[1]: Starting Ceph mds.e for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:26.405 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:26 smithi096 ceph-mon[34468]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:26.405 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 podman[63452]: 2024-09-05 06:38:26.037027668 +0000 UTC m=+0.015917396 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:26.406 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 podman[63452]: 2024-09-05 06:38:26.153279103 +0000 UTC m=+0.132168837 container create 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:26.406 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 podman[63452]: 2024-09-05 06:38:26.406115846 +0000 UTC m=+0.385005572 container init 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:26 smithi086 ceph-mon[30943]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:26 smithi115 ceph-mon[34791]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:26.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 podman[63452]: 2024-09-05 06:38:26.409214144 +0000 UTC m=+0.388103869 container start 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=) 2024-09-05T06:38:26.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 ceph-mds[63471]: -- 172.21.15.96:0/481368642 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e1de9d860 con 0x558e1de7b000 2024-09-05T06:38:26.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: starting mds.e at 2024-09-05T06:38:26.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 bash[63452]: 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 2024-09-05T06:38:26.807 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 06:38:26 smithi096 systemd[1]: Started Ceph mds.e for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:27.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:27.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.872 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: Deploying daemon mds.f on smithi115 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6834/3823127248,v1:172.21.15.96:6835/3823127248] up:boot 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: fsmap 5 up:standby 2024-09-05T06:38:27.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:27 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:38:27.901 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:27.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: Deploying daemon mds.f on smithi115 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6834/3823127248,v1:172.21.15.96:6835/3823127248] up:boot 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: fsmap 5 up:standby 2024-09-05T06:38:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:27 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:38:28.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: Deploying daemon mds.f on smithi115 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6834/3823127248,v1:172.21.15.96:6835/3823127248] up:boot 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: fsmap 5 up:standby 2024-09-05T06:38:28.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:27 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:38:28.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 -- 172.21.15.86:0/3894242500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 msgr2=0x7fab2410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:28.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 --2- 172.21.15.86:0/3894242500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab2410bbb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fab18009900 tx=0x7fab1802a480 comp rx=0 tx=0).stop 2024-09-05T06:38:28.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 -- 172.21.15.86:0/3894242500 shutdown_connections 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 --2- 172.21.15.86:0/3894242500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab2410c180 0x7fab2410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 --2- 172.21.15.86:0/3894242500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab2410bbb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 --2- 172.21.15.86:0/3894242500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24102da0 0x7fab24103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 -- 172.21.15.86:0/3894242500 >> 172.21.15.86:0/3894242500 conn(0x7fab240fc7a0 msgr2=0x7fab240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 -- 172.21.15.86:0/3894242500 shutdown_connections 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.389+0000 7fab29ec0640 1 -- 172.21.15.86:0/3894242500 wait complete. 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 Processor -- start 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 -- start start 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab24102da0 0x7fab24198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2410c180 0x7fab2419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111e10 con 0x7fab241036c0 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111c90 con 0x7fab24102da0 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab29ec0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111f90 con 0x7fab2410c180 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49190/0 (socket says 172.21.15.86:49190) 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 -- 172.21.15.86:0/592536917 learned_addr learned my addr 172.21.15.86:0/592536917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab23fff640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2410c180 0x7fab2419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab237fe640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab24102da0 0x7fab24198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 -- 172.21.15.86:0/592536917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2410c180 msgr2=0x7fab2419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2410c180 0x7fab2419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 -- 172.21.15.86:0/592536917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab24102da0 msgr2=0x7fab24198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab24102da0 0x7fab24198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.390+0000 7fab22ffd640 1 -- 172.21.15.86:0/592536917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2419d2c0 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab22ffd640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab18005940 tx=0x7fab180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1803f030 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2419d550 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab18003cb0 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18035070 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.391+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a4e90 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.392+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2406b010 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.393+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab18042030 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.394+0000 7fab20ff9640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 0x7fab0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.394+0000 7fab237fe640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 0x7fab0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.394+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fab180ef310 con 0x7fab241036c0 2024-09-05T06:38:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.394+0000 7fab237fe640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 0x7fab0c0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab14009bd0 tx=0x7fab14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:28.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.398+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab180bc0c0 con 0x7fab241036c0 2024-09-05T06:38:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.598+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab241071c0 con 0x7fab241036c0 2024-09-05T06:38:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.599+0000 7fab20ff9640 1 -- 172.21.15.86:0/592536917 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7fab180c03c0 con 0x7fab241036c0 2024-09-05T06:38:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:28.599 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"btime":"2024-09-05T06:38:27:268285+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T06:38:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-09-05T06:38:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 msgr2=0x7fab0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 0x7fab0c0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab14009bd0 tx=0x7fab14005f90 comp rx=0 tx=0).stop 2024-09-05T06:38:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 msgr2=0x7fab24198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab18005940 tx=0x7fab180042f0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 shutdown_connections 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2410c180 0x7fab2419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241036c0 0x7fab24198850 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab0c0a2b80 0x7fab0c0a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.602+0000 7fab29ec0640 1 --2- 172.21.15.86:0/592536917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab24102da0 0x7fab24198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.603+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 >> 172.21.15.86:0/592536917 conn(0x7fab240fc7a0 msgr2=0x7fab240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.603+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 shutdown_connections 2024-09-05T06:38:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:28.603+0000 7fab29ec0640 1 -- 172.21.15.86:0/592536917 wait complete. 2024-09-05T06:38:28.651 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 systemd[1]: Starting Ceph mds.f for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:28.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:28 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/592536917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:28.933 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 podman[64287]: 2024-09-05 06:38:28.588406294 +0000 UTC m=+0.015734256 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:28.933 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 podman[64287]: 2024-09-05 06:38:28.703847815 +0000 UTC m=+0.131175765 container create ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e) 2024-09-05T06:38:28.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:28 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/592536917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:29.054 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:28 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/592536917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:29.432 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 podman[64287]: 2024-09-05 06:38:28.956563559 +0000 UTC m=+0.383891512 container init ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-05T06:38:29.433 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 podman[64287]: 2024-09-05 06:38:28.959291649 +0000 UTC m=+0.386619602 container start ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-05T06:38:29.433 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 ceph-mds[64306]: -- 172.21.15.115:0/2227663948 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614b2469860 con 0x5614b2447400 2024-09-05T06:38:29.433 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: starting mds.f at 2024-09-05T06:38:29.433 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:29 smithi115 bash[64287]: ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a 2024-09-05T06:38:29.433 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 06:38:29 smithi115 systemd[1]: Started Ceph mds.f for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: Deploying daemon mds.c on smithi115 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6834/1272008759,v1:172.21.15.115:6835/1272008759] up:boot 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: fsmap 6 up:standby 2024-09-05T06:38:30.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:30 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: Deploying daemon mds.c on smithi115 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6834/1272008759,v1:172.21.15.115:6835/1272008759] up:boot 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: fsmap 6 up:standby 2024-09-05T06:38:30.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:30 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:38:30.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: Deploying daemon mds.c on smithi115 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6834/1272008759,v1:172.21.15.115:6835/1272008759] up:boot 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: fsmap 6 up:standby 2024-09-05T06:38:30.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:30 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:38:31.182 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:30 smithi115 systemd[1]: Starting Ceph mds.c for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:31.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:31.469 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 podman[64588]: 2024-09-05 06:38:31.108261295 +0000 UTC m=+0.015577468 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:31.470 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 podman[64588]: 2024-09-05 06:38:31.208082597 +0000 UTC m=+0.115398805 container create 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T06:38:31.470 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 podman[64588]: 2024-09-05 06:38:31.444163063 +0000 UTC m=+0.351479223 container init 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T06:38:31.470 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 podman[64588]: 2024-09-05 06:38:31.447200194 +0000 UTC m=+0.354516356 container start 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T06:38:31.470 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 ceph-mds[64607]: -- 172.21.15.115:0/2159821123 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c4fb4b860 con 0x559c4fb28800 2024-09-05T06:38:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:31.932 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: starting mds.c at 2024-09-05T06:38:31.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 bash[64588]: 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a 2024-09-05T06:38:31.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 06:38:31 smithi115 systemd[1]: Started Ceph mds.c for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- 172.21.15.86:0/508819637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4100e20 msgr2=0x7f17e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 --2- 172.21.15.86:0/508819637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4100e20 0x7f17e4101200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009ca0 tx=0x7f17d402a950 comp rx=0 tx=0).stop 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- 172.21.15.86:0/508819637 shutdown_connections 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 --2- 172.21.15.86:0/508819637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102ef0 0x7f17e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 --2- 172.21.15.86:0/508819637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4102570 0x7f17e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 --2- 172.21.15.86:0/508819637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4100e20 0x7f17e4101200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- 172.21.15.86:0/508819637 >> 172.21.15.86:0/508819637 conn(0x7f17e40fc7d0 msgr2=0x7f17e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- 172.21.15.86:0/508819637 shutdown_connections 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- 172.21.15.86:0/508819637 wait complete. 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 Processor -- start 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.199+0000 7f17ebed9640 1 -- start start 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4100e20 0x7f17e4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 0x7f17e41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4111f50 con 0x7f17e4102ef0 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4111dd0 con 0x7f17e4102570 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41120d0 con 0x7f17e4100e20 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49210/0 (socket says 172.21.15.86:49210) 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 -- 172.21.15.86:0/378108179 learned_addr learned my addr 172.21.15.86:0/378108179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e9c75640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4100e20 0x7f17e4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e9474640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 0x7f17e41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 -- 172.21.15.86:0/378108179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4100e20 msgr2=0x7f17e4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4100e20 0x7f17e4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 -- 172.21.15.86:0/378108179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 msgr2=0x7f17e41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 0x7f17e41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 -- 172.21.15.86:0/378108179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e419d330 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e9474640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 0x7f17e41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ea476640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00c970 tx=0x7f17dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc007bb0 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e419d620 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17dc007d50 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc018700 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.200+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e41a4f00 con 0x7f17e4102ef0 2024-09-05T06:38:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.201+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e4101200 con 0x7f17e4102ef0 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.202+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f17dc00fa20 con 0x7f17e4102ef0 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.203+0000 7f17e2ffd640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 0x7f17cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.203+0000 7f17e9c75640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 0x7f17cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.203+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f17dc0cec10 con 0x7f17e4102ef0 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.204+0000 7f17e9c75640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 0x7f17cc0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d4004ad0 tx=0x7f17d4004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:32.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.207+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17dc014030 con 0x7f17e4102ef0 2024-09-05T06:38:32.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:32.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:32 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:32.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.413+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17e4109090 con 0x7f17e4102ef0 2024-09-05T06:38:32.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.414+0000 7f17e2ffd640 1 -- 172.21.15.86:0/378108179 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f17dc09c010 con 0x7f17e4102ef0 2024-09-05T06:38:32.415 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:32.415 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":8,"btime":"2024-09-05T06:38:32:210983+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-05T06:38:32.415 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 8 2024-09-05T06:38:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.416+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 msgr2=0x7f17cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.416+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 0x7f17cc0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d4004ad0 tx=0x7f17d4004a60 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.416+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 msgr2=0x7f17e419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.416+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00c970 tx=0x7f17dc00ce30 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 shutdown_connections 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e4102ef0 0x7f17e419cc50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17cc0a2b80 0x7f17cc0a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17e4102570 0x7f17e41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 --2- 172.21.15.86:0/378108179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17e4100e20 0x7f17e4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 >> 172.21.15.86:0/378108179 conn(0x7f17e40fc7d0 msgr2=0x7f17e40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 shutdown_connections 2024-09-05T06:38:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:32.417+0000 7f17ebed9640 1 -- 172.21.15.86:0/378108179 wait complete. 2024-09-05T06:38:32.476 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:32.476 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:32.476 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:32 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:32.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:32.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:32.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:32 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:33.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:33 smithi096 ceph-mon[34468]: Deploying daemon mds.h on smithi096 2024-09-05T06:38:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:33 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6836/2087354694,v1:172.21.15.115:6837/2087354694] up:boot 2024-09-05T06:38:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:33 smithi096 ceph-mon[34468]: fsmap 7 up:standby 2024-09-05T06:38:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:33 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:38:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:33 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/378108179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:33.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:33 smithi086 ceph-mon[30943]: Deploying daemon mds.h on smithi096 2024-09-05T06:38:33.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:33 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6836/2087354694,v1:172.21.15.115:6837/2087354694] up:boot 2024-09-05T06:38:33.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:33 smithi086 ceph-mon[30943]: fsmap 7 up:standby 2024-09-05T06:38:33.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:33 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:38:33.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/378108179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:33 smithi115 ceph-mon[34791]: Deploying daemon mds.h on smithi096 2024-09-05T06:38:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:33 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6836/2087354694,v1:172.21.15.115:6837/2087354694] up:boot 2024-09-05T06:38:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:33 smithi115 ceph-mon[34791]: fsmap 7 up:standby 2024-09-05T06:38:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:33 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:38:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:33 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/378108179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:33.808 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:33 smithi096 systemd[1]: Starting Ceph mds.h for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:34.119 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:33 smithi096 podman[63753]: 2024-09-05 06:38:33.74236693 +0000 UTC m=+0.015614012 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:34.119 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:33 smithi096 podman[63753]: 2024-09-05 06:38:33.875548709 +0000 UTC m=+0.148795789 container create 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-05T06:38:34.119 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 podman[63753]: 2024-09-05 06:38:34.120092367 +0000 UTC m=+0.393339451 container init 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T06:38:34.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:34 smithi086 ceph-mon[30943]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:34.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:34 smithi096 ceph-mon[34468]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:34.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 podman[63753]: 2024-09-05 06:38:34.123007139 +0000 UTC m=+0.396254215 container start 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T06:38:34.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 ceph-mds[63772]: -- 172.21.15.96:0/1811633560 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56083cc2f860 con 0x56083cc0d000 2024-09-05T06:38:34.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: starting mds.h at 2024-09-05T06:38:34.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 bash[63753]: 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a 2024-09-05T06:38:34.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 06:38:34 smithi096 systemd[1]: Started Ceph mds.h for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:34 smithi115 ceph-mon[34791]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:35.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:35.415 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:boot 2024-09-05T06:38:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: fsmap 8 up:standby 2024-09-05T06:38:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:38:35.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:35.435 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:35.435 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:35.435 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:35 smithi115 ceph-mon[34791]: Deploying daemon mds.i on smithi115 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:boot 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: fsmap 8 up:standby 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:35 smithi086 ceph-mon[30943]: Deploying daemon mds.i on smithi115 2024-09-05T06:38:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:boot 2024-09-05T06:38:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: fsmap 8 up:standby 2024-09-05T06:38:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:38:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:35 smithi096 ceph-mon[34468]: Deploying daemon mds.i on smithi115 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- 172.21.15.86:0/1185674253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c102570 msgr2=0x7f9e4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1185674253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c102570 0x7f9e4c1029b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009900 tx=0x7f9e3402a480 comp rx=0 tx=0).stop 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- 172.21.15.86:0/1185674253 shutdown_connections 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1185674253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102ef0 0x7f9e4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1185674253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c102570 0x7f9e4c1029b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1185674253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c100e20 0x7f9e4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- 172.21.15.86:0/1185674253 >> 172.21.15.86:0/1185674253 conn(0x7f9e4c0fc7d0 msgr2=0x7f9e4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- 172.21.15.86:0/1185674253 shutdown_connections 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- 172.21.15.86:0/1185674253 wait complete. 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 Processor -- start 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.894+0000 7f9e51e88640 1 -- start start 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 0x7f9e4c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c102ef0 0x7f9e4c1a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c110e70 con 0x7f9e4c100e20 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c110cf0 con 0x7f9e4c102570 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e51e88640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c110ff0 con 0x7f9e4c102ef0 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38570/0 (socket says 172.21.15.86:38570) 2024-09-05T06:38:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 -- 172.21.15.86:0/1035330394 learned_addr learned my addr 172.21.15.86:0/1035330394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4bfff640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c102ef0 0x7f9e4c1a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e42ffd640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 0x7f9e4c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 -- 172.21.15.86:0/1035330394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c102ef0 msgr2=0x7f9e4c1a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c102ef0 0x7f9e4c1a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 -- 172.21.15.86:0/1035330394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 msgr2=0x7f9e4c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 0x7f9e4c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 -- 172.21.15.86:0/1035330394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c1a48e0 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e42ffd640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 0x7f9e4c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.895+0000 7f9e4b7fe640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00c970 tx=0x7f9e3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.896+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c007bb0 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.896+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c1a4ab0 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.896+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e3c007d50 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.896+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00f450 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.896+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c1a4fc0 con 0x7f9e4c100e20 2024-09-05T06:38:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.897+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c101200 con 0x7f9e4c100e20 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.898+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e3c004f20 con 0x7f9e4c100e20 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.899+0000 7f9e497fa640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 0x7f9e380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.899+0000 7f9e42ffd640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 0x7f9e380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.899+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f9e3c0ce9d0 con 0x7f9e4c100e20 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.899+0000 7f9e42ffd640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 0x7f9e380a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e34000c00 tx=0x7f9e34032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:35.903+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e3c014030 con 0x7f9e4c100e20 2024-09-05T06:38:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.102+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e4c109090 con 0x7f9e4c100e20 2024-09-05T06:38:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.103+0000 7f9e497fa640 1 -- 172.21.15.86:0/1035330394 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7f9e3c09bdd0 con 0x7f9e4c100e20 2024-09-05T06:38:36.104 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:36.104 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"btime":"2024-09-05T06:38:34:223023+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T06:38:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-09-05T06:38:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 msgr2=0x7f9e380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 0x7f9e380a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e34000c00 tx=0x7f9e34032d50 comp rx=0 tx=0).stop 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 msgr2=0x7f9e4c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00c970 tx=0x7f9e3c00ce30 comp rx=0 tx=0).stop 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 shutdown_connections 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e4c102ef0 0x7f9e4c1a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e380a2b80 0x7f9e380a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e4c102570 0x7f9e4c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 --2- 172.21.15.86:0/1035330394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4c100e20 0x7f9e4c199320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 >> 172.21.15.86:0/1035330394 conn(0x7f9e4c0fc7d0 msgr2=0x7f9e4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 shutdown_connections 2024-09-05T06:38:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:36.105+0000 7f9e51e88640 1 -- 172.21.15.86:0/1035330394 wait complete. 2024-09-05T06:38:36.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:36 smithi115 ceph-mon[34791]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:36.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:36 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1035330394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:36.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:36 smithi086 ceph-mon[30943]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:36.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:36 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1035330394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:36 smithi096 ceph-mon[34468]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:36 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1035330394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:36.709 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 06:38:36 smithi115 podman[64887]: 2024-09-05 06:38:36.686064988 +0000 UTC m=+0.346442879 container start 2bce40969aea963b4bbc5fd6b57f68f5f926d5f8aa24432c77cb3372cd750ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-05T06:38:37.182 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 06:38:36 smithi115 ceph-mds[64906]: -- 172.21.15.115:0/1918653900 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56547b13b860 con 0x56547b119400 2024-09-05T06:38:37.182 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 06:38:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: starting mds.i at 2024-09-05T06:38:37.183 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 06:38:36 smithi115 bash[64887]: 2bce40969aea963b4bbc5fd6b57f68f5f926d5f8aa24432c77cb3372cd750ae5 2024-09-05T06:38:37.183 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 06:38:36 smithi115 systemd[1]: Started Ceph mds.i for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:37.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:37.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:37.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:37.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: Deploying daemon mds.g on smithi086 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:boot 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: fsmap 9 up:standby 2024-09-05T06:38:37.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:37 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:38:38.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: Deploying daemon mds.g on smithi086 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:boot 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: fsmap 9 up:standby 2024-09-05T06:38:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:37 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: Deploying daemon mds.g on smithi086 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:boot 2024-09-05T06:38:38.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: fsmap 9 up:standby 2024-09-05T06:38:38.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:37 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:38:38.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:38.821 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:38 smithi086 systemd[1]: Starting Ceph mds.g for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:39.085 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:38 smithi086 podman[69285]: 2024-09-05 06:38:38.828499966 +0000 UTC m=+0.016757381 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:39.085 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:38 smithi086 podman[69285]: 2024-09-05 06:38:38.945620582 +0000 UTC m=+0.133877988 container create 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T06:38:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 podman[69285]: 2024-09-05 06:38:39.198434405 +0000 UTC m=+0.386691810 container init 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 podman[69285]: 2024-09-05 06:38:39.201618865 +0000 UTC m=+0.389876271 container start 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 ceph-mds[69341]: -- 172.21.15.86:0/1766825011 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608ccac7860 con 0x5608ccaa5400 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: starting mds.g at 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 bash[69285]: 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 2024-09-05T06:38:39.456 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 06:38:39 smithi086 systemd[1]: Started Ceph mds.g for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- 172.21.15.86:0/854178224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 msgr2=0x7fd310101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 --2- 172.21.15.86:0/854178224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd310101200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd308009900 tx=0x7fd30802a480 comp rx=0 tx=0).stop 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- 172.21.15.86:0/854178224 shutdown_connections 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 --2- 172.21.15.86:0/854178224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102ef0 0x7fd31010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 --2- 172.21.15.86:0/854178224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102570 0x7fd3101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 --2- 172.21.15.86:0/854178224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd310101200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:39.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- 172.21.15.86:0/854178224 >> 172.21.15.86:0/854178224 conn(0x7fd3100fc7d0 msgr2=0x7fd3100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- 172.21.15.86:0/854178224 shutdown_connections 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- 172.21.15.86:0/854178224 wait complete. 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 Processor -- start 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.800+0000 7fd31781c640 1 -- start start 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102570 0x7fd3101997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 0x7fd3101a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd310110e50 con 0x7fd310100e20 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd310110cd0 con 0x7fd310102ef0 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd310110fd0 con 0x7fd310102570 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:39.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38606/0 (socket says 172.21.15.86:38606) 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 -- 172.21.15.86:0/1291599841 learned_addr learned my addr 172.21.15.86:0/1291599841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd314db7640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102570 0x7fd3101997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd315db9640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 0x7fd3101a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 -- 172.21.15.86:0/1291599841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102570 msgr2=0x7fd3101997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102570 0x7fd3101997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 -- 172.21.15.86:0/1291599841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 msgr2=0x7fd3101a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 0x7fd3101a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 -- 172.21.15.86:0/1291599841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3101a4860 con 0x7fd310100e20 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd315db9640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 0x7fd3101a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3155b8640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd308006010 tx=0x7fd3080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:39.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30803f070 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd308003aa0 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 -- 172.21.15.86:0/1291599841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3101a4a30 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308035030 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.801+0000 7fd31781c640 1 -- 172.21.15.86:0/1291599841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a4ee0 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.802+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f0005290 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.803+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd308042070 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.804+0000 7fd3067fc640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 0x7fd2f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.804+0000 7fd314db7640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 0x7fd2f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.804+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd3080ef510 con 0x7fd310100e20 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.805+0000 7fd314db7640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 0x7fd2f80a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd31006aac0 tx=0x7fd30c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:39.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:39.809+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3080bc870 con 0x7fd310100e20 2024-09-05T06:38:40.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.022+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2f0005580 con 0x7fd310100e20 2024-09-05T06:38:40.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.023+0000 7fd3067fc640 1 -- 172.21.15.86:0/1291599841 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7fd3080bc170 con 0x7fd310100e20 2024-09-05T06:38:40.023 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:40.023 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"btime":"2024-09-05T06:38:37:292589+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T06:38:40.023 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 msgr2=0x7fd2f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 0x7fd2f80a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd31006aac0 tx=0x7fd30c00a2d0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 msgr2=0x7fd3101992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd308006010 tx=0x7fd3080042f0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 shutdown_connections 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd2f80a2c00 0x7fd2f80a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd310102ef0 0x7fd3101a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd310102570 0x7fd3101997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 --2- 172.21.15.86:0/1291599841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd310100e20 0x7fd3101992a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 >> 172.21.15.86:0/1291599841 conn(0x7fd3100fc7d0 msgr2=0x7fd3100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 shutdown_connections 2024-09-05T06:38:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:40.025+0000 7fd2f7fff640 1 -- 172.21.15.86:0/1291599841 wait complete. 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:40 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1291599841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:40 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1291599841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:40 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1291599841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:41.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:41 smithi086 ceph-mon[30943]: Deploying daemon mds.a on smithi086 2024-09-05T06:38:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:41 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6838/3075373024,v1:172.21.15.86:6839/3075373024] up:boot 2024-09-05T06:38:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:41 smithi086 ceph-mon[30943]: fsmap 10 up:standby 2024-09-05T06:38:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:41 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:38:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:41 smithi096 ceph-mon[34468]: Deploying daemon mds.a on smithi086 2024-09-05T06:38:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:41 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6838/3075373024,v1:172.21.15.86:6839/3075373024] up:boot 2024-09-05T06:38:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:41 smithi096 ceph-mon[34468]: fsmap 10 up:standby 2024-09-05T06:38:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:41 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:38:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:41 smithi115 ceph-mon[34791]: Deploying daemon mds.a on smithi086 2024-09-05T06:38:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:41 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6838/3075373024,v1:172.21.15.86:6839/3075373024] up:boot 2024-09-05T06:38:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:41 smithi115 ceph-mon[34791]: fsmap 10 up:standby 2024-09-05T06:38:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:41 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:38:42.224 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 systemd[1]: Starting Ceph mds.a for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T06:38:42.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:42 smithi086 ceph-mon[30943]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:42.487 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 podman[69695]: 2024-09-05 06:38:42.225015384 +0000 UTC m=+0.015949649 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T06:38:42.487 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 podman[69695]: 2024-09-05 06:38:42.334163376 +0000 UTC m=+0.125097632 container create 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T06:38:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:42 smithi096 ceph-mon[34468]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:42 smithi115 ceph-mon[34791]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:42.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 podman[69695]: 2024-09-05 06:38:42.570700623 +0000 UTC m=+0.361634878 container init 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 podman[69695]: 2024-09-05 06:38:42.573521174 +0000 UTC m=+0.364455442 container start 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 ceph-mds[69714]: -- 172.21.15.86:0/3926498064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ed3877860 con 0x561ed3854800 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: starting mds.a at 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 bash[69695]: 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b 2024-09-05T06:38:42.949 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 06:38:42 smithi086 systemd[1]: Started Ceph mds.a for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:43.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 -- 172.21.15.86:0/2117181228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474102ef0 msgr2=0x7f547410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:43.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 --2- 172.21.15.86:0/2117181228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474102ef0 0x7f547410da40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f546c009ca0 tx=0x7f546c02a950 comp rx=0 tx=0).stop 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 -- 172.21.15.86:0/2117181228 shutdown_connections 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 --2- 172.21.15.86:0/2117181228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474102ef0 0x7f547410da40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 --2- 172.21.15.86:0/2117181228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 --2- 172.21.15.86:0/2117181228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474100e20 0x7f5474101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 -- 172.21.15.86:0/2117181228 >> 172.21.15.86:0/2117181228 conn(0x7f54740fc7d0 msgr2=0x7f54740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 -- 172.21.15.86:0/2117181228 shutdown_connections 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.534+0000 7f547a94b640 1 -- 172.21.15.86:0/2117181228 wait complete. 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 Processor -- start 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 -- start start 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474102ef0 0x7f547419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5474110e30 con 0x7f5474100e20 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5474110cb0 con 0x7f5474102570 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f547a94b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5474110fb0 con 0x7f5474102ef0 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f5473fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f5473fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41504/0 (socket says 172.21.15.86:41504) 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f5473fff640 1 -- 172.21.15.86:0/1038786492 learned_addr learned my addr 172.21.15.86:0/1038786492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f5478ee8640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474102ef0 0x7f547419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 -- 172.21.15.86:0/1038786492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474102ef0 msgr2=0x7f547419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474102ef0 0x7f547419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 -- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 msgr2=0x7f5474198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f546bfff640 1 -- 172.21.15.86:0/1038786492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547419d330 con 0x7f5474102570 2024-09-05T06:38:43.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.535+0000 7f5473fff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f546bfff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741988c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c009c30 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547419d620 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f545c009dd0 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c003e40 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.536+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a4f00 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.537+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474101200 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.538+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f545c004000 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.538+0000 7f5471ffb640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 0x7f54600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.539+0000 7f5473fff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 0x7f54600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.539+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f545c0ce7e0 con 0x7f5474102570 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.539+0000 7f5473fff640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 0x7f54600a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54640104e0 tx=0x7f546400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.542+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f545c017020 con 0x7f5474102570 2024-09-05T06:38:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.743+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f547410a0a0 con 0x7f5474102570 2024-09-05T06:38:43.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.744+0000 7f5471ffb640 1 -- 172.21.15.86:0/1038786492 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f545c09b4f0 con 0x7f5474102570 2024-09-05T06:38:43.745 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"btime":"2024-09-05T06:38:43:224274+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T06:38:43.745 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 msgr2=0x7f54600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 0x7f54600a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54640104e0 tx=0x7f546400b410 comp rx=0 tx=0).stop 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 msgr2=0x7f54741988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741988c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c00dde0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 shutdown_connections 2024-09-05T06:38:43.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5474102ef0 0x7f547419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54600a2b80 0x7f54600a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5474102570 0x7f54741988c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 --2- 172.21.15.86:0/1038786492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5474100e20 0x7f5474198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 >> 172.21.15.86:0/1038786492 conn(0x7f54740fc7d0 msgr2=0x7f547406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 shutdown_connections 2024-09-05T06:38:43.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:43.746+0000 7f547a94b640 1 -- 172.21.15.86:0/1038786492 wait complete. 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: Deploying daemon mds.b on smithi096 2024-09-05T06:38:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:43.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6840/1507644375,v1:172.21.15.86:6841/1507644375] up:boot 2024-09-05T06:38:43.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: fsmap 11 up:standby 2024-09-05T06:38:43.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:43 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: Deploying daemon mds.b on smithi096 2024-09-05T06:38:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:44.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6840/1507644375,v1:172.21.15.86:6841/1507644375] up:boot 2024-09-05T06:38:44.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: fsmap 11 up:standby 2024-09-05T06:38:44.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:43 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:38:44.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: Deploying daemon mds.b on smithi096 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6840/1507644375,v1:172.21.15.86:6841/1507644375] up:boot 2024-09-05T06:38:44.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: fsmap 11 up:standby 2024-09-05T06:38:44.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:43 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:38:44.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:44 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1038786492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:44.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:44 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1038786492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:45.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:44 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1038786492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:45.744 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 06:38:45 smithi096 systemd[1]: Started Ceph mds.b for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T06:38:46.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:45 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:46.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:45 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:45 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:46.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:38:46.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:46 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:boot 2024-09-05T06:38:46.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:46 smithi086 ceph-mon[30943]: fsmap 12 up:standby 2024-09-05T06:38:46.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:38:46.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:46 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.792 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:46 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:boot 2024-09-05T06:38:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:46 smithi096 ceph-mon[34468]: fsmap 12 up:standby 2024-09-05T06:38:46.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:38:46.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:46 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:46.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:46 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:boot 2024-09-05T06:38:46.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:46 smithi115 ceph-mon[34791]: fsmap 12 up:standby 2024-09-05T06:38:46.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:38:46.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:46 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 -- 172.21.15.86:0/2288035390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc106f20 msgr2=0x7f2cfc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 --2- 172.21.15.86:0/2288035390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc106f20 0x7f2cfc10eaa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009900 tx=0x7f2cec02a410 comp rx=0 tx=0).stop 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 -- 172.21.15.86:0/2288035390 shutdown_connections 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 --2- 172.21.15.86:0/2288035390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc106f20 0x7f2cfc10eaa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 --2- 172.21.15.86:0/2288035390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 --2- 172.21.15.86:0/2288035390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 0x7f2cfc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 -- 172.21.15.86:0/2288035390 >> 172.21.15.86:0/2288035390 conn(0x7f2cfc0fd8b0 msgr2=0x7f2cfc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 -- 172.21.15.86:0/2288035390 shutdown_connections 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.417+0000 7f2d03f17640 1 -- 172.21.15.86:0/2288035390 wait complete. 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 Processor -- start 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 -- start start 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 0x7f2cfc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc106f20 0x7f2cfc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc111e30 con 0x7f2cfc1027a0 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc111cb0 con 0x7f2cfc106f20 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d03f17640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc111fb0 con 0x7f2cfc101df0 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41528/0 (socket says 172.21.15.86:41528) 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 -- 172.21.15.86:0/3742913606 learned_addr learned my addr 172.21.15.86:0/3742913606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d01cb3640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 0x7f2cfc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d024b4640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc106f20 0x7f2cfc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 -- 172.21.15.86:0/3742913606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 msgr2=0x7f2cfc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 0x7f2cfc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 -- 172.21.15.86:0/3742913606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc106f20 msgr2=0x7f2cfc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc106f20 0x7f2cfc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 -- 172.21.15.86:0/3742913606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a6750 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.418+0000 7f2d014b2640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000eab0 tx=0x7f2cf000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.419+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000ce40 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.419+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cf0011920 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.419+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0010a00 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.419+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a6a40 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.419+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a6fd0 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.420+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc1021d0 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.421+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2cf0011ad0 con 0x7f2cfc1027a0 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.421+0000 7f2cfaffd640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 0x7f2ce40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:47.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.421+0000 7f2d01cb3640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 0x7f2ce40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.422+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f2cf00cf000 con 0x7f2cfc1027a0 2024-09-05T06:38:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.422+0000 7f2d01cb3640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 0x7f2ce40a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4005fd0 tx=0x7f2cf400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.425+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cf0016030 con 0x7f2cfc1027a0 2024-09-05T06:38:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.644+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cfc075a20 con 0x7f2cfc1027a0 2024-09-05T06:38:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.644+0000 7f2cfaffd640 1 -- 172.21.15.86:0/3742913606 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f2cf009c480 con 0x7f2cfc1027a0 2024-09-05T06:38:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:47.646 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"btime":"2024-09-05T06:38:45:736003+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-05T06:38:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 msgr2=0x7f2ce40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 0x7f2ce40a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4005fd0 tx=0x7f2cf400b390 comp rx=0 tx=0).stop 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 msgr2=0x7f2cfc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000eab0 tx=0x7f2cf000ef70 comp rx=0 tx=0).stop 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 shutdown_connections 2024-09-05T06:38:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2ce40a2c00 0x7f2ce40a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cfc106f20 0x7f2cfc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc1027a0 0x7f2cfc074a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 --2- 172.21.15.86:0/3742913606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cfc101df0 0x7f2cfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 >> 172.21.15.86:0/3742913606 conn(0x7f2cfc0fd8b0 msgr2=0x7f2cfc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 shutdown_connections 2024-09-05T06:38:47.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:47.647+0000 7f2d03f17640 1 -- 172.21.15.86:0/3742913606 wait complete. 2024-09-05T06:38:48.291 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:48.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3742913606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:48.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-05T06:38:48.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:48.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3742913606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3742913606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:38:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- 172.21.15.86:0/785073261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 msgr2=0x7eff6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 --2- 172.21.15.86:0/785073261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c106e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7eff64009930 tx=0x7eff6402a420 comp rx=0 tx=0).stop 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- 172.21.15.86:0/785073261 shutdown_connections 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 --2- 172.21.15.86:0/785073261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c101420 0x7eff6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 --2- 172.21.15.86:0/785073261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c100a80 0x7eff6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 --2- 172.21.15.86:0/785073261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c106e60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- 172.21.15.86:0/785073261 >> 172.21.15.86:0/785073261 conn(0x7eff6c0fc7a0 msgr2=0x7eff6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- 172.21.15.86:0/785073261 shutdown_connections 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- 172.21.15.86:0/785073261 wait complete. 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 Processor -- start 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.058+0000 7eff72ec9640 1 -- start start 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c100a80 0x7eff6c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c101420 0x7eff6c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c110d50 con 0x7eff6c106a80 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c110bd0 con 0x7eff6c101420 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c110ed0 con 0x7eff6c100a80 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41540/0 (socket says 172.21.15.86:41540) 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 -- 172.21.15.86:0/499353236 learned_addr learned my addr 172.21.15.86:0/499353236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff70c65640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c100a80 0x7eff6c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff6bfff640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c101420 0x7eff6c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 -- 172.21.15.86:0/499353236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c100a80 msgr2=0x7eff6c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c100a80 0x7eff6c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 -- 172.21.15.86:0/499353236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c101420 msgr2=0x7eff6c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c101420 0x7eff6c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 -- 172.21.15.86:0/499353236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c19f5a0 con 0x7eff6c106a80 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff71466640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d750 tx=0x7eff5400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54017070 con 0x7eff6c106a80 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff540054d0 con 0x7eff6c106a80 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54004c60 con 0x7eff6c106a80 2024-09-05T06:38:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.059+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c19f890 con 0x7eff6c106a80 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.060+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a7170 con 0x7eff6c106a80 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.060+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c100ee0 con 0x7eff6c106a80 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.062+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7eff54004e00 con 0x7eff6c106a80 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.062+0000 7eff69ffb640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 0x7eff580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.062+0000 7eff70c65640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 0x7eff580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.062+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7eff540ce530 con 0x7eff6c106a80 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.063+0000 7eff70c65640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 0x7eff580a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff64009900 tx=0x7eff64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.066+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff5409b930 con 0x7eff6c106a80 2024-09-05T06:38:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.247+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7eff6c1a7450 con 0x7eff6c106a80 2024-09-05T06:38:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.247+0000 7eff69ffb640 1 -- 172.21.15.86:0/499353236 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7eff5409b230 con 0x7eff6c106a80 2024-09-05T06:38:49.248 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:38:49.249 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-09-05T06:38:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 msgr2=0x7eff580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 0x7eff580a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff64009900 tx=0x7eff64000bc0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 msgr2=0x7eff6c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d750 tx=0x7eff5400dc10 comp rx=0 tx=0).stop 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 shutdown_connections 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff6c106a80 0x7eff6c19eec0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7eff580a2c00 0x7eff580a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff6c101420 0x7eff6c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 --2- 172.21.15.86:0/499353236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6c100a80 0x7eff6c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 >> 172.21.15.86:0/499353236 conn(0x7eff6c0fc7a0 msgr2=0x7eff6c10c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 shutdown_connections 2024-09-05T06:38:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:49.249+0000 7eff72ec9640 1 -- 172.21.15.86:0/499353236 wait complete. 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.589 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:49.590 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:49.590 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:49 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/499353236' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T06:38:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:49 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/499353236' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T06:38:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:49 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/499353236' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T06:38:50.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-05T06:38:50.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-05T06:38:50.813 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:51.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:50 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:50 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:50 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:50 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 -- 172.21.15.86:0/1092871551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4101410 msgr2=0x7f21f410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 --2- 172.21.15.86:0/1092871551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4101410 0x7f21f410d940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009cd0 tx=0x7f21ec02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 -- 172.21.15.86:0/1092871551 shutdown_connections 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 --2- 172.21.15.86:0/1092871551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4101410 0x7f21f410d940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 --2- 172.21.15.86:0/1092871551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4100a70 0x7f21f4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 --2- 172.21.15.86:0/1092871551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4106a70 0x7f21f4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 -- 172.21.15.86:0/1092871551 >> 172.21.15.86:0/1092871551 conn(0x7f21f40fc790 msgr2=0x7f21f40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 -- 172.21.15.86:0/1092871551 shutdown_connections 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.258+0000 7f21f934a640 1 -- 172.21.15.86:0/1092871551 wait complete. 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 Processor -- start 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 -- start start 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4100a70 0x7f21f4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4101410 0x7f21f4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4110d20 con 0x7f21f4106a70 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4110ba0 con 0x7f21f4101410 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f934a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4110ea0 con 0x7f21f4100a70 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41562/0 (socket says 172.21.15.86:41562) 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 -- 172.21.15.86:0/1932585061 learned_addr learned my addr 172.21.15.86:0/1932585061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f2ffd640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4100a70 0x7f21f4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f27fc640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4101410 0x7f21f4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 -- 172.21.15.86:0/1932585061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4100a70 msgr2=0x7f21f4199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4100a70 0x7f21f4199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 -- 172.21.15.86:0/1932585061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4101410 msgr2=0x7f21f4199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4101410 0x7f21f4199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.259+0000 7f21f37fe640 1 -- 172.21.15.86:0/1932585061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41a4710 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21f37fe640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005cc0 tx=0x7f21ec003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec004450 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ec031c90 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41a48e0 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec030e70 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.260+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a4d90 con 0x7f21f4106a70 2024-09-05T06:38:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.261+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4100ed0 con 0x7f21f4106a70 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.262+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21ec0045f0 con 0x7f21f4106a70 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.262+0000 7f21dbfff640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.263+0000 7f21f2ffd640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.263+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f21ec02e030 con 0x7f21f4106a70 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.263+0000 7f21f2ffd640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 0x7f21dc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.267+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21ec04b060 con 0x7f21f4106a70 2024-09-05T06:38:51.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:50 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:50 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:50 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:50 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:50 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:50 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:50 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:50 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:51.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:51.445+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f21f4108fe0 con 0x7f21f4106a70 2024-09-05T06:38:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.338+0000 7f21dbfff640 1 -- 172.21.15.86:0/1932585061 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f21ec0098b0 con 0x7f21f4106a70 2024-09-05T06:38:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-09-05T06:38:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 msgr2=0x7f21dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 0x7f21dc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e400b390 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 msgr2=0x7f21f41a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005cc0 tx=0x7f21ec003c20 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 shutdown_connections 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21f4106a70 0x7f21f41a41d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4101410 0x7f21f4199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 --2- 172.21.15.86:0/1932585061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4100a70 0x7f21f4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.340+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 >> 172.21.15.86:0/1932585061 conn(0x7f21f40fc790 msgr2=0x7f21f40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.341+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 shutdown_connections 2024-09-05T06:38:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:52.341+0000 7f21f934a640 1 -- 172.21.15.86:0/1932585061 wait complete. 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.594 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:52 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T06:38:52.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:52.661 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.662 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.662 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:52.662 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:52.662 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.662 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:52 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T06:38:52.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:38:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:52 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T06:38:53.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-05T06:38:53.320 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:53.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:53 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T06:38:53.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:53 smithi086 ceph-mon[30943]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T06:38:53.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:53 smithi086 ceph-mon[30943]: pgmap v266: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:53 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T06:38:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:53 smithi115 ceph-mon[34791]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T06:38:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:53 smithi115 ceph-mon[34791]: pgmap v266: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:53 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1932585061' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T06:38:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:53 smithi096 ceph-mon[34468]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T06:38:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:53 smithi096 ceph-mon[34468]: pgmap v266: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:53.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 -- 172.21.15.86:0/1091516477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c101420 msgr2=0x7f172c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:53.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 --2- 172.21.15.86:0/1091516477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c101420 0x7f172c10d950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1724009cd0 tx=0x7f172402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 -- 172.21.15.86:0/1091516477 shutdown_connections 2024-09-05T06:38:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 --2- 172.21.15.86:0/1091516477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c101420 0x7f172c10d950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 --2- 172.21.15.86:0/1091516477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 0x7f172c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 --2- 172.21.15.86:0/1091516477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c106a80 0x7f172c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 -- 172.21.15.86:0/1091516477 >> 172.21.15.86:0/1091516477 conn(0x7f172c0fc7a0 msgr2=0x7f172c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 -- 172.21.15.86:0/1091516477 shutdown_connections 2024-09-05T06:38:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.804+0000 7f17333a1640 1 -- 172.21.15.86:0/1091516477 wait complete. 2024-09-05T06:38:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 Processor -- start 2024-09-05T06:38:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 -- start start 2024-09-05T06:38:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 0x7f172c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c101420 0x7f172c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:53.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:53.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f172c110d50 con 0x7f172c106a80 2024-09-05T06:38:53.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f172c110bd0 con 0x7f172c101420 2024-09-05T06:38:53.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f17333a1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f172c110ed0 con 0x7f172c100a80 2024-09-05T06:38:53.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f173193e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:53.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f173193e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54092/0 (socket says 172.21.15.86:54092) 2024-09-05T06:38:53.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f173193e640 1 -- 172.21.15.86:0/2095126005 learned_addr learned my addr 172.21.15.86:0/2095126005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:53.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.805+0000 7f173093c640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c101420 0x7f172c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:53.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173113d640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 0x7f172c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:53.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 -- 172.21.15.86:0/2095126005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 msgr2=0x7f172c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:53.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 0x7f172c197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 -- 172.21.15.86:0/2095126005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c101420 msgr2=0x7f172c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:53.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c101420 0x7f172c197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:53.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 -- 172.21.15.86:0/2095126005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c19c1e0 con 0x7f172c106a80 2024-09-05T06:38:53.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f173193e640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1724005cc0 tx=0x7f1724003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724004240 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17240305a0 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c19c470 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240466c0 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.806+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1a3db0 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.807+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c100ee0 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.808+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1724030ac0 con 0x7f172c106a80 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.809+0000 7f17227fc640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 0x7f17180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:53.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.809+0000 7f173113d640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 0x7f17180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:53.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.809+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f172402e030 con 0x7f172c106a80 2024-09-05T06:38:53.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.809+0000 7f173113d640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 0x7f17180a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f172800a810 tx=0x7f1728008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:53.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.813+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f172404b060 con 0x7f172c106a80 2024-09-05T06:38:53.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:53.979+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f172c109070 con 0x7f172c106a80 2024-09-05T06:38:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.352+0000 7f17227fc640 1 -- 172.21.15.86:0/2095126005 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7f17240bd590 con 0x7f172c106a80 2024-09-05T06:38:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-09-05T06:38:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 msgr2=0x7f17180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 0x7f17180a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f172800a810 tx=0x7f1728008040 comp rx=0 tx=0).stop 2024-09-05T06:38:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 msgr2=0x7f172c19bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1724005cc0 tx=0x7f1724003ba0 comp rx=0 tx=0).stop 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 shutdown_connections 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c106a80 0x7f172c19bb00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f17180a2b80 0x7f17180a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f172c101420 0x7f172c197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 --2- 172.21.15.86:0/2095126005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f172c100a80 0x7f172c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 >> 172.21.15.86:0/2095126005 conn(0x7f172c0fc7a0 msgr2=0x7f172c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 shutdown_connections 2024-09-05T06:38:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:54.354+0000 7f17333a1640 1 -- 172.21.15.86:0/2095126005 wait complete. 2024-09-05T06:38:54.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:54 smithi086 ceph-mon[30943]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:38:54.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:54 smithi086 ceph-mon[30943]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T06:38:54.680 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:54 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T06:38:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:54 smithi115 ceph-mon[34791]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:38:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:54 smithi115 ceph-mon[34791]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T06:38:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:54 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T06:38:54.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:54 smithi096 ceph-mon[34468]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:38:54.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:54 smithi096 ceph-mon[34468]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T06:38:54.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:54 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T06:38:55.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-05T06:38:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:55.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:55 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T06:38:55.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:55 smithi086 ceph-mon[30943]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T06:38:55.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:55 smithi086 ceph-mon[30943]: pgmap v269: 97 pgs: 12 active+clean, 15 creating+peering, 70 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:55 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T06:38:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:55 smithi115 ceph-mon[34791]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T06:38:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:55 smithi115 ceph-mon[34791]: pgmap v269: 97 pgs: 12 active+clean, 15 creating+peering, 70 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 -- 172.21.15.86:0/925005033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 msgr2=0x7ff198100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 --2- 172.21.15.86:0/925005033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff198100ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009930 tx=0x7ff18c02a420 comp rx=0 tx=0).stop 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 -- 172.21.15.86:0/925005033 shutdown_connections 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 --2- 172.21.15.86:0/925005033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198101430 0x7ff19810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 --2- 172.21.15.86:0/925005033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff198100ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 --2- 172.21.15.86:0/925005033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198106a90 0x7ff198106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 -- 172.21.15.86:0/925005033 >> 172.21.15.86:0/925005033 conn(0x7ff1980fc7d0 msgr2=0x7ff1980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 -- 172.21.15.86:0/925005033 shutdown_connections 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.716+0000 7ff19e552640 1 -- 172.21.15.86:0/925005033 wait complete. 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 Processor -- start 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 -- start start 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 0x7ff1980fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 0x7ff1980fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff198110d00 con 0x7ff198100a90 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff198110b80 con 0x7ff198101430 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19e552640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff198110e80 con 0x7ff198106a90 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54104/0 (socket says 172.21.15.86:54104) 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 -- 172.21.15.86:0/2558822210 learned_addr learned my addr 172.21.15.86:0/2558822210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19caef640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 0x7ff1980fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff1977fe640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 0x7ff1980fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 -- 172.21.15.86:0/2558822210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 msgr2=0x7ff1980fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 0x7ff1980fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 -- 172.21.15.86:0/2558822210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 msgr2=0x7ff1980fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 0x7ff1980fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff197fff640 1 -- 172.21.15.86:0/2558822210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981a6870 con 0x7ff198100a90 2024-09-05T06:38:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff19caef640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 0x7ff1980fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.717+0000 7ff1977fe640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 0x7ff1980fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff197fff640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff18800e9f0 tx=0x7ff18800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18800cd80 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981a6b60 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1880108e0 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188018920 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.718+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981a70c0 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.719+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff198100ef0 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.721+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff188010a90 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.721+0000 7ff1957fa640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.721+0000 7ff1977fe640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.722+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7ff1880cf6e0 con 0x7ff198100a90 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.722+0000 7ff1977fe640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000c00 tx=0x7ff18c009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:55.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.725+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff18809c7f0 con 0x7ff198100a90 2024-09-05T06:38:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:55 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2095126005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T06:38:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:55 smithi096 ceph-mon[34468]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T06:38:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:55 smithi096 ceph-mon[34468]: pgmap v269: 97 pgs: 12 active+clean, 15 creating+peering, 70 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:55.925+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff198109aa0 con 0x7ff198100a90 2024-09-05T06:38:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.364+0000 7ff1957fa640 1 -- 172.21.15.86:0/2558822210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff18809c0f0 con 0x7ff198100a90 2024-09-05T06:38:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 msgr2=0x7ff1800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000c00 tx=0x7ff18c009610 comp rx=0 tx=0).stop 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 msgr2=0x7ff1980fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff18800e9f0 tx=0x7ff18800eeb0 comp rx=0 tx=0).stop 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 shutdown_connections 2024-09-05T06:38:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff198106a90 0x7ff1980fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff198101430 0x7ff1980fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 --2- 172.21.15.86:0/2558822210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff198100a90 0x7ff1980fce50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 >> 172.21.15.86:0/2558822210 conn(0x7ff1980fc7d0 msgr2=0x7ff19806aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 shutdown_connections 2024-09-05T06:38:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:56.366+0000 7ff19e552640 1 -- 172.21.15.86:0/2558822210 wait complete. 2024-09-05T06:38:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:56 smithi115 ceph-mon[34791]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T06:38:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:56 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T06:38:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:56 smithi115 ceph-mon[34791]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T06:38:56.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:56 smithi086 ceph-mon[30943]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T06:38:56.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:56 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T06:38:56.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:56 smithi086 ceph-mon[30943]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T06:38:56.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:56 smithi096 ceph-mon[34468]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T06:38:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:56 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T06:38:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:56 smithi096 ceph-mon[34468]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T06:38:57.024 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-05T06:38:57.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-05T06:38:57.269 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:38:57.625 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:active 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:38:57.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:57 smithi086 ceph-mon[30943]: pgmap v272: 97 pgs: 27 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T06:38:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T06:38:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T06:38:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T06:38:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:active 2024-09-05T06:38:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:38:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:57 smithi115 ceph-mon[34791]: pgmap v272: 97 pgs: 27 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 -- 172.21.15.86:0/192187760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41036e0 msgr2=0x7f1ee4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 --2- 172.21.15.86:0/192187760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41036e0 0x7f1ee4107b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009cd0 tx=0x7f1ed802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 -- 172.21.15.86:0/192187760 shutdown_connections 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 --2- 172.21.15.86:0/192187760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ee41082a0 0x7f1ee410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 --2- 172.21.15.86:0/192187760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee41036e0 0x7f1ee4107b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 --2- 172.21.15.86:0/192187760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4102d30 0x7f1ee4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 -- 172.21.15.86:0/192187760 >> 172.21.15.86:0/192187760 conn(0x7f1ee40fc7d0 msgr2=0x7f1ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 -- 172.21.15.86:0/192187760 shutdown_connections 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.709+0000 7f1eec6de640 1 -- 172.21.15.86:0/192187760 wait complete. 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 Processor -- start 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 -- start start 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ee41036e0 0x7f1ee4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee41082a0 0x7f1ee419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4111f50 con 0x7f1ee4102d30 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4111dd0 con 0x7f1ee41036e0 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eec6de640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41120d0 con 0x7f1ee41082a0 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eea47a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eea47a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54128/0 (socket says 172.21.15.86:54128) 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.710+0000 7f1eea47a640 1 -- 172.21.15.86:0/1108732549 learned_addr learned my addr 172.21.15.86:0/1108732549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 -- 172.21.15.86:0/1108732549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee41082a0 msgr2=0x7f1ee419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee41082a0 0x7f1ee419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 -- 172.21.15.86:0/1108732549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ee41036e0 msgr2=0x7f1ee4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ee41036e0 0x7f1ee4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 -- 172.21.15.86:0/1108732549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee419d4a0 con 0x7f1ee4102d30 2024-09-05T06:38:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eea47a640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d770 tx=0x7f1edc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc017070 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee419d790 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1edc0054d0 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc004cf0 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.711+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee41a5070 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.712+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee4103110 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.713+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1edc004e90 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.713+0000 7f1ee37fe640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 0x7f1ecc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.714+0000 7f1ee9c79640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 0x7f1ecc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.714+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1edc0cea10 con 0x7f1ee4102d30 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.714+0000 7f1ee9c79640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 0x7f1ecc0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006010 tx=0x7f1ed8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:38:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.718+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1edc09bac0 con 0x7f1ee4102d30 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2558822210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] up:active 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:38:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:57 smithi096 ceph-mon[34468]: pgmap v272: 97 pgs: 27 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:38:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:57.894+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f1ee4199850 con 0x7f1ee4102d30 2024-09-05T06:38:58.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:58 smithi115 ceph-mon[34791]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T06:38:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:58 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T06:38:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:58 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T06:38:58.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:58 smithi086 ceph-mon[30943]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T06:38:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:58 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T06:38:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:58 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T06:38:58.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:58 smithi096 ceph-mon[34468]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T06:38:58.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:58 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T06:38:58.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:58 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T06:38:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.833+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1108732549 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f1edc09b3c0 con 0x7f1ee4102d30 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 msgr2=0x7f1ecc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 0x7f1ecc0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006010 tx=0x7f1ed8009360 comp rx=0 tx=0).stop 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 msgr2=0x7f1ee41984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d770 tx=0x7f1edc00dc30 comp rx=0 tx=0).stop 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 shutdown_connections 2024-09-05T06:38:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee41082a0 0x7f1ee419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1ecc0a2b80 0x7f1ecc0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ee41036e0 0x7f1ee4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 --2- 172.21.15.86:0/1108732549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee4102d30 0x7f1ee41984f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 >> 172.21.15.86:0/1108732549 conn(0x7f1ee40fc7d0 msgr2=0x7f1ee4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 shutdown_connections 2024-09-05T06:38:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:38:58.836+0000 7f1eec6de640 1 -- 172.21.15.86:0/1108732549 wait complete. 2024-09-05T06:38:59.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-05T06:38:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:00.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:59 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T06:39:00.168 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:59 smithi086 ceph-mon[30943]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T06:39:00.168 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:38:59 smithi086 ceph-mon[30943]: pgmap v274: 97 pgs: 65 active+clean, 14 creating+peering, 18 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 4 op/s 2024-09-05T06:39:00.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:59 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T06:39:00.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:59 smithi115 ceph-mon[34791]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T06:39:00.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:38:59 smithi115 ceph-mon[34791]: pgmap v274: 97 pgs: 65 active+clean, 14 creating+peering, 18 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 4 op/s 2024-09-05T06:39:00.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 -- 172.21.15.86:0/141010568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 msgr2=0x7f596c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:00.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 --2- 172.21.15.86:0/141010568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c1006f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5968009900 tx=0x7f596802a410 comp rx=0 tx=0).stop 2024-09-05T06:39:00.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 -- 172.21.15.86:0/141010568 shutdown_connections 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 --2- 172.21.15.86:0/141010568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f30 0x7f596c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 --2- 172.21.15.86:0/141010568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f596c100cc0 0x7f596c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 --2- 172.21.15.86:0/141010568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c1006f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 -- 172.21.15.86:0/141010568 >> 172.21.15.86:0/141010568 conn(0x7f596c077210 msgr2=0x7f596c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 -- 172.21.15.86:0/141010568 shutdown_connections 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.255+0000 7f59739d4640 1 -- 172.21.15.86:0/141010568 wait complete. 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 Processor -- start 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 -- start start 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f596c100cc0 0x7f596c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f30 0x7f596c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:00.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111e60 con 0x7f596c100310 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111ce0 con 0x7f596c100cc0 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f59739d4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111fe0 con 0x7f596c106f30 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f5971770640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f5971770640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54150/0 (socket says 172.21.15.86:54150) 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.256+0000 7f5971770640 1 -- 172.21.15.86:0/4218464698 learned_addr learned my addr 172.21.15.86:0/4218464698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 -- 172.21.15.86:0/4218464698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f30 msgr2=0x7f596c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f30 0x7f596c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 -- 172.21.15.86:0/4218464698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f596c100cc0 msgr2=0x7f596c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f596c100cc0 0x7f596c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 -- 172.21.15.86:0/4218464698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a68e0 con 0x7f596c100310 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f5971770640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5968006010 tx=0x7f5968004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596803f030 con 0x7f596c100310 2024-09-05T06:39:00.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59680035f0 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a6b70 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968035070 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.257+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a7050 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.258+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c1006f0 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.259+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5968003b10 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.260+0000 7f59627fc640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 0x7f59580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.260+0000 7f5970f6f640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 0x7f59580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.260+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f59680eff10 con 0x7f596c100310 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.260+0000 7f5970f6f640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 0x7f59580a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f596c102d20 tx=0x7f595c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:00.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.263+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59680bcde0 con 0x7f596c100310 2024-09-05T06:39:00.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:59 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1108732549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T06:39:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:59 smithi096 ceph-mon[34468]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T06:39:00.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:38:59 smithi096 ceph-mon[34468]: pgmap v274: 97 pgs: 65 active+clean, 14 creating+peering, 18 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 4 op/s 2024-09-05T06:39:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:00.428+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f596c1027c0 con 0x7f596c100310 2024-09-05T06:39:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:00 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T06:39:01.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 06:39:00 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T06:39:00.750+0000 7f2b817d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-05T06:39:01.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:00 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T06:39:01.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:00 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T06:39:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.855+0000 7f59627fc640 1 -- 172.21.15.86:0/4218464698 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f59680bc6e0 con 0x7f596c100310 2024-09-05T06:39:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 msgr2=0x7f59580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 0x7f59580a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f596c102d20 tx=0x7f595c006c90 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 msgr2=0x7f596c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5968006010 tx=0x7f5968004270 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 shutdown_connections 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f596c106f30 0x7f596c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59580a2b80 0x7f59580a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f596c100cc0 0x7f596c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 --2- 172.21.15.86:0/4218464698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f596c100310 0x7f596c101290 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:01.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 >> 172.21.15.86:0/4218464698 conn(0x7f596c077210 msgr2=0x7f596c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:01.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 shutdown_connections 2024-09-05T06:39:01.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:01.859+0000 7f59739d4640 1 -- 172.21.15.86:0/4218464698 wait complete. 2024-09-05T06:39:02.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:01 smithi115 ceph-mon[34791]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T06:39:02.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:01 smithi115 ceph-mon[34791]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-05T06:39:02.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:01 smithi086 ceph-mon[30943]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T06:39:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:01 smithi086 ceph-mon[30943]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-05T06:39:02.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:01 smithi096 ceph-mon[34468]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T06:39:02.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:01 smithi096 ceph-mon[34468]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-05T06:39:02.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-05T06:39:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:02.865 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:02 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T06:39:02.865 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:02 smithi086 ceph-mon[30943]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T06:39:03.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:02 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T06:39:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:02 smithi096 ceph-mon[34468]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T06:39:03.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:02 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4218464698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T06:39:03.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:02 smithi115 ceph-mon[34791]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T06:39:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/3224059999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 msgr2=0x7f3ac41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/3224059999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41029b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009900 tx=0x7f3aac02a480 comp rx=0 tx=0).stop 2024-09-05T06:39:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/3224059999 shutdown_connections 2024-09-05T06:39:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/3224059999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/3224059999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41029b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/3224059999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 0x7f3ac4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/3224059999 >> 172.21.15.86:0/3224059999 conn(0x7f3ac40fc7d0 msgr2=0x7f3ac40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/3224059999 shutdown_connections 2024-09-05T06:39:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.261+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/3224059999 wait complete. 2024-09-05T06:39:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 Processor -- start 2024-09-05T06:39:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 -- start start 2024-09-05T06:39:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 0x7f3ac4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4110e30 con 0x7f3ac4102570 2024-09-05T06:39:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4110cb0 con 0x7f3ac4102ef0 2024-09-05T06:39:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac8ee0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4110fb0 con 0x7f3ac4100e20 2024-09-05T06:39:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51408/0 (socket says 172.21.15.86:51408) 2024-09-05T06:39:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac1d9b640 1 -- 172.21.15.86:0/2214648895 learned_addr learned my addr 172.21.15.86:0/2214648895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac259c640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 0x7f3ac4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.262+0000 7f3ac2d9d640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 -- 172.21.15.86:0/2214648895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 msgr2=0x7f3ac4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 0x7f3ac4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 -- 172.21.15.86:0/2214648895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 msgr2=0x7f3ac419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 -- 172.21.15.86:0/2214648895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac419d420 con 0x7f3ac4102570 2024-09-05T06:39:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac2d9d640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:39:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac1d9b640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac03f070 con 0x7f3ac4102570 2024-09-05T06:39:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aac003aa0 con 0x7f3ac4102570 2024-09-05T06:39:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac419d6b0 con 0x7f3ac4102570 2024-09-05T06:39:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac035030 con 0x7f3ac4102570 2024-09-05T06:39:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.263+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41a4ff0 con 0x7f3ac4102570 2024-09-05T06:39:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.264+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac41021b0 con 0x7f3ac4102570 2024-09-05T06:39:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.265+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3aac042070 con 0x7f3ac4102570 2024-09-05T06:39:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.267+0000 7f3abb7fe640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 0x7f3ab00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.267+0000 7f3ac259c640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 0x7f3ab00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.267+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f3aac0efe60 con 0x7f3ac4102570 2024-09-05T06:39:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.267+0000 7f3ac259c640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 0x7f3ab00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40058c0 tx=0x7f3ab400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.270+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3aac0bcb40 con 0x7f3ac4102570 2024-09-05T06:39:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.445+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f3ac410a0a0 con 0x7f3ac4102570 2024-09-05T06:39:03.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.868+0000 7f3abb7fe640 1 -- 172.21.15.86:0/2214648895 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f3aac0bc440 con 0x7f3ac4102570 2024-09-05T06:39:03.869 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-09-05T06:39:03.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 msgr2=0x7f3ab00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:03.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 0x7f3ab00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40058c0 tx=0x7f3ab400a250 comp rx=0 tx=0).stop 2024-09-05T06:39:03.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 msgr2=0x7f3ac41989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:03.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac0042f0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 shutdown_connections 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f3ab00a2b80 0x7f3ab00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ac4102ef0 0x7f3ac419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac4102570 0x7f3ac41989b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 --2- 172.21.15.86:0/2214648895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ac4100e20 0x7f3ac4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 >> 172.21.15.86:0/2214648895 conn(0x7f3ac40fc7d0 msgr2=0x7f3ac406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 shutdown_connections 2024-09-05T06:39:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:03.871+0000 7f3ac8ee0640 1 -- 172.21.15.86:0/2214648895 wait complete. 2024-09-05T06:39:04.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:03 smithi115 ceph-mon[34791]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T06:39:04.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:03 smithi115 ceph-mon[34791]: pgmap v279: 129 pgs: 1 creating+activating, 2 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 5 op/s 2024-09-05T06:39:04.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:03 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T06:39:04.222 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:03 smithi086 ceph-mon[30943]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T06:39:04.222 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:03 smithi086 ceph-mon[30943]: pgmap v279: 129 pgs: 1 creating+activating, 2 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 5 op/s 2024-09-05T06:39:04.222 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:03 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T06:39:04.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:03 smithi096 ceph-mon[34468]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T06:39:04.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:03 smithi096 ceph-mon[34468]: pgmap v279: 129 pgs: 1 creating+activating, 2 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 5 op/s 2024-09-05T06:39:04.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:03 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T06:39:04.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-05T06:39:04.764 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:05.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:04 smithi086 ceph-mon[30943]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:39:05.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:04 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T06:39:05.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:04 smithi086 ceph-mon[30943]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T06:39:05.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:04 smithi115 ceph-mon[34791]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:39:05.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:04 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T06:39:05.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:04 smithi115 ceph-mon[34791]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.234+0000 7f0e3c616640 1 -- 172.21.15.86:0/1911323476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34103660 msgr2=0x7f0e3410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.234+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1911323476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34103660 0x7f0e3410bb50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009900 tx=0x7f0e2402a480 comp rx=0 tx=0).stop 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- 172.21.15.86:0/1911323476 shutdown_connections 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1911323476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e3410c170 0x7f0e3410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1911323476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34103660 0x7f0e3410bb50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1911323476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e34102d40 0x7f0e34103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- 172.21.15.86:0/1911323476 >> 172.21.15.86:0/1911323476 conn(0x7f0e340fc7a0 msgr2=0x7f0e340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- 172.21.15.86:0/1911323476 shutdown_connections 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- 172.21.15.86:0/1911323476 wait complete. 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 Processor -- start 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- start start 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e34103660 0x7f0e3419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 0x7f0e3419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34111e20 con 0x7f0e34102d40 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34111ca0 con 0x7f0e3410c170 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3c616640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34111fa0 con 0x7f0e34103660 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3a3b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:05.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3a3b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51424/0 (socket says 172.21.15.86:51424) 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.235+0000 7f0e3a3b2640 1 -- 172.21.15.86:0/1267557197 learned_addr learned my addr 172.21.15.86:0/1267557197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3abb3640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 0x7f0e3419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e39bb1640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e34103660 0x7f0e3419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 -- 172.21.15.86:0/1267557197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e34103660 msgr2=0x7f0e3419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e34103660 0x7f0e3419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 -- 172.21.15.86:0/1267557197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 msgr2=0x7f0e3419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 0x7f0e3419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 -- 172.21.15.86:0/1267557197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3419f4b0 con 0x7f0e34102d40 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3abb3640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 0x7f0e3419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3a3b2640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c920 tx=0x7f0e3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30007c70 con 0x7f0e34102d40 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e30007e10 con 0x7f0e34102d40 2024-09-05T06:39:05.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30018820 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3419f7a0 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.236+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a7080 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.237+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34103120 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.238+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e300189c0 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.239+0000 7f0e2b7fe640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 0x7f0e200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.239+0000 7f0e39bb1640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 0x7f0e200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.239+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f0e300cf6c0 con 0x7f0e34102d40 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.239+0000 7f0e39bb1640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 0x7f0e200a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0e24031440 tx=0x7f0e240095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:05.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.242+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e30014030 con 0x7f0e34102d40 2024-09-05T06:39:05.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:04 smithi096 ceph-mon[34468]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T06:39:05.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:04 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2214648895' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T06:39:05.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:04 smithi096 ceph-mon[34468]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T06:39:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.444+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f0e341a7380 con 0x7f0e34102d40 2024-09-05T06:39:05.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.880+0000 7f0e2b7fe640 1 -- 172.21.15.86:0/1267557197 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f0e3009c4b0 con 0x7f0e34102d40 2024-09-05T06:39:05.881 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 msgr2=0x7f0e200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 0x7f0e200a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0e24031440 tx=0x7f0e240095f0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 msgr2=0x7f0e3419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c920 tx=0x7f0e3000cde0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 shutdown_connections 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0e200a2c00 0x7f0e200a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e3410c170 0x7f0e3419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e34103660 0x7f0e3419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 --2- 172.21.15.86:0/1267557197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e34102d40 0x7f0e3419a500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 >> 172.21.15.86:0/1267557197 conn(0x7f0e340fc7a0 msgr2=0x7f0e340fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 shutdown_connections 2024-09-05T06:39:05.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:05.883+0000 7f0e3c616640 1 -- 172.21.15.86:0/1267557197 wait complete. 2024-09-05T06:39:06.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:05 smithi115 ceph-mon[34791]: pgmap v281: 129 pgs: 1 creating+activating, 2 creating+peering, 18 unknown, 108 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-09-05T06:39:06.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:05 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T06:39:06.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:05 smithi086 ceph-mon[30943]: pgmap v281: 129 pgs: 1 creating+activating, 2 creating+peering, 18 unknown, 108 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-09-05T06:39:06.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:05 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T06:39:06.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:05 smithi096 ceph-mon[34468]: pgmap v281: 129 pgs: 1 creating+activating, 2 creating+peering, 18 unknown, 108 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 1 op/s 2024-09-05T06:39:06.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:05 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T06:39:06.570 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:39:06.818 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:07.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:06 smithi086 ceph-mon[30943]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T06:39:07.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:06 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T06:39:07.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:06 smithi086 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:07.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:06 smithi115 ceph-mon[34791]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T06:39:07.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:06 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T06:39:07.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:06 smithi115 ceph-mon[34791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:07.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/3358494858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 msgr2=0x7f9d3810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:07.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/3358494858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3810e600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009ca0 tx=0x7f9d3002a950 comp rx=0 tx=0).stop 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/3358494858 shutdown_connections 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/3358494858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3810e600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/3358494858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 0x7f9d3810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/3358494858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 0x7f9d38103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/3358494858 >> 172.21.15.86:0/3358494858 conn(0x7f9d380fc7a0 msgr2=0x7f9d380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/3358494858 shutdown_connections 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/3358494858 wait complete. 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 Processor -- start 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.254+0000 7f9d3d9a8640 1 -- start start 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 0x7f9d381983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 0x7f9d381988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38111e10 con 0x7f9d3810c180 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38111c90 con 0x7f9d381036c0 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38111f90 con 0x7f9d38102da0 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51436/0 (socket says 172.21.15.86:51436) 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 -- 172.21.15.86:0/4204330225 learned_addr learned my addr 172.21.15.86:0/4204330225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d2ffff640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 0x7f9d381988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d36ffd640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 0x7f9d381983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 -- 172.21.15.86:0/4204330225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 msgr2=0x7f9d381983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 0x7f9d381983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 -- 172.21.15.86:0/4204330225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 msgr2=0x7f9d381988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 0x7f9d381988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 -- 172.21.15.86:0/4204330225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3819d350 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d377fe640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d3002ae70 tx=0x7f9d3002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30034040 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d300308f0 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3819d5e0 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.255+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d300047f0 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.256+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d381a4f20 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.256+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38103180 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.258+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9d30037020 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.258+0000 7f9d3c9a6640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 0x7f9d240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.258+0000 7f9d36ffd640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 0x7f9d240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.259+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9d300f0270 con 0x7f9d3810c180 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.259+0000 7f9d36ffd640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 0x7f9d240a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005fd0 tx=0x7f9d28005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.262+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d300bcf90 con 0x7f9d3810c180 2024-09-05T06:39:07.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:06 smithi096 ceph-mon[34468]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T06:39:07.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:06 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1267557197' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T06:39:07.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:06 smithi096 ceph-mon[34468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.461+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d38107220 con 0x7f9d3810c180 2024-09-05T06:39:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.461+0000 7f9d3c9a6640 1 -- 172.21.15.86:0/4204330225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11839 (secure 0 0 0) 0x7f9d300bc890 con 0x7f9d3810c180 2024-09-05T06:39:07.462 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:07.462 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"btime":"2024-09-05T06:39:05:870955+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:05.445549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24479,"qdb_cluster":[24479]},"id":1}]} 2024-09-05T06:39:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 msgr2=0x7f9d240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 0x7f9d240a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005fd0 tx=0x7f9d28005850 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 msgr2=0x7f9d3819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d3002ae70 tx=0x7f9d3002f5d0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 shutdown_connections 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3810c180 0x7f9d3819cc70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9d240a2c00 0x7f9d240a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d381036c0 0x7f9d381988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 --2- 172.21.15.86:0/4204330225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102da0 0x7f9d381983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 >> 172.21.15.86:0/4204330225 conn(0x7f9d380fc7a0 msgr2=0x7f9d380fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 shutdown_connections 2024-09-05T06:39:07.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:07.464+0000 7f9d3d9a8640 1 -- 172.21.15.86:0/4204330225 wait complete. 2024-09-05T06:39:08.129 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:07 smithi086 ceph-mon[30943]: pgmap v283: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:08.129 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:07 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4204330225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:08.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:08.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:07 smithi115 ceph-mon[34791]: pgmap v283: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:08.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:07 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4204330225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:08.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:07 smithi096 ceph-mon[34468]: pgmap v283: 129 pgs: 1 creating+activating, 2 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:08.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:07 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4204330225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:08.402 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:08.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 -- 172.21.15.86:0/365878092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c108300 msgr2=0x7fa14c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 --2- 172.21.15.86:0/365878092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c108300 0x7fa14c10ec70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa144009cd0 tx=0x7fa14402a8d0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 -- 172.21.15.86:0/365878092 shutdown_connections 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 --2- 172.21.15.86:0/365878092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c108300 0x7fa14c10ec70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 --2- 172.21.15.86:0/365878092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c103780 0x7fa14c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 --2- 172.21.15.86:0/365878092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102dd0 0x7fa14c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 -- 172.21.15.86:0/365878092 >> 172.21.15.86:0/365878092 conn(0x7fa14c0fc7d0 msgr2=0x7fa14c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 -- 172.21.15.86:0/365878092 shutdown_connections 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 -- 172.21.15.86:0/365878092 wait complete. 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.854+0000 7fa15418a640 1 Processor -- start 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 -- start start 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c102dd0 0x7fa14c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:08.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 0x7fa14c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c111f80 con 0x7fa14c103780 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c111e00 con 0x7fa14c108300 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c112100 con 0x7fa14c102dd0 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51458/0 (socket says 172.21.15.86:51458) 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 -- 172.21.15.86:0/2316175728 learned_addr learned my addr 172.21.15.86:0/2316175728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151f26640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c102dd0 0x7fa14c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa152727640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 0x7fa14c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 -- 172.21.15.86:0/2316175728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c102dd0 msgr2=0x7fa14c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c102dd0 0x7fa14c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 -- 172.21.15.86:0/2316175728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 msgr2=0x7fa14c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 0x7fa14c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 -- 172.21.15.86:0/2316175728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c19f5a0 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa152727640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 0x7fa14c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa151725640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c8c0 tx=0x7fa14000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007bb0 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa140007d50 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c19f890 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.855+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400187c0 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.856+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a7170 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.856+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c1031b0 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.858+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa14000f450 con 0x7fa14c103780 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.858+0000 7fa13effd640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 0x7fa1380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.858+0000 7fa151f26640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 0x7fa1380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.858+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fa1400cf450 con 0x7fa14c103780 2024-09-05T06:39:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.859+0000 7fa151f26640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 0x7fa1380a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa148005e10 tx=0x7fa148005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:08.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:08.862+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa140014030 con 0x7fa14c103780 2024-09-05T06:39:09.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.030+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa14c1a7340 con 0x7fa14c103780 2024-09-05T06:39:09.031 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:08 smithi086 ceph-mon[30943]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T06:39:09.031 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:08 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T06:39:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.032+0000 7fa13effd640 1 -- 172.21.15.86:0/2316175728 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24175 (secure 0 0 0) 0x7fa14001f300 con 0x7fa14c103780 2024-09-05T06:39:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:09.033 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:05.445420+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 msgr2=0x7fa1380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 0x7fa1380a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa148005e10 tx=0x7fa148005da0 comp rx=0 tx=0).stop 2024-09-05T06:39:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 msgr2=0x7fa14c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c8c0 tx=0x7fa14000cd80 comp rx=0 tx=0).stop 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 shutdown_connections 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa1380a2b80 0x7fa1380a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108300 0x7fa14c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa14c103780 0x7fa14c19ab30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 --2- 172.21.15.86:0/2316175728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14c102dd0 0x7fa14c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 >> 172.21.15.86:0/2316175728 conn(0x7fa14c0fc7d0 msgr2=0x7fa14c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 shutdown_connections 2024-09-05T06:39:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:09.034+0000 7fa15418a640 1 -- 172.21.15.86:0/2316175728 wait complete. 2024-09-05T06:39:09.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:08 smithi115 ceph-mon[34791]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T06:39:09.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:08 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T06:39:09.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:08 smithi096 ceph-mon[34468]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T06:39:09.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:08 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T06:39:09.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:09.964 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:10.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:09 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2316175728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:10.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:09 smithi115 ceph-mon[34791]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:10.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:09 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2316175728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:10.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:09 smithi086 ceph-mon[30943]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:10.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:09 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2316175728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:10.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:09 smithi096 ceph-mon[34468]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.462+0000 7f2a1c266640 1 -- 172.21.15.86:0/1229833554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 msgr2=0x7f2a141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.462+0000 7f2a1c266640 1 --2- 172.21.15.86:0/1229833554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a141021c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009900 tx=0x7f2a1002a410 comp rx=0 tx=0).stop 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- 172.21.15.86:0/1229833554 shutdown_connections 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- 172.21.15.86:0/1229833554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14106f10 0x7f2a1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- 172.21.15.86:0/1229833554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14102790 0x7f2a141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- 172.21.15.86:0/1229833554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a141021c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- 172.21.15.86:0/1229833554 >> 172.21.15.86:0/1229833554 conn(0x7f2a140fd8c0 msgr2=0x7f2a140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- 172.21.15.86:0/1229833554 shutdown_connections 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- 172.21.15.86:0/1229833554 wait complete. 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 Processor -- start 2024-09-05T06:39:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- start start 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14102790 0x7f2a141988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14106f10 0x7f2a1419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14111e20 con 0x7f2a14101de0 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14111ca0 con 0x7f2a14102790 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.463+0000 7f2a1c266640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14111fa0 con 0x7f2a14106f10 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51468/0 (socket says 172.21.15.86:51468) 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 -- 172.21.15.86:0/511011527 learned_addr learned my addr 172.21.15.86:0/511011527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a19801640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14102790 0x7f2a141988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a803640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14106f10 0x7f2a1419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 -- 172.21.15.86:0/511011527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14106f10 msgr2=0x7f2a1419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14106f10 0x7f2a1419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 -- 172.21.15.86:0/511011527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14102790 msgr2=0x7f2a141988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14102790 0x7f2a141988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 -- 172.21.15.86:0/511011527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1419d310 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1a002640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005940 tx=0x7f2a10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1003f030 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1419d5a0 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a10003aa0 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10035070 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.464+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141a4ee0 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.465+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a141032c0 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.466+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a10042030 con 0x7f2a14101de0 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.467+0000 7f2a0affd640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 0x7f2a000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.467+0000 7f2a19801640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 0x7f2a000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.467+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f2a100efae0 con 0x7f2a14101de0 2024-09-05T06:39:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.467+0000 7f2a19801640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 0x7f2a000a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.471+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a100bc800 con 0x7f2a14101de0 2024-09-05T06:39:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.640+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1410a0f0 con 0x7f2a14101de0 2024-09-05T06:39:10.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.641+0000 7f2a0affd640 1 -- 172.21.15.86:0/511011527 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24175 (secure 0 0 0) 0x7f2a100bc100 con 0x7f2a14101de0 2024-09-05T06:39:10.642 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:10.642 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:05.445420+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.643+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 msgr2=0x7f2a000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.643+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 0x7f2a000a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04005850 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.643+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 msgr2=0x7f2a14198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.643+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005940 tx=0x7f2a10004270 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 shutdown_connections 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a14106f10 0x7f2a1419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2a000a2b80 0x7f2a000a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a14102790 0x7f2a141988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 --2- 172.21.15.86:0/511011527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14101de0 0x7f2a14198360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 >> 172.21.15.86:0/511011527 conn(0x7f2a140fd8c0 msgr2=0x7f2a140fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 shutdown_connections 2024-09-05T06:39:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:10.644+0000 7f2a1c266640 1 -- 172.21.15.86:0/511011527 wait complete. 2024-09-05T06:39:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/511011527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:11.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/511011527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:11.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/511011527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:11.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:39:11.599 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T06:39:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T06:39:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T06:39:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-05T06:39:11.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:11 smithi086 ceph-mon[30943]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 -- 172.21.15.86:0/954811726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 msgr2=0x7f38001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 --2- 172.21.15.86:0/954811726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f38001067e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009900 tx=0x7f37f002a410 comp rx=0 tx=0).stop 2024-09-05T06:39:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 -- 172.21.15.86:0/954811726 shutdown_connections 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 --2- 172.21.15.86:0/954811726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800106f10 0x7f380010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 --2- 172.21.15.86:0/954811726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f38001067e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 --2- 172.21.15.86:0/954811726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800101de0 0x7f38001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 -- 172.21.15.86:0/954811726 >> 172.21.15.86:0/954811726 conn(0x7f38000fd8c0 msgr2=0x7f38000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 -- 172.21.15.86:0/954811726 shutdown_connections 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.030+0000 7f3807c1c640 1 -- 172.21.15.86:0/954811726 wait complete. 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 Processor -- start 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 -- start start 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 0x7f38000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 0x7f3800074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3800111e20 con 0x7f3800102790 2024-09-05T06:39:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3800111ca0 con 0x7f3800106f10 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f3807c1c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3800111fa0 con 0x7f3800101de0 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51492/0 (socket says 172.21.15.86:51492) 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 -- 172.21.15.86:0/2819993156 learned_addr learned my addr 172.21.15.86:0/2819993156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38059b8640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 0x7f38000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 -- 172.21.15.86:0/2819993156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 msgr2=0x7f38000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38061b9640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 0x7f3800074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 0x7f38000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 -- 172.21.15.86:0/2819993156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 msgr2=0x7f3800074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 0x7f3800074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38051b7640 1 -- 172.21.15.86:0/2819993156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38001a67c0 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38061b9640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 0x7f3800074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.031+0000 7f38059b8640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 0x7f38000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f38051b7640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37f0005940 tx=0x7f37f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f003f030 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38001a6a50 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37f0003aa0 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0035070 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.032+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38001a6f80 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.033+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38001021c0 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.034+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37f0042030 con 0x7f3800102790 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.034+0000 7f37f6ffd640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 0x7f37ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.035+0000 7f38059b8640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 0x7f37ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:12.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.035+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f37f00efc20 con 0x7f3800102790 2024-09-05T06:39:12.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.035+0000 7f38059b8640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 0x7f37ec0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00baa0 tx=0x7f37fc005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:12.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.039+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37f00bc840 con 0x7f3800102790 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-05T06:39:12.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:11 smithi115 ceph-mon[34791]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:12.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.238+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f380010a120 con 0x7f3800102790 2024-09-05T06:39:12.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.239+0000 7f37f6ffd640 1 -- 172.21.15.86:0/2819993156 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11839 (secure 0 0 0) 0x7f37f00bc140 con 0x7f3800102790 2024-09-05T06:39:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:12.239 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"btime":"2024-09-05T06:39:05:870955+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:05.445549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24479,"qdb_cluster":[24479]},"id":1}]} 2024-09-05T06:39:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-09-05T06:39:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 msgr2=0x7f37ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 0x7f37ec0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00baa0 tx=0x7f37fc005d90 comp rx=0 tx=0).stop 2024-09-05T06:39:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 msgr2=0x7f3800074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37f0005940 tx=0x7f37f0004270 comp rx=0 tx=0).stop 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 shutdown_connections 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f37ec0a2b80 0x7f37ec0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3800106f10 0x7f3800074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3800102790 0x7f3800074a30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 --2- 172.21.15.86:0/2819993156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3800101de0 0x7f38000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 >> 172.21.15.86:0/2819993156 conn(0x7f38000fd8c0 msgr2=0x7f38000fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 shutdown_connections 2024-09-05T06:39:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:12.241+0000 7f3807c1c640 1 -- 172.21.15.86:0/2819993156 wait complete. 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-05T06:39:12.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:11 smithi096 ceph-mon[34468]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:12.975 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T06:39:13.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2819993156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:13.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T06:39:13.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2819993156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 8]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T06:39:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2819993156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 -- 172.21.15.86:0/1119960528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef81027a0 msgr2=0x7f6ef81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 --2- 172.21.15.86:0/1119960528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef81027a0 0x7f6ef81067f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec02a410 comp rx=0 tx=0).stop 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 -- 172.21.15.86:0/1119960528 shutdown_connections 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 --2- 172.21.15.86:0/1119960528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8106f20 0x7f6ef810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 --2- 172.21.15.86:0/1119960528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef81027a0 0x7f6ef81067f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 --2- 172.21.15.86:0/1119960528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef8101df0 0x7f6ef81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 -- 172.21.15.86:0/1119960528 >> 172.21.15.86:0/1119960528 conn(0x7f6ef80fd8f0 msgr2=0x7f6ef80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.635+0000 7f6efe499640 1 -- 172.21.15.86:0/1119960528 shutdown_connections 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.636+0000 7f6efe499640 1 -- 172.21.15.86:0/1119960528 wait complete. 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.636+0000 7f6efe499640 1 Processor -- start 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.636+0000 7f6efe499640 1 -- start start 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8101df0 0x7f6ef80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef81027a0 0x7f6ef8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111e30 con 0x7f6ef8106f20 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111cb0 con 0x7f6ef81027a0 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efe499640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111fb0 con 0x7f6ef8101df0 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57218/0 (socket says 172.21.15.86:57218) 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 -- 172.21.15.86:0/582139234 learned_addr learned my addr 172.21.15.86:0/582139234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6ef7fff640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8101df0 0x7f6ef80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6ef77fe640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef81027a0 0x7f6ef8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 -- 172.21.15.86:0/582139234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8101df0 msgr2=0x7f6ef80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8101df0 0x7f6ef80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 -- 172.21.15.86:0/582139234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef81027a0 msgr2=0x7f6ef8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef81027a0 0x7f6ef8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 -- 172.21.15.86:0/582139234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef81a6800 con 0x7f6ef8106f20 2024-09-05T06:39:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6efca36640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000cce0 tx=0x7f6ef000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.637+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000f450 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.638+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef81a6af0 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.638+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ef000faa0 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.638+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0018b70 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.638+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a6fb0 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.639+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef81021d0 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.640+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6ef0018d10 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.640+0000 7f6ef57fa640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 0x7f6ee00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.640+0000 7f6ef7fff640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 0x7f6ee00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.641+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f6ef00cf6a0 con 0x7f6ef8106f20 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.641+0000 7f6ef7fff640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 0x7f6ee00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800bbe0 tx=0x7f6ee800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.644+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ef0014030 con 0x7f6ef8106f20 2024-09-05T06:39:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.814+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ef810a160 con 0x7f6ef8106f20 2024-09-05T06:39:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.816+0000 7f6ef57fa640 1 -- 172.21.15.86:0/582139234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7f6ef009c2c0 con 0x7f6ef8106f20 2024-09-05T06:39:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:13.816 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 msgr2=0x7f6ee00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 0x7f6ee00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800bbe0 tx=0x7f6ee800b390 comp rx=0 tx=0).stop 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 msgr2=0x7f6ef8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000cce0 tx=0x7f6ef000a620 comp rx=0 tx=0).stop 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 shutdown_connections 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ef8106f20 0x7f6ef8074f70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6ee00a2b80 0x7f6ee00a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ef81027a0 0x7f6ef8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 --2- 172.21.15.86:0/582139234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ef8101df0 0x7f6ef80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:13.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 >> 172.21.15.86:0/582139234 conn(0x7f6ef80fd8f0 msgr2=0x7f6ef80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:13.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 shutdown_connections 2024-09-05T06:39:13.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:13.818+0000 7f6efe499640 1 -- 172.21.15.86:0/582139234 wait complete. 2024-09-05T06:39:14.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:13 smithi086 ceph-mon[30943]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T06:39:14.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:13 smithi086 ceph-mon[30943]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:14.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:13 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/582139234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:14.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:13 smithi096 ceph-mon[34468]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T06:39:14.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:13 smithi096 ceph-mon[34468]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:14.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:13 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/582139234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:14.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:13 smithi115 ceph-mon[34791]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T06:39:14.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:13 smithi115 ceph-mon[34791]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:39:14.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:13 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/582139234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:14.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-05T06:38:55.927385+0000', 'modified': '2024-09-05T06:39:05.445549+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/2996609527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2996609527}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2996609527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24479, 'qdb_cluster': [24479]} 2024-09-05T06:39:14.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T06:39:14.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T06:39:14.863 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 -- 172.21.15.86:0/938009130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102570 msgr2=0x7f5bc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/938009130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102570 0x7f5bc81029b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009900 tx=0x7f5bb802a480 comp rx=0 tx=0).stop 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 -- 172.21.15.86:0/938009130 shutdown_connections 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/938009130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102ef0 0x7f5bc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/938009130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102570 0x7f5bc81029b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/938009130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 0x7f5bc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 -- 172.21.15.86:0/938009130 >> 172.21.15.86:0/938009130 conn(0x7f5bc80fc7d0 msgr2=0x7f5bc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 -- 172.21.15.86:0/938009130 shutdown_connections 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.369+0000 7f5bcf98e640 1 -- 172.21.15.86:0/938009130 wait complete. 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 Processor -- start 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 -- start start 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 0x7f5bc81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102570 0x7f5bc8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8110ed0 con 0x7f5bc8102ef0 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8110d50 con 0x7f5bc8100e20 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcf98e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111050 con 0x7f5bc8102570 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57226/0 (socket says 172.21.15.86:57226) 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 -- 172.21.15.86:0/2828361170 learned_addr learned my addr 172.21.15.86:0/2828361170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 0x7f5bc81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bccf29640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102570 0x7f5bc8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 -- 172.21.15.86:0/2828361170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102570 msgr2=0x7f5bc8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102570 0x7f5bc8197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 -- 172.21.15.86:0/2828361170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 msgr2=0x7f5bc81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 0x7f5bc81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.370+0000 7f5bcdf2b640 1 -- 172.21.15.86:0/2828361170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc819c380 con 0x7f5bc8102ef0 2024-09-05T06:39:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000db90 tx=0x7f5bc000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bbe7fc640 1 -- 172.21.15.86:0/2828361170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc000e880 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc819c610 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bbe7fc640 1 -- 172.21.15.86:0/2828361170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc0009580 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bbe7fc640 1 -- 172.21.15.86:0/2828361170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0016650 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.371+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc8069f70 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.373+0000 7f5bbe7fc640 1 -- 172.21.15.86:0/2828361170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5bc0009730 con 0x7f5bc8102ef0 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.373+0000 7f5bd1d67740 1 Processor -- start 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 -- start start 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bbe7fc640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f022f0 0x5629b5f0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f0cbb0 0x5629b5f0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629b5e68090 con 0x5629b5f019d0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629b5e0fa70 con 0x5629b5f0cbb0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629b5e64050 con 0x5629b5f022f0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bccf29640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57238/0 (socket says 172.21.15.86:57238) 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 -- 172.21.15.86:0/743918296 learned_addr learned my addr 172.21.15.86:0/743918296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bbe7fc640 1 -- 172.21.15.86:0/2828361170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f5bc00d0ab0 con 0x7f5bc8102ef0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bccf29640 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f022f0 0x5629b5f0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 -- 172.21.15.86:0/743918296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f022f0 msgr2=0x5629b5f0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f022f0 0x5629b5f0c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 -- 172.21.15.86:0/743918296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f0cbb0 msgr2=0x5629b5f0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f0cbb0 0x5629b5f0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 -- 172.21.15.86:0/743918296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b5e1c130 con 0x5629b5f019d0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bccf29640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 0x7f5bb40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006010 tx=0x7f5bb8031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.374+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009a70 tx=0x7f5bc4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937109ce76b3e3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bb37fe640 1 -- 172.21.15.86:0/743918296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4017070 con 0x5629b5f019d0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bb37fe640 1 -- 172.21.15.86:0/743918296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc4013470 con 0x5629b5f019d0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bb37fe640 1 -- 172.21.15.86:0/743918296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4012630 con 0x5629b5f019d0 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 -- 172.21.15.86:0/743918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 msgr2=0x5629b5f01db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009a70 tx=0x7f5bc4009f30 comp rx=0 tx=0).stop 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 -- 172.21.15.86:0/743918296 shutdown_connections 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f0cbb0 0x5629b5f0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f022f0 0x5629b5f0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/743918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f01db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 -- 172.21.15.86:0/743918296 >> 172.21.15.86:0/743918296 conn(0x5629b5efdf60 msgr2=0x5629b5efe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.375+0000 7f5bd1d67740 1 -- 172.21.15.86:0/743918296 shutdown_connections 2024-09-05T06:39:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 -- 172.21.15.86:0/743918296 wait complete. 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 -- start start 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 Processor -- start 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f0c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f0f780 con 0x5629b5f022f0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629b5e22f20 con 0x5629b5f019d0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f0f900 con 0x5629b5f0cbb0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bd1d67740 10 client.0 _ll_register_callbacks cb 0x5629b5dec030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.376+0000 7f5bccf29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57252/0 (socket says 172.21.15.86:57252) 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 -- 172.21.15.86:0/3058449561 learned_addr learned my addr 172.21.15.86:0/3058449561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f0c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 -- 172.21.15.86:0/3058449561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 msgr2=0x5629b5f09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 -- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 msgr2=0x5629b5f0c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f0c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 -- 172.21.15.86:0/3058449561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b5bd92b0 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bccf29640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8033000 tx=0x7f5bb802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6517828b409a8d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035050 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc4013470 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f5bb80046c0 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5bb8057050 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bd1d67740 10 client.14700 fetch_fsmap learned FSMap version 16 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.377+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5629b5f1b1f0 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc4013470 con 0x5629b5f022f0 2024-09-05T06:39:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f5bb8003a70 con 0x5629b5f022f0 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 10 client.14700 fetch_fsmap finished waiting for FSMap version 16 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 10 client.14700 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 20 client.14700 populate_metadata read hostname 'smithi086' 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f1d820 0x5629b5f1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 4 client.14700 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b5e75600 con 0x5629b5f1d820 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f1d820 0x5629b5f1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bb0ff9640 10 client.14700 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.378+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f1d820 0x5629b5f1ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f0c51304735d33 server_cookie=c8e180c0761517b0 in_seq=0 out_seq=0 2024-09-05T06:39:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.379+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/3058449561 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629b5e75600 con 0x5629b5f1d820 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.379+0000 7f5bb0ff9640 10 client.14700 handle_command_reply: tid=0 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.379+0000 7f5bd1d67740 1 client.14700 shutdown 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 msgr2=0x5629b5f098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8033000 tx=0x7f5bb802a480 comp rx=0 tx=0).stop 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 shutdown_connections 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f1d820 0x5629b5f1ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f098c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/3058449561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f0c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bb0ff9640 0 client.14700 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 >> 172.21.15.86:0/3058449561 conn(0x5629b5efdf60 msgr2=0x5629b5f03c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 shutdown_connections 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 1 -- 172.21.15.86:0/3058449561 wait complete. 2024-09-05T06:39:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.380+0000 7f5bd1d67740 20 client.14700 trim_cache size 0 max 16384 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.416+0000 7f5bd1d67740 1 Processor -- start 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.416+0000 7f5bd1d67740 1 -- start start 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f07060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1cfd0 con 0x5629b5f022f0 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1d6a0 con 0x5629b5f019d0 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1dd70 con 0x5629b5f0cbb0 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57264/0 (socket says 172.21.15.86:57264) 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 -- 172.21.15.86:0/812153456 learned_addr learned my addr 172.21.15.86:0/812153456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f07060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 -- 172.21.15.86:0/812153456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 msgr2=0x5629b5f07060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f07060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 -- 172.21.15.86:0/812153456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 msgr2=0x5629b5f09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f09180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 -- 172.21.15.86:0/812153456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b5f01800 con 0x5629b5f022f0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f07060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.417+0000 7f5bccf29640 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8003e90 tx=0x7f5bb8009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defe0bf87593148e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bb2ffd640 1 -- 172.21.15.86:0/812153456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8060070 con 0x5629b5f022f0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bb2ffd640 1 -- 172.21.15.86:0/812153456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb8041030 con 0x5629b5f022f0 2024-09-05T06:39:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bb2ffd640 1 -- 172.21.15.86:0/812153456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035070 con 0x5629b5f022f0 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 -- 172.21.15.86:0/812153456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 msgr2=0x5629b5f06b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8003e90 tx=0x7f5bb8009540 comp rx=0 tx=0).stop 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 -- 172.21.15.86:0/812153456 shutdown_connections 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f07060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f022f0 0x5629b5f06b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/812153456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f019d0 0x5629b5f09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.418+0000 7f5bd1d67740 1 -- 172.21.15.86:0/812153456 >> 172.21.15.86:0/812153456 conn(0x5629b5efdf60 msgr2=0x5629b5f09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- 172.21.15.86:0/812153456 shutdown_connections 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- 172.21.15.86:0/812153456 wait complete. 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- start start 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 Processor -- start 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f022f0 0x5629b5f20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1eb10 con 0x5629b5f019d0 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1e440 con 0x5629b5f022f0 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629b5f1f1e0 con 0x5629b5f0cbb0 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.419+0000 7f5bd1d67740 10 client.0 _ll_register_callbacks cb 0x7f5bb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:39:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57278/0 (socket says 172.21.15.86:57278) 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 -- 172.21.15.86:0/1675452478 learned_addr learned my addr 172.21.15.86:0/1675452478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bccf29640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f022f0 0x5629b5f20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcdf2b640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 -- 172.21.15.86:0/1675452478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 msgr2=0x5629b5f85f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f85f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 -- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f022f0 msgr2=0x5629b5f20770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f022f0 0x5629b5f20770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 -- 172.21.15.86:0/1675452478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b5eff0f0 con 0x5629b5f019d0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bcd72a640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40099c0 tx=0x7f5bc40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b89d1a29c1e7706 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402b060 con 0x5629b5f019d0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f5bc4013670 con 0x5629b5f019d0 2024-09-05T06:39:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb8041030 con 0x5629b5f019d0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5bc4032050 con 0x5629b5f019d0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bd1d67740 10 client.14706 fetch_fsmap learned FSMap version 16 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.420+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5629b5f87420 con 0x5629b5f019d0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb8041030 con 0x5629b5f019d0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f5bc401c450 con 0x5629b5f019d0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 10 client.14706 fetch_fsmap finished waiting for FSMap version 16 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 10 client.14706 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 20 client.14706 populate_metadata read hostname 'smithi086' 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f8bcd0 0x5629b5f8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 4 client.14706 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "status"}) -- 0x5629b5e6fb10 con 0x5629b5f8bcd0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.421+0000 7f5bccf29640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f8bcd0 0x5629b5f8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bb0ff9640 10 client.14706 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bccf29640 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f8bcd0 0x5629b5f8a9e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe76089bf49ff46 server_cookie=3ea2bba6e11176d2 in_seq=0 out_seq=0 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bb0ff9640 1 -- 172.21.15.86:0/1675452478 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5629b5e6fb10 con 0x5629b5f8bcd0 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bb0ff9640 10 client.14706 handle_command_reply: tid=0 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 client.14706 shutdown 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 msgr2=0x5629b5f20230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40099c0 tx=0x7f5bc40092c0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 shutdown_connections 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629b5f8bcd0 0x5629b5f8a9e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629b5f0cbb0 0x5629b5f85f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629b5f022f0 0x5629b5f20770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 --2- 172.21.15.86:0/1675452478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629b5f019d0 0x5629b5f20230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 >> 172.21.15.86:0/1675452478 conn(0x5629b5efdf60 msgr2=0x5629b5efc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bb0ff9640 0 client.14706 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 shutdown_connections 2024-09-05T06:39:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.422+0000 7f5bd1d67740 1 -- 172.21.15.86:0/1675452478 wait complete. 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bd1d67740 20 client.14706 trim_cache size 0 max 16384 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24479, 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 19.0560999, 2024-09-05T06:39:15.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 16, 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 87, 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 85, 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 30.046000669000001 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 msgr2=0x7f5bb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 0x7f5bb40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006010 tx=0x7f5bb8031430 comp rx=0 tx=0).stop 2024-09-05T06:39:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 msgr2=0x7f5bc819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000db90 tx=0x7f5bc000df90 comp rx=0 tx=0).stop 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 shutdown_connections 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bc8102ef0 0x7f5bc819bca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bc8102570 0x7f5bc8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5bb40a2b80 0x7f5bb40a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 --2- 172.21.15.86:0/2828361170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100e20 0x7f5bc81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 >> 172.21.15.86:0/2828361170 conn(0x7f5bc80fc7d0 msgr2=0x7f5bc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 shutdown_connections 2024-09-05T06:39:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:15.423+0000 7f5bcf98e640 1 -- 172.21.15.86:0/2828361170 wait complete. 2024-09-05T06:39:16.130 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T06:39:16.131 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:16.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T06:39:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:16.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:16 smithi086 ceph-mon[30943]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-05T06:39:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:16 smithi096 ceph-mon[34468]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-05T06:39:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:16 smithi115 ceph-mon[34791]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 201 B/s, 0 objects/s recovering 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1100744157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0101420 msgr2=0x7f48f010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1100744157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0101420 0x7f48f010d950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009cd0 tx=0x7f48e802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1100744157 shutdown_connections 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1100744157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0101420 0x7f48f010d950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1100744157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0100a80 0x7f48f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1100744157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0106a80 0x7f48f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1100744157 >> 172.21.15.86:0/1100744157 conn(0x7f48f00fc7a0 msgr2=0x7f48f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1100744157 shutdown_connections 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1100744157 wait complete. 2024-09-05T06:39:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.852+0000 7f48f5bc6640 1 Processor -- start 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- start start 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 0x7f48f019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 0x7f48f019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0110d70 con 0x7f48f0100a80 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0110bf0 con 0x7f48f0106a80 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0110ef0 con 0x7f48f0101420 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57300/0 (socket says 172.21.15.86:57300) 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 -- 172.21.15.86:0/1869713743 learned_addr learned my addr 172.21.15.86:0/1869713743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48e7fff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 0x7f48f019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48effff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 0x7f48f019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 -- 172.21.15.86:0/1869713743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 msgr2=0x7f48f019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 0x7f48f019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 -- 172.21.15.86:0/1869713743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 msgr2=0x7f48f019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 0x7f48f019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 -- 172.21.15.86:0/1869713743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f019f550 con 0x7f48f0100a80 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48e7fff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 0x7f48f019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48effff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 0x7f48f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ef7fe640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c970 tx=0x7f48e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0007bb0 con 0x7f48f0100a80 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f019f840 con 0x7f48f0100a80 2024-09-05T06:39:16.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48e0007d50 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e000f450 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.853+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a7120 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.854+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f0100ee0 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.856+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48e0004ee0 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.856+0000 7f48ed7fa640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.856+0000 7f48e7fff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.856+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f48e00cf5b0 con 0x7f48f0100a80 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.857+0000 7f48e7fff640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:16.859+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e0014030 con 0x7f48f0100a80 2024-09-05T06:39:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.057+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48f01a73b0 con 0x7f48f0100a80 2024-09-05T06:39:17.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.057+0000 7f48ed7fa640 1 -- 172.21.15.86:0/1869713743 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f48e009c1d0 con 0x7f48f0100a80 2024-09-05T06:39:17.057 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T06:39:17.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 msgr2=0x7f48dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:17.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 0x7f48dc0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8008040 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 msgr2=0x7f48f019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c970 tx=0x7f48e000ce30 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 shutdown_connections 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f48dc0a2b80 0x7f48dc0a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f0106a80 0x7f48f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f0101420 0x7f48f019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 --2- 172.21.15.86:0/1869713743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f0100a80 0x7f48f019a5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.059+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 >> 172.21.15.86:0/1869713743 conn(0x7f48f00fc7a0 msgr2=0x7f48f010c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.060+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 shutdown_connections 2024-09-05T06:39:17.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17.060+0000 7f48f5bc6640 1 -- 172.21.15.86:0/1869713743 wait complete. 2024-09-05T06:39:17.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1869713743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:39:17.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1869713743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:39:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1869713743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:39:17.872 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-05T06:39:17.872 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T06:39:17.872 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T06:39:17.873 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T06:39:17.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:17.873 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-09-05T06:39:17.890 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T06:39:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T06:39:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T06:39:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T06:39:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:39:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T06:39:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42244sec preferred_lft 42244sec 2024-09-05T06:39:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link noprefixroute 2024-09-05T06:39:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:39:17.893 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T06:39:17.893 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:39:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:39:18.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:18.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:18.088 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T06:39:18.113 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T06:39:18.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:18.114 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-09-05T06:39:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-09-05T06:39:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T06:39:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T06:39:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T06:39:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T06:39:18.189 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:39:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:39:18.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:18 smithi086 ceph-mon[30943]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:18.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:18.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:18.465 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-09-05T06:39:18.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:18.481 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-09-05T06:39:18.536 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:39:18.536 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:39:18.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:39:18.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-05T06:39:18.536 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:39:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:18 smithi096 ceph-mon[34468]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:39:18.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:18.654 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-05T06:39:18.655 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:39:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:18 smithi115 ceph-mon[34791]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:39:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:18.792 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:39:18.792 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:39:18.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-09-05T06:39:18.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T06:39:18.792 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:39:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:39:18.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:18.869 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-05T06:39:18.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T06:39:18.869 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:18.893 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T06:39:18.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T06:39:18.893 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:18.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:39:19.013 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:39:19.069 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:39:19.069 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:39:19.069 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:39:19.070 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:39:19.071 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:39:19.072 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:39:19.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:19.073 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:39:19.125 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T06:39:19.125 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-05T06:39:19.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:39:19.192 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:39:19.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:39:19.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:39:19.252 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:39:19.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:19.253 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:39:19.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-09-05T06:39:19.278+0000 7f0fe2f2b480 -1 init, newargv = 0x5638fd484b10 newargc=15 2024-09-05T06:39:19.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[72905]: starting ceph client 2024-09-05T06:39:19.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[72905]: starting fuse 2024-09-05T06:39:19.293 INFO:teuthology.orchestra.run.smithi086.stdout:121 2024-09-05T06:39:19.294 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-05T06:39:19.294 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:39:19.295 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-09-05T06:39:19.296 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-09-05T06:39:19.413 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.72905.asok 2024-09-05T06:39:19.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:19.424 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72905.asok 2024-09-05T06:39:19.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:19.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72905.asok status 2024-09-05T06:39:19.558 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:39:19.558 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-09-05T06:39:19.558 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T06:39:19.558 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T06:39:19.558 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "72905", 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 14718, 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-09-05T06:39:19.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 14718 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 119670558 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 119670558 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:39:19.560 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.14718 192.168.0.1:0/119670558", 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/119670558", 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 16, 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 87, 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-09-05T06:39:19.561 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:39:19.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:19.566 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-09-05T06:39:19.566 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:19.582 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-09-05T06:39:19.582 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:19.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:19.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:19.648 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-05T06:39:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T06:39:19.764 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-09-05T06:39:19.764 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-05T06:39:19.764 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:19.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:39:19.770 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:39:19.770 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T06:39:19.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T06:39:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:20.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:20 smithi086 ceph-mon[30943]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:20.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:20 smithi096 ceph-mon[34468]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2734566131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4100a80 msgr2=0x7fb9f4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2734566131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4100a80 0x7fb9f4100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009930 tx=0x7fb9dc02a420 comp rx=0 tx=0).stop 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2734566131 shutdown_connections 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2734566131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4101420 0x7fb9f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2734566131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4100a80 0x7fb9f4100ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2734566131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9f4106a80 0x7fb9f4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2734566131 >> 172.21.15.86:0/2734566131 conn(0x7fb9f40fc7a0 msgr2=0x7fb9f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2734566131 shutdown_connections 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2734566131 wait complete. 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.533+0000 7fb9fa6fe640 1 Processor -- start 2024-09-05T06:39:20.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 -- start start 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 0x7fb9f4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9f4106a80 0x7fb9f419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110d50 con 0x7fb9f4101420 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110bd0 con 0x7fb9f4100a80 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9fa6fe640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110ed0 con 0x7fb9f4106a80 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57316/0 (socket says 172.21.15.86:57316) 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 -- 172.21.15.86:0/2981126688 learned_addr learned my addr 172.21.15.86:0/2981126688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9f3fff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 0x7fb9f4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 -- 172.21.15.86:0/2981126688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9f4106a80 msgr2=0x7fb9f419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9f4106a80 0x7fb9f419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 -- 172.21.15.86:0/2981126688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 msgr2=0x7fb9f4197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 0x7fb9f4197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 -- 172.21.15.86:0/2981126688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f419c2e0 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9f3fff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 0x7fb9f4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9ebfff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc004770 tx=0x7fb9dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc03f070 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.534+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9dc03b460 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.535+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f419c570 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.535+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc035070 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.535+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f41a3eb0 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.536+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f41021c0 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.537+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9dc03b610 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.537+0000 7fb9f1ffb640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 0x7fb9e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.538+0000 7fb9f3fff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 0x7fb9e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.538+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb9dc0efa90 con 0x7fb9f4101420 2024-09-05T06:39:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.538+0000 7fb9f3fff640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 0x7fb9e00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006fd0 tx=0x7fb9e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.541+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9dc0bc6b0 con 0x7fb9f4101420 2024-09-05T06:39:20.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:20 smithi115 ceph-mon[34791]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s, 0 objects/s recovering 2024-09-05T06:39:20.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.718+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9f4100ee0 con 0x7fb9f4101420 2024-09-05T06:39:20.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.718+0000 7fb9f1ffb640 1 -- 172.21.15.86:0/2981126688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+589 (secure 0 0 0) 0x7fb9dc0bbfb0 con 0x7fb9f4101420 2024-09-05T06:39:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2555553887 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2886866246 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1227677145 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4008919610 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/740829125 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3421533074 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 msgr2=0x7fb9e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 0x7fb9e00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006fd0 tx=0x7fb9e4008040 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 msgr2=0x7fb9f4197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc004770 tx=0x7fb9dc0042f0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 shutdown_connections 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9f4106a80 0x7fb9f419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9f4101420 0x7fb9f4197870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9e00a2b80 0x7fb9e00a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 --2- 172.21.15.86:0/2981126688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f4100a80 0x7fb9f4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 >> 172.21.15.86:0/2981126688 conn(0x7fb9f40fc7a0 msgr2=0x7fb9f40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 shutdown_connections 2024-09-05T06:39:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:20.721+0000 7fb9fa6fe640 1 -- 172.21.15.86:0/2981126688 wait complete. 2024-09-05T06:39:21.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:21 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2981126688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:21.519 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:39:21.519 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T06:39:21.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T06:39:21.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:21 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2981126688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:21.583 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:21 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2981126688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:22.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/415799727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074eb0 msgr2=0x7f7b68112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/415799727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074eb0 0x7f7b68112660 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009900 tx=0x7f7b5002a480 comp rx=0 tx=0).stop 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/415799727 shutdown_connections 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/415799727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 0x7f7b68114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/415799727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074eb0 0x7f7b68112660 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/415799727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b68074590 0x7f7b68074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/415799727 >> 172.21.15.86:0/415799727 conn(0x7f7b680fd8b0 msgr2=0x7f7b680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/415799727 shutdown_connections 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.308+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/415799727 wait complete. 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 Processor -- start 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 -- start start 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b68074eb0 0x7f7b6819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 0x7f7b681a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b68118420 con 0x7f7b68074590 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681182a0 con 0x7f7b68112ba0 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b6d6d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681185a0 con 0x7f7b68074eb0 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57336/0 (socket says 172.21.15.86:57336) 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 -- 172.21.15.86:0/4150734861 learned_addr learned my addr 172.21.15.86:0/4150734861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b677fe640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 0x7f7b681a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 -- 172.21.15.86:0/4150734861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b68074eb0 msgr2=0x7f7b6819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b68074eb0 0x7f7b6819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 -- 172.21.15.86:0/4150734861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 msgr2=0x7f7b681a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 0x7f7b681a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.309+0000 7f7b66ffd640 1 -- 172.21.15.86:0/4150734861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b681a3a70 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b66ffd640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d770 tx=0x7f7b5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b580054d0 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b681a3d60 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58004cf0 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.310+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b681ab5f0 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.311+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b68074970 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.313+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7b58004e90 con 0x7f7b68074590 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.313+0000 7f7b64ff9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 0x7f7b540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.313+0000 7f7b5ffff640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 0x7f7b540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.313+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f7b580cf260 con 0x7f7b68074590 2024-09-05T06:39:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.314+0000 7f7b5ffff640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 0x7f7b540a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b500096c0 tx=0x7f7b50031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.317+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b5809bf00 con 0x7f7b68074590 2024-09-05T06:39:22.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:22 smithi086 ceph-mon[30943]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 221 B/s wr, 0 op/s; 176 B/s, 0 objects/s recovering 2024-09-05T06:39:22.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.496+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b68104fe0 con 0x7f7b68074590 2024-09-05T06:39:22.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.496+0000 7f7b64ff9640 1 -- 172.21.15.86:0/4150734861 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+589 (secure 0 0 0) 0x7f7b5809b800 con 0x7f7b68074590 2024-09-05T06:39:22.496 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2555553887 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2886866246 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1227677145 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4008919610 2024-09-06T06:32:11.079228+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/740829125 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3421533074 2024-09-06T06:31:48.412507+0000 2024-09-05T06:39:22.497 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-09-05T06:39:22.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.498+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 msgr2=0x7f7b540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.498+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 0x7f7b540a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b500096c0 tx=0x7f7b50031400 comp rx=0 tx=0).stop 2024-09-05T06:39:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.498+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 msgr2=0x7f7b6819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.498+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d770 tx=0x7f7b5800dc30 comp rx=0 tx=0).stop 2024-09-05T06:39:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 shutdown_connections 2024-09-05T06:39:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f7b540a2c00 0x7f7b540a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b68112ba0 0x7f7b681a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b68074eb0 0x7f7b6819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 --2- 172.21.15.86:0/4150734861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68074590 0x7f7b6819ea40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 >> 172.21.15.86:0/4150734861 conn(0x7f7b680fd8b0 msgr2=0x7f7b680ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 shutdown_connections 2024-09-05T06:39:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:22.499+0000 7f7b6d6d9640 1 -- 172.21.15.86:0/4150734861 wait complete. 2024-09-05T06:39:22.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:22 smithi096 ceph-mon[34468]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 221 B/s wr, 0 op/s; 176 B/s, 0 objects/s recovering 2024-09-05T06:39:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:22 smithi115 ceph-mon[34791]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 221 B/s wr, 0 op/s; 176 B/s, 0 objects/s recovering 2024-09-05T06:39:23.172 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-09-05T06:39:23.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:23.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-05T06:39:23.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[72905]: fuse finished with error 0 and tester_r 0 2024-09-05T06:39:23.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T06:39:23.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:39:23.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:39:23.271 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-05T06:39:23.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:23.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:23 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4150734861' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:23.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:23 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4150734861' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:23.587 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:23 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4150734861' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T06:39:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- 172.21.15.86:0/1025602469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 msgr2=0x7fe9601067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 --2- 172.21.15.86:0/1025602469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe9601067d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe948009900 tx=0x7fe94802a410 comp rx=0 tx=0).stop 2024-09-05T06:39:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- 172.21.15.86:0/1025602469 shutdown_connections 2024-09-05T06:39:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 --2- 172.21.15.86:0/1025602469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960106f00 0x7fe96010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 --2- 172.21.15.86:0/1025602469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe9601067d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 --2- 172.21.15.86:0/1025602469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe960101dd0 0x7fe9601021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- 172.21.15.86:0/1025602469 >> 172.21.15.86:0/1025602469 conn(0x7fe9600fd8b0 msgr2=0x7fe9600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- 172.21.15.86:0/1025602469 shutdown_connections 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- 172.21.15.86:0/1025602469 wait complete. 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 Processor -- start 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.095+0000 7fe965fad640 1 -- start start 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960101dd0 0x7fe9601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe960106f00 0x7fe96019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe960111e30 con 0x7fe960102780 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe960111cb0 con 0x7fe960101dd0 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe965fad640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe960111fb0 con 0x7fe960106f00 2024-09-05T06:39:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33666/0 (socket says 172.21.15.86:33666) 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 -- 172.21.15.86:0/2363781139 learned_addr learned my addr 172.21.15.86:0/2363781139 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe95f7fe640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960101dd0 0x7fe9601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 -- 172.21.15.86:0/2363781139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe960106f00 msgr2=0x7fe96019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe960106f00 0x7fe96019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 -- 172.21.15.86:0/2363781139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960101dd0 msgr2=0x7fe9601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960101dd0 0x7fe9601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 -- 172.21.15.86:0/2363781139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96019d380 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.096+0000 7fe956ffd640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe948005940 tx=0x7fe948004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.097+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94803f030 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.097+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96019d610 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.097+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe948003aa0 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.097+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948035070 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.097+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a4f50 con 0x7fe960102780 2024-09-05T06:39:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.098+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9601032b0 con 0x7fe960102780 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.099+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe948042030 con 0x7fe960102780 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.100+0000 7fe95d7fa640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.100+0000 7fe95f7fe640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.100+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fe9480efd80 con 0x7fe960102780 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.101+0000 7fe95f7fe640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 0x7fe94c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe950009990 tx=0x7fe950009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.103+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9480bc9a0 con 0x7fe960102780 2024-09-05T06:39:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.272+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe96010a110 con 0x7fe960102780 2024-09-05T06:39:24.273 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:24 smithi086 ceph-mon[30943]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 0 op/s; 159 B/s, 0 objects/s recovering 2024-09-05T06:39:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.274+0000 7fe95d7fa640 1 -- 172.21.15.86:0/2363781139 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7fe9480bc2a0 con 0x7fe960102780 2024-09-05T06:39:24.274 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:24.274 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 msgr2=0x7fe94c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 0x7fe94c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe950009990 tx=0x7fe950009110 comp rx=0 tx=0).stop 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 msgr2=0x7fe960198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe948005940 tx=0x7fe948004270 comp rx=0 tx=0).stop 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 shutdown_connections 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe960106f00 0x7fe96019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960102780 0x7fe960198910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 --2- 172.21.15.86:0/2363781139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe960101dd0 0x7fe9601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 >> 172.21.15.86:0/2363781139 conn(0x7fe9600fd8b0 msgr2=0x7fe9600fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 shutdown_connections 2024-09-05T06:39:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:24.276+0000 7fe965fad640 1 -- 172.21.15.86:0/2363781139 wait complete. 2024-09-05T06:39:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:24 smithi096 ceph-mon[34468]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 0 op/s; 159 B/s, 0 objects/s recovering 2024-09-05T06:39:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:24 smithi115 ceph-mon[34791]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 0 op/s; 159 B/s, 0 objects/s recovering 2024-09-05T06:39:24.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:25.502 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:25 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2363781139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:25.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:25 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2363781139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:25 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2363781139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 -- 172.21.15.86:0/1497216591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 msgr2=0x7ff834103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/1497216591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff834103120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff824009900 tx=0x7ff82402a480 comp rx=0 tx=0).stop 2024-09-05T06:39:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 -- 172.21.15.86:0/1497216591 shutdown_connections 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/1497216591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff83410c170 0x7ff83410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/1497216591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff834103660 0x7ff83410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/1497216591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff834103120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 -- 172.21.15.86:0/1497216591 >> 172.21.15.86:0/1497216591 conn(0x7ff8340fc7a0 msgr2=0x7ff8340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 -- 172.21.15.86:0/1497216591 shutdown_connections 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.699+0000 7ff83bbc1640 1 -- 172.21.15.86:0/1497216591 wait complete. 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 Processor -- start 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 -- start start 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 0x7ff834100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 0x7ff834101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff834111ee0 con 0x7ff834102d40 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff834111d60 con 0x7ff83410c170 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83bbc1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff834112060 con 0x7ff834103660 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33692/0 (socket says 172.21.15.86:33692) 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 -- 172.21.15.86:0/2372265822 learned_addr learned my addr 172.21.15.86:0/2372265822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83a15e640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 0x7ff834101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83915c640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 0x7ff834100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 -- 172.21.15.86:0/2372265822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 msgr2=0x7ff834100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 0x7ff834100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 -- 172.21.15.86:0/2372265822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 msgr2=0x7ff834101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 0x7ff834101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83995d640 1 -- 172.21.15.86:0/2372265822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8341a6950 con 0x7ff834102d40 2024-09-05T06:39:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83a15e640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 0x7ff834101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.700+0000 7ff83915c640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 0x7ff834100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff83995d640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff824006010 tx=0x7ff824004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82403f030 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8240035f0 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8341a6be0 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824035070 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.701+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8341a70c0 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.702+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8341031a0 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.703+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff824003b10 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.703+0000 7ff82affd640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 0x7ff8200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.704+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7ff8240effb0 con 0x7ff834102d40 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.704+0000 7ff83915c640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 0x7ff8200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.704+0000 7ff83915c640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 0x7ff8200a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff830006fd0 tx=0x7ff830008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:25.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.708+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8240bcbd0 con 0x7ff834102d40 2024-09-05T06:39:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.879+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff834106c80 con 0x7ff834102d40 2024-09-05T06:39:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.881+0000 7ff82affd640 1 -- 172.21.15.86:0/2372265822 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7ff8240bc4d0 con 0x7ff834102d40 2024-09-05T06:39:25.881 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:25.882 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.884+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 msgr2=0x7ff8200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.884+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 0x7ff8200a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff830006fd0 tx=0x7ff830008040 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.884+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 msgr2=0x7ff8341009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.884+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff824006010 tx=0x7ff824004270 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.884+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 shutdown_connections 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff8200a2b80 0x7ff8200a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff83410c170 0x7ff834101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff834103660 0x7ff834100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 --2- 172.21.15.86:0/2372265822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff834102d40 0x7ff8341009a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 >> 172.21.15.86:0/2372265822 conn(0x7ff8340fc7a0 msgr2=0x7ff8340fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 shutdown_connections 2024-09-05T06:39:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:25.885+0000 7ff83bbc1640 1 -- 172.21.15.86:0/2372265822 wait complete. 2024-09-05T06:39:26.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:26 smithi096 ceph-mon[34468]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 136 B/s, 0 objects/s recovering 2024-09-05T06:39:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:26 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2372265822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:26.599 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:26 smithi086 ceph-mon[30943]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 136 B/s, 0 objects/s recovering 2024-09-05T06:39:26.599 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:26 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2372265822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:26.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-05T06:39:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:26 smithi115 ceph-mon[34791]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 136 B/s, 0 objects/s recovering 2024-09-05T06:39:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:26 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2372265822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:27.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 -- 172.21.15.86:0/1392438768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 msgr2=0x7f1814100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 --2- 172.21.15.86:0/1392438768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f1814100ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1800009930 tx=0x7f180002a420 comp rx=0 tx=0).stop 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 -- 172.21.15.86:0/1392438768 shutdown_connections 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 --2- 172.21.15.86:0/1392438768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814101420 0x7f181410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 --2- 172.21.15.86:0/1392438768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f1814100ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 --2- 172.21.15.86:0/1392438768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814106a80 0x7f1814106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 -- 172.21.15.86:0/1392438768 >> 172.21.15.86:0/1392438768 conn(0x7f18140fc7a0 msgr2=0x7f18140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 -- 172.21.15.86:0/1392438768 shutdown_connections 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.361+0000 7f181af86640 1 -- 172.21.15.86:0/1392438768 wait complete. 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 Processor -- start 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 -- start start 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814101420 0x7f18140fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 0x7f18140fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110d30 con 0x7f1814100a80 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110bb0 con 0x7f1814106a80 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f181af86640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110eb0 con 0x7f1814101420 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33706/0 (socket says 172.21.15.86:33706) 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 -- 172.21.15.86:0/3050037694 learned_addr learned my addr 172.21.15.86:0/3050037694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1819523640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 0x7f18140fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1813fff640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814101420 0x7f18140fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 -- 172.21.15.86:0/3050037694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814101420 msgr2=0x7f18140fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814101420 0x7f18140fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 -- 172.21.15.86:0/3050037694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 msgr2=0x7f18140fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 0x7f18140fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1818d22640 1 -- 172.21.15.86:0/3050037694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18141a6840 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.362+0000 7f1819523640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 0x7f18140fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f1818d22640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f180800d950 tx=0x7f180800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808017070 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1808009c30 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141a6b30 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180800f620 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.363+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a7070 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.364+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1814100ee0 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.365+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f18080109a0 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.366+0000 7f1811ffb640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 0x7f18040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.366+0000 7f1813fff640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 0x7f18040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.366+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f18080cf000 con 0x7f1814100a80 2024-09-05T06:39:27.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.366+0000 7f1813fff640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 0x7f18040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:27.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.370+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1808009de0 con 0x7f1814100a80 2024-09-05T06:39:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.574+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f181402d7c0 con 0x7f1814100a80 2024-09-05T06:39:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.843+0000 7f1811ffb640 1 -- 172.21.15.86:0/3050037694 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f180809bc20 con 0x7f1814100a80 2024-09-05T06:39:27.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 msgr2=0x7f18040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:27.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 0x7f18040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 msgr2=0x7f18140fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:27.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f180800d950 tx=0x7f180800de10 comp rx=0 tx=0).stop 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 shutdown_connections 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f18040a2b80 0x7f18040a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1814106a80 0x7f18140fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1814101420 0x7f18140fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 --2- 172.21.15.86:0/3050037694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1814100a80 0x7f18140fce80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.844+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 >> 172.21.15.86:0/3050037694 conn(0x7f18140fc7a0 msgr2=0x7f181406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:27.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.845+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 shutdown_connections 2024-09-05T06:39:27.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:27.845+0000 7f181af86640 1 -- 172.21.15.86:0/3050037694 wait complete. 2024-09-05T06:39:28.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:27 smithi086 ceph-mon[30943]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T06:39:28.119 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:27 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T06:39:28.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:27 smithi115 ceph-mon[34791]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T06:39:28.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:27 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T06:39:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:27 smithi096 ceph-mon[34468]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T06:39:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:27 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T06:39:28.825 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-05T06:39:28.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-05T06:39:29.075 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:29.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:28 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T06:39:29.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:28 smithi115 ceph-mon[34791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:29.195 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:28 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T06:39:29.195 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:28 smithi086 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:29.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:28 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3050037694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T06:39:29.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:28 smithi096 ceph-mon[34468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 -- 172.21.15.86:0/2727519807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c10c170 msgr2=0x7fe81c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2727519807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c10c170 0x7fe81c10e610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe814009ca0 tx=0x7fe81402a950 comp rx=0 tx=0).stop 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 -- 172.21.15.86:0/2727519807 shutdown_connections 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2727519807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c10c170 0x7fe81c10e610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2727519807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe81c103660 0x7fe81c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2727519807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 -- 172.21.15.86:0/2727519807 >> 172.21.15.86:0/2727519807 conn(0x7fe81c0fc7a0 msgr2=0x7fe81c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:29.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 -- 172.21.15.86:0/2727519807 shutdown_connections 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.563+0000 7fe8236b4640 1 -- 172.21.15.86:0/2727519807 wait complete. 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 Processor -- start 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 -- start start 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe81c10c170 0x7fe81c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c111e00 con 0x7fe81c103660 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c111c80 con 0x7fe81c102d40 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe8236b4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c111f80 con 0x7fe81c10c170 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33724/0 (socket says 172.21.15.86:33724) 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 -- 172.21.15.86:0/2684164087 learned_addr learned my addr 172.21.15.86:0/2684164087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe821450640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 -- 172.21.15.86:0/2684164087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe81c10c170 msgr2=0x7fe81c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe81c10c170 0x7fe81c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 -- 172.21.15.86:0/2684164087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 msgr2=0x7fe81c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe820c4f640 1 -- 172.21.15.86:0/2684164087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c19d2c0 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.564+0000 7fe821450640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe820c4f640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00c970 tx=0x7fe80c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c007bb0 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe80c007d50 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c19d5b0 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c004e10 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.565+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c1a4e90 con 0x7fe81c103660 2024-09-05T06:39:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.566+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c103120 con 0x7fe81c103660 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.567+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe80c005020 con 0x7fe81c103660 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.567+0000 7fe8127fc640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 0x7fe8080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.568+0000 7fe821450640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 0x7fe8080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.568+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fe80c0cf0e0 con 0x7fe81c103660 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.568+0000 7fe821450640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 0x7fe8080a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe81800baa0 tx=0x7fe818005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:29.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.572+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe80c014030 con 0x7fe81c103660 2024-09-05T06:39:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.774+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe81c02d7c0 con 0x7fe81c103660 2024-09-05T06:39:29.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.849+0000 7fe8127fc640 1 -- 172.21.15.86:0/2684164087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fe80c09bd00 con 0x7fe81c103660 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.852+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 msgr2=0x7fe8080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.852+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 0x7fe8080a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe81800baa0 tx=0x7fe818005d90 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 msgr2=0x7fe81c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00c970 tx=0x7fe80c00ce30 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 shutdown_connections 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe81c10c170 0x7fe81c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe81c103660 0x7fe81c198850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8080a2c00 0x7fe8080a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 --2- 172.21.15.86:0/2684164087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe81c102d40 0x7fe81c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 >> 172.21.15.86:0/2684164087 conn(0x7fe81c0fc7a0 msgr2=0x7fe81c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 shutdown_connections 2024-09-05T06:39:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:29.853+0000 7fe8236b4640 1 -- 172.21.15.86:0/2684164087 wait complete. 2024-09-05T06:39:30.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:29 smithi115 ceph-mon[34791]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:30.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T06:39:30.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:29 smithi086 ceph-mon[30943]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:30.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T06:39:30.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:29 smithi096 ceph-mon[34468]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:30.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T06:39:30.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-05T06:39:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:30.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T06:39:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T06:39:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T06:39:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T06:39:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:30 smithi086 ceph-mon[30943]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T06:39:31.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T06:39:31.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T06:39:31.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T06:39:31.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:31.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T06:39:31.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:30 smithi115 ceph-mon[34791]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T06:39:31.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T06:39:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2684164087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T06:39:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T06:39:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T06:39:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-05T06:39:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:30 smithi096 ceph-mon[34468]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-05T06:39:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 -- 172.21.15.86:0/2573862695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840101420 msgr2=0x7f484010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 --2- 172.21.15.86:0/2573862695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840101420 0x7f484010d950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4838009cd0 tx=0x7f483802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 -- 172.21.15.86:0/2573862695 shutdown_connections 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 --2- 172.21.15.86:0/2573862695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840101420 0x7f484010d950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 --2- 172.21.15.86:0/2573862695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840100a80 0x7f4840100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 --2- 172.21.15.86:0/2573862695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840106a80 0x7f4840106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 -- 172.21.15.86:0/2573862695 >> 172.21.15.86:0/2573862695 conn(0x7f48400fc7a0 msgr2=0x7f48400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 -- 172.21.15.86:0/2573862695 shutdown_connections 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.414+0000 7f48477da640 1 -- 172.21.15.86:0/2573862695 wait complete. 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 Processor -- start 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 -- start start 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840101420 0x7f4840199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 0x7f48401a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4840110d30 con 0x7f4840100a80 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4840110bb0 con 0x7f4840106a80 2024-09-05T06:39:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f48477da640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4840110eb0 con 0x7f4840101420 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f4845576640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f4845576640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33742/0 (socket says 172.21.15.86:33742) 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f4845576640 1 -- 172.21.15.86:0/1359252116 learned_addr learned my addr 172.21.15.86:0/1359252116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f4844d75640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840101420 0x7f4840199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.415+0000 7f4845d77640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 0x7f48401a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 -- 172.21.15.86:0/1359252116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840101420 msgr2=0x7f4840199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840101420 0x7f4840199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 -- 172.21.15.86:0/1359252116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 msgr2=0x7f48401a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 0x7f48401a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 -- 172.21.15.86:0/1359252116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48401a4780 con 0x7f4840100a80 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845d77640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 0x7f48401a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f4845576640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d770 tx=0x7f483c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c017070 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f483c0054d0 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48401a4950 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c004cf0 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.416+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48401a4e10 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.417+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4840100ee0 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.418+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f483c004e90 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.418+0000 7f48367fc640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 0x7f482c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.419+0000 7f4844d75640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 0x7f482c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.419+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f483c0cf110 con 0x7f4840100a80 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.419+0000 7f4844d75640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 0x7f482c0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f484006b2e0 tx=0x7f483000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.422+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f483c09bd30 con 0x7f4840100a80 2024-09-05T06:39:31.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.623+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f4840109020 con 0x7f4840100a80 2024-09-05T06:39:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.836+0000 7f48367fc640 1 -- 172.21.15.86:0/1359252116 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f483c09b630 con 0x7f4840100a80 2024-09-05T06:39:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 msgr2=0x7f482c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 0x7f482c0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f484006b2e0 tx=0x7f483000b390 comp rx=0 tx=0).stop 2024-09-05T06:39:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 msgr2=0x7f48401991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d770 tx=0x7f483c00dc30 comp rx=0 tx=0).stop 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 shutdown_connections 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f482c0a2b80 0x7f482c0a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4840106a80 0x7f48401a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4840101420 0x7f4840199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 --2- 172.21.15.86:0/1359252116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4840100a80 0x7f48401991c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 >> 172.21.15.86:0/1359252116 conn(0x7f48400fc7a0 msgr2=0x7f48400fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.838+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 shutdown_connections 2024-09-05T06:39:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:31.839+0000 7f48477da640 1 -- 172.21.15.86:0/1359252116 wait complete. 2024-09-05T06:39:31.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:active 2024-09-05T06:39:31.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T06:39:31.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:active 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:31.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:31 smithi086 ceph-mon[30943]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-05T06:39:32.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:active 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:32.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:active 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:32.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:31 smithi115 ceph-mon[34791]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] up:active 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1359252116' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] up:active 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T06:39:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:32.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:31 smithi096 ceph-mon[34468]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-05T06:39:32.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-05T06:39:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- 172.21.15.86:0/2621123119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c102570 msgr2=0x7f023c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 --2- 172.21.15.86:0/2621123119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c102570 0x7f023c1029b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f022c009900 tx=0x7f022c02a480 comp rx=0 tx=0).stop 2024-09-05T06:39:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- 172.21.15.86:0/2621123119 shutdown_connections 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 --2- 172.21.15.86:0/2621123119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102ef0 0x7f023c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 --2- 172.21.15.86:0/2621123119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c102570 0x7f023c1029b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 --2- 172.21.15.86:0/2621123119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c100e20 0x7f023c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- 172.21.15.86:0/2621123119 >> 172.21.15.86:0/2621123119 conn(0x7f023c0fc7d0 msgr2=0x7f023c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- 172.21.15.86:0/2621123119 shutdown_connections 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- 172.21.15.86:0/2621123119 wait complete. 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 Processor -- start 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.475+0000 7f0242486640 1 -- start start 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102570 0x7f023c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c102ef0 0x7f023c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:33.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f023c110e30 con 0x7f023c100e20 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f023c110cb0 con 0x7f023c102570 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0242486640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f023c110fb0 con 0x7f023c102ef0 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48094/0 (socket says 172.21.15.86:48094) 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 -- 172.21.15.86:0/2719510830 learned_addr learned my addr 172.21.15.86:0/2719510830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023b7fe640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102570 0x7f023c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f0240a23640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c102ef0 0x7f023c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 -- 172.21.15.86:0/2719510830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c102ef0 msgr2=0x7f023c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c102ef0 0x7f023c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 -- 172.21.15.86:0/2719510830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102570 msgr2=0x7f023c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102570 0x7f023c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 -- 172.21.15.86:0/2719510830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c19d3f0 con 0x7f023c100e20 2024-09-05T06:39:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f023bfff640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f022400c970 tx=0x7f022400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.476+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224007bb0 con 0x7f023c100e20 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.477+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0224007d50 con 0x7f023c100e20 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.477+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022400f450 con 0x7f023c100e20 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.477+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c19d6e0 con 0x7f023c100e20 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.477+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c1a4fc0 con 0x7f023c100e20 2024-09-05T06:39:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.478+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c101200 con 0x7f023c100e20 2024-09-05T06:39:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.479+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0224004f20 con 0x7f023c100e20 2024-09-05T06:39:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.480+0000 7f02397fa640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 0x7f02280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.480+0000 7f023b7fe640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 0x7f02280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.480+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f02240cf130 con 0x7f023c100e20 2024-09-05T06:39:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.480+0000 7f023b7fe640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 0x7f02280a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f023c0ff0e0 tx=0x7f022c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.484+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0224014030 con 0x7f023c100e20 2024-09-05T06:39:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.695+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f023c10a0a0 con 0x7f023c100e20 2024-09-05T06:39:33.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.858+0000 7f02397fa640 1 -- 172.21.15.86:0/2719510830 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f022409bd50 con 0x7f023c100e20 2024-09-05T06:39:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 msgr2=0x7f02280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 0x7f02280a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f023c0ff0e0 tx=0x7f022c0095f0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 msgr2=0x7f023c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f022400c970 tx=0x7f022400ce30 comp rx=0 tx=0).stop 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 shutdown_connections 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f023c102ef0 0x7f023c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f02280a2c00 0x7f02280a50c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102570 0x7f023c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 --2- 172.21.15.86:0/2719510830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c100e20 0x7f023c198440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 >> 172.21.15.86:0/2719510830 conn(0x7f023c0fc7d0 msgr2=0x7f023c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 shutdown_connections 2024-09-05T06:39:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:33.861+0000 7f0242486640 1 -- 172.21.15.86:0/2719510830 wait complete. 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:active 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-05T06:39:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-05T06:39:33.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: pgmap v298: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-05T06:39:33.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T06:39:34.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:active 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: pgmap v298: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-05T06:39:34.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:33 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] up:active 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: pgmap v298: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-05T06:39:34.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:33 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T06:39:34.663 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-05T06:39:34.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-05T06:39:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:34.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:34 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T06:39:34.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:34 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:active 2024-09-05T06:39:34.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:34 smithi086 ceph-mon[30943]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-05T06:39:34.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:34 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:39:35.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:34 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T06:39:35.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:34 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:active 2024-09-05T06:39:35.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:34 smithi115 ceph-mon[34791]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-05T06:39:35.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:34 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:39:35.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:34 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2719510830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T06:39:35.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:34 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] up:active 2024-09-05T06:39:35.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:34 smithi096 ceph-mon[34468]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-05T06:39:35.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:34 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 -- 172.21.15.86:0/2681409030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c102570 msgr2=0x7fab1c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 --2- 172.21.15.86:0/2681409030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c102570 0x7fab1c1029b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002a480 comp rx=0 tx=0).stop 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 -- 172.21.15.86:0/2681409030 shutdown_connections 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 --2- 172.21.15.86:0/2681409030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102ef0 0x7fab1c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 --2- 172.21.15.86:0/2681409030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c102570 0x7fab1c1029b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 --2- 172.21.15.86:0/2681409030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c100e20 0x7fab1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 -- 172.21.15.86:0/2681409030 >> 172.21.15.86:0/2681409030 conn(0x7fab1c0fc7d0 msgr2=0x7fab1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 -- 172.21.15.86:0/2681409030 shutdown_connections 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.381+0000 7fab21054640 1 -- 172.21.15.86:0/2681409030 wait complete. 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 Processor -- start 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 -- start start 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102570 0x7fab1c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c102ef0 0x7fab1c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c110e80 con 0x7fab1c100e20 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c110d00 con 0x7fab1c102ef0 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab21054640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c111000 con 0x7fab1c102570 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48108/0 (socket says 172.21.15.86:48108) 2024-09-05T06:39:35.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1ad9d640 1 -- 172.21.15.86:0/1537043563 learned_addr learned my addr 172.21.15.86:0/1537043563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1b59e640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c102ef0 0x7fab1c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1a59c640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102570 0x7fab1c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1ad9d640 1 -- 172.21.15.86:0/1537043563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102570 msgr2=0x7fab1c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.382+0000 7fab1ad9d640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102570 0x7fab1c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab1ad9d640 1 -- 172.21.15.86:0/1537043563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c102ef0 msgr2=0x7fab1c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab1ad9d640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c102ef0 0x7fab1c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab1ad9d640 1 -- 172.21.15.86:0/1537043563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19d3c0 con 0x7fab1c100e20 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab1ad9d640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d950 tx=0x7fab0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017030 con 0x7fab1c100e20 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c19d6b0 con 0x7fab1c100e20 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0c004020 con 0x7fab1c100e20 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0047a0 con 0x7fab1c100e20 2024-09-05T06:39:35.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.383+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1a4f90 con 0x7fab1c100e20 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.384+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c101200 con 0x7fab1c100e20 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.385+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab0c0041d0 con 0x7fab1c100e20 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.386+0000 7fab03fff640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 0x7fab040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.386+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fab0c0cedb0 con 0x7fab1c100e20 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.386+0000 7fab1a59c640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 0x7fab040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.387+0000 7fab1a59c640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 0x7fab040a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fab10000c00 tx=0x7fab10005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.390+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab0c09b9d0 con 0x7fab1c100e20 2024-09-05T06:39:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.581+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fab1c062630 con 0x7fab040a2b80 2024-09-05T06:39:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.606+0000 7fab03fff640 1 -- 172.21.15.86:0/1537043563 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab1c062630 con 0x7fab040a2b80 2024-09-05T06:39:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 msgr2=0x7fab040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 0x7fab040a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fab10000c00 tx=0x7fab10005e80 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 msgr2=0x7fab1c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d950 tx=0x7fab0c00de10 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 shutdown_connections 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fab040a2b80 0x7fab040a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab1c102ef0 0x7fab1c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab1c102570 0x7fab1c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 --2- 172.21.15.86:0/1537043563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100e20 0x7fab1c198410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 >> 172.21.15.86:0/1537043563 conn(0x7fab1c0fc7d0 msgr2=0x7fab1c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 shutdown_connections 2024-09-05T06:39:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:35.608+0000 7fab21054640 1 -- 172.21.15.86:0/1537043563 wait complete. 2024-09-05T06:39:35.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:39:35.586+0000 7fb02898e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:39:35.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:39:35.586+0000 7fb02898e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:39:35.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:39:35.586+0000 7fb02898e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:39:35.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:39:35.586+0000 7fb02898e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:39:35.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:39:35.586+0000 7fb02898e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:39:36.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:35 smithi096 ceph-mon[34468]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-05T06:39:36.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 06:39:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T06:39:35.752+0000 7f2b817d1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-05T06:39:36.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 06:39:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T06:39:35.752+0000 7f2b817d1640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-05T06:39:36.058 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 06:39:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T06:39:35.752+0000 7f2b817d1640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-05T06:39:36.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:35 smithi115 ceph-mon[34791]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-05T06:39:36.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:35 smithi086 ceph-mon[30943]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-05T06:39:36.327 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-05T06:39:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:36.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:36 smithi086 ceph-mon[30943]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- 172.21.15.86:0/110286414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8105a90 msgr2=0x7f59d8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 --2- 172.21.15.86:0/110286414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8105a90 0x7f59d8105e70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009900 tx=0x7f59c802a410 comp rx=0 tx=0).stop 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- 172.21.15.86:0/110286414 shutdown_connections 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 --2- 172.21.15.86:0/110286414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101bc0 0x7f59d810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 --2- 172.21.15.86:0/110286414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8101240 0x7f59d8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 --2- 172.21.15.86:0/110286414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8105a90 0x7f59d8105e70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- 172.21.15.86:0/110286414 >> 172.21.15.86:0/110286414 conn(0x7f59d8077210 msgr2=0x7f59d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- 172.21.15.86:0/110286414 shutdown_connections 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- 172.21.15.86:0/110286414 wait complete. 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 Processor -- start 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.093+0000 7f59ddc90640 1 -- start start 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101240 0x7f59d80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8105a90 0x7f59d80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8110e20 con 0x7f59d8101bc0 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8110ca0 con 0x7f59d8101240 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8110fa0 con 0x7f59d8105a90 2024-09-05T06:39:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48144/0 (socket says 172.21.15.86:48144) 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 -- 172.21.15.86:0/1349907155 learned_addr learned my addr 172.21.15.86:0/1349907155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59d7fff640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8105a90 0x7f59d80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59d77fe640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101240 0x7f59d80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 -- 172.21.15.86:0/1349907155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8105a90 msgr2=0x7f59d80fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8105a90 0x7f59d80fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 -- 172.21.15.86:0/1349907155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101240 msgr2=0x7f59d80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101240 0x7f59d80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 -- 172.21.15.86:0/1349907155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d81a68f0 con 0x7f59d8101bc0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59cffff640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59c000d770 tx=0x7f59c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0017070 con 0x7f59d8101bc0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c00054d0 con 0x7f59d8101bc0 2024-09-05T06:39:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d81a6be0 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0004cf0 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.094+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d81a7120 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.096+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d8101680 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.096+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f59c0004e90 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.097+0000 7f59d57fa640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 0x7f59c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.097+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f59c00ced70 con 0x7f59d8101bc0 2024-09-05T06:39:37.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.097+0000 7f59d77fe640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 0x7f59c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.098+0000 7f59d77fe640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 0x7f59c40a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005ab0 tx=0x7f59c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.101+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59c009b990 con 0x7f59d8101bc0 2024-09-05T06:39:37.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:36 smithi115 ceph-mon[34791]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:37.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.293+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f59d8062630 con 0x7f59c40a2c00 2024-09-05T06:39:37.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:36 smithi096 ceph-mon[34468]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:37.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.315+0000 7f59d57fa640 1 -- 172.21.15.86:0/1349907155 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59d8062630 con 0x7f59c40a2c00 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 msgr2=0x7f59c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 0x7f59c40a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005ab0 tx=0x7f59c8005a20 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 msgr2=0x7f59d80fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59c000d770 tx=0x7f59c000dc30 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 shutdown_connections 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59d8105a90 0x7f59d80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d8101bc0 0x7f59d80fcbd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f59c40a2c00 0x7f59c40a50c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 --2- 172.21.15.86:0/1349907155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59d8101240 0x7f59d80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 >> 172.21.15.86:0/1349907155 conn(0x7f59d8077210 msgr2=0x7f59d8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 shutdown_connections 2024-09-05T06:39:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:37.318+0000 7f59ddc90640 1 -- 172.21.15.86:0/1349907155 wait complete. 2024-09-05T06:39:37.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:37 smithi086 ceph-mon[30943]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-05T06:39:37.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:37 smithi086 ceph-mon[30943]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:38.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-05T06:39:38.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:37 smithi115 ceph-mon[34791]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-05T06:39:38.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:37 smithi115 ceph-mon[34791]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:38.302 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:38.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:37 smithi096 ceph-mon[34468]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-05T06:39:38.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:37 smithi096 ceph-mon[34468]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- 172.21.15.86:0/1957347449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650106f10 msgr2=0x7f065010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- 172.21.15.86:0/1957347449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650106f10 0x7f065010ea90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0648009ca0 tx=0x7f064802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- 172.21.15.86:0/1957347449 shutdown_connections 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- 172.21.15.86:0/1957347449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650106f10 0x7f065010ea90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- 172.21.15.86:0/1957347449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650102790 0x7f06501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- 172.21.15.86:0/1957347449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650101de0 0x7f06501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- 172.21.15.86:0/1957347449 >> 172.21.15.86:0/1957347449 conn(0x7f06500fd8c0 msgr2=0x7f06500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- 172.21.15.86:0/1957347449 shutdown_connections 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- 172.21.15.86:0/1957347449 wait complete. 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 Processor -- start 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- start start 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650101de0 0x7f0650198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 0x7f065019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0650111ec0 con 0x7f0650102790 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.753+0000 7f06570e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0650111d40 con 0x7f0650101de0 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f06570e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0650112040 con 0x7f0650106f10 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48164/0 (socket says 172.21.15.86:48164) 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 -- 172.21.15.86:0/1106365402 learned_addr learned my addr 172.21.15.86:0/1106365402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f0654e7f640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650101de0 0x7f0650198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f0655680640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 0x7f065019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 -- 172.21.15.86:0/1106365402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 msgr2=0x7f065019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 0x7f065019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 -- 172.21.15.86:0/1106365402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650101de0 msgr2=0x7f0650198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650101de0 0x7f0650198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 -- 172.21.15.86:0/1106365402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065019d430 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f0655680640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 0x7f065019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064ffff640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f063c00b4a0 tx=0x7f063c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c014070 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065019d720 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f063c010ac0 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.754+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c017c30 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.755+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06501a5000 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.755+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06501021c0 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.756+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f063c017070 con 0x7f0650102790 2024-09-05T06:39:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.757+0000 7f064dffb640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 0x7f06400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.757+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f063c0ceb90 con 0x7f0650102790 2024-09-05T06:39:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.757+0000 7f0654e7f640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 0x7f06400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.758+0000 7f0654e7f640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 0x7f06400a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0644009890 tx=0x7f0644008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.761+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f063c09b760 con 0x7f0650102790 2024-09-05T06:39:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.957+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f065010a120 con 0x7f06400a2c10 2024-09-05T06:39:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.962+0000 7f064dffb640 1 -- 172.21.15.86:0/1106365402 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f065010a120 con 0x7f06400a2c10 2024-09-05T06:39:38.963 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/f90eafeb-ef3e-47d2-90fd-8c07576c6e3c 2024-09-05T06:39:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 msgr2=0x7f06400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 0x7f06400a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0644009890 tx=0x7f0644008040 comp rx=0 tx=0).stop 2024-09-05T06:39:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 msgr2=0x7f06501989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f063c00b4a0 tx=0x7f063c00b960 comp rx=0 tx=0).stop 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 shutdown_connections 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0650106f10 0x7f065019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0650102790 0x7f06501989c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f06400a2c10 0x7f06400a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 --2- 172.21.15.86:0/1106365402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0650101de0 0x7f0650198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 >> 172.21.15.86:0/1106365402 conn(0x7f06500fd8c0 msgr2=0x7f06500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 shutdown_connections 2024-09-05T06:39:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:38.964+0000 7f06570e3640 1 -- 172.21.15.86:0/1106365402 wait complete. 2024-09-05T06:39:39.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:38 smithi115 ceph-mon[34791]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T06:39:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:38 smithi086 ceph-mon[30943]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T06:39:39.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:38 smithi096 ceph-mon[34468]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T06:39:39.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-05T06:39:39.901 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:40.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:39 smithi086 ceph-mon[30943]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:40.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:39 smithi086 ceph-mon[30943]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-05T06:39:40.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:39 smithi115 ceph-mon[34791]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:40.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:39 smithi115 ceph-mon[34791]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-05T06:39:40.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:39 smithi096 ceph-mon[34468]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:39 smithi096 ceph-mon[34468]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-05T06:39:40.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 -- 172.21.15.86:0/1091073137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 msgr2=0x7fda441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:40.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1091073137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441021d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda40009900 tx=0x7fda4002a410 comp rx=0 tx=0).stop 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 -- 172.21.15.86:0/1091073137 shutdown_connections 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1091073137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda44106f20 0x7fda4410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1091073137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda441027a0 0x7fda441067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1091073137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441021d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 -- 172.21.15.86:0/1091073137 >> 172.21.15.86:0/1091073137 conn(0x7fda440fd8f0 msgr2=0x7fda440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 -- 172.21.15.86:0/1091073137 shutdown_connections 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.397+0000 7fda4b81a640 1 -- 172.21.15.86:0/1091073137 wait complete. 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 Processor -- start 2024-09-05T06:39:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 -- start start 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 0x7fda44198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda44106f20 0x7fda4419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111e30 con 0x7fda44101df0 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111cb0 con 0x7fda44106f20 2024-09-05T06:39:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda4b81a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111fb0 con 0x7fda441027a0 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda495b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda495b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48180/0 (socket says 172.21.15.86:48180) 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda48db5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 0x7fda44198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.398+0000 7fda495b6640 1 -- 172.21.15.86:0/1428457615 learned_addr learned my addr 172.21.15.86:0/1428457615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda49db7640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda44106f20 0x7fda4419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 -- 172.21.15.86:0/1428457615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda44106f20 msgr2=0x7fda4419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda44106f20 0x7fda4419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 -- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 msgr2=0x7fda441982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 -- 172.21.15.86:0/1428457615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4419d2a0 con 0x7fda441027a0 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda495b6640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:39:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda48db5640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 0x7fda44198830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d6d0 tx=0x7fda3400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34017070 con 0x7fda441027a0 2024-09-05T06:39:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda34009820 con 0x7fda441027a0 2024-09-05T06:39:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4419d590 con 0x7fda441027a0 2024-09-05T06:39:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34004c40 con 0x7fda441027a0 2024-09-05T06:39:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.399+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441a4e70 con 0x7fda441027a0 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.400+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda441021d0 con 0x7fda441027a0 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.401+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda340099d0 con 0x7fda441027a0 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.402+0000 7fda3a7fc640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 0x7fda300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.402+0000 7fda495b6640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 0x7fda300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.402+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fda340cf140 con 0x7fda441027a0 2024-09-05T06:39:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.403+0000 7fda495b6640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 0x7fda300a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda40031440 tx=0x7fda400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.406+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda3409b550 con 0x7fda441027a0 2024-09-05T06:39:40.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.596+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda4410a160 con 0x7fda300a2c10 2024-09-05T06:39:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.619+0000 7fda3a7fc640 1 -- 172.21.15.86:0/1428457615 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda4410a160 con 0x7fda300a2c10 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 msgr2=0x7fda300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 0x7fda300a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda40031440 tx=0x7fda400058d0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 msgr2=0x7fda44198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 0x7fda44198830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d6d0 tx=0x7fda3400db90 comp rx=0 tx=0).stop 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 shutdown_connections 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fda300a2c10 0x7fda300a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda44106f20 0x7fda4419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda441027a0 0x7fda44198830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 --2- 172.21.15.86:0/1428457615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda44101df0 0x7fda441982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 >> 172.21.15.86:0/1428457615 conn(0x7fda440fd8f0 msgr2=0x7fda440fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 shutdown_connections 2024-09-05T06:39:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:40.621+0000 7fda4b81a640 1 -- 172.21.15.86:0/1428457615 wait complete. 2024-09-05T06:39:41.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-05T06:39:41.711 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 -- 172.21.15.86:0/159884553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0103660 msgr2=0x7fb2b010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 --2- 172.21.15.86:0/159884553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0103660 0x7fb2b010bb50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009900 tx=0x7fb2a402a480 comp rx=0 tx=0).stop 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 -- 172.21.15.86:0/159884553 shutdown_connections 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 --2- 172.21.15.86:0/159884553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b010c170 0x7fb2b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 --2- 172.21.15.86:0/159884553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b0103660 0x7fb2b010bb50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 --2- 172.21.15.86:0/159884553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0102d40 0x7fb2b0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 -- 172.21.15.86:0/159884553 >> 172.21.15.86:0/159884553 conn(0x7fb2b00fc7a0 msgr2=0x7fb2b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 -- 172.21.15.86:0/159884553 shutdown_connections 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.217+0000 7fb2b5303640 1 -- 172.21.15.86:0/159884553 wait complete. 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 Processor -- start 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 -- start start 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b0102d40 0x7fb2b01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:42.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0103660 0x7fb2b0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0111e60 con 0x7fb2b010c170 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0111ce0 con 0x7fb2b0103660 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2b5303640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0111fe0 con 0x7fb2b0102d40 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48200/0 (socket says 172.21.15.86:48200) 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.218+0000 7fb2af7fe640 1 -- 172.21.15.86:0/2970240398 learned_addr learned my addr 172.21.15.86:0/2970240398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2ae7fc640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0103660 0x7fb2b0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2aeffd640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b0102d40 0x7fb2b01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 -- 172.21.15.86:0/2970240398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b0102d40 msgr2=0x7fb2b01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b0102d40 0x7fb2b01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 -- 172.21.15.86:0/2970240398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0103660 msgr2=0x7fb2b0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0103660 0x7fb2b0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 -- 172.21.15.86:0/2970240398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b019d3a0 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2af7fe640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b970 tx=0x7fb2a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8019070 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b019d690 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8014440 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80075d0 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.219+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b01a4f70 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.220+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b0103120 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.221+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb2a801c070 con 0x7fb2b010c170 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.222+0000 7fb297fff640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 0x7fb2980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.222+0000 7fb2aeffd640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 0x7fb2980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.222+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb2a80cea00 con 0x7fb2b010c170 2024-09-05T06:39:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.222+0000 7fb2aeffd640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 0x7fb2980a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.226+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2a80d0e20 con 0x7fb2b010c170 2024-09-05T06:39:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.419+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb2b0062630 con 0x7fb2980a2c10 2024-09-05T06:39:42.419 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:42 smithi086 ceph-mon[30943]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:42.420 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:42 smithi086 ceph-mon[30943]: pgmap v302: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-05T06:39:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.423+0000 7fb297fff640 1 -- 172.21.15.86:0/2970240398 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb2b0062630 con 0x7fb2980a2c10 2024-09-05T06:39:42.423 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/65c9bd09-af32-4725-a5f9-e483c80d767c 2024-09-05T06:39:42.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.425+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 msgr2=0x7fb2980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:42.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.425+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 0x7fb2980a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0005850 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.425+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 msgr2=0x7fb2b019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.425+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b970 tx=0x7fb2a800be30 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 shutdown_connections 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b010c170 0x7fb2b019ccc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb2980a2c10 0x7fb2980a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2b0103660 0x7fb2b0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 --2- 172.21.15.86:0/2970240398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2b0102d40 0x7fb2b01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 >> 172.21.15.86:0/2970240398 conn(0x7fb2b00fc7a0 msgr2=0x7fb2b00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 shutdown_connections 2024-09-05T06:39:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:42.426+0000 7fb2b5303640 1 -- 172.21.15.86:0/2970240398 wait complete. 2024-09-05T06:39:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:42 smithi096 ceph-mon[34468]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:42 smithi096 ceph-mon[34468]: pgmap v302: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-05T06:39:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:42 smithi115 ceph-mon[34791]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:42.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:42 smithi115 ceph-mon[34791]: pgmap v302: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-05T06:39:43.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:39:43.431 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 -- 172.21.15.86:0/3862041073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae88106f20 msgr2=0x7fae8810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 --2- 172.21.15.86:0/3862041073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae88106f20 0x7fae8810eaa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ca0 tx=0x7fae8002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 -- 172.21.15.86:0/3862041073 shutdown_connections 2024-09-05T06:39:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 --2- 172.21.15.86:0/3862041073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae88106f20 0x7fae8810eaa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 --2- 172.21.15.86:0/3862041073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae881027a0 0x7fae881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 --2- 172.21.15.86:0/3862041073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 0x7fae881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 -- 172.21.15.86:0/3862041073 >> 172.21.15.86:0/3862041073 conn(0x7fae880fd8b0 msgr2=0x7fae880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 -- 172.21.15.86:0/3862041073 shutdown_connections 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.898+0000 7fae8cb86640 1 -- 172.21.15.86:0/3862041073 wait complete. 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 Processor -- start 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 -- start start 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 0x7fae88198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae88106f20 0x7fae8819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae88111e30 con 0x7fae881027a0 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae88111cb0 con 0x7fae88106f20 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8cb86640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae88111fb0 con 0x7fae88101df0 2024-09-05T06:39:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49938/0 (socket says 172.21.15.86:49938) 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae85d9b640 1 -- 172.21.15.86:0/1537903951 learned_addr learned my addr 172.21.15.86:0/1537903951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae86d9d640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae88106f20 0x7fae8819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.899+0000 7fae8659c640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 0x7fae88198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 -- 172.21.15.86:0/1537903951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 msgr2=0x7fae88198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 0x7fae88198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 -- 172.21.15.86:0/1537903951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae88106f20 msgr2=0x7fae8819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae88106f20 0x7fae8819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 -- 172.21.15.86:0/1537903951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8819d2e0 con 0x7fae881027a0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae85d9b640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d950 tx=0x7fae7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78017030 con 0x7fae881027a0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8819d5d0 con 0x7fae881027a0 2024-09-05T06:39:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae78004020 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae780047a0 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.900+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a4eb0 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.901+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae881021d0 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.902+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae780041d0 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.903+0000 7fae777fe640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 0x7fae700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.903+0000 7fae8659c640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 0x7fae700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.903+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fae780cee90 con 0x7fae881027a0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.903+0000 7fae8659c640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 0x7fae700a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00bbe0 tx=0x7fae7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:43.907+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae7809b950 con 0x7fae881027a0 2024-09-05T06:39:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.111+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae8810a160 con 0x7fae881027a0 2024-09-05T06:39:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.112+0000 7fae777fe640 1 -- 172.21.15.86:0/1537903951 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7fae7809b250 con 0x7fae881027a0 2024-09-05T06:39:44.112 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:44.113 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:39:44.113 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:39:44.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 msgr2=0x7fae700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:44.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 0x7fae700a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00bbe0 tx=0x7fae7c00b390 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 msgr2=0x7fae88198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d950 tx=0x7fae7800de10 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 shutdown_connections 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae700a2c10 0x7fae700a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae88106f20 0x7fae8819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae881027a0 0x7fae88198870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 --2- 172.21.15.86:0/1537903951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae88101df0 0x7fae88198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.114+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 >> 172.21.15.86:0/1537903951 conn(0x7fae880fd8b0 msgr2=0x7fae880febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.115+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 shutdown_connections 2024-09-05T06:39:44.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:44.115+0000 7fae8cb86640 1 -- 172.21.15.86:0/1537903951 wait complete. 2024-09-05T06:39:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:44 smithi086 ceph-mon[30943]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:44 smithi086 ceph-mon[30943]: pgmap v303: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-05T06:39:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:44 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1537903951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:44 smithi096 ceph-mon[34468]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:44 smithi096 ceph-mon[34468]: pgmap v303: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-05T06:39:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:44 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1537903951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:44 smithi115 ceph-mon[34791]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:39:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:44 smithi115 ceph-mon[34791]: pgmap v303: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-05T06:39:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:44 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1537903951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:39:44.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:45.240 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- 172.21.15.86:0/1665332894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102ef0 msgr2=0x7f1fc810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/1665332894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102ef0 0x7f1fc810da40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ca0 tx=0x7f1fc002a950 comp rx=0 tx=0).stop 2024-09-05T06:39:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- 172.21.15.86:0/1665332894 shutdown_connections 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/1665332894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102ef0 0x7f1fc810da40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/1665332894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8102570 0x7f1fc81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/1665332894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8100e20 0x7f1fc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- 172.21.15.86:0/1665332894 >> 172.21.15.86:0/1665332894 conn(0x7f1fc80fc7d0 msgr2=0x7f1fc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- 172.21.15.86:0/1665332894 shutdown_connections 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- 172.21.15.86:0/1665332894 wait complete. 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 Processor -- start 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.709+0000 7f1fcf13b640 1 -- start start 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8102ef0 0x7f1fc819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8110e80 con 0x7f1fc8102570 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8110d00 con 0x7f1fc8102ef0 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8111000 con 0x7f1fc8100e20 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49960/0 (socket says 172.21.15.86:49960) 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 -- 172.21.15.86:0/520675374 learned_addr learned my addr 172.21.15.86:0/520675374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcd6d8640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8102ef0 0x7f1fc819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcced7640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 -- 172.21.15.86:0/520675374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 msgr2=0x7f1fc8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 -- 172.21.15.86:0/520675374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8102ef0 msgr2=0x7f1fc819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8102ef0 0x7f1fc819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 -- 172.21.15.86:0/520675374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc819d3e0 con 0x7f1fc8102570 2024-09-05T06:39:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc7fff640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4017070 con 0x7f1fc8102570 2024-09-05T06:39:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc819d6d0 con 0x7f1fc8102570 2024-09-05T06:39:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb4009c30 con 0x7f1fc8102570 2024-09-05T06:39:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.710+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400f620 con 0x7f1fc8102570 2024-09-05T06:39:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.711+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a4fb0 con 0x7f1fc8102570 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.712+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8101200 con 0x7f1fc8102570 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.713+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fb40109a0 con 0x7f1fc8102570 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.713+0000 7f1fc5ffb640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 0x7f1fb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.714+0000 7f1fcced7640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 0x7f1fb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.714+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f1fb40cf040 con 0x7f1fc8102570 2024-09-05T06:39:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.714+0000 7f1fcced7640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 0x7f1fb80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009bd0 tx=0x7f1fbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.718+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fb4009de0 con 0x7f1fc8102570 2024-09-05T06:39:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.886+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fc81a51e0 con 0x7f1fc8102570 2024-09-05T06:39:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.887+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/520675374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7f1fb409b4b0 con 0x7f1fc8102570 2024-09-05T06:39:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:45.888 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 msgr2=0x7f1fb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 0x7f1fb80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009bd0 tx=0x7f1fbc005f90 comp rx=0 tx=0).stop 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 msgr2=0x7f1fc8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400de10 comp rx=0 tx=0).stop 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 shutdown_connections 2024-09-05T06:39:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1fb80a2c10 0x7f1fb80a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fc8102ef0 0x7f1fc819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc8102570 0x7f1fc8198970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 --2- 172.21.15.86:0/520675374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 >> 172.21.15.86:0/520675374 conn(0x7f1fc80fc7d0 msgr2=0x7f1fc8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 shutdown_connections 2024-09-05T06:39:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:45.890+0000 7f1fcf13b640 1 -- 172.21.15.86:0/520675374 wait complete. 2024-09-05T06:39:46.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:46 smithi086 ceph-mon[30943]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-05T06:39:46.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:46 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/520675374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:46.556 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T06:39:46.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:39:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:46 smithi096 ceph-mon[34468]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-05T06:39:46.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:46 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/520675374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:46 smithi115 ceph-mon[34791]: pgmap v304: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2024-09-05T06:39:46.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:46 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/520675374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/2750501741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4106f20 msgr2=0x7fb6a410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/2750501741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4106f20 0x7fb6a410eaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009ca0 tx=0x7fb69c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/2750501741 shutdown_connections 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/2750501741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4106f20 0x7fb6a410eaa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/2750501741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a41027a0 0x7fb6a41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/2750501741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a4101df0 0x7fb6a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/2750501741 >> 172.21.15.86:0/2750501741 conn(0x7fb6a40fd8f0 msgr2=0x7fb6a40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/2750501741 shutdown_connections 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.259+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/2750501741 wait complete. 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 Processor -- start 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 -- start start 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a41027a0 0x7fb6a4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a4106f20 0x7fb6a419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4111e80 con 0x7fb6a4101df0 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4111d00 con 0x7fb6a4106f20 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6aa0fe640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4112000 con 0x7fb6a41027a0 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49972/0 (socket says 172.21.15.86:49972) 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.260+0000 7fb6a37fe640 1 -- 172.21.15.86:0/4261460769 learned_addr learned my addr 172.21.15.86:0/4261460769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a3fff640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a4106f20 0x7fb6a419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a2ffd640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a41027a0 0x7fb6a4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 -- 172.21.15.86:0/4261460769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a41027a0 msgr2=0x7fb6a4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a41027a0 0x7fb6a4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 -- 172.21.15.86:0/4261460769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a4106f20 msgr2=0x7fb6a419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a4106f20 0x7fb6a419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 -- 172.21.15.86:0/4261460769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a419d2b0 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a37fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb69400d950 tx=0x7fb69400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694017030 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb694004020 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6940047a0 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a419d5a0 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.261+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a41a4e80 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.262+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a41021d0 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.264+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb6940041d0 con 0x7fb6a4101df0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.264+0000 7fb6a0ff9640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 0x7fb68c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.264+0000 7fb6a2ffd640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 0x7fb68c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.264+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb6940cf5d0 con 0x7fb6a4101df0 2024-09-05T06:39:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.265+0000 7fb6a2ffd640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 0x7fb68c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40fdd10 tx=0x7fb698005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.267+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb69409c110 con 0x7fb6a4101df0 2024-09-05T06:39:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.433+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6a4062630 con 0x7fb68c0a2c90 2024-09-05T06:39:47.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.442+0000 7fb6a0ff9640 1 -- 172.21.15.86:0/4261460769 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340386 (secure 0 0 0) 0x7fb6a4062630 con 0x7fb68c0a2c90 2024-09-05T06:39:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 msgr2=0x7fb68c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 0x7fb68c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40fdd10 tx=0x7fb698005f90 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 msgr2=0x7fb6a4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb69400d950 tx=0x7fb69400de10 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 shutdown_connections 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb68c0a2c90 0x7fb68c0a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6a4106f20 0x7fb6a419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6a41027a0 0x7fb6a4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 --2- 172.21.15.86:0/4261460769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6a4101df0 0x7fb6a4198300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.445+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 >> 172.21.15.86:0/4261460769 conn(0x7fb6a40fd8f0 msgr2=0x7fb6a40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.446+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 shutdown_connections 2024-09-05T06:39:47.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:47.446+0000 7fb6aa0fe640 1 -- 172.21.15.86:0/4261460769 wait complete. 2024-09-05T06:39:48.171 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-05T06:39:47.194302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641155,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":299,"num_read_kb":258,"num_write":415,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388020,"kb_used_data":13812,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553804,"statfs":{"total":1151940427776,"available":1151543095296,"internally_reserved":0,"allocated":14143488,"data_stored":7805931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41289,"num_objects":21,"num_object_clones":0,"num_object_copies":65,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":133,"num_read_kb":112,"num_write":66,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.887045+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:39:43.887045+0000","last_peered":"2024-09-05T06:39:43.887045+0000","last_clean":"2024-09-05T06:39:43.887045+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:39:43.887045+0000","last_undegraded":"2024-09-05T06:39:43.887045+0000","last_fullsized":"2024-09-05T06:39:43.887045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927748+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:39:12.927748+0000","last_peered":"2024-09-05T06:39:12.927748+0000","last_clean":"2024-09-05T06:39:12.927748+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:39:12.927748+0000","last_undegraded":"2024-09-05T06:39:12.927748+0000","last_fullsized":"2024-09-05T06:39:12.927748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928430+0000","last_change":"2024-09-05T06:39:02.873883+0000","last_active":"2024-09-05T06:39:12.928430+0000","last_peered":"2024-09-05T06:39:12.928430+0000","last_clean":"2024-09-05T06:39:12.928430+0000","last_became_active":"2024-09-05T06:39:02.873471+0000","last_became_peered":"2024-09-05T06:39:02.873471+0000","last_unstale":"2024-09-05T06:39:12.928430+0000","last_undegraded":"2024-09-05T06:39:12.928430+0000","last_fullsized":"2024-09-05T06:39:12.928430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:54:30.025583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928804+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:39:12.928804+0000","last_peered":"2024-09-05T06:39:12.928804+0000","last_clean":"2024-09-05T06:39:12.928804+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:39:12.928804+0000","last_undegraded":"2024-09-05T06:39:12.928804+0000","last_fullsized":"2024-09-05T06:39:12.928804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924103+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:39:12.924103+0000","last_peered":"2024-09-05T06:39:12.924103+0000","last_clean":"2024-09-05T06:39:12.924103+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:39:12.924103+0000","last_undegraded":"2024-09-05T06:39:12.924103+0000","last_fullsized":"2024-09-05T06:39:12.924103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929854+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:39:12.929854+0000","last_peered":"2024-09-05T06:39:12.929854+0000","last_clean":"2024-09-05T06:39:12.929854+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:39:12.929854+0000","last_undegraded":"2024-09-05T06:39:12.929854+0000","last_fullsized":"2024-09-05T06:39:12.929854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:33.871541+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:39:33.871541+0000","last_peered":"2024-09-05T06:39:33.871541+0000","last_clean":"2024-09-05T06:39:33.871541+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:39:33.871541+0000","last_undegraded":"2024-09-05T06:39:33.871541+0000","last_fullsized":"2024-09-05T06:39:33.871541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932308+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:39:12.932308+0000","last_peered":"2024-09-05T06:39:12.932308+0000","last_clean":"2024-09-05T06:39:12.932308+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:39:12.932308+0000","last_undegraded":"2024-09-05T06:39:12.932308+0000","last_fullsized":"2024-09-05T06:39:12.932308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930662+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:39:12.930662+0000","last_peered":"2024-09-05T06:39:12.930662+0000","last_clean":"2024-09-05T06:39:12.930662+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:39:12.930662+0000","last_undegraded":"2024-09-05T06:39:12.930662+0000","last_fullsized":"2024-09-05T06:39:12.930662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.860355+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:39:32.860355+0000","last_peered":"2024-09-05T06:39:32.860355+0000","last_clean":"2024-09-05T06:39:32.860355+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:39:32.860355+0000","last_undegraded":"2024-09-05T06:39:32.860355+0000","last_fullsized":"2024-09-05T06:39:32.860355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931310+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:39:12.931310+0000","last_peered":"2024-09-05T06:39:12.931310+0000","last_clean":"2024-09-05T06:39:12.931310+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:39:12.931310+0000","last_undegraded":"2024-09-05T06:39:12.931310+0000","last_fullsized":"2024-09-05T06:39:12.931310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930102+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:39:12.930102+0000","last_peered":"2024-09-05T06:39:12.930102+0000","last_clean":"2024-09-05T06:39:12.930102+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:39:12.930102+0000","last_undegraded":"2024-09-05T06:39:12.930102+0000","last_fullsized":"2024-09-05T06:39:12.930102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924768+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:39:12.924768+0000","last_peered":"2024-09-05T06:39:12.924768+0000","last_clean":"2024-09-05T06:39:12.924768+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:39:12.924768+0000","last_undegraded":"2024-09-05T06:39:12.924768+0000","last_fullsized":"2024-09-05T06:39:12.924768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:29.869845+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:39:29.869845+0000","last_peered":"2024-09-05T06:39:29.869845+0000","last_clean":"2024-09-05T06:39:29.869845+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:39:29.869845+0000","last_undegraded":"2024-09-05T06:39:29.869845+0000","last_fullsized":"2024-09-05T06:39:29.869845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930481+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:39:12.930481+0000","last_peered":"2024-09-05T06:39:12.930481+0000","last_clean":"2024-09-05T06:39:12.930481+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:39:12.930481+0000","last_undegraded":"2024-09-05T06:39:12.930481+0000","last_fullsized":"2024-09-05T06:39:12.930481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930743+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:39:12.930743+0000","last_peered":"2024-09-05T06:39:12.930743+0000","last_clean":"2024-09-05T06:39:12.930743+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:39:12.930743+0000","last_undegraded":"2024-09-05T06:39:12.930743+0000","last_fullsized":"2024-09-05T06:39:12.930743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854799+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:39:31.854799+0000","last_peered":"2024-09-05T06:39:31.854799+0000","last_clean":"2024-09-05T06:39:31.854799+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:39:31.854799+0000","last_undegraded":"2024-09-05T06:39:31.854799+0000","last_fullsized":"2024-09-05T06:39:31.854799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931985+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:39:12.931985+0000","last_peered":"2024-09-05T06:39:12.931985+0000","last_clean":"2024-09-05T06:39:12.931985+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:39:12.931985+0000","last_undegraded":"2024-09-05T06:39:12.931985+0000","last_fullsized":"2024-09-05T06:39:12.931985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927766+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:39:12.927766+0000","last_peered":"2024-09-05T06:39:12.927766+0000","last_clean":"2024-09-05T06:39:12.927766+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:39:12.927766+0000","last_undegraded":"2024-09-05T06:39:12.927766+0000","last_fullsized":"2024-09-05T06:39:12.927766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859651+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:39:32.859651+0000","last_peered":"2024-09-05T06:39:32.859651+0000","last_clean":"2024-09-05T06:39:32.859651+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:39:32.859651+0000","last_undegraded":"2024-09-05T06:39:32.859651+0000","last_fullsized":"2024-09-05T06:39:32.859651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925738+0000","last_change":"2024-09-05T06:39:02.872883+0000","last_active":"2024-09-05T06:39:12.925738+0000","last_peered":"2024-09-05T06:39:12.925738+0000","last_clean":"2024-09-05T06:39:12.925738+0000","last_became_active":"2024-09-05T06:39:02.872523+0000","last_became_peered":"2024-09-05T06:39:02.872523+0000","last_unstale":"2024-09-05T06:39:12.925738+0000","last_undegraded":"2024-09-05T06:39:12.925738+0000","last_fullsized":"2024-09-05T06:39:12.925738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:51:46.797271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924420+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:39:12.924420+0000","last_peered":"2024-09-05T06:39:12.924420+0000","last_clean":"2024-09-05T06:39:12.924420+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:39:12.924420+0000","last_undegraded":"2024-09-05T06:39:12.924420+0000","last_fullsized":"2024-09-05T06:39:12.924420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:29.869008+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:39:29.869008+0000","last_peered":"2024-09-05T06:39:29.869008+0000","last_clean":"2024-09-05T06:39:29.869008+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:39:29.869008+0000","last_undegraded":"2024-09-05T06:39:29.869008+0000","last_fullsized":"2024-09-05T06:39:29.869008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930518+0000","last_change":"2024-09-05T06:39:02.962341+0000","last_active":"2024-09-05T06:39:12.930518+0000","last_peered":"2024-09-05T06:39:12.930518+0000","last_clean":"2024-09-05T06:39:12.930518+0000","last_became_active":"2024-09-05T06:39:02.961849+0000","last_became_peered":"2024-09-05T06:39:02.961849+0000","last_unstale":"2024-09-05T06:39:12.930518+0000","last_undegraded":"2024-09-05T06:39:12.930518+0000","last_fullsized":"2024-09-05T06:39:12.930518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:54:11.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930785+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:39:12.930785+0000","last_peered":"2024-09-05T06:39:12.930785+0000","last_clean":"2024-09-05T06:39:12.930785+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:39:12.930785+0000","last_undegraded":"2024-09-05T06:39:12.930785+0000","last_fullsized":"2024-09-05T06:39:12.930785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.858514+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:39:32.858514+0000","last_peered":"2024-09-05T06:39:32.858514+0000","last_clean":"2024-09-05T06:39:32.858514+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:39:32.858514+0000","last_undegraded":"2024-09-05T06:39:32.858514+0000","last_fullsized":"2024-09-05T06:39:32.858514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928546+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:39:12.928546+0000","last_peered":"2024-09-05T06:39:12.928546+0000","last_clean":"2024-09-05T06:39:12.928546+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:39:12.928546+0000","last_undegraded":"2024-09-05T06:39:12.928546+0000","last_fullsized":"2024-09-05T06:39:12.928546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928319+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:39:12.928319+0000","last_peered":"2024-09-05T06:39:12.928319+0000","last_clean":"2024-09-05T06:39:12.928319+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:39:12.928319+0000","last_undegraded":"2024-09-05T06:39:12.928319+0000","last_fullsized":"2024-09-05T06:39:12.928319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:41.378348+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:39:41.378348+0000","last_peered":"2024-09-05T06:39:41.378348+0000","last_clean":"2024-09-05T06:39:41.378348+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:39:41.378348+0000","last_undegraded":"2024-09-05T06:39:41.378348+0000","last_fullsized":"2024-09-05T06:39:41.378348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928200+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:39:12.928200+0000","last_peered":"2024-09-05T06:39:12.928200+0000","last_clean":"2024-09-05T06:39:12.928200+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:39:12.928200+0000","last_undegraded":"2024-09-05T06:39:12.928200+0000","last_fullsized":"2024-09-05T06:39:12.928200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928906+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:39:12.928906+0000","last_peered":"2024-09-05T06:39:12.928906+0000","last_clean":"2024-09-05T06:39:12.928906+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:39:12.928906+0000","last_undegraded":"2024-09-05T06:39:12.928906+0000","last_fullsized":"2024-09-05T06:39:12.928906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854092+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:39:31.854092+0000","last_peered":"2024-09-05T06:39:31.854092+0000","last_clean":"2024-09-05T06:39:31.854092+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:39:31.854092+0000","last_undegraded":"2024-09-05T06:39:31.854092+0000","last_fullsized":"2024-09-05T06:39:31.854092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928066+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:39:12.928066+0000","last_peered":"2024-09-05T06:39:12.928066+0000","last_clean":"2024-09-05T06:39:12.928066+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:39:12.928066+0000","last_undegraded":"2024-09-05T06:39:12.928066+0000","last_fullsized":"2024-09-05T06:39:12.928066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927876+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:39:12.927876+0000","last_peered":"2024-09-05T06:39:12.927876+0000","last_clean":"2024-09-05T06:39:12.927876+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:39:12.927876+0000","last_undegraded":"2024-09-05T06:39:12.927876+0000","last_fullsized":"2024-09-05T06:39:12.927876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925688+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:39:12.925688+0000","last_peered":"2024-09-05T06:39:12.925688+0000","last_clean":"2024-09-05T06:39:12.925688+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:39:12.925688+0000","last_undegraded":"2024-09-05T06:39:12.925688+0000","last_fullsized":"2024-09-05T06:39:12.925688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924413+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:39:12.924413+0000","last_peered":"2024-09-05T06:39:12.924413+0000","last_clean":"2024-09-05T06:39:12.924413+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:39:12.924413+0000","last_undegraded":"2024-09-05T06:39:12.924413+0000","last_fullsized":"2024-09-05T06:39:12.924413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928021+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:39:12.928021+0000","last_peered":"2024-09-05T06:39:12.928021+0000","last_clean":"2024-09-05T06:39:12.928021+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:39:12.928021+0000","last_undegraded":"2024-09-05T06:39:12.928021+0000","last_fullsized":"2024-09-05T06:39:12.928021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:41.884862+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:39:41.884862+0000","last_peered":"2024-09-05T06:39:41.884862+0000","last_clean":"2024-09-05T06:39:41.884862+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:39:41.884862+0000","last_undegraded":"2024-09-05T06:39:41.884862+0000","last_fullsized":"2024-09-05T06:39:41.884862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928684+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:39:12.928684+0000","last_peered":"2024-09-05T06:39:12.928684+0000","last_clean":"2024-09-05T06:39:12.928684+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:39:12.928684+0000","last_undegraded":"2024-09-05T06:39:12.928684+0000","last_fullsized":"2024-09-05T06:39:12.928684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929530+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:39:12.929530+0000","last_peered":"2024-09-05T06:39:12.929530+0000","last_clean":"2024-09-05T06:39:12.929530+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:39:12.929530+0000","last_undegraded":"2024-09-05T06:39:12.929530+0000","last_fullsized":"2024-09-05T06:39:12.929530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:42.864712+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:39:42.864712+0000","last_peered":"2024-09-05T06:39:42.864712+0000","last_clean":"2024-09-05T06:39:42.864712+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:39:42.864712+0000","last_undegraded":"2024-09-05T06:39:42.864712+0000","last_fullsized":"2024-09-05T06:39:42.864712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929391+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:39:12.929391+0000","last_peered":"2024-09-05T06:39:12.929391+0000","last_clean":"2024-09-05T06:39:12.929391+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:39:12.929391+0000","last_undegraded":"2024-09-05T06:39:12.929391+0000","last_fullsized":"2024-09-05T06:39:12.929391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928004+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:39:12.928004+0000","last_peered":"2024-09-05T06:39:12.928004+0000","last_clean":"2024-09-05T06:39:12.928004+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:39:12.928004+0000","last_undegraded":"2024-09-05T06:39:12.928004+0000","last_fullsized":"2024-09-05T06:39:12.928004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:30.875833+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:39:30.875833+0000","last_peered":"2024-09-05T06:39:30.875833+0000","last_clean":"2024-09-05T06:39:30.875833+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:39:30.875833+0000","last_undegraded":"2024-09-05T06:39:30.875833+0000","last_fullsized":"2024-09-05T06:39:30.875833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931687+0000","last_change":"2024-09-05T06:39:02.962445+0000","last_active":"2024-09-05T06:39:12.931687+0000","last_peered":"2024-09-05T06:39:12.931687+0000","last_clean":"2024-09-05T06:39:12.931687+0000","last_became_active":"2024-09-05T06:39:02.961982+0000","last_became_peered":"2024-09-05T06:39:02.961982+0000","last_unstale":"2024-09-05T06:39:12.931687+0000","last_undegraded":"2024-09-05T06:39:12.931687+0000","last_fullsized":"2024-09-05T06:39:12.931687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:41:37.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928488+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:39:12.928488+0000","last_peered":"2024-09-05T06:39:12.928488+0000","last_clean":"2024-09-05T06:39:12.928488+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:39:12.928488+0000","last_undegraded":"2024-09-05T06:39:12.928488+0000","last_fullsized":"2024-09-05T06:39:12.928488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.858956+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:39:32.858956+0000","last_peered":"2024-09-05T06:39:32.858956+0000","last_clean":"2024-09-05T06:39:32.858956+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:39:32.858956+0000","last_undegraded":"2024-09-05T06:39:32.858956+0000","last_fullsized":"2024-09-05T06:39:32.858956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925164+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:39:12.925164+0000","last_peered":"2024-09-05T06:39:12.925164+0000","last_clean":"2024-09-05T06:39:12.925164+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:39:12.925164+0000","last_undegraded":"2024-09-05T06:39:12.925164+0000","last_fullsized":"2024-09-05T06:39:12.925164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924996+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:39:12.924996+0000","last_peered":"2024-09-05T06:39:12.924996+0000","last_clean":"2024-09-05T06:39:12.924996+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:39:12.924996+0000","last_undegraded":"2024-09-05T06:39:12.924996+0000","last_fullsized":"2024-09-05T06:39:12.924996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925065+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:39:12.925065+0000","last_peered":"2024-09-05T06:39:12.925065+0000","last_clean":"2024-09-05T06:39:12.925065+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:39:12.925065+0000","last_undegraded":"2024-09-05T06:39:12.925065+0000","last_fullsized":"2024-09-05T06:39:12.925065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:44.899422+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:39:44.899422+0000","last_peered":"2024-09-05T06:39:44.899422+0000","last_clean":"2024-09-05T06:39:44.899422+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:39:44.899422+0000","last_undegraded":"2024-09-05T06:39:44.899422+0000","last_fullsized":"2024-09-05T06:39:44.899422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931567+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:39:12.931567+0000","last_peered":"2024-09-05T06:39:12.931567+0000","last_clean":"2024-09-05T06:39:12.931567+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:39:12.931567+0000","last_undegraded":"2024-09-05T06:39:12.931567+0000","last_fullsized":"2024-09-05T06:39:12.931567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924990+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:39:12.924990+0000","last_peered":"2024-09-05T06:39:12.924990+0000","last_clean":"2024-09-05T06:39:12.924990+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:39:12.924990+0000","last_undegraded":"2024-09-05T06:39:12.924990+0000","last_fullsized":"2024-09-05T06:39:12.924990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854285+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:39:31.854285+0000","last_peered":"2024-09-05T06:39:31.854285+0000","last_clean":"2024-09-05T06:39:31.854285+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:39:31.854285+0000","last_undegraded":"2024-09-05T06:39:31.854285+0000","last_fullsized":"2024-09-05T06:39:31.854285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923807+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:39:12.923807+0000","last_peered":"2024-09-05T06:39:12.923807+0000","last_clean":"2024-09-05T06:39:12.923807+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:39:12.923807+0000","last_undegraded":"2024-09-05T06:39:12.923807+0000","last_fullsized":"2024-09-05T06:39:12.923807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925257+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:39:12.925257+0000","last_peered":"2024-09-05T06:39:12.925257+0000","last_clean":"2024-09-05T06:39:12.925257+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:39:12.925257+0000","last_undegraded":"2024-09-05T06:39:12.925257+0000","last_fullsized":"2024-09-05T06:39:12.925257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859107+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:39:32.859107+0000","last_peered":"2024-09-05T06:39:32.859107+0000","last_clean":"2024-09-05T06:39:32.859107+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:39:32.859107+0000","last_undegraded":"2024-09-05T06:39:32.859107+0000","last_fullsized":"2024-09-05T06:39:32.859107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929213+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:39:12.929213+0000","last_peered":"2024-09-05T06:39:12.929213+0000","last_clean":"2024-09-05T06:39:12.929213+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:39:12.929213+0000","last_undegraded":"2024-09-05T06:39:12.929213+0000","last_fullsized":"2024-09-05T06:39:12.929213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928351+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:39:12.928351+0000","last_peered":"2024-09-05T06:39:12.928351+0000","last_clean":"2024-09-05T06:39:12.928351+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:39:12.928351+0000","last_undegraded":"2024-09-05T06:39:12.928351+0000","last_fullsized":"2024-09-05T06:39:12.928351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.844687+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:39:31.844687+0000","last_peered":"2024-09-05T06:39:31.844687+0000","last_clean":"2024-09-05T06:39:31.844687+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:39:31.844687+0000","last_undegraded":"2024-09-05T06:39:31.844687+0000","last_fullsized":"2024-09-05T06:39:31.844687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930143+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:39:12.930143+0000","last_peered":"2024-09-05T06:39:12.930143+0000","last_clean":"2024-09-05T06:39:12.930143+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:39:12.930143+0000","last_undegraded":"2024-09-05T06:39:12.930143+0000","last_fullsized":"2024-09-05T06:39:12.930143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931025+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:39:12.931025+0000","last_peered":"2024-09-05T06:39:12.931025+0000","last_clean":"2024-09-05T06:39:12.931025+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:39:12.931025+0000","last_undegraded":"2024-09-05T06:39:12.931025+0000","last_fullsized":"2024-09-05T06:39:12.931025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854721+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:39:31.854721+0000","last_peered":"2024-09-05T06:39:31.854721+0000","last_clean":"2024-09-05T06:39:31.854721+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:39:31.854721+0000","last_undegraded":"2024-09-05T06:39:31.854721+0000","last_fullsized":"2024-09-05T06:39:31.854721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929035+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:39:12.929035+0000","last_peered":"2024-09-05T06:39:12.929035+0000","last_clean":"2024-09-05T06:39:12.929035+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:39:12.929035+0000","last_undegraded":"2024-09-05T06:39:12.929035+0000","last_fullsized":"2024-09-05T06:39:12.929035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924474+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:39:12.924474+0000","last_peered":"2024-09-05T06:39:12.924474+0000","last_clean":"2024-09-05T06:39:12.924474+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:39:12.924474+0000","last_undegraded":"2024-09-05T06:39:12.924474+0000","last_fullsized":"2024-09-05T06:39:12.924474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:44.868358+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:39:44.868358+0000","last_peered":"2024-09-05T06:39:44.868358+0000","last_clean":"2024-09-05T06:39:44.868358+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:39:44.868358+0000","last_undegraded":"2024-09-05T06:39:44.868358+0000","last_fullsized":"2024-09-05T06:39:44.868358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925797+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:39:12.925797+0000","last_peered":"2024-09-05T06:39:12.925797+0000","last_clean":"2024-09-05T06:39:12.925797+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:39:12.925797+0000","last_undegraded":"2024-09-05T06:39:12.925797+0000","last_fullsized":"2024-09-05T06:39:12.925797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924664+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:39:12.924664+0000","last_peered":"2024-09-05T06:39:12.924664+0000","last_clean":"2024-09-05T06:39:12.924664+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:39:12.924664+0000","last_undegraded":"2024-09-05T06:39:12.924664+0000","last_fullsized":"2024-09-05T06:39:12.924664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859070+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:39:32.859070+0000","last_peered":"2024-09-05T06:39:32.859070+0000","last_clean":"2024-09-05T06:39:32.859070+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:39:32.859070+0000","last_undegraded":"2024-09-05T06:39:32.859070+0000","last_fullsized":"2024-09-05T06:39:32.859070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928077+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:39:12.928077+0000","last_peered":"2024-09-05T06:39:12.928077+0000","last_clean":"2024-09-05T06:39:12.928077+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:39:12.928077+0000","last_undegraded":"2024-09-05T06:39:12.928077+0000","last_fullsized":"2024-09-05T06:39:12.928077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928477+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:39:12.928477+0000","last_peered":"2024-09-05T06:39:12.928477+0000","last_clean":"2024-09-05T06:39:12.928477+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:39:12.928477+0000","last_undegraded":"2024-09-05T06:39:12.928477+0000","last_fullsized":"2024-09-05T06:39:12.928477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.860569+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:39:32.860569+0000","last_peered":"2024-09-05T06:39:32.860569+0000","last_clean":"2024-09-05T06:39:32.860569+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:39:32.860569+0000","last_undegraded":"2024-09-05T06:39:32.860569+0000","last_fullsized":"2024-09-05T06:39:32.860569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931370+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:39:12.931370+0000","last_peered":"2024-09-05T06:39:12.931370+0000","last_clean":"2024-09-05T06:39:12.931370+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:39:12.931370+0000","last_undegraded":"2024-09-05T06:39:12.931370+0000","last_fullsized":"2024-09-05T06:39:12.931370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924799+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:39:12.924799+0000","last_peered":"2024-09-05T06:39:12.924799+0000","last_clean":"2024-09-05T06:39:12.924799+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:39:12.924799+0000","last_undegraded":"2024-09-05T06:39:12.924799+0000","last_fullsized":"2024-09-05T06:39:12.924799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.870686+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:39:43.870686+0000","last_peered":"2024-09-05T06:39:43.870686+0000","last_clean":"2024-09-05T06:39:43.870686+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:39:43.870686+0000","last_undegraded":"2024-09-05T06:39:43.870686+0000","last_fullsized":"2024-09-05T06:39:43.870686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":58,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925363+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:39:12.925363+0000","last_peered":"2024-09-05T06:39:12.925363+0000","last_clean":"2024-09-05T06:39:12.925363+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:39:12.925363+0000","last_undegraded":"2024-09-05T06:39:12.925363+0000","last_fullsized":"2024-09-05T06:39:12.925363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932416+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:39:12.932416+0000","last_peered":"2024-09-05T06:39:12.932416+0000","last_clean":"2024-09-05T06:39:12.932416+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:39:12.932416+0000","last_undegraded":"2024-09-05T06:39:12.932416+0000","last_fullsized":"2024-09-05T06:39:12.932416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.847959+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:39:31.847959+0000","last_peered":"2024-09-05T06:39:31.847959+0000","last_clean":"2024-09-05T06:39:31.847959+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:39:31.847959+0000","last_undegraded":"2024-09-05T06:39:31.847959+0000","last_fullsized":"2024-09-05T06:39:31.847959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931541+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:39:12.931541+0000","last_peered":"2024-09-05T06:39:12.931541+0000","last_clean":"2024-09-05T06:39:12.931541+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:39:12.931541+0000","last_undegraded":"2024-09-05T06:39:12.931541+0000","last_fullsized":"2024-09-05T06:39:12.931541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931561+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:39:12.931561+0000","last_peered":"2024-09-05T06:39:12.931561+0000","last_clean":"2024-09-05T06:39:12.931561+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:39:12.931561+0000","last_undegraded":"2024-09-05T06:39:12.931561+0000","last_fullsized":"2024-09-05T06:39:12.931561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859600+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:39:32.859600+0000","last_peered":"2024-09-05T06:39:32.859600+0000","last_clean":"2024-09-05T06:39:32.859600+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:39:32.859600+0000","last_undegraded":"2024-09-05T06:39:32.859600+0000","last_fullsized":"2024-09-05T06:39:32.859600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925123+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:39:12.925123+0000","last_peered":"2024-09-05T06:39:12.925123+0000","last_clean":"2024-09-05T06:39:12.925123+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:39:12.925123+0000","last_undegraded":"2024-09-05T06:39:12.925123+0000","last_fullsized":"2024-09-05T06:39:12.925123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.933049+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:39:12.933049+0000","last_peered":"2024-09-05T06:39:12.933049+0000","last_clean":"2024-09-05T06:39:12.933049+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:39:12.933049+0000","last_undegraded":"2024-09-05T06:39:12.933049+0000","last_fullsized":"2024-09-05T06:39:12.933049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.865960+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:39:43.865960+0000","last_peered":"2024-09-05T06:39:43.865960+0000","last_clean":"2024-09-05T06:39:43.865960+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:39:43.865960+0000","last_undegraded":"2024-09-05T06:39:43.865960+0000","last_fullsized":"2024-09-05T06:39:43.865960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927699+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:39:12.927699+0000","last_peered":"2024-09-05T06:39:12.927699+0000","last_clean":"2024-09-05T06:39:12.927699+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:39:12.927699+0000","last_undegraded":"2024-09-05T06:39:12.927699+0000","last_fullsized":"2024-09-05T06:39:12.927699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923934+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:39:12.923934+0000","last_peered":"2024-09-05T06:39:12.923934+0000","last_clean":"2024-09-05T06:39:12.923934+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:39:12.923934+0000","last_undegraded":"2024-09-05T06:39:12.923934+0000","last_fullsized":"2024-09-05T06:39:12.923934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925236+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:39:12.925236+0000","last_peered":"2024-09-05T06:39:12.925236+0000","last_clean":"2024-09-05T06:39:12.925236+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:39:12.925236+0000","last_undegraded":"2024-09-05T06:39:12.925236+0000","last_fullsized":"2024-09-05T06:39:12.925236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859600+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:39:32.859600+0000","last_peered":"2024-09-05T06:39:32.859600+0000","last_clean":"2024-09-05T06:39:32.859600+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:39:32.859600+0000","last_undegraded":"2024-09-05T06:39:32.859600+0000","last_fullsized":"2024-09-05T06:39:32.859600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929091+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:39:12.929091+0000","last_peered":"2024-09-05T06:39:12.929091+0000","last_clean":"2024-09-05T06:39:12.929091+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:39:12.929091+0000","last_undegraded":"2024-09-05T06:39:12.929091+0000","last_fullsized":"2024-09-05T06:39:12.929091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928436+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:39:12.928436+0000","last_peered":"2024-09-05T06:39:12.928436+0000","last_clean":"2024-09-05T06:39:12.928436+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:39:12.928436+0000","last_undegraded":"2024-09-05T06:39:12.928436+0000","last_fullsized":"2024-09-05T06:39:12.928436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:30.875871+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:39:30.875871+0000","last_peered":"2024-09-05T06:39:30.875871+0000","last_clean":"2024-09-05T06:39:30.875871+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:39:30.875871+0000","last_undegraded":"2024-09-05T06:39:30.875871+0000","last_fullsized":"2024-09-05T06:39:30.875871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854015+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:39:31.854015+0000","last_peered":"2024-09-05T06:39:31.854015+0000","last_clean":"2024-09-05T06:39:31.854015+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:39:31.854015+0000","last_undegraded":"2024-09-05T06:39:31.854015+0000","last_fullsized":"2024-09-05T06:39:31.854015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932497+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:39:12.932497+0000","last_peered":"2024-09-05T06:39:12.932497+0000","last_clean":"2024-09-05T06:39:12.932497+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:39:12.932497+0000","last_undegraded":"2024-09-05T06:39:12.932497+0000","last_fullsized":"2024-09-05T06:39:12.932497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928650+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:39:12.928650+0000","last_peered":"2024-09-05T06:39:12.928650+0000","last_clean":"2024-09-05T06:39:12.928650+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:39:12.928650+0000","last_undegraded":"2024-09-05T06:39:12.928650+0000","last_fullsized":"2024-09-05T06:39:12.928650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:42.865209+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:39:42.865209+0000","last_peered":"2024-09-05T06:39:42.865209+0000","last_clean":"2024-09-05T06:39:42.865209+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:39:42.865209+0000","last_undegraded":"2024-09-05T06:39:42.865209+0000","last_fullsized":"2024-09-05T06:39:42.865209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924487+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:39:12.924487+0000","last_peered":"2024-09-05T06:39:12.924487+0000","last_clean":"2024-09-05T06:39:12.924487+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:39:12.924487+0000","last_undegraded":"2024-09-05T06:39:12.924487+0000","last_fullsized":"2024-09-05T06:39:12.924487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929786+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:39:12.929786+0000","last_peered":"2024-09-05T06:39:12.929786+0000","last_clean":"2024-09-05T06:39:12.929786+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:39:12.929786+0000","last_undegraded":"2024-09-05T06:39:12.929786+0000","last_fullsized":"2024-09-05T06:39:12.929786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927704+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:39:12.927704+0000","last_peered":"2024-09-05T06:39:12.927704+0000","last_clean":"2024-09-05T06:39:12.927704+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:39:12.927704+0000","last_undegraded":"2024-09-05T06:39:12.927704+0000","last_fullsized":"2024-09-05T06:39:12.927704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.933654+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:39:12.933654+0000","last_peered":"2024-09-05T06:39:12.933654+0000","last_clean":"2024-09-05T06:39:12.933654+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:39:12.933654+0000","last_undegraded":"2024-09-05T06:39:12.933654+0000","last_fullsized":"2024-09-05T06:39:12.933654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928391+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:39:12.928391+0000","last_peered":"2024-09-05T06:39:12.928391+0000","last_clean":"2024-09-05T06:39:12.928391+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:39:12.928391+0000","last_undegraded":"2024-09-05T06:39:12.928391+0000","last_fullsized":"2024-09-05T06:39:12.928391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923839+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:39:12.923839+0000","last_peered":"2024-09-05T06:39:12.923839+0000","last_clean":"2024-09-05T06:39:12.923839+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:39:12.923839+0000","last_undegraded":"2024-09-05T06:39:12.923839+0000","last_fullsized":"2024-09-05T06:39:12.923839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928825+0000","last_change":"2024-09-05T06:38:55.376697+0000","last_active":"2024-09-05T06:39:12.928825+0000","last_peered":"2024-09-05T06:39:12.928825+0000","last_clean":"2024-09-05T06:39:12.928825+0000","last_became_active":"2024-09-05T06:38:55.376401+0000","last_became_peered":"2024-09-05T06:38:55.376401+0000","last_unstale":"2024-09-05T06:39:12.928825+0000","last_undegraded":"2024-09-05T06:39:12.928825+0000","last_fullsized":"2024-09-05T06:39:12.928825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:41:31.881708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928614+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:39:12.928614+0000","last_peered":"2024-09-05T06:39:12.928614+0000","last_clean":"2024-09-05T06:39:12.928614+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:39:12.928614+0000","last_undegraded":"2024-09-05T06:39:12.928614+0000","last_fullsized":"2024-09-05T06:39:12.928614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930228+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:39:12.930228+0000","last_peered":"2024-09-05T06:39:12.930228+0000","last_clean":"2024-09-05T06:39:12.930228+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:39:12.930228+0000","last_undegraded":"2024-09-05T06:39:12.930228+0000","last_fullsized":"2024-09-05T06:39:12.930228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928203+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:39:12.928203+0000","last_peered":"2024-09-05T06:39:12.928203+0000","last_clean":"2024-09-05T06:39:12.928203+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:39:12.928203+0000","last_undegraded":"2024-09-05T06:39:12.928203+0000","last_fullsized":"2024-09-05T06:39:12.928203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929428+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:39:12.929428+0000","last_peered":"2024-09-05T06:39:12.929428+0000","last_clean":"2024-09-05T06:39:12.929428+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:39:12.929428+0000","last_undegraded":"2024-09-05T06:39:12.929428+0000","last_fullsized":"2024-09-05T06:39:12.929428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927942+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:39:12.927942+0000","last_peered":"2024-09-05T06:39:12.927942+0000","last_clean":"2024-09-05T06:39:12.927942+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:39:12.927942+0000","last_undegraded":"2024-09-05T06:39:12.927942+0000","last_fullsized":"2024-09-05T06:39:12.927942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932168+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:39:12.932168+0000","last_peered":"2024-09-05T06:39:12.932168+0000","last_clean":"2024-09-05T06:39:12.932168+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:39:12.932168+0000","last_undegraded":"2024-09-05T06:39:12.932168+0000","last_fullsized":"2024-09-05T06:39:12.932168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928920+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:39:12.928920+0000","last_peered":"2024-09-05T06:39:12.928920+0000","last_clean":"2024-09-05T06:39:12.928920+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:39:12.928920+0000","last_undegraded":"2024-09-05T06:39:12.928920+0000","last_fullsized":"2024-09-05T06:39:12.928920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928196+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:39:12.928196+0000","last_peered":"2024-09-05T06:39:12.928196+0000","last_clean":"2024-09-05T06:39:12.928196+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:39:12.928196+0000","last_undegraded":"2024-09-05T06:39:12.928196+0000","last_fullsized":"2024-09-05T06:39:12.928196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929955+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:39:12.929955+0000","last_peered":"2024-09-05T06:39:12.929955+0000","last_clean":"2024-09-05T06:39:12.929955+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:39:12.929955+0000","last_undegraded":"2024-09-05T06:39:12.929955+0000","last_fullsized":"2024-09-05T06:39:12.929955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925729+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:39:12.925729+0000","last_peered":"2024-09-05T06:39:12.925729+0000","last_clean":"2024-09-05T06:39:12.925729+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:39:12.925729+0000","last_undegraded":"2024-09-05T06:39:12.925729+0000","last_fullsized":"2024-09-05T06:39:12.925729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929506+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:39:12.929506+0000","last_peered":"2024-09-05T06:39:12.929506+0000","last_clean":"2024-09-05T06:39:12.929506+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:39:12.929506+0000","last_undegraded":"2024-09-05T06:39:12.929506+0000","last_fullsized":"2024-09-05T06:39:12.929506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932041+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:39:12.932041+0000","last_peered":"2024-09-05T06:39:12.932041+0000","last_clean":"2024-09-05T06:39:12.932041+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:39:12.932041+0000","last_undegraded":"2024-09-05T06:39:12.932041+0000","last_fullsized":"2024-09-05T06:39:12.932041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923572+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:39:12.923572+0000","last_peered":"2024-09-05T06:39:12.923572+0000","last_clean":"2024-09-05T06:39:12.923572+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:39:12.923572+0000","last_undegraded":"2024-09-05T06:39:12.923572+0000","last_fullsized":"2024-09-05T06:39:12.923572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923620+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:39:12.923620+0000","last_peered":"2024-09-05T06:39:12.923620+0000","last_clean":"2024-09-05T06:39:12.923620+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:39:12.923620+0000","last_undegraded":"2024-09-05T06:39:12.923620+0000","last_fullsized":"2024-09-05T06:39:12.923620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928843+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:39:12.928843+0000","last_peered":"2024-09-05T06:39:12.928843+0000","last_clean":"2024-09-05T06:39:12.928843+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:39:12.928843+0000","last_undegraded":"2024-09-05T06:39:12.928843+0000","last_fullsized":"2024-09-05T06:39:12.928843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932178+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:39:12.932178+0000","last_peered":"2024-09-05T06:39:12.932178+0000","last_clean":"2024-09-05T06:39:12.932178+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:39:12.932178+0000","last_undegraded":"2024-09-05T06:39:12.932178+0000","last_fullsized":"2024-09-05T06:39:12.932178+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925123+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:39:12.925123+0000","last_peered":"2024-09-05T06:39:12.925123+0000","last_clean":"2024-09-05T06:39:12.925123+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:39:12.925123+0000","last_undegraded":"2024-09-05T06:39:12.925123+0000","last_fullsized":"2024-09-05T06:39:12.925123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928612+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:39:12.928612+0000","last_peered":"2024-09-05T06:39:12.928612+0000","last_clean":"2024-09-05T06:39:12.928612+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:39:12.928612+0000","last_undegraded":"2024-09-05T06:39:12.928612+0000","last_fullsized":"2024-09-05T06:39:12.928612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931839+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:39:12.931839+0000","last_peered":"2024-09-05T06:39:12.931839+0000","last_clean":"2024-09-05T06:39:12.931839+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:39:12.931839+0000","last_undegraded":"2024-09-05T06:39:12.931839+0000","last_fullsized":"2024-09-05T06:39:12.931839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928359+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:39:12.928359+0000","last_peered":"2024-09-05T06:39:12.928359+0000","last_clean":"2024-09-05T06:39:12.928359+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:39:12.928359+0000","last_undegraded":"2024-09-05T06:39:12.928359+0000","last_fullsized":"2024-09-05T06:39:12.928359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924609+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:39:12.924609+0000","last_peered":"2024-09-05T06:39:12.924609+0000","last_clean":"2024-09-05T06:39:12.924609+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:39:12.924609+0000","last_undegraded":"2024-09-05T06:39:12.924609+0000","last_fullsized":"2024-09-05T06:39:12.924609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927750+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:39:12.927750+0000","last_peered":"2024-09-05T06:39:12.927750+0000","last_clean":"2024-09-05T06:39:12.927750+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:39:12.927750+0000","last_undegraded":"2024-09-05T06:39:12.927750+0000","last_fullsized":"2024-09-05T06:39:12.927750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924470+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:39:12.924470+0000","last_peered":"2024-09-05T06:39:12.924470+0000","last_clean":"2024-09-05T06:39:12.924470+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:39:12.924470+0000","last_undegraded":"2024-09-05T06:39:12.924470+0000","last_fullsized":"2024-09-05T06:39:12.924470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932050+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:39:12.932050+0000","last_peered":"2024-09-05T06:39:12.932050+0000","last_clean":"2024-09-05T06:39:12.932050+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:39:12.932050+0000","last_undegraded":"2024-09-05T06:39:12.932050+0000","last_fullsized":"2024-09-05T06:39:12.932050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923679+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:39:12.923679+0000","last_peered":"2024-09-05T06:39:12.923679+0000","last_clean":"2024-09-05T06:39:12.923679+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:39:12.923679+0000","last_undegraded":"2024-09-05T06:39:12.923679+0000","last_fullsized":"2024-09-05T06:39:12.923679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923827+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:39:12.923827+0000","last_peered":"2024-09-05T06:39:12.923827+0000","last_clean":"2024-09-05T06:39:12.923827+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:39:12.923827+0000","last_undegraded":"2024-09-05T06:39:12.923827+0000","last_fullsized":"2024-09-05T06:39:12.923827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929407+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:39:12.929407+0000","last_peered":"2024-09-05T06:39:12.929407+0000","last_clean":"2024-09-05T06:39:12.929407+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:39:12.929407+0000","last_undegraded":"2024-09-05T06:39:12.929407+0000","last_fullsized":"2024-09-05T06:39:12.929407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50553,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":163,"num_read_kb":139,"num_write":232,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":166975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612631,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":491567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1122099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":491831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462954,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1088745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626480,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":486979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855412,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1083934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":550107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":488622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":488435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:39:48.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:39:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:48.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:48 smithi086 ceph-mon[30943]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-05T06:39:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:48 smithi096 ceph-mon[34468]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-05T06:39:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:48 smithi115 ceph-mon[34791]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-05T06:39:48.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.900+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/1427263789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 msgr2=0x7f4a9c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.900+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/1427263789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c103110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009cd0 tx=0x7f4a9402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/1427263789 shutdown_connections 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/1427263789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c1082a0 0x7f4a9c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/1427263789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 0x7f4a9c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/1427263789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c103110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/1427263789 >> 172.21.15.86:0/1427263789 conn(0x7f4a9c0fc7d0 msgr2=0x7f4a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/1427263789 shutdown_connections 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.901+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/1427263789 wait complete. 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 Processor -- start 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 -- start start 2024-09-05T06:39:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 0x7f4a9c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c1082a0 0x7f4a9c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c111fb0 con 0x7f4a9c102d30 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c111e30 con 0x7f4a9c1036e0 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4aa0aa5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112130 con 0x7f4a9c1082a0 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50002/0 (socket says 172.21.15.86:50002) 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 -- 172.21.15.86:0/691016715 learned_addr learned my addr 172.21.15.86:0/691016715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a99d9b640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 0x7f4a9c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 -- 172.21.15.86:0/691016715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c1082a0 msgr2=0x7f4a9c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c1082a0 0x7f4a9c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 -- 172.21.15.86:0/691016715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 msgr2=0x7f4a9c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 0x7f4a9c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 -- 172.21.15.86:0/691016715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c19d460 con 0x7f4a9c102d30 2024-09-05T06:39:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.902+0000 7f4a9a59c640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006010 tx=0x7f4a94003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.903+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004450 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.903+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a940045f0 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.903+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c19d6f0 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.903+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a940466e0 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.903+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a5030 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.904+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c103110 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.905+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a940305a0 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.906+0000 7f4a937fe640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 0x7f4a880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.906+0000 7f4a99d9b640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 0x7f4a880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.906+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f4a9402e030 con 0x7f4a9c102d30 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.906+0000 7f4a99d9b640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 0x7f4a880a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0fedc0 tx=0x7f4a8400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:48.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:48.910+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a9404b060 con 0x7f4a9c102d30 2024-09-05T06:39:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.072+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a9c10a300 con 0x7f4a880a2c10 2024-09-05T06:39:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.081+0000 7f4a937fe640 1 -- 172.21.15.86:0/691016715 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340386 (secure 0 0 0) 0x7f4a9c10a300 con 0x7f4a880a2c10 2024-09-05T06:39:49.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:49.086 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 msgr2=0x7f4a880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 0x7f4a880a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0fedc0 tx=0x7f4a8400a2d0 comp rx=0 tx=0).stop 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 msgr2=0x7f4a9c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006010 tx=0x7f4a94003c70 comp rx=0 tx=0).stop 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 shutdown_connections 2024-09-05T06:39:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c1082a0 0x7f4a9c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4a880a2c10 0x7f4a880a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a9c1036e0 0x7f4a9c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 --2- 172.21.15.86:0/691016715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c102d30 0x7f4a9c1984b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.084+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 >> 172.21.15.86:0/691016715 conn(0x7f4a9c0fc7d0 msgr2=0x7f4a9c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.085+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 shutdown_connections 2024-09-05T06:39:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:49.085+0000 7f4aa0aa5640 1 -- 172.21.15.86:0/691016715 wait complete. 2024-09-05T06:39:49.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:49 smithi086 ceph-mon[30943]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:49 smithi096 ceph-mon[34468]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:49 smithi115 ceph-mon[34791]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-05T06:39:47.194302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641155,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":299,"num_read_kb":258,"num_write":415,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388020,"kb_used_data":13812,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553804,"statfs":{"total":1151940427776,"available":1151543095296,"internally_reserved":0,"allocated":14143488,"data_stored":7805931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41289,"num_objects":21,"num_object_clones":0,"num_object_copies":65,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":133,"num_read_kb":112,"num_write":66,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.887045+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:39:43.887045+0000","last_peered":"2024-09-05T06:39:43.887045+0000","last_clean":"2024-09-05T06:39:43.887045+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:39:43.887045+0000","last_undegraded":"2024-09-05T06:39:43.887045+0000","last_fullsized":"2024-09-05T06:39:43.887045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927748+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:39:12.927748+0000","last_peered":"2024-09-05T06:39:12.927748+0000","last_clean":"2024-09-05T06:39:12.927748+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:39:12.927748+0000","last_undegraded":"2024-09-05T06:39:12.927748+0000","last_fullsized":"2024-09-05T06:39:12.927748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928430+0000","last_change":"2024-09-05T06:39:02.873883+0000","last_active":"2024-09-05T06:39:12.928430+0000","last_peered":"2024-09-05T06:39:12.928430+0000","last_clean":"2024-09-05T06:39:12.928430+0000","last_became_active":"2024-09-05T06:39:02.873471+0000","last_became_peered":"2024-09-05T06:39:02.873471+0000","last_unstale":"2024-09-05T06:39:12.928430+0000","last_undegraded":"2024-09-05T06:39:12.928430+0000","last_fullsized":"2024-09-05T06:39:12.928430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:54:30.025583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928804+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:39:12.928804+0000","last_peered":"2024-09-05T06:39:12.928804+0000","last_clean":"2024-09-05T06:39:12.928804+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:39:12.928804+0000","last_undegraded":"2024-09-05T06:39:12.928804+0000","last_fullsized":"2024-09-05T06:39:12.928804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924103+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:39:12.924103+0000","last_peered":"2024-09-05T06:39:12.924103+0000","last_clean":"2024-09-05T06:39:12.924103+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:39:12.924103+0000","last_undegraded":"2024-09-05T06:39:12.924103+0000","last_fullsized":"2024-09-05T06:39:12.924103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929854+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:39:12.929854+0000","last_peered":"2024-09-05T06:39:12.929854+0000","last_clean":"2024-09-05T06:39:12.929854+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:39:12.929854+0000","last_undegraded":"2024-09-05T06:39:12.929854+0000","last_fullsized":"2024-09-05T06:39:12.929854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:33.871541+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:39:33.871541+0000","last_peered":"2024-09-05T06:39:33.871541+0000","last_clean":"2024-09-05T06:39:33.871541+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:39:33.871541+0000","last_undegraded":"2024-09-05T06:39:33.871541+0000","last_fullsized":"2024-09-05T06:39:33.871541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932308+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:39:12.932308+0000","last_peered":"2024-09-05T06:39:12.932308+0000","last_clean":"2024-09-05T06:39:12.932308+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:39:12.932308+0000","last_undegraded":"2024-09-05T06:39:12.932308+0000","last_fullsized":"2024-09-05T06:39:12.932308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930662+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:39:12.930662+0000","last_peered":"2024-09-05T06:39:12.930662+0000","last_clean":"2024-09-05T06:39:12.930662+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:39:12.930662+0000","last_undegraded":"2024-09-05T06:39:12.930662+0000","last_fullsized":"2024-09-05T06:39:12.930662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.860355+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:39:32.860355+0000","last_peered":"2024-09-05T06:39:32.860355+0000","last_clean":"2024-09-05T06:39:32.860355+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:39:32.860355+0000","last_undegraded":"2024-09-05T06:39:32.860355+0000","last_fullsized":"2024-09-05T06:39:32.860355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931310+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:39:12.931310+0000","last_peered":"2024-09-05T06:39:12.931310+0000","last_clean":"2024-09-05T06:39:12.931310+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:39:12.931310+0000","last_undegraded":"2024-09-05T06:39:12.931310+0000","last_fullsized":"2024-09-05T06:39:12.931310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930102+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:39:12.930102+0000","last_peered":"2024-09-05T06:39:12.930102+0000","last_clean":"2024-09-05T06:39:12.930102+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:39:12.930102+0000","last_undegraded":"2024-09-05T06:39:12.930102+0000","last_fullsized":"2024-09-05T06:39:12.930102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924768+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:39:12.924768+0000","last_peered":"2024-09-05T06:39:12.924768+0000","last_clean":"2024-09-05T06:39:12.924768+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:39:12.924768+0000","last_undegraded":"2024-09-05T06:39:12.924768+0000","last_fullsized":"2024-09-05T06:39:12.924768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:29.869845+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:39:29.869845+0000","last_peered":"2024-09-05T06:39:29.869845+0000","last_clean":"2024-09-05T06:39:29.869845+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:39:29.869845+0000","last_undegraded":"2024-09-05T06:39:29.869845+0000","last_fullsized":"2024-09-05T06:39:29.869845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930481+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:39:12.930481+0000","last_peered":"2024-09-05T06:39:12.930481+0000","last_clean":"2024-09-05T06:39:12.930481+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:39:12.930481+0000","last_undegraded":"2024-09-05T06:39:12.930481+0000","last_fullsized":"2024-09-05T06:39:12.930481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930743+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:39:12.930743+0000","last_peered":"2024-09-05T06:39:12.930743+0000","last_clean":"2024-09-05T06:39:12.930743+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:39:12.930743+0000","last_undegraded":"2024-09-05T06:39:12.930743+0000","last_fullsized":"2024-09-05T06:39:12.930743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854799+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:39:31.854799+0000","last_peered":"2024-09-05T06:39:31.854799+0000","last_clean":"2024-09-05T06:39:31.854799+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:39:31.854799+0000","last_undegraded":"2024-09-05T06:39:31.854799+0000","last_fullsized":"2024-09-05T06:39:31.854799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931985+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:39:12.931985+0000","last_peered":"2024-09-05T06:39:12.931985+0000","last_clean":"2024-09-05T06:39:12.931985+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:39:12.931985+0000","last_undegraded":"2024-09-05T06:39:12.931985+0000","last_fullsized":"2024-09-05T06:39:12.931985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927766+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:39:12.927766+0000","last_peered":"2024-09-05T06:39:12.927766+0000","last_clean":"2024-09-05T06:39:12.927766+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:39:12.927766+0000","last_undegraded":"2024-09-05T06:39:12.927766+0000","last_fullsized":"2024-09-05T06:39:12.927766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859651+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:39:32.859651+0000","last_peered":"2024-09-05T06:39:32.859651+0000","last_clean":"2024-09-05T06:39:32.859651+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:39:32.859651+0000","last_undegraded":"2024-09-05T06:39:32.859651+0000","last_fullsized":"2024-09-05T06:39:32.859651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925738+0000","last_change":"2024-09-05T06:39:02.872883+0000","last_active":"2024-09-05T06:39:12.925738+0000","last_peered":"2024-09-05T06:39:12.925738+0000","last_clean":"2024-09-05T06:39:12.925738+0000","last_became_active":"2024-09-05T06:39:02.872523+0000","last_became_peered":"2024-09-05T06:39:02.872523+0000","last_unstale":"2024-09-05T06:39:12.925738+0000","last_undegraded":"2024-09-05T06:39:12.925738+0000","last_fullsized":"2024-09-05T06:39:12.925738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:51:46.797271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924420+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:39:12.924420+0000","last_peered":"2024-09-05T06:39:12.924420+0000","last_clean":"2024-09-05T06:39:12.924420+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:39:12.924420+0000","last_undegraded":"2024-09-05T06:39:12.924420+0000","last_fullsized":"2024-09-05T06:39:12.924420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:29.869008+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:39:29.869008+0000","last_peered":"2024-09-05T06:39:29.869008+0000","last_clean":"2024-09-05T06:39:29.869008+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:39:29.869008+0000","last_undegraded":"2024-09-05T06:39:29.869008+0000","last_fullsized":"2024-09-05T06:39:29.869008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930518+0000","last_change":"2024-09-05T06:39:02.962341+0000","last_active":"2024-09-05T06:39:12.930518+0000","last_peered":"2024-09-05T06:39:12.930518+0000","last_clean":"2024-09-05T06:39:12.930518+0000","last_became_active":"2024-09-05T06:39:02.961849+0000","last_became_peered":"2024-09-05T06:39:02.961849+0000","last_unstale":"2024-09-05T06:39:12.930518+0000","last_undegraded":"2024-09-05T06:39:12.930518+0000","last_fullsized":"2024-09-05T06:39:12.930518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:54:11.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930785+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:39:12.930785+0000","last_peered":"2024-09-05T06:39:12.930785+0000","last_clean":"2024-09-05T06:39:12.930785+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:39:12.930785+0000","last_undegraded":"2024-09-05T06:39:12.930785+0000","last_fullsized":"2024-09-05T06:39:12.930785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.858514+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:39:32.858514+0000","last_peered":"2024-09-05T06:39:32.858514+0000","last_clean":"2024-09-05T06:39:32.858514+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:39:32.858514+0000","last_undegraded":"2024-09-05T06:39:32.858514+0000","last_fullsized":"2024-09-05T06:39:32.858514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928546+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:39:12.928546+0000","last_peered":"2024-09-05T06:39:12.928546+0000","last_clean":"2024-09-05T06:39:12.928546+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:39:12.928546+0000","last_undegraded":"2024-09-05T06:39:12.928546+0000","last_fullsized":"2024-09-05T06:39:12.928546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928319+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:39:12.928319+0000","last_peered":"2024-09-05T06:39:12.928319+0000","last_clean":"2024-09-05T06:39:12.928319+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:39:12.928319+0000","last_undegraded":"2024-09-05T06:39:12.928319+0000","last_fullsized":"2024-09-05T06:39:12.928319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:41.378348+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:39:41.378348+0000","last_peered":"2024-09-05T06:39:41.378348+0000","last_clean":"2024-09-05T06:39:41.378348+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:39:41.378348+0000","last_undegraded":"2024-09-05T06:39:41.378348+0000","last_fullsized":"2024-09-05T06:39:41.378348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928200+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:39:12.928200+0000","last_peered":"2024-09-05T06:39:12.928200+0000","last_clean":"2024-09-05T06:39:12.928200+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:39:12.928200+0000","last_undegraded":"2024-09-05T06:39:12.928200+0000","last_fullsized":"2024-09-05T06:39:12.928200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928906+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:39:12.928906+0000","last_peered":"2024-09-05T06:39:12.928906+0000","last_clean":"2024-09-05T06:39:12.928906+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:39:12.928906+0000","last_undegraded":"2024-09-05T06:39:12.928906+0000","last_fullsized":"2024-09-05T06:39:12.928906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854092+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:39:31.854092+0000","last_peered":"2024-09-05T06:39:31.854092+0000","last_clean":"2024-09-05T06:39:31.854092+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:39:31.854092+0000","last_undegraded":"2024-09-05T06:39:31.854092+0000","last_fullsized":"2024-09-05T06:39:31.854092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928066+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:39:12.928066+0000","last_peered":"2024-09-05T06:39:12.928066+0000","last_clean":"2024-09-05T06:39:12.928066+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:39:12.928066+0000","last_undegraded":"2024-09-05T06:39:12.928066+0000","last_fullsized":"2024-09-05T06:39:12.928066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927876+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:39:12.927876+0000","last_peered":"2024-09-05T06:39:12.927876+0000","last_clean":"2024-09-05T06:39:12.927876+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:39:12.927876+0000","last_undegraded":"2024-09-05T06:39:12.927876+0000","last_fullsized":"2024-09-05T06:39:12.927876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925688+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:39:12.925688+0000","last_peered":"2024-09-05T06:39:12.925688+0000","last_clean":"2024-09-05T06:39:12.925688+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:39:12.925688+0000","last_undegraded":"2024-09-05T06:39:12.925688+0000","last_fullsized":"2024-09-05T06:39:12.925688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924413+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:39:12.924413+0000","last_peered":"2024-09-05T06:39:12.924413+0000","last_clean":"2024-09-05T06:39:12.924413+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:39:12.924413+0000","last_undegraded":"2024-09-05T06:39:12.924413+0000","last_fullsized":"2024-09-05T06:39:12.924413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928021+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:39:12.928021+0000","last_peered":"2024-09-05T06:39:12.928021+0000","last_clean":"2024-09-05T06:39:12.928021+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:39:12.928021+0000","last_undegraded":"2024-09-05T06:39:12.928021+0000","last_fullsized":"2024-09-05T06:39:12.928021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:41.884862+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:39:41.884862+0000","last_peered":"2024-09-05T06:39:41.884862+0000","last_clean":"2024-09-05T06:39:41.884862+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:39:41.884862+0000","last_undegraded":"2024-09-05T06:39:41.884862+0000","last_fullsized":"2024-09-05T06:39:41.884862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928684+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:39:12.928684+0000","last_peered":"2024-09-05T06:39:12.928684+0000","last_clean":"2024-09-05T06:39:12.928684+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:39:12.928684+0000","last_undegraded":"2024-09-05T06:39:12.928684+0000","last_fullsized":"2024-09-05T06:39:12.928684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929530+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:39:12.929530+0000","last_peered":"2024-09-05T06:39:12.929530+0000","last_clean":"2024-09-05T06:39:12.929530+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:39:12.929530+0000","last_undegraded":"2024-09-05T06:39:12.929530+0000","last_fullsized":"2024-09-05T06:39:12.929530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:42.864712+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:39:42.864712+0000","last_peered":"2024-09-05T06:39:42.864712+0000","last_clean":"2024-09-05T06:39:42.864712+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:39:42.864712+0000","last_undegraded":"2024-09-05T06:39:42.864712+0000","last_fullsized":"2024-09-05T06:39:42.864712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929391+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:39:12.929391+0000","last_peered":"2024-09-05T06:39:12.929391+0000","last_clean":"2024-09-05T06:39:12.929391+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:39:12.929391+0000","last_undegraded":"2024-09-05T06:39:12.929391+0000","last_fullsized":"2024-09-05T06:39:12.929391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928004+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:39:12.928004+0000","last_peered":"2024-09-05T06:39:12.928004+0000","last_clean":"2024-09-05T06:39:12.928004+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:39:12.928004+0000","last_undegraded":"2024-09-05T06:39:12.928004+0000","last_fullsized":"2024-09-05T06:39:12.928004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:30.875833+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:39:30.875833+0000","last_peered":"2024-09-05T06:39:30.875833+0000","last_clean":"2024-09-05T06:39:30.875833+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:39:30.875833+0000","last_undegraded":"2024-09-05T06:39:30.875833+0000","last_fullsized":"2024-09-05T06:39:30.875833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931687+0000","last_change":"2024-09-05T06:39:02.962445+0000","last_active":"2024-09-05T06:39:12.931687+0000","last_peered":"2024-09-05T06:39:12.931687+0000","last_clean":"2024-09-05T06:39:12.931687+0000","last_became_active":"2024-09-05T06:39:02.961982+0000","last_became_peered":"2024-09-05T06:39:02.961982+0000","last_unstale":"2024-09-05T06:39:12.931687+0000","last_undegraded":"2024-09-05T06:39:12.931687+0000","last_fullsized":"2024-09-05T06:39:12.931687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:41:37.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928488+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:39:12.928488+0000","last_peered":"2024-09-05T06:39:12.928488+0000","last_clean":"2024-09-05T06:39:12.928488+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:39:12.928488+0000","last_undegraded":"2024-09-05T06:39:12.928488+0000","last_fullsized":"2024-09-05T06:39:12.928488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.858956+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:39:32.858956+0000","last_peered":"2024-09-05T06:39:32.858956+0000","last_clean":"2024-09-05T06:39:32.858956+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:39:32.858956+0000","last_undegraded":"2024-09-05T06:39:32.858956+0000","last_fullsized":"2024-09-05T06:39:32.858956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925164+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:39:12.925164+0000","last_peered":"2024-09-05T06:39:12.925164+0000","last_clean":"2024-09-05T06:39:12.925164+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:39:12.925164+0000","last_undegraded":"2024-09-05T06:39:12.925164+0000","last_fullsized":"2024-09-05T06:39:12.925164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924996+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:39:12.924996+0000","last_peered":"2024-09-05T06:39:12.924996+0000","last_clean":"2024-09-05T06:39:12.924996+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:39:12.924996+0000","last_undegraded":"2024-09-05T06:39:12.924996+0000","last_fullsized":"2024-09-05T06:39:12.924996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925065+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:39:12.925065+0000","last_peered":"2024-09-05T06:39:12.925065+0000","last_clean":"2024-09-05T06:39:12.925065+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:39:12.925065+0000","last_undegraded":"2024-09-05T06:39:12.925065+0000","last_fullsized":"2024-09-05T06:39:12.925065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:44.899422+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:39:44.899422+0000","last_peered":"2024-09-05T06:39:44.899422+0000","last_clean":"2024-09-05T06:39:44.899422+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:39:44.899422+0000","last_undegraded":"2024-09-05T06:39:44.899422+0000","last_fullsized":"2024-09-05T06:39:44.899422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931567+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:39:12.931567+0000","last_peered":"2024-09-05T06:39:12.931567+0000","last_clean":"2024-09-05T06:39:12.931567+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:39:12.931567+0000","last_undegraded":"2024-09-05T06:39:12.931567+0000","last_fullsized":"2024-09-05T06:39:12.931567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924990+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:39:12.924990+0000","last_peered":"2024-09-05T06:39:12.924990+0000","last_clean":"2024-09-05T06:39:12.924990+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:39:12.924990+0000","last_undegraded":"2024-09-05T06:39:12.924990+0000","last_fullsized":"2024-09-05T06:39:12.924990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854285+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:39:31.854285+0000","last_peered":"2024-09-05T06:39:31.854285+0000","last_clean":"2024-09-05T06:39:31.854285+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:39:31.854285+0000","last_undegraded":"2024-09-05T06:39:31.854285+0000","last_fullsized":"2024-09-05T06:39:31.854285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923807+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:39:12.923807+0000","last_peered":"2024-09-05T06:39:12.923807+0000","last_clean":"2024-09-05T06:39:12.923807+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:39:12.923807+0000","last_undegraded":"2024-09-05T06:39:12.923807+0000","last_fullsized":"2024-09-05T06:39:12.923807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925257+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:39:12.925257+0000","last_peered":"2024-09-05T06:39:12.925257+0000","last_clean":"2024-09-05T06:39:12.925257+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:39:12.925257+0000","last_undegraded":"2024-09-05T06:39:12.925257+0000","last_fullsized":"2024-09-05T06:39:12.925257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859107+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:39:32.859107+0000","last_peered":"2024-09-05T06:39:32.859107+0000","last_clean":"2024-09-05T06:39:32.859107+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:39:32.859107+0000","last_undegraded":"2024-09-05T06:39:32.859107+0000","last_fullsized":"2024-09-05T06:39:32.859107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929213+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:39:12.929213+0000","last_peered":"2024-09-05T06:39:12.929213+0000","last_clean":"2024-09-05T06:39:12.929213+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:39:12.929213+0000","last_undegraded":"2024-09-05T06:39:12.929213+0000","last_fullsized":"2024-09-05T06:39:12.929213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928351+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:39:12.928351+0000","last_peered":"2024-09-05T06:39:12.928351+0000","last_clean":"2024-09-05T06:39:12.928351+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:39:12.928351+0000","last_undegraded":"2024-09-05T06:39:12.928351+0000","last_fullsized":"2024-09-05T06:39:12.928351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.844687+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:39:31.844687+0000","last_peered":"2024-09-05T06:39:31.844687+0000","last_clean":"2024-09-05T06:39:31.844687+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:39:31.844687+0000","last_undegraded":"2024-09-05T06:39:31.844687+0000","last_fullsized":"2024-09-05T06:39:31.844687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930143+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:39:12.930143+0000","last_peered":"2024-09-05T06:39:12.930143+0000","last_clean":"2024-09-05T06:39:12.930143+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:39:12.930143+0000","last_undegraded":"2024-09-05T06:39:12.930143+0000","last_fullsized":"2024-09-05T06:39:12.930143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931025+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:39:12.931025+0000","last_peered":"2024-09-05T06:39:12.931025+0000","last_clean":"2024-09-05T06:39:12.931025+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:39:12.931025+0000","last_undegraded":"2024-09-05T06:39:12.931025+0000","last_fullsized":"2024-09-05T06:39:12.931025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854721+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:39:31.854721+0000","last_peered":"2024-09-05T06:39:31.854721+0000","last_clean":"2024-09-05T06:39:31.854721+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:39:31.854721+0000","last_undegraded":"2024-09-05T06:39:31.854721+0000","last_fullsized":"2024-09-05T06:39:31.854721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929035+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:39:12.929035+0000","last_peered":"2024-09-05T06:39:12.929035+0000","last_clean":"2024-09-05T06:39:12.929035+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:39:12.929035+0000","last_undegraded":"2024-09-05T06:39:12.929035+0000","last_fullsized":"2024-09-05T06:39:12.929035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924474+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:39:12.924474+0000","last_peered":"2024-09-05T06:39:12.924474+0000","last_clean":"2024-09-05T06:39:12.924474+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:39:12.924474+0000","last_undegraded":"2024-09-05T06:39:12.924474+0000","last_fullsized":"2024-09-05T06:39:12.924474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:44.868358+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:39:44.868358+0000","last_peered":"2024-09-05T06:39:44.868358+0000","last_clean":"2024-09-05T06:39:44.868358+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:39:44.868358+0000","last_undegraded":"2024-09-05T06:39:44.868358+0000","last_fullsized":"2024-09-05T06:39:44.868358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925797+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:39:12.925797+0000","last_peered":"2024-09-05T06:39:12.925797+0000","last_clean":"2024-09-05T06:39:12.925797+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:39:12.925797+0000","last_undegraded":"2024-09-05T06:39:12.925797+0000","last_fullsized":"2024-09-05T06:39:12.925797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924664+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:39:12.924664+0000","last_peered":"2024-09-05T06:39:12.924664+0000","last_clean":"2024-09-05T06:39:12.924664+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:39:12.924664+0000","last_undegraded":"2024-09-05T06:39:12.924664+0000","last_fullsized":"2024-09-05T06:39:12.924664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859070+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:39:32.859070+0000","last_peered":"2024-09-05T06:39:32.859070+0000","last_clean":"2024-09-05T06:39:32.859070+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:39:32.859070+0000","last_undegraded":"2024-09-05T06:39:32.859070+0000","last_fullsized":"2024-09-05T06:39:32.859070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928077+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:39:12.928077+0000","last_peered":"2024-09-05T06:39:12.928077+0000","last_clean":"2024-09-05T06:39:12.928077+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:39:12.928077+0000","last_undegraded":"2024-09-05T06:39:12.928077+0000","last_fullsized":"2024-09-05T06:39:12.928077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928477+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:39:12.928477+0000","last_peered":"2024-09-05T06:39:12.928477+0000","last_clean":"2024-09-05T06:39:12.928477+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:39:12.928477+0000","last_undegraded":"2024-09-05T06:39:12.928477+0000","last_fullsized":"2024-09-05T06:39:12.928477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.860569+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:39:32.860569+0000","last_peered":"2024-09-05T06:39:32.860569+0000","last_clean":"2024-09-05T06:39:32.860569+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:39:32.860569+0000","last_undegraded":"2024-09-05T06:39:32.860569+0000","last_fullsized":"2024-09-05T06:39:32.860569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931370+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:39:12.931370+0000","last_peered":"2024-09-05T06:39:12.931370+0000","last_clean":"2024-09-05T06:39:12.931370+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:39:12.931370+0000","last_undegraded":"2024-09-05T06:39:12.931370+0000","last_fullsized":"2024-09-05T06:39:12.931370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924799+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:39:12.924799+0000","last_peered":"2024-09-05T06:39:12.924799+0000","last_clean":"2024-09-05T06:39:12.924799+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:39:12.924799+0000","last_undegraded":"2024-09-05T06:39:12.924799+0000","last_fullsized":"2024-09-05T06:39:12.924799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.870686+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:39:43.870686+0000","last_peered":"2024-09-05T06:39:43.870686+0000","last_clean":"2024-09-05T06:39:43.870686+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:39:43.870686+0000","last_undegraded":"2024-09-05T06:39:43.870686+0000","last_fullsized":"2024-09-05T06:39:43.870686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":58,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925363+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:39:12.925363+0000","last_peered":"2024-09-05T06:39:12.925363+0000","last_clean":"2024-09-05T06:39:12.925363+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:39:12.925363+0000","last_undegraded":"2024-09-05T06:39:12.925363+0000","last_fullsized":"2024-09-05T06:39:12.925363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932416+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:39:12.932416+0000","last_peered":"2024-09-05T06:39:12.932416+0000","last_clean":"2024-09-05T06:39:12.932416+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:39:12.932416+0000","last_undegraded":"2024-09-05T06:39:12.932416+0000","last_fullsized":"2024-09-05T06:39:12.932416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.847959+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:39:31.847959+0000","last_peered":"2024-09-05T06:39:31.847959+0000","last_clean":"2024-09-05T06:39:31.847959+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:39:31.847959+0000","last_undegraded":"2024-09-05T06:39:31.847959+0000","last_fullsized":"2024-09-05T06:39:31.847959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931541+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:39:12.931541+0000","last_peered":"2024-09-05T06:39:12.931541+0000","last_clean":"2024-09-05T06:39:12.931541+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:39:12.931541+0000","last_undegraded":"2024-09-05T06:39:12.931541+0000","last_fullsized":"2024-09-05T06:39:12.931541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931561+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:39:12.931561+0000","last_peered":"2024-09-05T06:39:12.931561+0000","last_clean":"2024-09-05T06:39:12.931561+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:39:12.931561+0000","last_undegraded":"2024-09-05T06:39:12.931561+0000","last_fullsized":"2024-09-05T06:39:12.931561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859600+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:39:32.859600+0000","last_peered":"2024-09-05T06:39:32.859600+0000","last_clean":"2024-09-05T06:39:32.859600+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:39:32.859600+0000","last_undegraded":"2024-09-05T06:39:32.859600+0000","last_fullsized":"2024-09-05T06:39:32.859600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925123+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:39:12.925123+0000","last_peered":"2024-09-05T06:39:12.925123+0000","last_clean":"2024-09-05T06:39:12.925123+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:39:12.925123+0000","last_undegraded":"2024-09-05T06:39:12.925123+0000","last_fullsized":"2024-09-05T06:39:12.925123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.933049+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:39:12.933049+0000","last_peered":"2024-09-05T06:39:12.933049+0000","last_clean":"2024-09-05T06:39:12.933049+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:39:12.933049+0000","last_undegraded":"2024-09-05T06:39:12.933049+0000","last_fullsized":"2024-09-05T06:39:12.933049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:43.865960+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:39:43.865960+0000","last_peered":"2024-09-05T06:39:43.865960+0000","last_clean":"2024-09-05T06:39:43.865960+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:39:43.865960+0000","last_undegraded":"2024-09-05T06:39:43.865960+0000","last_fullsized":"2024-09-05T06:39:43.865960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927699+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:39:12.927699+0000","last_peered":"2024-09-05T06:39:12.927699+0000","last_clean":"2024-09-05T06:39:12.927699+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:39:12.927699+0000","last_undegraded":"2024-09-05T06:39:12.927699+0000","last_fullsized":"2024-09-05T06:39:12.927699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923934+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:39:12.923934+0000","last_peered":"2024-09-05T06:39:12.923934+0000","last_clean":"2024-09-05T06:39:12.923934+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:39:12.923934+0000","last_undegraded":"2024-09-05T06:39:12.923934+0000","last_fullsized":"2024-09-05T06:39:12.923934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925236+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:39:12.925236+0000","last_peered":"2024-09-05T06:39:12.925236+0000","last_clean":"2024-09-05T06:39:12.925236+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:39:12.925236+0000","last_undegraded":"2024-09-05T06:39:12.925236+0000","last_fullsized":"2024-09-05T06:39:12.925236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:32.859600+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:39:32.859600+0000","last_peered":"2024-09-05T06:39:32.859600+0000","last_clean":"2024-09-05T06:39:32.859600+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:39:32.859600+0000","last_undegraded":"2024-09-05T06:39:32.859600+0000","last_fullsized":"2024-09-05T06:39:32.859600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929091+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:39:12.929091+0000","last_peered":"2024-09-05T06:39:12.929091+0000","last_clean":"2024-09-05T06:39:12.929091+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:39:12.929091+0000","last_undegraded":"2024-09-05T06:39:12.929091+0000","last_fullsized":"2024-09-05T06:39:12.929091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928436+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:39:12.928436+0000","last_peered":"2024-09-05T06:39:12.928436+0000","last_clean":"2024-09-05T06:39:12.928436+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:39:12.928436+0000","last_undegraded":"2024-09-05T06:39:12.928436+0000","last_fullsized":"2024-09-05T06:39:12.928436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:30.875871+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:39:30.875871+0000","last_peered":"2024-09-05T06:39:30.875871+0000","last_clean":"2024-09-05T06:39:30.875871+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:39:30.875871+0000","last_undegraded":"2024-09-05T06:39:30.875871+0000","last_fullsized":"2024-09-05T06:39:30.875871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:31.854015+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:39:31.854015+0000","last_peered":"2024-09-05T06:39:31.854015+0000","last_clean":"2024-09-05T06:39:31.854015+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:39:31.854015+0000","last_undegraded":"2024-09-05T06:39:31.854015+0000","last_fullsized":"2024-09-05T06:39:31.854015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932497+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:39:12.932497+0000","last_peered":"2024-09-05T06:39:12.932497+0000","last_clean":"2024-09-05T06:39:12.932497+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:39:12.932497+0000","last_undegraded":"2024-09-05T06:39:12.932497+0000","last_fullsized":"2024-09-05T06:39:12.932497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928650+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:39:12.928650+0000","last_peered":"2024-09-05T06:39:12.928650+0000","last_clean":"2024-09-05T06:39:12.928650+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:39:12.928650+0000","last_undegraded":"2024-09-05T06:39:12.928650+0000","last_fullsized":"2024-09-05T06:39:12.928650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:42.865209+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:39:42.865209+0000","last_peered":"2024-09-05T06:39:42.865209+0000","last_clean":"2024-09-05T06:39:42.865209+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:39:42.865209+0000","last_undegraded":"2024-09-05T06:39:42.865209+0000","last_fullsized":"2024-09-05T06:39:42.865209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924487+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:39:12.924487+0000","last_peered":"2024-09-05T06:39:12.924487+0000","last_clean":"2024-09-05T06:39:12.924487+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:39:12.924487+0000","last_undegraded":"2024-09-05T06:39:12.924487+0000","last_fullsized":"2024-09-05T06:39:12.924487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929786+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:39:12.929786+0000","last_peered":"2024-09-05T06:39:12.929786+0000","last_clean":"2024-09-05T06:39:12.929786+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:39:12.929786+0000","last_undegraded":"2024-09-05T06:39:12.929786+0000","last_fullsized":"2024-09-05T06:39:12.929786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927704+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:39:12.927704+0000","last_peered":"2024-09-05T06:39:12.927704+0000","last_clean":"2024-09-05T06:39:12.927704+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:39:12.927704+0000","last_undegraded":"2024-09-05T06:39:12.927704+0000","last_fullsized":"2024-09-05T06:39:12.927704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.933654+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:39:12.933654+0000","last_peered":"2024-09-05T06:39:12.933654+0000","last_clean":"2024-09-05T06:39:12.933654+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:39:12.933654+0000","last_undegraded":"2024-09-05T06:39:12.933654+0000","last_fullsized":"2024-09-05T06:39:12.933654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928391+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:39:12.928391+0000","last_peered":"2024-09-05T06:39:12.928391+0000","last_clean":"2024-09-05T06:39:12.928391+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:39:12.928391+0000","last_undegraded":"2024-09-05T06:39:12.928391+0000","last_fullsized":"2024-09-05T06:39:12.928391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923839+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:39:12.923839+0000","last_peered":"2024-09-05T06:39:12.923839+0000","last_clean":"2024-09-05T06:39:12.923839+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:39:12.923839+0000","last_undegraded":"2024-09-05T06:39:12.923839+0000","last_fullsized":"2024-09-05T06:39:12.923839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928825+0000","last_change":"2024-09-05T06:38:55.376697+0000","last_active":"2024-09-05T06:39:12.928825+0000","last_peered":"2024-09-05T06:39:12.928825+0000","last_clean":"2024-09-05T06:39:12.928825+0000","last_became_active":"2024-09-05T06:38:55.376401+0000","last_became_peered":"2024-09-05T06:38:55.376401+0000","last_unstale":"2024-09-05T06:39:12.928825+0000","last_undegraded":"2024-09-05T06:39:12.928825+0000","last_fullsized":"2024-09-05T06:39:12.928825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:41:31.881708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928614+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:39:12.928614+0000","last_peered":"2024-09-05T06:39:12.928614+0000","last_clean":"2024-09-05T06:39:12.928614+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:39:12.928614+0000","last_undegraded":"2024-09-05T06:39:12.928614+0000","last_fullsized":"2024-09-05T06:39:12.928614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.930228+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:39:12.930228+0000","last_peered":"2024-09-05T06:39:12.930228+0000","last_clean":"2024-09-05T06:39:12.930228+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:39:12.930228+0000","last_undegraded":"2024-09-05T06:39:12.930228+0000","last_fullsized":"2024-09-05T06:39:12.930228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928203+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:39:12.928203+0000","last_peered":"2024-09-05T06:39:12.928203+0000","last_clean":"2024-09-05T06:39:12.928203+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:39:12.928203+0000","last_undegraded":"2024-09-05T06:39:12.928203+0000","last_fullsized":"2024-09-05T06:39:12.928203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929428+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:39:12.929428+0000","last_peered":"2024-09-05T06:39:12.929428+0000","last_clean":"2024-09-05T06:39:12.929428+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:39:12.929428+0000","last_undegraded":"2024-09-05T06:39:12.929428+0000","last_fullsized":"2024-09-05T06:39:12.929428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927942+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:39:12.927942+0000","last_peered":"2024-09-05T06:39:12.927942+0000","last_clean":"2024-09-05T06:39:12.927942+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:39:12.927942+0000","last_undegraded":"2024-09-05T06:39:12.927942+0000","last_fullsized":"2024-09-05T06:39:12.927942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932168+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:39:12.932168+0000","last_peered":"2024-09-05T06:39:12.932168+0000","last_clean":"2024-09-05T06:39:12.932168+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:39:12.932168+0000","last_undegraded":"2024-09-05T06:39:12.932168+0000","last_fullsized":"2024-09-05T06:39:12.932168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928920+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:39:12.928920+0000","last_peered":"2024-09-05T06:39:12.928920+0000","last_clean":"2024-09-05T06:39:12.928920+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:39:12.928920+0000","last_undegraded":"2024-09-05T06:39:12.928920+0000","last_fullsized":"2024-09-05T06:39:12.928920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928196+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:39:12.928196+0000","last_peered":"2024-09-05T06:39:12.928196+0000","last_clean":"2024-09-05T06:39:12.928196+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:39:12.928196+0000","last_undegraded":"2024-09-05T06:39:12.928196+0000","last_fullsized":"2024-09-05T06:39:12.928196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929955+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:39:12.929955+0000","last_peered":"2024-09-05T06:39:12.929955+0000","last_clean":"2024-09-05T06:39:12.929955+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:39:12.929955+0000","last_undegraded":"2024-09-05T06:39:12.929955+0000","last_fullsized":"2024-09-05T06:39:12.929955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925729+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:39:12.925729+0000","last_peered":"2024-09-05T06:39:12.925729+0000","last_clean":"2024-09-05T06:39:12.925729+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:39:12.925729+0000","last_undegraded":"2024-09-05T06:39:12.925729+0000","last_fullsized":"2024-09-05T06:39:12.925729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929506+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:39:12.929506+0000","last_peered":"2024-09-05T06:39:12.929506+0000","last_clean":"2024-09-05T06:39:12.929506+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:39:12.929506+0000","last_undegraded":"2024-09-05T06:39:12.929506+0000","last_fullsized":"2024-09-05T06:39:12.929506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932041+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:39:12.932041+0000","last_peered":"2024-09-05T06:39:12.932041+0000","last_clean":"2024-09-05T06:39:12.932041+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:39:12.932041+0000","last_undegraded":"2024-09-05T06:39:12.932041+0000","last_fullsized":"2024-09-05T06:39:12.932041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923572+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:39:12.923572+0000","last_peered":"2024-09-05T06:39:12.923572+0000","last_clean":"2024-09-05T06:39:12.923572+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:39:12.923572+0000","last_undegraded":"2024-09-05T06:39:12.923572+0000","last_fullsized":"2024-09-05T06:39:12.923572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923620+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:39:12.923620+0000","last_peered":"2024-09-05T06:39:12.923620+0000","last_clean":"2024-09-05T06:39:12.923620+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:39:12.923620+0000","last_undegraded":"2024-09-05T06:39:12.923620+0000","last_fullsized":"2024-09-05T06:39:12.923620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928843+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:39:12.928843+0000","last_peered":"2024-09-05T06:39:12.928843+0000","last_clean":"2024-09-05T06:39:12.928843+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:39:12.928843+0000","last_undegraded":"2024-09-05T06:39:12.928843+0000","last_fullsized":"2024-09-05T06:39:12.928843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932178+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:39:12.932178+0000","last_peered":"2024-09-05T06:39:12.932178+0000","last_clean":"2024-09-05T06:39:12.932178+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:39:12.932178+0000","last_undegraded":"2024-09-05T06:39:12.932178+0000","last_fullsized":"2024-09-05T06:39:12.932178+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.925123+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:39:12.925123+0000","last_peered":"2024-09-05T06:39:12.925123+0000","last_clean":"2024-09-05T06:39:12.925123+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:39:12.925123+0000","last_undegraded":"2024-09-05T06:39:12.925123+0000","last_fullsized":"2024-09-05T06:39:12.925123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928612+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:39:12.928612+0000","last_peered":"2024-09-05T06:39:12.928612+0000","last_clean":"2024-09-05T06:39:12.928612+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:39:12.928612+0000","last_undegraded":"2024-09-05T06:39:12.928612+0000","last_fullsized":"2024-09-05T06:39:12.928612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.931839+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:39:12.931839+0000","last_peered":"2024-09-05T06:39:12.931839+0000","last_clean":"2024-09-05T06:39:12.931839+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:39:12.931839+0000","last_undegraded":"2024-09-05T06:39:12.931839+0000","last_fullsized":"2024-09-05T06:39:12.931839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.928359+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:39:12.928359+0000","last_peered":"2024-09-05T06:39:12.928359+0000","last_clean":"2024-09-05T06:39:12.928359+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:39:12.928359+0000","last_undegraded":"2024-09-05T06:39:12.928359+0000","last_fullsized":"2024-09-05T06:39:12.928359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924609+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:39:12.924609+0000","last_peered":"2024-09-05T06:39:12.924609+0000","last_clean":"2024-09-05T06:39:12.924609+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:39:12.924609+0000","last_undegraded":"2024-09-05T06:39:12.924609+0000","last_fullsized":"2024-09-05T06:39:12.924609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.927750+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:39:12.927750+0000","last_peered":"2024-09-05T06:39:12.927750+0000","last_clean":"2024-09-05T06:39:12.927750+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:39:12.927750+0000","last_undegraded":"2024-09-05T06:39:12.927750+0000","last_fullsized":"2024-09-05T06:39:12.927750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.924470+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:39:12.924470+0000","last_peered":"2024-09-05T06:39:12.924470+0000","last_clean":"2024-09-05T06:39:12.924470+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:39:12.924470+0000","last_undegraded":"2024-09-05T06:39:12.924470+0000","last_fullsized":"2024-09-05T06:39:12.924470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.932050+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:39:12.932050+0000","last_peered":"2024-09-05T06:39:12.932050+0000","last_clean":"2024-09-05T06:39:12.932050+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:39:12.932050+0000","last_undegraded":"2024-09-05T06:39:12.932050+0000","last_fullsized":"2024-09-05T06:39:12.932050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923679+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:39:12.923679+0000","last_peered":"2024-09-05T06:39:12.923679+0000","last_clean":"2024-09-05T06:39:12.923679+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:39:12.923679+0000","last_undegraded":"2024-09-05T06:39:12.923679+0000","last_fullsized":"2024-09-05T06:39:12.923679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.923827+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:39:12.923827+0000","last_peered":"2024-09-05T06:39:12.923827+0000","last_clean":"2024-09-05T06:39:12.923827+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:39:12.923827+0000","last_undegraded":"2024-09-05T06:39:12.923827+0000","last_fullsized":"2024-09-05T06:39:12.923827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-05T06:39:12.929407+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:39:12.929407+0000","last_peered":"2024-09-05T06:39:12.929407+0000","last_clean":"2024-09-05T06:39:12.929407+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:39:12.929407+0000","last_undegraded":"2024-09-05T06:39:12.929407+0000","last_fullsized":"2024-09-05T06:39:12.929407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50553,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":163,"num_read_kb":139,"num_write":232,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":166975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612631,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":491567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1122099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":491831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462954,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1088745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626480,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":486979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855412,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1083934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":550107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":488622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":488435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:39:49.796 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T06:39:49.796 INFO:tasks.cephadm:Setting up client nodes... 2024-09-05T06:39:49.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-05T06:39:50.043 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:50.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:50 smithi086 ceph-mon[30943]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:50.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:50 smithi086 ceph-mon[30943]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-05T06:39:50.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- 172.21.15.86:0/2032457738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 msgr2=0x7f6284103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 --2- 172.21.15.86:0/2032457738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284103330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6274009930 tx=0x7f627402a420 comp rx=0 tx=0).stop 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- 172.21.15.86:0/2032457738 shutdown_connections 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 --2- 172.21.15.86:0/2032457738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6284103870 0x7f628410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 --2- 172.21.15.86:0/2032457738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284103330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 --2- 172.21.15.86:0/2032457738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62840ffed0 0x7f62841002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- 172.21.15.86:0/2032457738 >> 172.21.15.86:0/2032457738 conn(0x7f62840fba40 msgr2=0x7f62840fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- 172.21.15.86:0/2032457738 shutdown_connections 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- 172.21.15.86:0/2032457738 wait complete. 2024-09-05T06:39:50.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 Processor -- start 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.554+0000 7f628b52b640 1 -- start start 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840ffed0 0x7f6284197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 0x7f628419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f628410d7c0 con 0x7f6284102eb0 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f628410d640 con 0x7f6284103870 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f628410d940 con 0x7f62840ffed0 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50012/0 (socket says 172.21.15.86:50012) 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 -- 172.21.15.86:0/2986728684 learned_addr learned my addr 172.21.15.86:0/2986728684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f62892c7640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840ffed0 0x7f6284197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6289ac8640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 0x7f628419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 -- 172.21.15.86:0/2986728684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840ffed0 msgr2=0x7f6284197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840ffed0 0x7f6284197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 -- 172.21.15.86:0/2986728684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 msgr2=0x7f628419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 0x7f628419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 -- 172.21.15.86:0/2986728684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628419fb00 con 0x7f6284102eb0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6289ac8640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 0x7f628419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f6288ac6640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6274005940 tx=0x7f6274004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403f030 con 0x7f6284102eb0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6274003cb0 con 0x7f6284102eb0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628419fd90 con 0x7f6284102eb0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.555+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274035070 con 0x7f6284102eb0 2024-09-05T06:39:50.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.556+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62841a0290 con 0x7f6284102eb0 2024-09-05T06:39:50.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:50 smithi096 ceph-mon[34468]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:50.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:50 smithi096 ceph-mon[34468]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.556+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6284109120 con 0x7f6284102eb0 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.557+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6274042030 con 0x7f6284102eb0 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.558+0000 7f627a7fc640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 0x7f62700a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.558+0000 7f62892c7640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 0x7f62700a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.558+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f62740ef860 con 0x7f6284102eb0 2024-09-05T06:39:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.559+0000 7f62892c7640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 0x7f62700a5100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f628406a590 tx=0x7f6280007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.562+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62740bc2c0 con 0x7f6284102eb0 2024-09-05T06:39:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:50 smithi115 ceph-mon[34791]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:39:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:50 smithi115 ceph-mon[34791]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-05T06:39:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.791+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f628402d7c0 con 0x7f6284102eb0 2024-09-05T06:39:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.793+0000 7f627a7fc640 1 -- 172.21.15.86:0/2986728684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f62740bbbc0 con 0x7f6284102eb0 2024-09-05T06:39:50.793 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-09-05T06:39:50.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQA2Utlm5PNBLxAAwPbqdNeqjuJoF3PoTRXk9Q== 2024-09-05T06:39:50.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 msgr2=0x7f62700a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 0x7f62700a5100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f628406a590 tx=0x7f6280007600 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 msgr2=0x7f6284198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6274005940 tx=0x7f6274004340 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 shutdown_connections 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f62700a2c40 0x7f62700a5100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6284103870 0x7f628419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6284102eb0 0x7f6284198410 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 --2- 172.21.15.86:0/2986728684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840ffed0 0x7f6284197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.795+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 >> 172.21.15.86:0/2986728684 conn(0x7f62840fba40 msgr2=0x7f62841062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.796+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 shutdown_connections 2024-09-05T06:39:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:50.796+0000 7f628b52b640 1 -- 172.21.15.86:0/2986728684 wait complete. 2024-09-05T06:39:51.475 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T06:39:51.475 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T06:39:51.476 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:39:51.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-05T06:39:51.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-05T06:39:51.512 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T06:39:51.513 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-05T06:39:51.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph mgr dump --format=json 2024-09-05T06:39:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T06:39:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T06:39:51.583 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T06:39:51.583 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2986728684' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T06:39:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:52.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:52 smithi096 ceph-mon[34468]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-05T06:39:52.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:52 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:39:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.321+0000 7fb92c85a640 1 -- 172.21.15.86:0/3258221514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102ef0 msgr2=0x7fb92810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.321+0000 7fb92c85a640 1 --2- 172.21.15.86:0/3258221514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102ef0 0x7fb92810da40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb920009900 tx=0x7fb92002a480 comp rx=0 tx=0).stop 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 -- 172.21.15.86:0/3258221514 shutdown_connections 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 --2- 172.21.15.86:0/3258221514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102ef0 0x7fb92810da40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 --2- 172.21.15.86:0/3258221514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928102570 0x7fb9281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 --2- 172.21.15.86:0/3258221514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928100e20 0x7fb928101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 -- 172.21.15.86:0/3258221514 >> 172.21.15.86:0/3258221514 conn(0x7fb9280fc7d0 msgr2=0x7fb9280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 -- 172.21.15.86:0/3258221514 shutdown_connections 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 -- 172.21.15.86:0/3258221514 wait complete. 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 Processor -- start 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.322+0000 7fb92c85a640 1 -- start start 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100e20 0x7fb9281993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928102ef0 0x7fb9281a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb928110ed0 con 0x7fb928102570 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb928110d50 con 0x7fb928100e20 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb928111050 con 0x7fb928102ef0 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50032/0 (socket says 172.21.15.86:50032) 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 -- 172.21.15.86:0/1721197620 learned_addr learned my addr 172.21.15.86:0/1721197620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb926d9d640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928102ef0 0x7fb9281a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92659c640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100e20 0x7fb9281993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 -- 172.21.15.86:0/1721197620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928102ef0 msgr2=0x7fb9281a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928102ef0 0x7fb9281a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 -- 172.21.15.86:0/1721197620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100e20 msgr2=0x7fb9281993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100e20 0x7fb9281993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 -- 172.21.15.86:0/1721197620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9281a4960 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb925d9b640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b4a0 tx=0x7fb91000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910014070 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb910010ac0 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910017c30 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9281a4b30 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.323+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281a5070 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.325+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928101200 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.326+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb910017070 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.326+0000 7fb91f7fe640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 0x7fb9140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.327+0000 7fb92659c640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 0x7fb9140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.327+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb9100ce9e0 con 0x7fb928102570 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.327+0000 7fb92659c640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 0x7fb9140a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a400 tx=0x7fb91800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.329+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb91009af60 con 0x7fb928102570 2024-09-05T06:39:52.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:52 smithi115 ceph-mon[34791]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-05T06:39:52.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:52 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:39:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.546+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb928109090 con 0x7fb928102570 2024-09-05T06:39:52.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:52 smithi086 ceph-mon[30943]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-05T06:39:52.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:52 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:39:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.551+0000 7fb91f7fe640 1 -- 172.21.15.86:0/1721197620 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7fb91009f260 con 0x7fb928102570 2024-09-05T06:39:52.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 msgr2=0x7fb9140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 0x7fb9140a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb91800a400 tx=0x7fb91800d040 comp rx=0 tx=0).stop 2024-09-05T06:39:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 msgr2=0x7fb9281998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b4a0 tx=0x7fb91000b960 comp rx=0 tx=0).stop 2024-09-05T06:39:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 shutdown_connections 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb928102ef0 0x7fb9281a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb928102570 0x7fb9281998e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9140a2c90 0x7fb9140a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 --2- 172.21.15.86:0/1721197620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100e20 0x7fb9281993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 >> 172.21.15.86:0/1721197620 conn(0x7fb9280fc7d0 msgr2=0x7fb9280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 shutdown_connections 2024-09-05T06:39:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:52.554+0000 7fb92c85a640 1 -- 172.21.15.86:0/1721197620 wait complete. 2024-09-05T06:39:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:53 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1721197620' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T06:39:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:53 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:53 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1721197620' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T06:39:53.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:53 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:53 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1721197620' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T06:39:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:53 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:53.704 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3708924769},{"type":"v1","addr":"172.21.15.86:6801","nonce":3708924769}]},"active_addr":"172.21.15.86:6801/3708924769","active_change":"2024-09-05T06:32:11.079334+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3625555544}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4040819651}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2959599766}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4279196951}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4062552821}]}]} 2024-09-05T06:39:53.709 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-05T06:39:53.709 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-05T06:39:53.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:54.003 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:54.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:54 smithi086 ceph-mon[30943]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 -- 172.21.15.86:0/9142841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48100e20 msgr2=0x7fae48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/9142841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48100e20 0x7fae48101200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fae38009900 tx=0x7fae3802a480 comp rx=0 tx=0).stop 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 -- 172.21.15.86:0/9142841 shutdown_connections 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/9142841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 0x7fae4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/9142841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48102570 0x7fae481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/9142841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48100e20 0x7fae48101200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 -- 172.21.15.86:0/9142841 >> 172.21.15.86:0/9142841 conn(0x7fae480fc7d0 msgr2=0x7fae480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 -- 172.21.15.86:0/9142841 shutdown_connections 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 -- 172.21.15.86:0/9142841 wait complete. 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.478+0000 7fae4d6d3640 1 Processor -- start 2024-09-05T06:39:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 -- start start 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48100e20 0x7fae48198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 0x7fae4819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110ed0 con 0x7fae48102570 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110d50 con 0x7fae48100e20 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae4d6d3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae48111050 con 0x7fae48102ef0 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37334/0 (socket says 172.21.15.86:37334) 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 -- 172.21.15.86:0/1591518290 learned_addr learned my addr 172.21.15.86:0/1591518290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae46ffd640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48100e20 0x7fae48198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae477fe640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 0x7fae4819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 -- 172.21.15.86:0/1591518290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 msgr2=0x7fae4819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 0x7fae4819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 -- 172.21.15.86:0/1591518290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48100e20 msgr2=0x7fae48198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48100e20 0x7fae48198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 -- 172.21.15.86:0/1591518290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4819d4f0 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae467fc640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b4a0 tx=0x7fae3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.479+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c014070 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.480+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4819d7e0 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.480+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae3c010ac0 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.480+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c017c30 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.480+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae481a50c0 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.481+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae48101200 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.482+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae3c017070 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.482+0000 7fae2ffff640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 0x7fae300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.482+0000 7fae46ffd640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 0x7fae300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.482+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fae3c0ceb80 con 0x7fae48102570 2024-09-05T06:39:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.483+0000 7fae46ffd640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 0x7fae300a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae38005ab0 tx=0x7fae38005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:54.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.486+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae3c09b750 con 0x7fae48102570 2024-09-05T06:39:54.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:54 smithi096 ceph-mon[34468]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-05T06:39:54.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.680+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae4810a0a0 con 0x7fae48102570 2024-09-05T06:39:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:54 smithi115 ceph-mon[34791]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-05T06:39:54.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.683+0000 7fae2ffff640 1 -- 172.21.15.86:0/1591518290 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7fae3c09b050 con 0x7fae48102570 2024-09-05T06:39:54.683 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:54.683 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 msgr2=0x7fae300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 0x7fae300a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae38005ab0 tx=0x7fae38005a20 comp rx=0 tx=0).stop 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 msgr2=0x7fae48198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b4a0 tx=0x7fae3c00b960 comp rx=0 tx=0).stop 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 shutdown_connections 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae48102ef0 0x7fae4819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae48102570 0x7fae48198a80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fae300a2c10 0x7fae300a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 --2- 172.21.15.86:0/1591518290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae48100e20 0x7fae48198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 >> 172.21.15.86:0/1591518290 conn(0x7fae480fc7d0 msgr2=0x7fae4806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 shutdown_connections 2024-09-05T06:39:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:54.686+0000 7fae4d6d3640 1 -- 172.21.15.86:0/1591518290 wait complete. 2024-09-05T06:39:55.422 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-05T06:39:55.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:39:55.667 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1591518290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:55 smithi086 ceph-mon[30943]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-05T06:39:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1591518290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:55 smithi115 ceph-mon[34791]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1591518290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:39:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:55 smithi096 ceph-mon[34468]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-05T06:39:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 -- 172.21.15.86:0/1287683397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 msgr2=0x7f6490100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 --2- 172.21.15.86:0/1287683397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f6490100ee0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6484009930 tx=0x7f648402a420 comp rx=0 tx=0).stop 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 -- 172.21.15.86:0/1287683397 shutdown_connections 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 --2- 172.21.15.86:0/1287683397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6490101420 0x7f649010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 --2- 172.21.15.86:0/1287683397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f6490100ee0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 --2- 172.21.15.86:0/1287683397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490106a80 0x7f6490106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 -- 172.21.15.86:0/1287683397 >> 172.21.15.86:0/1287683397 conn(0x7f64900fc7a0 msgr2=0x7f64900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 -- 172.21.15.86:0/1287683397 shutdown_connections 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.152+0000 7f6494ce8640 1 -- 172.21.15.86:0/1287683397 wait complete. 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 Processor -- start 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 -- start start 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 0x7f649019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6490106a80 0x7f649019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110d50 con 0x7f6490100a80 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110bd0 con 0x7f6490101420 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f6494ce8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110ed0 con 0x7f6490106a80 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f648e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f648e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37346/0 (socket says 172.21.15.86:37346) 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.153+0000 7f648e59c640 1 -- 172.21.15.86:0/3883861450 learned_addr learned my addr 172.21.15.86:0/3883861450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 -- 172.21.15.86:0/3883861450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6490106a80 msgr2=0x7f649019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6490106a80 0x7f649019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 -- 172.21.15.86:0/3883861450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 msgr2=0x7f649019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648dd9b640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 0x7f649019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 0x7f649019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 -- 172.21.15.86:0/3883861450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649019f530 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648dd9b640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 0x7f649019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f648e59c640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f648000c920 tx=0x7f648000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480007c70 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649019f820 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6480007e10 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480018820 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.154+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64901a7100 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.155+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6490100ee0 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.156+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f64800189c0 con 0x7f6490100a80 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.157+0000 7f647f7fe640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.157+0000 7f648dd9b640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.157+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f64800cf730 con 0x7f6490100a80 2024-09-05T06:39:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.157+0000 7f648dd9b640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 0x7f64780a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6484000c00 tx=0x7f6484031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.161+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6480014030 con 0x7f6490100a80 2024-09-05T06:39:56.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.330+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64901a7450 con 0x7f6490100a80 2024-09-05T06:39:56.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.331+0000 7f647f7fe640 1 -- 172.21.15.86:0/3883861450 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24671 (secure 0 0 0) 0x7f648009c1f0 con 0x7f6490100a80 2024-09-05T06:39:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:39:56.332 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:39:12.920662+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.333+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 msgr2=0x7f64780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 0x7f64780a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6484000c00 tx=0x7f6484031400 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 msgr2=0x7f649019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f648000c920 tx=0x7f648000cde0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 shutdown_connections 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6490106a80 0x7f649019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6490101420 0x7f649019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 --2- 172.21.15.86:0/3883861450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6490100a80 0x7f649019a580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 >> 172.21.15.86:0/3883861450 conn(0x7f64900fc7a0 msgr2=0x7f649006a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 shutdown_connections 2024-09-05T06:39:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:56.334+0000 7f6494ce8640 1 -- 172.21.15.86:0/3883861450 wait complete. 2024-09-05T06:39:56.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-05T06:39:56.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-05T06:39:56.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-05T06:39:56.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-05T06:39:56.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-05T06:39:56.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-05T06:39:56.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-05T06:39:56.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-05T06:39:56.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-05T06:39:56.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-05T06:39:56.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-05T06:39:56.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-05T06:39:57.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:57 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3883861450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:57.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:57 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3883861450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:57.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:57 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3883861450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:39:57.668 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:57.689 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.002 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:39:58.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:39:58 smithi086 ceph-mon[30943]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T06:39:58.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:39:58 smithi115 ceph-mon[34791]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T06:39:58.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:39:58 smithi096 ceph-mon[34468]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3329140603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108106a80 msgr2=0x7fb108106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3329140603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108106a80 0x7fb108106e60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb104009930 tx=0x7fb10402a420 comp rx=0 tx=0).stop 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3329140603 shutdown_connections 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3329140603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb108101420 0x7fb10810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3329140603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb108100a80 0x7fb108100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3329140603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108106a80 0x7fb108106e60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3329140603 >> 172.21.15.86:0/3329140603 conn(0x7fb1080fc7a0 msgr2=0x7fb1080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.124+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3329140603 shutdown_connections 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.125+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3329140603 wait complete. 2024-09-05T06:39:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.125+0000 7fb10f8a9640 1 Processor -- start 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.125+0000 7fb10f8a9640 1 -- start start 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb108100a80 0x7fb108197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 0x7fb108197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb108106a80 0x7fb10819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb108110d50 con 0x7fb108101420 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb108110bd0 con 0x7fb108100a80 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb108110ed0 con 0x7fb108106a80 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 0x7fb108197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 0x7fb108197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37368/0 (socket says 172.21.15.86:37368) 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 -- 172.21.15.86:0/3116367203 learned_addr learned my addr 172.21.15.86:0/3116367203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10d645640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb108100a80 0x7fb108197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 -- 172.21.15.86:0/3116367203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb108106a80 msgr2=0x7fb10819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb108106a80 0x7fb10819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 -- 172.21.15.86:0/3116367203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb108100a80 msgr2=0x7fb108197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb108100a80 0x7fb108197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10819c300 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10ce44640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 0x7fb108197890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d990 tx=0x7fb0f800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f801b070 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0f800edf0 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10819c5f0 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8014800 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.126+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10806a790 con 0x7fb108101420 2024-09-05T06:39:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.127+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb10806ada0 con 0x7fb108101420 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.128+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0f801e070 con 0x7fb108101420 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb0fe7fc640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb0f40a2c10 0x7fb0f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb10d645640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb0f40a2c10 0x7fb0f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb0f80d13e0 con 0x7fb108101420 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb0fe7fc640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] conn(0x7fb0f40add60 0x7fb0f40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0f40b08d0 con 0x7fb0f40add60 2024-09-05T06:39:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb0f80d1830 con 0x7fb108101420 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.129+0000 7fb10de46640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] conn(0x7fb0f40add60 0x7fb0f40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.130+0000 7fb10d645640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb0f40a2c10 0x7fb0f40a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb104000c00 tx=0x7fb104032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.130+0000 7fb10de46640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] conn(0x7fb0f40add60 0x7fb0f40b0200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.130+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== osd.0 v2:172.21.15.86:6802/2287131107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0f40b08d0 con 0x7fb0f40add60 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.144+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 --> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb10806b220 con 0x7fb0f40add60 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.144+0000 7fb0fe7fc640 1 -- 172.21.15.86:0/3116367203 <== osd.0 v2:172.21.15.86:6802/2287131107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb10806b220 con 0x7fb0f40add60 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.144+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] conn(0x7fb0f40add60 msgr2=0x7fb0f40b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] conn(0x7fb0f40add60 0x7fb0f40b0200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb0f40a2c10 msgr2=0x7fb0f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb0f40a2c10 0x7fb0f40a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb104000c00 tx=0x7fb104032c80 comp rx=0 tx=0).stop 2024-09-05T06:39:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 msgr2=0x7fb108197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 --2- 172.21.15.86:0/3116367203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb108101420 0x7fb108197890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d990 tx=0x7fb0f800de50 comp rx=0 tx=0).stop 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10de46640 1 -- 172.21.15.86:0/3116367203 reap_dead start 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 shutdown_connections 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 >> 172.21.15.86:0/3116367203 conn(0x7fb1080fc7a0 msgr2=0x7fb1080fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 shutdown_connections 2024-09-05T06:39:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.145+0000 7fb10f8a9640 1 -- 172.21.15.86:0/3116367203 wait complete. 2024-09-05T06:39:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 -- 172.21.15.86:0/2437408790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 msgr2=0x7f5c3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 --2- 172.21.15.86:0/2437408790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c3410da40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ca0 tx=0x7f5c2c02a950 comp rx=0 tx=0).stop 2024-09-05T06:39:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 -- 172.21.15.86:0/2437408790 shutdown_connections 2024-09-05T06:39:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 --2- 172.21.15.86:0/2437408790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c3410da40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 --2- 172.21.15.86:0/2437408790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 0x7f5c341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 --2- 172.21.15.86:0/2437408790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 -- 172.21.15.86:0/2437408790 >> 172.21.15.86:0/2437408790 conn(0x7f5c340fc7d0 msgr2=0x7f5c340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 -- 172.21.15.86:0/2437408790 shutdown_connections 2024-09-05T06:39:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.602+0000 7f5c3a408640 1 -- 172.21.15.86:0/2437408790 wait complete. 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 Processor -- start 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 -- start start 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c34198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102ef0 0x7f5c3419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110e30 con 0x7f5c34102570 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110cb0 con 0x7f5c34102ef0 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c3a408640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110fb0 con 0x7f5c34100e20 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c34198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c34198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37386/0 (socket says 172.21.15.86:37386) 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.603+0000 7f5c337fe640 1 -- 172.21.15.86:0/1920143308 learned_addr learned my addr 172.21.15.86:0/1920143308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c389a5640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102ef0 0x7f5c3419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c33fff640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 -- 172.21.15.86:0/1920143308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 msgr2=0x7f5c34198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 -- 172.21.15.86:0/1920143308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102ef0 msgr2=0x7f5c3419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102ef0 0x7f5c3419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3419d2f0 con 0x7f5c34102570 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.604+0000 7f5c337fe640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c34198880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c920 tx=0x7f5c2800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28007c70 con 0x7f5c34102570 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3419d5e0 con 0x7f5c34102570 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c28007e10 con 0x7f5c34102570 2024-09-05T06:39:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a4ec0 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800f450 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.605+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c34062980 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.607+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5c28004e80 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.607+0000 7f5c317fa640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c1c0a2af0 0x7f5c1c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c33fff640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c1c0a2af0 0x7f5c1c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f5c280cf630 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c317fa640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] conn(0x7f5c1c0adc40 0x7f5c1c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c28019e50 con 0x7f5c1c0adc40 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f5c280cfa80 con 0x7f5c34102570 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c389a5640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] conn(0x7f5c1c0adc40 0x7f5c1c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c33fff640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c1c0a2af0 0x7f5c1c0a4fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c24005fd0 tx=0x7f5c24005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.608+0000 7f5c389a5640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] conn(0x7f5c1c0adc40 0x7f5c1c0b00e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.609+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== osd.3 v2:172.21.15.86:6810/3414369107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c28019e50 con 0x7f5c1c0adc40 2024-09-05T06:39:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.622+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 --> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c34062e00 con 0x7f5c1c0adc40 2024-09-05T06:39:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c317fa640 1 -- 172.21.15.86:0/1920143308 <== osd.3 v2:172.21.15.86:6810/3414369107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5c34062e00 con 0x7f5c1c0adc40 2024-09-05T06:39:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] conn(0x7f5c1c0adc40 msgr2=0x7f5c1c0b00e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] conn(0x7f5c1c0adc40 0x7f5c1c0b00e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c1c0a2af0 msgr2=0x7f5c1c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5c1c0a2af0 0x7f5c1c0a4fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c24005fd0 tx=0x7f5c24005e20 comp rx=0 tx=0).stop 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 msgr2=0x7f5c34198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 --2- 172.21.15.86:0/1920143308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c34198880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c920 tx=0x7f5c2800cde0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c389a5640 1 -- 172.21.15.86:0/1920143308 reap_dead start 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 shutdown_connections 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 >> 172.21.15.86:0/1920143308 conn(0x7f5c340fc7d0 msgr2=0x7f5c3406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 shutdown_connections 2024-09-05T06:39:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.623+0000 7f5c3a408640 1 -- 172.21.15.86:0/1920143308 wait complete. 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- 172.21.15.86:0/1419109176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 msgr2=0x7f4d70100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 --2- 172.21.15.86:0/1419109176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d70100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009930 tx=0x7f4d6402a420 comp rx=0 tx=0).stop 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- 172.21.15.86:0/1419109176 shutdown_connections 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 --2- 172.21.15.86:0/1419109176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d70101420 0x7f4d7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 --2- 172.21.15.86:0/1419109176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d70100ee0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 --2- 172.21.15.86:0/1419109176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d70106a80 0x7f4d70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- 172.21.15.86:0/1419109176 >> 172.21.15.86:0/1419109176 conn(0x7f4d700fc7a0 msgr2=0x7f4d700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- 172.21.15.86:0/1419109176 shutdown_connections 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- 172.21.15.86:0/1419109176 wait complete. 2024-09-05T06:39:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 Processor -- start 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.897+0000 7f4d7542f640 1 -- start start 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d701991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d70101420 0x7f4d70199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d70106a80 0x7f4d701a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70110d30 con 0x7f4d70100a80 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70110bb0 con 0x7f4d70106a80 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d7542f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70110eb0 con 0x7f4d70101420 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d701991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d701991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37404/0 (socket says 172.21.15.86:37404) 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 -- 172.21.15.86:0/4200708795 learned_addr learned my addr 172.21.15.86:0/4200708795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6e7fc640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d70101420 0x7f4d70199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6f7fe640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d70106a80 0x7f4d701a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 -- 172.21.15.86:0/4200708795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d70101420 msgr2=0x7f4d70199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d70101420 0x7f4d70199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 -- 172.21.15.86:0/4200708795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d70106a80 msgr2=0x7f4d701a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d70106a80 0x7f4d701a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d701a47a0 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d6effd640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d701991e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c920 tx=0x7f4d6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.898+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60007c70 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.899+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d60007e10 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.899+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60018820 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.899+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d701a4970 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.899+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d701a4e30 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.900+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d70062900 con 0x7f4d70100a80 2024-09-05T06:39:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.901+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4d600189c0 con 0x7f4d70100a80 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.901+0000 7f4d57fff640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4d580a2c10 0x7f4d580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d6e7fc640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4d580a2c10 0x7f4d580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f4d600cf520 con 0x7f4d70100a80 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d57fff640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] conn(0x7f4d580add60 0x7f4d580b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d60012ec0 con 0x7f4d580add60 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f4d600d6050 con 0x7f4d70100a80 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d6f7fe640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] conn(0x7f4d580add60 0x7f4d580b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d6e7fc640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4d580a2c10 0x7f4d580a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.902+0000 7f4d6f7fe640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] conn(0x7f4d580add60 0x7f4d580b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.903+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== osd.6 v2:172.21.15.86:6818/1960864620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d60012ec0 con 0x7f4d580add60 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.916+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 --> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d70062db0 con 0x7f4d580add60 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.916+0000 7f4d57fff640 1 -- 172.21.15.86:0/4200708795 <== osd.6 v2:172.21.15.86:6818/1960864620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d70062db0 con 0x7f4d580add60 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] conn(0x7f4d580add60 msgr2=0x7f4d580b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] conn(0x7f4d580add60 0x7f4d580b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4d580a2c10 msgr2=0x7f4d580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f4d580a2c10 0x7f4d580a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d64000bc0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 msgr2=0x7f4d701991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 --2- 172.21.15.86:0/4200708795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70100a80 0x7f4d701991e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c920 tx=0x7f4d6000cde0 comp rx=0 tx=0).stop 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d6f7fe640 1 -- 172.21.15.86:0/4200708795 reap_dead start 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 shutdown_connections 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 >> 172.21.15.86:0/4200708795 conn(0x7f4d700fc7a0 msgr2=0x7f4d700fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 shutdown_connections 2024-09-05T06:39:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:39:59.917+0000 7f4d7542f640 1 -- 172.21.15.86:0/4200708795 wait complete. 2024-09-05T06:40:00.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:00 smithi096 ceph-mon[34468]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-05T06:40:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:00 smithi096 ceph-mon[34468]: overall HEALTH_OK 2024-09-05T06:40:00.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:00 smithi086 ceph-mon[30943]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-05T06:40:00.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:00 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T06:40:00.584 INFO:teuthology.orchestra.run.smithi086.stdout:34359738448 2024-09-05T06:40:00.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-05T06:40:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:00 smithi115 ceph-mon[34791]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-05T06:40:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:00 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T06:40:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:103079215170 2024-09-05T06:40:01.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-05T06:40:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 -- 172.21.15.86:0/997638764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 msgr2=0x7f93f8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 --2- 172.21.15.86:0/997638764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8103130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009ca0 tx=0x7f93f002a950 comp rx=0 tx=0).stop 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 -- 172.21.15.86:0/997638764 shutdown_connections 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 --2- 172.21.15.86:0/997638764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f810c180 0x7f93f810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 --2- 172.21.15.86:0/997638764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 0x7f93f810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 --2- 172.21.15.86:0/997638764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8103130 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 -- 172.21.15.86:0/997638764 >> 172.21.15.86:0/997638764 conn(0x7f93f80fc7d0 msgr2=0x7f93f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 -- 172.21.15.86:0/997638764 shutdown_connections 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.810+0000 7f93fcf78640 1 -- 172.21.15.86:0/997638764 wait complete. 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 Processor -- start 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 -- start start 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 0x7f93f8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:01.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f810c180 0x7f93f819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111ef0 con 0x7f93f8102d50 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111d70 con 0x7f93f8103670 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93fcf78640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112070 con 0x7f93f810c180 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37416/0 (socket says 172.21.15.86:37416) 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 -- 172.21.15.86:0/274878349 learned_addr learned my addr 172.21.15.86:0/274878349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 -- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f810c180 msgr2=0x7f93f819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f810c180 0x7f93f819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 -- 172.21.15.86:0/274878349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 msgr2=0x7f93f8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f5d9b640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 0x7f93f8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 0x7f93f8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f659c640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819d4f0 con 0x7f93f8102d50 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.811+0000 7f93f5d9b640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93f8103670 0x7f93f8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93f659c640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8198540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93f0005cc0 tx=0x7f93f002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0034040 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819d780 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93f0003e40 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0045480 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.812+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a50c0 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.813+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93f8062900 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.814+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f93f0037020 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93ef7fe640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f93e40a2c10 0x7f93e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93f5d9b640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f93e40a2c10 0x7f93e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f93f00f0a60 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93ef7fe640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] conn(0x7f93e40add60 0x7f93e40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93e40b08d0 con 0x7f93e40add60 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f93f00f0eb0 con 0x7f93f8102d50 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.815+0000 7f93f5d9b640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f93e40a2c10 0x7f93e40a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.816+0000 7f93f6d9d640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] conn(0x7f93e40add60 0x7f93e40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.816+0000 7f93f6d9d640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] conn(0x7f93e40add60 0x7f93e40b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.817+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== osd.2 v2:172.21.15.115:6800/1161034740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93e40b08d0 con 0x7f93e40add60 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.830+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 --> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93f8062d00 con 0x7f93e40add60 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93ef7fe640 1 -- 172.21.15.86:0/274878349 <== osd.2 v2:172.21.15.115:6800/1161034740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f93f8062d00 con 0x7f93e40add60 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] conn(0x7f93e40add60 msgr2=0x7f93e40b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] conn(0x7f93e40add60 0x7f93e40b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f93e40a2c10 msgr2=0x7f93e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f93e40a2c10 0x7f93e40a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0008040 comp rx=0 tx=0).stop 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 msgr2=0x7f93f8198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:01.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 --2- 172.21.15.86:0/274878349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8102d50 0x7f93f8198540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93f0005cc0 tx=0x7f93f002f650 comp rx=0 tx=0).stop 2024-09-05T06:40:01.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93f6d9d640 1 -- 172.21.15.86:0/274878349 reap_dead start 2024-09-05T06:40:01.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 shutdown_connections 2024-09-05T06:40:01.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 >> 172.21.15.86:0/274878349 conn(0x7f93f80fc7d0 msgr2=0x7f93f80fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:01.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 shutdown_connections 2024-09-05T06:40:01.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:01.831+0000 7f93fcf78640 1 -- 172.21.15.86:0/274878349 wait complete. 2024-09-05T06:40:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.276+0000 7fd812d3d640 1 -- 172.21.15.86:0/2327977475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 msgr2=0x7fd80c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.276+0000 7fd812d3d640 1 --2- 172.21.15.86:0/2327977475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c106810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009900 tx=0x7fd7f402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 -- 172.21.15.86:0/2327977475 shutdown_connections 2024-09-05T06:40:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 --2- 172.21.15.86:0/2327977475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd80c106f40 0x7fd80c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 --2- 172.21.15.86:0/2327977475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c106810 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 --2- 172.21.15.86:0/2327977475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd80c101e10 0x7fd80c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 -- 172.21.15.86:0/2327977475 >> 172.21.15.86:0/2327977475 conn(0x7fd80c0fd8d0 msgr2=0x7fd80c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 -- 172.21.15.86:0/2327977475 shutdown_connections 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 -- 172.21.15.86:0/2327977475 wait complete. 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 Processor -- start 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 -- start start 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.277+0000 7fd812d3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd80c101e10 0x7fd80c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd80c106f40 0x7fd80c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111ef0 con 0x7fd80c1027c0 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111d70 con 0x7fd80c101e10 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c112070 con 0x7fd80c106f40 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37424/0 (socket says 172.21.15.86:37424) 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 -- 172.21.15.86:0/3803016706 learned_addr learned my addr 172.21.15.86:0/3803016706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd8112da640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd80c106f40 0x7fd80c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd810ad9640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd80c101e10 0x7fd80c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 -- 172.21.15.86:0/3803016706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd80c106f40 msgr2=0x7fd80c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd80c106f40 0x7fd80c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 -- 172.21.15.86:0/3803016706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd80c101e10 msgr2=0x7fd80c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd80c101e10 0x7fd80c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c19d4a0 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd80bfff640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c198a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006010 tx=0x7fd7f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f403f030 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7f40035f0 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c19d730 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.278+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4035070 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.279+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1a5070 con 0x7fd80c1027c0 2024-09-05T06:40:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.279+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd80c062980 con 0x7fd80c1027c0 2024-09-05T06:40:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.281+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd7f4003b10 con 0x7fd80c1027c0 2024-09-05T06:40:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.281+0000 7fd809ffb640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7f80a2c10 0x7fd7f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd810ad9640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7f80a2c10 0x7fd7f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fd7f40f0700 con 0x7fd80c1027c0 2024-09-05T06:40:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd809ffb640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] conn(0x7fd7f80adce0 0x7fd7f80b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7f4001d30 con 0x7fd7f80adce0 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd7f40f0b50 con 0x7fd80c1027c0 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd8112da640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] conn(0x7fd7f80adce0 0x7fd7f80b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd810ad9640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7f80a2c10 0x7fd7f80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009f20 tx=0x7fd7fc0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.282+0000 7fd8112da640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] conn(0x7fd7f80adce0 0x7fd7f80b0180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.283+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== osd.9 v2:172.21.15.86:6826/174097683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7f4001d30 con 0x7fd7f80adce0 2024-09-05T06:40:02.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.297+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 --> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd80c062e00 con 0x7fd7f80adce0 2024-09-05T06:40:02.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.297+0000 7fd809ffb640 1 -- 172.21.15.86:0/3803016706 <== osd.9 v2:172.21.15.86:6826/174097683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd80c062e00 con 0x7fd7f80adce0 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] conn(0x7fd7f80adce0 msgr2=0x7fd7f80b0180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] conn(0x7fd7f80adce0 0x7fd7f80b0180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7f80a2c10 msgr2=0x7fd7f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd7f80a2c10 0x7fd7f80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009f20 tx=0x7fd7fc0094f0 comp rx=0 tx=0).stop 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 msgr2=0x7fd80c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 --2- 172.21.15.86:0/3803016706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd80c1027c0 0x7fd80c198a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006010 tx=0x7fd7f4004270 comp rx=0 tx=0).stop 2024-09-05T06:40:02.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd8112da640 1 -- 172.21.15.86:0/3803016706 reap_dead start 2024-09-05T06:40:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 shutdown_connections 2024-09-05T06:40:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 >> 172.21.15.86:0/3803016706 conn(0x7fd80c0fd8d0 msgr2=0x7fd80c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 shutdown_connections 2024-09-05T06:40:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.298+0000 7fd812d3d640 1 -- 172.21.15.86:0/3803016706 wait complete. 2024-09-05T06:40:02.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:02 smithi086 ceph-mon[30943]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-05T06:40:02.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:02 smithi096 ceph-mon[34468]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-05T06:40:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:02 smithi115 ceph-mon[34791]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-05T06:40:03.473 INFO:teuthology.orchestra.run.smithi086.stdout:261993005093 2024-09-05T06:40:03.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-05T06:40:03.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 -- 172.21.15.86:0/404925661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102790 msgr2=0x7fc5bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/404925661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102790 0x7fc5bc1067e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009900 tx=0x7fc5b002a410 comp rx=0 tx=0).stop 2024-09-05T06:40:03.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 -- 172.21.15.86:0/404925661 shutdown_connections 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/404925661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc106f10 0x7fc5bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/404925661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102790 0x7fc5bc1067e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/404925661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc101de0 0x7fc5bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 -- 172.21.15.86:0/404925661 >> 172.21.15.86:0/404925661 conn(0x7fc5bc0fd8c0 msgr2=0x7fc5bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 -- 172.21.15.86:0/404925661 shutdown_connections 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 -- 172.21.15.86:0/404925661 wait complete. 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.767+0000 7fc5c1d99640 1 Processor -- start 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 -- start start 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc101de0 0x7fc5bc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc102790 0x7fc5bc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 0x7fc5bc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:03.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc111de0 con 0x7fc5bc106f10 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc111c60 con 0x7fc5bc102790 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5c1d99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc111f60 con 0x7fc5bc101de0 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 0x7fc5bc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 0x7fc5bc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35170/0 (socket says 172.21.15.86:35170) 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 -- 172.21.15.86:0/1646937835 learned_addr learned my addr 172.21.15.86:0/1646937835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5baffd640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc102790 0x7fc5bc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bb7fe640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc101de0 0x7fc5bc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 -- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc101de0 msgr2=0x7fc5bc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc101de0 0x7fc5bc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 -- 172.21.15.86:0/1646937835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc102790 msgr2=0x7fc5bc198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc102790 0x7fc5bc198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bbfff640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc19d290 con 0x7fc5bc106f10 2024-09-05T06:40:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.768+0000 7fc5bb7fe640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5bc101de0 0x7fc5bc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5bbfff640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 0x7fc5bc19cbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400eaa0 tx=0x7fc5b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400cdb0 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b4005020 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc19d580 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40116d0 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.769+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc1a4e60 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.770+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5bc062980 con 0x7fc5bc106f10 2024-09-05T06:40:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.771+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5b40051d0 con 0x7fc5bc106f10 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.771+0000 7fc5b8ff9640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc5a40a2c10 0x7fc5a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.771+0000 7fc5bb7fe640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc5a40a2c10 0x7fc5a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.771+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fc5b40cf6a0 con 0x7fc5bc106f10 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5b8ff9640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] conn(0x7fc5a40add60 0x7fc5a40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5b4018ec0 con 0x7fc5a40add60 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc5b40d6050 con 0x7fc5bc106f10 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5bb7fe640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc5a40a2c10 0x7fc5a40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00baa0 tx=0x7fc5ac005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5baffd640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] conn(0x7fc5a40add60 0x7fc5a40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.772+0000 7fc5baffd640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] conn(0x7fc5a40add60 0x7fc5a40b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.773+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== osd.11 v2:172.21.15.115:6824/3733177277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5b4018ec0 con 0x7fc5a40add60 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.787+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 --> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5bc062e30 con 0x7fc5a40add60 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.787+0000 7fc5b8ff9640 1 -- 172.21.15.86:0/1646937835 <== osd.11 v2:172.21.15.115:6824/3733177277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5bc062e30 con 0x7fc5a40add60 2024-09-05T06:40:03.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.787+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] conn(0x7fc5a40add60 msgr2=0x7fc5a40b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.787+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] conn(0x7fc5a40add60 0x7fc5a40b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc5a40a2c10 msgr2=0x7fc5a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc5a40a2c10 0x7fc5a40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00baa0 tx=0x7fc5ac005d90 comp rx=0 tx=0).stop 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 msgr2=0x7fc5bc19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 --2- 172.21.15.86:0/1646937835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc106f10 0x7fc5bc19cbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400eaa0 tx=0x7fc5b400ef60 comp rx=0 tx=0).stop 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5bbfff640 1 -- 172.21.15.86:0/1646937835 reap_dead start 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 shutdown_connections 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 >> 172.21.15.86:0/1646937835 conn(0x7fc5bc0fd8c0 msgr2=0x7fc5bc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 shutdown_connections 2024-09-05T06:40:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:03.788+0000 7fc5c1d99640 1 -- 172.21.15.86:0/1646937835 wait complete. 2024-09-05T06:40:04.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:04 smithi096 ceph-mon[34468]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:40:04.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- 172.21.15.86:0/1620896349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c101df0 msgr2=0x7fed4c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 --2- 172.21.15.86:0/1620896349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c101df0 0x7fed4c1021d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009ca0 tx=0x7fed3c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- 172.21.15.86:0/1620896349 shutdown_connections 2024-09-05T06:40:04.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 --2- 172.21.15.86:0/1620896349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed4c106f20 0x7fed4c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 --2- 172.21.15.86:0/1620896349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 0x7fed4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 --2- 172.21.15.86:0/1620896349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c101df0 0x7fed4c1021d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- 172.21.15.86:0/1620896349 >> 172.21.15.86:0/1620896349 conn(0x7fed4c0fd8f0 msgr2=0x7fed4c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- 172.21.15.86:0/1620896349 shutdown_connections 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- 172.21.15.86:0/1620896349 wait complete. 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 Processor -- start 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.815+0000 7fed52eec640 1 -- start start 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed4c101df0 0x7fed4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 0x7fed4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 0x7fed4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c111ed0 con 0x7fed4c106f20 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c111d50 con 0x7fed4c1027a0 2024-09-05T06:40:04.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112050 con 0x7fed4c101df0 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 0x7fed4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 0x7fed4c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35150/0 (socket says 172.21.15.86:35150) 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 -- 172.21.15.86:0/2703318219 learned_addr learned my addr 172.21.15.86:0/2703318219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed4bfff640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 0x7fed4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed50c88640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed4c101df0 0x7fed4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 -- 172.21.15.86:0/2703318219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed4c101df0 msgr2=0x7fed4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed4c101df0 0x7fed4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 -- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 msgr2=0x7fed4c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 0x7fed4c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c1a6930 con 0x7fed4c106f20 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed4bfff640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed4c1027a0 0x7fed4c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed51489640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 0x7fed4c074f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fed44009fd0 tx=0x7fed4400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44011830 con 0x7fed4c106f20 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed44016070 con 0x7fed4c106f20 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c1a6c20 con 0x7fed4c106f20 2024-09-05T06:40:04.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.816+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44010de0 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.817+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c1a7110 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.817+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed4c062880 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fed440119d0 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fed380a2c10 0x7fed380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed50c88640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fed380a2c10 0x7fed380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fed440cf640 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] conn(0x7fed380add60 0x7fed380b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed44004450 con 0x7fed380add60 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.819+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fed440cfa90 con 0x7fed4c106f20 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.820+0000 7fed50c88640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fed380a2c10 0x7fed380a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005fa0 tx=0x7fed3c031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.820+0000 7fed4bfff640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] conn(0x7fed380add60 0x7fed380b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.820+0000 7fed4bfff640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] conn(0x7fed380add60 0x7fed380b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.821+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== osd.10 v2:172.21.15.96:6824/1196038229 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed44004450 con 0x7fed380add60 2024-09-05T06:40:04.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.834+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 --> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed4c062d00 con 0x7fed380add60 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed49ffb640 1 -- 172.21.15.86:0/2703318219 <== osd.10 v2:172.21.15.96:6824/1196038229 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed4c062d00 con 0x7fed380add60 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] conn(0x7fed380add60 msgr2=0x7fed380b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] conn(0x7fed380add60 0x7fed380b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fed380a2c10 msgr2=0x7fed380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fed380a2c10 0x7fed380a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005fa0 tx=0x7fed3c031000 comp rx=0 tx=0).stop 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 msgr2=0x7fed4c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 --2- 172.21.15.86:0/2703318219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed4c106f20 0x7fed4c074f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fed44009fd0 tx=0x7fed4400b8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed51489640 1 -- 172.21.15.86:0/2703318219 reap_dead start 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 shutdown_connections 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.835+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 >> 172.21.15.86:0/2703318219 conn(0x7fed4c0fd8f0 msgr2=0x7fed4c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.836+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 shutdown_connections 2024-09-05T06:40:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:02.836+0000 7fed52eec640 1 -- 172.21.15.86:0/2703318219 wait complete. 2024-09-05T06:40:04.573 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:04 smithi086 ceph-mon[30943]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:40:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:04 smithi115 ceph-mon[34791]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:40:05.240 INFO:teuthology.orchestra.run.smithi086.stdout:77309411400 2024-09-05T06:40:05.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-05T06:40:05.391 INFO:teuthology.orchestra.run.smithi086.stdout:283467841567 2024-09-05T06:40:05.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-05T06:40:05.823 INFO:teuthology.orchestra.run.smithi086.stdout:313532612635 2024-09-05T06:40:05.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-05T06:40:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 -- 172.21.15.86:0/1277575679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 msgr2=0x7f316c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 --2- 172.21.15.86:0/1277575679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c100ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3158009930 tx=0x7f315802a420 comp rx=0 tx=0).stop 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 -- 172.21.15.86:0/1277575679 shutdown_connections 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 --2- 172.21.15.86:0/1277575679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f316c101430 0x7f316c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 --2- 172.21.15.86:0/1277575679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c100ef0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 --2- 172.21.15.86:0/1277575679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c106a90 0x7f316c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 -- 172.21.15.86:0/1277575679 >> 172.21.15.86:0/1277575679 conn(0x7f316c0fc7d0 msgr2=0x7f316c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 -- 172.21.15.86:0/1277575679 shutdown_connections 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.158+0000 7f3172f62640 1 -- 172.21.15.86:0/1277575679 wait complete. 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 Processor -- start 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 -- start start 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c101430 0x7f316c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f316c106a90 0x7f316c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f316c110d60 con 0x7f316c100a90 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f316c110be0 con 0x7f316c101430 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3172f62640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f316c110ee0 con 0x7f316c106a90 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35194/0 (socket says 172.21.15.86:35194) 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 -- 172.21.15.86:0/2250826363 learned_addr learned my addr 172.21.15.86:0/2250826363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f31714ff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f316c106a90 0x7f316c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f316bfff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c101430 0x7f316c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 -- 172.21.15.86:0/2250826363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f316c106a90 msgr2=0x7f316c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f316c106a90 0x7f316c19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 -- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c101430 msgr2=0x7f316c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c101430 0x7f316c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f3170cfe640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c19c2b0 con 0x7f316c100a90 2024-09-05T06:40:06.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.159+0000 7f316bfff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f316c101430 0x7f316c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3170cfe640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c197300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3160009970 tx=0x7f3160009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316000d020 con 0x7f316c100a90 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c19c5a0 con 0x7f316c100a90 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f316000d1c0 con 0x7f316c100a90 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316000cc50 con 0x7f316c100a90 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.160+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c06a790 con 0x7f316c100a90 2024-09-05T06:40:06.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.161+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f316c06ad40 con 0x7f316c100a90 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.162+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f316000c460 con 0x7f316c100a90 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f3169ffb640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f315c0a2c10 0x7f315c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f31600d11e0 con 0x7f316c100a90 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f316bfff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f315c0a2c10 0x7f315c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f3169ffb640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] conn(0x7f315c0add60 0x7f315c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f315c0b08d0 con 0x7f315c0add60 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.163+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f31600d1630 con 0x7f316c100a90 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.164+0000 7f316bfff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f315c0a2c10 0x7f315c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3158000c00 tx=0x7f3158031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.164+0000 7f31714ff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] conn(0x7f315c0add60 0x7f315c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.164+0000 7f31714ff640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] conn(0x7f315c0add60 0x7f315c0b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.165+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== osd.7 v2:172.21.15.96:6816/2748292566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315c0b08d0 con 0x7f315c0add60 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.178+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 --> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f316c06b140 con 0x7f315c0add60 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3169ffb640 1 -- 172.21.15.86:0/2250826363 <== osd.7 v2:172.21.15.96:6816/2748292566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f316c06b140 con 0x7f315c0add60 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] conn(0x7f315c0add60 msgr2=0x7f315c0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] conn(0x7f315c0add60 0x7f315c0b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f315c0a2c10 msgr2=0x7f315c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f315c0a2c10 0x7f315c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3158000c00 tx=0x7f3158031400 comp rx=0 tx=0).stop 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 msgr2=0x7f316c197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 --2- 172.21.15.86:0/2250826363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f316c100a90 0x7f316c197300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3160009970 tx=0x7f3160009e30 comp rx=0 tx=0).stop 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f31714ff640 1 -- 172.21.15.86:0/2250826363 reap_dead start 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 shutdown_connections 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 >> 172.21.15.86:0/2250826363 conn(0x7f316c0fc7d0 msgr2=0x7f316c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 shutdown_connections 2024-09-05T06:40:06.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.179+0000 7f3172f62640 1 -- 172.21.15.86:0/2250826363 wait complete. 2024-09-05T06:40:06.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:06 smithi086 ceph-mon[30943]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-05T06:40:06.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:06 smithi096 ceph-mon[34468]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-05T06:40:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:06 smithi115 ceph-mon[34791]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-05T06:40:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- 172.21.15.86:0/682020503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 msgr2=0x7f6be81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 --2- 172.21.15.86:0/682020503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be81029b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009900 tx=0x7f6bd402a480 comp rx=0 tx=0).stop 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- 172.21.15.86:0/682020503 shutdown_connections 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 --2- 172.21.15.86:0/682020503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8102ef0 0x7f6be810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 --2- 172.21.15.86:0/682020503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be81029b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 --2- 172.21.15.86:0/682020503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6be8100e20 0x7f6be8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- 172.21.15.86:0/682020503 >> 172.21.15.86:0/682020503 conn(0x7f6be80fc7d0 msgr2=0x7f6be80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- 172.21.15.86:0/682020503 shutdown_connections 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- 172.21.15.86:0/682020503 wait complete. 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 Processor -- start 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.665+0000 7f6beebb0640 1 -- start start 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8100e20 0x7f6be81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6be8102ef0 0x7f6be819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8111fd0 con 0x7f6be8102570 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8111e50 con 0x7f6be8102ef0 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8112150 con 0x7f6be8100e20 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35214/0 (socket says 172.21.15.86:35214) 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 -- 172.21.15.86:0/2340895567 learned_addr learned my addr 172.21.15.86:0/2340895567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6bec94c640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8100e20 0x7f6be81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6bed14d640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6be8102ef0 0x7f6be819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 -- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8100e20 msgr2=0x7f6be81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8100e20 0x7f6be81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 -- 172.21.15.86:0/2340895567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6be8102ef0 msgr2=0x7f6be819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6be8102ef0 0x7f6be819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be819d480 con 0x7f6be8102570 2024-09-05T06:40:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be7fff640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be8198a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4006010 tx=0x7f6bd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd403f070 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bd4003aa0 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be819d710 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.666+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4035030 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.667+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81a5050 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.667+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6be8062980 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.669+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6bd4042070 con 0x7f6be8102570 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.669+0000 7f6be5ffb640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6bd80a2c90 0x7f6bd80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.669+0000 7f6bec94c640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6bd80a2c90 0x7f6bd80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.669+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f6bd40f04b0 con 0x7f6be8102570 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6be5ffb640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] conn(0x7f6bd80adde0 0x7f6bd80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bd4038e90 con 0x7f6bd80adde0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6bd40f0900 con 0x7f6be8102570 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6bec94c640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6bd80a2c90 0x7f6bd80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009990 tx=0x7f6bdc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6bed14d640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] conn(0x7f6bd80adde0 0x7f6bd80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.670+0000 7f6bed14d640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] conn(0x7f6bd80adde0 0x7f6bd80b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.671+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== osd.5 v2:172.21.15.115:6808/464616540 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bd4038e90 con 0x7f6bd80adde0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.684+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 --> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6be8062d80 con 0x7f6bd80adde0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6be5ffb640 1 -- 172.21.15.86:0/2340895567 <== osd.5 v2:172.21.15.115:6808/464616540 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6be8062d80 con 0x7f6bd80adde0 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] conn(0x7f6bd80adde0 msgr2=0x7f6bd80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] conn(0x7f6bd80adde0 0x7f6bd80b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6bd80a2c90 msgr2=0x7f6bd80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f6bd80a2c90 0x7f6bd80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009990 tx=0x7f6bdc009110 comp rx=0 tx=0).stop 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 msgr2=0x7f6be8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 --2- 172.21.15.86:0/2340895567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6be8102570 0x7f6be8198a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4006010 tx=0x7f6bd40042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6bed14d640 1 -- 172.21.15.86:0/2340895567 reap_dead start 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 shutdown_connections 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.685+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 >> 172.21.15.86:0/2340895567 conn(0x7f6be80fc7d0 msgr2=0x7f6be80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:06.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.686+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 shutdown_connections 2024-09-05T06:40:06.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:06.686+0000 7f6beebb0640 1 -- 172.21.15.86:0/2340895567 wait complete. 2024-09-05T06:40:07.378 INFO:teuthology.orchestra.run.smithi086.stdout:201863462959 2024-09-05T06:40:07.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-05T06:40:07.926 INFO:teuthology.orchestra.run.smithi086.stdout:180388626484 2024-09-05T06:40:07.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-05T06:40:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- 172.21.15.86:0/2912099117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8100a70 msgr2=0x7f52b8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 --2- 172.21.15.86:0/2912099117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8100a70 0x7f52b8100ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009930 tx=0x7f52a402a420 comp rx=0 tx=0).stop 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- 172.21.15.86:0/2912099117 shutdown_connections 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 --2- 172.21.15.86:0/2912099117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8101410 0x7f52b810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 --2- 172.21.15.86:0/2912099117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8100a70 0x7f52b8100ed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 --2- 172.21.15.86:0/2912099117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52b8106a70 0x7f52b8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- 172.21.15.86:0/2912099117 >> 172.21.15.86:0/2912099117 conn(0x7f52b80fc790 msgr2=0x7f52b80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- 172.21.15.86:0/2912099117 shutdown_connections 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- 172.21.15.86:0/2912099117 wait complete. 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 Processor -- start 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 -- start start 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.244+0000 7f52bef2b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8100a70 0x7f52b81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 0x7f52b8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52b8106a70 0x7f52b819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111e40 con 0x7f52b8101410 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111cc0 con 0x7f52b8106a70 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111fc0 con 0x7f52b8100a70 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 0x7f52b8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 0x7f52b8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35232/0 (socket says 172.21.15.86:35232) 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 -- 172.21.15.86:0/3087875174 learned_addr learned my addr 172.21.15.86:0/3087875174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bd4c8640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52b8106a70 0x7f52b819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bccc7640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8100a70 0x7f52b81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 -- 172.21.15.86:0/3087875174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8100a70 msgr2=0x7f52b81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8100a70 0x7f52b81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 -- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52b8106a70 msgr2=0x7f52b819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52b8106a70 0x7f52b819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819d370 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bccc7640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8100a70 0x7f52b81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b7fff640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 0x7f52b8198900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004770 tx=0x7f52a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a403f070 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819d600 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52a403b460 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.245+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4035070 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.246+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a4f40 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.246+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52b8062980 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f52a403b610 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52b5ffb640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f52a80a2c10 0x7f52a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52bccc7640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f52a80a2c10 0x7f52a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f52a40ef950 con 0x7f52b8101410 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52b5ffb640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] conn(0x7f52a80add60 0x7f52a80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.248+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52a80b08d0 con 0x7f52a80add60 2024-09-05T06:40:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.249+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f52a40f42c0 con 0x7f52b8101410 2024-09-05T06:40:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.249+0000 7f52bccc7640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f52a80a2c10 0x7f52a80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005fd0 tx=0x7f52ac0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.249+0000 7f52bd4c8640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] conn(0x7f52a80add60 0x7f52a80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.249+0000 7f52bd4c8640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] conn(0x7f52a80add60 0x7f52a80b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.250+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== osd.4 v2:172.21.15.96:6808/208598279 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52a80b08d0 con 0x7f52a80add60 2024-09-05T06:40:08.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.263+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 --> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52b8062d80 con 0x7f52a80add60 2024-09-05T06:40:08.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52b5ffb640 1 -- 172.21.15.86:0/3087875174 <== osd.4 v2:172.21.15.96:6808/208598279 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f52b8062d80 con 0x7f52a80add60 2024-09-05T06:40:08.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] conn(0x7f52a80add60 msgr2=0x7f52a80b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] conn(0x7f52a80add60 0x7f52a80b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f52a80a2c10 msgr2=0x7f52a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f52a80a2c10 0x7f52a80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005fd0 tx=0x7f52ac0094d0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 msgr2=0x7f52b8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 --2- 172.21.15.86:0/3087875174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b8101410 0x7f52b8198900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004770 tx=0x7f52a40042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bd4c8640 1 -- 172.21.15.86:0/3087875174 reap_dead start 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 shutdown_connections 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.264+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 >> 172.21.15.86:0/3087875174 conn(0x7f52b80fc790 msgr2=0x7f52b80fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.265+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 shutdown_connections 2024-09-05T06:40:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.265+0000 7f52bef2b640 1 -- 172.21.15.86:0/3087875174 wait complete. 2024-09-05T06:40:08.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:08 smithi086 ceph-mon[30943]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T06:40:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 -- 172.21.15.86:0/2070786054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 msgr2=0x7ff21c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 --2- 172.21.15.86:0/2070786054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c100ee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009930 tx=0x7ff20c02a420 comp rx=0 tx=0).stop 2024-09-05T06:40:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 -- 172.21.15.86:0/2070786054 shutdown_connections 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 --2- 172.21.15.86:0/2070786054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c101420 0x7ff21c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 --2- 172.21.15.86:0/2070786054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c100ee0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 --2- 172.21.15.86:0/2070786054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c106a80 0x7ff21c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 -- 172.21.15.86:0/2070786054 >> 172.21.15.86:0/2070786054 conn(0x7ff21c0fc7a0 msgr2=0x7ff21c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 -- 172.21.15.86:0/2070786054 shutdown_connections 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.462+0000 7ff22401b640 1 -- 172.21.15.86:0/2070786054 wait complete. 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 Processor -- start 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 -- start start 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c101420 0x7ff21c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c106a80 0x7ff21c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c110d80 con 0x7ff21c100a80 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c110c00 con 0x7ff21c101420 2024-09-05T06:40:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff22401b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c110f00 con 0x7ff21c106a80 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35250/0 (socket says 172.21.15.86:35250) 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 -- 172.21.15.86:0/4286419332 learned_addr learned my addr 172.21.15.86:0/4286419332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff2215b6640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c101420 0x7ff21c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff2225b8640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c106a80 0x7ff21c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 -- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c106a80 msgr2=0x7ff21c19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c106a80 0x7ff21c19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 -- 172.21.15.86:0/4286419332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c101420 msgr2=0x7ff21c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c101420 0x7ff21c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff221db7640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c19c230 con 0x7ff21c100a80 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff2215b6640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff21c101420 0x7ff21c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.463+0000 7ff2225b8640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff21c106a80 0x7ff21c19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff221db7640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c197280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff21800e9f0 tx=0x7ff21800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21800cd80 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c19c520 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2180108e0 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218018920 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.464+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a3e00 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.465+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff21c062980 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.466+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff218010a90 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff212ffd640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff2080a2c10 0x7ff2080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff2215b6640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff2080a2c10 0x7ff2080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7ff2180d00e0 con 0x7ff21c100a80 2024-09-05T06:40:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff212ffd640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] conn(0x7ff2080add60 0x7ff2080b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2080b08d0 con 0x7ff2080add60 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff2180d0530 con 0x7ff21c100a80 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.467+0000 7ff2215b6640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff2080a2c10 0x7ff2080a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff20c02a940 tx=0x7ff20c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.468+0000 7ff2225b8640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] conn(0x7ff2080add60 0x7ff2080b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.468+0000 7ff2225b8640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] conn(0x7ff2080add60 0x7ff2080b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.469+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== osd.8 v2:172.21.15.115:6816/3761965299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2080b08d0 con 0x7ff2080add60 2024-09-05T06:40:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.482+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 --> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff21c062d80 con 0x7ff2080add60 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.483+0000 7ff212ffd640 1 -- 172.21.15.86:0/4286419332 <== osd.8 v2:172.21.15.115:6816/3761965299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff21c062d80 con 0x7ff2080add60 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.483+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] conn(0x7ff2080add60 msgr2=0x7ff2080b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.483+0000 7ff22401b640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] conn(0x7ff2080add60 0x7ff2080b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff2080a2c10 msgr2=0x7ff2080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7ff2080a2c10 0x7ff2080a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff20c02a940 tx=0x7ff20c005fd0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 msgr2=0x7ff21c197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 --2- 172.21.15.86:0/4286419332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21c100a80 0x7ff21c197280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff21800e9f0 tx=0x7ff21800eeb0 comp rx=0 tx=0).stop 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff2225b8640 1 -- 172.21.15.86:0/4286419332 reap_dead start 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 shutdown_connections 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 >> 172.21.15.86:0/4286419332 conn(0x7ff21c0fc7a0 msgr2=0x7ff21c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 shutdown_connections 2024-09-05T06:40:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:08.484+0000 7ff22401b640 1 -- 172.21.15.86:0/4286419332 wait complete. 2024-09-05T06:40:08.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:08 smithi096 ceph-mon[34468]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T06:40:08.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:08 smithi115 ceph-mon[34791]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-05T06:40:09.183 INFO:teuthology.orchestra.run.smithi086.stdout:231928234027 2024-09-05T06:40:09.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-05T06:40:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:09.608 INFO:teuthology.orchestra.run.smithi086.stdout:150323855417 2024-09-05T06:40:09.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-05T06:40:10.242 INFO:teuthology.orchestra.run.smithi086.stdout:133143986239 2024-09-05T06:40:10.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-05T06:40:10.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:10 smithi086 ceph-mon[30943]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T06:40:10.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:10 smithi096 ceph-mon[34468]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T06:40:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:10 smithi115 ceph-mon[34791]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- 172.21.15.86:0/1201949143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0106a80 msgr2=0x7efcf0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 --2- 172.21.15.86:0/1201949143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0106a80 0x7efcf0106e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efce0009930 tx=0x7efce002a420 comp rx=0 tx=0).stop 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- 172.21.15.86:0/1201949143 shutdown_connections 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 --2- 172.21.15.86:0/1201949143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0101420 0x7efcf010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 --2- 172.21.15.86:0/1201949143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0100a80 0x7efcf0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 --2- 172.21.15.86:0/1201949143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0106a80 0x7efcf0106e60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- 172.21.15.86:0/1201949143 >> 172.21.15.86:0/1201949143 conn(0x7efcf00fc7a0 msgr2=0x7efcf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- 172.21.15.86:0/1201949143 shutdown_connections 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- 172.21.15.86:0/1201949143 wait complete. 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 Processor -- start 2024-09-05T06:40:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.812+0000 7efcf6f45640 1 -- start start 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0100a80 0x7efcf0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 0x7efcf0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0106a80 0x7efcf01a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0110cf0 con 0x7efcf0101420 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0110b70 con 0x7efcf0106a80 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0110e70 con 0x7efcf0100a80 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 0x7efcf0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 0x7efcf0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35262/0 (socket says 172.21.15.86:35262) 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 -- 172.21.15.86:0/3657065323 learned_addr learned my addr 172.21.15.86:0/3657065323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf4ce1640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0100a80 0x7efcf0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf54e2640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0106a80 0x7efcf01a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 -- 172.21.15.86:0/3657065323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0100a80 msgr2=0x7efcf0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0100a80 0x7efcf0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 -- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0106a80 msgr2=0x7efcf01a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0106a80 0x7efcf01a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf01a47c0 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf54e2640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcf0106a80 0x7efcf01a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efceffff640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 0x7efcf0199740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efce400c890 tx=0x7efce400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4004220 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce40043c0 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4004a00 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.813+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf01a4990 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.814+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf01a4e80 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.815+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcf0062900 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efce4004ba0 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcedffb640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efcd80a2c90 0x7efcd80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcf4ce1640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efcd80a2c90 0x7efcd80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7efce40cf150 con 0x7efcf0101420 2024-09-05T06:40:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcedffb640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] conn(0x7efcd80adde0 0x7efcd80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.816+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcd80b0950 con 0x7efcd80adde0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.817+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7efce40d6050 con 0x7efcf0101420 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.817+0000 7efcf4ce1640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efcd80a2c90 0x7efcd80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efce0009900 tx=0x7efce0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.817+0000 7efcf54e2640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] conn(0x7efcd80adde0 0x7efcd80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.817+0000 7efcf54e2640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] conn(0x7efcd80adde0 0x7efcd80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.818+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== osd.1 v2:172.21.15.96:6800/3435233109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcd80b0950 con 0x7efcd80adde0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.832+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 --> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efcf0062dd0 con 0x7efcd80adde0 2024-09-05T06:40:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.832+0000 7efcedffb640 1 -- 172.21.15.86:0/3657065323 <== osd.1 v2:172.21.15.96:6800/3435233109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efcf0062dd0 con 0x7efcd80adde0 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] conn(0x7efcd80adde0 msgr2=0x7efcd80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] conn(0x7efcd80adde0 0x7efcd80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efcd80a2c90 msgr2=0x7efcd80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efcd80a2c90 0x7efcd80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efce0009900 tx=0x7efce0000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 msgr2=0x7efcf0199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 --2- 172.21.15.86:0/3657065323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcf0101420 0x7efcf0199740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efce400c890 tx=0x7efce400cd50 comp rx=0 tx=0).stop 2024-09-05T06:40:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf54e2640 1 -- 172.21.15.86:0/3657065323 reap_dead start 2024-09-05T06:40:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 shutdown_connections 2024-09-05T06:40:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 >> 172.21.15.86:0/3657065323 conn(0x7efcf00fc7a0 msgr2=0x7efcf00fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 shutdown_connections 2024-09-05T06:40:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:10.833+0000 7efcf6f45640 1 -- 172.21.15.86:0/3657065323 wait complete. 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 -- 172.21.15.86:0/3080567918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 msgr2=0x7f22c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/3080567918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c810ea90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009900 tx=0x7f22b802a410 comp rx=0 tx=0).stop 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 -- 172.21.15.86:0/3080567918 shutdown_connections 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/3080567918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c810ea90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/3080567918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8102790 0x7f22c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/3080567918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8101de0 0x7f22c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 -- 172.21.15.86:0/3080567918 >> 172.21.15.86:0/3080567918 conn(0x7f22c80fd8c0 msgr2=0x7f22c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 -- 172.21.15.86:0/3080567918 shutdown_connections 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.508+0000 7f22ccdfc640 1 -- 172.21.15.86:0/3080567918 wait complete. 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 Processor -- start 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 -- start start 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8101de0 0x7f22c8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8102790 0x7f22c8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8111e40 con 0x7f22c8106f10 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8111cc0 con 0x7f22c8101de0 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22ccdfc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8111fc0 con 0x7f22c8102790 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35278/0 (socket says 172.21.15.86:35278) 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 -- 172.21.15.86:0/4058728782 learned_addr learned my addr 172.21.15.86:0/4058728782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c659c640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8101de0 0x7f22c8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c5d9b640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8102790 0x7f22c8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 -- 172.21.15.86:0/4058728782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8102790 msgr2=0x7f22c8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8102790 0x7f22c8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 -- 172.21.15.86:0/4058728782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8101de0 msgr2=0x7f22c8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8101de0 0x7f22c8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.509+0000 7f22c6d9d640 1 -- 172.21.15.86:0/4058728782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c819d2d0 con 0x7f22c8106f10 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22c6d9d640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009a20 tx=0x7f22b80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b803f030 con 0x7f22c8106f10 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c819d560 con 0x7f22c8106f10 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22b80044d0 con 0x7f22c8106f10 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8035070 con 0x7f22c8106f10 2024-09-05T06:40:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.510+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c81a4ea0 con 0x7f22c8106f10 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.511+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c81032c0 con 0x7f22c8106f10 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.512+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f22b8042030 con 0x7f22c8106f10 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.513+0000 7f22bf7fe640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 0x7f22b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.513+0000 7f22c659c640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 0x7f22b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.513+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f22b80efe50 con 0x7f22c8106f10 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.513+0000 7f22c659c640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 0x7f22b40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22c000a400 tx=0x7f22c000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.517+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22b80bc910 con 0x7f22c8106f10 2024-09-05T06:40:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.690+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f22c810a0f0 con 0x7f22c8106f10 2024-09-05T06:40:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.690+0000 7f22bf7fe640 1 -- 172.21.15.86:0/4058728782 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f22b80bc210 con 0x7f22c8106f10 2024-09-05T06:40:11.690 INFO:teuthology.orchestra.run.smithi086.stdout:261993005095 2024-09-05T06:40:11.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 msgr2=0x7f22b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 0x7f22b40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22c000a400 tx=0x7f22c000d040 comp rx=0 tx=0).stop 2024-09-05T06:40:11.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 msgr2=0x7f22c819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009a20 tx=0x7f22b80042e0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 shutdown_connections 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c8106f10 0x7f22c819cbf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22c8102790 0x7f22c8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f22b40a2c10 0x7f22b40a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 --2- 172.21.15.86:0/4058728782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22c8101de0 0x7f22c8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 >> 172.21.15.86:0/4058728782 conn(0x7f22c80fd8c0 msgr2=0x7f22c80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 shutdown_connections 2024-09-05T06:40:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.692+0000 7f22ccdfc640 1 -- 172.21.15.86:0/4058728782 wait complete. 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- 172.21.15.86:0/3001659313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde50102d30 msgr2=0x7fde50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 --2- 172.21.15.86:0/3001659313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde50102d30 0x7fde50103110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde4402a480 comp rx=0 tx=0).stop 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- 172.21.15.86:0/3001659313 shutdown_connections 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 --2- 172.21.15.86:0/3001659313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5010c160 0x7fde5010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 --2- 172.21.15.86:0/3001659313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50103650 0x7fde5010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 --2- 172.21.15.86:0/3001659313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde50102d30 0x7fde50103110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- 172.21.15.86:0/3001659313 >> 172.21.15.86:0/3001659313 conn(0x7fde500fc790 msgr2=0x7fde500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- 172.21.15.86:0/3001659313 shutdown_connections 2024-09-05T06:40:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- 172.21.15.86:0/3001659313 wait complete. 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 Processor -- start 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.715+0000 7fde56c05640 1 -- start start 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 0x7fde50198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde50103650 0x7fde501987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111e10 con 0x7fde5010c160 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111c90 con 0x7fde50102d30 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde56c05640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111f90 con 0x7fde50103650 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35290/0 (socket says 172.21.15.86:35290) 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 -- 172.21.15.86:0/2259735035 learned_addr learned my addr 172.21.15.86:0/2259735035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde4ffff640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde50103650 0x7fde501987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde549a1640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 0x7fde50198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 -- 172.21.15.86:0/2259735035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde50103650 msgr2=0x7fde501987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde50103650 0x7fde501987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 -- 172.21.15.86:0/2259735035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 msgr2=0x7fde50198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 0x7fde50198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 -- 172.21.15.86:0/2259735035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5019d220 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde549a1640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 0x7fde50198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.716+0000 7fde551a2640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde4800cce0 tx=0x7fde4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.717+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4800f450 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.717+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde4800faa0 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.717+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5019d510 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.717+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48018b70 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.717+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde501008c0 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.718+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5006b070 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.719+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fde48018d10 con 0x7fde5010c160 2024-09-05T06:40:11.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.720+0000 7fde4dffb640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 0x7fde400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.720+0000 7fde549a1640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 0x7fde400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.720+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fde480cf920 con 0x7fde5010c160 2024-09-05T06:40:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.721+0000 7fde549a1640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 0x7fde400a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde44031440 tx=0x7fde440095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.724+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde48014030 con 0x7fde5010c160 2024-09-05T06:40:11.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.896+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fde50106c70 con 0x7fde5010c160 2024-09-05T06:40:11.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.897+0000 7fde4dffb640 1 -- 172.21.15.86:0/2259735035 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fde4809c380 con 0x7fde5010c160 2024-09-05T06:40:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:313532612637 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 msgr2=0x7fde400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 0x7fde400a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde44031440 tx=0x7fde440095f0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 msgr2=0x7fde5019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fde4800cce0 tx=0x7fde4800a620 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 shutdown_connections 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde5010c160 0x7fde5019cb40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde50103650 0x7fde501987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fde400a2c10 0x7fde400a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 --2- 172.21.15.86:0/2259735035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102d30 0x7fde50198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.899+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 >> 172.21.15.86:0/2259735035 conn(0x7fde500fc790 msgr2=0x7fde500fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.900+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 shutdown_connections 2024-09-05T06:40:11.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:11.900+0000 7fde56c05640 1 -- 172.21.15.86:0/2259735035 wait complete. 2024-09-05T06:40:12.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4058728782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2259735035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-05T06:40:12.542 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:12 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-09-05T06:40:12.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4058728782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2259735035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-05T06:40:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:12 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-09-05T06:40:12.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005093 got 261993005095 for osd.9 2024-09-05T06:40:12.569 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-05T06:40:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4058728782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2259735035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-05T06:40:12.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:12 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-09-05T06:40:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:13.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T06:40:13.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:13 smithi086 ceph-mon[30943]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T06:40:13.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 -- 172.21.15.86:0/2811633505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 msgr2=0x7f9e6410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/2811633505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e6410e9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009cd0 tx=0x7f9e5c02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 -- 172.21.15.86:0/2811633505 shutdown_connections 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/2811633505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e6410e9c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/2811633505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e641019e0 0x7f9e64101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/2811633505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e64107af0 0x7f9e64107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 -- 172.21.15.86:0/2811633505 >> 172.21.15.86:0/2811633505 conn(0x7f9e640fd700 msgr2=0x7f9e640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 -- 172.21.15.86:0/2811633505 shutdown_connections 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 -- 172.21.15.86:0/2811633505 wait complete. 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.450+0000 7f9e68f9f640 1 Processor -- start 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 -- start start 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e641019e0 0x7f9e641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e64107af0 0x7f9e6419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64111dc0 con 0x7f9e64102380 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64111c40 con 0x7f9e641019e0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e68f9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64111f40 con 0x7f9e64107af0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51574/0 (socket says 172.21.15.86:51574) 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 -- 172.21.15.86:0/227194616 learned_addr learned my addr 172.21.15.86:0/227194616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e637fe640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e641019e0 0x7f9e641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e63fff640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e64107af0 0x7f9e6419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 -- 172.21.15.86:0/227194616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e64107af0 msgr2=0x7f9e6419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e64107af0 0x7f9e6419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 -- 172.21.15.86:0/227194616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e641019e0 msgr2=0x7f9e641982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e641019e0 0x7f9e641982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.451+0000 7f9e62ffd640 1 -- 172.21.15.86:0/227194616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6419d290 con 0x7f9e64102380 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e62ffd640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800b4a0 tx=0x7f9e5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58014070 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6419d580 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e58010ac0 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58017c30 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.452+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e641a4e60 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.453+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e64101e40 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.454+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e58017070 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.455+0000 7f9e60ff9640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 0x7f9e4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.455+0000 7f9e637fe640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 0x7f9e4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.455+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f9e580cec10 con 0x7f9e64102380 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.455+0000 7f9e637fe640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 0x7f9e4c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005fd0 tx=0x7f9e54005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.459+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e5809b750 con 0x7f9e64102380 2024-09-05T06:40:13.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T06:40:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:13 smithi096 ceph-mon[34468]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T06:40:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.633+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9e64103180 con 0x7f9e64102380 2024-09-05T06:40:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.633+0000 7f9e60ff9640 1 -- 172.21.15.86:0/227194616 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e5809b050 con 0x7f9e64102380 2024-09-05T06:40:13.633 INFO:teuthology.orchestra.run.smithi086.stdout:201863462960 2024-09-05T06:40:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 msgr2=0x7f9e4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 0x7f9e4c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005fd0 tx=0x7f9e54005850 comp rx=0 tx=0).stop 2024-09-05T06:40:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 msgr2=0x7f9e64198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800b4a0 tx=0x7f9e5800b960 comp rx=0 tx=0).stop 2024-09-05T06:40:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 shutdown_connections 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e64107af0 0x7f9e6419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e64102380 0x7f9e64198820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f9e4c0a2c10 0x7f9e4c0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 --2- 172.21.15.86:0/227194616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e641019e0 0x7f9e641982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 >> 172.21.15.86:0/227194616 conn(0x7f9e640fd700 msgr2=0x7f9e640fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.635+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 shutdown_connections 2024-09-05T06:40:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:13.636+0000 7f9e68f9f640 1 -- 172.21.15.86:0/227194616 wait complete. 2024-09-05T06:40:13.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T06:40:13.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-05T06:40:13.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-05T06:40:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-05T06:40:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-09-05T06:40:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T06:40:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:13 smithi115 ceph-mon[34791]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T06:40:13.936 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:13.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612637 for osd.11 2024-09-05T06:40:13.964 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 -- 172.21.15.86:0/1597577505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102ef0 msgr2=0x7fe5dc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 --2- 172.21.15.86:0/1597577505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc10da40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ca0 tx=0x7fe5d402a950 comp rx=0 tx=0).stop 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 -- 172.21.15.86:0/1597577505 shutdown_connections 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 --2- 172.21.15.86:0/1597577505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc10da40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 --2- 172.21.15.86:0/1597577505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc102570 0x7fe5dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 --2- 172.21.15.86:0/1597577505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc100e20 0x7fe5dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 -- 172.21.15.86:0/1597577505 >> 172.21.15.86:0/1597577505 conn(0x7fe5dc0fc7d0 msgr2=0x7fe5dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 -- 172.21.15.86:0/1597577505 shutdown_connections 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.228+0000 7fe5e389c640 1 -- 172.21.15.86:0/1597577505 wait complete. 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 Processor -- start 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 -- start start 2024-09-05T06:40:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc100e20 0x7fe5dc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110e30 con 0x7fe5dc102570 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110cb0 con 0x7fe5dc100e20 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e389c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110fb0 con 0x7fe5dc102ef0 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e0e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e0e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51592/0 (socket says 172.21.15.86:51592) 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e0e37640 1 -- 172.21.15.86:0/231552543 learned_addr learned my addr 172.21.15.86:0/231552543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e1638640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc100e20 0x7fe5dc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.229+0000 7fe5e1e39640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 -- 172.21.15.86:0/231552543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 msgr2=0x7fe5dc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 -- 172.21.15.86:0/231552543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc100e20 msgr2=0x7fe5dc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc100e20 0x7fe5dc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 -- 172.21.15.86:0/231552543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc1a6910 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e1e39640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e0e37640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c890 tx=0x7fe5cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004220 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5cc0043c0 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004a00 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1a6c00 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.230+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a7140 con 0x7fe5dc102570 2024-09-05T06:40:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.231+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc101200 con 0x7fe5dc102570 2024-09-05T06:40:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.232+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe5cc004ba0 con 0x7fe5dc102570 2024-09-05T06:40:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.233+0000 7fe5d27fc640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 0x7fe5c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.233+0000 7fe5e1638640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 0x7fe5c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.234+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fe5cc0cf650 con 0x7fe5dc102570 2024-09-05T06:40:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.234+0000 7fe5e1638640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 0x7fe5c80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80058c0 tx=0x7fe5d800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.237+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5cc014030 con 0x7fe5dc102570 2024-09-05T06:40:14.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.406+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe5dc0fe1b0 con 0x7fe5dc102570 2024-09-05T06:40:14.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.406+0000 7fe5d27fc640 1 -- 172.21.15.86:0/231552543 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5dc0fe1b0 con 0x7fe5dc102570 2024-09-05T06:40:14.408 INFO:teuthology.orchestra.run.smithi086.stdout:34359738451 2024-09-05T06:40:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 msgr2=0x7fe5c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 0x7fe5c80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80058c0 tx=0x7fe5d800a2d0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 msgr2=0x7fe5dc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c890 tx=0x7fe5cc00cd50 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 shutdown_connections 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5dc102ef0 0x7fe5dc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.408+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5dc102570 0x7fe5dc0fd1c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.409+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe5c80a2c10 0x7fe5c80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.409+0000 7fe5e389c640 1 --2- 172.21.15.86:0/231552543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc100e20 0x7fe5dc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.409+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 >> 172.21.15.86:0/231552543 conn(0x7fe5dc0fc7d0 msgr2=0x7fe5dc06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.409+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 shutdown_connections 2024-09-05T06:40:14.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:14.409+0000 7fe5e389c640 1 -- 172.21.15.86:0/231552543 wait complete. 2024-09-05T06:40:14.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:14 smithi086 ceph-mon[30943]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-09-05T06:40:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:14 smithi086 ceph-mon[30943]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-05T06:40:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:14 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/227194616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T06:40:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:14 smithi096 ceph-mon[34468]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-09-05T06:40:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:14 smithi096 ceph-mon[34468]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-05T06:40:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:14 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/227194616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T06:40:14.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:14 smithi115 ceph-mon[34791]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-09-05T06:40:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:14 smithi115 ceph-mon[34791]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-05T06:40:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:14 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/227194616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T06:40:15.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738451 for osd.0 2024-09-05T06:40:15.223 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:15.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:15 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/231552543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T06:40:15.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:15 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/231552543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T06:40:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:15 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/231552543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T06:40:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 -- 172.21.15.86:0/1653427653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 msgr2=0x7f44681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 --2- 172.21.15.86:0/1653427653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681067e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f445c009900 tx=0x7f445c02a410 comp rx=0 tx=0).stop 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 -- 172.21.15.86:0/1653427653 shutdown_connections 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 --2- 172.21.15.86:0/1653427653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 0x7f446810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 --2- 172.21.15.86:0/1653427653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681067e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 --2- 172.21.15.86:0/1653427653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 0x7f44681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 -- 172.21.15.86:0/1653427653 >> 172.21.15.86:0/1653427653 conn(0x7f44680fd8c0 msgr2=0x7f44680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 -- 172.21.15.86:0/1653427653 shutdown_connections 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.868+0000 7f446d3ae640 1 -- 172.21.15.86:0/1653427653 wait complete. 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 Processor -- start 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 -- start start 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 0x7f4468198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 0x7f446819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4468111e40 con 0x7f4468102790 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4468111cc0 con 0x7f4468106f10 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f446d3ae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4468111fc0 con 0x7f4468101de0 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f44667fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f44667fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51610/0 (socket says 172.21.15.86:51610) 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f44667fc640 1 -- 172.21.15.86:0/4160987222 learned_addr learned my addr 172.21.15.86:0/4160987222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.869+0000 7f4466ffd640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 0x7f4468198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44677fe640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 0x7f446819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 -- 172.21.15.86:0/4160987222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 msgr2=0x7f4468198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 0x7f4468198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 -- 172.21.15.86:0/4160987222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 msgr2=0x7f446819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 0x7f446819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 -- 172.21.15.86:0/4160987222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446819d410 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f44667fc640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f445c006010 tx=0x7f445c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c03f030 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446819d6a0 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a4fe0 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f445c0035f0 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.870+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c035070 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.871+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4454005290 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.872+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f445c031af0 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.873+0000 7f444ffff640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 0x7f44500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.873+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f445c0efe40 con 0x7f4468102790 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.873+0000 7f4466ffd640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 0x7f44500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.874+0000 7f4466ffd640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 0x7f44500a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f445800a400 tx=0x7f445800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:15.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:15.877+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f445c031de0 con 0x7f4468102790 2024-09-05T06:40:16.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.051+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4454005850 con 0x7f4468102790 2024-09-05T06:40:16.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.051+0000 7f444ffff640 1 -- 172.21.15.86:0/4160987222 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f445c0bc220 con 0x7f4468102790 2024-09-05T06:40:16.052 INFO:teuthology.orchestra.run.smithi086.stdout:77309411402 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 msgr2=0x7f44500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 0x7f44500a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f445800a400 tx=0x7f445800d040 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 msgr2=0x7f44681989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f445c006010 tx=0x7f445c004270 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 shutdown_connections 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44500a2c90 0x7f44500a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4468106f10 0x7f446819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4468102790 0x7f44681989a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 --2- 172.21.15.86:0/4160987222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4468101de0 0x7f4468198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 >> 172.21.15.86:0/4160987222 conn(0x7f44680fd8c0 msgr2=0x7f44680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:16.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 shutdown_connections 2024-09-05T06:40:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.054+0000 7f446d3ae640 1 -- 172.21.15.86:0/4160987222 wait complete. 2024-09-05T06:40:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 -- 172.21.15.86:0/676070729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 msgr2=0x7fb9a4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/676070729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a4106840 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb990009900 tx=0x7fb99002a410 comp rx=0 tx=0).stop 2024-09-05T06:40:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 -- 172.21.15.86:0/676070729 shutdown_connections 2024-09-05T06:40:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/676070729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9a4106f70 0x7fb9a410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/676070729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a4106840 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/676070729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 0x7fb9a41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 -- 172.21.15.86:0/676070729 >> 172.21.15.86:0/676070729 conn(0x7fb9a40fc5c0 msgr2=0x7fb9a40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 -- 172.21.15.86:0/676070729 shutdown_connections 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 -- 172.21.15.86:0/676070729 wait complete. 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.122+0000 7fb9ab13e640 1 Processor -- start 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 -- start start 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 0x7fb9a4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9a4106f70 0x7fb9a4101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4111f50 con 0x7fb9a4100eb0 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4111dd0 con 0x7fb9a4100500 2024-09-05T06:40:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41120d0 con 0x7fb9a4106f70 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51630/0 (socket says 172.21.15.86:51630) 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 -- 172.21.15.86:0/2699538713 learned_addr learned my addr 172.21.15.86:0/2699538713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a8eda640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 0x7fb9a4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 -- 172.21.15.86:0/2699538713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9a4106f70 msgr2=0x7fb9a4101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9a4106f70 0x7fb9a4101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 -- 172.21.15.86:0/2699538713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 msgr2=0x7fb9a4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 0x7fb9a4101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 -- 172.21.15.86:0/2699538713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a41a6a10 con 0x7fb9a4100eb0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a3fff640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb990031440 tx=0x7fb990003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99003f070 con 0x7fb9a4100eb0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a41a6ca0 con 0x7fb9a4100eb0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb990031d60 con 0x7fb9a4100eb0 2024-09-05T06:40:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.123+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990035030 con 0x7fb9a4100eb0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.124+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41a7180 con 0x7fb9a4100eb0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.125+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a4100960 con 0x7fb9a4100eb0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.126+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb990042070 con 0x7fb9a4100eb0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.127+0000 7fb9a1ffb640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 0x7fb9940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.127+0000 7fb9a8eda640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 0x7fb9940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.127+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fb9900f02d0 con 0x7fb9a4100eb0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.127+0000 7fb9a8eda640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 0x7fb9940a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4069fc0 tx=0x7fb998008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.130+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9900bcd00 con 0x7fb9a4100eb0 2024-09-05T06:40:16.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.302+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb9a410a0e0 con 0x7fb9a4100eb0 2024-09-05T06:40:16.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:16 smithi086 ceph-mon[30943]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-05T06:40:16.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:16 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4160987222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T06:40:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.302+0000 7fb9a1ffb640 1 -- 172.21.15.86:0/2699538713 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9900bc600 con 0x7fb9a4100eb0 2024-09-05T06:40:16.304 INFO:teuthology.orchestra.run.smithi086.stdout:103079215173 2024-09-05T06:40:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 msgr2=0x7fb9940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 0x7fb9940a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4069fc0 tx=0x7fb998008040 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 msgr2=0x7fb9a41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb990031440 tx=0x7fb990003d50 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 shutdown_connections 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9a4106f70 0x7fb9a4101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a4100eb0 0x7fb9a41019c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb9940a2c10 0x7fb9940a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 --2- 172.21.15.86:0/2699538713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4100500 0x7fb9a4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 >> 172.21.15.86:0/2699538713 conn(0x7fb9a40fc5c0 msgr2=0x7fb9a4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 shutdown_connections 2024-09-05T06:40:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:16.304+0000 7fb9ab13e640 1 -- 172.21.15.86:0/2699538713 wait complete. 2024-09-05T06:40:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:16 smithi096 ceph-mon[34468]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-05T06:40:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:16 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4160987222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T06:40:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:16 smithi115 ceph-mon[34791]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-05T06:40:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:16 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4160987222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T06:40:16.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411402 for osd.2 2024-09-05T06:40:16.853 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:17.208 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462960 for osd.7 2024-09-05T06:40:17.208 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:17.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2699538713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T06:40:17.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2699538713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T06:40:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2699538713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 -- 172.21.15.86:0/2105023917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 msgr2=0x7f685410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 --2- 172.21.15.86:0/2105023917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685410ea80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f684c009ca0 tx=0x7f684c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 -- 172.21.15.86:0/2105023917 shutdown_connections 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 --2- 172.21.15.86:0/2105023917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685410ea80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 --2- 172.21.15.86:0/2105023917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 0x7f68541067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 --2- 172.21.15.86:0/2105023917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 0x7f68541021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 -- 172.21.15.86:0/2105023917 >> 172.21.15.86:0/2105023917 conn(0x7f68540fd8b0 msgr2=0x7f68540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:18.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 -- 172.21.15.86:0/2105023917 shutdown_connections 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.298+0000 7f685c6ac640 1 -- 172.21.15.86:0/2105023917 wait complete. 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 Processor -- start 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 -- start start 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 0x7f68541982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 0x7f6854198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6854111e10 con 0x7f6854106f00 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6854111c90 con 0x7f6854101dd0 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685c6ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6854111f90 con 0x7f6854102780 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51640/0 (socket says 172.21.15.86:51640) 2024-09-05T06:40:18.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 -- 172.21.15.86:0/3123646768 learned_addr learned my addr 172.21.15.86:0/3123646768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685a448640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 0x7f68541982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f6859c47640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 0x7f6854198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 -- 172.21.15.86:0/3123646768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 msgr2=0x7f6854198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 0x7f6854198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 -- 172.21.15.86:0/3123646768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 msgr2=0x7f68541982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 0x7f68541982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.299+0000 7f685ac49640 1 -- 172.21.15.86:0/3123646768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685419d2a0 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f685ac49640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f684c009dc0 tx=0x7f684c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c003cf0 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f684c0044f0 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685419d530 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c030bf0 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.300+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68541a4e70 con 0x7f6854106f00 2024-09-05T06:40:18.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.301+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68541021b0 con 0x7f6854106f00 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.302+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f684c004040 con 0x7f6854106f00 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.302+0000 7f684b7fe640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 0x7f68400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.303+0000 7f685a448640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 0x7f68400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.303+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f684c0f0a20 con 0x7f6854106f00 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.303+0000 7f685a448640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 0x7f68400a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f685000bbe0 tx=0x7f685000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:18.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.307+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f684c0bd560 con 0x7f6854106f00 2024-09-05T06:40:18.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:18 smithi086 ceph-mon[30943]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.479+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f685410a0e0 con 0x7f6854106f00 2024-09-05T06:40:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.479+0000 7f684b7fe640 1 -- 172.21.15.86:0/3123646768 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f684c0bce60 con 0x7f6854106f00 2024-09-05T06:40:18.480 INFO:teuthology.orchestra.run.smithi086.stdout:180388626487 2024-09-05T06:40:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 msgr2=0x7f68400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 0x7f68400a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f685000bbe0 tx=0x7f685000b390 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 msgr2=0x7f685419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f684c009dc0 tx=0x7f684c032ff0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 shutdown_connections 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6854106f00 0x7f685419cbc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6854102780 0x7f6854198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f68400a2c10 0x7f68400a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 --2- 172.21.15.86:0/3123646768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6854101dd0 0x7f68541982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 >> 172.21.15.86:0/3123646768 conn(0x7f68540fd8b0 msgr2=0x7f68540fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 shutdown_connections 2024-09-05T06:40:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.481+0000 7f685c6ac640 1 -- 172.21.15.86:0/3123646768 wait complete. 2024-09-05T06:40:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.536+0000 7fb256208640 1 -- 172.21.15.86:0/2066984378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250101400 msgr2=0x7fb25010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.536+0000 7fb256208640 1 --2- 172.21.15.86:0/2066984378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250101400 0x7fb25010d930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb248009ca0 tx=0x7fb24802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 -- 172.21.15.86:0/2066984378 shutdown_connections 2024-09-05T06:40:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- 172.21.15.86:0/2066984378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250101400 0x7fb25010d930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- 172.21.15.86:0/2066984378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250100a60 0x7fb250100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- 172.21.15.86:0/2066984378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb250106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 -- 172.21.15.86:0/2066984378 >> 172.21.15.86:0/2066984378 conn(0x7fb2500fc780 msgr2=0x7fb2500feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 -- 172.21.15.86:0/2066984378 shutdown_connections 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 -- 172.21.15.86:0/2066984378 wait complete. 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 Processor -- start 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 -- start start 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250101400 0x7fb250198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.537+0000 7fb256208640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb25019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb256208640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb250111df0 con 0x7fb250100a60 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb256208640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb250111c70 con 0x7fb250101400 2024-09-05T06:40:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb256208640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb250111f70 con 0x7fb250106a60 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51654/0 (socket says 172.21.15.86:51654) 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 -- 172.21.15.86:0/2861126480 learned_addr learned my addr 172.21.15.86:0/2861126480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb247fff640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250101400 0x7fb250198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24ffff640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb25019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 -- 172.21.15.86:0/2861126480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 msgr2=0x7fb25019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb25019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 -- 172.21.15.86:0/2861126480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250101400 msgr2=0x7fb250198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250101400 0x7fb250198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 -- 172.21.15.86:0/2861126480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25019d300 con 0x7fb250100a60 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24ffff640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb25019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24f7fe640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c970 tx=0x7fb24000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240007bb0 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25019d5f0 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb240007d50 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24000f450 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.538+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a4ed0 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.539+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250100ec0 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.540+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb240004f00 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.541+0000 7fb24d7fa640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 0x7fb23c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.541+0000 7fb247fff640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 0x7fb23c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.541+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fb2400cf620 con 0x7fb250100a60 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.541+0000 7fb247fff640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 0x7fb23c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2500ffea0 tx=0x7fb23800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:18.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.544+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb240014030 con 0x7fb250100a60 2024-09-05T06:40:18.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:18 smithi096 ceph-mon[34468]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:18 smithi115 ceph-mon[34791]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:18.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.716+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb2501996b0 con 0x7fb250100a60 2024-09-05T06:40:18.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.716+0000 7fb24d7fa640 1 -- 172.21.15.86:0/2861126480 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb24009c050 con 0x7fb250100a60 2024-09-05T06:40:18.717 INFO:teuthology.orchestra.run.smithi086.stdout:231928234028 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 msgr2=0x7fb23c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 0x7fb23c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2500ffea0 tx=0x7fb23800b390 comp rx=0 tx=0).stop 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 msgr2=0x7fb250198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c970 tx=0x7fb24000ce30 comp rx=0 tx=0).stop 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 shutdown_connections 2024-09-05T06:40:18.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb250106a60 0x7fb25019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb23c0a2c10 0x7fb23c0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb250101400 0x7fb250198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 --2- 172.21.15.86:0/2861126480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb250100a60 0x7fb250198350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 >> 172.21.15.86:0/2861126480 conn(0x7fb2500fc780 msgr2=0x7fb2500fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 shutdown_connections 2024-09-05T06:40:18.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:18.718+0000 7fb256208640 1 -- 172.21.15.86:0/2861126480 wait complete. 2024-09-05T06:40:19.229 INFO:teuthology.orchestra.run.smithi086.stdout:55834574926 2024-09-05T06:40:19.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-05T06:40:19.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3123646768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T06:40:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2861126480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T06:40:19.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3123646768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T06:40:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2861126480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T06:40:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.401+0000 7f2b1e50b640 1 -- 172.21.15.86:0/1845138562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102570 msgr2=0x7f2b181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.401+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/1845138562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102570 0x7f2b181029b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009900 tx=0x7f2b0c02a480 comp rx=0 tx=0).stop 2024-09-05T06:40:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.401+0000 7f2b1e50b640 1 -- 172.21.15.86:0/1845138562 shutdown_connections 2024-09-05T06:40:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.401+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/1845138562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102ef0 0x7f2b1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/1845138562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102570 0x7f2b181029b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/1845138562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 0x7f2b18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 -- 172.21.15.86:0/1845138562 >> 172.21.15.86:0/1845138562 conn(0x7f2b180fc7d0 msgr2=0x7f2b180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 -- 172.21.15.86:0/1845138562 shutdown_connections 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 -- 172.21.15.86:0/1845138562 wait complete. 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 Processor -- start 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.402+0000 7f2b1e50b640 1 -- start start 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 0x7f2b18198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 0x7f2b181989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18110e50 con 0x7f2b18102ef0 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18110cd0 con 0x7f2b18100e20 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18110fd0 con 0x7f2b18102570 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51670/0 (socket says 172.21.15.86:51670) 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 -- 172.21.15.86:0/2267776632 learned_addr learned my addr 172.21.15.86:0/2267776632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b177fe640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 0x7f2b181989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b17fff640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 0x7f2b18198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stdout:150323855419 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 -- 172.21.15.86:0/2267776632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 msgr2=0x7f2b181989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 0x7f2b181989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 -- 172.21.15.86:0/2267776632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 msgr2=0x7f2b18198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 0x7f2b18198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 -- 172.21.15.86:0/2267776632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1819d420 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b177fe640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 0x7f2b181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1caa8640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b970 tx=0x7f2b1000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10019070 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1819d710 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b10014440 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100075d0 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.403+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181a4ff0 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.404+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18101200 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.406+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2b1001c070 con 0x7f2b18102ef0 2024-09-05T06:40:19.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.406+0000 7f2b157fa640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 0x7f2b000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.406+0000 7f2b17fff640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 0x7f2b000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.406+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f2b100cef40 con 0x7f2b18102ef0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.407+0000 7f2b17fff640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 0x7f2b000a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b080058c0 tx=0x7f2b0800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.410+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b1009ba80 con 0x7f2b18102ef0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.592+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2b1810a0a0 con 0x7f2b18102ef0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.592+0000 7f2b157fa640 1 -- 172.21.15.86:0/2267776632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2b1009b380 con 0x7f2b18102ef0 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 msgr2=0x7f2b000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 0x7f2b000a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b080058c0 tx=0x7f2b0800a4c0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 msgr2=0x7f2b1819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b970 tx=0x7f2b1000be30 comp rx=0 tx=0).stop 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 shutdown_connections 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b18102ef0 0x7f2b1819cd40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b18102570 0x7f2b181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f2b000a2c10 0x7f2b000a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 --2- 172.21.15.86:0/2267776632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b18100e20 0x7f2b18198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 >> 172.21.15.86:0/2267776632 conn(0x7f2b180fc7d0 msgr2=0x7f2b1806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 shutdown_connections 2024-09-05T06:40:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:19.594+0000 7f2b1e50b640 1 -- 172.21.15.86:0/2267776632 wait complete. 2024-09-05T06:40:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3123646768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T06:40:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2861126480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T06:40:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:20.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626484 got 180388626487 for osd.6 2024-09-05T06:40:20.507 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:20.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:20 smithi096 ceph-mon[34468]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-05T06:40:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:20 smithi096 ceph-mon[34468]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-05T06:40:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:20 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T06:40:20.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:20 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2267776632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T06:40:20.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:20 smithi115 ceph-mon[34791]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-05T06:40:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:20 smithi115 ceph-mon[34791]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-05T06:40:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:20 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T06:40:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:20 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2267776632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T06:40:20.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:20 smithi086 ceph-mon[30943]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-05T06:40:20.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:20 smithi086 ceph-mon[30943]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-05T06:40:20.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:20 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T06:40:20.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:20 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2267776632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T06:40:20.902 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855417 got 150323855419 for osd.5 2024-09-05T06:40:20.902 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 -- 172.21.15.86:0/1788910953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0102de0 msgr2=0x7efdf01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 --2- 172.21.15.86:0/1788910953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0102de0 0x7efdf01031c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efde8009900 tx=0x7efde802a480 comp rx=0 tx=0).stop 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 -- 172.21.15.86:0/1788910953 shutdown_connections 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 --2- 172.21.15.86:0/1788910953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 0x7efdf010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 --2- 172.21.15.86:0/1788910953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0103700 0x7efdf010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 --2- 172.21.15.86:0/1788910953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0102de0 0x7efdf01031c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 -- 172.21.15.86:0/1788910953 >> 172.21.15.86:0/1788910953 conn(0x7efdf00fc7a0 msgr2=0x7efdf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 -- 172.21.15.86:0/1788910953 shutdown_connections 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.358+0000 7efdf5876640 1 -- 172.21.15.86:0/1788910953 wait complete. 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 Processor -- start 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 -- start start 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 0x7efdf0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 0x7efdf019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0111ed0 con 0x7efdf0103700 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0111d50 con 0x7efdf0102de0 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdf5876640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0112050 con 0x7efdf010c180 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.359+0000 7efdee7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdeeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 0x7efdf0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdeeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 0x7efdf0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45568/0 (socket says 172.21.15.86:45568) 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51688/0 (socket says 172.21.15.86:51688) 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdeeffd640 1 -- 172.21.15.86:0/1568847812 learned_addr learned my addr 172.21.15.86:0/1568847812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdef7fe640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 0x7efdf019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 -- 172.21.15.86:0/1568847812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 msgr2=0x7efdf019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 0x7efdf019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 -- 172.21.15.86:0/1568847812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 msgr2=0x7efdf0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 0x7efdf0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 -- 172.21.15.86:0/1568847812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf019d3b0 con 0x7efdf0103700 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdee7fc640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdd800b3f0 tx=0x7efdd800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8015020 con 0x7efdf0103700 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf019d6a0 con 0x7efdf0103700 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a4f80 con 0x7efdf0103700 2024-09-05T06:40:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.360+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdd8011ce0 con 0x7efdf0103700 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.361+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8010d80 con 0x7efdf0103700 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.362+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf01031c0 con 0x7efdf0103700 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.363+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efdd8018020 con 0x7efdf0103700 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.364+0000 7efdf4874640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 0x7efddc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.364+0000 7efdeeffd640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 0x7efddc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.364+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7efdd80cec80 con 0x7efdf0103700 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.364+0000 7efdeeffd640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 0x7efddc0a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efde80096c0 tx=0x7efde8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.368+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdd801f020 con 0x7efdf0103700 2024-09-05T06:40:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.541+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efdf0106cd0 con 0x7efdf0103700 2024-09-05T06:40:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.541+0000 7efdf4874640 1 -- 172.21.15.86:0/1568847812 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efdd809b7c0 con 0x7efdf0103700 2024-09-05T06:40:21.542 INFO:teuthology.orchestra.run.smithi086.stdout:133143986241 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 msgr2=0x7efddc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 0x7efddc0a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efde80096c0 tx=0x7efde8005a20 comp rx=0 tx=0).stop 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 msgr2=0x7efdf0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdd800b3f0 tx=0x7efdd800b8b0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 shutdown_connections 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdf010c180 0x7efdf019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdf0103700 0x7efdf0198940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7efddc0a2c90 0x7efddc0a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 --2- 172.21.15.86:0/1568847812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efdf0102de0 0x7efdf0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 >> 172.21.15.86:0/1568847812 conn(0x7efdf00fc7a0 msgr2=0x7efdf00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:21.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 shutdown_connections 2024-09-05T06:40:21.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.544+0000 7efdf5876640 1 -- 172.21.15.86:0/1568847812 wait complete. 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 -- 172.21.15.86:0/2804501349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa230102d30 msgr2=0x7fa230103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 --2- 172.21.15.86:0/2804501349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa230102d30 0x7fa230103110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa220009930 tx=0x7fa22002a440 comp rx=0 tx=0).stop 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 -- 172.21.15.86:0/2804501349 shutdown_connections 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 --2- 172.21.15.86:0/2804501349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2301082a0 0x7fa23010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 --2- 172.21.15.86:0/2804501349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 0x7fa230107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 --2- 172.21.15.86:0/2804501349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa230102d30 0x7fa230103110 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 -- 172.21.15.86:0/2804501349 >> 172.21.15.86:0/2804501349 conn(0x7fa2300fc7d0 msgr2=0x7fa2300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 -- 172.21.15.86:0/2804501349 shutdown_connections 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.590+0000 7fa236340640 1 -- 172.21.15.86:0/2804501349 wait complete. 2024-09-05T06:40:21.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 Processor -- start 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 -- start start 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 0x7fa230198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 0x7fa230198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa230111f90 con 0x7fa2301082a0 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa230111e10 con 0x7fa230102d30 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa236340640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa230112110 con 0x7fa2301036e0 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa2348dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa2348dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51714/0 (socket says 172.21.15.86:51714) 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa2348dd640 1 -- 172.21.15.86:0/4072358473 learned_addr learned my addr 172.21.15.86:0/4072358473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.591+0000 7fa22f7fe640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 0x7fa230198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa22ffff640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 0x7fa230198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 -- 172.21.15.86:0/4072358473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 msgr2=0x7fa230198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 0x7fa230198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 -- 172.21.15.86:0/4072358473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 msgr2=0x7fa230198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 0x7fa230198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 -- 172.21.15.86:0/4072358473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23019d4c0 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa22ffff640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 0x7fa230198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa2348dd640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa22800cce0 tx=0x7fa22800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800f450 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23019d7b0 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa22800faa0 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a5090 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.592+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228018b70 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.593+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23006a850 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.594+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa228018d10 con 0x7fa2301082a0 2024-09-05T06:40:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.595+0000 7fa22d7fa640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.595+0000 7fa22ffff640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.595+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fa2280cfb60 con 0x7fa2301082a0 2024-09-05T06:40:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.595+0000 7fa22ffff640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 0x7fa2180a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa220006010 tx=0x7fa220000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.598+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2280cfe20 con 0x7fa2301082a0 2024-09-05T06:40:21.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.769+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa230103110 con 0x7fa2301082a0 2024-09-05T06:40:21.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.769+0000 7fa22d7fa640 1 -- 172.21.15.86:0/4072358473 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa230103110 con 0x7fa2301082a0 2024-09-05T06:40:21.770 INFO:teuthology.orchestra.run.smithi086.stdout:283467841571 2024-09-05T06:40:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 msgr2=0x7fa2180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 0x7fa2180a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa220006010 tx=0x7fa220000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 msgr2=0x7fa23019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa22800cce0 tx=0x7fa22800a620 comp rx=0 tx=0).stop 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 shutdown_connections 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2301082a0 0x7fa23019cde0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2301036e0 0x7fa230198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 --2- 172.21.15.86:0/4072358473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa230102d30 0x7fa230198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 >> 172.21.15.86:0/4072358473 conn(0x7fa2300fc7d0 msgr2=0x7fa230105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 shutdown_connections 2024-09-05T06:40:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:21.771+0000 7fa236340640 1 -- 172.21.15.86:0/4072358473 wait complete. 2024-09-05T06:40:22.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986239 got 133143986241 for osd.4 2024-09-05T06:40:22.312 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:22 smithi096 ceph-mon[34468]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-05T06:40:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:22 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1568847812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T06:40:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:22 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4072358473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T06:40:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:22 smithi115 ceph-mon[34791]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-05T06:40:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:22 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1568847812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T06:40:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:22 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4072358473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T06:40:22.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:22 smithi086 ceph-mon[30943]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-05T06:40:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:22 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1568847812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T06:40:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:22 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4072358473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T06:40:22.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215170 got 103079215173 for osd.3 2024-09-05T06:40:22.963 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:23.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234027 got 231928234028 for osd.8 2024-09-05T06:40:23.343 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:23.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841567 got 283467841571 for osd.10 2024-09-05T06:40:23.796 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:23.839 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- 172.21.15.86:0/2200564733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 msgr2=0x7fce18106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 --2- 172.21.15.86:0/2200564733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18106810 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009900 tx=0x7fce0c02a410 comp rx=0 tx=0).stop 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- 172.21.15.86:0/2200564733 shutdown_connections 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 --2- 172.21.15.86:0/2200564733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce1810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 --2- 172.21.15.86:0/2200564733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18106810 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 --2- 172.21.15.86:0/2200564733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce181021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- 172.21.15.86:0/2200564733 >> 172.21.15.86:0/2200564733 conn(0x7fce180fd8d0 msgr2=0x7fce180ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- 172.21.15.86:0/2200564733 shutdown_connections 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- 172.21.15.86:0/2200564733 wait complete. 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 Processor -- start 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.309+0000 7fce1dee7640 1 -- start start 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce18111e50 con 0x7fce181027c0 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce18111cd0 con 0x7fce18101e10 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce1dee7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce18111fd0 con 0x7fce18106f40 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36394/0 (socket says 172.21.15.86:36394) 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce17fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 -- 172.21.15.86:0/3340516507 learned_addr learned my addr 172.21.15.86:0/3340516507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 -- 172.21.15.86:0/3340516507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 msgr2=0x7fce18074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce18074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 -- 172.21.15.86:0/3340516507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 msgr2=0x7fce180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 -- 172.21.15.86:0/3340516507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce181a6810 con 0x7fce181027c0 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce17fff640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce18074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce177fe640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:40:24.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.310+0000 7fce16ffd640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005940 tx=0x7fce0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.311+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c03f030 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.311+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181a6aa0 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.311+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0c003cb0 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.311+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c035070 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.311+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181a6f80 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.312+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce181021f0 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.313+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce0c042030 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.314+0000 7fce14ff9640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 0x7fce000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.314+0000 7fce177fe640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 0x7fce000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.314+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fce0c0efd00 con 0x7fce181027c0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.314+0000 7fce177fe640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 0x7fce000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce08005d00 tx=0x7fce0800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:24.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.318+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce0c0bc0e0 con 0x7fce181027c0 2024-09-05T06:40:24.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.484+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fce1802d7c0 con 0x7fce181027c0 2024-09-05T06:40:24.485 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:24 smithi086 ceph-mon[30943]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-05T06:40:24.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.485+0000 7fce14ff9640 1 -- 172.21.15.86:0/3340516507 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce0c0c03e0 con 0x7fce181027c0 2024-09-05T06:40:24.485 INFO:teuthology.orchestra.run.smithi086.stdout:55834574929 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 msgr2=0x7fce000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 0x7fce000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce08005d00 tx=0x7fce0800b410 comp rx=0 tx=0).stop 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 msgr2=0x7fce18074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005940 tx=0x7fce0c0042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 shutdown_connections 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce18106f40 0x7fce18074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce181027c0 0x7fce18074a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce000a2c10 0x7fce000a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 --2- 172.21.15.86:0/3340516507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce18101e10 0x7fce180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:24.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 >> 172.21.15.86:0/3340516507 conn(0x7fce180fd8d0 msgr2=0x7fce180fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:24.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 shutdown_connections 2024-09-05T06:40:24.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:24.488+0000 7fce1dee7640 1 -- 172.21.15.86:0/3340516507 wait complete. 2024-09-05T06:40:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:24 smithi096 ceph-mon[34468]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-05T06:40:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:24 smithi115 ceph-mon[34791]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-05T06:40:25.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574929 for osd.1 2024-09-05T06:40:25.139 DEBUG:teuthology.parallel:result is None 2024-09-05T06:40:25.139 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T06:40:25.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:40:25.395 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:25.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:25 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3340516507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T06:40:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:25 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3340516507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T06:40:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:25 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3340516507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 -- 172.21.15.86:0/250536306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 msgr2=0x7f51001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 --2- 172.21.15.86:0/250536306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f51001029b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009900 tx=0x7f50f002a480 comp rx=0 tx=0).stop 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 -- 172.21.15.86:0/250536306 shutdown_connections 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 --2- 172.21.15.86:0/250536306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 0x7f510010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 --2- 172.21.15.86:0/250536306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f51001029b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 --2- 172.21.15.86:0/250536306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 -- 172.21.15.86:0/250536306 >> 172.21.15.86:0/250536306 conn(0x7f51000fc7d0 msgr2=0x7f51000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 -- 172.21.15.86:0/250536306 shutdown_connections 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 -- 172.21.15.86:0/250536306 wait complete. 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.901+0000 7f51076b3640 1 Processor -- start 2024-09-05T06:40:25.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 -- start start 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110e50 con 0x7f5100102570 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110cd0 con 0x7f5100100e20 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f51076b3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110fd0 con 0x7f5100102ef0 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36408/0 (socket says 172.21.15.86:36408) 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 -- 172.21.15.86:0/907239803 learned_addr learned my addr 172.21.15.86:0/907239803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5105c50640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f510544f640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 -- 172.21.15.86:0/907239803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 msgr2=0x7f510019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 -- 172.21.15.86:0/907239803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 msgr2=0x7f5100197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 -- 172.21.15.86:0/907239803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510019c300 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f510544f640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.902+0000 7f5104c4e640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006010 tx=0x7f50f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.903+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f003f070 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.903+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510019c590 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.903+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f0003aa0 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.903+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0035030 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.903+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51001a3ed0 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.904+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51001021b0 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.905+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f50f0042070 con 0x7f5100102570 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.905+0000 7f50f67fc640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 0x7f50ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.906+0000 7f510544f640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 0x7f50ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.906+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f50f00f0640 con 0x7f5100102570 2024-09-05T06:40:25.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.906+0000 7f510544f640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 0x7f50ec0a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005c40 tx=0x7f50fc0082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:25.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:25.909+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50f00bd070 con 0x7f5100102570 2024-09-05T06:40:26.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.073+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5100062630 con 0x7f50ec0a2c10 2024-09-05T06:40:26.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.081+0000 7f50f67fc640 1 -- 172.21.15.86:0/907239803 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7f5100062630 con 0x7f50ec0a2c10 2024-09-05T06:40:26.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:26.086 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:40:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 msgr2=0x7f50ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 0x7f50ec0a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005c40 tx=0x7f50fc0082f0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 msgr2=0x7f5100197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006010 tx=0x7f50f00042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 shutdown_connections 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f50ec0a2c10 0x7f50ec0a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 --2- 172.21.15.86:0/907239803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 >> 172.21.15.86:0/907239803 conn(0x7f51000fc7d0 msgr2=0x7f51000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 shutdown_connections 2024-09-05T06:40:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:26.084+0000 7f51076b3640 1 -- 172.21.15.86:0/907239803 wait complete. 2024-09-05T06:40:26.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:26 smithi086 ceph-mon[30943]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T06:40:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:26 smithi115 ceph-mon[34791]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T06:40:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:26 smithi096 ceph-mon[34468]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T06:40:26.831 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-05T06:40:25.205712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661874,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":687,"num_read_kb":475,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388520,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553304,"statfs":{"total":1151940427776,"available":1151542583296,"internally_reserved":0,"allocated":14581760,"data_stored":8080976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.936539"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:18.931092+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:40:18.931092+0000","last_peered":"2024-09-05T06:40:18.931092+0000","last_clean":"2024-09-05T06:40:18.931092+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:40:18.931092+0000","last_undegraded":"2024-09-05T06:40:18.931092+0000","last_fullsized":"2024-09-05T06:40:18.931092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420916+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:40:13.420916+0000","last_peered":"2024-09-05T06:40:13.420916+0000","last_clean":"2024-09-05T06:40:13.420916+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:40:13.420916+0000","last_undegraded":"2024-09-05T06:40:13.420916+0000","last_fullsized":"2024-09-05T06:40:13.420916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.500473+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:40:13.500473+0000","last_peered":"2024-09-05T06:40:13.500473+0000","last_clean":"2024-09-05T06:40:13.500473+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:40:13.500473+0000","last_undegraded":"2024-09-05T06:40:13.500473+0000","last_fullsized":"2024-09-05T06:40:13.500473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420216+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:40:13.420216+0000","last_peered":"2024-09-05T06:40:13.420216+0000","last_clean":"2024-09-05T06:40:13.420216+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:40:13.420216+0000","last_undegraded":"2024-09-05T06:40:13.420216+0000","last_fullsized":"2024-09-05T06:40:13.420216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499751+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:40:13.499751+0000","last_peered":"2024-09-05T06:40:13.499751+0000","last_clean":"2024-09-05T06:40:13.499751+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:40:13.499751+0000","last_undegraded":"2024-09-05T06:40:13.499751+0000","last_fullsized":"2024-09-05T06:40:13.499751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272429+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:40:13.272429+0000","last_peered":"2024-09-05T06:40:13.272429+0000","last_clean":"2024-09-05T06:40:13.272429+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:40:13.272429+0000","last_undegraded":"2024-09-05T06:40:13.272429+0000","last_fullsized":"2024-09-05T06:40:13.272429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274275+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:40:13.274275+0000","last_peered":"2024-09-05T06:40:13.274275+0000","last_clean":"2024-09-05T06:40:13.274275+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:40:13.274275+0000","last_undegraded":"2024-09-05T06:40:13.274275+0000","last_fullsized":"2024-09-05T06:40:13.274275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420939+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:40:13.420939+0000","last_peered":"2024-09-05T06:40:13.420939+0000","last_clean":"2024-09-05T06:40:13.420939+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:40:13.420939+0000","last_undegraded":"2024-09-05T06:40:13.420939+0000","last_fullsized":"2024-09-05T06:40:13.420939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272466+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:40:13.272466+0000","last_peered":"2024-09-05T06:40:13.272466+0000","last_clean":"2024-09-05T06:40:13.272466+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:40:13.272466+0000","last_undegraded":"2024-09-05T06:40:13.272466+0000","last_fullsized":"2024-09-05T06:40:13.272466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273822+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:40:13.273822+0000","last_peered":"2024-09-05T06:40:13.273822+0000","last_clean":"2024-09-05T06:40:13.273822+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:40:13.273822+0000","last_undegraded":"2024-09-05T06:40:13.273822+0000","last_fullsized":"2024-09-05T06:40:13.273822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274553+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:40:13.274553+0000","last_peered":"2024-09-05T06:40:13.274553+0000","last_clean":"2024-09-05T06:40:13.274553+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:40:13.274553+0000","last_undegraded":"2024-09-05T06:40:13.274553+0000","last_fullsized":"2024-09-05T06:40:13.274553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272285+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:40:13.272285+0000","last_peered":"2024-09-05T06:40:13.272285+0000","last_clean":"2024-09-05T06:40:13.272285+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:40:13.272285+0000","last_undegraded":"2024-09-05T06:40:13.272285+0000","last_fullsized":"2024-09-05T06:40:13.272285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271466+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:40:13.271466+0000","last_peered":"2024-09-05T06:40:13.271466+0000","last_clean":"2024-09-05T06:40:13.271466+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:40:13.271466+0000","last_undegraded":"2024-09-05T06:40:13.271466+0000","last_fullsized":"2024-09-05T06:40:13.271466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273333+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:40:13.273333+0000","last_peered":"2024-09-05T06:40:13.273333+0000","last_clean":"2024-09-05T06:40:13.273333+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:40:13.273333+0000","last_undegraded":"2024-09-05T06:40:13.273333+0000","last_fullsized":"2024-09-05T06:40:13.273333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272679+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:40:13.272679+0000","last_peered":"2024-09-05T06:40:13.272679+0000","last_clean":"2024-09-05T06:40:13.272679+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:40:13.272679+0000","last_undegraded":"2024-09-05T06:40:13.272679+0000","last_fullsized":"2024-09-05T06:40:13.272679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270300+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:40:13.270300+0000","last_peered":"2024-09-05T06:40:13.270300+0000","last_clean":"2024-09-05T06:40:13.270300+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:40:13.270300+0000","last_undegraded":"2024-09-05T06:40:13.270300+0000","last_fullsized":"2024-09-05T06:40:13.270300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271737+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:40:13.271737+0000","last_peered":"2024-09-05T06:40:13.271737+0000","last_clean":"2024-09-05T06:40:13.271737+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:40:13.271737+0000","last_undegraded":"2024-09-05T06:40:13.271737+0000","last_fullsized":"2024-09-05T06:40:13.271737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272895+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:40:13.272895+0000","last_peered":"2024-09-05T06:40:13.272895+0000","last_clean":"2024-09-05T06:40:13.272895+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:40:13.272895+0000","last_undegraded":"2024-09-05T06:40:13.272895+0000","last_fullsized":"2024-09-05T06:40:13.272895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420722+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:40:13.420722+0000","last_peered":"2024-09-05T06:40:13.420722+0000","last_clean":"2024-09-05T06:40:13.420722+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:40:13.420722+0000","last_undegraded":"2024-09-05T06:40:13.420722+0000","last_fullsized":"2024-09-05T06:40:13.420722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420062+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:40:13.420062+0000","last_peered":"2024-09-05T06:40:13.420062+0000","last_clean":"2024-09-05T06:40:13.420062+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:40:13.420062+0000","last_undegraded":"2024-09-05T06:40:13.420062+0000","last_fullsized":"2024-09-05T06:40:13.420062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.422027+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:40:13.422027+0000","last_peered":"2024-09-05T06:40:13.422027+0000","last_clean":"2024-09-05T06:40:13.422027+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:40:13.422027+0000","last_undegraded":"2024-09-05T06:40:13.422027+0000","last_fullsized":"2024-09-05T06:40:13.422027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273427+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:40:13.273427+0000","last_peered":"2024-09-05T06:40:13.273427+0000","last_clean":"2024-09-05T06:40:13.273427+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:40:13.273427+0000","last_undegraded":"2024-09-05T06:40:13.273427+0000","last_fullsized":"2024-09-05T06:40:13.273427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.419805+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:40:13.419805+0000","last_peered":"2024-09-05T06:40:13.419805+0000","last_clean":"2024-09-05T06:40:13.419805+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:40:13.419805+0000","last_undegraded":"2024-09-05T06:40:13.419805+0000","last_fullsized":"2024-09-05T06:40:13.419805+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.423201+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:40:13.423201+0000","last_peered":"2024-09-05T06:40:13.423201+0000","last_clean":"2024-09-05T06:40:13.423201+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:40:13.423201+0000","last_undegraded":"2024-09-05T06:40:13.423201+0000","last_fullsized":"2024-09-05T06:40:13.423201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499550+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:40:13.499550+0000","last_peered":"2024-09-05T06:40:13.499550+0000","last_clean":"2024-09-05T06:40:13.499550+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:40:13.499550+0000","last_undegraded":"2024-09-05T06:40:13.499550+0000","last_fullsized":"2024-09-05T06:40:13.499550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273119+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:40:13.273119+0000","last_peered":"2024-09-05T06:40:13.273119+0000","last_clean":"2024-09-05T06:40:13.273119+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:40:13.273119+0000","last_undegraded":"2024-09-05T06:40:13.273119+0000","last_fullsized":"2024-09-05T06:40:13.273119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272486+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:40:13.272486+0000","last_peered":"2024-09-05T06:40:13.272486+0000","last_clean":"2024-09-05T06:40:13.272486+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:40:13.272486+0000","last_undegraded":"2024-09-05T06:40:13.272486+0000","last_fullsized":"2024-09-05T06:40:13.272486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421101+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:40:13.421101+0000","last_peered":"2024-09-05T06:40:13.421101+0000","last_clean":"2024-09-05T06:40:13.421101+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:40:13.421101+0000","last_undegraded":"2024-09-05T06:40:13.421101+0000","last_fullsized":"2024-09-05T06:40:13.421101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270681+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:40:13.270681+0000","last_peered":"2024-09-05T06:40:13.270681+0000","last_clean":"2024-09-05T06:40:13.270681+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:40:13.270681+0000","last_undegraded":"2024-09-05T06:40:13.270681+0000","last_fullsized":"2024-09-05T06:40:13.270681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420270+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:40:13.420270+0000","last_peered":"2024-09-05T06:40:13.420270+0000","last_clean":"2024-09-05T06:40:13.420270+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:40:13.420270+0000","last_undegraded":"2024-09-05T06:40:13.420270+0000","last_fullsized":"2024-09-05T06:40:13.420270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273838+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:40:13.273838+0000","last_peered":"2024-09-05T06:40:13.273838+0000","last_clean":"2024-09-05T06:40:13.273838+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:40:13.273838+0000","last_undegraded":"2024-09-05T06:40:13.273838+0000","last_fullsized":"2024-09-05T06:40:13.273838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274166+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:40:13.274166+0000","last_peered":"2024-09-05T06:40:13.274166+0000","last_clean":"2024-09-05T06:40:13.274166+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:40:13.274166+0000","last_undegraded":"2024-09-05T06:40:13.274166+0000","last_fullsized":"2024-09-05T06:40:13.274166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420779+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:40:13.420779+0000","last_peered":"2024-09-05T06:40:13.420779+0000","last_clean":"2024-09-05T06:40:13.420779+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:40:13.420779+0000","last_undegraded":"2024-09-05T06:40:13.420779+0000","last_fullsized":"2024-09-05T06:40:13.420779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421906+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:40:13.421906+0000","last_peered":"2024-09-05T06:40:13.421906+0000","last_clean":"2024-09-05T06:40:13.421906+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:40:13.421906+0000","last_undegraded":"2024-09-05T06:40:13.421906+0000","last_fullsized":"2024-09-05T06:40:13.421906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270588+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:40:13.270588+0000","last_peered":"2024-09-05T06:40:13.270588+0000","last_clean":"2024-09-05T06:40:13.270588+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:40:13.270588+0000","last_undegraded":"2024-09-05T06:40:13.270588+0000","last_fullsized":"2024-09-05T06:40:13.270588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272986+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:40:13.272986+0000","last_peered":"2024-09-05T06:40:13.272986+0000","last_clean":"2024-09-05T06:40:13.272986+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:40:13.272986+0000","last_undegraded":"2024-09-05T06:40:13.272986+0000","last_fullsized":"2024-09-05T06:40:13.272986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421889+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:40:13.421889+0000","last_peered":"2024-09-05T06:40:13.421889+0000","last_clean":"2024-09-05T06:40:13.421889+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:40:13.421889+0000","last_undegraded":"2024-09-05T06:40:13.421889+0000","last_fullsized":"2024-09-05T06:40:13.421889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:21.936129+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:40:21.936129+0000","last_peered":"2024-09-05T06:40:21.936129+0000","last_clean":"2024-09-05T06:40:21.936129+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:40:21.936129+0000","last_undegraded":"2024-09-05T06:40:21.936129+0000","last_fullsized":"2024-09-05T06:40:21.936129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273566+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:40:13.273566+0000","last_peered":"2024-09-05T06:40:13.273566+0000","last_clean":"2024-09-05T06:40:13.273566+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:40:13.273566+0000","last_undegraded":"2024-09-05T06:40:13.273566+0000","last_fullsized":"2024-09-05T06:40:13.273566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273613+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:40:13.273613+0000","last_peered":"2024-09-05T06:40:13.273613+0000","last_clean":"2024-09-05T06:40:13.273613+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:40:13.273613+0000","last_undegraded":"2024-09-05T06:40:13.273613+0000","last_fullsized":"2024-09-05T06:40:13.273613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:22.921500+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:40:22.921500+0000","last_peered":"2024-09-05T06:40:22.921500+0000","last_clean":"2024-09-05T06:40:22.921500+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:40:22.921500+0000","last_undegraded":"2024-09-05T06:40:22.921500+0000","last_fullsized":"2024-09-05T06:40:22.921500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273656+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:40:13.273656+0000","last_peered":"2024-09-05T06:40:13.273656+0000","last_clean":"2024-09-05T06:40:13.273656+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:40:13.273656+0000","last_undegraded":"2024-09-05T06:40:13.273656+0000","last_fullsized":"2024-09-05T06:40:13.273656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273179+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:40:13.273179+0000","last_peered":"2024-09-05T06:40:13.273179+0000","last_clean":"2024-09-05T06:40:13.273179+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:40:13.273179+0000","last_undegraded":"2024-09-05T06:40:13.273179+0000","last_fullsized":"2024-09-05T06:40:13.273179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271351+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:40:13.271351+0000","last_peered":"2024-09-05T06:40:13.271351+0000","last_clean":"2024-09-05T06:40:13.271351+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:40:13.271351+0000","last_undegraded":"2024-09-05T06:40:13.271351+0000","last_fullsized":"2024-09-05T06:40:13.271351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.423422+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:40:13.423422+0000","last_peered":"2024-09-05T06:40:13.423422+0000","last_clean":"2024-09-05T06:40:13.423422+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:40:13.423422+0000","last_undegraded":"2024-09-05T06:40:13.423422+0000","last_fullsized":"2024-09-05T06:40:13.423422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420625+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:40:13.420625+0000","last_peered":"2024-09-05T06:40:13.420625+0000","last_clean":"2024-09-05T06:40:13.420625+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:40:13.420625+0000","last_undegraded":"2024-09-05T06:40:13.420625+0000","last_fullsized":"2024-09-05T06:40:13.420625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272828+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:40:13.272828+0000","last_peered":"2024-09-05T06:40:13.272828+0000","last_clean":"2024-09-05T06:40:13.272828+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:40:13.272828+0000","last_undegraded":"2024-09-05T06:40:13.272828+0000","last_fullsized":"2024-09-05T06:40:13.272828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271453+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:40:13.271453+0000","last_peered":"2024-09-05T06:40:13.271453+0000","last_clean":"2024-09-05T06:40:13.271453+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:40:13.271453+0000","last_undegraded":"2024-09-05T06:40:13.271453+0000","last_fullsized":"2024-09-05T06:40:13.271453+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271746+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:40:13.271746+0000","last_peered":"2024-09-05T06:40:13.271746+0000","last_clean":"2024-09-05T06:40:13.271746+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:40:13.271746+0000","last_undegraded":"2024-09-05T06:40:13.271746+0000","last_fullsized":"2024-09-05T06:40:13.271746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271880+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:40:13.271880+0000","last_peered":"2024-09-05T06:40:13.271880+0000","last_clean":"2024-09-05T06:40:13.271880+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:40:13.271880+0000","last_undegraded":"2024-09-05T06:40:13.271880+0000","last_fullsized":"2024-09-05T06:40:13.271880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:19.953563+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:40:19.953563+0000","last_peered":"2024-09-05T06:40:19.953563+0000","last_clean":"2024-09-05T06:40:19.953563+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:40:19.953563+0000","last_undegraded":"2024-09-05T06:40:19.953563+0000","last_fullsized":"2024-09-05T06:40:19.953563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420683+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:40:13.420683+0000","last_peered":"2024-09-05T06:40:13.420683+0000","last_clean":"2024-09-05T06:40:13.420683+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:40:13.420683+0000","last_undegraded":"2024-09-05T06:40:13.420683+0000","last_fullsized":"2024-09-05T06:40:13.420683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271735+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:40:13.271735+0000","last_peered":"2024-09-05T06:40:13.271735+0000","last_clean":"2024-09-05T06:40:13.271735+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:40:13.271735+0000","last_undegraded":"2024-09-05T06:40:13.271735+0000","last_fullsized":"2024-09-05T06:40:13.271735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274695+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:40:13.274695+0000","last_peered":"2024-09-05T06:40:13.274695+0000","last_clean":"2024-09-05T06:40:13.274695+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:40:13.274695+0000","last_undegraded":"2024-09-05T06:40:13.274695+0000","last_fullsized":"2024-09-05T06:40:13.274695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499354+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:40:13.499354+0000","last_peered":"2024-09-05T06:40:13.499354+0000","last_clean":"2024-09-05T06:40:13.499354+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:40:13.499354+0000","last_undegraded":"2024-09-05T06:40:13.499354+0000","last_fullsized":"2024-09-05T06:40:13.499354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271972+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:40:13.271972+0000","last_peered":"2024-09-05T06:40:13.271972+0000","last_clean":"2024-09-05T06:40:13.271972+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:40:13.271972+0000","last_undegraded":"2024-09-05T06:40:13.271972+0000","last_fullsized":"2024-09-05T06:40:13.271972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273770+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:40:13.273770+0000","last_peered":"2024-09-05T06:40:13.273770+0000","last_clean":"2024-09-05T06:40:13.273770+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:40:13.273770+0000","last_undegraded":"2024-09-05T06:40:13.273770+0000","last_fullsized":"2024-09-05T06:40:13.273770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273295+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:40:13.273295+0000","last_peered":"2024-09-05T06:40:13.273295+0000","last_clean":"2024-09-05T06:40:13.273295+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:40:13.273295+0000","last_undegraded":"2024-09-05T06:40:13.273295+0000","last_fullsized":"2024-09-05T06:40:13.273295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421344+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:40:13.421344+0000","last_peered":"2024-09-05T06:40:13.421344+0000","last_clean":"2024-09-05T06:40:13.421344+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:40:13.421344+0000","last_undegraded":"2024-09-05T06:40:13.421344+0000","last_fullsized":"2024-09-05T06:40:13.421344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421423+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:40:13.421423+0000","last_peered":"2024-09-05T06:40:13.421423+0000","last_clean":"2024-09-05T06:40:13.421423+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:40:13.421423+0000","last_undegraded":"2024-09-05T06:40:13.421423+0000","last_fullsized":"2024-09-05T06:40:13.421423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273940+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:40:13.273940+0000","last_peered":"2024-09-05T06:40:13.273940+0000","last_clean":"2024-09-05T06:40:13.273940+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:40:13.273940+0000","last_undegraded":"2024-09-05T06:40:13.273940+0000","last_fullsized":"2024-09-05T06:40:13.273940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274096+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:40:13.274096+0000","last_peered":"2024-09-05T06:40:13.274096+0000","last_clean":"2024-09-05T06:40:13.274096+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:40:13.274096+0000","last_undegraded":"2024-09-05T06:40:13.274096+0000","last_fullsized":"2024-09-05T06:40:13.274096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272815+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:40:13.272815+0000","last_peered":"2024-09-05T06:40:13.272815+0000","last_clean":"2024-09-05T06:40:13.272815+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:40:13.272815+0000","last_undegraded":"2024-09-05T06:40:13.272815+0000","last_fullsized":"2024-09-05T06:40:13.272815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273665+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:40:13.273665+0000","last_peered":"2024-09-05T06:40:13.273665+0000","last_clean":"2024-09-05T06:40:13.273665+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:40:13.273665+0000","last_undegraded":"2024-09-05T06:40:13.273665+0000","last_fullsized":"2024-09-05T06:40:13.273665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271237+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:40:13.271237+0000","last_peered":"2024-09-05T06:40:13.271237+0000","last_clean":"2024-09-05T06:40:13.271237+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:40:13.271237+0000","last_undegraded":"2024-09-05T06:40:13.271237+0000","last_fullsized":"2024-09-05T06:40:13.271237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:19.917897+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:40:19.917897+0000","last_peered":"2024-09-05T06:40:19.917897+0000","last_clean":"2024-09-05T06:40:19.917897+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:40:19.917897+0000","last_undegraded":"2024-09-05T06:40:19.917897+0000","last_fullsized":"2024-09-05T06:40:19.917897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270590+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:40:13.270590+0000","last_peered":"2024-09-05T06:40:13.270590+0000","last_clean":"2024-09-05T06:40:13.270590+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:40:13.270590+0000","last_undegraded":"2024-09-05T06:40:13.270590+0000","last_fullsized":"2024-09-05T06:40:13.270590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271609+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:40:13.271609+0000","last_peered":"2024-09-05T06:40:13.271609+0000","last_clean":"2024-09-05T06:40:13.271609+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:40:13.271609+0000","last_undegraded":"2024-09-05T06:40:13.271609+0000","last_fullsized":"2024-09-05T06:40:13.271609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274424+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:40:13.274424+0000","last_peered":"2024-09-05T06:40:13.274424+0000","last_clean":"2024-09-05T06:40:13.274424+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:40:13.274424+0000","last_undegraded":"2024-09-05T06:40:13.274424+0000","last_fullsized":"2024-09-05T06:40:13.274424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420976+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:40:13.420976+0000","last_peered":"2024-09-05T06:40:13.420976+0000","last_clean":"2024-09-05T06:40:13.420976+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:40:13.420976+0000","last_undegraded":"2024-09-05T06:40:13.420976+0000","last_fullsized":"2024-09-05T06:40:13.420976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273373+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:40:13.273373+0000","last_peered":"2024-09-05T06:40:13.273373+0000","last_clean":"2024-09-05T06:40:13.273373+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:40:13.273373+0000","last_undegraded":"2024-09-05T06:40:13.273373+0000","last_fullsized":"2024-09-05T06:40:13.273373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271907+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:40:13.271907+0000","last_peered":"2024-09-05T06:40:13.271907+0000","last_clean":"2024-09-05T06:40:13.271907+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:40:13.271907+0000","last_undegraded":"2024-09-05T06:40:13.271907+0000","last_fullsized":"2024-09-05T06:40:13.271907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274549+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:40:13.274549+0000","last_peered":"2024-09-05T06:40:13.274549+0000","last_clean":"2024-09-05T06:40:13.274549+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:40:13.274549+0000","last_undegraded":"2024-09-05T06:40:13.274549+0000","last_fullsized":"2024-09-05T06:40:13.274549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273402+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:40:13.273402+0000","last_peered":"2024-09-05T06:40:13.273402+0000","last_clean":"2024-09-05T06:40:13.273402+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:40:13.273402+0000","last_undegraded":"2024-09-05T06:40:13.273402+0000","last_fullsized":"2024-09-05T06:40:13.273402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:23.925780+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:40:23.925780+0000","last_peered":"2024-09-05T06:40:23.925780+0000","last_clean":"2024-09-05T06:40:23.925780+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:40:23.925780+0000","last_undegraded":"2024-09-05T06:40:23.925780+0000","last_fullsized":"2024-09-05T06:40:23.925780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":120,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270241+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:40:13.270241+0000","last_peered":"2024-09-05T06:40:13.270241+0000","last_clean":"2024-09-05T06:40:13.270241+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:40:13.270241+0000","last_undegraded":"2024-09-05T06:40:13.270241+0000","last_fullsized":"2024-09-05T06:40:13.270241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421066+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:40:13.421066+0000","last_peered":"2024-09-05T06:40:13.421066+0000","last_clean":"2024-09-05T06:40:13.421066+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:40:13.421066+0000","last_undegraded":"2024-09-05T06:40:13.421066+0000","last_fullsized":"2024-09-05T06:40:13.421066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274651+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:40:13.274651+0000","last_peered":"2024-09-05T06:40:13.274651+0000","last_clean":"2024-09-05T06:40:13.274651+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:40:13.274651+0000","last_undegraded":"2024-09-05T06:40:13.274651+0000","last_fullsized":"2024-09-05T06:40:13.274651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272601+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:40:13.272601+0000","last_peered":"2024-09-05T06:40:13.272601+0000","last_clean":"2024-09-05T06:40:13.272601+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:40:13.272601+0000","last_undegraded":"2024-09-05T06:40:13.272601+0000","last_fullsized":"2024-09-05T06:40:13.272601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499809+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:40:13.499809+0000","last_peered":"2024-09-05T06:40:13.499809+0000","last_clean":"2024-09-05T06:40:13.499809+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:40:13.499809+0000","last_undegraded":"2024-09-05T06:40:13.499809+0000","last_fullsized":"2024-09-05T06:40:13.499809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272033+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:40:13.272033+0000","last_peered":"2024-09-05T06:40:13.272033+0000","last_clean":"2024-09-05T06:40:13.272033+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:40:13.272033+0000","last_undegraded":"2024-09-05T06:40:13.272033+0000","last_fullsized":"2024-09-05T06:40:13.272033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270061+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:40:13.270061+0000","last_peered":"2024-09-05T06:40:13.270061+0000","last_clean":"2024-09-05T06:40:13.270061+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:40:13.270061+0000","last_undegraded":"2024-09-05T06:40:13.270061+0000","last_fullsized":"2024-09-05T06:40:13.270061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420966+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:40:13.420966+0000","last_peered":"2024-09-05T06:40:13.420966+0000","last_clean":"2024-09-05T06:40:13.420966+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:40:13.420966+0000","last_undegraded":"2024-09-05T06:40:13.420966+0000","last_fullsized":"2024-09-05T06:40:13.420966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:23.922608+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:40:23.922608+0000","last_peered":"2024-09-05T06:40:23.922608+0000","last_clean":"2024-09-05T06:40:23.922608+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:40:23.922608+0000","last_undegraded":"2024-09-05T06:40:23.922608+0000","last_fullsized":"2024-09-05T06:40:23.922608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420546+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:40:13.420546+0000","last_peered":"2024-09-05T06:40:13.420546+0000","last_clean":"2024-09-05T06:40:13.420546+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:40:13.420546+0000","last_undegraded":"2024-09-05T06:40:13.420546+0000","last_fullsized":"2024-09-05T06:40:13.420546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499446+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:40:13.499446+0000","last_peered":"2024-09-05T06:40:13.499446+0000","last_clean":"2024-09-05T06:40:13.499446+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:40:13.499446+0000","last_undegraded":"2024-09-05T06:40:13.499446+0000","last_fullsized":"2024-09-05T06:40:13.499446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270151+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:40:13.270151+0000","last_peered":"2024-09-05T06:40:13.270151+0000","last_clean":"2024-09-05T06:40:13.270151+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:40:13.270151+0000","last_undegraded":"2024-09-05T06:40:13.270151+0000","last_fullsized":"2024-09-05T06:40:13.270151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273811+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:40:13.273811+0000","last_peered":"2024-09-05T06:40:13.273811+0000","last_clean":"2024-09-05T06:40:13.273811+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:40:13.273811+0000","last_undegraded":"2024-09-05T06:40:13.273811+0000","last_fullsized":"2024-09-05T06:40:13.273811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421224+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:40:13.421224+0000","last_peered":"2024-09-05T06:40:13.421224+0000","last_clean":"2024-09-05T06:40:13.421224+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:40:13.421224+0000","last_undegraded":"2024-09-05T06:40:13.421224+0000","last_fullsized":"2024-09-05T06:40:13.421224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420434+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:40:13.420434+0000","last_peered":"2024-09-05T06:40:13.420434+0000","last_clean":"2024-09-05T06:40:13.420434+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:40:13.420434+0000","last_undegraded":"2024-09-05T06:40:13.420434+0000","last_fullsized":"2024-09-05T06:40:13.420434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273182+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:40:13.273182+0000","last_peered":"2024-09-05T06:40:13.273182+0000","last_clean":"2024-09-05T06:40:13.273182+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:40:13.273182+0000","last_undegraded":"2024-09-05T06:40:13.273182+0000","last_fullsized":"2024-09-05T06:40:13.273182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420547+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:40:13.420547+0000","last_peered":"2024-09-05T06:40:13.420547+0000","last_clean":"2024-09-05T06:40:13.420547+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:40:13.420547+0000","last_undegraded":"2024-09-05T06:40:13.420547+0000","last_fullsized":"2024-09-05T06:40:13.420547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270342+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:40:13.270342+0000","last_peered":"2024-09-05T06:40:13.270342+0000","last_clean":"2024-09-05T06:40:13.270342+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:40:13.270342+0000","last_undegraded":"2024-09-05T06:40:13.270342+0000","last_fullsized":"2024-09-05T06:40:13.270342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271719+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:40:13.271719+0000","last_peered":"2024-09-05T06:40:13.271719+0000","last_clean":"2024-09-05T06:40:13.271719+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:40:13.271719+0000","last_undegraded":"2024-09-05T06:40:13.271719+0000","last_fullsized":"2024-09-05T06:40:13.271719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:22.921986+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:40:22.921986+0000","last_peered":"2024-09-05T06:40:22.921986+0000","last_clean":"2024-09-05T06:40:22.921986+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:40:22.921986+0000","last_undegraded":"2024-09-05T06:40:22.921986+0000","last_fullsized":"2024-09-05T06:40:22.921986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273080+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:40:13.273080+0000","last_peered":"2024-09-05T06:40:13.273080+0000","last_clean":"2024-09-05T06:40:13.273080+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:40:13.273080+0000","last_undegraded":"2024-09-05T06:40:13.273080+0000","last_fullsized":"2024-09-05T06:40:13.273080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272054+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:40:13.272054+0000","last_peered":"2024-09-05T06:40:13.272054+0000","last_clean":"2024-09-05T06:40:13.272054+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:40:13.272054+0000","last_undegraded":"2024-09-05T06:40:13.272054+0000","last_fullsized":"2024-09-05T06:40:13.272054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420700+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:40:13.420700+0000","last_peered":"2024-09-05T06:40:13.420700+0000","last_clean":"2024-09-05T06:40:13.420700+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:40:13.420700+0000","last_undegraded":"2024-09-05T06:40:13.420700+0000","last_fullsized":"2024-09-05T06:40:13.420700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272923+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:40:13.272923+0000","last_peered":"2024-09-05T06:40:13.272923+0000","last_clean":"2024-09-05T06:40:13.272923+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:40:13.272923+0000","last_undegraded":"2024-09-05T06:40:13.272923+0000","last_fullsized":"2024-09-05T06:40:13.272923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271605+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:40:13.271605+0000","last_peered":"2024-09-05T06:40:13.271605+0000","last_clean":"2024-09-05T06:40:13.271605+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:40:13.271605+0000","last_undegraded":"2024-09-05T06:40:13.271605+0000","last_fullsized":"2024-09-05T06:40:13.271605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499407+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:40:13.499407+0000","last_peered":"2024-09-05T06:40:13.499407+0000","last_clean":"2024-09-05T06:40:13.499407+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:40:13.499407+0000","last_undegraded":"2024-09-05T06:40:13.499407+0000","last_fullsized":"2024-09-05T06:40:13.499407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.500615+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:40:13.500615+0000","last_peered":"2024-09-05T06:40:13.500615+0000","last_clean":"2024-09-05T06:40:13.500615+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:40:13.500615+0000","last_undegraded":"2024-09-05T06:40:13.500615+0000","last_fullsized":"2024-09-05T06:40:13.500615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421442+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:40:13.421442+0000","last_peered":"2024-09-05T06:40:13.421442+0000","last_clean":"2024-09-05T06:40:13.421442+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:40:13.421442+0000","last_undegraded":"2024-09-05T06:40:13.421442+0000","last_fullsized":"2024-09-05T06:40:13.421442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272490+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:40:13.272490+0000","last_peered":"2024-09-05T06:40:13.272490+0000","last_clean":"2024-09-05T06:40:13.272490+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:40:13.272490+0000","last_undegraded":"2024-09-05T06:40:13.272490+0000","last_fullsized":"2024-09-05T06:40:13.272490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421097+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:40:13.421097+0000","last_peered":"2024-09-05T06:40:13.421097+0000","last_clean":"2024-09-05T06:40:13.421097+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:40:13.421097+0000","last_undegraded":"2024-09-05T06:40:13.421097+0000","last_fullsized":"2024-09-05T06:40:13.421097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273517+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:40:13.273517+0000","last_peered":"2024-09-05T06:40:13.273517+0000","last_clean":"2024-09-05T06:40:13.273517+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:40:13.273517+0000","last_undegraded":"2024-09-05T06:40:13.273517+0000","last_fullsized":"2024-09-05T06:40:13.273517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420760+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:40:13.420760+0000","last_peered":"2024-09-05T06:40:13.420760+0000","last_clean":"2024-09-05T06:40:13.420760+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:40:13.420760+0000","last_undegraded":"2024-09-05T06:40:13.420760+0000","last_fullsized":"2024-09-05T06:40:13.420760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420806+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:40:13.420806+0000","last_peered":"2024-09-05T06:40:13.420806+0000","last_clean":"2024-09-05T06:40:13.420806+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:40:13.420806+0000","last_undegraded":"2024-09-05T06:40:13.420806+0000","last_fullsized":"2024-09-05T06:40:13.420806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273546+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:40:13.273546+0000","last_peered":"2024-09-05T06:40:13.273546+0000","last_clean":"2024-09-05T06:40:13.273546+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:40:13.273546+0000","last_undegraded":"2024-09-05T06:40:13.273546+0000","last_fullsized":"2024-09-05T06:40:13.273546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420982+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:40:13.420982+0000","last_peered":"2024-09-05T06:40:13.420982+0000","last_clean":"2024-09-05T06:40:13.420982+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:40:13.420982+0000","last_undegraded":"2024-09-05T06:40:13.420982+0000","last_fullsized":"2024-09-05T06:40:13.420982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272205+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:40:13.272205+0000","last_peered":"2024-09-05T06:40:13.272205+0000","last_clean":"2024-09-05T06:40:13.272205+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:40:13.272205+0000","last_undegraded":"2024-09-05T06:40:13.272205+0000","last_fullsized":"2024-09-05T06:40:13.272205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270572+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:40:13.270572+0000","last_peered":"2024-09-05T06:40:13.270572+0000","last_clean":"2024-09-05T06:40:13.270572+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:40:13.270572+0000","last_undegraded":"2024-09-05T06:40:13.270572+0000","last_fullsized":"2024-09-05T06:40:13.270572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421524+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:40:13.421524+0000","last_peered":"2024-09-05T06:40:13.421524+0000","last_clean":"2024-09-05T06:40:13.421524+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:40:13.421524+0000","last_undegraded":"2024-09-05T06:40:13.421524+0000","last_fullsized":"2024-09-05T06:40:13.421524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420103+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:40:13.420103+0000","last_peered":"2024-09-05T06:40:13.420103+0000","last_clean":"2024-09-05T06:40:13.420103+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:40:13.420103+0000","last_undegraded":"2024-09-05T06:40:13.420103+0000","last_fullsized":"2024-09-05T06:40:13.420103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499051+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:40:13.499051+0000","last_peered":"2024-09-05T06:40:13.499051+0000","last_clean":"2024-09-05T06:40:13.499051+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:40:13.499051+0000","last_undegraded":"2024-09-05T06:40:13.499051+0000","last_fullsized":"2024-09-05T06:40:13.499051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499099+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:40:13.499099+0000","last_peered":"2024-09-05T06:40:13.499099+0000","last_clean":"2024-09-05T06:40:13.499099+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:40:13.499099+0000","last_undegraded":"2024-09-05T06:40:13.499099+0000","last_fullsized":"2024-09-05T06:40:13.499099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273476+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:40:13.273476+0000","last_peered":"2024-09-05T06:40:13.273476+0000","last_clean":"2024-09-05T06:40:13.273476+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:40:13.273476+0000","last_undegraded":"2024-09-05T06:40:13.273476+0000","last_fullsized":"2024-09-05T06:40:13.273476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273159+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:40:13.273159+0000","last_peered":"2024-09-05T06:40:13.273159+0000","last_clean":"2024-09-05T06:40:13.273159+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:40:13.273159+0000","last_undegraded":"2024-09-05T06:40:13.273159+0000","last_fullsized":"2024-09-05T06:40:13.273159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271852+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:40:13.271852+0000","last_peered":"2024-09-05T06:40:13.271852+0000","last_clean":"2024-09-05T06:40:13.271852+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:40:13.271852+0000","last_undegraded":"2024-09-05T06:40:13.271852+0000","last_fullsized":"2024-09-05T06:40:13.271852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.422370+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:40:13.422370+0000","last_peered":"2024-09-05T06:40:13.422370+0000","last_clean":"2024-09-05T06:40:13.422370+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:40:13.422370+0000","last_undegraded":"2024-09-05T06:40:13.422370+0000","last_fullsized":"2024-09-05T06:40:13.422370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420810+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:40:13.420810+0000","last_peered":"2024-09-05T06:40:13.420810+0000","last_clean":"2024-09-05T06:40:13.420810+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:40:13.420810+0000","last_undegraded":"2024-09-05T06:40:13.420810+0000","last_fullsized":"2024-09-05T06:40:13.420810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420712+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:40:13.420712+0000","last_peered":"2024-09-05T06:40:13.420712+0000","last_clean":"2024-09-05T06:40:13.420712+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:40:13.420712+0000","last_undegraded":"2024-09-05T06:40:13.420712+0000","last_fullsized":"2024-09-05T06:40:13.420712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271346+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:40:13.271346+0000","last_peered":"2024-09-05T06:40:13.271346+0000","last_clean":"2024-09-05T06:40:13.271346+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:40:13.271346+0000","last_undegraded":"2024-09-05T06:40:13.271346+0000","last_fullsized":"2024-09-05T06:40:13.271346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421735+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:40:13.421735+0000","last_peered":"2024-09-05T06:40:13.421735+0000","last_clean":"2024-09-05T06:40:13.421735+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:40:13.421735+0000","last_undegraded":"2024-09-05T06:40:13.421735+0000","last_fullsized":"2024-09-05T06:40:13.421735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273219+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:40:13.273219+0000","last_peered":"2024-09-05T06:40:13.273219+0000","last_clean":"2024-09-05T06:40:13.273219+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:40:13.273219+0000","last_undegraded":"2024-09-05T06:40:13.273219+0000","last_fullsized":"2024-09-05T06:40:13.273219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273124+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:40:13.273124+0000","last_peered":"2024-09-05T06:40:13.273124+0000","last_clean":"2024-09-05T06:40:13.273124+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:40:13.273124+0000","last_undegraded":"2024-09-05T06:40:13.273124+0000","last_fullsized":"2024-09-05T06:40:13.273124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499452+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:40:13.499452+0000","last_peered":"2024-09-05T06:40:13.499452+0000","last_clean":"2024-09-05T06:40:13.499452+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:40:13.499452+0000","last_undegraded":"2024-09-05T06:40:13.499452+0000","last_fullsized":"2024-09-05T06:40:13.499452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499502+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:40:13.499502+0000","last_peered":"2024-09-05T06:40:13.499502+0000","last_clean":"2024-09-05T06:40:13.499502+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:40:13.499502+0000","last_undegraded":"2024-09-05T06:40:13.499502+0000","last_fullsized":"2024-09-05T06:40:13.499502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421504+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:40:13.421504+0000","last_peered":"2024-09-05T06:40:13.421504+0000","last_clean":"2024-09-05T06:40:13.421504+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:40:13.421504+0000","last_undegraded":"2024-09-05T06:40:13.421504+0000","last_fullsized":"2024-09-05T06:40:13.421504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71272,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":551,"num_read_kb":356,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":213816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612639,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":510584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1728512,"data_stored":1161835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1060864,"data_stored":514588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1048576,"data_stored":510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462963,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1650688,"data_stored":1107762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":505996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855421,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1658880,"data_stored":1102951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215175,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":505704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":507639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1101824,"data_stored":571090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":507452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:40:26.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:40:27.081 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:27.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:27 smithi086 ceph-mon[30943]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 -- 172.21.15.86:0/2000462225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 msgr2=0x7f82ec100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 --2- 172.21.15.86:0/2000462225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec100ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009930 tx=0x7f82dc02a420 comp rx=0 tx=0).stop 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 -- 172.21.15.86:0/2000462225 shutdown_connections 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 --2- 172.21.15.86:0/2000462225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 0x7f82ec10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 --2- 172.21.15.86:0/2000462225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec100ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 --2- 172.21.15.86:0/2000462225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 -- 172.21.15.86:0/2000462225 >> 172.21.15.86:0/2000462225 conn(0x7f82ec0fc780 msgr2=0x7f82ec0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:27.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 -- 172.21.15.86:0/2000462225 shutdown_connections 2024-09-05T06:40:27.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 -- 172.21.15.86:0/2000462225 wait complete. 2024-09-05T06:40:27.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.529+0000 7f82f21f6640 1 Processor -- start 2024-09-05T06:40:27.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 -- start start 2024-09-05T06:40:27.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:27.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 0x7f82ec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:27.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:27.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec110d30 con 0x7f82ec100a60 2024-09-05T06:40:27.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec110bb0 con 0x7f82ec106a60 2024-09-05T06:40:27.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec110eb0 con 0x7f82ec101400 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36434/0 (socket says 172.21.15.86:36434) 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 -- 172.21.15.86:0/3326253342 learned_addr learned my addr 172.21.15.86:0/3326253342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eaffd640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 0x7f82ec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82ebfff640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 -- 172.21.15.86:0/3326253342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 msgr2=0x7f82ec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 0x7f82ec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 -- 172.21.15.86:0/3326253342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 msgr2=0x7f82ec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 -- 172.21.15.86:0/3326253342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec19d3f0 con 0x7f82ec100a60 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82ebfff640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82eb7fe640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82e000b4a0 tx=0x7f82e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:27.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0014070 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82e0010ac0 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.530+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec19d6e0 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.531+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0017c30 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.531+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a4fc0 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.531+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec100ec0 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.533+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f82e0017070 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.534+0000 7f82e8ff9640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 0x7f82d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.534+0000 7f82eaffd640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 0x7f82d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.534+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f82e00cec50 con 0x7f82ec100a60 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.534+0000 7f82eaffd640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 0x7f82d40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82dc000c00 tx=0x7f82dc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:27.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.537+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82e009b790 con 0x7f82ec100a60 2024-09-05T06:40:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:27 smithi115 ceph-mon[34791]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.700+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82ec06a790 con 0x7f82d40a2c10 2024-09-05T06:40:27.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.708+0000 7f82e8ff9640 1 -- 172.21.15.86:0/3326253342 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340397 (secure 0 0 0) 0x7f82ec06a790 con 0x7f82d40a2c10 2024-09-05T06:40:27.711 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:27.713 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:40:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 msgr2=0x7f82d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 0x7f82d40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82dc000c00 tx=0x7f82dc031400 comp rx=0 tx=0).stop 2024-09-05T06:40:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 msgr2=0x7f82ec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82e000b4a0 tx=0x7f82e000b960 comp rx=0 tx=0).stop 2024-09-05T06:40:27.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 shutdown_connections 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f82d40a2c10 0x7f82d40a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82ec106a60 0x7f82ec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82ec101400 0x7f82ec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 --2- 172.21.15.86:0/3326253342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ec100a60 0x7f82ec198440 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 >> 172.21.15.86:0/3326253342 conn(0x7f82ec0fc780 msgr2=0x7f82ec104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 shutdown_connections 2024-09-05T06:40:27.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:27.711+0000 7f82f21f6640 1 -- 172.21.15.86:0/3326253342 wait complete. 2024-09-05T06:40:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:27 smithi096 ceph-mon[34468]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:28.390 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-05T06:40:27.206329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661874,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":702,"num_read_kb":480,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388520,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553304,"statfs":{"total":1151940427776,"available":1151542583296,"internally_reserved":0,"allocated":14581760,"data_stored":8080976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":104,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:23.938110+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:40:23.938110+0000","last_peered":"2024-09-05T06:40:23.938110+0000","last_clean":"2024-09-05T06:40:23.938110+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:40:23.938110+0000","last_undegraded":"2024-09-05T06:40:23.938110+0000","last_fullsized":"2024-09-05T06:40:23.938110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420916+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:40:13.420916+0000","last_peered":"2024-09-05T06:40:13.420916+0000","last_clean":"2024-09-05T06:40:13.420916+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:40:13.420916+0000","last_undegraded":"2024-09-05T06:40:13.420916+0000","last_fullsized":"2024-09-05T06:40:13.420916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.500473+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:40:13.500473+0000","last_peered":"2024-09-05T06:40:13.500473+0000","last_clean":"2024-09-05T06:40:13.500473+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:40:13.500473+0000","last_undegraded":"2024-09-05T06:40:13.500473+0000","last_fullsized":"2024-09-05T06:40:13.500473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420216+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:40:13.420216+0000","last_peered":"2024-09-05T06:40:13.420216+0000","last_clean":"2024-09-05T06:40:13.420216+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:40:13.420216+0000","last_undegraded":"2024-09-05T06:40:13.420216+0000","last_fullsized":"2024-09-05T06:40:13.420216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499751+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:40:13.499751+0000","last_peered":"2024-09-05T06:40:13.499751+0000","last_clean":"2024-09-05T06:40:13.499751+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:40:13.499751+0000","last_undegraded":"2024-09-05T06:40:13.499751+0000","last_fullsized":"2024-09-05T06:40:13.499751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272429+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:40:13.272429+0000","last_peered":"2024-09-05T06:40:13.272429+0000","last_clean":"2024-09-05T06:40:13.272429+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:40:13.272429+0000","last_undegraded":"2024-09-05T06:40:13.272429+0000","last_fullsized":"2024-09-05T06:40:13.272429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274275+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:40:13.274275+0000","last_peered":"2024-09-05T06:40:13.274275+0000","last_clean":"2024-09-05T06:40:13.274275+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:40:13.274275+0000","last_undegraded":"2024-09-05T06:40:13.274275+0000","last_fullsized":"2024-09-05T06:40:13.274275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420939+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:40:13.420939+0000","last_peered":"2024-09-05T06:40:13.420939+0000","last_clean":"2024-09-05T06:40:13.420939+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:40:13.420939+0000","last_undegraded":"2024-09-05T06:40:13.420939+0000","last_fullsized":"2024-09-05T06:40:13.420939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272466+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:40:13.272466+0000","last_peered":"2024-09-05T06:40:13.272466+0000","last_clean":"2024-09-05T06:40:13.272466+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:40:13.272466+0000","last_undegraded":"2024-09-05T06:40:13.272466+0000","last_fullsized":"2024-09-05T06:40:13.272466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273822+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:40:13.273822+0000","last_peered":"2024-09-05T06:40:13.273822+0000","last_clean":"2024-09-05T06:40:13.273822+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:40:13.273822+0000","last_undegraded":"2024-09-05T06:40:13.273822+0000","last_fullsized":"2024-09-05T06:40:13.273822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274553+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:40:13.274553+0000","last_peered":"2024-09-05T06:40:13.274553+0000","last_clean":"2024-09-05T06:40:13.274553+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:40:13.274553+0000","last_undegraded":"2024-09-05T06:40:13.274553+0000","last_fullsized":"2024-09-05T06:40:13.274553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272285+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:40:13.272285+0000","last_peered":"2024-09-05T06:40:13.272285+0000","last_clean":"2024-09-05T06:40:13.272285+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:40:13.272285+0000","last_undegraded":"2024-09-05T06:40:13.272285+0000","last_fullsized":"2024-09-05T06:40:13.272285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271466+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:40:13.271466+0000","last_peered":"2024-09-05T06:40:13.271466+0000","last_clean":"2024-09-05T06:40:13.271466+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:40:13.271466+0000","last_undegraded":"2024-09-05T06:40:13.271466+0000","last_fullsized":"2024-09-05T06:40:13.271466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273333+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:40:13.273333+0000","last_peered":"2024-09-05T06:40:13.273333+0000","last_clean":"2024-09-05T06:40:13.273333+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:40:13.273333+0000","last_undegraded":"2024-09-05T06:40:13.273333+0000","last_fullsized":"2024-09-05T06:40:13.273333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272679+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:40:13.272679+0000","last_peered":"2024-09-05T06:40:13.272679+0000","last_clean":"2024-09-05T06:40:13.272679+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:40:13.272679+0000","last_undegraded":"2024-09-05T06:40:13.272679+0000","last_fullsized":"2024-09-05T06:40:13.272679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270300+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:40:13.270300+0000","last_peered":"2024-09-05T06:40:13.270300+0000","last_clean":"2024-09-05T06:40:13.270300+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:40:13.270300+0000","last_undegraded":"2024-09-05T06:40:13.270300+0000","last_fullsized":"2024-09-05T06:40:13.270300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271737+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:40:13.271737+0000","last_peered":"2024-09-05T06:40:13.271737+0000","last_clean":"2024-09-05T06:40:13.271737+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:40:13.271737+0000","last_undegraded":"2024-09-05T06:40:13.271737+0000","last_fullsized":"2024-09-05T06:40:13.271737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272895+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:40:13.272895+0000","last_peered":"2024-09-05T06:40:13.272895+0000","last_clean":"2024-09-05T06:40:13.272895+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:40:13.272895+0000","last_undegraded":"2024-09-05T06:40:13.272895+0000","last_fullsized":"2024-09-05T06:40:13.272895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420722+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:40:13.420722+0000","last_peered":"2024-09-05T06:40:13.420722+0000","last_clean":"2024-09-05T06:40:13.420722+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:40:13.420722+0000","last_undegraded":"2024-09-05T06:40:13.420722+0000","last_fullsized":"2024-09-05T06:40:13.420722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420062+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:40:13.420062+0000","last_peered":"2024-09-05T06:40:13.420062+0000","last_clean":"2024-09-05T06:40:13.420062+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:40:13.420062+0000","last_undegraded":"2024-09-05T06:40:13.420062+0000","last_fullsized":"2024-09-05T06:40:13.420062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.422027+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:40:13.422027+0000","last_peered":"2024-09-05T06:40:13.422027+0000","last_clean":"2024-09-05T06:40:13.422027+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:40:13.422027+0000","last_undegraded":"2024-09-05T06:40:13.422027+0000","last_fullsized":"2024-09-05T06:40:13.422027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273427+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:40:13.273427+0000","last_peered":"2024-09-05T06:40:13.273427+0000","last_clean":"2024-09-05T06:40:13.273427+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:40:13.273427+0000","last_undegraded":"2024-09-05T06:40:13.273427+0000","last_fullsized":"2024-09-05T06:40:13.273427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.419805+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:40:13.419805+0000","last_peered":"2024-09-05T06:40:13.419805+0000","last_clean":"2024-09-05T06:40:13.419805+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:40:13.419805+0000","last_undegraded":"2024-09-05T06:40:13.419805+0000","last_fullsized":"2024-09-05T06:40:13.419805+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.423201+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:40:13.423201+0000","last_peered":"2024-09-05T06:40:13.423201+0000","last_clean":"2024-09-05T06:40:13.423201+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:40:13.423201+0000","last_undegraded":"2024-09-05T06:40:13.423201+0000","last_fullsized":"2024-09-05T06:40:13.423201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499550+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:40:13.499550+0000","last_peered":"2024-09-05T06:40:13.499550+0000","last_clean":"2024-09-05T06:40:13.499550+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:40:13.499550+0000","last_undegraded":"2024-09-05T06:40:13.499550+0000","last_fullsized":"2024-09-05T06:40:13.499550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273119+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:40:13.273119+0000","last_peered":"2024-09-05T06:40:13.273119+0000","last_clean":"2024-09-05T06:40:13.273119+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:40:13.273119+0000","last_undegraded":"2024-09-05T06:40:13.273119+0000","last_fullsized":"2024-09-05T06:40:13.273119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272486+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:40:13.272486+0000","last_peered":"2024-09-05T06:40:13.272486+0000","last_clean":"2024-09-05T06:40:13.272486+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:40:13.272486+0000","last_undegraded":"2024-09-05T06:40:13.272486+0000","last_fullsized":"2024-09-05T06:40:13.272486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421101+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:40:13.421101+0000","last_peered":"2024-09-05T06:40:13.421101+0000","last_clean":"2024-09-05T06:40:13.421101+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:40:13.421101+0000","last_undegraded":"2024-09-05T06:40:13.421101+0000","last_fullsized":"2024-09-05T06:40:13.421101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270681+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:40:13.270681+0000","last_peered":"2024-09-05T06:40:13.270681+0000","last_clean":"2024-09-05T06:40:13.270681+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:40:13.270681+0000","last_undegraded":"2024-09-05T06:40:13.270681+0000","last_fullsized":"2024-09-05T06:40:13.270681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420270+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:40:13.420270+0000","last_peered":"2024-09-05T06:40:13.420270+0000","last_clean":"2024-09-05T06:40:13.420270+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:40:13.420270+0000","last_undegraded":"2024-09-05T06:40:13.420270+0000","last_fullsized":"2024-09-05T06:40:13.420270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273838+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:40:13.273838+0000","last_peered":"2024-09-05T06:40:13.273838+0000","last_clean":"2024-09-05T06:40:13.273838+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:40:13.273838+0000","last_undegraded":"2024-09-05T06:40:13.273838+0000","last_fullsized":"2024-09-05T06:40:13.273838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274166+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:40:13.274166+0000","last_peered":"2024-09-05T06:40:13.274166+0000","last_clean":"2024-09-05T06:40:13.274166+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:40:13.274166+0000","last_undegraded":"2024-09-05T06:40:13.274166+0000","last_fullsized":"2024-09-05T06:40:13.274166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420779+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:40:13.420779+0000","last_peered":"2024-09-05T06:40:13.420779+0000","last_clean":"2024-09-05T06:40:13.420779+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:40:13.420779+0000","last_undegraded":"2024-09-05T06:40:13.420779+0000","last_fullsized":"2024-09-05T06:40:13.420779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421906+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:40:13.421906+0000","last_peered":"2024-09-05T06:40:13.421906+0000","last_clean":"2024-09-05T06:40:13.421906+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:40:13.421906+0000","last_undegraded":"2024-09-05T06:40:13.421906+0000","last_fullsized":"2024-09-05T06:40:13.421906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270588+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:40:13.270588+0000","last_peered":"2024-09-05T06:40:13.270588+0000","last_clean":"2024-09-05T06:40:13.270588+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:40:13.270588+0000","last_undegraded":"2024-09-05T06:40:13.270588+0000","last_fullsized":"2024-09-05T06:40:13.270588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272986+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:40:13.272986+0000","last_peered":"2024-09-05T06:40:13.272986+0000","last_clean":"2024-09-05T06:40:13.272986+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:40:13.272986+0000","last_undegraded":"2024-09-05T06:40:13.272986+0000","last_fullsized":"2024-09-05T06:40:13.272986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421889+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:40:13.421889+0000","last_peered":"2024-09-05T06:40:13.421889+0000","last_clean":"2024-09-05T06:40:13.421889+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:40:13.421889+0000","last_undegraded":"2024-09-05T06:40:13.421889+0000","last_fullsized":"2024-09-05T06:40:13.421889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:21.936129+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:40:21.936129+0000","last_peered":"2024-09-05T06:40:21.936129+0000","last_clean":"2024-09-05T06:40:21.936129+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:40:21.936129+0000","last_undegraded":"2024-09-05T06:40:21.936129+0000","last_fullsized":"2024-09-05T06:40:21.936129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273566+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:40:13.273566+0000","last_peered":"2024-09-05T06:40:13.273566+0000","last_clean":"2024-09-05T06:40:13.273566+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:40:13.273566+0000","last_undegraded":"2024-09-05T06:40:13.273566+0000","last_fullsized":"2024-09-05T06:40:13.273566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273613+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:40:13.273613+0000","last_peered":"2024-09-05T06:40:13.273613+0000","last_clean":"2024-09-05T06:40:13.273613+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:40:13.273613+0000","last_undegraded":"2024-09-05T06:40:13.273613+0000","last_fullsized":"2024-09-05T06:40:13.273613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:22.921500+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:40:22.921500+0000","last_peered":"2024-09-05T06:40:22.921500+0000","last_clean":"2024-09-05T06:40:22.921500+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:40:22.921500+0000","last_undegraded":"2024-09-05T06:40:22.921500+0000","last_fullsized":"2024-09-05T06:40:22.921500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273656+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:40:13.273656+0000","last_peered":"2024-09-05T06:40:13.273656+0000","last_clean":"2024-09-05T06:40:13.273656+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:40:13.273656+0000","last_undegraded":"2024-09-05T06:40:13.273656+0000","last_fullsized":"2024-09-05T06:40:13.273656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273179+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:40:13.273179+0000","last_peered":"2024-09-05T06:40:13.273179+0000","last_clean":"2024-09-05T06:40:13.273179+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:40:13.273179+0000","last_undegraded":"2024-09-05T06:40:13.273179+0000","last_fullsized":"2024-09-05T06:40:13.273179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271351+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:40:13.271351+0000","last_peered":"2024-09-05T06:40:13.271351+0000","last_clean":"2024-09-05T06:40:13.271351+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:40:13.271351+0000","last_undegraded":"2024-09-05T06:40:13.271351+0000","last_fullsized":"2024-09-05T06:40:13.271351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.423422+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:40:13.423422+0000","last_peered":"2024-09-05T06:40:13.423422+0000","last_clean":"2024-09-05T06:40:13.423422+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:40:13.423422+0000","last_undegraded":"2024-09-05T06:40:13.423422+0000","last_fullsized":"2024-09-05T06:40:13.423422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420625+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:40:13.420625+0000","last_peered":"2024-09-05T06:40:13.420625+0000","last_clean":"2024-09-05T06:40:13.420625+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:40:13.420625+0000","last_undegraded":"2024-09-05T06:40:13.420625+0000","last_fullsized":"2024-09-05T06:40:13.420625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272828+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:40:13.272828+0000","last_peered":"2024-09-05T06:40:13.272828+0000","last_clean":"2024-09-05T06:40:13.272828+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:40:13.272828+0000","last_undegraded":"2024-09-05T06:40:13.272828+0000","last_fullsized":"2024-09-05T06:40:13.272828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271453+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:40:13.271453+0000","last_peered":"2024-09-05T06:40:13.271453+0000","last_clean":"2024-09-05T06:40:13.271453+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:40:13.271453+0000","last_undegraded":"2024-09-05T06:40:13.271453+0000","last_fullsized":"2024-09-05T06:40:13.271453+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271746+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:40:13.271746+0000","last_peered":"2024-09-05T06:40:13.271746+0000","last_clean":"2024-09-05T06:40:13.271746+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:40:13.271746+0000","last_undegraded":"2024-09-05T06:40:13.271746+0000","last_fullsized":"2024-09-05T06:40:13.271746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271880+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:40:13.271880+0000","last_peered":"2024-09-05T06:40:13.271880+0000","last_clean":"2024-09-05T06:40:13.271880+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:40:13.271880+0000","last_undegraded":"2024-09-05T06:40:13.271880+0000","last_fullsized":"2024-09-05T06:40:13.271880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:24.961302+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:40:24.961302+0000","last_peered":"2024-09-05T06:40:24.961302+0000","last_clean":"2024-09-05T06:40:24.961302+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:40:24.961302+0000","last_undegraded":"2024-09-05T06:40:24.961302+0000","last_fullsized":"2024-09-05T06:40:24.961302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420683+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:40:13.420683+0000","last_peered":"2024-09-05T06:40:13.420683+0000","last_clean":"2024-09-05T06:40:13.420683+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:40:13.420683+0000","last_undegraded":"2024-09-05T06:40:13.420683+0000","last_fullsized":"2024-09-05T06:40:13.420683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271735+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:40:13.271735+0000","last_peered":"2024-09-05T06:40:13.271735+0000","last_clean":"2024-09-05T06:40:13.271735+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:40:13.271735+0000","last_undegraded":"2024-09-05T06:40:13.271735+0000","last_fullsized":"2024-09-05T06:40:13.271735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274695+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:40:13.274695+0000","last_peered":"2024-09-05T06:40:13.274695+0000","last_clean":"2024-09-05T06:40:13.274695+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:40:13.274695+0000","last_undegraded":"2024-09-05T06:40:13.274695+0000","last_fullsized":"2024-09-05T06:40:13.274695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499354+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:40:13.499354+0000","last_peered":"2024-09-05T06:40:13.499354+0000","last_clean":"2024-09-05T06:40:13.499354+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:40:13.499354+0000","last_undegraded":"2024-09-05T06:40:13.499354+0000","last_fullsized":"2024-09-05T06:40:13.499354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271972+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:40:13.271972+0000","last_peered":"2024-09-05T06:40:13.271972+0000","last_clean":"2024-09-05T06:40:13.271972+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:40:13.271972+0000","last_undegraded":"2024-09-05T06:40:13.271972+0000","last_fullsized":"2024-09-05T06:40:13.271972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273770+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:40:13.273770+0000","last_peered":"2024-09-05T06:40:13.273770+0000","last_clean":"2024-09-05T06:40:13.273770+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:40:13.273770+0000","last_undegraded":"2024-09-05T06:40:13.273770+0000","last_fullsized":"2024-09-05T06:40:13.273770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273295+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:40:13.273295+0000","last_peered":"2024-09-05T06:40:13.273295+0000","last_clean":"2024-09-05T06:40:13.273295+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:40:13.273295+0000","last_undegraded":"2024-09-05T06:40:13.273295+0000","last_fullsized":"2024-09-05T06:40:13.273295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421344+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:40:13.421344+0000","last_peered":"2024-09-05T06:40:13.421344+0000","last_clean":"2024-09-05T06:40:13.421344+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:40:13.421344+0000","last_undegraded":"2024-09-05T06:40:13.421344+0000","last_fullsized":"2024-09-05T06:40:13.421344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421423+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:40:13.421423+0000","last_peered":"2024-09-05T06:40:13.421423+0000","last_clean":"2024-09-05T06:40:13.421423+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:40:13.421423+0000","last_undegraded":"2024-09-05T06:40:13.421423+0000","last_fullsized":"2024-09-05T06:40:13.421423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273940+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:40:13.273940+0000","last_peered":"2024-09-05T06:40:13.273940+0000","last_clean":"2024-09-05T06:40:13.273940+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:40:13.273940+0000","last_undegraded":"2024-09-05T06:40:13.273940+0000","last_fullsized":"2024-09-05T06:40:13.273940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274096+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:40:13.274096+0000","last_peered":"2024-09-05T06:40:13.274096+0000","last_clean":"2024-09-05T06:40:13.274096+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:40:13.274096+0000","last_undegraded":"2024-09-05T06:40:13.274096+0000","last_fullsized":"2024-09-05T06:40:13.274096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272815+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:40:13.272815+0000","last_peered":"2024-09-05T06:40:13.272815+0000","last_clean":"2024-09-05T06:40:13.272815+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:40:13.272815+0000","last_undegraded":"2024-09-05T06:40:13.272815+0000","last_fullsized":"2024-09-05T06:40:13.272815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273665+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:40:13.273665+0000","last_peered":"2024-09-05T06:40:13.273665+0000","last_clean":"2024-09-05T06:40:13.273665+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:40:13.273665+0000","last_undegraded":"2024-09-05T06:40:13.273665+0000","last_fullsized":"2024-09-05T06:40:13.273665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271237+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:40:13.271237+0000","last_peered":"2024-09-05T06:40:13.271237+0000","last_clean":"2024-09-05T06:40:13.271237+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:40:13.271237+0000","last_undegraded":"2024-09-05T06:40:13.271237+0000","last_fullsized":"2024-09-05T06:40:13.271237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:24.924969+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:40:24.924969+0000","last_peered":"2024-09-05T06:40:24.924969+0000","last_clean":"2024-09-05T06:40:24.924969+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:40:24.924969+0000","last_undegraded":"2024-09-05T06:40:24.924969+0000","last_fullsized":"2024-09-05T06:40:24.924969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270590+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:40:13.270590+0000","last_peered":"2024-09-05T06:40:13.270590+0000","last_clean":"2024-09-05T06:40:13.270590+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:40:13.270590+0000","last_undegraded":"2024-09-05T06:40:13.270590+0000","last_fullsized":"2024-09-05T06:40:13.270590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271609+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:40:13.271609+0000","last_peered":"2024-09-05T06:40:13.271609+0000","last_clean":"2024-09-05T06:40:13.271609+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:40:13.271609+0000","last_undegraded":"2024-09-05T06:40:13.271609+0000","last_fullsized":"2024-09-05T06:40:13.271609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274424+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:40:13.274424+0000","last_peered":"2024-09-05T06:40:13.274424+0000","last_clean":"2024-09-05T06:40:13.274424+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:40:13.274424+0000","last_undegraded":"2024-09-05T06:40:13.274424+0000","last_fullsized":"2024-09-05T06:40:13.274424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420976+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:40:13.420976+0000","last_peered":"2024-09-05T06:40:13.420976+0000","last_clean":"2024-09-05T06:40:13.420976+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:40:13.420976+0000","last_undegraded":"2024-09-05T06:40:13.420976+0000","last_fullsized":"2024-09-05T06:40:13.420976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273373+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:40:13.273373+0000","last_peered":"2024-09-05T06:40:13.273373+0000","last_clean":"2024-09-05T06:40:13.273373+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:40:13.273373+0000","last_undegraded":"2024-09-05T06:40:13.273373+0000","last_fullsized":"2024-09-05T06:40:13.273373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271907+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:40:13.271907+0000","last_peered":"2024-09-05T06:40:13.271907+0000","last_clean":"2024-09-05T06:40:13.271907+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:40:13.271907+0000","last_undegraded":"2024-09-05T06:40:13.271907+0000","last_fullsized":"2024-09-05T06:40:13.271907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274549+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:40:13.274549+0000","last_peered":"2024-09-05T06:40:13.274549+0000","last_clean":"2024-09-05T06:40:13.274549+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:40:13.274549+0000","last_undegraded":"2024-09-05T06:40:13.274549+0000","last_fullsized":"2024-09-05T06:40:13.274549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273402+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:40:13.273402+0000","last_peered":"2024-09-05T06:40:13.273402+0000","last_clean":"2024-09-05T06:40:13.273402+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:40:13.273402+0000","last_undegraded":"2024-09-05T06:40:13.273402+0000","last_fullsized":"2024-09-05T06:40:13.273402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:23.925780+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:40:23.925780+0000","last_peered":"2024-09-05T06:40:23.925780+0000","last_clean":"2024-09-05T06:40:23.925780+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:40:23.925780+0000","last_undegraded":"2024-09-05T06:40:23.925780+0000","last_fullsized":"2024-09-05T06:40:23.925780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":120,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270241+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:40:13.270241+0000","last_peered":"2024-09-05T06:40:13.270241+0000","last_clean":"2024-09-05T06:40:13.270241+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:40:13.270241+0000","last_undegraded":"2024-09-05T06:40:13.270241+0000","last_fullsized":"2024-09-05T06:40:13.270241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421066+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:40:13.421066+0000","last_peered":"2024-09-05T06:40:13.421066+0000","last_clean":"2024-09-05T06:40:13.421066+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:40:13.421066+0000","last_undegraded":"2024-09-05T06:40:13.421066+0000","last_fullsized":"2024-09-05T06:40:13.421066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.274651+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:40:13.274651+0000","last_peered":"2024-09-05T06:40:13.274651+0000","last_clean":"2024-09-05T06:40:13.274651+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:40:13.274651+0000","last_undegraded":"2024-09-05T06:40:13.274651+0000","last_fullsized":"2024-09-05T06:40:13.274651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272601+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:40:13.272601+0000","last_peered":"2024-09-05T06:40:13.272601+0000","last_clean":"2024-09-05T06:40:13.272601+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:40:13.272601+0000","last_undegraded":"2024-09-05T06:40:13.272601+0000","last_fullsized":"2024-09-05T06:40:13.272601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499809+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:40:13.499809+0000","last_peered":"2024-09-05T06:40:13.499809+0000","last_clean":"2024-09-05T06:40:13.499809+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:40:13.499809+0000","last_undegraded":"2024-09-05T06:40:13.499809+0000","last_fullsized":"2024-09-05T06:40:13.499809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272033+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:40:13.272033+0000","last_peered":"2024-09-05T06:40:13.272033+0000","last_clean":"2024-09-05T06:40:13.272033+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:40:13.272033+0000","last_undegraded":"2024-09-05T06:40:13.272033+0000","last_fullsized":"2024-09-05T06:40:13.272033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270061+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:40:13.270061+0000","last_peered":"2024-09-05T06:40:13.270061+0000","last_clean":"2024-09-05T06:40:13.270061+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:40:13.270061+0000","last_undegraded":"2024-09-05T06:40:13.270061+0000","last_fullsized":"2024-09-05T06:40:13.270061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420966+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:40:13.420966+0000","last_peered":"2024-09-05T06:40:13.420966+0000","last_clean":"2024-09-05T06:40:13.420966+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:40:13.420966+0000","last_undegraded":"2024-09-05T06:40:13.420966+0000","last_fullsized":"2024-09-05T06:40:13.420966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:23.922608+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:40:23.922608+0000","last_peered":"2024-09-05T06:40:23.922608+0000","last_clean":"2024-09-05T06:40:23.922608+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:40:23.922608+0000","last_undegraded":"2024-09-05T06:40:23.922608+0000","last_fullsized":"2024-09-05T06:40:23.922608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420546+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:40:13.420546+0000","last_peered":"2024-09-05T06:40:13.420546+0000","last_clean":"2024-09-05T06:40:13.420546+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:40:13.420546+0000","last_undegraded":"2024-09-05T06:40:13.420546+0000","last_fullsized":"2024-09-05T06:40:13.420546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499446+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:40:13.499446+0000","last_peered":"2024-09-05T06:40:13.499446+0000","last_clean":"2024-09-05T06:40:13.499446+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:40:13.499446+0000","last_undegraded":"2024-09-05T06:40:13.499446+0000","last_fullsized":"2024-09-05T06:40:13.499446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270151+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:40:13.270151+0000","last_peered":"2024-09-05T06:40:13.270151+0000","last_clean":"2024-09-05T06:40:13.270151+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:40:13.270151+0000","last_undegraded":"2024-09-05T06:40:13.270151+0000","last_fullsized":"2024-09-05T06:40:13.270151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273811+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:40:13.273811+0000","last_peered":"2024-09-05T06:40:13.273811+0000","last_clean":"2024-09-05T06:40:13.273811+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:40:13.273811+0000","last_undegraded":"2024-09-05T06:40:13.273811+0000","last_fullsized":"2024-09-05T06:40:13.273811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421224+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:40:13.421224+0000","last_peered":"2024-09-05T06:40:13.421224+0000","last_clean":"2024-09-05T06:40:13.421224+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:40:13.421224+0000","last_undegraded":"2024-09-05T06:40:13.421224+0000","last_fullsized":"2024-09-05T06:40:13.421224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420434+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:40:13.420434+0000","last_peered":"2024-09-05T06:40:13.420434+0000","last_clean":"2024-09-05T06:40:13.420434+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:40:13.420434+0000","last_undegraded":"2024-09-05T06:40:13.420434+0000","last_fullsized":"2024-09-05T06:40:13.420434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273182+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:40:13.273182+0000","last_peered":"2024-09-05T06:40:13.273182+0000","last_clean":"2024-09-05T06:40:13.273182+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:40:13.273182+0000","last_undegraded":"2024-09-05T06:40:13.273182+0000","last_fullsized":"2024-09-05T06:40:13.273182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420547+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:40:13.420547+0000","last_peered":"2024-09-05T06:40:13.420547+0000","last_clean":"2024-09-05T06:40:13.420547+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:40:13.420547+0000","last_undegraded":"2024-09-05T06:40:13.420547+0000","last_fullsized":"2024-09-05T06:40:13.420547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270342+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:40:13.270342+0000","last_peered":"2024-09-05T06:40:13.270342+0000","last_clean":"2024-09-05T06:40:13.270342+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:40:13.270342+0000","last_undegraded":"2024-09-05T06:40:13.270342+0000","last_fullsized":"2024-09-05T06:40:13.270342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271719+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:40:13.271719+0000","last_peered":"2024-09-05T06:40:13.271719+0000","last_clean":"2024-09-05T06:40:13.271719+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:40:13.271719+0000","last_undegraded":"2024-09-05T06:40:13.271719+0000","last_fullsized":"2024-09-05T06:40:13.271719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:22.921986+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:40:22.921986+0000","last_peered":"2024-09-05T06:40:22.921986+0000","last_clean":"2024-09-05T06:40:22.921986+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:40:22.921986+0000","last_undegraded":"2024-09-05T06:40:22.921986+0000","last_fullsized":"2024-09-05T06:40:22.921986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273080+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:40:13.273080+0000","last_peered":"2024-09-05T06:40:13.273080+0000","last_clean":"2024-09-05T06:40:13.273080+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:40:13.273080+0000","last_undegraded":"2024-09-05T06:40:13.273080+0000","last_fullsized":"2024-09-05T06:40:13.273080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272054+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:40:13.272054+0000","last_peered":"2024-09-05T06:40:13.272054+0000","last_clean":"2024-09-05T06:40:13.272054+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:40:13.272054+0000","last_undegraded":"2024-09-05T06:40:13.272054+0000","last_fullsized":"2024-09-05T06:40:13.272054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420700+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:40:13.420700+0000","last_peered":"2024-09-05T06:40:13.420700+0000","last_clean":"2024-09-05T06:40:13.420700+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:40:13.420700+0000","last_undegraded":"2024-09-05T06:40:13.420700+0000","last_fullsized":"2024-09-05T06:40:13.420700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272923+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:40:13.272923+0000","last_peered":"2024-09-05T06:40:13.272923+0000","last_clean":"2024-09-05T06:40:13.272923+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:40:13.272923+0000","last_undegraded":"2024-09-05T06:40:13.272923+0000","last_fullsized":"2024-09-05T06:40:13.272923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271605+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:40:13.271605+0000","last_peered":"2024-09-05T06:40:13.271605+0000","last_clean":"2024-09-05T06:40:13.271605+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:40:13.271605+0000","last_undegraded":"2024-09-05T06:40:13.271605+0000","last_fullsized":"2024-09-05T06:40:13.271605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499407+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:40:13.499407+0000","last_peered":"2024-09-05T06:40:13.499407+0000","last_clean":"2024-09-05T06:40:13.499407+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:40:13.499407+0000","last_undegraded":"2024-09-05T06:40:13.499407+0000","last_fullsized":"2024-09-05T06:40:13.499407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.500615+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:40:13.500615+0000","last_peered":"2024-09-05T06:40:13.500615+0000","last_clean":"2024-09-05T06:40:13.500615+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:40:13.500615+0000","last_undegraded":"2024-09-05T06:40:13.500615+0000","last_fullsized":"2024-09-05T06:40:13.500615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421442+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:40:13.421442+0000","last_peered":"2024-09-05T06:40:13.421442+0000","last_clean":"2024-09-05T06:40:13.421442+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:40:13.421442+0000","last_undegraded":"2024-09-05T06:40:13.421442+0000","last_fullsized":"2024-09-05T06:40:13.421442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272490+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:40:13.272490+0000","last_peered":"2024-09-05T06:40:13.272490+0000","last_clean":"2024-09-05T06:40:13.272490+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:40:13.272490+0000","last_undegraded":"2024-09-05T06:40:13.272490+0000","last_fullsized":"2024-09-05T06:40:13.272490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421097+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:40:13.421097+0000","last_peered":"2024-09-05T06:40:13.421097+0000","last_clean":"2024-09-05T06:40:13.421097+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:40:13.421097+0000","last_undegraded":"2024-09-05T06:40:13.421097+0000","last_fullsized":"2024-09-05T06:40:13.421097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273517+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:40:13.273517+0000","last_peered":"2024-09-05T06:40:13.273517+0000","last_clean":"2024-09-05T06:40:13.273517+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:40:13.273517+0000","last_undegraded":"2024-09-05T06:40:13.273517+0000","last_fullsized":"2024-09-05T06:40:13.273517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420760+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:40:13.420760+0000","last_peered":"2024-09-05T06:40:13.420760+0000","last_clean":"2024-09-05T06:40:13.420760+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:40:13.420760+0000","last_undegraded":"2024-09-05T06:40:13.420760+0000","last_fullsized":"2024-09-05T06:40:13.420760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420806+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:40:13.420806+0000","last_peered":"2024-09-05T06:40:13.420806+0000","last_clean":"2024-09-05T06:40:13.420806+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:40:13.420806+0000","last_undegraded":"2024-09-05T06:40:13.420806+0000","last_fullsized":"2024-09-05T06:40:13.420806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273546+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:40:13.273546+0000","last_peered":"2024-09-05T06:40:13.273546+0000","last_clean":"2024-09-05T06:40:13.273546+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:40:13.273546+0000","last_undegraded":"2024-09-05T06:40:13.273546+0000","last_fullsized":"2024-09-05T06:40:13.273546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420982+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:40:13.420982+0000","last_peered":"2024-09-05T06:40:13.420982+0000","last_clean":"2024-09-05T06:40:13.420982+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:40:13.420982+0000","last_undegraded":"2024-09-05T06:40:13.420982+0000","last_fullsized":"2024-09-05T06:40:13.420982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.272205+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:40:13.272205+0000","last_peered":"2024-09-05T06:40:13.272205+0000","last_clean":"2024-09-05T06:40:13.272205+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:40:13.272205+0000","last_undegraded":"2024-09-05T06:40:13.272205+0000","last_fullsized":"2024-09-05T06:40:13.272205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.270572+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:40:13.270572+0000","last_peered":"2024-09-05T06:40:13.270572+0000","last_clean":"2024-09-05T06:40:13.270572+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:40:13.270572+0000","last_undegraded":"2024-09-05T06:40:13.270572+0000","last_fullsized":"2024-09-05T06:40:13.270572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421524+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:40:13.421524+0000","last_peered":"2024-09-05T06:40:13.421524+0000","last_clean":"2024-09-05T06:40:13.421524+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:40:13.421524+0000","last_undegraded":"2024-09-05T06:40:13.421524+0000","last_fullsized":"2024-09-05T06:40:13.421524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420103+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:40:13.420103+0000","last_peered":"2024-09-05T06:40:13.420103+0000","last_clean":"2024-09-05T06:40:13.420103+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:40:13.420103+0000","last_undegraded":"2024-09-05T06:40:13.420103+0000","last_fullsized":"2024-09-05T06:40:13.420103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499051+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:40:13.499051+0000","last_peered":"2024-09-05T06:40:13.499051+0000","last_clean":"2024-09-05T06:40:13.499051+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:40:13.499051+0000","last_undegraded":"2024-09-05T06:40:13.499051+0000","last_fullsized":"2024-09-05T06:40:13.499051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499099+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:40:13.499099+0000","last_peered":"2024-09-05T06:40:13.499099+0000","last_clean":"2024-09-05T06:40:13.499099+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:40:13.499099+0000","last_undegraded":"2024-09-05T06:40:13.499099+0000","last_fullsized":"2024-09-05T06:40:13.499099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273476+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:40:13.273476+0000","last_peered":"2024-09-05T06:40:13.273476+0000","last_clean":"2024-09-05T06:40:13.273476+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:40:13.273476+0000","last_undegraded":"2024-09-05T06:40:13.273476+0000","last_fullsized":"2024-09-05T06:40:13.273476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273159+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:40:13.273159+0000","last_peered":"2024-09-05T06:40:13.273159+0000","last_clean":"2024-09-05T06:40:13.273159+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:40:13.273159+0000","last_undegraded":"2024-09-05T06:40:13.273159+0000","last_fullsized":"2024-09-05T06:40:13.273159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271852+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:40:13.271852+0000","last_peered":"2024-09-05T06:40:13.271852+0000","last_clean":"2024-09-05T06:40:13.271852+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:40:13.271852+0000","last_undegraded":"2024-09-05T06:40:13.271852+0000","last_fullsized":"2024-09-05T06:40:13.271852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.422370+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:40:13.422370+0000","last_peered":"2024-09-05T06:40:13.422370+0000","last_clean":"2024-09-05T06:40:13.422370+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:40:13.422370+0000","last_undegraded":"2024-09-05T06:40:13.422370+0000","last_fullsized":"2024-09-05T06:40:13.422370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420810+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:40:13.420810+0000","last_peered":"2024-09-05T06:40:13.420810+0000","last_clean":"2024-09-05T06:40:13.420810+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:40:13.420810+0000","last_undegraded":"2024-09-05T06:40:13.420810+0000","last_fullsized":"2024-09-05T06:40:13.420810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.420712+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:40:13.420712+0000","last_peered":"2024-09-05T06:40:13.420712+0000","last_clean":"2024-09-05T06:40:13.420712+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:40:13.420712+0000","last_undegraded":"2024-09-05T06:40:13.420712+0000","last_fullsized":"2024-09-05T06:40:13.420712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.271346+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:40:13.271346+0000","last_peered":"2024-09-05T06:40:13.271346+0000","last_clean":"2024-09-05T06:40:13.271346+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:40:13.271346+0000","last_undegraded":"2024-09-05T06:40:13.271346+0000","last_fullsized":"2024-09-05T06:40:13.271346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421735+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:40:13.421735+0000","last_peered":"2024-09-05T06:40:13.421735+0000","last_clean":"2024-09-05T06:40:13.421735+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:40:13.421735+0000","last_undegraded":"2024-09-05T06:40:13.421735+0000","last_fullsized":"2024-09-05T06:40:13.421735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273219+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:40:13.273219+0000","last_peered":"2024-09-05T06:40:13.273219+0000","last_clean":"2024-09-05T06:40:13.273219+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:40:13.273219+0000","last_undegraded":"2024-09-05T06:40:13.273219+0000","last_fullsized":"2024-09-05T06:40:13.273219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.273124+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:40:13.273124+0000","last_peered":"2024-09-05T06:40:13.273124+0000","last_clean":"2024-09-05T06:40:13.273124+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:40:13.273124+0000","last_undegraded":"2024-09-05T06:40:13.273124+0000","last_fullsized":"2024-09-05T06:40:13.273124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499452+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:40:13.499452+0000","last_peered":"2024-09-05T06:40:13.499452+0000","last_clean":"2024-09-05T06:40:13.499452+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:40:13.499452+0000","last_undegraded":"2024-09-05T06:40:13.499452+0000","last_fullsized":"2024-09-05T06:40:13.499452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.499502+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:40:13.499502+0000","last_peered":"2024-09-05T06:40:13.499502+0000","last_clean":"2024-09-05T06:40:13.499502+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:40:13.499502+0000","last_undegraded":"2024-09-05T06:40:13.499502+0000","last_fullsized":"2024-09-05T06:40:13.499502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-05T06:40:13.421504+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:40:13.421504+0000","last_peered":"2024-09-05T06:40:13.421504+0000","last_clean":"2024-09-05T06:40:13.421504+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:40:13.421504+0000","last_undegraded":"2024-09-05T06:40:13.421504+0000","last_fullsized":"2024-09-05T06:40:13.421504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71272,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":566,"num_read_kb":361,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":213816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612640,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":510584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1728512,"data_stored":1161835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1060864,"data_stored":514588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1048576,"data_stored":510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462963,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1650688,"data_stored":1107762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":505996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855421,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1658880,"data_stored":1102951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215175,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":505704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":507639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1101824,"data_stored":571090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":507452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:40:28.396 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T06:40:28.396 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T06:40:28.397 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-05T06:40:28.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph health --format=json 2024-09-05T06:40:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:28 smithi115 ceph-mon[34791]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:28.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:28 smithi086 ceph-mon[30943]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:28.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:28 smithi096 ceph-mon[34468]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 -- 172.21.15.86:0/3441404282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2118102da0 msgr2=0x7f2118103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 --2- 172.21.15.86:0/3441404282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2118102da0 0x7f2118103180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2108009900 tx=0x7f210802a480 comp rx=0 tx=0).stop 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 -- 172.21.15.86:0/3441404282 shutdown_connections 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 --2- 172.21.15.86:0/3441404282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f211810c180 0x7f211810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 --2- 172.21.15.86:0/3441404282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f211810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 --2- 172.21.15.86:0/3441404282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2118102da0 0x7f2118103180 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 -- 172.21.15.86:0/3441404282 >> 172.21.15.86:0/3441404282 conn(0x7f21180fc7a0 msgr2=0x7f21180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 -- 172.21.15.86:0/3441404282 shutdown_connections 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 -- 172.21.15.86:0/3441404282 wait complete. 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.118+0000 7f211d1fe640 1 Processor -- start 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 -- start start 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2118102da0 0x7f2118198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f21181988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2118111e10 con 0x7f211810c180 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2118111c90 con 0x7f2118102da0 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211d1fe640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2118111f90 con 0x7f21181036c0 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:29.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36460/0 (socket says 172.21.15.86:36460) 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 -- 172.21.15.86:0/2655821205 learned_addr learned my addr 172.21.15.86:0/2655821205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f2116d9d640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2118102da0 0x7f2118198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211659c640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f21181988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 -- 172.21.15.86:0/2655821205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 msgr2=0x7f21181988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f21181988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 -- 172.21.15.86:0/2655821205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2118102da0 msgr2=0x7f2118198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2118102da0 0x7f2118198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211759e640 1 -- 172.21.15.86:0/2655821205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211819d320 con 0x7f211810c180 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.119+0000 7f211659c640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f21181988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f211759e640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f211000cad0 tx=0x7f211000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000ae20 con 0x7f211810c180 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2110005020 con 0x7f211810c180 2024-09-05T06:40:29.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211819d610 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000f6b0 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.120+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21181a4ef0 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.121+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2118103180 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.122+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f21100051d0 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.123+0000 7f20fffff640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 0x7f21000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.123+0000 7f2116d9d640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 0x7f21000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.123+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f21100cf9c0 con 0x7f211810c180 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.124+0000 7f2116d9d640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 0x7f21000a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2108031440 tx=0x7f2108005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:29.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.127+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2110014030 con 0x7f211810c180 2024-09-05T06:40:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.332+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f21181996d0 con 0x7f211810c180 2024-09-05T06:40:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.332+0000 7f20fffff640 1 -- 172.21.15.86:0/2655821205 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f21181996d0 con 0x7f211810c180 2024-09-05T06:40:29.333 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:29.333 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 msgr2=0x7f21000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 0x7f21000a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2108031440 tx=0x7f2108005e80 comp rx=0 tx=0).stop 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 msgr2=0x7f211819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f211000cad0 tx=0x7f211000cf90 comp rx=0 tx=0).stop 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 shutdown_connections 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f211810c180 0x7f211819cc40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21181036c0 0x7f21181988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f21000a2c10 0x7f21000a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 --2- 172.21.15.86:0/2655821205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2118102da0 0x7f2118198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 >> 172.21.15.86:0/2655821205 conn(0x7f21180fc7a0 msgr2=0x7f21180fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 shutdown_connections 2024-09-05T06:40:29.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:29.335+0000 7f211d1fe640 1 -- 172.21.15.86:0/2655821205 wait complete. 2024-09-05T06:40:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:29 smithi115 ceph-mon[34791]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:29.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:29 smithi086 ceph-mon[30943]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:29 smithi096 ceph-mon[34468]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:40:30.157 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-05T06:40:30.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:40:30.402 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:30.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:30 smithi086 ceph-mon[30943]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:30.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:30 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2655821205' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T06:40:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:30 smithi115 ceph-mon[34791]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:30.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:30 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2655821205' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T06:40:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:30 smithi096 ceph-mon[34468]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:30 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2655821205' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T06:40:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.930+0000 7f1333ff5640 1 -- 172.21.15.86:0/985864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102ef0 msgr2=0x7f132c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.930+0000 7f1333ff5640 1 --2- 172.21.15.86:0/985864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102ef0 0x7f132c10da40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1324009ca0 tx=0x7f132402a950 comp rx=0 tx=0).stop 2024-09-05T06:40:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 -- 172.21.15.86:0/985864585 shutdown_connections 2024-09-05T06:40:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 --2- 172.21.15.86:0/985864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102ef0 0x7f132c10da40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 --2- 172.21.15.86:0/985864585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102570 0x7f132c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 --2- 172.21.15.86:0/985864585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 -- 172.21.15.86:0/985864585 >> 172.21.15.86:0/985864585 conn(0x7f132c0fc7d0 msgr2=0x7f132c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 -- 172.21.15.86:0/985864585 shutdown_connections 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 -- 172.21.15.86:0/985864585 wait complete. 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 Processor -- start 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.931+0000 7f1333ff5640 1 -- start start 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 0x7f132c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110e50 con 0x7f132c102570 2024-09-05T06:40:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110cd0 con 0x7f132c100e20 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f132c110fd0 con 0x7f132c102ef0 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36492/0 (socket says 172.21.15.86:36492) 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 -- 172.21.15.86:0/3985656196 learned_addr learned my addr 172.21.15.86:0/3985656196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331d91640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1332592640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 0x7f132c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 -- 172.21.15.86:0/3985656196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 msgr2=0x7f132c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 0x7f132c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 -- 172.21.15.86:0/3985656196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 msgr2=0x7f132c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 -- 172.21.15.86:0/3985656196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c19c240 con 0x7f132c102570 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1332592640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 0x7f132c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331d91640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1331590640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f131c00b4a0 tx=0x7f131c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c014070 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f131c010ac0 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c19c530 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.932+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c017c30 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.933+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1a3e10 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.934+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c101200 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.935+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f131c017070 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.935+0000 7f1322ffd640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 0x7f13180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.935+0000 7f1331d91640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 0x7f13180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.935+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f131c0cec90 con 0x7f132c102570 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.936+0000 7f1331d91640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 0x7f13180a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f132800abe0 tx=0x7f132800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:30.938+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f131c09b7d0 con 0x7f132c102570 2024-09-05T06:40:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.139+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f132c1a4070 con 0x7f132c102570 2024-09-05T06:40:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.140+0000 7f1322ffd640 1 -- 172.21.15.86:0/3985656196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7f131c09b0d0 con 0x7f132c102570 2024-09-05T06:40:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:31.140 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:40:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 msgr2=0x7f13180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 0x7f13180a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f132800abe0 tx=0x7f132800a5c0 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 msgr2=0x7f132c1977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f131c00b4a0 tx=0x7f131c00b960 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 shutdown_connections 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f132c102ef0 0x7f132c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c102570 0x7f132c1977d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f13180a2c10 0x7f13180a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 --2- 172.21.15.86:0/3985656196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c100e20 0x7f132c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 >> 172.21.15.86:0/3985656196 conn(0x7f132c0fc7d0 msgr2=0x7f132c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 shutdown_connections 2024-09-05T06:40:31.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:31.143+0000 7f1333ff5640 1 -- 172.21.15.86:0/3985656196 wait complete. 2024-09-05T06:40:31.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:31 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3985656196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:31 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3985656196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:31 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3985656196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:31.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:40:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:32.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:32 smithi086 ceph-mon[30943]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:32 smithi115 ceph-mon[34791]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 -- 172.21.15.86:0/2250935657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774101de0 msgr2=0x7f97741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 --2- 172.21.15.86:0/2250935657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774101de0 0x7f97741021c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9764009900 tx=0x7f976402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 -- 172.21.15.86:0/2250935657 shutdown_connections 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 --2- 172.21.15.86:0/2250935657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774106f10 0x7f977410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 --2- 172.21.15.86:0/2250935657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774102790 0x7f97741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 --2- 172.21.15.86:0/2250935657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774101de0 0x7f97741021c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 -- 172.21.15.86:0/2250935657 >> 172.21.15.86:0/2250935657 conn(0x7f97740fd8c0 msgr2=0x7f97740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 -- 172.21.15.86:0/2250935657 shutdown_connections 2024-09-05T06:40:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.663+0000 7f9779e7e640 1 -- 172.21.15.86:0/2250935657 wait complete. 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 Processor -- start 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 -- start start 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774101de0 0x7f97741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774102790 0x7f9774198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111de0 con 0x7f9774106f10 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111c60 con 0x7f9774102790 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9779e7e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111f60 con 0x7f9774101de0 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52656/0 (socket says 172.21.15.86:52656) 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 -- 172.21.15.86:0/1796505697 learned_addr learned my addr 172.21.15.86:0/1796505697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9772ffd640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774102790 0x7f9774198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f97737fe640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774101de0 0x7f97741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 -- 172.21.15.86:0/1796505697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774101de0 msgr2=0x7f97741983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774101de0 0x7f97741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 -- 172.21.15.86:0/1796505697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774102790 msgr2=0x7f9774198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774102790 0x7f9774198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.664+0000 7f9773fff640 1 -- 172.21.15.86:0/1796505697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977419d370 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9773fff640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f976c00cad0 tx=0x7f976c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c00ae20 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f976c005020 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977419d660 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c00f6b0 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.665+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a4f40 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.666+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97741021c0 con 0x7f9774106f10 2024-09-05T06:40:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.667+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f976c0051d0 con 0x7f9774106f10 2024-09-05T06:40:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.668+0000 7f9770ff9640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 0x7f97600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.668+0000 7f97737fe640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 0x7f97600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.668+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f976c0cf2c0 con 0x7f9774106f10 2024-09-05T06:40:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.669+0000 7f97737fe640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 0x7f97600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9764006010 tx=0x7f9764000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.672+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f976c014030 con 0x7f9774106f10 2024-09-05T06:40:32.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:32 smithi096 ceph-mon[34468]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.874+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9774109b60 con 0x7f9774106f10 2024-09-05T06:40:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.875+0000 7f9770ff9640 1 -- 172.21.15.86:0/1796505697 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7f976c09bcf0 con 0x7f9774106f10 2024-09-05T06:40:32.875 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:32.875 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:40:32.875 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 msgr2=0x7f97600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 0x7f97600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9764006010 tx=0x7f9764000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 msgr2=0x7f977419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f976c00cad0 tx=0x7f976c00cf90 comp rx=0 tx=0).stop 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 shutdown_connections 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9774106f10 0x7f977419cc90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f97600a2c10 0x7f97600a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9774102790 0x7f9774198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 --2- 172.21.15.86:0/1796505697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9774101de0 0x7f97741983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 >> 172.21.15.86:0/1796505697 conn(0x7f97740fd8c0 msgr2=0x7f97740fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 shutdown_connections 2024-09-05T06:40:32.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:32.877+0000 7f9779e7e640 1 -- 172.21.15.86:0/1796505697 wait complete. 2024-09-05T06:40:33.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1796505697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:33.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:40:33.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:33 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1796505697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:33 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1796505697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:34.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/3046145488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e74102d30 msgr2=0x7f5e74103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/3046145488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e74102d30 0x7f5e74103110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009930 tx=0x7f5e6402a420 comp rx=0 tx=0).stop 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/3046145488 shutdown_connections 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/3046145488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e741082a0 0x7f5e7410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/3046145488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741036e0 0x7f5e74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/3046145488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e74102d30 0x7f5e74103110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/3046145488 >> 172.21.15.86:0/3046145488 conn(0x7f5e740fc7d0 msgr2=0x7f5e740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.353+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/3046145488 shutdown_connections 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.354+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/3046145488 wait complete. 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.354+0000 7f5e7a4b5640 1 Processor -- start 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.354+0000 7f5e7a4b5640 1 -- start start 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74102d30 0x7f5e741984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:34.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 0x7f5e7419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74111fb0 con 0x7f5e741036e0 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74111e30 con 0x7f5e741082a0 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e7a4b5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74112130 con 0x7f5e74102d30 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52660/0 (socket says 172.21.15.86:52660) 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 -- 172.21.15.86:0/718789004 learned_addr learned my addr 172.21.15.86:0/718789004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e73fff640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74102d30 0x7f5e741984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e78a52640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 0x7f5e7419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 -- 172.21.15.86:0/718789004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74102d30 msgr2=0x7f5e741984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74102d30 0x7f5e741984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 -- 172.21.15.86:0/718789004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 msgr2=0x7f5e7419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:34.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 0x7f5e7419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 -- 172.21.15.86:0/718789004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7419d460 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e78a52640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 0x7f5e7419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e737fe640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c8c0 tx=0x7f5e6800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.355+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68007bb0 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.356+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7419d750 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.356+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e68007d50 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.356+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e680187c0 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.356+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e741a5030 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.356+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7406a850 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.358+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e6800f450 con 0x7f5e741036e0 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.358+0000 7f5e717fa640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 0x7f5e5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:34.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.359+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f5e680cf990 con 0x7f5e741036e0 2024-09-05T06:40:34.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.359+0000 7f5e73fff640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 0x7f5e5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:34.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.359+0000 7f5e73fff640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 0x7f5e5c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5e64006010 tx=0x7f5e64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:34.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.366+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e68014030 con 0x7f5e741036e0 2024-09-05T06:40:34.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:34 smithi086 ceph-mon[30943]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.540+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e74103110 con 0x7f5e741036e0 2024-09-05T06:40:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.542+0000 7f5e717fa640 1 -- 172.21.15.86:0/718789004 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24905 (secure 0 0 0) 0x7f5e74103110 con 0x7f5e741036e0 2024-09-05T06:40:34.543 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:34.543 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:40:13.267085+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 msgr2=0x7f5e5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 0x7f5e5c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5e64006010 tx=0x7f5e64000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 msgr2=0x7f5e741989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c8c0 tx=0x7f5e6800cd80 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 shutdown_connections 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f5e5c0a2c10 0x7f5e5c0a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741082a0 0x7f5e7419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e741036e0 0x7f5e741989f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 --2- 172.21.15.86:0/718789004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74102d30 0x7f5e741984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 >> 172.21.15.86:0/718789004 conn(0x7f5e740fc7d0 msgr2=0x7f5e74105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:34.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 shutdown_connections 2024-09-05T06:40:34.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:34.545+0000 7f5e7a4b5640 1 -- 172.21.15.86:0/718789004 wait complete. 2024-09-05T06:40:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:34 smithi115 ceph-mon[34791]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:34.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:34 smithi096 ceph-mon[34468]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:35.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:40:35.405 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:35.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:35 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/718789004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:40:35.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:35 smithi086 ceph-mon[30943]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:35.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:35 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/718789004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:40:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:35 smithi115 ceph-mon[34791]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:35 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/718789004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:40:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:35 smithi096 ceph-mon[34468]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 -- 172.21.15.86:0/1896414044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848106a80 msgr2=0x7f2848106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 --2- 172.21.15.86:0/1896414044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848106a80 0x7f2848106e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2838009930 tx=0x7f283802a420 comp rx=0 tx=0).stop 2024-09-05T06:40:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 -- 172.21.15.86:0/1896414044 shutdown_connections 2024-09-05T06:40:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 --2- 172.21.15.86:0/1896414044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848101420 0x7f284810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 --2- 172.21.15.86:0/1896414044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848100a80 0x7f2848100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 --2- 172.21.15.86:0/1896414044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848106a80 0x7f2848106e60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 -- 172.21.15.86:0/1896414044 >> 172.21.15.86:0/1896414044 conn(0x7f28480fc7a0 msgr2=0x7f28480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 -- 172.21.15.86:0/1896414044 shutdown_connections 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 -- 172.21.15.86:0/1896414044 wait complete. 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.901+0000 7f284e385640 1 Processor -- start 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 -- start start 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848101420 0x7f2848198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 0x7f284819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2848110d50 con 0x7f2848100a80 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2848110bd0 con 0x7f2848106a80 2024-09-05T06:40:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284e385640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2848110ed0 con 0x7f2848101420 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52686/0 (socket says 172.21.15.86:52686) 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 -- 172.21.15.86:0/3146041969 learned_addr learned my addr 172.21.15.86:0/3146041969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284c922640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 0x7f284819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f283ffff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848101420 0x7f2848198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 -- 172.21.15.86:0/3146041969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848101420 msgr2=0x7f2848198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848101420 0x7f2848198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 -- 172.21.15.86:0/3146041969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 msgr2=0x7f284819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 0x7f284819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 -- 172.21.15.86:0/3146041969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284819d3f0 con 0x7f2848100a80 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f284c922640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 0x7f284819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.902+0000 7f2847fff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2838004770 tx=0x7f28380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.903+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283803f070 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.903+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283803b460 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.903+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284819d680 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.903+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838035070 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.903+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28481a4fc0 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.904+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28481021c0 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.905+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f283803b610 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.905+0000 7f2845ffb640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 0x7f28340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.906+0000 7f283ffff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 0x7f28340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.906+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f28380ef9e0 con 0x7f2848100a80 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.906+0000 7f283ffff640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 0x7f28340a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28481062f0 tx=0x7f2830008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:35.910+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28380bc410 con 0x7f2848100a80 2024-09-05T06:40:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.111+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f284810a000 con 0x7f2848100a80 2024-09-05T06:40:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.111+0000 7f2845ffb640 1 -- 172.21.15.86:0/3146041969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7f28380bbd10 con 0x7f2848100a80 2024-09-05T06:40:36.112 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:36.112 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:40:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 msgr2=0x7f28340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 0x7f28340a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28481062f0 tx=0x7f2830008040 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 msgr2=0x7f2848198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2838004770 tx=0x7f28380042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 shutdown_connections 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f28340a2c10 0x7f28340a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2848106a80 0x7f284819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2848101420 0x7f2848198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 --2- 172.21.15.86:0/3146041969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2848100a80 0x7f2848198440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 >> 172.21.15.86:0/3146041969 conn(0x7f28480fc7a0 msgr2=0x7f284806ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 shutdown_connections 2024-09-05T06:40:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:36.114+0000 7f284e385640 1 -- 172.21.15.86:0/3146041969 wait complete. 2024-09-05T06:40:36.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:36 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3146041969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:36.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:36 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3146041969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:36.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:36 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3146041969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:36.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T06:38:55.927385+0000', 'modified': '2024-09-05T06:39:33.863378+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24466, 'mds_2': 24461, 'mds_3': 24422, 'mds_4': 24403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3075373024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3075373024}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3075373024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/1507644375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 1507644375}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 1507644375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.115:6833/3498674074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3498674074}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3498674074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.96:6833/1668283475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 1668283475}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 1668283475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/1272008759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 1272008759}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 1272008759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/3823127248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3823127248}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3823127248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6837/2087354694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 2087354694}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2087354694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.96:6837/1007288434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 1007288434}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 1007288434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.115:6839/4249007973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4249007973}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4249007973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/2996609527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2996609527}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2996609527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24466, 24461, 24422, 24403]} 2024-09-05T06:40:36.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T06:40:36.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.l status 2024-09-05T06:40:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:37 smithi086 ceph-mon[30943]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- 172.21.15.86:0/3454588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 msgr2=0x7fb38810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 --2- 172.21.15.86:0/3454588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb38810d950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb380009cd0 tx=0x7fb38002a8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- 172.21.15.86:0/3454588284 shutdown_connections 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 --2- 172.21.15.86:0/3454588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb38810d950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 --2- 172.21.15.86:0/3454588284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388100a80 0x7fb388100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 --2- 172.21.15.86:0/3454588284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388106a80 0x7fb388106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- 172.21.15.86:0/3454588284 >> 172.21.15.86:0/3454588284 conn(0x7fb3880fc7a0 msgr2=0x7fb3880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- 172.21.15.86:0/3454588284 shutdown_connections 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- 172.21.15.86:0/3454588284 wait complete. 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 Processor -- start 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.610+0000 7fb38cfea640 1 -- start start 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 0x7fb388198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388106a80 0x7fb38819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb388110d30 con 0x7fb388101420 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb388110bb0 con 0x7fb388100a80 2024-09-05T06:40:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb388110eb0 con 0x7fb388106a80 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52708/0 (socket says 172.21.15.86:52708) 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 -- 172.21.15.86:0/2252553840 learned_addr learned my addr 172.21.15.86:0/2252553840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb386d9d640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388106a80 0x7fb38819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38659c640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 0x7fb388198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 -- 172.21.15.86:0/2252553840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388106a80 msgr2=0x7fb38819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388106a80 0x7fb38819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 -- 172.21.15.86:0/2252553840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 msgr2=0x7fb388198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 0x7fb388198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 -- 172.21.15.86:0/2252553840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38819d330 con 0x7fb388101420 2024-09-05T06:40:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38659c640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 0x7fb388198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c970 tx=0x7fb37000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37f7fe640 1 -- 172.21.15.86:0/2252553840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007bb0 con 0x7fb388101420 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38819d620 con 0x7fb388101420 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37f7fe640 1 -- 172.21.15.86:0/2252553840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb370007d50 con 0x7fb388101420 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb37f7fe640 1 -- 172.21.15.86:0/2252553840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f450 con 0x7fb388101420 2024-09-05T06:40:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.611+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881a4f00 con 0x7fb388101420 2024-09-05T06:40:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.614+0000 7fb37f7fe640 1 -- 172.21.15.86:0/2252553840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb370004f20 con 0x7fb388101420 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.614+0000 7fb38f3c3740 1 Processor -- start 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 -- start start 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad316090 con 0x55f8ad3b01b0 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad2bda70 con 0x55f8ad3af890 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad312050 con 0x55f8ad3befc0 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52710/0 (socket says 172.21.15.86:52710) 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:60670/0 (socket says 172.21.15.86:60670) 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 -- 172.21.15.86:0/4138158477 learned_addr learned my addr 172.21.15.86:0/4138158477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb37ffff640 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb37f7fe640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 0x7fb3740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 -- 172.21.15.86:0/4138158477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 msgr2=0x55f8ad3c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3c13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 -- 172.21.15.86:0/4138158477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 msgr2=0x55f8ad3afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.615+0000 7fb38659c640 1 -- 172.21.15.86:0/4138158477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ad2ca130 con 0x55f8ad3b01b0 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb386d9d640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 0x7fb3740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb37f7fe640 1 -- 172.21.15.86:0/2252553840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fb3700cf320 con 0x7fb388101420 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb38659c640 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb37800baf0 tx=0x7fb37800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c64f6bcbf731cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb37cff9640 1 -- 172.21.15.86:0/4138158477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378017420 con 0x55f8ad3b01b0 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb37cff9640 1 -- 172.21.15.86:0/4138158477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb378017a70 con 0x55f8ad3b01b0 2024-09-05T06:40:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.616+0000 7fb37cff9640 1 -- 172.21.15.86:0/4138158477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800f9e0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- 172.21.15.86:0/4138158477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 msgr2=0x55f8ad3bea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb37800baf0 tx=0x7fb37800d340 comp rx=0 tx=0).stop 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb386d9d640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 0x7fb3740a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb380009890 tx=0x7fb3800095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- 172.21.15.86:0/4138158477 shutdown_connections 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3c13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bea80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/4138158477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- 172.21.15.86:0/4138158477 >> 172.21.15.86:0/4138158477 conn(0x55f8ad3abf60 msgr2=0x55f8ad3ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- 172.21.15.86:0/4138158477 shutdown_connections 2024-09-05T06:40:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- 172.21.15.86:0/4138158477 wait complete. 2024-09-05T06:40:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 -- start start 2024-09-05T06:40:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 Processor -- start 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.617+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3c1b90 con 0x55f8ad3b01b0 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad2d0f20 con 0x55f8ad3af890 2024-09-05T06:40:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3c1d10 con 0x55f8ad3befc0 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 10 client.0 _ll_register_callbacks cb 0x55f8ad29a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52722/0 (socket says 172.21.15.86:52722) 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 -- 172.21.15.86:0/612616337 learned_addr learned my addr 172.21.15.86:0/612616337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb386d9d640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38659c640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 -- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 msgr2=0x55f8ad3bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3bc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 -- 172.21.15.86:0/612616337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 msgr2=0x55f8ad3be7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3be7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 -- 172.21.15.86:0/612616337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ad0872b0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38659c640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb37ffff640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb37009b5e0 tx=0x7fb37000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34cdde749c8b297 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001d040 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fb37009b8b0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb378017a70 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.618+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb37009b190 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 10 client.15048 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8ad3cd4b0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb378017a70 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb3700a3e30 con 0x55f8ad3b01b0 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 10 client.15048 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 20 client.15048 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad3cfb10 0x55f8ad3d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 4 client.15048 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ad323600 con 0x55f8ad3cfb10 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.619+0000 7fb38659c640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad3cfb10 0x55f8ad3d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.620+0000 7fb38659c640 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad3cfb10 0x55f8ad3d0f80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2343fe0a7ca77602 server_cookie=f62e41cc550e7945 in_seq=0 out_seq=0 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.620+0000 7fb36dffb640 10 client.15048 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.620+0000 7fb36dffb640 1 -- 172.21.15.86:0/612616337 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8ad323600 con 0x55f8ad3cfb10 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb36dffb640 10 client.15048 handle_command_reply: tid=0 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 client.15048 shutdown 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 msgr2=0x55f8ad3bbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb37009b5e0 tx=0x7fb37000a570 comp rx=0 tx=0).stop 2024-09-05T06:40:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 shutdown_connections 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad3cfb10 0x55f8ad3d0f80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3bbb30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb36dffb640 0 client.15048 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/612616337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 >> 172.21.15.86:0/612616337 conn(0x55f8ad3abf60 msgr2=0x55f8ad3b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 shutdown_connections 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 1 -- 172.21.15.86:0/612616337 wait complete. 2024-09-05T06:40:37.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.621+0000 7fb38f3c3740 20 client.15048 trim_cache size 0 max 16384 2024-09-05T06:40:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 Processor -- start 2024-09-05T06:40:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 -- start start 2024-09-05T06:40:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3cf2c0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3cf990 con 0x55f8ad3af890 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3d0060 con 0x55f8ad3befc0 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52738/0 (socket says 172.21.15.86:52738) 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 -- 172.21.15.86:0/2458380945 learned_addr learned my addr 172.21.15.86:0/2458380945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38659c640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 -- 172.21.15.86:0/2458380945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 msgr2=0x55f8ad3b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb386d9d640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 -- 172.21.15.86:0/2458380945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 msgr2=0x55f8ad3bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 -- 172.21.15.86:0/2458380945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ad3bc670 con 0x55f8ad3b01b0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb38659c640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb386d9d640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.639+0000 7fb37ffff640 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb37009b350 tx=0x7fb3700076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7539e4ec831ffab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb36ffff640 1 -- 172.21.15.86:0/2458380945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001d070 con 0x55f8ad3b01b0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb36ffff640 1 -- 172.21.15.86:0/2458380945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3700cf170 con 0x55f8ad3b01b0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb36ffff640 1 -- 172.21.15.86:0/2458380945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37009afb0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 -- 172.21.15.86:0/2458380945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 msgr2=0x55f8ad3b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb37009b350 tx=0x7fb3700076c0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 -- 172.21.15.86:0/2458380945 shutdown_connections 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3befc0 0x55f8ad3b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3b6a20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/2458380945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3af890 0x55f8ad3bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 -- 172.21.15.86:0/2458380945 >> 172.21.15.86:0/2458380945 conn(0x55f8ad3abf60 msgr2=0x55f8ad3ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 -- 172.21.15.86:0/2458380945 shutdown_connections 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.640+0000 7fb38f3c3740 1 -- 172.21.15.86:0/2458380945 wait complete. 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 -- start start 2024-09-05T06:40:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 Processor -- start 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3befc0 0x55f8ad3d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3d0e00 con 0x55f8ad3af890 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3d0730 con 0x55f8ad3befc0 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad3d14d0 con 0x55f8ad3b01b0 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38f3c3740 10 client.0 _ll_register_callbacks cb 0x7fb37000c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52742/0 (socket says 172.21.15.86:52742) 2024-09-05T06:40:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 -- 172.21.15.86:0/3202377709 learned_addr learned my addr 172.21.15.86:0/3202377709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb386d9d640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3befc0 0x55f8ad3d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb37ffff640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 -- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 msgr2=0x55f8ad3d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3d98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 -- 172.21.15.86:0/3202377709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3befc0 msgr2=0x55f8ad3d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3befc0 0x55f8ad3d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 -- 172.21.15.86:0/3202377709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ad3d2e70 con 0x55f8ad3af890 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb37ffff640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb38659c640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb378019040 tx=0x7fb378005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643ffdee3971dfdf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.641+0000 7fb36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800b6c0 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fb37800f6b0 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3700cf170 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb378037050 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 10 client.15054 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8ad3d7860 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3700cf170 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb378028900 con 0x55f8ad3af890 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 10 client.15054 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 20 client.15054 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad43e610 0x55f8ad43d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 4 client.15054 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:40:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.642+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "status"}) -- 0x55f8ad31db10 con 0x55f8ad43e610 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb37ffff640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad43e610 0x55f8ad43d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb37ffff640 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad43e610 0x55f8ad43d320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d8b19e7d877e714 server_cookie=890d6f217dc4264 in_seq=0 out_seq=0 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb36dffb640 10 client.15054 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb36dffb640 1 -- 172.21.15.86:0/3202377709 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fb3700d2e50 con 0x55f8ad43e610 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb36dffb640 10 client.15054 handle_command_reply: tid=0 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb38f3c3740 1 client.15054 shutdown 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 msgr2=0x55f8ad3d9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.643+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb378019040 tx=0x7fb378005ee0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 shutdown_connections 2024-09-05T06:40:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f8ad43e610 0x55f8ad43d320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad3befc0 0x55f8ad3d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb36dffb640 0 client.15054 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad3b01b0 0x55f8ad3d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 --2- 172.21.15.86:0/3202377709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad3af890 0x55f8ad3d9370 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 >> 172.21.15.86:0/3202377709 conn(0x55f8ad3abf60 msgr2=0x55f8ad3ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 shutdown_connections 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 1 -- 172.21.15.86:0/3202377709 wait complete. 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38f3c3740 20 client.15054 trim_cache size 0 max 16384 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:40:37.651 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24403, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 64.794135792999995, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 143.05597576400001 2024-09-05T06:40:37.652 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:40:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 msgr2=0x7fb3740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 0x7fb3740a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb380009890 tx=0x7fb3800095c0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 msgr2=0x7fb3881988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c970 tx=0x7fb37000ce30 comp rx=0 tx=0).stop 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 shutdown_connections 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb388106a80 0x7fb38819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb388101420 0x7fb3881988c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fb3740a2c10 0x7fb3740a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 --2- 172.21.15.86:0/2252553840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb388100a80 0x7fb388198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 >> 172.21.15.86:0/2252553840 conn(0x7fb3880fc7a0 msgr2=0x7fb38806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 shutdown_connections 2024-09-05T06:40:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:37.644+0000 7fb38cfea640 1 -- 172.21.15.86:0/2252553840 wait complete. 2024-09-05T06:40:37.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:37 smithi115 ceph-mon[34791]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:37 smithi096 ceph-mon[34468]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:38.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.k status 2024-09-05T06:40:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:39.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.152+0000 7fac6c8ac640 1 -- 172.21.15.86:0/3570418665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68102790 msgr2=0x7fac681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.152+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/3570418665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68102790 0x7fac681067e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009900 tx=0x7fac5c02a410 comp rx=0 tx=0).stop 2024-09-05T06:40:39.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 -- 172.21.15.86:0/3570418665 shutdown_connections 2024-09-05T06:40:39.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/3570418665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68106f10 0x7fac6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/3570418665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68102790 0x7fac681067e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/3570418665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68101de0 0x7fac681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 -- 172.21.15.86:0/3570418665 >> 172.21.15.86:0/3570418665 conn(0x7fac680fd8c0 msgr2=0x7fac680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 -- 172.21.15.86:0/3570418665 shutdown_connections 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 -- 172.21.15.86:0/3570418665 wait complete. 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 Processor -- start 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.153+0000 7fac6c8ac640 1 -- start start 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 0x7fac680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68102790 0x7fac68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac68111e40 con 0x7fac68106f10 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac68111cc0 con 0x7fac68101de0 2024-09-05T06:40:39.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac68111fc0 con 0x7fac68102790 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52756/0 (socket says 172.21.15.86:52756) 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 -- 172.21.15.86:0/916603705 learned_addr learned my addr 172.21.15.86:0/916603705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6659c640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 0x7fac680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac65d9b640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68102790 0x7fac68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 -- 172.21.15.86:0/916603705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68102790 msgr2=0x7fac68074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68102790 0x7fac68074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 -- 172.21.15.86:0/916603705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 msgr2=0x7fac680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 0x7fac680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 -- 172.21.15.86:0/916603705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac681a68a0 con 0x7fac68106f10 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6659c640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 0x7fac680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac66d9d640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac60009fd0 tx=0x7fac6000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac577fe640 1 -- 172.21.15.86:0/916603705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60011830 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac577fe640 1 -- 172.21.15.86:0/916603705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac60016070 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac681a6b90 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.154+0000 7fac577fe640 1 -- 172.21.15.86:0/916603705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60010de0 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.155+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681a7080 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac577fe640 1 -- 172.21.15.86:0/916603705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac600119d0 con 0x7fac68106f10 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 Processor -- start 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 -- start start 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 0x55f38420ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f38421afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f384174090 con 0x55f38420e2f0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f38411ba70 con 0x55f384218bb0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6ec85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f384170050 con 0x55f38420d9d0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.157+0000 7fac6659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52758/0 (socket says 172.21.15.86:52758) 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 -- 172.21.15.86:0/3261082793 learned_addr learned my addr 172.21.15.86:0/3261082793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac577fe640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 0x7fac500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac65d9b640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 0x55f38420ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac65d9b640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f38421afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac66d9d640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 0x7fac500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 -- 172.21.15.86:0/3261082793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 msgr2=0x55f38420ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 0x55f38420ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 -- 172.21.15.86:0/3261082793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 msgr2=0x55f38421afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f38421afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 -- 172.21.15.86:0/3261082793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f384128130 con 0x55f38420e2f0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac65d9b640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 0x55f38420ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac577fe640 1 -- 172.21.15.86:0/916603705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fac600cec20 con 0x7fac68106f10 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac6659c640 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac5800af40 tx=0x7fac5800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c130f449e5379de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac54ff9640 1 -- 172.21.15.86:0/3261082793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58004d20 con 0x55f38420e2f0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac54ff9640 1 -- 172.21.15.86:0/3261082793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac58011b00 con 0x55f38420e2f0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.158+0000 7fac54ff9640 1 -- 172.21.15.86:0/3261082793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58005630 con 0x55f38420e2f0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- 172.21.15.86:0/3261082793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 msgr2=0x55f384218670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac5800af40 tx=0x7fac5800f7f0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac66d9d640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 0x7fac500a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac60097a30 tx=0x7fac6000bf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- 172.21.15.86:0/3261082793 shutdown_connections 2024-09-05T06:40:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f38421afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f384218670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 --2- 172.21.15.86:0/3261082793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420d9d0 0x55f38420ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- 172.21.15.86:0/3261082793 >> 172.21.15.86:0/3261082793 conn(0x55f384209f60 msgr2=0x55f38420a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- 172.21.15.86:0/3261082793 shutdown_connections 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- 172.21.15.86:0/3261082793 wait complete. 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.159+0000 7fac6ec85740 1 -- start start 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 Processor -- start 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f38420d9d0 0x55f384218480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 0x55f384215d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f38421b780 con 0x55f38420e2f0 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f38412ef20 con 0x55f38420d9d0 2024-09-05T06:40:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f38421b900 con 0x55f384218bb0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6ec85740 10 client.0 _ll_register_callbacks cb 0x55f3840f8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52768/0 (socket says 172.21.15.86:52768) 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 -- 172.21.15.86:0/1304515441 learned_addr learned my addr 172.21.15.86:0/1304515441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f38420d9d0 0x55f384218480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 0x55f384215d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 -- 172.21.15.86:0/1304515441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 msgr2=0x55f384215d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 0x55f384215d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 -- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f38420d9d0 msgr2=0x55f384218480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f38420d9d0 0x55f384218480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 -- 172.21.15.86:0/1304515441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f383ee52b0 con 0x55f38420e2f0 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 0x55f384215d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:39.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac6659c640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f80 tx=0x7fac5800ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab0ed9900bab8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.160+0000 7fac4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac580118e0 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fac58004a40 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac58011b00 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac5802f050 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 10 client.15066 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3842270d0 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac58011b00 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fac58005780 con 0x55f38420e2f0 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 10 client.15066 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 20 client.15066 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384229330 0x55f38422a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 4 client.15066 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.161+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f384181600 con 0x55f384229330 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.162+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384229330 0x55f38422a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.162+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384229330 0x55f38422a7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3092c43124d5922 server_cookie=139f3c1ea7cb5218 in_seq=0 out_seq=0 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.162+0000 7fac4dffb640 10 client.15066 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac4dffb640 1 -- 172.21.15.86:0/1304515441 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f384181600 con 0x55f384229330 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac4dffb640 10 client.15066 handle_command_reply: tid=0 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 client.15066 shutdown 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 msgr2=0x55f3842157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f80 tx=0x7fac5800ac80 comp rx=0 tx=0).stop 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 shutdown_connections 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384229330 0x55f38422a7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f384218bb0 0x55f384215d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420e2f0 0x55f3842157d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1304515441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f38420d9d0 0x55f384218480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 >> 172.21.15.86:0/1304515441 conn(0x55f384209f60 msgr2=0x55f38420fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac4dffb640 0 client.15066 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 shutdown_connections 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 1 -- 172.21.15.86:0/1304515441 wait complete. 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.163+0000 7fac6ec85740 20 client.15066 trim_cache size 0 max 16384 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 Processor -- start 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 -- start start 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f384218480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f3842177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f384228ae0 con 0x55f38420d9d0 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f3842291b0 con 0x55f384218bb0 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.180+0000 7fac6ec85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f384229880 con 0x55f38420e2f0 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52770/0 (socket says 172.21.15.86:52770) 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 -- 172.21.15.86:0/1655969798 learned_addr learned my addr 172.21.15.86:0/1655969798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac6659c640 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f384218480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 -- 172.21.15.86:0/1655969798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 msgr2=0x55f384218480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f384218480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 -- 172.21.15.86:0/1655969798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 msgr2=0x55f3842177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f3842177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 -- 172.21.15.86:0/1655969798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38420d800 con 0x55f38420d9d0 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac65d9b640 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009230 tx=0x7fac5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1b3b084514c76a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac4ffff640 1 -- 172.21.15.86:0/1655969798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c04b020 con 0x55f38420d9d0 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac4ffff640 1 -- 172.21.15.86:0/1655969798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac5c0317e0 con 0x55f38420d9d0 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.181+0000 7fac4ffff640 1 -- 172.21.15.86:0/1655969798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c035040 con 0x55f38420d9d0 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- 172.21.15.86:0/1655969798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 msgr2=0x55f38422a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009230 tx=0x7fac5c000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- 172.21.15.86:0/1655969798 shutdown_connections 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f3842177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f384218480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- 172.21.15.86:0/1655969798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422a0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- 172.21.15.86:0/1655969798 >> 172.21.15.86:0/1655969798 conn(0x55f384209f60 msgr2=0x55f38420a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- 172.21.15.86:0/1655969798 shutdown_connections 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- 172.21.15.86:0/1655969798 wait complete. 2024-09-05T06:40:39.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- start start 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 Processor -- start 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f38422e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f384291a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f38422a620 con 0x55f38420d9d0 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f384229f50 con 0x55f384218bb0 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f38422acf0 con 0x55f38420e2f0 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.182+0000 7fac6ec85740 10 client.0 _ll_register_callbacks cb 0x7fac5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52778/0 (socket says 172.21.15.86:52778) 2024-09-05T06:40:39.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 -- 172.21.15.86:0/492637390 learned_addr learned my addr 172.21.15.86:0/492637390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac6659c640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f38422e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f384291a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 -- 172.21.15.86:0/492637390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 msgr2=0x55f38422e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f38422e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 -- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 msgr2=0x55f384291a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f384291a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 -- 172.21.15.86:0/492637390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f384291f70 con 0x55f38420d9d0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac65d9b640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac5c031440 tx=0x7fac5c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418bb06b6da909c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c058060 con 0x55f38420d9d0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fac5c004450 con 0x55f38420d9d0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5c0317e0 con 0x55f38420d9d0 2024-09-05T06:40:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac5c04b070 con 0x55f38420d9d0 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac6ec85740 10 client.15072 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.183+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3842930c0 con 0x55f38420d9d0 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac5c0317e0 con 0x55f38420d9d0 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fac5c050040 con 0x55f38420d9d0 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24422, 2024-09-05T06:40:39.205 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 67.341513402000004, 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 141.980356868 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 10 client.15072 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:40:39.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 20 client.15072 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384297840 0x55f384296550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 4 client.15072 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "status"}) -- 0x55f38417bb10 con 0x55f384297840 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6659c640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384297840 0x55f384296550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac6659c640 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384297840 0x55f384296550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b049ce46b9f0763 server_cookie=75b5f85624af4a4 in_seq=0 out_seq=0 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.184+0000 7fac4dffb640 10 client.15072 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac4dffb640 1 -- 172.21.15.86:0/492637390 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f38417bb10 con 0x55f384297840 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac4dffb640 10 client.15072 handle_command_reply: tid=0 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 client.15072 shutdown 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 msgr2=0x55f38422db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac5c031440 tx=0x7fac5c009640 comp rx=0 tx=0).stop 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 shutdown_connections 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f384297840 0x55f384296550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f384218bb0 0x55f384291a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f38420e2f0 0x55f38422e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac4dffb640 0 client.15072 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 --2- 172.21.15.86:0/492637390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f38420d9d0 0x55f38422db50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 >> 172.21.15.86:0/492637390 conn(0x55f384209f60 msgr2=0x55f384218480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 shutdown_connections 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 1 -- 172.21.15.86:0/492637390 wait complete. 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6ec85740 20 client.15072 trim_cache size 0 max 16384 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 msgr2=0x7fac500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 0x7fac500a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac60097a30 tx=0x7fac6000bf90 comp rx=0 tx=0).stop 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 msgr2=0x7fac68074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac60009fd0 tx=0x7fac6000b8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 shutdown_connections 2024-09-05T06:40:39.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac68106f10 0x7fac68074f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac68102790 0x7fac68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac500a2c10 0x7fac500a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 --2- 172.21.15.86:0/916603705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101de0 0x7fac680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.185+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 >> 172.21.15.86:0/916603705 conn(0x7fac680fd8c0 msgr2=0x7fac680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.186+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 shutdown_connections 2024-09-05T06:40:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:39.186+0000 7fac6c8ac640 1 -- 172.21.15.86:0/916603705 wait complete. 2024-09-05T06:40:39.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T06:40:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:40.499 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:40 smithi086 ceph-mon[30943]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:40.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:40 smithi096 ceph-mon[34468]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:40.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 -- 172.21.15.86:0/3677493412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858102790 msgr2=0x7f88581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 --2- 172.21.15.86:0/3677493412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858102790 0x7f88581067e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8844009900 tx=0x7f884402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 -- 172.21.15.86:0/3677493412 shutdown_connections 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 --2- 172.21.15.86:0/3677493412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858106f10 0x7f885810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 --2- 172.21.15.86:0/3677493412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858102790 0x7f88581067e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 --2- 172.21.15.86:0/3677493412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 -- 172.21.15.86:0/3677493412 >> 172.21.15.86:0/3677493412 conn(0x7f88580fd8c0 msgr2=0x7f88580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 -- 172.21.15.86:0/3677493412 shutdown_connections 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.581+0000 7f885efde640 1 -- 172.21.15.86:0/3677493412 wait complete. 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 Processor -- start 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 -- start start 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858102790 0x7f8858074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8858111e40 con 0x7f8858106f10 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8858111cc0 con 0x7f8858102790 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885efde640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8858111fc0 con 0x7f8858101de0 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52800/0 (socket says 172.21.15.86:52800) 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 -- 172.21.15.86:0/1859500863 learned_addr learned my addr 172.21.15.86:0/1859500863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:40.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f8857fff640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858102790 0x7f8858074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885cd7a640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 -- 172.21.15.86:0/1859500863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 msgr2=0x7f88580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 -- 172.21.15.86:0/1859500863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858102790 msgr2=0x7f8858074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858102790 0x7f8858074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 -- 172.21.15.86:0/1859500863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88581a68c0 con 0x7f8858106f10 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885cd7a640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.582+0000 7f885d57b640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f885000cce0 tx=0x7f885000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.583+0000 7f8855ffb640 1 -- 172.21.15.86:0/1859500863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885000f450 con 0x7f8858106f10 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.583+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581a6bb0 con 0x7f8858106f10 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.583+0000 7f8855ffb640 1 -- 172.21.15.86:0/1859500863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f885000faa0 con 0x7f8858106f10 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.583+0000 7f8855ffb640 1 -- 172.21.15.86:0/1859500863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850018b70 con 0x7f8858106f10 2024-09-05T06:40:40.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.583+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581a7070 con 0x7f8858106f10 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.585+0000 7f8855ffb640 1 -- 172.21.15.86:0/1859500863 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8850018d10 con 0x7f8858106f10 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.585+0000 7f88613b7740 1 Processor -- start 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 -- start start 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f8855ffb640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 0x7f88480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55657b999090 con 0x55657ba3dbb0 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55657b940a70 con 0x55657ba329d0 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f88613b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55657b995050 con 0x55657ba332f0 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f8857fff640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 0x7f88480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f8855ffb640 1 -- 172.21.15.86:0/1859500863 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f88500cf9f0 con 0x7f8858106f10 2024-09-05T06:40:40.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52812/0 (socket says 172.21.15.86:52812) 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 -- 172.21.15.86:0/1007263599 learned_addr learned my addr 172.21.15.86:0/1007263599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885cd7a640 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 -- 172.21.15.86:0/1007263599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 msgr2=0x55657ba3d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 -- 172.21.15.86:0/1007263599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 msgr2=0x55657ba32db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba32db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 -- 172.21.15.86:0/1007263599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657b94d130 con 0x55657ba3dbb0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f8857fff640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 0x7f88480a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8844031440 tx=0x7f8844031c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.586+0000 7f885d57b640 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8850098180 tx=0x7f885009b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b10f4a6804ea69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f8842ffd640 1 -- 172.21.15.86:0/1007263599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885009bd20 con 0x55657ba3dbb0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f8842ffd640 1 -- 172.21.15.86:0/1007263599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88500a00d0 con 0x55657ba3dbb0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f8842ffd640 1 -- 172.21.15.86:0/1007263599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885009bf80 con 0x55657ba3dbb0 2024-09-05T06:40:40.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 -- 172.21.15.86:0/1007263599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 msgr2=0x55657ba3ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8850098180 tx=0x7f885009b740 comp rx=0 tx=0).stop 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 -- 172.21.15.86:0/1007263599 shutdown_connections 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba3ffa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba3d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 --2- 172.21.15.86:0/1007263599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba32db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 -- 172.21.15.86:0/1007263599 >> 172.21.15.86:0/1007263599 conn(0x55657ba2ef60 msgr2=0x55657ba2f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 -- 172.21.15.86:0/1007263599 shutdown_connections 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.587+0000 7f88613b7740 1 -- 172.21.15.86:0/1007263599 wait complete. 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 -- start start 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 Processor -- start 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 0x55657ba3d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba3ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55657ba40780 con 0x55657ba332f0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55657b953f20 con 0x55657ba3dbb0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55657ba40900 con 0x55657ba329d0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f88613b7740 10 client.0 _ll_register_callbacks cb 0x55657b91d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f8857fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f8857fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52814/0 (socket says 172.21.15.86:52814) 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f8857fff640 1 -- 172.21.15.86:0/3081326711 learned_addr learned my addr 172.21.15.86:0/3081326711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f885d57b640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba3ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.588+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 0x55657ba3d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 -- 172.21.15.86:0/3081326711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 msgr2=0x55657ba3d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 0x55657ba3d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 -- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 msgr2=0x55657ba3ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba3ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 -- 172.21.15.86:0/3081326711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657b70a2b0 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 0x55657ba3d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8857fff640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8844009900 tx=0x7f884402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9598f33c02b06856 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844035040 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f88440035f0 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88500a00d0 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8844057050 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f88613b7740 10 client.15084 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55657ba4c100 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88500a00d0 con 0x55657ba332f0 2024-09-05T06:40:40.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f8844049960 con 0x55657ba332f0 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f88613b7740 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.589+0000 7f88613b7740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f88613b7740 20 client.15084 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657ba4e780 0x55657ba4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f88613b7740 4 client.15084 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55657b9a6600 con 0x55657ba4e780 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657ba4e780 0x55657ba4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f8837fff640 10 client.15084 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.590+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657ba4e780 0x55657ba4fbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f36ee3c21c9de7ea server_cookie=318beb26ebc7a465 in_seq=0 out_seq=0 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f8837fff640 1 -- 172.21.15.86:0/3081326711 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55657b9a6600 con 0x55657ba4e780 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f8837fff640 10 client.15084 handle_command_reply: tid=0 2024-09-05T06:40:40.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 client.15084 shutdown 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 msgr2=0x55657ba3a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8844009900 tx=0x7f884402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 shutdown_connections 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657ba4e780 0x55657ba4fbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba3ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba332f0 0x55657ba3a7d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f8837fff640 0 client.15084 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 --2- 172.21.15.86:0/3081326711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba329d0 0x55657ba3d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 >> 172.21.15.86:0/3081326711 conn(0x55657ba2ef60 msgr2=0x55657ba34c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 shutdown_connections 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 1 -- 172.21.15.86:0/3081326711 wait complete. 2024-09-05T06:40:40.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.591+0000 7f88613b7740 20 client.15084 trim_cache size 0 max 16384 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 Processor -- start 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 -- start start 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba37a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55657ba4df30 con 0x55657ba329d0 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55657ba4e600 con 0x55657ba3dbb0 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f88613b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55657ba4ecd0 con 0x55657ba332f0 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52816/0 (socket says 172.21.15.86:52816) 2024-09-05T06:40:40.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 -- 172.21.15.86:0/447695493 learned_addr learned my addr 172.21.15.86:0/447695493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f8857fff640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885d57b640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba37a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 -- 172.21.15.86:0/447695493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 msgr2=0x55657ba390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 -- 172.21.15.86:0/447695493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 msgr2=0x55657ba37a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.609+0000 7f885cd7a640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba37a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f885cd7a640 1 -- 172.21.15.86:0/447695493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657ba39780 con 0x55657ba329d0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f885d57b640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba37a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f885cd7a640 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55657ba50700 tx=0x7f884c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bf9bddfe270ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f88427fc640 1 -- 172.21.15.86:0/447695493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c016b80 con 0x55657ba329d0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f88427fc640 1 -- 172.21.15.86:0/447695493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f884c020030 con 0x55657ba329d0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f88427fc640 1 -- 172.21.15.86:0/447695493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c024dd0 con 0x55657ba329d0 2024-09-05T06:40:40.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f88613b7740 1 -- 172.21.15.86:0/447695493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 msgr2=0x55657ba4fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.610+0000 7f88613b7740 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55657ba50700 tx=0x7f884c008040 comp rx=0 tx=0).stop 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- 172.21.15.86:0/447695493 shutdown_connections 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba3dbb0 0x55657ba37a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- 172.21.15.86:0/447695493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba329d0 0x55657ba4fbf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- 172.21.15.86:0/447695493 >> 172.21.15.86:0/447695493 conn(0x55657ba2ef60 msgr2=0x55657ba35de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- 172.21.15.86:0/447695493 shutdown_connections 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- 172.21.15.86:0/447695493 wait complete. 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- start start 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 Processor -- start 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55657ba4fa70 con 0x55657ba3dbb0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55657ba4f3a0 con 0x55657ba329d0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.611+0000 7f88613b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55657ba50140 con 0x55657ba332f0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f88613b7740 10 client.0 _ll_register_callbacks cb 0x7f8844031ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52828/0 (socket says 172.21.15.86:52828) 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 -- 172.21.15.86:0/3650501690 learned_addr learned my addr 172.21.15.86:0/3650501690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f8857fff640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 -- 172.21.15.86:0/3650501690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 msgr2=0x55657ba37ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba37ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 -- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 msgr2=0x55657ba379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 -- 172.21.15.86:0/3650501690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657ba33120 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f885d57b640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8850004970 tx=0x7f885000af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e6ade9b3b0d410 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f8837fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88500ba070 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f885009bf80 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c020030 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f885009bd20 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f88613b7740 10 client.15090 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.612+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55657bab9140 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f884c020030 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f885009b170 con 0x55657ba3dbb0 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:40:40.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 20 client.15090 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657babd7c0 0x55657babc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 4 client.15090 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "status"}) -- 0x55657b9a0b10 con 0x55657babd7c0 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657babd7c0 0x55657babc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f8837fff640 10 client.15090 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.613+0000 7f885cd7a640 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657babd7c0 0x55657babc4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=516aee9140fec4ac server_cookie=c0a13e323172e688 in_seq=0 out_seq=0 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f8837fff640 1 -- 172.21.15.86:0/3650501690 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55657b9a0b10 con 0x55657babd7c0 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f8837fff640 10 client.15090 handle_command_reply: tid=0 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 client.15090 shutdown 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 msgr2=0x55657ba38420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8850004970 tx=0x7f885000af30 comp rx=0 tx=0).stop 2024-09-05T06:40:40.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 shutdown_connections 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55657babd7c0 0x55657babc4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55657ba3dbb0 0x55657ba38420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55657ba332f0 0x55657ba37ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 --2- 172.21.15.86:0/3650501690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55657ba329d0 0x55657ba379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 >> 172.21.15.86:0/3650501690 conn(0x55657ba2ef60 msgr2=0x55657ba35de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f8837fff640 0 client.15090 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 shutdown_connections 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 1 -- 172.21.15.86:0/3650501690 wait complete. 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.614+0000 7f88613b7740 20 client.15090 trim_cache size 0 max 16384 2024-09-05T06:40:40.621 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24461, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 69.746913309999996, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 126.465333856 2024-09-05T06:40:40.622 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 msgr2=0x7f88480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 0x7f88480a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8844031440 tx=0x7f8844031c90 comp rx=0 tx=0).stop 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 msgr2=0x7f8858074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f885000cce0 tx=0x7f885000a620 comp rx=0 tx=0).stop 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 shutdown_connections 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8858106f10 0x7f8858074f70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f88480a2c10 0x7f88480a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858102790 0x7f8858074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 --2- 172.21.15.86:0/1859500863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8858101de0 0x7f88580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 >> 172.21.15.86:0/1859500863 conn(0x7f88580fd8c0 msgr2=0x7f88580fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 shutdown_connections 2024-09-05T06:40:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:40.615+0000 7f885efde640 1 -- 172.21.15.86:0/1859500863 wait complete. 2024-09-05T06:40:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:40 smithi115 ceph-mon[34791]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:41.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T06:40:41.711 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 -- 172.21.15.86:0/3473806411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458101de0 msgr2=0x7f54581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 --2- 172.21.15.86:0/3473806411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458101de0 0x7f54581021c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5454009900 tx=0x7f545402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 -- 172.21.15.86:0/3473806411 shutdown_connections 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 --2- 172.21.15.86:0/3473806411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458106f10 0x7f545810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 --2- 172.21.15.86:0/3473806411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458102790 0x7f54581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 --2- 172.21.15.86:0/3473806411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458101de0 0x7f54581021c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 -- 172.21.15.86:0/3473806411 >> 172.21.15.86:0/3473806411 conn(0x7f54580fd8c0 msgr2=0x7f54580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.189+0000 7f54603e2640 1 -- 172.21.15.86:0/3473806411 shutdown_connections 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 -- 172.21.15.86:0/3473806411 wait complete. 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 Processor -- start 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 -- start start 2024-09-05T06:40:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 0x7f54580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 0x7f5458074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5458111e40 con 0x7f5458106f10 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5458111cc0 con 0x7f5458102790 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f54603e2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5458111fc0 con 0x7f5458101de0 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f545e97f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f545e97f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52834/0 (socket says 172.21.15.86:52834) 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.190+0000 7f545e97f640 1 -- 172.21.15.86:0/2442955421 learned_addr learned my addr 172.21.15.86:0/2442955421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545d97d640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 0x7f5458074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e17e640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 0x7f54580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 -- 172.21.15.86:0/2442955421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 msgr2=0x7f54580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 0x7f54580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 -- 172.21.15.86:0/2442955421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 msgr2=0x7f5458074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 0x7f5458074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 -- 172.21.15.86:0/2442955421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581aad10 con 0x7f5458106f10 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545d97d640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 0x7f5458074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e17e640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 0x7f54580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f545e97f640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f545000b970 tx=0x7f545000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f544f7fe640 1 -- 172.21.15.86:0/2442955421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450019070 con 0x7f5458106f10 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581aaf40 con 0x7f5458106f10 2024-09-05T06:40:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f544f7fe640 1 -- 172.21.15.86:0/2442955421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5450014440 con 0x7f5458106f10 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f544f7fe640 1 -- 172.21.15.86:0/2442955421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500075d0 con 0x7f5458106f10 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.191+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581ab3a0 con 0x7f5458106f10 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.194+0000 7f544f7fe640 1 -- 172.21.15.86:0/2442955421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f545001c070 con 0x7f5458106f10 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.194+0000 7f54627bb740 1 Processor -- start 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.194+0000 7f54627bb740 1 -- start start 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.194+0000 7f54627bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.194+0000 7f54627bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285c2f0 0x55e352866670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f54627bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e352866bb0 0x55e352868fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f54627bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3527c2090 con 0x55e35285b9d0 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f54627bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e352769a70 con 0x55e352866bb0 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f54627bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e3527be050 con 0x55e35285c2f0 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52848/0 (socket says 172.21.15.86:52848) 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 -- 172.21.15.86:0/3573664156 learned_addr learned my addr 172.21.15.86:0/3573664156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f544f7fe640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 0x7f54440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545e17e640 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285c2f0 0x55e352866670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e352866bb0 0x55e352868fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545e97f640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 0x7f54440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 -- 172.21.15.86:0/3573664156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285c2f0 msgr2=0x55e352866670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285c2f0 0x55e352866670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 -- 172.21.15.86:0/3573664156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e352866bb0 msgr2=0x55e352868fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e352866bb0 0x55e352868fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f544f7fe640 1 -- 172.21.15.86:0/2442955421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f54500ceaf0 con 0x7f5458106f10 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 -- 172.21.15.86:0/3573664156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e352776130 con 0x55e35285b9d0 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545d97d640 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e10 tx=0x7f5448008600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daf6bbb78d7cf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.195+0000 7f545e97f640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 0x7f54440a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5450097390 tx=0x7f54500bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f544cdea640 1 -- 172.21.15.86:0/3573664156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448018030 con 0x55e35285b9d0 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f544cdea640 1 -- 172.21.15.86:0/3573664156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54480054d0 con 0x55e35285b9d0 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f544cdea640 1 -- 172.21.15.86:0/3573664156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800c750 con 0x55e35285b9d0 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 -- 172.21.15.86:0/3573664156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 msgr2=0x55e35285bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e10 tx=0x7f5448008600 comp rx=0 tx=0).stop 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 -- 172.21.15.86:0/3573664156 shutdown_connections 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e352866bb0 0x55e352868fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285c2f0 0x55e352866670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 --2- 172.21.15.86:0/3573664156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35285b9d0 0x55e35285bdb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 -- 172.21.15.86:0/3573664156 >> 172.21.15.86:0/3573664156 conn(0x55e352857f60 msgr2=0x55e352858340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 -- 172.21.15.86:0/3573664156 shutdown_connections 2024-09-05T06:40:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.196+0000 7f54627bb740 1 -- 172.21.15.86:0/3573664156 wait complete. 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 -- start start 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 Processor -- start 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e3528663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352863710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e352869780 con 0x55e352866bb0 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e35277cf20 con 0x55e35285c2f0 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e352869900 con 0x55e35285b9d0 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f54627bb740 10 client.0 _ll_register_callbacks cb 0x55e352746030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52852/0 (socket says 172.21.15.86:52852) 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 -- 172.21.15.86:0/218271708 learned_addr learned my addr 172.21.15.86:0/218271708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545e17e640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352863710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e3528663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 -- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 msgr2=0x55e3528663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.197+0000 7f545d97d640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e3528663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f545d97d640 1 -- 172.21.15.86:0/218271708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 msgr2=0x55e352863710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f545d97d640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352863710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f545d97d640 1 -- 172.21.15.86:0/218271708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3525332b0 con 0x55e352866bb0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f545d97d640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5448004940 tx=0x7f5448004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe42b547220fd41d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800c8d0 con 0x55e352866bb0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54480054d0 con 0x55e352866bb0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f544800ca70 con 0x55e352866bb0 2024-09-05T06:40:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f544800cc70 con 0x55e352866bb0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f54627bb740 10 client.15102 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e352875040 con 0x55e352866bb0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54480054d0 con 0x55e352866bb0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f544800fbe0 con 0x55e352866bb0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.198+0000 7f54627bb740 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f54627bb740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f54627bb740 20 client.15102 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528773c0 0x55e352878830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f54627bb740 4 client.15102 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3527cf600 con 0x55e3528773c0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f545e17e640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528773c0 0x55e352878830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f5441ffb640 10 client.15102 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.199+0000 7f545e17e640 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528773c0 0x55e352878830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac7b0f35545d8414 server_cookie=43b6dedba6e80a01 in_seq=0 out_seq=0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f5441ffb640 1 -- 172.21.15.86:0/218271708 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3527cf600 con 0x55e3528773c0 2024-09-05T06:40:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f5441ffb640 10 client.15102 handle_command_reply: tid=0 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 client.15102 shutdown 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 msgr2=0x55e352863c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5448004940 tx=0x7f5448004d60 comp rx=0 tx=0).stop 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 shutdown_connections 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528773c0 0x55e352878830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352863c50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352863710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f5441ffb640 0 client.15102 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 --2- 172.21.15.86:0/218271708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e3528663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 >> 172.21.15.86:0/218271708 conn(0x55e352857f60 msgr2=0x55e35285dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 shutdown_connections 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 1 -- 172.21.15.86:0/218271708 wait complete. 2024-09-05T06:40:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.200+0000 7f54627bb740 20 client.15102 trim_cache size 0 max 16384 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 Processor -- start 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 -- start start 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e352876cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352877a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e352876b70 con 0x55e352866bb0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e352877240 con 0x55e35285c2f0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.218+0000 7f54627bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e352877910 con 0x55e35285b9d0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52860/0 (socket says 172.21.15.86:52860) 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 -- 172.21.15.86:0/3636434984 learned_addr learned my addr 172.21.15.86:0/3636434984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545e17e640 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352877a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e352876cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 -- 172.21.15.86:0/3636434984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 msgr2=0x55e352876cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e352876cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 -- 172.21.15.86:0/3636434984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 msgr2=0x55e352877a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352877a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 -- 172.21.15.86:0/3636434984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e352865fe0 con 0x55e352866bb0 2024-09-05T06:40:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f545d97d640 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f544802b960 tx=0x7f544800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70c6061fee96f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f5443fff640 1 -- 172.21.15.86:0/3636434984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448018070 con 0x55e352866bb0 2024-09-05T06:40:42.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f5443fff640 1 -- 172.21.15.86:0/3636434984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5448004dd0 con 0x55e352866bb0 2024-09-05T06:40:42.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.219+0000 7f5443fff640 1 -- 172.21.15.86:0/3636434984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480120c0 con 0x55e352866bb0 2024-09-05T06:40:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- 172.21.15.86:0/3636434984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 msgr2=0x55e352862020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f544802b960 tx=0x7f544800cf30 comp rx=0 tx=0).stop 2024-09-05T06:40:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- 172.21.15.86:0/3636434984 shutdown_connections 2024-09-05T06:40:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e352862020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e352877a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 --2- 172.21.15.86:0/3636434984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e352876cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- 172.21.15.86:0/3636434984 >> 172.21.15.86:0/3636434984 conn(0x55e352857f60 msgr2=0x55e3528636d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- 172.21.15.86:0/3636434984 shutdown_connections 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- 172.21.15.86:0/3636434984 wait complete. 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 -- start start 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.220+0000 7f54627bb740 1 Processor -- start 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e35287b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e35287bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3528786b0 con 0x55e352866bb0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e352877fe0 con 0x55e35285c2f0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e352878d80 con 0x55e35285b9d0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 10 client.0 _ll_register_callbacks cb 0x7f5454009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52870/0 (socket says 172.21.15.86:52870) 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 -- 172.21.15.86:0/3142063261 learned_addr learned my addr 172.21.15.86:0/3142063261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e35287b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545e17e640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e35287bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 -- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 msgr2=0x55e35287b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e35287b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 -- 172.21.15.86:0/3142063261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 msgr2=0x55e35287bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e35287bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 -- 172.21.15.86:0/3142063261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3528dfbf0 con 0x55e352866bb0 2024-09-05T06:40:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f545d97d640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5448008650 tx=0x7f5448014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae9188e0ef2204d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f5441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448038020 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f5448019070 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448004dd0 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5448018030 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.221+0000 7f54627bb740 10 client.15108 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3528e0eb0 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5448004dd0 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f5448004090 con 0x55e352866bb0 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 20 client.15108 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528e5670 0x55e3528e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 4 client.15108 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "status"}) -- 0x55e3527c9b10 con 0x55e3528e5670 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.222+0000 7f545e17e640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528e5670 0x55e3528e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f545e17e640 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528e5670 0x55e3528e4380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5bf6a10fc9e2388 server_cookie=97408dc64137f592 in_seq=0 out_seq=0 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f5441ffb640 10 client.15108 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f5441ffb640 1 -- 172.21.15.86:0/3142063261 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e3527c9b10 con 0x55e3528e5670 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f5441ffb640 10 client.15108 handle_command_reply: tid=0 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 client.15108 shutdown 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 msgr2=0x55e35287c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5448008650 tx=0x7f5448014040 comp rx=0 tx=0).stop 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 shutdown_connections 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55e3528e5670 0x55e3528e4380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e352866bb0 0x55e35287c410 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e35285c2f0 0x55e35287bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 --2- 172.21.15.86:0/3142063261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e35285b9d0 0x55e35287b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 >> 172.21.15.86:0/3142063261 conn(0x55e352857f60 msgr2=0x55e352861bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f5441ffb640 0 client.15108 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 shutdown_connections 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 1 -- 172.21.15.86:0/3142063261 wait complete. 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.223+0000 7f54627bb740 20 client.15108 trim_cache size 0 max 16384 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24466, 2024-09-05T06:40:42.229 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 72.362744985000006, 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 125.511762547 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 msgr2=0x7f54440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 0x7f54440a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5450097390 tx=0x7f54500bb040 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 msgr2=0x7f5458074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f545000b970 tx=0x7f545000be30 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 shutdown_connections 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5458106f10 0x7f5458074f70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f54440a2c10 0x7f54440a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5458102790 0x7f5458074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 --2- 172.21.15.86:0/2442955421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5458101de0 0x7f54580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 >> 172.21.15.86:0/2442955421 conn(0x7f54580fd8c0 msgr2=0x7f54580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 shutdown_connections 2024-09-05T06:40:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:42.224+0000 7f54603e2640 1 -- 172.21.15.86:0/2442955421 wait complete. 2024-09-05T06:40:42.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:42 smithi086 ceph-mon[30943]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:42.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:42 smithi096 ceph-mon[34468]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:42 smithi115 ceph-mon[34791]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:42.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T06:40:43.121 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.574+0000 7fe8ec711640 1 -- 172.21.15.86:0/1874115208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e41036e0 msgr2=0x7fe8e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.574+0000 7fe8ec711640 1 --2- 172.21.15.86:0/1874115208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e41036e0 0x7fe8e4107b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009930 tx=0x7fe8d402a420 comp rx=0 tx=0).stop 2024-09-05T06:40:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- 172.21.15.86:0/1874115208 shutdown_connections 2024-09-05T06:40:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- 172.21.15.86:0/1874115208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41082a0 0x7fe8e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- 172.21.15.86:0/1874115208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e41036e0 0x7fe8e4107b90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- 172.21.15.86:0/1874115208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e4102d30 0x7fe8e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- 172.21.15.86:0/1874115208 >> 172.21.15.86:0/1874115208 conn(0x7fe8e40fc7d0 msgr2=0x7fe8e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- 172.21.15.86:0/1874115208 shutdown_connections 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- 172.21.15.86:0/1874115208 wait complete. 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 Processor -- start 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- start start 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41036e0 0x7fe8e419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e41082a0 0x7fe8e419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4111f70 con 0x7fe8e4102d30 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4111df0 con 0x7fe8e41082a0 2024-09-05T06:40:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.575+0000 7fe8ec711640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41120f0 con 0x7fe8e41036e0 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41580/0 (socket says 172.21.15.86:41580) 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/129366712 learned_addr learned my addr 172.21.15.86:0/129366712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41036e0 0x7fe8e419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8eacae640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e41082a0 0x7fe8e419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/129366712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41036e0 msgr2=0x7fe8e419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41036e0 0x7fe8e419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/129366712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e41082a0 msgr2=0x7fe8e419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e41082a0 0x7fe8e419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/129366712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e419f700 con 0x7fe8e4102d30 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ba50 tx=0x7fe8e000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8db7fe640 1 -- 172.21.15.86:0/129366712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0019070 con 0x7fe8e4102d30 2024-09-05T06:40:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8db7fe640 1 -- 172.21.15.86:0/129366712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e000cbe0 con 0x7fe8e4102d30 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e419f9f0 con 0x7fe8e4102d30 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8db7fe640 1 -- 172.21.15.86:0/129366712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00075d0 con 0x7fe8e4102d30 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.576+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41a72d0 con 0x7fe8e4102d30 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8db7fe640 1 -- 172.21.15.86:0/129366712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8e001c070 con 0x7fe8e4102d30 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 Processor -- start 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 -- start start 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241325db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413262f0 0x556241330670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55624128c090 con 0x556241330bb0 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556241233a70 con 0x5562413262f0 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.579+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556241288050 con 0x5562413259d0 2024-09-05T06:40:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8db7fe640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 0x7fe8d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41582/0 (socket says 172.21.15.86:41582) 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3695166571 learned_addr learned my addr 172.21.15.86:0/3695166571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413262f0 0x556241330670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8eacae640 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241325db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8eacae640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 0x7fe8d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8db7fe640 1 -- 172.21.15.86:0/129366712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fe8e00cea40 con 0x7fe8e4102d30 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3695166571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 msgr2=0x556241325db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241325db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3695166571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413262f0 msgr2=0x556241330670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413262f0 0x556241330670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3695166571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556241240130 con 0x556241330bb0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40697d0 tx=0x7fe8d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b49bfc5e0804d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8eacae640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 0x7fe8d00a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005fd0 tx=0x7fe8dc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8d8ff9640 1 -- 172.21.15.86:0/3695166571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4035040 con 0x556241330bb0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8d8ff9640 1 -- 172.21.15.86:0/3695166571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d40316a0 con 0x556241330bb0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.580+0000 7fe8d8ff9640 1 -- 172.21.15.86:0/3695166571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40047a0 con 0x556241330bb0 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3695166571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 msgr2=0x556241332fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40697d0 tx=0x7fe8d4005fd0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3695166571 shutdown_connections 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x556241332fa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413262f0 0x556241330670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3695166571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241325db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3695166571 >> 172.21.15.86:0/3695166571 conn(0x556241321f60 msgr2=0x556241322340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3695166571 shutdown_connections 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3695166571 wait complete. 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.581+0000 7fe8eeaea740 1 -- start start 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 Processor -- start 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413259d0 0x5562413303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x55624132d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556241333780 con 0x556241330bb0 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556241246f20 con 0x5562413259d0 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556241333900 con 0x5562413262f0 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eeaea740 10 client.0 _ll_register_callbacks cb 0x556241210030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41598/0 (socket says 172.21.15.86:41598) 2024-09-05T06:40:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 -- 172.21.15.86:0/1860357820 learned_addr learned my addr 172.21.15.86:0/1860357820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413259d0 0x5562413303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x55624132d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 -- 172.21.15.86:0/1860357820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 msgr2=0x55624132d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x55624132d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 -- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413259d0 msgr2=0x5562413303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413259d0 0x5562413303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 -- 172.21.15.86:0/1860357820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556240ffd2b0 con 0x556241330bb0 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x55624132d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8eacae640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc010fc0 tx=0x7fe8dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60ba75dd60977c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.582+0000 7fe8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01e640 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d40316a0 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fe8dc01e7e0 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc00e960 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 10 client.15120 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55624133f0a0 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d40316a0 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fe8dc01c040 con 0x556241330bb0 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 20 client.15120 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413417a0 0x556241342c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 4 client.15120 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:40:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556241299600 con 0x5562413417a0 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.583+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413417a0 0x556241342c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.584+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413417a0 0x556241342c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a957a5e24d3b5438 server_cookie=c27cf570a9badc01 in_seq=0 out_seq=0 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.584+0000 7fe8cdffb640 10 client.15120 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8cdffb640 1 -- 172.21.15.86:0/1860357820 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556241299600 con 0x5562413417a0 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8cdffb640 10 client.15120 handle_command_reply: tid=0 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 client.15120 shutdown 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 msgr2=0x55624132dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc010fc0 tx=0x7fe8dc004730 comp rx=0 tx=0).stop 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 shutdown_connections 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413417a0 0x556241342c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556241330bb0 0x55624132dc80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x55624132d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/1860357820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562413259d0 0x5562413303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8cdffb640 0 client.15120 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:40:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 >> 172.21.15.86:0/1860357820 conn(0x556241321f60 msgr2=0x556241327c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 shutdown_connections 2024-09-05T06:40:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 1 -- 172.21.15.86:0/1860357820 wait complete. 2024-09-05T06:40:43.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.585+0000 7fe8eeaea740 20 client.15120 trim_cache size 0 max 16384 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 Processor -- start 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 -- start start 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241342c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x55624132c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556241340f50 con 0x5562413262f0 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556241341620 con 0x556241330bb0 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556241341cf0 con 0x5562413259d0 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41612/0 (socket says 172.21.15.86:41612) 2024-09-05T06:40:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3580909848 learned_addr learned my addr 172.21.15.86:0/3580909848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8eacae640 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x55624132c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241342c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3580909848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 msgr2=0x556241342c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241342c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3580909848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 msgr2=0x55624132c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x55624132c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.603+0000 7fe8e9cac640 1 -- 172.21.15.86:0/3580909848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562413219e0 con 0x5562413262f0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5562413232e0 tx=0x7fe8d402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3303fc1fccd1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8cffff640 1 -- 172.21.15.86:0/3580909848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4035040 con 0x5562413262f0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8cffff640 1 -- 172.21.15.86:0/3580909848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d40054d0 con 0x5562413262f0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8cffff640 1 -- 172.21.15.86:0/3580909848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4004cf0 con 0x5562413262f0 2024-09-05T06:40:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3580909848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 msgr2=0x55624131cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5562413232e0 tx=0x7fe8d402a420 comp rx=0 tx=0).stop 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3580909848 shutdown_connections 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x55624132c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413262f0 0x55624131cdf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/3580909848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413259d0 0x556241342c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.604+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3580909848 >> 172.21.15.86:0/3580909848 conn(0x556241321f60 msgr2=0x556241322340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3580909848 shutdown_connections 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- 172.21.15.86:0/3580909848 wait complete. 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- start start 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 Processor -- start 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x556241344260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x5562413447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556241342a90 con 0x5562413259d0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562413423c0 con 0x556241330bb0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556241343160 con 0x5562413262f0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eeaea740 10 client.0 _ll_register_callbacks cb 0x7fe8dc010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8ea4ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8ea4ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41614/0 (socket says 172.21.15.86:41614) 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/295813546 learned_addr learned my addr 172.21.15.86:0/295813546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x556241344260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.605+0000 7fe8eacae640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x5562413447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/295813546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 msgr2=0x556241344260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x556241344260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 msgr2=0x5562413447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x5562413447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 -- 172.21.15.86:0/295813546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562413a9ea0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8eacae640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x5562413447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8ea4ad640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0097900 tx=0x7fe8e0065950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6037b58551ff3262 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00a3af0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fe8e00cc270 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d40054d0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8e00975a0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8eeaea740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562413ab1d0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d40054d0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e006d4e0 con 0x5562413259d0 2024-09-05T06:40:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.606+0000 7fe8eeaea740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 20 client.15126 populate_metadata read hostname 'smithi086' 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413afdc0 0x5562413aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 4 client.15126 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "status"}) -- 0x556241293b10 con 0x5562413afdc0 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413afdc0 0x5562413aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8cdffb640 10 client.15126 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8e9cac640 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413afdc0 0x5562413aead0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9f305fa5c58c33 server_cookie=2aca4ff9e568bb75 in_seq=0 out_seq=0 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8cdffb640 1 -- 172.21.15.86:0/295813546 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556241293b10 con 0x5562413afdc0 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8cdffb640 10 client.15126 handle_command_reply: tid=0 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 1 client.15126 shutdown 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 msgr2=0x556241343d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.607+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0097900 tx=0x7fe8e0065950 comp rx=0 tx=0).stop 2024-09-05T06:40:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 shutdown_connections 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5562413afdc0 0x5562413aead0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556241330bb0 0x5562413447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5562413262f0 0x556241344260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 --2- 172.21.15.86:0/295813546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562413259d0 0x556241343d20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 >> 172.21.15.86:0/295813546 conn(0x556241321f60 msgr2=0x556241328580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8cdffb640 0 client.15126 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 shutdown_connections 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 1 -- 172.21.15.86:0/295813546 wait complete. 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8eeaea740 20 client.15126 trim_cache size 0 max 16384 2024-09-05T06:40:43.616 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24479, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 107.24150553600001, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 118.231407891 2024-09-05T06:40:43.617 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:40:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 msgr2=0x7fe8d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 0x7fe8d00a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005fd0 tx=0x7fe8dc0046c0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 msgr2=0x7fe8e419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ba50 tx=0x7fe8e000bf10 comp rx=0 tx=0).stop 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 shutdown_connections 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fe8d00a2c90 0x7fe8d00a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e41082a0 0x7fe8e419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8e41036e0 0x7fe8e419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 --2- 172.21.15.86:0/129366712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4102d30 0x7fe8e419a750 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 >> 172.21.15.86:0/129366712 conn(0x7fe8e40fc7d0 msgr2=0x7fe8e4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 shutdown_connections 2024-09-05T06:40:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:43.608+0000 7fe8ec711640 1 -- 172.21.15.86:0/129366712 wait complete. 2024-09-05T06:40:44.411 INFO:tasks.cephadm:Setup complete, yielding 2024-09-05T06:40:44.411 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T06:40:44.421 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:40:44.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch status' 2024-09-05T06:40:44.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:44 smithi096 ceph-mon[34468]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:44.664 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:44 smithi115 ceph-mon[34791]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:44.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:44 smithi086 ceph-mon[30943]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1925697892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac074c50 msgr2=0x7f58ac0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1925697892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac074c50 0x7f58ac0750b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009900 tx=0x7f58a002a410 comp rx=0 tx=0).stop 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1925697892 shutdown_connections 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1925697892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1925697892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac074c50 0x7f58ac0750b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1925697892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac076220 0x7f58ac074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1925697892 >> 172.21.15.86:0/1925697892 conn(0x7f58ac0fd390 msgr2=0x7f58ac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1925697892 shutdown_connections 2024-09-05T06:40:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.235+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1925697892 wait complete. 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 Processor -- start 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 -- start start 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 0x7f58ac19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac113d50 con 0x7f58ac076220 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac113bd0 con 0x7f58ac074c50 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58b1ddc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac113ed0 con 0x7f58ac10e1e0 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58aaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58aaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41636/0 (socket says 172.21.15.86:41636) 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.236+0000 7f58aaffd640 1 -- 172.21.15.86:0/1195035549 learned_addr learned my addr 172.21.15.86:0/1195035549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58abfff640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:45.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58ab7fe640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 0x7f58ac19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 -- 172.21.15.86:0/1195035549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 msgr2=0x7f58ac1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 -- 172.21.15.86:0/1195035549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 msgr2=0x7f58ac19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 0x7f58ac19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 -- 172.21.15.86:0/1195035549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac1a6070 con 0x7f58ac076220 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58abfff640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58ab7fe640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 0x7f58ac19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58aaffd640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f58a0004770 tx=0x7f58a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a003f030 con 0x7f58ac076220 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58a0003cb0 con 0x7f58ac076220 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac1a6300 con 0x7f58ac076220 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0035070 con 0x7f58ac076220 2024-09-05T06:40:45.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.237+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac1a67e0 con 0x7f58ac076220 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.238+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac1043b0 con 0x7f58ac076220 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.239+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f58a0042030 con 0x7f58ac076220 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.240+0000 7f58a8ff9640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 0x7f58940a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.240+0000 7f58ab7fe640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 0x7f58940a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.240+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f58a00efc90 con 0x7f58ac076220 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.240+0000 7f58ab7fe640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 0x7f58940a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f589c0058c0 tx=0x7f589c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:45.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.244+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58a00f5020 con 0x7f58ac076220 2024-09-05T06:40:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.427+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f58ac062630 con 0x7f58940a2c40 2024-09-05T06:40:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.429+0000 7f58a8ff9640 1 -- 172.21.15.86:0/1195035549 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f58ac062630 con 0x7f58940a2c40 2024-09-05T06:40:45.429 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-09-05T06:40:45.429 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-09-05T06:40:45.429 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 msgr2=0x7f58940a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 0x7f58940a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f589c0058c0 tx=0x7f589c005d30 comp rx=0 tx=0).stop 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 msgr2=0x7f58ac19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f58a0004770 tx=0x7f58a00042f0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 shutdown_connections 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58ac10e1e0 0x7f58ac1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58ac076220 0x7f58ac19e980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f58940a2c40 0x7f58940a5100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 --2- 172.21.15.86:0/1195035549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ac074c50 0x7f58ac19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:45.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 >> 172.21.15.86:0/1195035549 conn(0x7f58ac0fd390 msgr2=0x7f58ac101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:45.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 shutdown_connections 2024-09-05T06:40:45.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:45.431+0000 7f58b1ddc640 1 -- 172.21.15.86:0/1195035549 wait complete. 2024-09-05T06:40:46.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-05T06:40:46.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:46 smithi086 ceph-mon[30943]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:46 smithi096 ceph-mon[34468]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:46 smithi115 ceph-mon[34791]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 -- 172.21.15.86:0/3261653705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 msgr2=0x7f0394108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 --2- 172.21.15.86:0/3261653705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f0394108f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f038c009ca0 tx=0x7f038c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 -- 172.21.15.86:0/3261653705 shutdown_connections 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 --2- 172.21.15.86:0/3261653705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f0394108f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 --2- 172.21.15.86:0/3261653705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394101cd0 0x7f0394102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 --2- 172.21.15.86:0/3261653705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394100ad0 0x7f0394100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 -- 172.21.15.86:0/3261653705 >> 172.21.15.86:0/3261653705 conn(0x7f03940fc280 msgr2=0x7f03940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 -- 172.21.15.86:0/3261653705 shutdown_connections 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 -- 172.21.15.86:0/3261653705 wait complete. 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.938+0000 7f039958b640 1 Processor -- start 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 -- start start 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394100ad0 0x7f03941049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394101cd0 0x7f03941030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f039410c650 con 0x7f0394102690 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f039410c4d0 con 0x7f0394100ad0 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f039958b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f039410c7d0 con 0x7f0394101cd0 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41654/0 (socket says 172.21.15.86:41654) 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 -- 172.21.15.86:0/1358772113 learned_addr learned my addr 172.21.15.86:0/1358772113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03927fc640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394101cd0 0x7f03941030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f0392ffd640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394100ad0 0x7f03941049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 -- 172.21.15.86:0/1358772113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394101cd0 msgr2=0x7f03941030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394101cd0 0x7f03941030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 -- 172.21.15.86:0/1358772113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394100ad0 msgr2=0x7f03941049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394100ad0 0x7f03941049f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.939+0000 7f03937fe640 1 -- 172.21.15.86:0/1358772113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0394103e10 con 0x7f0394102690 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f03937fe640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f038c031040 tx=0x7f038c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c004040 con 0x7f0394102690 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f038c004690 con 0x7f0394102690 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03941a1e30 con 0x7f0394102690 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c03b410 con 0x7f0394102690 2024-09-05T06:40:46.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.940+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03941a22e0 con 0x7f0394102690 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.941+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03941a2540 con 0x7f0394102690 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.942+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f038c0305c0 con 0x7f0394102690 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.943+0000 7f037bfff640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 0x7f037c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.943+0000 7f0392ffd640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 0x7f037c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.943+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f038c0f0cb0 con 0x7f0394102690 2024-09-05T06:40:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.943+0000 7f0392ffd640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 0x7f037c0a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03840058c0 tx=0x7f038400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:46.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:46.947+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f038c0bd910 con 0x7f0394102690 2024-09-05T06:40:47.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.122+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f03941a2710 con 0x7f037c0a2c10 2024-09-05T06:40:47.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.133+0000 7f037bfff640 1 -- 172.21.15.86:0/1358772113 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f03941a2710 con 0x7f037c0a2c10 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (2m) 52s ago 2m 17.2M - 19.1.1-259-g91c8a7e5 77770cf7e895 113a8c7c1239 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi096 running (2m) 54s ago 2m 25.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 22f4f023cf60 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi115 running (2m) 54s ago 2m 17.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 534a97c1e314 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (2m) 52s ago 2m 14.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 f72d40b29bcc 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi096 running (2m) 54s ago 2m 17.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 8d73361f83ac 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi115 running (2m) 54s ago 2m 15.6M - 19.1.1-259-g91c8a7e5 77770cf7e895 ff33ea77cdd1 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (2m) 52s ago 2m 17.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 032f3ad3e747 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi096 running (2m) 54s ago 2m 17.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 136a750f0099 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi115 running (2m) 54s ago 2m 17.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 2bce40969aea 2024-09-05T06:40:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (2m) 52s ago 2m 14.2M - 19.1.1-259-g91c8a7e5 77770cf7e895 4ddc3b180a38 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi096 running (2m) 54s ago 2m 19.2M - 19.1.1-259-g91c8a7e5 77770cf7e895 440a1bb0b310 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi115 running (2m) 54s ago 2m 18.6M - 19.1.1-259-g91c8a7e5 77770cf7e895 05e4d7cfd4c7 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (9m) 52s ago 9m 509M - 19.1.1-259-g91c8a7e5 77770cf7e895 058d60dfd4a8 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi096 *:8443,8765 running (7m) 54s ago 7m 442M - 19.1.1-259-g91c8a7e5 77770cf7e895 fb5b4d804b90 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi115 *:8443,8765 running (7m) 54s ago 7m 441M - 19.1.1-259-g91c8a7e5 77770cf7e895 dbbbd764a2c6 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (9m) 52s ago 9m 50.0M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 07527dcb92f7 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi096 running (7m) 54s ago 7m 39.6M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 e06d482d9f34 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi115 running (8m) 54s ago 8m 40.8M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 10e8b9151908 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (7m) 52s ago 7m 45.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 0fc10a7af3ca 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi096 running (6m) 54s ago 6m 46.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 a155f923dcdf 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi115 running (6m) 54s ago 6m 46.1M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 1038e3565477 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (6m) 52s ago 6m 52.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 0071b51cd474 2024-09-05T06:40:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi096 running (5m) 54s ago 5m 53.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 7d258aeaeda7 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi115 running (5m) 54s ago 5m 47.2M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 b90fb8878d04 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (4m) 52s ago 4m 50.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 ab9a4b41cfdb 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi096 running (4m) 54s ago 4m 55.8M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 9ee75ac65ee9 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi115 running (4m) 54s ago 4m 52.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 01724e5f53b3 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (3m) 52s ago 3m 44.7M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 bf8e969ffc53 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi096 running (3m) 54s ago 3m 44.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 deb301d36b67 2024-09-05T06:40:47.136 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi115 running (2m) 54s ago 2m 50.6M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 e8670243ec65 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 msgr2=0x7f037c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 0x7f037c0a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03840058c0 tx=0x7f038400a2d0 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 msgr2=0x7f03941035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f038c031040 tx=0x7f038c003c10 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 shutdown_connections 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0394102690 0x7f03941035f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0394101cd0 0x7f03941030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f037c0a2c10 0x7f037c0a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 --2- 172.21.15.86:0/1358772113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0394100ad0 0x7f03941049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.135+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 >> 172.21.15.86:0/1358772113 conn(0x7f03940fc280 msgr2=0x7f03940fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.136+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 shutdown_connections 2024-09-05T06:40:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:47.136+0000 7f039958b640 1 -- 172.21.15.86:0/1358772113 wait complete. 2024-09-05T06:40:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:47 smithi096 ceph-mon[34468]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:47 smithi115 ceph-mon[34791]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:47.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:47 smithi086 ceph-mon[30943]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:47.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-05T06:40:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:48 smithi086 ceph-mon[30943]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:48.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:48 smithi086 ceph-mon[30943]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:48.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:48 smithi096 ceph-mon[34468]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:48 smithi096 ceph-mon[34468]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- 172.21.15.86:0/4067364922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 msgr2=0x7fac28109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 --2- 172.21.15.86:0/4067364922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac28109d10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac18009900 tx=0x7fac1802a410 comp rx=0 tx=0).stop 2024-09-05T06:40:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- 172.21.15.86:0/4067364922 shutdown_connections 2024-09-05T06:40:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 --2- 172.21.15.86:0/4067364922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac28109d10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 --2- 172.21.15.86:0/4067364922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28104f90 0x7fac28107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 --2- 172.21.15.86:0/4067364922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28102660 0x7fac28104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- 172.21.15.86:0/4067364922 >> 172.21.15.86:0/4067364922 conn(0x7fac280fc280 msgr2=0x7fac280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- 172.21.15.86:0/4067364922 shutdown_connections 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- 172.21.15.86:0/4067364922 wait complete. 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 Processor -- start 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.621+0000 7fac2dd30640 1 -- start start 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28102660 0x7fac28197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28104f90 0x7fac281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac2810d790 con 0x7fac281078c0 2024-09-05T06:40:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac2810d610 con 0x7fac28102660 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac2dd30640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac2810d910 con 0x7fac28104f90 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41684/0 (socket says 172.21.15.86:41684) 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 -- 172.21.15.86:0/2816924065 learned_addr learned my addr 172.21.15.86:0/2816924065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac277fe640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28102660 0x7fac28197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac26ffd640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28104f90 0x7fac281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 -- 172.21.15.86:0/2816924065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28104f90 msgr2=0x7fac281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28104f90 0x7fac281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 -- 172.21.15.86:0/2816924065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28102660 msgr2=0x7fac28197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28102660 0x7fac28197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 -- 172.21.15.86:0/2816924065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2819faa0 con 0x7fac281078c0 2024-09-05T06:40:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.622+0000 7fac27fff640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac18005940 tx=0x7fac18031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.623+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1803f030 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.623+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac18003aa0 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.623+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2819fd30 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.623+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18035070 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.623+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac281a0240 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.624+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac280fdd70 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.625+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac18042030 con 0x7fac281078c0 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.626+0000 7fac24ff9640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 0x7fac140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.626+0000 7fac277fe640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 0x7fac140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.626+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fac180eff30 con 0x7fac281078c0 2024-09-05T06:40:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.626+0000 7fac277fe640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 0x7fac140a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac2000a400 tx=0x7fac2000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.629+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac180bc310 con 0x7fac281078c0 2024-09-05T06:40:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:48 smithi115 ceph-mon[34791]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:48 smithi115 ceph-mon[34791]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:48.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.802+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fac28062630 con 0x7fac140a2c40 2024-09-05T06:40:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.807+0000 7fac24ff9640 1 -- 172.21.15.86:0/2816924065 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fac28062630 con 0x7fac140a2c40 2024-09-05T06:40:48.807 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-05T06:40:48.807 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 55s ago 2m smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2024-09-05T06:40:48.807 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 55s ago 7m smithi086=x;smithi096=y;smithi115=z;count:3 2024-09-05T06:40:48.808 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 55s ago 8m smithi086:172.21.15.86=a;smithi096:172.21.15.96=b;smithi115:172.21.15.115=c;count:3 2024-09-05T06:40:48.808 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 55s ago - 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.809+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 msgr2=0x7fac140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.809+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 0x7fac140a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fac2000a400 tx=0x7fac2000d040 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.809+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 msgr2=0x7fac2819f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.809+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac18005940 tx=0x7fac18031a30 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 shutdown_connections 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac281078c0 0x7fac2819f430 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac28104f90 0x7fac281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fac140a2c40 0x7fac140a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 --2- 172.21.15.86:0/2816924065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac28102660 0x7fac28197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 >> 172.21.15.86:0/2816924065 conn(0x7fac280fc280 msgr2=0x7fac28102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 shutdown_connections 2024-09-05T06:40:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:48.810+0000 7fac2dd30640 1 -- 172.21.15.86:0/2816924065 wait complete. 2024-09-05T06:40:49.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-05T06:40:49.788 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 -- 172.21.15.86:0/758903803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 msgr2=0x7f44a0109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 --2- 172.21.15.86:0/758903803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0109d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4498009ca0 tx=0x7f449802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 -- 172.21.15.86:0/758903803 shutdown_connections 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 --2- 172.21.15.86:0/758903803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0109d70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 --2- 172.21.15.86:0/758903803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0105050 0x7f44a0107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 --2- 172.21.15.86:0/758903803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0102720 0x7f44a0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 -- 172.21.15.86:0/758903803 >> 172.21.15.86:0/758903803 conn(0x7f44a00fc280 msgr2=0x7f44a00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 -- 172.21.15.86:0/758903803 shutdown_connections 2024-09-05T06:40:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.283+0000 7f44a7252640 1 -- 172.21.15.86:0/758903803 wait complete. 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 Processor -- start 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 -- start start 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0102720 0x7f44a0102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0105050 0x7f44a01006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d720 con 0x7f44a0107980 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d5a0 con 0x7f44a0102720 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a7252640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d8a0 con 0x7f44a0105050 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41696/0 (socket says 172.21.15.86:41696) 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a4fee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0102720 0x7f44a0102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 -- 172.21.15.86:0/836006102 learned_addr learned my addr 172.21.15.86:0/836006102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f449ffff640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0105050 0x7f44a01006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 -- 172.21.15.86:0/836006102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0105050 msgr2=0x7f44a01006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0105050 0x7f44a01006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 -- 172.21.15.86:0/836006102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0102720 msgr2=0x7f44a0102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0102720 0x7f44a0102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.284+0000 7f44a57ef640 1 -- 172.21.15.86:0/836006102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a01014d0 con 0x7f44a0107980 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f44a57ef640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4498009dc0 tx=0x7f4498003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498032ec0 con 0x7f44a0107980 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44980044f0 con 0x7f44a0107980 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a01a1ee0 con 0x7f44a0107980 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498030d40 con 0x7f44a0107980 2024-09-05T06:40:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.285+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a2390 con 0x7f44a0107980 2024-09-05T06:40:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.286+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a00fdd70 con 0x7f44a0107980 2024-09-05T06:40:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.287+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4498004040 con 0x7f44a0107980 2024-09-05T06:40:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.288+0000 7f449dffb640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 0x7f44900a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.288+0000 7f44a4fee640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 0x7f44900a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.288+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f44980f0b20 con 0x7f44a0107980 2024-09-05T06:40:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.288+0000 7f44a4fee640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 0x7f44900a5270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f449400bbe0 tx=0x7f449400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.291+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44980bd660 con 0x7f44a0107980 2024-09-05T06:40:50.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.462+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f44a0062630 con 0x7f44900a2db0 2024-09-05T06:40:50.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:50 smithi086 ceph-mon[30943]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:50.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:50 smithi086 ceph-mon[30943]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.463+0000 7f449dffb640 1 -- 172.21.15.86:0/836006102 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f44a0062630 con 0x7f44900a2db0 2024-09-05T06:40:50.463 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-09-05T06:40:50.463 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-09-05T06:40:50.464 INFO:teuthology.orchestra.run.smithi086.stdout:smithi096 172.21.15.96 2024-09-05T06:40:50.464 INFO:teuthology.orchestra.run.smithi086.stdout:smithi115 172.21.15.115 2024-09-05T06:40:50.464 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 msgr2=0x7f44900a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 0x7f44900a5270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f449400bbe0 tx=0x7f449400b390 comp rx=0 tx=0).stop 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 msgr2=0x7f44a0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4498009dc0 tx=0x7f4498003ba0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 shutdown_connections 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a0107980 0x7f44a0100c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44a0105050 0x7f44a01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f44900a2db0 0x7f44900a5270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 --2- 172.21.15.86:0/836006102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a0102720 0x7f44a0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 >> 172.21.15.86:0/836006102 conn(0x7f44a00fc280 msgr2=0x7f44a01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 shutdown_connections 2024-09-05T06:40:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:50.466+0000 7f44a7252640 1 -- 172.21.15.86:0/836006102 wait complete. 2024-09-05T06:40:50.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:50 smithi096 ceph-mon[34468]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:50.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:50 smithi096 ceph-mon[34468]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:50 smithi115 ceph-mon[34791]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:50 smithi115 ceph-mon[34791]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:51.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-05T06:40:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:51.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.939+0000 7f49e0e56640 1 -- 172.21.15.86:0/1083873923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc100ad0 msgr2=0x7f49dc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.939+0000 7f49e0e56640 1 --2- 172.21.15.86:0/1083873923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc100ad0 0x7f49dc100ed0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009900 tx=0x7f49d002a410 comp rx=0 tx=0).stop 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 -- 172.21.15.86:0/1083873923 shutdown_connections 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 --2- 172.21.15.86:0/1083873923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc102690 0x7f49dc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 --2- 172.21.15.86:0/1083873923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 0x7f49dc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 --2- 172.21.15.86:0/1083873923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc100ad0 0x7f49dc100ed0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 -- 172.21.15.86:0/1083873923 >> 172.21.15.86:0/1083873923 conn(0x7f49dc0fc280 msgr2=0x7f49dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 -- 172.21.15.86:0/1083873923 shutdown_connections 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 -- 172.21.15.86:0/1083873923 wait complete. 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.940+0000 7f49e0e56640 1 Processor -- start 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 -- start start 2024-09-05T06:40:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc100ad0 0x7f49dc197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 0x7f49dc198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d730 con 0x7f49dc102690 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d5b0 con 0x7f49dc100ad0 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49e0e56640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d8b0 con 0x7f49dc101cd0 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41710/0 (socket says 172.21.15.86:41710) 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 -- 172.21.15.86:0/539354299 learned_addr learned my addr 172.21.15.86:0/539354299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49da59c640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc100ad0 0x7f49dc197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49d9d9b640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 0x7f49dc198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 -- 172.21.15.86:0/539354299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 msgr2=0x7f49dc198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 0x7f49dc198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:51.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 -- 172.21.15.86:0/539354299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc100ad0 msgr2=0x7f49dc197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc100ad0 0x7f49dc197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 -- 172.21.15.86:0/539354299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc19fa30 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.941+0000 7f49dad9d640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f49d400cce0 tx=0x7f49d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.942+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d400f450 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.942+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49d400faa0 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.942+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4018b70 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.942+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc19fd20 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.942+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a0200 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.943+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc107f20 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.944+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49d4018d10 con 0x7f49dc102690 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.944+0000 7f49cb7fe640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 0x7f49c40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.944+0000 7f49da59c640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 0x7f49c40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:51.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.944+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f49d40cf9d0 con 0x7f49dc102690 2024-09-05T06:40:51.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.945+0000 7f49da59c640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 0x7f49c40a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:51.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:51.948+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49d4014030 con 0x7f49dc102690 2024-09-05T06:40:52.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.122+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 --> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f49dc062630 con 0x7f49c40a2cc0 2024-09-05T06:40:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.124+0000 7f49cb7fe640 1 -- 172.21.15.86:0/539354299 <== mgr.14150 v2:172.21.15.86:6800/3708924769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f49dc062630 con 0x7f49c40a2cc0 2024-09-05T06:40:52.124 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-05T06:40:52.124 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T06:40:52.124 INFO:teuthology.orchestra.run.smithi086.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T06:40:52.124 INFO:teuthology.orchestra.run.smithi086.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-05T06:40:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 msgr2=0x7f49c40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 0x7f49c40a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d00314a0 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 msgr2=0x7f49dc19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f49d400cce0 tx=0x7f49d400a620 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 shutdown_connections 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49dc102690 0x7f49dc19f3c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49dc101cd0 0x7f49dc198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f49c40a2cc0 0x7f49c40a5180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 --2- 172.21.15.86:0/539354299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc100ad0 0x7f49dc197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 >> 172.21.15.86:0/539354299 conn(0x7f49dc0fc280 msgr2=0x7f49dc1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.126+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 shutdown_connections 2024-09-05T06:40:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:52.127+0000 7f49e0e56640 1 -- 172.21.15.86:0/539354299 wait complete. 2024-09-05T06:40:52.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:52 smithi086 ceph-mon[30943]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:52.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:52 smithi086 ceph-mon[30943]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:52.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:52 smithi096 ceph-mon[34468]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:52 smithi096 ceph-mon[34468]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:52 smithi115 ceph-mon[34791]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:52 smithi115 ceph-mon[34791]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:40:52.887 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T06:40:52.900 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:40:52.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-05T06:40:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:53.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:53 smithi086 ceph-mon[30943]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:53.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:53 smithi096 ceph-mon[34468]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:53.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- 172.21.15.86:0/3944641684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10700ffef0 msgr2=0x7f10701002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:53.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 --2- 172.21.15.86:0/3944641684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10700ffef0 0x7f10701002f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1060009930 tx=0x7f106002a420 comp rx=0 tx=0).stop 2024-09-05T06:40:53.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- 172.21.15.86:0/3944641684 shutdown_connections 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 --2- 172.21.15.86:0/3944641684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1070103930 0x7f107010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 --2- 172.21.15.86:0/3944641684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070102f70 0x7f10701033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 --2- 172.21.15.86:0/3944641684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10700ffef0 0x7f10701002f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- 172.21.15.86:0/3944641684 >> 172.21.15.86:0/3944641684 conn(0x7f10700fba40 msgr2=0x7f10700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- 172.21.15.86:0/3944641684 shutdown_connections 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- 172.21.15.86:0/3944641684 wait complete. 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 Processor -- start 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.616+0000 7f1075078640 1 -- start start 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10700ffef0 0x7f1070197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 0x7f107019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f107010d760 con 0x7f1070102f70 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f107010d5e0 con 0x7f10700ffef0 2024-09-05T06:40:53.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f107010d8e0 con 0x7f1070103930 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33786/0 (socket says 172.21.15.86:33786) 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 -- 172.21.15.86:0/3350841494 learned_addr learned my addr 172.21.15.86:0/3350841494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106f59e640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 0x7f107019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106ed9d640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10700ffef0 0x7f1070197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 -- 172.21.15.86:0/3350841494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 msgr2=0x7f107019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 0x7f107019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 -- 172.21.15.86:0/3350841494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10700ffef0 msgr2=0x7f1070197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10700ffef0 0x7f1070197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 -- 172.21.15.86:0/3350841494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107019fb70 con 0x7f1070102f70 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106f59e640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 0x7f107019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1067fff640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f105800c970 tx=0x7f105800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:53.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058007bb0 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1058007d50 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107019fe60 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105800f450 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.617+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701a02c0 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.618+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10701092b0 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.620+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1058004ee0 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.620+0000 7f106cd99640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 0x7f105c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.621+0000 7f106ed9d640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 0x7f105c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.621+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f10580cf6d0 con 0x7f1070102f70 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.621+0000 7f106ed9d640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 0x7f105c0a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1060006010 tx=0x7f1060000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.624+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1058014030 con 0x7f1070102f70 2024-09-05T06:40:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:53 smithi115 ceph-mon[34791]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:40:53.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.825+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f107002d7c0 con 0x7f1070102f70 2024-09-05T06:40:53.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.825+0000 7f106cd99640 1 -- 172.21.15.86:0/3350841494 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3130 (secure 0 0 0) 0x7f105809c100 con 0x7f1070102f70 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:e27 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:btime 2024-09-05T06:39:33:863395+0000 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 27 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-09-05T06:38:55.927385+0000 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-09-05T06:39:33.863378+0000 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-09-05T06:40:53.826 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24479,1=24466,2=24461,3=24422,4=24403} 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-09-05T06:40:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3,4] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:qdb_cluster leader: 24479 members: 24479,24466,24461,24422,24403 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24479} state up:active seq 4 addr [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.115:6836/2087354694,v1:172.21.15.115:6837/2087354694] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{1:24466} state up:active seq 15 addr [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{1:24446} state up:standby-replay seq 1 addr [v2:172.21.15.96:6834/3823127248,v1:172.21.15.96:6835/3823127248] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{2:24461} state up:active seq 16 addr [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{2:24442} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/1272008759,v1:172.21.15.115:6835/1272008759] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:24422} state up:active seq 20 addr [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{3:14598} state up:standby-replay seq 1 addr [v2:172.21.15.86:6840/1507644375,v1:172.21.15.86:6841/1507644375] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.828 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{4:24403} state up:active seq 21 addr [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{4:14586} state up:standby-replay seq 1 addr [v2:172.21.15.86:6838/3075373024,v1:172.21.15.86:6839/3075373024] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.86:6834/3431663557,v1:172.21.15.86:6835/3431663557] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.829 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.86:6836/78876235,v1:172.21.15.86:6837/78876235] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 msgr2=0x7f105c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 0x7f105c0a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1060006010 tx=0x7f1060000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 msgr2=0x7f1070198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f105800c970 tx=0x7f105800ce30 comp rx=0 tx=0).stop 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 shutdown_connections 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1070103930 0x7f107019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1070102f70 0x7f1070198480 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f105c0a2c40 0x7f105c0a5100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 --2- 172.21.15.86:0/3350841494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10700ffef0 0x7f1070197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 >> 172.21.15.86:0/3350841494 conn(0x7f10700fba40 msgr2=0x7f1070106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 shutdown_connections 2024-09-05T06:40:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:53.827+0000 7f1075078640 1 -- 172.21.15.86:0/3350841494 wait complete. 2024-09-05T06:40:54.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:54 smithi096 ceph-mon[34468]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:54 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3350841494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T06:40:54.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:54 smithi086 ceph-mon[30943]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:54.610 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:54 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3350841494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T06:40:54.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-05T06:40:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:54 smithi115 ceph-mon[34791]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:40:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:54 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3350841494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T06:40:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 -- 172.21.15.86:0/2870019027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710101d10 msgr2=0x7fc710102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 --2- 172.21.15.86:0/2870019027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710101d10 0x7fc710102190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc704009900 tx=0x7fc70402a410 comp rx=0 tx=0).stop 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 -- 172.21.15.86:0/2870019027 shutdown_connections 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 --2- 172.21.15.86:0/2870019027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 0x7fc710108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 --2- 172.21.15.86:0/2870019027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710101d10 0x7fc710102190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 --2- 172.21.15.86:0/2870019027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710100b10 0x7fc710100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.422+0000 7fc71613e640 1 -- 172.21.15.86:0/2870019027 >> 172.21.15.86:0/2870019027 conn(0x7fc7100fc2a0 msgr2=0x7fc7100fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.423+0000 7fc71613e640 1 -- 172.21.15.86:0/2870019027 shutdown_connections 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.423+0000 7fc71613e640 1 -- 172.21.15.86:0/2870019027 wait complete. 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 Processor -- start 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 -- start start 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710101d10 0x7fc710103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 0x7fc7101038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c640 con 0x7fc710100b10 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c4c0 con 0x7fc7101026d0 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc71613e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c7c0 con 0x7fc710101d10 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33812/0 (socket says 172.21.15.86:33812) 2024-09-05T06:40:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 -- 172.21.15.86:0/575806798 learned_addr learned my addr 172.21.15.86:0/575806798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70effd640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710101d10 0x7fc710103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70ffff640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 0x7fc7101038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 -- 172.21.15.86:0/575806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710101d10 msgr2=0x7fc710103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710101d10 0x7fc710103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 -- 172.21.15.86:0/575806798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 msgr2=0x7fc7101038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 0x7fc7101038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.424+0000 7fc70f7fe640 1 -- 172.21.15.86:0/575806798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc710104180 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc70f7fe640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9c0 tx=0x7fc70000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000cd80 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101a1fe0 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7000035f0 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700010690 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.425+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a2380 con 0x7fc710100b10 2024-09-05T06:40:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.426+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7101a2610 con 0x7fc710100b10 2024-09-05T06:40:55.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.428+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc7000037a0 con 0x7fc710100b10 2024-09-05T06:40:55.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.428+0000 7fc70cff9640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:55.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.428+0000 7fc70effd640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:55.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.429+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fc7000cfc00 con 0x7fc710100b10 2024-09-05T06:40:55.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.429+0000 7fc70effd640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc704000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.432+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc70009c630 con 0x7fc710100b10 2024-09-05T06:40:55.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:55 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:40:55.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.639+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc710108b30 con 0x7fc710100b10 2024-09-05T06:40:55.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:55 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.640+0000 7fc70cff9640 1 -- 172.21.15.86:0/575806798 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5690 (secure 0 0 0) 0x7fc70009bf30 con 0x7fc710100b10 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 89 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-09-05T06:31:26.636788+0000 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-09-05T06:40:13.267085+0000 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 27 2024-09-05T06:40:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/2287131107,v1:172.21.15.86:6803/2287131107] [v2:172.21.15.86:6804/2287131107,v1:172.21.15.86:6805/2287131107] exists,up 964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f 2024-09-05T06:40:55.641 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/3435233109,v1:172.21.15.96:6801/3435233109] [v2:172.21.15.96:6802/3435233109,v1:172.21.15.96:6803/3435233109] exists,up e2892f95-df5c-4d96-955a-d3d741302bc2 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/1161034740,v1:172.21.15.115:6801/1161034740] [v2:172.21.15.115:6802/1161034740,v1:172.21.15.115:6803/1161034740] exists,up 6ec111fa-9256-44e0-b091-da1f005c82d7 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/3414369107,v1:172.21.15.86:6811/3414369107] [v2:172.21.15.86:6812/3414369107,v1:172.21.15.86:6813/3414369107] exists,up d41eff29-18b3-48ac-8d2b-5ef2945f4144 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 31 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/208598279,v1:172.21.15.96:6809/208598279] [v2:172.21.15.96:6810/208598279,v1:172.21.15.96:6811/208598279] exists,up 83c0b669-bfcd-4de2-8349-2eb03e0f28fb 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 35 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/464616540,v1:172.21.15.115:6809/464616540] [v2:172.21.15.115:6810/464616540,v1:172.21.15.115:6811/464616540] exists,up 6110ec1a-05cb-42e3-9e2f-8442e1e2abf2 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 42 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/1960864620,v1:172.21.15.86:6819/1960864620] [v2:172.21.15.86:6820/1960864620,v1:172.21.15.86:6821/1960864620] exists,up 637d7ec0-a913-4e18-92a2-9503fedd4a6b 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 47 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/2748292566,v1:172.21.15.96:6817/2748292566] [v2:172.21.15.96:6818/2748292566,v1:172.21.15.96:6819/2748292566] exists,up 76945c8d-4539-4690-a020-b9af497ee44e 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 54 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/3761965299,v1:172.21.15.115:6817/3761965299] [v2:172.21.15.115:6818/3761965299,v1:172.21.15.115:6819/3761965299] exists,up 608c50ef-1ed5-407d-8375-a0eff5fa26f0 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 61 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/174097683,v1:172.21.15.86:6827/174097683] [v2:172.21.15.86:6828/174097683,v1:172.21.15.86:6829/174097683] exists,up 311dbd75-2e56-4242-b1b4-536719722c62 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 66 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/1196038229,v1:172.21.15.96:6825/1196038229] [v2:172.21.15.96:6826/1196038229,v1:172.21.15.96:6827/1196038229] exists,up d9e00a49-77d5-4256-8ec1-9f43c99a1326 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/3733177277,v1:172.21.15.115:6825/3733177277] [v2:172.21.15.115:6826/3733177277,v1:172.21.15.115:6827/3733177277] exists,up dfda802c-c8b3-4230-81b0-48a3a331b1cf 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.6 [8,11] 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.e [2,1] 2024-09-05T06:40:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.f [8,11] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.14 [2,4] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.6 [4,7] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.b [3,9] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.17 [4,6,5,8] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.1b [5,9] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.21 [4,8] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.24 [4,9] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.6 [9,11] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.e [5,8] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.f [9,8] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.19 [9,10] 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2555553887 expires 2024-09-06T06:32:11.079228+0000 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/1793567579 expires 2024-09-06T06:32:11.079228+0000 2024-09-05T06:40:55.643 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/1793567579 expires 2024-09-06T06:32:11.079228+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2886866246 expires 2024-09-06T06:32:11.079228+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1227677145 expires 2024-09-06T06:31:48.412507+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/164214999 expires 2024-09-06T06:31:48.412507+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/4008919610 expires 2024-09-06T06:32:11.079228+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/164214999 expires 2024-09-06T06:31:48.412507+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/740829125 expires 2024-09-06T06:31:48.412507+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3421533074 expires 2024-09-06T06:31:48.412507+0000 2024-09-05T06:40:55.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 msgr2=0x7fc6f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc704000bc0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 msgr2=0x7fc710104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9c0 tx=0x7fc70000ee80 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 shutdown_connections 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101026d0 0x7fc7101038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc710101d10 0x7fc710103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 --2- 172.21.15.86:0/575806798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc710100b10 0x7fc710104d30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 >> 172.21.15.86:0/575806798 conn(0x7fc7100fc2a0 msgr2=0x7fc7100fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 shutdown_connections 2024-09-05T06:40:55.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:55.642+0000 7fc71613e640 1 -- 172.21.15.86:0/575806798 wait complete. 2024-09-05T06:40:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:55 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:40:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:56 smithi096 ceph-mon[34468]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:56 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/575806798' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T06:40:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:56 smithi115 ceph-mon[34791]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:56 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/575806798' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T06:40:56.701 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:56 smithi086 ceph-mon[30943]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:40:56.701 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:56 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/575806798' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T06:40:56.727 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-05T06:40:56.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:40:57.124 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:57.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.534+0000 7f10899ec640 1 -- 172.21.15.86:0/2459176807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1084102d30 msgr2=0x7f1084103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:57.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.534+0000 7f10899ec640 1 --2- 172.21.15.86:0/2459176807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1084102d30 0x7f1084103110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1074009cd0 tx=0x7f107402a910 comp rx=0 tx=0).stop 2024-09-05T06:40:57.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 -- 172.21.15.86:0/2459176807 shutdown_connections 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 --2- 172.21.15.86:0/2459176807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10841082a0 0x7f108410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 --2- 172.21.15.86:0/2459176807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841036e0 0x7f1084107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 --2- 172.21.15.86:0/2459176807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1084102d30 0x7f1084103110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 -- 172.21.15.86:0/2459176807 >> 172.21.15.86:0/2459176807 conn(0x7f10840fc7d0 msgr2=0x7f10840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 -- 172.21.15.86:0/2459176807 shutdown_connections 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 -- 172.21.15.86:0/2459176807 wait complete. 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 Processor -- start 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.535+0000 7f10899ec640 1 -- start start 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 0x7f1084198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841082a0 0x7f108419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112010 con 0x7f10841036e0 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1084111e90 con 0x7f10841082a0 2024-09-05T06:40:57.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112190 con 0x7f1084102d30 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33824/0 (socket says 172.21.15.86:33824) 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 -- 172.21.15.86:0/3313450167 learned_addr learned my addr 172.21.15.86:0/3313450167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10837fe640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841082a0 0x7f108419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f1082ffd640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 0x7f1084198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 -- 172.21.15.86:0/3313450167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 msgr2=0x7f1084198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 0x7f1084198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 -- 172.21.15.86:0/3313450167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841082a0 msgr2=0x7f108419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841082a0 0x7f108419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 -- 172.21.15.86:0/3313450167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108419d5e0 con 0x7f10841036e0 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f1082ffd640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 0x7f1084198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10827fc640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f106c006fd0 tx=0x7f106c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c011830 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f106c016070 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108419d8d0 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.536+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c010df0 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.537+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a51b0 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.537+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108406a850 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.539+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f106c0119d0 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.540+0000 7f10889ea640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 0x7f10700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.540+0000 7f1082ffd640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 0x7f10700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.540+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f106c0cf1c0 con 0x7f10841036e0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.540+0000 7f1082ffd640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 0x7f10700a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f80 tx=0x7f1074005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.543+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f106c09bd00 con 0x7f10841036e0 2024-09-05T06:40:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.774+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1084103190 con 0x7f10841036e0 2024-09-05T06:40:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.775+0000 7f10889ea640 1 -- 172.21.15.86:0/3313450167 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7f1084103190 con 0x7f10841036e0 2024-09-05T06:40:57.775 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:57.775 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:40:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 msgr2=0x7f10700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 0x7f10700a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f80 tx=0x7f1074005f40 comp rx=0 tx=0).stop 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 msgr2=0x7f1084198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f106c006fd0 tx=0x7f106c00b8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 shutdown_connections 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f10700a2c10 0x7f10700a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10841082a0 0x7f108419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10841036e0 0x7f1084198b70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 --2- 172.21.15.86:0/3313450167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1084102d30 0x7f1084198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 >> 172.21.15.86:0/3313450167 conn(0x7f10840fc7d0 msgr2=0x7f1084105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 shutdown_connections 2024-09-05T06:40:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:57.777+0000 7f10899ec640 1 -- 172.21.15.86:0/3313450167 wait complete. 2024-09-05T06:40:57.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:57 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:57 smithi086 ceph-mon[30943]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:57 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:57 smithi115 ceph-mon[34791]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:58.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:57 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:57 smithi096 ceph-mon[34468]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:40:58.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:40:58.829 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:40:58.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:58 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3313450167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:58 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:58 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3313450167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:58.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:58.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:58 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:59.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:58 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3313450167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:59.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:59.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:58 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:59.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.321+0000 7f55b7519640 1 -- 172.21.15.86:0/1520075740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0101420 msgr2=0x7f55b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:59.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 --2- 172.21.15.86:0/1520075740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0101420 0x7f55b010d950 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009cd0 tx=0x7f55a802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 -- 172.21.15.86:0/1520075740 shutdown_connections 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 --2- 172.21.15.86:0/1520075740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0101420 0x7f55b010d950 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 --2- 172.21.15.86:0/1520075740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 --2- 172.21.15.86:0/1520075740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0106a80 0x7f55b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 -- 172.21.15.86:0/1520075740 >> 172.21.15.86:0/1520075740 conn(0x7f55b00fc7a0 msgr2=0x7f55b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 -- 172.21.15.86:0/1520075740 shutdown_connections 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 -- 172.21.15.86:0/1520075740 wait complete. 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 Processor -- start 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.322+0000 7f55b7519640 1 -- start start 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0101420 0x7f55b0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:59.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0110e20 con 0x7f55b0106a80 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0110ca0 con 0x7f55b0100a80 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0110fa0 con 0x7f55b0101420 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33838/0 (socket says 172.21.15.86:33838) 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 -- 172.21.15.86:0/1658593777 learned_addr learned my addr 172.21.15.86:0/1658593777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b4ab4640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0101420 0x7f55b0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b52b5640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 -- 172.21.15.86:0/1658593777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0101420 msgr2=0x7f55b0197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0101420 0x7f55b0197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 -- 172.21.15.86:0/1658593777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 msgr2=0x7f55b0197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 -- 172.21.15.86:0/1658593777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b01a6990 con 0x7f55b0106a80 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b52b5640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b5ab6640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009ca0 tx=0x7f55a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004240 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55a80305a0 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8045670 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b01a6c20 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.323+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b01a7160 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.324+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b0100f60 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.325+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f55a8030750 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.326+0000 7f55a67fc640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 0x7f559c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.326+0000 7f55b52b5640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 0x7f559c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.326+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f55a802e030 con 0x7f55b0106a80 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.326+0000 7f55b52b5640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 0x7f559c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006fd0 tx=0x7f55ac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:40:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.330+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55a804a060 con 0x7f55b0106a80 2024-09-05T06:40:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.532+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55b002d7c0 con 0x7f55b0106a80 2024-09-05T06:40:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.533+0000 7f55a67fc640 1 -- 172.21.15.86:0/1658593777 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7f55a80bd490 con 0x7f55b0106a80 2024-09-05T06:40:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:40:59.534 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"btime":"2024-09-05T06:39:33:863395+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:39:33.863378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:40:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 msgr2=0x7f559c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 0x7f559c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006fd0 tx=0x7f55ac008040 comp rx=0 tx=0).stop 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 msgr2=0x7f55b0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009ca0 tx=0x7f55a8003ba0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 shutdown_connections 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55b0106a80 0x7f55b0197ba0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b0101420 0x7f55b0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f559c0a2c10 0x7f559c0a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 --2- 172.21.15.86:0/1658593777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55b0100a80 0x7f55b0197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:40:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 >> 172.21.15.86:0/1658593777 conn(0x7f55b00fc7a0 msgr2=0x7f55b00fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:40:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 shutdown_connections 2024-09-05T06:40:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:40:59.535+0000 7f55b7519640 1 -- 172.21.15.86:0/1658593777 wait complete. 2024-09-05T06:40:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:40:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:40:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:59 smithi115 ceph-mon[34791]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:59 smithi115 ceph-mon[34791]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:40:59 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1658593777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:40:59.974 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:40:59.974 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:40:59.974 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:59 smithi086 ceph-mon[30943]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:40:59.974 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:59 smithi086 ceph-mon[30943]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:40:59.974 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:40:59 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1658593777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:59 smithi096 ceph-mon[34468]: from='mgr.14150 172.21.15.86:0/1375911555' entity='mgr.x' 2024-09-05T06:41:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:59 smithi096 ceph-mon[34468]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:41:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:40:59 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1658593777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:00.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:41:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 -- 172.21.15.86:0/972968275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 msgr2=0x7f802410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 --2- 172.21.15.86:0/972968275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802410ea90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f801c009ca0 tx=0x7f801c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 -- 172.21.15.86:0/972968275 shutdown_connections 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 --2- 172.21.15.86:0/972968275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802410ea90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 --2- 172.21.15.86:0/972968275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 0x7f80241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 --2- 172.21.15.86:0/972968275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 0x7f80241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 -- 172.21.15.86:0/972968275 >> 172.21.15.86:0/972968275 conn(0x7f80240fd8c0 msgr2=0x7f80240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 -- 172.21.15.86:0/972968275 shutdown_connections 2024-09-05T06:41:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.087+0000 7f80295a5640 1 -- 172.21.15.86:0/972968275 wait complete. 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 Processor -- start 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 -- start start 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 0x7f80241982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 0x7f80241987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111e20 con 0x7f8024106f10 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111ca0 con 0x7f8024101de0 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.088+0000 7f80295a5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111fa0 con 0x7f8024102790 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33842/0 (socket says 172.21.15.86:33842) 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 -- 172.21.15.86:0/774956997 learned_addr learned my addr 172.21.15.86:0/774956997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80227fc640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 0x7f80241987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f8022ffd640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 0x7f80241982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 -- 172.21.15.86:0/774956997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 msgr2=0x7f80241987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 0x7f80241987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 -- 172.21.15.86:0/774956997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 msgr2=0x7f80241982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 0x7f80241982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 -- 172.21.15.86:0/774956997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802419d250 con 0x7f8024106f10 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80237fe640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f801c009dc0 tx=0x7f801c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c004040 con 0x7f8024106f10 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802419d4e0 con 0x7f8024106f10 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801c004690 con 0x7f8024106f10 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c030c90 con 0x7f8024106f10 2024-09-05T06:41:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.089+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a4e20 con 0x7f8024106f10 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.090+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80241021c0 con 0x7f8024106f10 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.091+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f801c0041e0 con 0x7f8024106f10 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.092+0000 7f800bfff640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 0x7f800c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.092+0000 7f8022ffd640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 0x7f800c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.092+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f801c0f0ca0 con 0x7f8024106f10 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.093+0000 7f8022ffd640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 0x7f800c0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80140058c0 tx=0x7f801400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.097+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f801c0bd7e0 con 0x7f8024106f10 2024-09-05T06:41:01.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.264+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f802410a180 con 0x7f8024106f10 2024-09-05T06:41:01.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.265+0000 7f800bfff640 1 -- 172.21.15.86:0/774956997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24905 (secure 0 0 0) 0x7f801c0bd0e0 con 0x7f8024106f10 2024-09-05T06:41:01.266 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:40:13.267085+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:41:01.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.267+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 msgr2=0x7f800c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.267+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 0x7f800c0a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80140058c0 tx=0x7f801400a4c0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.267+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 msgr2=0x7f802419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.267+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f801c009dc0 tx=0x7f801c003ba0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 shutdown_connections 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024106f10 0x7f802419cb70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8024102790 0x7f80241987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f800c0a2c10 0x7f800c0a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 --2- 172.21.15.86:0/774956997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8024101de0 0x7f80241982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 >> 172.21.15.86:0/774956997 conn(0x7f80240fd8c0 msgr2=0x7f80240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 shutdown_connections 2024-09-05T06:41:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:01.268+0000 7f80295a5640 1 -- 172.21.15.86:0/774956997 wait complete. 2024-09-05T06:41:01.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:01 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/774956997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:01 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/774956997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:01 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/774956997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:01.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T06:38:55.927385+0000', 'modified': '2024-09-05T06:39:33.863378+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24466, 'mds_2': 24461, 'mds_3': 24422, 'mds_4': 24403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3075373024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3075373024}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3075373024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/1507644375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 1507644375}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 1507644375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.115:6833/3498674074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3498674074}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3498674074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.96:6833/1668283475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 1668283475}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 1668283475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/1272008759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 1272008759}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 1272008759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/3823127248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3823127248}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3823127248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6837/2087354694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 2087354694}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2087354694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.96:6837/1007288434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 1007288434}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 1007288434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.115:6839/4249007973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4249007973}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4249007973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/2996609527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2996609527}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2996609527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24466, 24461, 24422, 24403]} 2024-09-05T06:41:01.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T06:41:01.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.l status 2024-09-05T06:41:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:02.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:02 smithi086 ceph-mon[30943]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:41:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:02 smithi115 ceph-mon[34791]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:41:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 -- 172.21.15.86:0/211461381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 msgr2=0x7f75f010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 --2- 172.21.15.86:0/211461381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f010bb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009900 tx=0x7f75e402a480 comp rx=0 tx=0).stop 2024-09-05T06:41:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 -- 172.21.15.86:0/211461381 shutdown_connections 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 --2- 172.21.15.86:0/211461381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f010c170 0x7f75f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 --2- 172.21.15.86:0/211461381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f010bb50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 --2- 172.21.15.86:0/211461381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f0102d40 0x7f75f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 -- 172.21.15.86:0/211461381 >> 172.21.15.86:0/211461381 conn(0x7f75f00fc7a0 msgr2=0x7f75f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 -- 172.21.15.86:0/211461381 shutdown_connections 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.755+0000 7f75f6009640 1 -- 172.21.15.86:0/211461381 wait complete. 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 Processor -- start 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 -- start start 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f0102d40 0x7f75f01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f010c170 0x7f75f019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0111e20 con 0x7f75f0103660 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0111ca0 con 0x7f75f0102d40 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75f6009640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0111fa0 con 0x7f75f010c170 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42342/0 (socket says 172.21.15.86:42342) 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75eeffd640 1 -- 172.21.15.86:0/2982167050 learned_addr learned my addr 172.21.15.86:0/2982167050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.756+0000 7f75effff640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f010c170 0x7f75f019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f0102d40 0x7f75f01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 -- 172.21.15.86:0/2982167050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f010c170 msgr2=0x7f75f019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f010c170 0x7f75f019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 -- 172.21.15.86:0/2982167050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f0102d40 msgr2=0x7f75f01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f0102d40 0x7f75f01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 -- 172.21.15.86:0/2982167050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f019d370 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75eeffd640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005940 tx=0x7f75e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75ecff9640 1 -- 172.21.15.86:0/2982167050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e403f030 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f019d600 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75ecff9640 1 -- 172.21.15.86:0/2982167050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75e4003cb0 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75ecff9640 1 -- 172.21.15.86:0/2982167050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4035070 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.757+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a4f40 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.759+0000 7f75ecff9640 1 -- 172.21.15.86:0/2982167050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f75e4042030 con 0x7f75f0103660 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.759+0000 7f75f83e2740 1 Processor -- start 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.759+0000 7f75f83e2740 1 -- start start 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c76733090 con 0x561c767cc9d0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c766daa70 con 0x561c767d7bb0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75f83e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c7672f050 con 0x561c767cd2f0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75ecff9640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 0x7f75d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42358/0 (socket says 172.21.15.86:42358) 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 -- 172.21.15.86:0/1166824262 learned_addr learned my addr 172.21.15.86:0/1166824262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 0x7f75d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75ecff9640 1 -- 172.21.15.86:0/2982167050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f75e40efde0 con 0x7f75f0103660 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 -- 172.21.15.86:0/1166824262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 msgr2=0x561c767d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767d7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 -- 172.21.15.86:0/1166824262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 msgr2=0x561c767d9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767d9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 -- 172.21.15.86:0/1166824262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c766e7130 con 0x561c767cc9d0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75e8014ea0 tx=0x7f75e8014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89731734a026eb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75effff640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 0x7f75d80a5150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75e801acc0 tx=0x7f75e8021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.760+0000 7f75d5ffb640 1 -- 172.21.15.86:0/1166824262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8027070 con 0x561c767cc9d0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75d5ffb640 1 -- 172.21.15.86:0/1166824262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75e8009c70 con 0x561c767cc9d0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75d5ffb640 1 -- 172.21.15.86:0/1166824262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e80054e0 con 0x561c767cc9d0 2024-09-05T06:41:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 -- 172.21.15.86:0/1166824262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 msgr2=0x561c767ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75e8014ea0 tx=0x7f75e8014f90 comp rx=0 tx=0).stop 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 -- 172.21.15.86:0/1166824262 shutdown_connections 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1166824262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767ccdb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 -- 172.21.15.86:0/1166824262 >> 172.21.15.86:0/1166824262 conn(0x561c767c8f60 msgr2=0x561c767c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 -- 172.21.15.86:0/1166824262 shutdown_connections 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.761+0000 7f75f83e2740 1 -- 172.21.15.86:0/1166824262 wait complete. 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 -- start start 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 Processor -- start 2024-09-05T06:41:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 0x561c767d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c767da780 con 0x561c767cc9d0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c766edf20 con 0x561c767cd2f0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c767da900 con 0x561c767d7bb0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75f83e2740 10 client.0 _ll_register_callbacks cb 0x561c766b7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42368/0 (socket says 172.21.15.86:42368) 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75ef7fe640 1 -- 172.21.15.86:0/309784418 learned_addr learned my addr 172.21.15.86:0/309784418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75effff640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75ef7fe640 1 -- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 msgr2=0x561c767d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.762+0000 7f75eeffd640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 0x561c767d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ef7fe640 1 -- 172.21.15.86:0/309784418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 msgr2=0x561c767d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 0x561c767d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ef7fe640 1 -- 172.21.15.86:0/309784418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c764a42b0 con 0x561c767cc9d0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75effff640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75eeffd640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 0x561c767d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75e00106b0 tx=0x7f75e0010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d0cb54da8363 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e001d060 con 0x561c767cc9d0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e8009c70 con 0x561c767cc9d0 2024-09-05T06:41:02.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f75e000ed20 con 0x561c767cc9d0 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75e0024050 con 0x561c767cc9d0 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 10 client.15198 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c767e61a0 con 0x561c767cc9d0 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75e8009c70 con 0x561c767cc9d0 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f75e000d4c0 con 0x561c767cc9d0 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 10 client.15198 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.763+0000 7f75f83e2740 20 client.15198 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c767e87c0 0x561c767e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75f83e2740 4 client.15198 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c76740600 con 0x561c767e87c0 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75eeffd640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c767e87c0 0x561c767e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75eeffd640 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c767e87c0 0x561c767e9c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c995e1bc77da1541 server_cookie=b62f468a5ac60b9a in_seq=0 out_seq=0 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.764+0000 7f75ceffd640 10 client.15198 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75ceffd640 1 -- 172.21.15.86:0/309784418 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c76740600 con 0x561c767e87c0 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75ceffd640 10 client.15198 handle_command_reply: tid=0 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 client.15198 shutdown 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 msgr2=0x561c767d74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75e00106b0 tx=0x7f75e0010b70 comp rx=0 tx=0).stop 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 shutdown_connections 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c767e87c0 0x561c767e9c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cd2f0 0x561c767d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75ceffd640 0 client.15198 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 --2- 172.21.15.86:0/309784418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767d74b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 >> 172.21.15.86:0/309784418 conn(0x561c767c8f60 msgr2=0x561c767cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 shutdown_connections 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.765+0000 7f75f83e2740 1 -- 172.21.15.86:0/309784418 wait complete. 2024-09-05T06:41:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.766+0000 7f75f83e2740 20 client.15198 trim_cache size 0 max 16384 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 Processor -- start 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 -- start start 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cc9d0 0x561c767e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c767e7f70 con 0x561c767cd2f0 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c767e8640 con 0x561c767cc9d0 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75f83e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c767e8d10 con 0x561c767d7bb0 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42378/0 (socket says 172.21.15.86:42378) 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.783+0000 7f75eeffd640 1 -- 172.21.15.86:0/1085901305 learned_addr learned my addr 172.21.15.86:0/1085901305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:02.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75effff640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 -- 172.21.15.86:0/1085901305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 msgr2=0x561c767d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cc9d0 0x561c767e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 -- 172.21.15.86:0/1085901305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cc9d0 msgr2=0x561c767e9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cc9d0 0x561c767e9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 -- 172.21.15.86:0/1085901305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c767d5340 con 0x561c767cd2f0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75effff640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f75e4003fd0 tx=0x7f75e40dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb2369d742c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75d57fa640 1 -- 172.21.15.86:0/1085901305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40d4070 con 0x561c767cd2f0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75d57fa640 1 -- 172.21.15.86:0/1085901305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75e40bc110 con 0x561c767cd2f0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.784+0000 7f75d57fa640 1 -- 172.21.15.86:0/1085901305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4049040 con 0x561c767cd2f0 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- 172.21.15.86:0/1085901305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 msgr2=0x561c767c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f75e4003fd0 tx=0x7f75e40dc040 comp rx=0 tx=0).stop 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- 172.21.15.86:0/1085901305 shutdown_connections 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767d7bb0 0x561c767d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cd2f0 0x561c767c9340 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1085901305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767cc9d0 0x561c767e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- 172.21.15.86:0/1085901305 >> 172.21.15.86:0/1085901305 conn(0x561c767c8f60 msgr2=0x561c767d5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- 172.21.15.86:0/1085901305 shutdown_connections 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- 172.21.15.86:0/1085901305 wait complete. 2024-09-05T06:41:02.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 -- start start 2024-09-05T06:41:02.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 Processor -- start 2024-09-05T06:41:02.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.785+0000 7f75f83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c767e9ab0 con 0x561c767cc9d0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c767e93e0 con 0x561c767d7bb0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c767ea180 con 0x561c767cd2f0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 10 client.0 _ll_register_callbacks cb 0x7f75e000b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42390/0 (socket says 172.21.15.86:42390) 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 -- 172.21.15.86:0/1842664539 learned_addr learned my addr 172.21.15.86:0/1842664539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75effff640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 -- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 msgr2=0x561c767ed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767ed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 -- 172.21.15.86:0/1842664539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 msgr2=0x561c767edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767edcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 -- 172.21.15.86:0/1842664539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c767ee2b0 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ef7fe640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bbb0 tx=0x7f75e0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96422dc34cc6921 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e000d5a0 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f75e000eb20 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e40bc110 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75e001d020 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 10 client.15204 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.786+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c76852c30 con 0x561c767cc9d0 2024-09-05T06:41:02.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75e40bc110 con 0x561c767cc9d0 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f75e0004660 con 0x561c767cc9d0 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 10 client.15204 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 20 client.15204 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c76857820 0x561c76856530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 4 client.15204 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "status"}) -- 0x561c7673ab10 con 0x561c76857820 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.787+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c76857820 0x561c76856530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75eeffd640 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c76857820 0x561c76856530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f51c3fd74ed9885a server_cookie=8bc8129ec95ed83d in_seq=0 out_seq=0 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75ceffd640 10 client.15204 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75ceffd640 1 -- 172.21.15.86:0/1842664539 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561c7673ab10 con 0x561c76857820 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75ceffd640 10 client.15204 handle_command_reply: tid=0 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 client.15204 shutdown 2024-09-05T06:41:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 msgr2=0x561c767f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bbb0 tx=0x7f75e0005d20 comp rx=0 tx=0).stop 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 shutdown_connections 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c76857820 0x561c76856530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c767d7bb0 0x561c767edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c767cd2f0 0x561c767ed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75ceffd640 0 client.15204 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 --2- 172.21.15.86:0/1842664539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c767cc9d0 0x561c767f23e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 >> 172.21.15.86:0/1842664539 conn(0x561c767c8f60 msgr2=0x561c767c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 shutdown_connections 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 1 -- 172.21.15.86:0/1842664539 wait complete. 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.788+0000 7f75f83e2740 20 client.15204 trim_cache size 0 max 16384 2024-09-05T06:41:02.795 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24403, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 89.939554904999994, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 168.20139523 2024-09-05T06:41:02.796 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 msgr2=0x7f75d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 0x7f75d80a5150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75e801acc0 tx=0x7f75e8021000 comp rx=0 tx=0).stop 2024-09-05T06:41:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 msgr2=0x7f75f0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005940 tx=0x7f75e40042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 shutdown_connections 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75f010c170 0x7f75f019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0103660 0x7f75f0198900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f75d80a2c90 0x7f75d80a5150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 --2- 172.21.15.86:0/2982167050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75f0102d40 0x7f75f01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 >> 172.21.15.86:0/2982167050 conn(0x7f75f00fc7a0 msgr2=0x7f75f00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 shutdown_connections 2024-09-05T06:41:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:02.789+0000 7f75f6009640 1 -- 172.21.15.86:0/2982167050 wait complete. 2024-09-05T06:41:02.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:02 smithi096 ceph-mon[34468]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T06:41:03.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.k status 2024-09-05T06:41:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:04.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 -- 172.21.15.86:0/1536530664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 msgr2=0x7fd538107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 --2- 172.21.15.86:0/1536530664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538107b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009930 tx=0x7fd52c02a420 comp rx=0 tx=0).stop 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 -- 172.21.15.86:0/1536530664 shutdown_connections 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 --2- 172.21.15.86:0/1536530664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5381082a0 0x7fd53810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 --2- 172.21.15.86:0/1536530664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538107b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 --2- 172.21.15.86:0/1536530664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd538102d30 0x7fd538103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 -- 172.21.15.86:0/1536530664 >> 172.21.15.86:0/1536530664 conn(0x7fd5380fc7d0 msgr2=0x7fd5380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 -- 172.21.15.86:0/1536530664 shutdown_connections 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.234+0000 7fd53e11b640 1 -- 172.21.15.86:0/1536530664 wait complete. 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 Processor -- start 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 -- start start 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd538102d30 0x7fd5381984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5381082a0 0x7fd53819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd538111f50 con 0x7fd5381036e0 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd538111dd0 con 0x7fd5381082a0 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.235+0000 7fd53e11b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381120d0 con 0x7fd538102d30 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42410/0 (socket says 172.21.15.86:42410) 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 -- 172.21.15.86:0/311451898 learned_addr learned my addr 172.21.15.86:0/311451898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd5377fe640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd538102d30 0x7fd5381984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd537fff640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5381082a0 0x7fd53819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 -- 172.21.15.86:0/311451898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd538102d30 msgr2=0x7fd5381984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd538102d30 0x7fd5381984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 -- 172.21.15.86:0/311451898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5381082a0 msgr2=0x7fd53819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5381082a0 0x7fd53819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 -- 172.21.15.86:0/311451898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53819d480 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd536ffd640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009230 tx=0x7fd52c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd534ff9640 1 -- 172.21.15.86:0/311451898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c03f030 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53819d710 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd534ff9640 1 -- 172.21.15.86:0/311451898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52c0035f0 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd534ff9640 1 -- 172.21.15.86:0/311451898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c035070 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.236+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381a5050 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.238+0000 7fd534ff9640 1 -- 172.21.15.86:0/311451898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd52c003aa0 con 0x7fd5381036e0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 Processor -- start 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 -- start start 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 0x55deb8647db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55deb85ae090 con 0x55deb8652bb0 2024-09-05T06:41:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55deb8555a70 con 0x55deb86482f0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5404f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55deb85aa050 con 0x55deb86479d0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd534ff9640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 0x7fd5200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd537fff640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 0x7fd5200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd534ff9640 1 -- 172.21.15.86:0/311451898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fd52c0eff60 con 0x7fd5381036e0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:36524/0 (socket says 172.21.15.86:36524) 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42414/0 (socket says 172.21.15.86:42414) 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.239+0000 7fd5377fe640 1 -- 172.21.15.86:0/3438810579 learned_addr learned my addr 172.21.15.86:0/3438810579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd537fff640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 0x55deb8647db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 -- 172.21.15.86:0/3438810579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 msgr2=0x55deb8647db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 0x55deb8647db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 -- 172.21.15.86:0/3438810579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 msgr2=0x55deb8652670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 -- 172.21.15.86:0/3438810579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb8562130 con 0x55deb8652bb0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd5377fe640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd537fff640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 0x55deb8647db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd537fff640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 0x7fd5200a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd530005cc0 tx=0x7fd530012a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0ed540 tx=0x7fd52c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23da43808da2ecb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd51dffb640 1 -- 172.21.15.86:0/3438810579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c049040 con 0x55deb8652bb0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd51dffb640 1 -- 172.21.15.86:0/3438810579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52c0bc290 con 0x55deb8652bb0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.240+0000 7fd51dffb640 1 -- 172.21.15.86:0/3438810579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0037a0 con 0x55deb8652bb0 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- 172.21.15.86:0/3438810579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 msgr2=0x55deb8654fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0ed540 tx=0x7fd52c009260 comp rx=0 tx=0).stop 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- 172.21.15.86:0/3438810579 shutdown_connections 2024-09-05T06:41:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb8654fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86482f0 0x55deb8652670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3438810579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86479d0 0x55deb8647db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- 172.21.15.86:0/3438810579 >> 172.21.15.86:0/3438810579 conn(0x55deb8643f60 msgr2=0x55deb8644340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- 172.21.15.86:0/3438810579 shutdown_connections 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- 172.21.15.86:0/3438810579 wait complete. 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.241+0000 7fd5404f4740 1 -- start start 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 Processor -- start 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb8652460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86482f0 0x55deb864f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55deb8655540 con 0x55deb8652bb0 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55deb8568f20 con 0x55deb86479d0 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55deb86556c0 con 0x55deb86482f0 2024-09-05T06:41:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5404f4740 10 client.0 _ll_register_callbacks cb 0x55deb8532030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd537fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd537fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42416/0 (socket says 172.21.15.86:42416) 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd537fff640 1 -- 172.21.15.86:0/480491968 learned_addr learned my addr 172.21.15.86:0/480491968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd5377fe640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb8652460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.242+0000 7fd536ffd640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86482f0 0x55deb864f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 -- 172.21.15.86:0/480491968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86482f0 msgr2=0x55deb864f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86482f0 0x55deb864f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 -- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 msgr2=0x55deb8652460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb8652460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 -- 172.21.15.86:0/480491968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb831f2b0 con 0x55deb8652bb0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd537fff640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd53000ab50 tx=0x7fd530018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8cc26b322fb323 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530024070 con 0x55deb8652bb0 2024-09-05T06:41:04.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fd530003b70 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c0bc290 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd53002f050 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd5404f4740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55deb8660d50 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.243+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52c0bc290 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fd530003d70 con 0x55deb8652bb0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 10 client.15216 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 20 client.15216 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb8663390 0x55deb8664800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 4 client.15216 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb85bb600 con 0x55deb8663390 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5377fe640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb8663390 0x55deb8664800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd5377fe640 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb8663390 0x55deb8664800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1a3230cb41b40b2 server_cookie=7e9f405b97f978b9 in_seq=0 out_seq=0 2024-09-05T06:41:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.244+0000 7fd516ffd640 10 client.15216 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd516ffd640 1 -- 172.21.15.86:0/480491968 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deb85bb600 con 0x55deb8663390 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd516ffd640 10 client.15216 handle_command_reply: tid=0 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 client.15216 shutdown 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 msgr2=0x55deb864fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd53000ab50 tx=0x7fd530018000 comp rx=0 tx=0).stop 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 shutdown_connections 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb8663390 0x55deb8664800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb8652bb0 0x55deb864fcf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb86482f0 0x55deb864f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 --2- 172.21.15.86:0/480491968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb8652460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd516ffd640 0 client.15216 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 >> 172.21.15.86:0/480491968 conn(0x55deb8643f60 msgr2=0x55deb8649c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 shutdown_connections 2024-09-05T06:41:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.245+0000 7fd5404f4740 1 -- 172.21.15.86:0/480491968 wait complete. 2024-09-05T06:41:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.246+0000 7fd5404f4740 20 client.15216 trim_cache size 0 max 16384 2024-09-05T06:41:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.263+0000 7fd5404f4740 1 Processor -- start 2024-09-05T06:41:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.263+0000 7fd5404f4740 1 -- start start 2024-09-05T06:41:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb865e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55deb8662b40 con 0x55deb86482f0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55deb8663210 con 0x55deb86479d0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5404f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55deb86638e0 con 0x55deb8652bb0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42418/0 (socket says 172.21.15.86:42418) 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 -- 172.21.15.86:0/3825860134 learned_addr learned my addr 172.21.15.86:0/3825860134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd537fff640 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd5377fe640 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb865e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 -- 172.21.15.86:0/3825860134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 msgr2=0x55deb864e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 -- 172.21.15.86:0/3825860134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 msgr2=0x55deb865e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb865e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 -- 172.21.15.86:0/3825860134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb8644b10 con 0x55deb86482f0 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.264+0000 7fd536ffd640 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd52c000c00 tx=0x7fd52c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e31a0e2ffaf0804 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd51d7fa640 1 -- 172.21.15.86:0/3825860134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0da070 con 0x55deb86482f0 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd51d7fa640 1 -- 172.21.15.86:0/3825860134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52c0c4f00 con 0x55deb86482f0 2024-09-05T06:41:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd51d7fa640 1 -- 172.21.15.86:0/3825860134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c049070 con 0x55deb86482f0 2024-09-05T06:41:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 -- 172.21.15.86:0/3825860134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 msgr2=0x55deb865e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd52c000c00 tx=0x7fd52c031400 comp rx=0 tx=0).stop 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 -- 172.21.15.86:0/3825860134 shutdown_connections 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86482f0 0x55deb865e660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 --2- 172.21.15.86:0/3825860134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb86479d0 0x55deb865e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 -- 172.21.15.86:0/3825860134 >> 172.21.15.86:0/3825860134 conn(0x55deb8643f60 msgr2=0x55deb864f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.265+0000 7fd5404f4740 1 -- 172.21.15.86:0/3825860134 shutdown_connections 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 -- 172.21.15.86:0/3825860134 wait complete. 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 -- start start 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 Processor -- start 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb864c900 0x55deb866bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55deb8664680 con 0x55deb86479d0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55deb8663fb0 con 0x55deb864c900 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55deb8664d50 con 0x55deb8652bb0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5404f4740 10 client.0 _ll_register_callbacks cb 0x7fd53000a5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.266+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42428/0 (socket says 172.21.15.86:42428) 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5377fe640 1 -- 172.21.15.86:0/4107157939 learned_addr learned my addr 172.21.15.86:0/4107157939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd537fff640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb864c900 0x55deb866bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 -- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb864c900 msgr2=0x55deb866bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb864c900 0x55deb866bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 -- 172.21.15.86:0/4107157939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 msgr2=0x55deb864bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 -- 172.21.15.86:0/4107157939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb866c2c0 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5377fe640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd536ffd640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864c320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55deb8644d40 tx=0x7fd52c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345375a3555364bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c086060 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd52c0c4f00 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fd52c0b8900 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd52c049070 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5404f4740 10 client.24808 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.267+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55deb866d580 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52c0c4f00 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fd52c0c4a70 con 0x55deb8652bb0 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 10 client.24808 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 10 client.24808 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:41:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 20 client.24808 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb86d61a0 0x55deb86d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 4 client.24808 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "status"}) -- 0x55deb85b5b10 con 0x55deb86d61a0 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.268+0000 7fd5377fe640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb86d61a0 0x55deb86d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5377fe640 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb86d61a0 0x55deb86d4eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef247f04cd900de2 server_cookie=6777b109a832bb9f in_seq=0 out_seq=0 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd516ffd640 10 client.24808 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd516ffd640 1 -- 172.21.15.86:0/4107157939 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55deb85b5b10 con 0x55deb86d61a0 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd516ffd640 10 client.24808 handle_command_reply: tid=0 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 client.24808 shutdown 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 msgr2=0x55deb864c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864c320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55deb8644d40 tx=0x7fd52c02a420 comp rx=0 tx=0).stop 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 shutdown_connections 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55deb86d61a0 0x55deb86d4eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55deb864c900 0x55deb866bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55deb8652bb0 0x55deb864c320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd516ffd640 0 client.24808 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 --2- 172.21.15.86:0/4107157939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55deb86479d0 0x55deb864bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 >> 172.21.15.86:0/4107157939 conn(0x55deb8643f60 msgr2=0x55deb864a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 shutdown_connections 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 1 -- 172.21.15.86:0/4107157939 wait complete. 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.269+0000 7fd5404f4740 20 client.24808 trim_cache size 0 max 16384 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24422, 2024-09-05T06:41:04.275 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 92.426691238000004, 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 167.06553479999999 2024-09-05T06:41:04.276 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 msgr2=0x7fd5200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 0x7fd5200a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd530005cc0 tx=0x7fd530012a10 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 msgr2=0x7fd538198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009230 tx=0x7fd52c004270 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 shutdown_connections 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fd5200a2c10 0x7fd5200a50d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5381082a0 0x7fd53819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381036e0 0x7fd538198a10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 --2- 172.21.15.86:0/311451898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd538102d30 0x7fd5381984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 >> 172.21.15.86:0/311451898 conn(0x7fd5380fc7d0 msgr2=0x7fd538105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 shutdown_connections 2024-09-05T06:41:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:04.270+0000 7fd53e11b640 1 -- 172.21.15.86:0/311451898 wait complete. 2024-09-05T06:41:04.425 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:04 smithi086 ceph-mon[30943]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:41:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:04 smithi115 ceph-mon[34791]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:41:04.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:04 smithi096 ceph-mon[34468]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T06:41:04.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T06:41:05.256 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 -- 172.21.15.86:0/3172718726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4106a80 msgr2=0x7f84b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 --2- 172.21.15.86:0/3172718726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4106a80 0x7f84b4106e60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009930 tx=0x7f84a802a420 comp rx=0 tx=0).stop 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 -- 172.21.15.86:0/3172718726 shutdown_connections 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 --2- 172.21.15.86:0/3172718726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4101420 0x7f84b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 --2- 172.21.15.86:0/3172718726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4100a80 0x7f84b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 --2- 172.21.15.86:0/3172718726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4106a80 0x7f84b4106e60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 -- 172.21.15.86:0/3172718726 >> 172.21.15.86:0/3172718726 conn(0x7f84b40fc7a0 msgr2=0x7f84b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 -- 172.21.15.86:0/3172718726 shutdown_connections 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.750+0000 7f84b91c6640 1 -- 172.21.15.86:0/3172718726 wait complete. 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 Processor -- start 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 -- start start 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4100a80 0x7f84b41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 0x7f84b419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4110d50 con 0x7f84b4101420 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4110bd0 con 0x7f84b4106a80 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b91c6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4110ed0 con 0x7f84b4100a80 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42446/0 (socket says 172.21.15.86:42446) 2024-09-05T06:41:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b259c640 1 -- 172.21.15.86:0/1893622747 learned_addr learned my addr 172.21.15.86:0/1893622747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.751+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4100a80 0x7f84b41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b359e640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 0x7f84b419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 -- 172.21.15.86:0/1893622747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4100a80 msgr2=0x7f84b41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4100a80 0x7f84b41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 -- 172.21.15.86:0/1893622747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 msgr2=0x7f84b419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 0x7f84b419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 -- 172.21.15.86:0/1893622747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b419d380 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b359e640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 0x7f84b419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b259c640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84a400cce0 tx=0x7f84a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f849bfff640 1 -- 172.21.15.86:0/1893622747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4004b20 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b419d670 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f849bfff640 1 -- 172.21.15.86:0/1893622747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a4004cc0 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f849bfff640 1 -- 172.21.15.86:0/1893622747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400f6b0 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.752+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a4f50 con 0x7f84b4101420 2024-09-05T06:41:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.754+0000 7f849bfff640 1 -- 172.21.15.86:0/1893622747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f84a400f850 con 0x7f84b4101420 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 Processor -- start 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 -- start start 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 0x55827782ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x55827783afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558277794090 con 0x55827782e2f0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55827773ba70 con 0x55827782d9d0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84bb59f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558277790050 con 0x558277838bb0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42452/0 (socket says 172.21.15.86:42452) 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 -- 172.21.15.86:0/2398035375 learned_addr learned my addr 172.21.15.86:0/2398035375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b359e640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 0x55827782ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f849bfff640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b359e640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 -- 172.21.15.86:0/2398035375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 msgr2=0x55827783afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b259c640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x55827783afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f849bfff640 1 -- 172.21.15.86:0/1893622747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f84a40cf660 con 0x7f84b4101420 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x55827783afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 -- 172.21.15.86:0/2398035375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 msgr2=0x55827782ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 0x55827782ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b2d9d640 1 -- 172.21.15.86:0/2398035375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277748130 con 0x55827782e2f0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b359e640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 0x55827782ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.755+0000 7f84b259c640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x55827783afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.756+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009930 tx=0x7f84a8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a7f8b860d76972 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.756+0000 7f84b359e640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 0x7f849c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84ac013360 tx=0x7f84ac0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.756+0000 7f84997fa640 1 -- 172.21.15.86:0/2398035375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8035040 con 0x55827782e2f0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.756+0000 7f84997fa640 1 -- 172.21.15.86:0/2398035375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80316a0 con 0x55827782e2f0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.756+0000 7f84997fa640 1 -- 172.21.15.86:0/2398035375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80047a0 con 0x55827782e2f0 2024-09-05T06:41:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- 172.21.15.86:0/2398035375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 msgr2=0x558277838670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009930 tx=0x7f84a8032d60 comp rx=0 tx=0).stop 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- 172.21.15.86:0/2398035375 shutdown_connections 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x55827783afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277838670 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- 172.21.15.86:0/2398035375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782d9d0 0x55827782ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- 172.21.15.86:0/2398035375 >> 172.21.15.86:0/2398035375 conn(0x558277829f60 msgr2=0x55827782a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- 172.21.15.86:0/2398035375 shutdown_connections 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- 172.21.15.86:0/2398035375 wait complete. 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- start start 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 Processor -- start 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55827782d9d0 0x5582778383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558277838bb0 0x558277835c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55827783b780 con 0x55827782e2f0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55827774ef20 con 0x558277838bb0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55827783b900 con 0x55827782d9d0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84bb59f740 10 client.0 _ll_register_callbacks cb 0x558277718030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42454/0 (socket says 172.21.15.86:42454) 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.757+0000 7f84b259c640 1 -- 172.21.15.86:0/4069795831 learned_addr learned my addr 172.21.15.86:0/4069795831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b359e640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558277838bb0 0x558277835c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 -- 172.21.15.86:0/4069795831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55827782d9d0 msgr2=0x5582778383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55827782d9d0 0x5582778383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 -- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558277838bb0 msgr2=0x558277835c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558277838bb0 0x558277835c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 -- 172.21.15.86:0/4069795831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582775052b0 con 0x55827782e2f0 2024-09-05T06:41:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84b259c640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005c40 tx=0x7f84a401d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b504d1f1cc9cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a409c090 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f84a40a4170 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a80316a0 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84a409b990 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84bb59f740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558277846f80 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80316a0 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.758+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f84a400fc60 con 0x55827782e2f0 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 20 client.15234 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778495b0 0x55827784aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 4 client.15234 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582777a1600 con 0x5582778495b0 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778495b0 0x55827784aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778495b0 0x55827784aa20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4b6df2571100b9c server_cookie=c95909e943d2775f in_seq=0 out_seq=0 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.759+0000 7f8496ff5640 10 client.15234 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f8496ff5640 1 -- 172.21.15.86:0/4069795831 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582777a1600 con 0x5582778495b0 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f8496ff5640 10 client.15234 handle_command_reply: tid=0 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 client.15234 shutdown 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 msgr2=0x558277835700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005c40 tx=0x7f84a401d3f0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 shutdown_connections 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778495b0 0x55827784aa20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558277838bb0 0x558277835c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782e2f0 0x558277835700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f8496ff5640 0 client.15234 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 --2- 172.21.15.86:0/4069795831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55827782d9d0 0x5582778383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 >> 172.21.15.86:0/4069795831 conn(0x558277829f60 msgr2=0x55827782fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 shutdown_connections 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 1 -- 172.21.15.86:0/4069795831 wait complete. 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.760+0000 7f84bb59f740 20 client.15234 trim_cache size 0 max 16384 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 Processor -- start 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 -- start start 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x5582778530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x558277832960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558277848d60 con 0x55827782d9d0 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558277849430 con 0x55827782e2f0 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.778+0000 7f84bb59f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558277849b00 con 0x558277838bb0 2024-09-05T06:41:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42460/0 (socket says 172.21.15.86:42460) 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 -- 172.21.15.86:0/1801952368 learned_addr learned my addr 172.21.15.86:0/1801952368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b259c640 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x5582778530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b359e640 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x558277832960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 -- 172.21.15.86:0/1801952368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 msgr2=0x558277832960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x558277832960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 -- 172.21.15.86:0/1801952368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 msgr2=0x5582778530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x5582778530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 -- 172.21.15.86:0/1801952368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277838720 con 0x55827782d9d0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55827782adc0 tx=0x7f84a8005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fa13d1486ea8b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f8498ff9640 1 -- 172.21.15.86:0/1801952368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8054070 con 0x55827782d9d0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f8498ff9640 1 -- 172.21.15.86:0/1801952368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8004240 con 0x55827782d9d0 2024-09-05T06:41:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.779+0000 7f8498ff9640 1 -- 172.21.15.86:0/1801952368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8035040 con 0x55827782d9d0 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- 172.21.15.86:0/1801952368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 msgr2=0x558277834fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55827782adc0 tx=0x7f84a8005460 comp rx=0 tx=0).stop 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- 172.21.15.86:0/1801952368 shutdown_connections 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x558277832960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x5582778530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 --2- 172.21.15.86:0/1801952368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x558277834fc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- 172.21.15.86:0/1801952368 >> 172.21.15.86:0/1801952368 conn(0x558277829f60 msgr2=0x5582778356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- 172.21.15.86:0/1801952368 shutdown_connections 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- 172.21.15.86:0/1801952368 wait complete. 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 -- start start 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.780+0000 7f84bb59f740 1 Processor -- start 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x55827784c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x5582778b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55827784a8a0 con 0x55827782d9d0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55827784a1d0 con 0x55827782e2f0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55827784af70 con 0x558277838bb0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 10 client.0 _ll_register_callbacks cb 0x7f84a4004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x55827784c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42474/0 (socket says 172.21.15.86:42474) 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 -- 172.21.15.86:0/120619776 learned_addr learned my addr 172.21.15.86:0/120619776 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b359e640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x5582778b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 -- 172.21.15.86:0/120619776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 msgr2=0x5582778b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x5582778b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 -- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 msgr2=0x55827784c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x55827784c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 -- 172.21.15.86:0/120619776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582778b6390 con 0x55827782d9d0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b259c640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x55827784c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84b2d9d640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005760 tx=0x7f84a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d565a0e8b0bb29bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f8496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8054040 con 0x55827782d9d0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f84a80035f0 con 0x55827782d9d0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.781+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a8004240 con 0x55827782d9d0 2024-09-05T06:41:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84a8035040 con 0x55827782d9d0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582778b7670 con 0x55827782d9d0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8004240 con 0x55827782d9d0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f84a8060030 con 0x55827782d9d0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 20 client.15240 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778bbbe0 0x5582778ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 4 client.15240 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "status"}) -- 0x55827779bb10 con 0x5582778bbbe0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.782+0000 7f84b259c640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778bbbe0 0x5582778ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84b259c640 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778bbbe0 0x5582778ba8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d69bedd1d3c8a154 server_cookie=306b076143b7bba6 in_seq=0 out_seq=0 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f8496ff5640 10 client.15240 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f8496ff5640 1 -- 172.21.15.86:0/120619776 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55827779bb10 con 0x5582778bbbe0 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f8496ff5640 10 client.15240 handle_command_reply: tid=0 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 client.15240 shutdown 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 msgr2=0x55827784c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005760 tx=0x7f84a8005850 comp rx=0 tx=0).stop 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 shutdown_connections 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5582778bbbe0 0x5582778ba8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558277838bb0 0x5582778b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55827782e2f0 0x55827784c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 --2- 172.21.15.86:0/120619776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55827782d9d0 0x55827784c420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f8496ff5640 0 client.15240 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 >> 172.21.15.86:0/120619776 conn(0x558277829f60 msgr2=0x558277833fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 shutdown_connections 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 1 -- 172.21.15.86:0/120619776 wait complete. 2024-09-05T06:41:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.783+0000 7f84bb59f740 20 client.15240 trim_cache size 0 max 16384 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24461, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 94.917126733000003, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:41:05.800 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 151.63554752900001 2024-09-05T06:41:05.801 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:05.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 msgr2=0x7f849c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 0x7f849c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84ac013360 tx=0x7f84ac0046c0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 msgr2=0x7f84b4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84a400cce0 tx=0x7f84a400a620 comp rx=0 tx=0).stop 2024-09-05T06:41:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 shutdown_connections 2024-09-05T06:41:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b4106a80 0x7f84b419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84b4101420 0x7f84b4198910 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 --2- 172.21.15.86:0/1893622747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b4100a80 0x7f84b41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 >> 172.21.15.86:0/1893622747 conn(0x7f84b40fc7a0 msgr2=0x7f84b41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 shutdown_connections 2024-09-05T06:41:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:05.784+0000 7f84b91c6640 1 -- 172.21.15.86:0/1893622747 wait complete. 2024-09-05T06:41:06.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:06 smithi086 ceph-mon[30943]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:41:06.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T06:41:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:06 smithi115 ceph-mon[34791]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:41:06.765 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:06 smithi096 ceph-mon[34468]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-05T06:41:07.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 -- 172.21.15.86:0/1513723009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 msgr2=0x7fce5410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/1513723009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5410e610 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009ca0 tx=0x7fce4c02a950 comp rx=0 tx=0).stop 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 -- 172.21.15.86:0/1513723009 shutdown_connections 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/1513723009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5410e610 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/1513723009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 0x7fce5410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/1513723009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce541031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 -- 172.21.15.86:0/1513723009 >> 172.21.15.86:0/1513723009 conn(0x7fce540fc7a0 msgr2=0x7fce540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 -- 172.21.15.86:0/1513723009 shutdown_connections 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.258+0000 7fce5b5d3640 1 -- 172.21.15.86:0/1513723009 wait complete. 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 Processor -- start 2024-09-05T06:41:07.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 -- start start 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce54198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 0x7fce541989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111e60 con 0x7fce5410c1a0 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111ce0 con 0x7fce54103720 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5b5d3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111fe0 con 0x7fce54102e00 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42488/0 (socket says 172.21.15.86:42488) 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 -- 172.21.15.86:0/761964791 learned_addr learned my addr 172.21.15.86:0/761964791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce58b6e640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 0x7fce541989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5936f640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce54198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 -- 172.21.15.86:0/761964791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 msgr2=0x7fce54198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce54198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 -- 172.21.15.86:0/761964791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 msgr2=0x7fce541989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 0x7fce541989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce59b70640 1 -- 172.21.15.86:0/761964791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5419d440 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.259+0000 7fce5936f640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce54198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce59b70640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce4c006010 tx=0x7fce4c02f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce4a7fc640 1 -- 172.21.15.86:0/761964791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c034040 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce4a7fc640 1 -- 172.21.15.86:0/761964791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce4c003e00 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce4a7fc640 1 -- 172.21.15.86:0/761964791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c045480 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5419d6d0 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.260+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce541a5010 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce4a7fc640 1 -- 172.21.15.86:0/761964791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce4c037020 con 0x7fce5410c1a0 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 Processor -- start 2024-09-05T06:41:07.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 -- start start 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8322f0 0x55d5dc83c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc83efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc798090 con 0x55d5dc8319d0 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc73fa70 con 0x55d5dc8322f0 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc794050 con 0x55d5dc83cbb0 2024-09-05T06:41:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42500/0 (socket says 172.21.15.86:42500) 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 -- 172.21.15.86:0/2423648249 learned_addr learned my addr 172.21.15.86:0/2423648249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce5936f640 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8322f0 0x55d5dc83c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce4a7fc640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 0x7fce400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 -- 172.21.15.86:0/2423648249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 msgr2=0x55d5dc83efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc83efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 -- 172.21.15.86:0/2423648249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8322f0 msgr2=0x55d5dc83c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8322f0 0x55d5dc83c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce58b6e640 1 -- 172.21.15.86:0/2423648249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5dc74c130 con 0x55d5dc8319d0 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.263+0000 7fce59b70640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 0x7fce400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.264+0000 7fce58b6e640 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce4400ac70 tx=0x7fce440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f27b6df5796cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.264+0000 7fce4a7fc640 1 -- 172.21.15.86:0/761964791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fce4c0f05c0 con 0x7fce5410c1a0 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.264+0000 7fce3f7fe640 1 -- 172.21.15.86:0/2423648249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44005020 con 0x55d5dc8319d0 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.264+0000 7fce3f7fe640 1 -- 172.21.15.86:0/2423648249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce440047a0 con 0x55d5dc8319d0 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.264+0000 7fce3f7fe640 1 -- 172.21.15.86:0/2423648249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440126b0 con 0x55d5dc8319d0 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- 172.21.15.86:0/2423648249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 msgr2=0x55d5dc831db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce4400ac70 tx=0x7fce440117b0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce59b70640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 0x7fce400a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0bc790 tx=0x7fce4c0a3be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- 172.21.15.86:0/2423648249 shutdown_connections 2024-09-05T06:41:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc83efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8322f0 0x55d5dc83c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/2423648249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc831db0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- 172.21.15.86:0/2423648249 >> 172.21.15.86:0/2423648249 conn(0x55d5dc82df60 msgr2=0x55d5dc82e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- 172.21.15.86:0/2423648249 shutdown_connections 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- 172.21.15.86:0/2423648249 wait complete. 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- start start 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 Processor -- start 2024-09-05T06:41:07.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8319d0 0x55d5dc83c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc839ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc83f780 con 0x55d5dc8322f0 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc752f20 con 0x55d5dc8319d0 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.265+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc83f900 con 0x55d5dc83cbb0 2024-09-05T06:41:07.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5d9ac740 10 client.0 _ll_register_callbacks cb 0x55d5dc71c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:07.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42506/0 (socket says 172.21.15.86:42506) 2024-09-05T06:41:07.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 -- 172.21.15.86:0/1731424060 learned_addr learned my addr 172.21.15.86:0/1731424060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:07.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8319d0 0x55d5dc83c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc839ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 -- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 msgr2=0x55d5dc839ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc839ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 -- 172.21.15.86:0/1731424060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8319d0 msgr2=0x55d5dc83c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8319d0 0x55d5dc83c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 -- 172.21.15.86:0/1731424060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5dc5092b0 con 0x55d5dc8322f0 2024-09-05T06:41:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc839ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5936f640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce5000f530 tx=0x7fce5000f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46983fa45094a06a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50013850 con 0x55d5dc8322f0 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fce500139f0 con 0x55d5dc8322f0 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce440047a0 con 0x55d5dc8322f0 2024-09-05T06:41:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce50024050 con 0x55d5dc8322f0 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5d9ac740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.266+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5dc84b100 con 0x55d5dc8322f0 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce440047a0 con 0x55d5dc8322f0 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fce500113e0 con 0x55d5dc8322f0 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:41:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 20 client.15252 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc84d750 0x55d5dc84ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 4 client.15252 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5dc7a5600 con 0x55d5dc84d750 2024-09-05T06:41:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.267+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc84d750 0x55d5dc84ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce3cff9640 10 client.15252 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc84d750 0x55d5dc84ebc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c3faf1c0121ef2 server_cookie=906271719c843ef4 in_seq=0 out_seq=0 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce3cff9640 1 -- 172.21.15.86:0/1731424060 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5dc7a5600 con 0x55d5dc84d750 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce3cff9640 10 client.15252 handle_command_reply: tid=0 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce5d9ac740 1 client.15252 shutdown 2024-09-05T06:41:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 msgr2=0x55d5dc8397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.268+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce5000f530 tx=0x7fce5000f9f0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 shutdown_connections 2024-09-05T06:41:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc84d750 0x55d5dc84ebc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc83cbb0 0x55d5dc839ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc8397a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1731424060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc8319d0 0x55d5dc83c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce3cff9640 0 client.15252 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 >> 172.21.15.86:0/1731424060 conn(0x55d5dc82df60 msgr2=0x55d5dc833c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 shutdown_connections 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1731424060 wait complete. 2024-09-05T06:41:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.269+0000 7fce5d9ac740 20 client.15252 trim_cache size 0 max 16384 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 Processor -- start 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 -- start start 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 0x55d5dc84e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc8573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84cf00 con 0x55d5dc8322f0 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84d5d0 con 0x55d5dc83cbb0 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84dca0 con 0x55d5dc8319d0 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42522/0 (socket says 172.21.15.86:42522) 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 -- 172.21.15.86:0/1910537514 learned_addr learned my addr 172.21.15.86:0/1910537514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc8573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 0x55d5dc84e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 -- 172.21.15.86:0/1910537514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 msgr2=0x55d5dc84e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 0x55d5dc84e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 -- 172.21.15.86:0/1910537514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 msgr2=0x55d5dc8573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc8573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce5936f640 1 -- 172.21.15.86:0/1910537514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5dc82dd90 con 0x55d5dc8322f0 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.287+0000 7fce58b6e640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 0x55d5dc84e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5936f640 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fce50011970 tx=0x7fce5000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827e2b64c8af49f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce3effd640 1 -- 172.21.15.86:0/1910537514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5002e030 con 0x55d5dc8322f0 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce3effd640 1 -- 172.21.15.86:0/1910537514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce500113e0 con 0x55d5dc8322f0 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce3effd640 1 -- 172.21.15.86:0/1910537514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50013940 con 0x55d5dc8322f0 2024-09-05T06:41:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1910537514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 msgr2=0x55d5dc856ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fce50011970 tx=0x7fce5000aed0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1910537514 shutdown_connections 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc8573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8322f0 0x55d5dc856ea0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/1910537514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8319d0 0x55d5dc84e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.288+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1910537514 >> 172.21.15.86:0/1910537514 conn(0x55d5dc82df60 msgr2=0x55d5dc838890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1910537514 shutdown_connections 2024-09-05T06:41:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- 172.21.15.86:0/1910537514 wait complete. 2024-09-05T06:41:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- start start 2024-09-05T06:41:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 Processor -- start 2024-09-05T06:41:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8322f0 0x55d5dc852750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc852c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84ea40 con 0x55d5dc8319d0 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84e370 con 0x55d5dc83cbb0 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5dc84f110 con 0x55d5dc8322f0 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce5d9ac740 10 client.0 _ll_register_callbacks cb 0x7fce5000a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce58b6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce58b6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42538/0 (socket says 172.21.15.86:42538) 2024-09-05T06:41:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.289+0000 7fce58b6e640 1 -- 172.21.15.86:0/3264195042 learned_addr learned my addr 172.21.15.86:0/3264195042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce5936f640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8322f0 0x55d5dc852750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc852c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 -- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8322f0 msgr2=0x55d5dc852750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8322f0 0x55d5dc852750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 -- 172.21.15.86:0/3264195042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 msgr2=0x55d5dc852c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc852c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 -- 172.21.15.86:0/3264195042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5dc8532d0 con 0x55d5dc8319d0 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce58b6e640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fce44004f30 tx=0x7fce5000ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef16aee0380ef421 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44024040 con 0x55d5dc8319d0 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7fce440110c0 con 0x55d5dc8319d0 2024-09-05T06:41:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce500113e0 con 0x55d5dc8319d0 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce4403b050 con 0x55d5dc8319d0 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce5d9ac740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5dc8b7c20 con 0x55d5dc8319d0 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.290+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce500113e0 con 0x55d5dc8319d0 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fce44034060 con 0x55d5dc8319d0 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 20 client.15258 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc8bc670 0x55d5dc8bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 4 client.15258 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "status"}) -- 0x55d5dc79fb10 con 0x55d5dc8bc670 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5936f640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc8bc670 0x55d5dc8bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce5936f640 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc8bc670 0x55d5dc8bb380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e48b0facb93b89ed server_cookie=969a15f461c8dc4 in_seq=0 out_seq=0 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.291+0000 7fce3cff9640 10 client.15258 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce3cff9640 1 -- 172.21.15.86:0/3264195042 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d5dc79fb10 con 0x55d5dc8bc670 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce3cff9640 10 client.15258 handle_command_reply: tid=0 2024-09-05T06:41:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 client.15258 shutdown 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 msgr2=0x55d5dc857420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fce44004f30 tx=0x7fce5000ac30 comp rx=0 tx=0).stop 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 shutdown_connections 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d5dc8bc670 0x55d5dc8bb380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d5dc83cbb0 0x55d5dc852c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce3cff9640 0 client.15258 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d5dc8322f0 0x55d5dc852750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 --2- 172.21.15.86:0/3264195042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5dc8319d0 0x55d5dc857420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 >> 172.21.15.86:0/3264195042 conn(0x55d5dc82df60 msgr2=0x55d5dc847b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 shutdown_connections 2024-09-05T06:41:07.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 1 -- 172.21.15.86:0/3264195042 wait complete. 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5d9ac740 20 client.15258 trim_cache size 0 max 16384 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24466, 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:07.327 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 97.432540536999994, 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 150.58154978900001 2024-09-05T06:41:07.328 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.292+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 msgr2=0x7fce400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 0x7fce400a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0bc790 tx=0x7fce4c0a3be0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 msgr2=0x7fce5419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce4c006010 tx=0x7fce4c02f640 comp rx=0 tx=0).stop 2024-09-05T06:41:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 shutdown_connections 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5410c1a0 0x7fce5419cd60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7fce400a2c10 0x7fce400a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54103720 0x7fce541989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 --2- 172.21.15.86:0/761964791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce54102e00 0x7fce54198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 >> 172.21.15.86:0/761964791 conn(0x7fce540fc7a0 msgr2=0x7fce540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 shutdown_connections 2024-09-05T06:41:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:07.293+0000 7fce5b5d3640 1 -- 172.21.15.86:0/761964791 wait complete. 2024-09-05T06:41:08.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T06:41:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:08.679 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:08 smithi086 ceph-mon[30943]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:41:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:08 smithi115 ceph-mon[34791]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:41:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.758+0000 7f6081fa0640 1 -- 172.21.15.86:0/415369925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c101270 msgr2=0x7f607c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.758+0000 7f6081fa0640 1 --2- 172.21.15.86:0/415369925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c101270 0x7f607c10d7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6074009cd0 tx=0x7f607402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- 172.21.15.86:0/415369925 shutdown_connections 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- 172.21.15.86:0/415369925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c101270 0x7f607c10d7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- 172.21.15.86:0/415369925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 0x7f607c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- 172.21.15.86:0/415369925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c1068d0 0x7f607c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- 172.21.15.86:0/415369925 >> 172.21.15.86:0/415369925 conn(0x7f607c0fc610 msgr2=0x7f607c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- 172.21.15.86:0/415369925 shutdown_connections 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- 172.21.15.86:0/415369925 wait complete. 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 Processor -- start 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- start start 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 0x7f607c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 0x7f607c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f607c110b80 con 0x7f607c1068d0 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f607c110a00 con 0x7f607c101270 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.759+0000 7f6081fa0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f607c110d00 con 0x7f607c1008d0 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42544/0 (socket says 172.21.15.86:42544) 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 -- 172.21.15.86:0/1022459607 learned_addr learned my addr 172.21.15.86:0/1022459607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6080f9e640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 0x7f607c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6073fff640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 0x7f607c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 -- 172.21.15.86:0/1022459607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 msgr2=0x7f607c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 0x7f607c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 -- 172.21.15.86:0/1022459607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 msgr2=0x7f607c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 0x7f607c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 -- 172.21.15.86:0/1022459607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c19d140 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6073fff640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 0x7f607c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f608179f640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6074005cc0 tx=0x7f6074003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6071ffb640 1 -- 172.21.15.86:0/1022459607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074004450 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6071ffb640 1 -- 172.21.15.86:0/1022459607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6074031c90 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c19d3d0 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.760+0000 7f6071ffb640 1 -- 172.21.15.86:0/1022459607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607403b400 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.761+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1a4d10 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f6071ffb640 1 -- 172.21.15.86:0/1022459607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6074004680 con 0x7f607c1068d0 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 Processor -- start 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 -- start start 2024-09-05T06:41:08.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987abbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987acac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e987a22090 con 0x55e987acb1c0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9879c9a70 con 0x55e987abc3d0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f60855db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e987a1e050 con 0x55e987abbab0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f6071ffb640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 0x7f60640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42548/0 (socket says 172.21.15.86:42548) 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.763+0000 7f6073fff640 1 -- 172.21.15.86:0/68712246 learned_addr learned my addr 172.21.15.86:0/68712246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f608179f640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 0x7f60640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987abbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6080f9e640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987acac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 -- 172.21.15.86:0/68712246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 msgr2=0x55e987abbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987abbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 -- 172.21.15.86:0/68712246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 msgr2=0x55e987acac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987acac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 -- 172.21.15.86:0/68712246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9879d6130 con 0x55e987acb1c0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6080f9e640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987acac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6071ffb640 1 -- 172.21.15.86:0/1022459607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f607402e030 con 0x7f607c1068d0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6073fff640 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f606c013550 tx=0x7f606c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6dff29f75f0066 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f608179f640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 0x7f60640a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60740c5220 tx=0x7f60740de040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6062ffd640 1 -- 172.21.15.86:0/68712246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c019070 con 0x55e987acb1c0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6062ffd640 1 -- 172.21.15.86:0/68712246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f606c0054d0 con 0x55e987acb1c0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.764+0000 7f6062ffd640 1 -- 172.21.15.86:0/68712246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c004c70 con 0x55e987acb1c0 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- 172.21.15.86:0/68712246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 msgr2=0x55e987acd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f606c013550 tx=0x7f606c00b6b0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- 172.21.15.86:0/68712246 shutdown_connections 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987acd5b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987acac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 --2- 172.21.15.86:0/68712246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987abbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- 172.21.15.86:0/68712246 >> 172.21.15.86:0/68712246 conn(0x55e987ab8160 msgr2=0x55e987ab8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- 172.21.15.86:0/68712246 shutdown_connections 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- 172.21.15.86:0/68712246 wait complete. 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 -- start start 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.765+0000 7f60855db740 1 Processor -- start 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ac7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987ac8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e987acdd90 con 0x55e987abbab0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9879dcf20 con 0x55e987abc3d0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e987acdf10 con 0x55e987acb1c0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f60855db740 10 client.0 _ll_register_callbacks cb 0x55e9879a6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42554/0 (socket says 172.21.15.86:42554) 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 -- 172.21.15.86:0/523862431 learned_addr learned my addr 172.21.15.86:0/523862431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6080f9e640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ac7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987ac8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 -- 172.21.15.86:0/523862431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 msgr2=0x55e987ac8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987ac8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 -- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 msgr2=0x55e987ac7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ac7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 -- 172.21.15.86:0/523862431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9877932b0 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f6073fff640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b680 tx=0x7f606c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f361efcfaed30 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f605bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.766+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c019040 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f606c0035f0 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c0054d0 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f606c033050 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e987ad9770 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f606c0054d0 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f606c004dc0 con 0x55e987abbab0 2024-09-05T06:41:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 20 client.15270 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987adb9f0 0x55e987adce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 4 client.15270 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e987a2f600 con 0x55e987adb9f0 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.767+0000 7f6080f9e640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987adb9f0 0x55e987adce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.768+0000 7f605bfff640 10 client.15270 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.768+0000 7f6080f9e640 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987adb9f0 0x55e987adce60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c29847ca2e3954 server_cookie=dffc92bf7e1de019 in_seq=0 out_seq=0 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f605bfff640 1 -- 172.21.15.86:0/523862431 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e987a2f600 con 0x55e987adb9f0 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f605bfff640 10 client.15270 handle_command_reply: tid=0 2024-09-05T06:41:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 client.15270 shutdown 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 msgr2=0x55e987acaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b680 tx=0x7f606c0044a0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 shutdown_connections 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987adb9f0 0x55e987adce60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987ac8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ac7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 --2- 172.21.15.86:0/523862431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abbab0 0x55e987acaa90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f605bfff640 0 client.15270 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 >> 172.21.15.86:0/523862431 conn(0x55e987ab8160 msgr2=0x55e987abdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 shutdown_connections 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 1 -- 172.21.15.86:0/523862431 wait complete. 2024-09-05T06:41:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.769+0000 7f60855db740 20 client.15270 trim_cache size 0 max 16384 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 Processor -- start 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 -- start start 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abbab0 0x55e987adce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987adb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e987adb1a0 con 0x55e987abc3d0 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e987adb870 con 0x55e987abbab0 2024-09-05T06:41:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f60855db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e987adbf40 con 0x55e987acb1c0 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42564/0 (socket says 172.21.15.86:42564) 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 -- 172.21.15.86:0/2493616070 learned_addr learned my addr 172.21.15.86:0/2493616070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 -- 172.21.15.86:0/2493616070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 msgr2=0x55e987adb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987adb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 -- 172.21.15.86:0/2493616070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abbab0 msgr2=0x55e987adce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abbab0 0x55e987adce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.787+0000 7f6080f9e640 1 -- 172.21.15.86:0/2493616070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e987acae00 con 0x55e987abc3d0 2024-09-05T06:41:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f6080f9e640 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e987ac7670 tx=0x7f607800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c90b64a4f46bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60627fc640 1 -- 172.21.15.86:0/2493616070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078003d20 con 0x55e987abc3d0 2024-09-05T06:41:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60627fc640 1 -- 172.21.15.86:0/2493616070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60780054d0 con 0x55e987abc3d0 2024-09-05T06:41:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60627fc640 1 -- 172.21.15.86:0/2493616070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078004e60 con 0x55e987abc3d0 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 -- 172.21.15.86:0/2493616070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 msgr2=0x55e987adc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55e987ac7670 tx=0x7f607800bc10 comp rx=0 tx=0).stop 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 -- 172.21.15.86:0/2493616070 shutdown_connections 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987acb1c0 0x55e987adb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987abc3d0 0x55e987adc0c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 --2- 172.21.15.86:0/2493616070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abbab0 0x55e987adce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.788+0000 7f60855db740 1 -- 172.21.15.86:0/2493616070 >> 172.21.15.86:0/2493616070 conn(0x55e987ab8160 msgr2=0x55e987ab8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- 172.21.15.86:0/2493616070 shutdown_connections 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- 172.21.15.86:0/2493616070 wait complete. 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- start start 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 Processor -- start 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987ade060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ade5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e987adcce0 con 0x55e987acb1c0 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e987adc610 con 0x55e987abc3d0 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e987add3b0 con 0x55e987abbab0 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f60855db740 10 client.0 _ll_register_callbacks cb 0x7f606c013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f6073fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42574/0 (socket says 172.21.15.86:42574) 2024-09-05T06:41:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.789+0000 7f6073fff640 1 -- 172.21.15.86:0/3584113051 learned_addr learned my addr 172.21.15.86:0/3584113051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6080f9e640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ade5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987ade060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 -- 172.21.15.86:0/3584113051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 msgr2=0x55e987ade060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987ade060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 -- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 msgr2=0x55e987ade5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ade5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 -- 172.21.15.86:0/3584113051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e987b44250 con 0x55e987acb1c0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6080f9e640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ade5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f6073fff640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f606c013550 tx=0x7f606c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51063369181af10f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f605bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c019070 con 0x55e987acb1c0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f606c0118c0 con 0x55e987acb1c0 2024-09-05T06:41:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60780054d0 con 0x55e987acb1c0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f606c044050 con 0x55e987acb1c0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f60855db740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e987b45550 con 0x55e987acb1c0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.790+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60780054d0 con 0x55e987acb1c0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f606c003cc0 con 0x55e987acb1c0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 20 client.15276 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987b4a0a0 0x55e987b48db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 4 client.15276 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "status"}) -- 0x55e987a29b10 con 0x55e987b4a0a0 2024-09-05T06:41:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f6080f9e640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987b4a0a0 0x55e987b48db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f605bfff640 10 client.15276 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.791+0000 7f6080f9e640 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987b4a0a0 0x55e987b48db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907520a5b191fcd server_cookie=d36ac62f404b0195 in_seq=0 out_seq=0 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f605bfff640 1 -- 172.21.15.86:0/3584113051 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e987a29b10 con 0x55e987b4a0a0 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f605bfff640 10 client.15276 handle_command_reply: tid=0 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 client.15276 shutdown 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 msgr2=0x55e987adeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f606c013550 tx=0x7f606c00b6b0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 shutdown_connections 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e987b4a0a0 0x55e987b48db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e987acb1c0 0x55e987adeae0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e987abc3d0 0x55e987ade5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f605bfff640 0 client.15276 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 --2- 172.21.15.86:0/3584113051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e987abbab0 0x55e987ade060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 >> 172.21.15.86:0/3584113051 conn(0x55e987ab8160 msgr2=0x55e987ab2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 shutdown_connections 2024-09-05T06:41:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 1 -- 172.21.15.86:0/3584113051 wait complete. 2024-09-05T06:41:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.792+0000 7f60855db740 20 client.15276 trim_cache size 0 max 16384 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24479, 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 132.42582221699999, 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 143.415725716 2024-09-05T06:41:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 msgr2=0x7f60640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 0x7f60640a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60740c5220 tx=0x7f60740de040 comp rx=0 tx=0).stop 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 msgr2=0x7f607c19ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6074005cc0 tx=0x7f6074003c20 comp rx=0 tx=0).stop 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 shutdown_connections 2024-09-05T06:41:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c1068d0 0x7f607c19ca60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f60640a2c10 0x7f60640a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f607c101270 0x7f607c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 --2- 172.21.15.86:0/1022459607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f607c1008d0 0x7f607c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 >> 172.21.15.86:0/1022459607 conn(0x7f607c0fc610 msgr2=0x7f607c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 shutdown_connections 2024-09-05T06:41:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:08.793+0000 7f6081fa0640 1 -- 172.21.15.86:0/1022459607 wait complete. 2024-09-05T06:41:08.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:08 smithi096 ceph-mon[34468]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T06:41:09.511 INFO:teuthology.run_tasks:Running task sequential... 2024-09-05T06:41:09.521 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-05T06:41:09.521 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-05T06:41:09.521 INFO:teuthology.task.print:Enabling mgr modules 2024-09-05T06:41:09.521 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-05T06:41:09.522 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T06:41:09.522 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:41:09.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/1888501138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 msgr2=0x7f1a9c100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/1888501138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c100ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009900 tx=0x7f1a9002a410 comp rx=0 tx=0).stop 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/1888501138 shutdown_connections 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/1888501138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c101530 0x7f1a9c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/1888501138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c100ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/1888501138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c0ffe10 0x7f1a9c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/1888501138 >> 172.21.15.86:0/1888501138 conn(0x7f1a9c0fb970 msgr2=0x7f1a9c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/1888501138 shutdown_connections 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/1888501138 wait complete. 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 Processor -- start 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- start start 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ffe10 0x7f1a9c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c101530 0x7f1a9c1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113cc0 con 0x7f1a9c100b70 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113b40 con 0x7f1a9c101530 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.624+0000 7f1aa3f3b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113e40 con 0x7f1a9c0ffe10 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42600/0 (socket says 172.21.15.86:42600) 2024-09-05T06:41:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 -- 172.21.15.86:0/3896234087 learned_addr learned my addr 172.21.15.86:0/3896234087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa1cd7640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ffe10 0x7f1a9c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa24d8640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c101530 0x7f1a9c1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 -- 172.21.15.86:0/3896234087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ffe10 msgr2=0x7f1a9c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ffe10 0x7f1a9c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 -- 172.21.15.86:0/3896234087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c101530 msgr2=0x7f1a9c1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c101530 0x7f1a9c1a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 -- 172.21.15.86:0/3896234087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c1a5ea0 con 0x7f1a9c100b70 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa14d6640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009230 tx=0x7f1a90004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9003f030 con 0x7f1a9c100b70 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a90003cb0 con 0x7f1a9c100b70 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c1a6130 con 0x7f1a9c100b70 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90035070 con 0x7f1a9c100b70 2024-09-05T06:41:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.625+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c1a6620 con 0x7f1a9c100b70 2024-09-05T06:41:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.626+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9c1a6940 con 0x7f1a9c100b70 2024-09-05T06:41:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.627+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a90042030 con 0x7f1a9c100b70 2024-09-05T06:41:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.628+0000 7f1a9affd640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 0x7f1a840a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.628+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f1a900f0050 con 0x7f1a9c100b70 2024-09-05T06:41:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.628+0000 7f1aa1cd7640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 0x7f1a840a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:09.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.628+0000 7f1aa1cd7640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 0x7f1a840a5190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c007ec0 tx=0x7f1a8c0075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:09.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.631+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a900bca80 con 0x7f1a9c100b70 2024-09-05T06:41:09.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:09.845+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1a9c1a6b10 con 0x7f1a9c100b70 2024-09-05T06:41:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.337+0000 7f1a9affd640 1 -- 172.21.15.86:0/3896234087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f1a900bc380 con 0x7f1a9c100b70 2024-09-05T06:41:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 msgr2=0x7f1a840a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 0x7f1a840a5190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c007ec0 tx=0x7f1a8c0075b0 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 msgr2=0x7f1a9c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009230 tx=0x7f1a90004340 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 shutdown_connections 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f1a840a2cd0 0x7f1a840a5190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a9c101530 0x7f1a9c1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a9c100b70 0x7f1a9c19e7c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 --2- 172.21.15.86:0/3896234087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ffe10 0x7f1a9c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 >> 172.21.15.86:0/3896234087 conn(0x7f1a9c0fb970 msgr2=0x7f1a9c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 shutdown_connections 2024-09-05T06:41:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:10.339+0000 7f1aa3f3b640 1 -- 172.21.15.86:0/3896234087 wait complete. 2024-09-05T06:41:10.356 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-05T06:41:10.368 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-05T06:41:10.368 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-05T06:41:10.368 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-05T06:41:10.369 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T06:41:10.369 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:10.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:10.369 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-09-05T06:41:10.388 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-05T06:41:10.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:10.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T06:41:10.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:10.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-09-05T06:41:10.575 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-05T06:41:10.575 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-05T06:41:10.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T06:41:10.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-mon[30943]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:41:10.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T06:41:10.585 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-05T06:41:10.585 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-05T06:41:10.585 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:10.493+0000 7f90a0793040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:41:10.586 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:10.555+0000 7f90a0793040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-mon[34791]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:41:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T06:41:10.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: ignoring --setuser ceph since I am not root 2024-09-05T06:41:10.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: ignoring --setgroup ceph since I am not root 2024-09-05T06:41:10.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-mgr[36601]: -- 172.21.15.115:0/733228351 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561297d0b0e0 con 0x561297ce9000 2024-09-05T06:41:10.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:10.492+0000 7f8f59f17040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:41:10.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:10.549+0000 7f8f59f17040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-mon[34468]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: ignoring --setuser ceph since I am not root 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: ignoring --setgroup ceph since I am not root 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-mgr[35833]: -- 172.21.15.96:0/134588748 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a28ce1f0e0 con 0x55a28cdfc800 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:10.490+0000 7fd47d873040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:10.547+0000 7fd47d873040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:10.697 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:10.696+0000 7fd47d873040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:41:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:10.942 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:10.711+0000 7f90a0793040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:41:10.943 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:10.784+0000 7f90a0793040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:41:11.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:10.762+0000 7fd47d873040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:41:11.050 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:10.909+0000 7fd47d873040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:11.055 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:10.697+0000 7f8f59f17040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T06:41:11.055 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:10.763+0000 7f8f59f17040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T06:41:11.055 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:10.913+0000 7f8f59f17040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:11.055 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:11.055+0000 7f8f59f17040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:41:11.236 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:10.942+0000 7f90a0793040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:11.237 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:11.089+0000 7f90a0793040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 -- 172.21.15.86:0/188462986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec102d90 msgr2=0x7f0aec103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 --2- 172.21.15.86:0/188462986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec102d90 0x7f0aec103170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009930 tx=0x7f0adc02a420 comp rx=0 tx=0).stop 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 -- 172.21.15.86:0/188462986 shutdown_connections 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 --2- 172.21.15.86:0/188462986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec108290 0x7f0aec10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 --2- 172.21.15.86:0/188462986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec103740 0x7f0aec107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 --2- 172.21.15.86:0/188462986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec102d90 0x7f0aec103170 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 -- 172.21.15.86:0/188462986 >> 172.21.15.86:0/188462986 conn(0x7f0aec0fc7d0 msgr2=0x7f0aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 -- 172.21.15.86:0/188462986 shutdown_connections 2024-09-05T06:41:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 -- 172.21.15.86:0/188462986 wait complete. 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.342+0000 7f0af1431640 1 Processor -- start 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 -- start start 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 0x7f0aec1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec108290 0x7f0aec19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec111f60 con 0x7f0aec103740 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec111de0 con 0x7f0aec102d90 2024-09-05T06:41:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0af1431640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec1120e0 con 0x7f0aec108290 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aeaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 0x7f0aec1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42644/0 (socket says 172.21.15.86:42644) 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 -- 172.21.15.86:0/1508209593 learned_addr learned my addr 172.21.15.86:0/1508209593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aeb7fe640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec108290 0x7f0aec19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec108290 msgr2=0x7f0aec19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec108290 0x7f0aec19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 msgr2=0x7f0aec1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 0x7f0aec1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aea7fc640 1 -- 172.21.15.86:0/1508209593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec19d480 con 0x7f0aec103740 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.343+0000 7f0aeaffd640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 0x7f0aec1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0aea7fc640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005fd0 tx=0x7f0ae000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x7f0aec103740 2024-09-05T06:41:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec19d770 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ae0009c30 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a5050 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.344+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0003d40 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.345+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8005290 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.346+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137901+0+0 (secure 0 0 0) 0x7f0ae0003ee0 con 0x7f0aec103740 2024-09-05T06:41:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.347+0000 7f0ad3fff640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 0x7f0ad40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.347+0000 7f0aeaffd640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 msgr2=0x7f0ad40a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3708924769 2024-09-05T06:41:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.347+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9216+0+0 (secure 0 0 0) 0x7f0ae00cf070 con 0x7f0aec103740 2024-09-05T06:41:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.347+0000 7f0aeaffd640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 0x7f0ad40a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T06:41:11.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.351+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ae0020030 con 0x7f0aec103740 2024-09-05T06:41:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.548+0000 7f0aeaffd640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 msgr2=0x7f0ad40a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3708924769 2024-09-05T06:41:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.548+0000 7f0aeaffd640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 0x7f0ad40a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T06:41:11.548 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:11.418+0000 7f90a0793040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:41:11.548 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:11.480+0000 7f90a0793040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:41:11.548 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T06:41:11.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-mon[30943]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-05T06:41:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.564+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0ad8005850 con 0x7f0aec103740 2024-09-05T06:41:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.564+0000 7f0ad3fff640 1 -- 172.21.15.86:0/1508209593 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f0ae009baa0 con 0x7f0aec103740 2024-09-05T06:41:11.564 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 msgr2=0x7f0ad40a51a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 0x7f0ad40a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 msgr2=0x7f0aec198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005fd0 tx=0x7f0ae000dde0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 shutdown_connections 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aec108290 0x7f0aec19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec103740 0x7f0aec198a10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.86:6800/3708924769,v1:172.21.15.86:6801/3708924769] conn(0x7f0ad40a2ce0 0x7f0ad40a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 --2- 172.21.15.86:0/1508209593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aec102d90 0x7f0aec1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 >> 172.21.15.86:0/1508209593 conn(0x7f0aec0fc7d0 msgr2=0x7f0aec105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 shutdown_connections 2024-09-05T06:41:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:11.567+0000 7f0af1431640 1 -- 172.21.15.86:0/1508209593 wait complete. 2024-09-05T06:41:11.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T06:41:11.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-mon[34468]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-05T06:41:11.612 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:11.049+0000 7fd47d873040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T06:41:11.612 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:11.364+0000 7fd47d873040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:41:11.613 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:11.428+0000 7fd47d873040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:41:11.613 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:11.612+0000 7fd47d873040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:41:11.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3896234087' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T06:41:11.616 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-mon[34791]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-05T06:41:11.616 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:11.368+0000 7f8f59f17040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T06:41:11.616 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:11.432+0000 7f8f59f17040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T06:41:11.895 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:11.679+0000 7f90a0793040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:41:11.932 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:11.616+0000 7f8f59f17040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T06:41:12.234 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-05T06:41:12.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:12.234 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-09-05T06:41:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-09-05T06:41:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link noprefixroute 2024-09-05T06:41:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-09-05T06:41:12.292 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T06:41:12.292 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:41:12.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:41:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:12.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:12.434 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T06:41:12.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:12 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1508209593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:41:12.458 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T06:41:12.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:12.460 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-09-05T06:41:12.516 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-09-05T06:41:12.517 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T06:41:12.517 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T06:41:12.517 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T06:41:12.517 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T06:41:12.517 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:41:12.517 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:41:12.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T06:41:12.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T06:41:12.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T06:41:12.518 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:41:12.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:41:12.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:12.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:12.647 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-09-05T06:41:12.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:12.662 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-09-05T06:41:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:12 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1508209593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:41:12.682 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:12.495+0000 7f8f59f17040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:41:12.682 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:12.561+0000 7f8f59f17040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:41:12.718 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:41:12.718 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:41:12.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:12.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-05T06:41:12.718 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:41:12.736 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:12.595+0000 7f90a0793040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:41:12.736 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:12.668+0000 7f90a0793040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:41:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:41:12.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:12 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1508209593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T06:41:12.806 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:12.503+0000 7fd47d873040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T06:41:12.807 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:12.569+0000 7fd47d873040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T06:41:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:12.814 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-05T06:41:12.814 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-05T06:41:12.815 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:41:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:41:12.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:12.930 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T06:41:12.930 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-09-05T06:41:12.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-09-05T06:41:12.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T06:41:12.930 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-09-05T06:41:12.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T06:41:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:13.008 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-05T06:41:13.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T06:41:13.009 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-05T06:41:13.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T06:41:13.024 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:13.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:41:13.146 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:41:13.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:41:13.204 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:41:13.205 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:13.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:13.206 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:41:13.259 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T06:41:13.259 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/f90eafeb-ef3e-47d2-90fd-8c07576c6e3c --client_fs=cephfs) 2024-09-05T06:41:13.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:41:13.326 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:41:13.384 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:41:13.385 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:41:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:13.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:13.387 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:41:13.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-09-05T06:41:13.386+0000 7fd0e751f480 -1 init, newargv = 0x557e7f567a80 newargc=15 2024-09-05T06:41:13.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83137]: starting ceph client 2024-09-05T06:41:13.523 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.267+0000 7f8f59f17040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:41:13.524 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.380+0000 7f8f59f17040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:41:13.538 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.277+0000 7fd47d873040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:41:13.538 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.393+0000 7fd47d873040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:41:13.692 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.437+0000 7f90a0793040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T06:41:13.692 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.551+0000 7f90a0793040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T06:41:13.799 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.523+0000 7f8f59f17040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:13.799 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.584+0000 7f8f59f17040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:41:13.799 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.732+0000 7f8f59f17040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:41:13.799 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.799+0000 7f8f59f17040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:41:13.806 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.538+0000 7fd47d873040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:13.806 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.599+0000 7fd47d873040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:41:13.806 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.746+0000 7fd47d873040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:41:13.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.692+0000 7f90a0793040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T06:41:13.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.753+0000 7f90a0793040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T06:41:13.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.901+0000 7f90a0793040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T06:41:13.969 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:13.968+0000 7f90a0793040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:41:14.081 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:13.859+0000 7f8f59f17040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:41:14.081 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.017+0000 7f8f59f17040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:41:14.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.814+0000 7fd47d873040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T06:41:14.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:13.874+0000 7fd47d873040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:41:14.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.032+0000 7fd47d873040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:41:14.098 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.097+0000 7fd47d873040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:41:14.236 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.030+0000 7f90a0793040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T06:41:14.237 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.187+0000 7f90a0793040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T06:41:14.346 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.081+0000 7f8f59f17040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:41:14.346 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.147+0000 7f8f59f17040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:41:14.346 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.228+0000 7f8f59f17040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:14.346 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.287+0000 7f8f59f17040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:41:14.346 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:14.346+0000 7f8f59f17040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:41:14.364 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.164+0000 7fd47d873040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:41:14.364 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.245+0000 7fd47d873040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:14.364 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.304+0000 7fd47d873040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:41:14.364 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:14.363+0000 7fd47d873040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:41:14.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:41:14.526 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.253+0000 7f90a0793040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T06:41:14.526 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.320+0000 7f90a0793040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T06:41:14.526 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.404+0000 7f90a0793040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T06:41:14.526 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.464+0000 7f90a0793040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T06:41:14.526 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:14.526+0000 7f90a0793040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T06:41:14.532 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:41:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:41:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:41:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:41:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:41:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:41:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:41:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:41:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:41:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:41:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:41:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:41:14.591 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:41:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:41:14.593 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:41:14.593 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:14.593 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:14.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:14.593 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:41:15.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83137]: starting fuse 2024-09-05T06:41:15.432 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z[36597]: 2024-09-05T06:41:15.132+0000 7f8f59f17040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: Standby manager daemon z restarted 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: Standby manager daemon z started 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: Standby manager daemon y restarted 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: Standby manager daemon y started 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:41:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:15 smithi115 ceph-mon[34791]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.486 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.326+0000 7f90a0793040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: Standby manager daemon z restarted 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: Standby manager daemon z started 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: Standby manager daemon y restarted 2024-09-05T06:41:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: Standby manager daemon y started 2024-09-05T06:41:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:41:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:41:15.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-mon[30943]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.556 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y[35829]: 2024-09-05T06:41:15.155+0000 7fd47d873040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T06:41:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: Standby manager daemon z restarted 2024-09-05T06:41:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: Standby manager daemon z started 2024-09-05T06:41:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.115:0/2385104140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: Standby manager daemon y restarted 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: Standby manager daemon y started 2024-09-05T06:41:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T06:41:15.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T06:41:15.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T06:41:15.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:15 smithi096 ceph-mon[34468]: from='mgr.? 172.21.15.96:0/3737293522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T06:41:15.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-09-05T06:41:15.772 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T06:41:15.828 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:15.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/544e57c75ffdbec4ac2aff8fd19cf9a110cfc5bc4d1e58b4badc09b8e51943a3/merged 2024-09-05T06:41:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/1f36391dc68815b75d61c334168c1327a2dfcd95788dce23ace6901076a0bd84/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0586f1c9a5f1df3b04d3670a6951baae260e71277b1751985c16b9f9aeb468cb/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/22c85881afbfa8502447bf3dcd3c8370b8da7aa8e2f7497b0c6bca8d857d25a3/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/26ba3719a2be9c199845beabac9858142fe066389df84d7e83a7c46f6e5671a0/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/556628bdc5e2e4117736a57a0509279947731df69aa91a7d12d52cd8bf1ed370/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c792cd5789946af6ba7b9a86af7b8d482644ace9ee7c766ee8f8190a58e64c2e/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c687f20f1c1887522330ae58cbe67052d04e161b7ce21a423164d97b67485aa2/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/9b315aa8095e446c875bffe0597c58baffad5d2880145b928ce925789e8c08ec/merged 2024-09-05T06:41:15.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/800db2aad941cab46bf8f2d9a0ccbbddf99d1ef0f13a0727984543c748f03841/merged 2024-09-05T06:41:15.832 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-09-05T06:41:15.832 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:15.832 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T06:41:15.832 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:15.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:15.832 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-09-05T06:41:15.885 INFO:teuthology.orchestra.run.smithi086.stdout:121 2024-09-05T06:41:15.885 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-09-05T06:41:15.886 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-09-05T06:41:15.887 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.622+0000 7f900a5ab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.622+0000 7f900a5ab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.622+0000 7f900a5ab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.622+0000 7f900a5ab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.622+0000 7f900a5ab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.623+0000 7f9007d26640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.623+0000 7f9007d26640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.623+0000 7f9007d26640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.623+0000 7f9007d26640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:15.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:15.623+0000 7f9007d26640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:41:16.007 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.83137.asok 2024-09-05T06:41:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T06:41:16.018 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83137.asok 2024-09-05T06:41:16.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:16.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83137.asok status 2024-09-05T06:41:16.159 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-09-05T06:41:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T06:41:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "83137", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/volumes/qa/sv_0/f90eafeb-ef3e-47d2-90fd-8c07576c6e3c" 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 15312, 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 15312 2024-09-05T06:41:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3883825824 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3883825824 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.15312 192.168.0.1:0/3883825824", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3883825824", 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 5, 2024-09-05T06:41:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 27, 2024-09-05T06:41:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 89, 2024-09-05T06:41:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-09-05T06:41:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-09-05T06:41:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-09-05T06:41:16.162 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:16.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:16.169 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-09-05T06:41:16.169 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:16.185 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-09-05T06:41:16.185 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:16.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T06:41:16.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: Active manager daemon x restarted 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: Activating manager daemon x 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: mgrmap e20: x(active, starting, since 0.0140778s), standbys: z, y 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:41:16.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:41:16.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: Manager daemon x is now available 2024-09-05T06:41:16.203 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:16.232 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T06:41:16.242 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T06:41:16.242 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:41:16.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 -- 172.21.15.86:0/2946334989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c100a40 msgr2=0x7ff40c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2946334989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c100a40 0x7ff40c100e40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff408009900 tx=0x7ff40802a410 comp rx=0 tx=0).stop 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 -- 172.21.15.86:0/2946334989 shutdown_connections 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2946334989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 0x7ff40c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2946334989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c101c30 0x7ff40c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2946334989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c100a40 0x7ff40c100e40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 -- 172.21.15.86:0/2946334989 >> 172.21.15.86:0/2946334989 conn(0x7ff40c0fc1e0 msgr2=0x7ff40c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 -- 172.21.15.86:0/2946334989 shutdown_connections 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.381+0000 7ff4136e6640 1 -- 172.21.15.86:0/2946334989 wait complete. 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 Processor -- start 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 -- start start 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c100a40 0x7ff40c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 0x7ff40c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c10c570 con 0x7ff40c101c30 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c10c3f0 con 0x7ff40c100a40 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff4136e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c10c6f0 con 0x7ff40c1025f0 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59002/0 (socket says 172.21.15.86:59002) 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 -- 172.21.15.86:0/2987211416 learned_addr learned my addr 172.21.15.86:0/2987211416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff411482640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c100a40 0x7ff40c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff411c83640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 0x7ff40c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 -- 172.21.15.86:0/2987211416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 msgr2=0x7ff40c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 0x7ff40c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 -- 172.21.15.86:0/2987211416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c100a40 msgr2=0x7ff40c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c100a40 0x7ff40c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.382+0000 7ff410c81640 1 -- 172.21.15.86:0/2987211416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c19e830 con 0x7ff40c101c30 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff410c81640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00e9f0 tx=0x7ff3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00cd80 con 0x7ff40c101c30 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3fc0108e0 con 0x7ff40c101c30 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c19eb20 con 0x7ff40c101c30 2024-09-05T06:41:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc018920 con 0x7ff40c101c30 2024-09-05T06:41:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.383+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c19f070 con 0x7ff40c101c30 2024-09-05T06:41:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.384+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c107e20 con 0x7ff40c101c30 2024-09-05T06:41:16.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.385+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7ff3fc010a90 con 0x7ff40c101c30 2024-09-05T06:41:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.386+0000 7ff4027fc640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 0x7ff3f80a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.386+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ff3fc0cf600 con 0x7ff40c101c30 2024-09-05T06:41:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.386+0000 7ff411482640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 0x7ff3f80a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.386+0000 7ff411482640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 0x7ff3f80a4f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff408006010 tx=0x7ff408000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.390+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3fc09c350 con 0x7ff40c101c30 2024-09-05T06:41:16.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:41:16.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: Active manager daemon x restarted 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: Activating manager daemon x 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: mgrmap e20: x(active, starting, since 0.0140778s), standbys: z, y 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:41:16.519 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:41:16.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: Manager daemon x is now available 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:41:16.521 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:16 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: Active manager daemon x restarted 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: Activating manager daemon x 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: mgrmap e20: x(active, starting, since 0.0140778s), standbys: z, y 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T06:41:16.524 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T06:41:16.525 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T06:41:16.526 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: Manager daemon x is now available 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T06:41:16.527 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:16 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T06:41:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:16.602+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff40c02d7c0 con 0x7ff40c101c30 2024-09-05T06:41:16.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:16.584+0000 7f8fee7b4640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:06:41:16] ENGINE Error in HTTPServer.serve 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-05T06:41:16.738 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:41:16.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-05T06:41:16.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:41:16.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-05T06:41:16.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:41:16.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:41:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:41:17.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.342+0000 7ff4027fc640 1 -- 172.21.15.86:0/2987211416 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7ff3fc09bc50 con 0x7ff40c101c30 2024-09-05T06:41:17.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 msgr2=0x7ff3f80a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 0x7ff3f80a4f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff408006010 tx=0x7ff408000bc0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 msgr2=0x7ff40c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00e9f0 tx=0x7ff3fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 shutdown_connections 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff3f80a2a40 0x7ff3f80a4f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.344+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff40c1025f0 0x7ff40c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.345+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff40c101c30 0x7ff40c197160 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.345+0000 7ff4136e6640 1 --2- 172.21.15.86:0/2987211416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff40c100a40 0x7ff40c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.345+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 >> 172.21.15.86:0/2987211416 conn(0x7ff40c0fc1e0 msgr2=0x7ff40c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.345+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 shutdown_connections 2024-09-05T06:41:17.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.345+0000 7ff4136e6640 1 -- 172.21.15.86:0/2987211416 wait complete. 2024-09-05T06:41:17.360 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-05T06:41:17.371 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-05T06:41:17.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2609435681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4102650 msgr2=0x7f43f4104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2609435681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4102650 0x7f43f4104a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009900 tx=0x7f43e802a410 comp rx=0 tx=0).stop 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2609435681 shutdown_connections 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2609435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 0x7f43f4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2609435681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4104f80 0x7f43f4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2609435681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4102650 0x7f43f4104a40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2609435681 >> 172.21.15.86:0/2609435681 conn(0x7f43f40fc170 msgr2=0x7f43f40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2609435681 shutdown_connections 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.487+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2609435681 wait complete. 2024-09-05T06:41:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 Processor -- start 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 -- start start 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4102650 0x7f43f4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 0x7f43f4100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d6c0 con 0x7f43f4104f80 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d540 con 0x7f43f41078b0 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fbf2a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d840 con 0x7f43f4102650 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59016/0 (socket says 172.21.15.86:59016) 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 -- 172.21.15.86:0/2480646648 learned_addr learned my addr 172.21.15.86:0/2480646648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f9cc6640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4102650 0x7f43f4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43fa4c7640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 0x7f43f4100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 -- 172.21.15.86:0/2480646648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4102650 msgr2=0x7f43f4101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4102650 0x7f43f4101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 -- 172.21.15.86:0/2480646648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 msgr2=0x7f43f4100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 0x7f43f4100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 -- 172.21.15.86:0/2480646648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f41013d0 con 0x7f43f4104f80 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43f94c5640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e9f0 tx=0x7f43f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.488+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f000ccc0 con 0x7f43f4104f80 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.489+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41a1f20 con 0x7f43f4104f80 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.489+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43f00054d0 con 0x7f43f4104f80 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.489+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0010430 con 0x7f43f4104f80 2024-09-05T06:41:17.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.489+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a2330 con 0x7f43f4104f80 2024-09-05T06:41:17.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.490+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f40fdc60 con 0x7f43f4104f80 2024-09-05T06:41:17.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.492+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f43f0005020 con 0x7f43f4104f80 2024-09-05T06:41:17.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.492+0000 7f43e6ffd640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 0x7f43e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.493+0000 7f43f9cc6640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 0x7f43e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.493+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f43f00cfbd0 con 0x7f43f4104f80 2024-09-05T06:41:17.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.493+0000 7f43f9cc6640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 0x7f43e00a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e80314e0 tx=0x7f43e8005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:17.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.497+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43f009c410 con 0x7f43f4104f80 2024-09-05T06:41:17.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.672+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43f402da10 con 0x7f43f4104f80 2024-09-05T06:41:17.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.674+0000 7f43e6ffd640 1 -- 172.21.15.86:0/2480646648 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25345 (secure 0 0 0) 0x7f43f009bd10 con 0x7f43f4104f80 2024-09-05T06:41:17.674 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:17.674 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:41:15.328284+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4062552821":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/2959599766":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/3708924769":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4040819651":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3625555544":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4279196951":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6800/3708924769":"2024-09-06T06:41:15.328251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:41:17.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 msgr2=0x7f43e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 0x7f43e00a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43e80314e0 tx=0x7f43e8005a20 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 msgr2=0x7f43f4100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e9f0 tx=0x7f43f000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 shutdown_connections 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43e00a2d00 0x7f43e00a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43f41078b0 0x7f43f4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f4104f80 0x7f43f4100600 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 --2- 172.21.15.86:0/2480646648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f4102650 0x7f43f4101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 >> 172.21.15.86:0/2480646648 conn(0x7f43f40fc170 msgr2=0x7f43f4103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 shutdown_connections 2024-09-05T06:41:17.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.676+0000 7f43fbf2a640 1 -- 172.21.15.86:0/2480646648 wait complete. 2024-09-05T06:41:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:17 smithi115 ceph-mon[34791]: mgrmap e21: x(active, since 1.01866s), standbys: z, y 2024-09-05T06:41:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:17 smithi115 ceph-mon[34791]: [05/Sep/2024:06:41:16] ENGINE Bus STARTING 2024-09-05T06:41:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T06:41:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:17 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:17.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:17 smithi086 ceph-mon[30943]: mgrmap e21: x(active, since 1.01866s), standbys: z, y 2024-09-05T06:41:17.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:17 smithi086 ceph-mon[30943]: [05/Sep/2024:06:41:16] ENGINE Bus STARTING 2024-09-05T06:41:17.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T06:41:17.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:17 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:17.688 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T06:34:21.377500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T06:38:51.679225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T06:38:54.212638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T06:39:01.086734+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T06:41:17.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 -- 172.21.15.86:0/1345221882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 msgr2=0x7f06381049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 --2- 172.21.15.86:0/1345221882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f06381049f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f062c009900 tx=0x7f062c02a410 comp rx=0 tx=0).stop 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 -- 172.21.15.86:0/1345221882 shutdown_connections 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 --2- 172.21.15.86:0/1345221882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638107860 0x7f0638109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 --2- 172.21.15.86:0/1345221882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638104f30 0x7f0638107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 --2- 172.21.15.86:0/1345221882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f06381049f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 -- 172.21.15.86:0/1345221882 >> 172.21.15.86:0/1345221882 conn(0x7f06380fc140 msgr2=0x7f06380fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 -- 172.21.15.86:0/1345221882 shutdown_connections 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.793+0000 7f063eb8d640 1 -- 172.21.15.86:0/1345221882 wait complete. 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 Processor -- start 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 -- start start 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638104f30 0x7f0638198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 0x7f063819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f063810d710 con 0x7f0638102600 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f063810d590 con 0x7f0638107860 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063eb8d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f063810d890 con 0x7f0638104f30 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063c929640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063c929640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59036/0 (socket says 172.21.15.86:59036) 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063c929640 1 -- 172.21.15.86:0/970376394 learned_addr learned my addr 172.21.15.86:0/970376394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f0637fff640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638104f30 0x7f0638198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.794+0000 7f063d12a640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 0x7f063819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 -- 172.21.15.86:0/970376394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638104f30 msgr2=0x7f0638198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638104f30 0x7f0638198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 -- 172.21.15.86:0/970376394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 msgr2=0x7f063819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 0x7f063819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 -- 172.21.15.86:0/970376394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063819fae0 con 0x7f0638102600 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063d12a640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 0x7f063819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063c929640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f062c004770 tx=0x7f062c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c03f030 con 0x7f0638102600 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063819fd70 con 0x7f0638102600 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f062c003cb0 con 0x7f0638102600 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c035070 con 0x7f0638102600 2024-09-05T06:41:17.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.795+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381a0230 con 0x7f0638102600 2024-09-05T06:41:17.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.796+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06380fdc30 con 0x7f0638102600 2024-09-05T06:41:17.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.797+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f062c042030 con 0x7f0638102600 2024-09-05T06:41:17.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.798+0000 7f0635ffb640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 0x7f06280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:17.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.798+0000 7f0637fff640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 0x7f06280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:17.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.798+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f062c0f03b0 con 0x7f0638102600 2024-09-05T06:41:17.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.799+0000 7f0637fff640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 0x7f06280a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06381993d0 tx=0x7f0624009110 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.801+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f062c0bcbf0 con 0x7f0638102600 2024-09-05T06:41:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:17 smithi096 ceph-mon[34468]: mgrmap e21: x(active, since 1.01866s), standbys: z, y 2024-09-05T06:41:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:17 smithi096 ceph-mon[34468]: [05/Sep/2024:06:41:16] ENGINE Bus STARTING 2024-09-05T06:41:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T06:41:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:17 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.976+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f063802d950 con 0x7f0638102600 2024-09-05T06:41:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.976+0000 7f0635ffb640 1 -- 172.21.15.86:0/970376394 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f062c0bc4f0 con 0x7f0638102600 2024-09-05T06:41:17.977 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 msgr2=0x7f06280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 0x7f06280a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06381993d0 tx=0x7f0624009110 comp rx=0 tx=0).stop 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 msgr2=0x7f0638197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f062c004770 tx=0x7f062c0042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 shutdown_connections 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f06280a2d00 0x7f06280a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0638107860 0x7f063819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0638104f30 0x7f0638198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 --2- 172.21.15.86:0/970376394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0638102600 0x7f0638197ec0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 >> 172.21.15.86:0/970376394 conn(0x7f06380fc140 msgr2=0x7f0638102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 shutdown_connections 2024-09-05T06:41:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:17.979+0000 7f063eb8d640 1 -- 172.21.15.86:0/970376394 wait complete. 2024-09-05T06:41:17.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T06:41:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 -- 172.21.15.86:0/898311758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a80100a00 msgr2=0x7f0a80100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/898311758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a80100a00 0x7f0a80100e00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009900 tx=0x7f0a7002a410 comp rx=0 tx=0).stop 2024-09-05T06:41:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 -- 172.21.15.86:0/898311758 shutdown_connections 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/898311758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a801025b0 0x7f0a80108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/898311758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/898311758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a80100a00 0x7f0a80100e00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 -- 172.21.15.86:0/898311758 >> 172.21.15.86:0/898311758 conn(0x7f0a800fc140 msgr2=0x7f0a800fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 -- 172.21.15.86:0/898311758 shutdown_connections 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.102+0000 7f0a85d6f640 1 -- 172.21.15.86:0/898311758 wait complete. 2024-09-05T06:41:18.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 Processor -- start 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 -- start start 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 0x7f0a80197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8010c580 con 0x7f0a801025b0 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8010c400 con 0x7f0a80100a00 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a85d6f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8010c700 con 0x7f0a80101bf0 2024-09-05T06:41:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59052/0 (socket says 172.21.15.86:59052) 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 0x7f0a80197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:45446/0 (socket says 172.21.15.86:45446) 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 -- 172.21.15.86:0/1721304330 learned_addr learned my addr 172.21.15.86:0/1721304330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 -- 172.21.15.86:0/1721304330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 msgr2=0x7f0a80198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 -- 172.21.15.86:0/1721304330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 msgr2=0x7f0a80197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 0x7f0a80197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7ffff640 1 -- 172.21.15.86:0/1721304330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8019fa10 con 0x7f0a801025b0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.103+0000 7f0a7effd640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a7f7fe640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 0x7f0a80197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a7ffff640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800eb10 tx=0x7f0a7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78004b20 con 0x7f0a801025b0 2024-09-05T06:41:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a78004cc0 con 0x7f0a801025b0 2024-09-05T06:41:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8019fd00 con 0x7f0a801025b0 2024-09-05T06:41:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.104+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78010690 con 0x7f0a801025b0 2024-09-05T06:41:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.105+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a801a01f0 con 0x7f0a801025b0 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.106+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a801a0480 con 0x7f0a801025b0 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.107+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f0a78005020 con 0x7f0a801025b0 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.108+0000 7f0a7cff9640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 0x7f0a680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.108+0000 7f0a7f7fe640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 0x7f0a680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.108+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0a780d2120 con 0x7f0a801025b0 2024-09-05T06:41:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.108+0000 7f0a7f7fe640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 0x7f0a680a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005ab0 tx=0x7f0a70005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.112+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a7809c580 con 0x7f0a801025b0 2024-09-05T06:41:18.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.316+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a801a06f0 con 0x7f0a801025b0 2024-09-05T06:41:18.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.316+0000 7f0a7cff9640 1 -- 172.21.15.86:0/1721304330 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0a7809be80 con 0x7f0a801025b0 2024-09-05T06:41:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-09-05T06:41:18.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 msgr2=0x7f0a680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 0x7f0a680a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005ab0 tx=0x7f0a70005a20 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 msgr2=0x7f0a8019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800eb10 tx=0x7f0a7800c2d0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 shutdown_connections 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0a680a2d80 0x7f0a680a5240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a801025b0 0x7f0a8019f3a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.318+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a80101bf0 0x7f0a80198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.319+0000 7f0a85d6f640 1 --2- 172.21.15.86:0/1721304330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80100a00 0x7f0a80197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.319+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 >> 172.21.15.86:0/1721304330 conn(0x7f0a800fc140 msgr2=0x7f0a80069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.319+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 shutdown_connections 2024-09-05T06:41:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.319+0000 7f0a85d6f640 1 -- 172.21.15.86:0/1721304330 wait complete. 2024-09-05T06:41:18.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 -- 172.21.15.86:0/311543376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d81025e0 msgr2=0x7fb3d8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/311543376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d81025e0 0x7fb3d8108e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ca0 tx=0x7fb3d002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 -- 172.21.15.86:0/311543376 shutdown_connections 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/311543376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d81025e0 0x7fb3d8108e70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/311543376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8101c20 0x7fb3d81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/311543376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d8100a30 0x7fb3d8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 -- 172.21.15.86:0/311543376 >> 172.21.15.86:0/311543376 conn(0x7fb3d80fc1b0 msgr2=0x7fb3d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 -- 172.21.15.86:0/311543376 shutdown_connections 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 -- 172.21.15.86:0/311543376 wait complete. 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.437+0000 7fb3dfa50640 1 Processor -- start 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 -- start start 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8100a30 0x7fb3d8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81025e0 0x7fb3d819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810c5b0 con 0x7fb3d8101c20 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810c430 con 0x7fb3d81025e0 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dfa50640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810c730 con 0x7fb3d8100a30 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59060/0 (socket says 172.21.15.86:59060) 2024-09-05T06:41:18.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 -- 172.21.15.86:0/1001965297 learned_addr learned my addr 172.21.15.86:0/1001965297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3ddfed640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81025e0 0x7fb3d819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dd7ec640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8100a30 0x7fb3d8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 -- 172.21.15.86:0/1001965297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8100a30 msgr2=0x7fb3d8197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8100a30 0x7fb3d8197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 -- 172.21.15.86:0/1001965297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81025e0 msgr2=0x7fb3d819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81025e0 0x7fb3d819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 -- 172.21.15.86:0/1001965297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819f940 con 0x7fb3d8101c20 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.438+0000 7fb3dcfeb640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800e9f0 tx=0x7fb3c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.439+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800cd80 con 0x7fb3d8101c20 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.439+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819fc30 con 0x7fb3d8101c20 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.439+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c80054d0 con 0x7fb3d8101c20 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.439+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8010490 con 0x7fb3d8101c20 2024-09-05T06:41:18.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.439+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a0180 con 0x7fb3d8101c20 2024-09-05T06:41:18.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.440+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d81a0350 con 0x7fb3d8101c20 2024-09-05T06:41:18.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.441+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fb3c8005000 con 0x7fb3d8101c20 2024-09-05T06:41:18.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.442+0000 7fb3ce7fc640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 0x7fb3c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.442+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb3c80cfe60 con 0x7fb3d8101c20 2024-09-05T06:41:18.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.442+0000 7fb3dd7ec640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 0x7fb3c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.442+0000 7fb3dd7ec640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 0x7fb3c40a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8199200 tx=0x7fb3d4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.445+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3c80541b0 con 0x7fb3d8101c20 2024-09-05T06:41:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.622+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3d81a0520 con 0x7fb3d8101c20 2024-09-05T06:41:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.622+0000 7fb3ce7fc640 1 -- 172.21.15.86:0/1001965297 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb3c80052f0 con 0x7fb3d8101c20 2024-09-05T06:41:18.622 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-09-05T06:41:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 msgr2=0x7fb3c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 0x7fb3c40a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8199200 tx=0x7fb3d4005f90 comp rx=0 tx=0).stop 2024-09-05T06:41:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 msgr2=0x7fb3d8198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800e9f0 tx=0x7fb3c800edf0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 shutdown_connections 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c40a2d00 0x7fb3c40a51c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81025e0 0x7fb3d819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8101c20 0x7fb3d8198270 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 --2- 172.21.15.86:0/1001965297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d8100a30 0x7fb3d8197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 >> 172.21.15.86:0/1001965297 conn(0x7fb3d80fc1b0 msgr2=0x7fb3d80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 shutdown_connections 2024-09-05T06:41:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.624+0000 7fb3dfa50640 1 -- 172.21.15.86:0/1001965297 wait complete. 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: [05/Sep/2024:06:41:16] ENGINE Serving on http://172.21.15.86:8765 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: [05/Sep/2024:06:41:16] ENGINE Serving on https://172.21.15.86:7150 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: [05/Sep/2024:06:41:16] ENGINE Bus STARTED 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: [05/Sep/2024:06:41:16] ENGINE Error in HTTPServer.serve 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: Traceback (most recent call last): 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._connections.run(self.expiration_interval) 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._run(expiration_interval) 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:41:18.635 INFO:journalctl@ceph.mon.a.smithi086.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: s = self.context.wrap_socket( 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: return self.sslsocket_class._create( 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: self.do_handshake() 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: self._sslobj.do_handshake() 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.636 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2480646648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/970376394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.638 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:18.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:18.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1721304330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: [05/Sep/2024:06:41:16] ENGINE Serving on http://172.21.15.86:8765 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: [05/Sep/2024:06:41:16] ENGINE Serving on https://172.21.15.86:7150 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: [05/Sep/2024:06:41:16] ENGINE Bus STARTED 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: [05/Sep/2024:06:41:16] ENGINE Error in HTTPServer.serve 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: Traceback (most recent call last): 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._run(expiration_interval) 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:41:18.683 INFO:journalctl@ceph.mon.c.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: s = self.context.wrap_socket( 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: return self.sslsocket_class._create( 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: self.do_handshake() 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._sslobj.do_handshake() 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:41:18.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2480646648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/970376394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:18.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1721304330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 -- 172.21.15.86:0/535756905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 msgr2=0x7f77c0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 --2- 172.21.15.86:0/535756905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0102070 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009900 tx=0x7f77b402a410 comp rx=0 tx=0).stop 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 -- 172.21.15.86:0/535756905 shutdown_connections 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 --2- 172.21.15.86:0/535756905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 --2- 172.21.15.86:0/535756905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0102070 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 --2- 172.21.15.86:0/535756905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 0x7f77c0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 -- 172.21.15.86:0/535756905 >> 172.21.15.86:0/535756905 conn(0x7f77c00fc140 msgr2=0x7f77c00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 -- 172.21.15.86:0/535756905 shutdown_connections 2024-09-05T06:41:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.746+0000 7f77c54ac640 1 -- 172.21.15.86:0/535756905 wait complete. 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 Processor -- start 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 -- start start 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 0x7f77c0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d6e0 con 0x7f77c0101bf0 2024-09-05T06:41:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d560 con 0x7f77c01025b0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.747+0000 7f77c54ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d860 con 0x7f77c0100a00 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77beffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 0x7f77c0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59066/0 (socket says 172.21.15.86:59066) 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 -- 172.21.15.86:0/616557324 learned_addr learned my addr 172.21.15.86:0/616557324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77bf7fe640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 -- 172.21.15.86:0/616557324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 msgr2=0x7f77c0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 0x7f77c0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 -- 172.21.15.86:0/616557324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 msgr2=0x7f77c019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 -- 172.21.15.86:0/616557324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c019f9e0 con 0x7f77c0101bf0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77bf7fe640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77be7fc640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77b40314e0 tx=0x7f77b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b403f070 con 0x7f77c0101bf0 2024-09-05T06:41:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c019fc70 con 0x7f77c0101bf0 2024-09-05T06:41:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77b4031c40 con 0x7f77c0101bf0 2024-09-05T06:41:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4035030 con 0x7f77c0101bf0 2024-09-05T06:41:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.748+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01a0160 con 0x7f77c0101bf0 2024-09-05T06:41:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.749+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c0107ee0 con 0x7f77c0101bf0 2024-09-05T06:41:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.751+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f77b4042070 con 0x7f77c0101bf0 2024-09-05T06:41:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.751+0000 7f77a7fff640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 0x7f77a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.751+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f77b40f0660 con 0x7f77c0101bf0 2024-09-05T06:41:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.752+0000 7f77beffd640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 0x7f77a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.752+0000 7f77beffd640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 0x7f77a80a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77c0101a50 tx=0x7f77b0005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.755+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77b40bc850 con 0x7f77c0101bf0 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: [05/Sep/2024:06:41:16] ENGINE Serving on http://172.21.15.86:8765 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: [05/Sep/2024:06:41:16] ENGINE Serving on https://172.21.15.86:7150 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: [05/Sep/2024:06:41:16] ENGINE Bus STARTED 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: [05/Sep/2024:06:41:16] ENGINE Error in HTTPServer.serve 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: Traceback (most recent call last): 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._connections.run(self.expiration_interval) 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._run(expiration_interval) 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T06:41:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: s = self.context.wrap_socket( 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: return self.sslsocket_class._create( 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: self.do_handshake() 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._sslobj.do_handshake() 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2987211416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T06:41:18.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2480646648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/970376394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:18.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:18.811 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1721304330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T06:41:18.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.935+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77c002da10 con 0x7f77c0101bf0 2024-09-05T06:41:18.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.935+0000 7f77a7fff640 1 -- 172.21.15.86:0/616557324 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f77b40c0b50 con 0x7f77c0101bf0 2024-09-05T06:41:18.935 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-09-05T06:41:18.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 msgr2=0x7f77a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 0x7f77a80a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77c0101a50 tx=0x7f77b0005f90 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 msgr2=0x7f77c0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77b40314e0 tx=0x7f77b4003c10 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 shutdown_connections 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77a80a2d00 0x7f77a80a51c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c01025b0 0x7f77c019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77c0101bf0 0x7f77c0198300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 --2- 172.21.15.86:0/616557324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c0100a00 0x7f77c0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.937+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 >> 172.21.15.86:0/616557324 conn(0x7f77c00fc140 msgr2=0x7f77c01050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.938+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 shutdown_connections 2024-09-05T06:41:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:18.938+0000 7f77c54ac640 1 -- 172.21.15.86:0/616557324 wait complete. 2024-09-05T06:41:18.949 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-05T06:41:18.949 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:41:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Updating smithi086:/etc/ceph/ceph.conf 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-05T06:41:19.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1001965297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/616557324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:19.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:19 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Updating smithi086:/etc/ceph/ceph.conf 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-05T06:41:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1001965297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/616557324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:19.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:19 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 -- 172.21.15.86:0/1573733237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102ef0 msgr2=0x7f866010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:19.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 --2- 172.21.15.86:0/1573733237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102ef0 0x7f866010da40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8658009ca0 tx=0x7f865802a950 comp rx=0 tx=0).stop 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 -- 172.21.15.86:0/1573733237 shutdown_connections 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 --2- 172.21.15.86:0/1573733237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102ef0 0x7f866010da40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 --2- 172.21.15.86:0/1573733237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102570 0x7f86601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 --2- 172.21.15.86:0/1573733237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 0x7f8660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 -- 172.21.15.86:0/1573733237 >> 172.21.15.86:0/1573733237 conn(0x7f86600fc7d0 msgr2=0x7f86600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 -- 172.21.15.86:0/1573733237 shutdown_connections 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.691+0000 7f8666718640 1 -- 172.21.15.86:0/1573733237 wait complete. 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.692+0000 7f8666718640 1 Processor -- start 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.692+0000 7f8666718640 1 -- start start 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 0x7f86601983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102ef0 0x7f866019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8660111f50 con 0x7f8660102570 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8660111dd0 con 0x7f8660100e20 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86601120d0 con 0x7f8660102ef0 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59076/0 (socket says 172.21.15.86:59076) 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 -- 172.21.15.86:0/651074804 learned_addr learned my addr 172.21.15.86:0/651074804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8664cb5640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102ef0 0x7f866019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865ffff640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 0x7f86601983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 -- 172.21.15.86:0/651074804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102ef0 msgr2=0x7f866019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102ef0 0x7f866019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 -- 172.21.15.86:0/651074804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 msgr2=0x7f86601983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 0x7f86601983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 -- 172.21.15.86:0/651074804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866019d3a0 con 0x7f8660102570 2024-09-05T06:41:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865f7fe640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864800e9f0 tx=0x7f864800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864800cd80 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866019d690 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86480054d0 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648010490 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.693+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86601a4f70 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.694+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8660101200 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.696+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f8648005020 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.696+0000 7f865d7fa640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 0x7f864c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.696+0000 7f865ffff640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 0x7f864c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.696+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f86480cfbc0 con 0x7f8660102570 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.697+0000 7f865ffff640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 0x7f864c0a51b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86500058c0 tx=0x7f865000a2d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.700+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f864809c400 con 0x7f8660102570 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Updating smithi086:/etc/ceph/ceph.conf 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-05T06:41:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1001965297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/616557324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:19.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:19 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.897+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8660109090 con 0x7f8660102570 2024-09-05T06:41:19.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.898+0000 7f865d7fa640 1 -- 172.21.15.86:0/651074804 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f864809bd00 con 0x7f8660102570 2024-09-05T06:41:19.898 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:19.898 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:41:19.898 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:41:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 msgr2=0x7f864c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 0x7f864c0a51b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86500058c0 tx=0x7f865000a2d0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 msgr2=0x7f8660198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864800e9f0 tx=0x7f864800edf0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 shutdown_connections 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f864c0a2cf0 0x7f864c0a51b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8660102ef0 0x7f866019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8660102570 0x7f8660198930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 --2- 172.21.15.86:0/651074804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8660100e20 0x7f86601983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 >> 172.21.15.86:0/651074804 conn(0x7f86600fc7d0 msgr2=0x7f86600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.900+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 shutdown_connections 2024-09-05T06:41:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:19.901+0000 7f8666718640 1 -- 172.21.15.86:0/651074804 wait complete. 2024-09-05T06:41:20.585 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-05T06:41:20.585 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-05T06:41:20.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T06:41:20.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/651074804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:20.609 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:20 smithi086 ceph-mon[30943]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-05T06:41:20.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/651074804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:20 smithi115 ceph-mon[34791]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 -- 172.21.15.86:0/2180934767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 msgr2=0x7fb9ec10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2180934767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec10e970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009cd0 tx=0x7fb9e402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 -- 172.21.15.86:0/2180934767 shutdown_connections 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2180934767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec10e970 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2180934767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 0x7fb9ec101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2180934767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 0x7fb9ec107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 -- 172.21.15.86:0/2180934767 >> 172.21.15.86:0/2180934767 conn(0x7fb9ec0fd7c0 msgr2=0x7fb9ec0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:20.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 -- 172.21.15.86:0/2180934767 shutdown_connections 2024-09-05T06:41:20.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 -- 172.21.15.86:0/2180934767 wait complete. 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.687+0000 7fb9f192c640 1 Processor -- start 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 -- start start 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 0x7fb9ec1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 0x7fb9ec19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec111da0 con 0x7fb9ec102440 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec111c20 con 0x7fb9ec107ab0 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9f192c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec111f20 con 0x7fb9ec101aa0 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:20.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59092/0 (socket says 172.21.15.86:59092) 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 -- 172.21.15.86:0/2085861367 learned_addr learned my addr 172.21.15.86:0/2085861367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9eb7fe640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 0x7fb9ec19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9eaffd640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 0x7fb9ec1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 -- 172.21.15.86:0/2085861367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 msgr2=0x7fb9ec1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 0x7fb9ec1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 -- 172.21.15.86:0/2085861367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 msgr2=0x7fb9ec19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 0x7fb9ec19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.688+0000 7fb9ea7fc640 1 -- 172.21.15.86:0/2085861367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec19d1e0 con 0x7fb9ec102440 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9ea7fc640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c920 tx=0x7fb9dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc007c70 con 0x7fb9ec102440 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9dc007e10 con 0x7fb9ec102440 2024-09-05T06:41:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ec19d4d0 con 0x7fb9ec102440 2024-09-05T06:41:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc018820 con 0x7fb9ec102440 2024-09-05T06:41:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.689+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ec1a4d60 con 0x7fb9ec102440 2024-09-05T06:41:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.690+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec101f00 con 0x7fb9ec102440 2024-09-05T06:41:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.691+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb9dc0189c0 con 0x7fb9ec102440 2024-09-05T06:41:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.691+0000 7fb9f092a640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 0x7fb9d40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.692+0000 7fb9eaffd640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 0x7fb9d40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.692+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb9dc0cfb00 con 0x7fb9ec102440 2024-09-05T06:41:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.692+0000 7fb9eaffd640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 0x7fb9d40a5250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0005e00 tx=0x7fb9e0005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:20.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.696+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9dc014030 con 0x7fb9ec102440 2024-09-05T06:41:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: Updating smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/651074804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:20.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:20 smithi096 ceph-mon[34468]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-05T06:41:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.858+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9ec0ff630 con 0x7fb9d40a2d90 2024-09-05T06:41:20.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.866+0000 7fb9f092a640 1 -- 172.21.15.86:0/2085861367 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339593 (secure 0 0 0) 0x7fb9ec0ff630 con 0x7fb9d40a2d90 2024-09-05T06:41:20.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:20.872 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:20.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 msgr2=0x7fb9d40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:20.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 0x7fb9d40a5250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0005e00 tx=0x7fb9e0005c50 comp rx=0 tx=0).stop 2024-09-05T06:41:20.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 msgr2=0x7fb9ec198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:20.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c920 tx=0x7fb9dc00cde0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 shutdown_connections 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9d40a2d90 0x7fb9d40a5250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9ec107ab0 0x7fb9ec19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec102440 0x7fb9ec198730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 --2- 172.21.15.86:0/2085861367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec101aa0 0x7fb9ec1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.869+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 >> 172.21.15.86:0/2085861367 conn(0x7fb9ec0fd7c0 msgr2=0x7fb9ec0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.870+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 shutdown_connections 2024-09-05T06:41:20.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:20.870+0000 7fb9f192c640 1 -- 172.21.15.86:0/2085861367 wait complete. 2024-09-05T06:41:20.880 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-05T06:41:19.349648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191268,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1318,"num_read_kb":826,"num_write":566,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389708,"kb_used_data":15464,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552116,"statfs":{"total":1151940427776,"available":1151541366784,"internally_reserved":0,"allocated":15835136,"data_stored":9262593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.993319"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375225+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:15.375225+0000","last_peered":"2024-09-05T06:41:15.375225+0000","last_clean":"2024-09-05T06:41:15.375225+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:15.375225+0000","last_undegraded":"2024-09-05T06:41:15.375225+0000","last_fullsized":"2024-09-05T06:41:15.375225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354898+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354898+0000","last_peered":"2024-09-05T06:41:15.354898+0000","last_clean":"2024-09-05T06:41:15.354898+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354898+0000","last_undegraded":"2024-09-05T06:41:15.354898+0000","last_fullsized":"2024-09-05T06:41:15.354898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383888+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.383888+0000","last_peered":"2024-09-05T06:41:15.383888+0000","last_clean":"2024-09-05T06:41:15.383888+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.383888+0000","last_undegraded":"2024-09-05T06:41:15.383888+0000","last_fullsized":"2024-09-05T06:41:15.383888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350604+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350604+0000","last_peered":"2024-09-05T06:41:15.350604+0000","last_clean":"2024-09-05T06:41:15.350604+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350604+0000","last_undegraded":"2024-09-05T06:41:15.350604+0000","last_fullsized":"2024-09-05T06:41:15.350604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385607+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385607+0000","last_peered":"2024-09-05T06:41:15.385607+0000","last_clean":"2024-09-05T06:41:15.385607+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385607+0000","last_undegraded":"2024-09-05T06:41:15.385607+0000","last_fullsized":"2024-09-05T06:41:15.385607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345930+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345930+0000","last_peered":"2024-09-05T06:41:15.345930+0000","last_clean":"2024-09-05T06:41:15.345930+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345930+0000","last_undegraded":"2024-09-05T06:41:15.345930+0000","last_fullsized":"2024-09-05T06:41:15.345930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391262+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391262+0000","last_peered":"2024-09-05T06:41:15.391262+0000","last_clean":"2024-09-05T06:41:15.391262+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391262+0000","last_undegraded":"2024-09-05T06:41:15.391262+0000","last_fullsized":"2024-09-05T06:41:15.391262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345709+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345709+0000","last_peered":"2024-09-05T06:41:15.345709+0000","last_clean":"2024-09-05T06:41:15.345709+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345709+0000","last_undegraded":"2024-09-05T06:41:15.345709+0000","last_fullsized":"2024-09-05T06:41:15.345709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346004+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346004+0000","last_peered":"2024-09-05T06:41:15.346004+0000","last_clean":"2024-09-05T06:41:15.346004+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346004+0000","last_undegraded":"2024-09-05T06:41:15.346004+0000","last_fullsized":"2024-09-05T06:41:15.346004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'50","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.569551+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:15.569551+0000","last_peered":"2024-09-05T06:41:15.569551+0000","last_clean":"2024-09-05T06:41:15.569551+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:15.569551+0000","last_undegraded":"2024-09-05T06:41:15.569551+0000","last_fullsized":"2024-09-05T06:41:15.569551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.004241+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:16.004241+0000","last_peered":"2024-09-05T06:41:16.004241+0000","last_clean":"2024-09-05T06:41:16.004241+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:16.004241+0000","last_undegraded":"2024-09-05T06:41:16.004241+0000","last_fullsized":"2024-09-05T06:41:16.004241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336294+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:15.336294+0000","last_peered":"2024-09-05T06:41:15.336294+0000","last_clean":"2024-09-05T06:41:15.336294+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:15.336294+0000","last_undegraded":"2024-09-05T06:41:15.336294+0000","last_fullsized":"2024-09-05T06:41:15.336294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338545+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:15.338545+0000","last_peered":"2024-09-05T06:41:15.338545+0000","last_clean":"2024-09-05T06:41:15.338545+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:15.338545+0000","last_undegraded":"2024-09-05T06:41:15.338545+0000","last_fullsized":"2024-09-05T06:41:15.338545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.062661+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:16.062661+0000","last_peered":"2024-09-05T06:41:16.062661+0000","last_clean":"2024-09-05T06:41:16.062661+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:16.062661+0000","last_undegraded":"2024-09-05T06:41:16.062661+0000","last_fullsized":"2024-09-05T06:41:16.062661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.566938+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:15.566938+0000","last_peered":"2024-09-05T06:41:15.566938+0000","last_clean":"2024-09-05T06:41:15.566938+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:15.566938+0000","last_undegraded":"2024-09-05T06:41:15.566938+0000","last_fullsized":"2024-09-05T06:41:15.566938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336610+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336610+0000","last_peered":"2024-09-05T06:41:15.336610+0000","last_clean":"2024-09-05T06:41:15.336610+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336610+0000","last_undegraded":"2024-09-05T06:41:15.336610+0000","last_fullsized":"2024-09-05T06:41:15.336610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.994711+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:15.994711+0000","last_peered":"2024-09-05T06:41:15.994711+0000","last_clean":"2024-09-05T06:41:15.994711+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:15.994711+0000","last_undegraded":"2024-09-05T06:41:15.994711+0000","last_fullsized":"2024-09-05T06:41:15.994711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390252+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390252+0000","last_peered":"2024-09-05T06:41:15.390252+0000","last_clean":"2024-09-05T06:41:15.390252+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390252+0000","last_undegraded":"2024-09-05T06:41:15.390252+0000","last_fullsized":"2024-09-05T06:41:15.390252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353154+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353154+0000","last_peered":"2024-09-05T06:41:15.353154+0000","last_clean":"2024-09-05T06:41:15.353154+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353154+0000","last_undegraded":"2024-09-05T06:41:15.353154+0000","last_fullsized":"2024-09-05T06:41:15.353154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381394+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381394+0000","last_peered":"2024-09-05T06:41:15.381394+0000","last_clean":"2024-09-05T06:41:15.381394+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381394+0000","last_undegraded":"2024-09-05T06:41:15.381394+0000","last_fullsized":"2024-09-05T06:41:15.381394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336176+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336176+0000","last_peered":"2024-09-05T06:41:15.336176+0000","last_clean":"2024-09-05T06:41:15.336176+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336176+0000","last_undegraded":"2024-09-05T06:41:15.336176+0000","last_fullsized":"2024-09-05T06:41:15.336176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368864+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368864+0000","last_peered":"2024-09-05T06:41:15.368864+0000","last_clean":"2024-09-05T06:41:15.368864+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368864+0000","last_undegraded":"2024-09-05T06:41:15.368864+0000","last_fullsized":"2024-09-05T06:41:15.368864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":317,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.994479+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:15.994479+0000","last_peered":"2024-09-05T06:41:15.994479+0000","last_clean":"2024-09-05T06:41:15.994479+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:15.994479+0000","last_undegraded":"2024-09-05T06:41:15.994479+0000","last_fullsized":"2024-09-05T06:41:15.994479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65201,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357230+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357230+0000","last_peered":"2024-09-05T06:41:15.357230+0000","last_clean":"2024-09-05T06:41:15.357230+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357230+0000","last_undegraded":"2024-09-05T06:41:15.357230+0000","last_fullsized":"2024-09-05T06:41:15.357230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384938+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.384938+0000","last_peered":"2024-09-05T06:41:15.384938+0000","last_clean":"2024-09-05T06:41:15.384938+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.384938+0000","last_undegraded":"2024-09-05T06:41:15.384938+0000","last_fullsized":"2024-09-05T06:41:15.384938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346385+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346385+0000","last_peered":"2024-09-05T06:41:15.346385+0000","last_clean":"2024-09-05T06:41:15.346385+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346385+0000","last_undegraded":"2024-09-05T06:41:15.346385+0000","last_fullsized":"2024-09-05T06:41:15.346385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392388+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392388+0000","last_peered":"2024-09-05T06:41:15.392388+0000","last_clean":"2024-09-05T06:41:15.392388+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392388+0000","last_undegraded":"2024-09-05T06:41:15.392388+0000","last_fullsized":"2024-09-05T06:41:15.392388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356999+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.356999+0000","last_peered":"2024-09-05T06:41:15.356999+0000","last_clean":"2024-09-05T06:41:15.356999+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.356999+0000","last_undegraded":"2024-09-05T06:41:15.356999+0000","last_fullsized":"2024-09-05T06:41:15.356999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381534+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.381534+0000","last_peered":"2024-09-05T06:41:15.381534+0000","last_clean":"2024-09-05T06:41:15.381534+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.381534+0000","last_undegraded":"2024-09-05T06:41:15.381534+0000","last_fullsized":"2024-09-05T06:41:15.381534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":254,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.993502+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:15.993502+0000","last_peered":"2024-09-05T06:41:15.993502+0000","last_clean":"2024-09-05T06:41:15.993502+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:15.993502+0000","last_undegraded":"2024-09-05T06:41:15.993502+0000","last_fullsized":"2024-09-05T06:41:15.993502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351143+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351143+0000","last_peered":"2024-09-05T06:41:15.351143+0000","last_clean":"2024-09-05T06:41:15.351143+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351143+0000","last_undegraded":"2024-09-05T06:41:15.351143+0000","last_fullsized":"2024-09-05T06:41:15.351143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388662+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388662+0000","last_peered":"2024-09-05T06:41:15.388662+0000","last_clean":"2024-09-05T06:41:15.388662+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388662+0000","last_undegraded":"2024-09-05T06:41:15.388662+0000","last_fullsized":"2024-09-05T06:41:15.388662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335966+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.335966+0000","last_peered":"2024-09-05T06:41:15.335966+0000","last_clean":"2024-09-05T06:41:15.335966+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.335966+0000","last_undegraded":"2024-09-05T06:41:15.335966+0000","last_fullsized":"2024-09-05T06:41:15.335966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.378926+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.378926+0000","last_peered":"2024-09-05T06:41:15.378926+0000","last_clean":"2024-09-05T06:41:15.378926+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.378926+0000","last_undegraded":"2024-09-05T06:41:15.378926+0000","last_fullsized":"2024-09-05T06:41:15.378926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336108+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336108+0000","last_peered":"2024-09-05T06:41:15.336108+0000","last_clean":"2024-09-05T06:41:15.336108+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336108+0000","last_undegraded":"2024-09-05T06:41:15.336108+0000","last_fullsized":"2024-09-05T06:41:15.336108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346029+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346029+0000","last_peered":"2024-09-05T06:41:15.346029+0000","last_clean":"2024-09-05T06:41:15.346029+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346029+0000","last_undegraded":"2024-09-05T06:41:15.346029+0000","last_fullsized":"2024-09-05T06:41:15.346029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.993957+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:15.993957+0000","last_peered":"2024-09-05T06:41:15.993957+0000","last_clean":"2024-09-05T06:41:15.993957+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:15.993957+0000","last_undegraded":"2024-09-05T06:41:15.993957+0000","last_fullsized":"2024-09-05T06:41:15.993957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379740+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379740+0000","last_peered":"2024-09-05T06:41:15.379740+0000","last_clean":"2024-09-05T06:41:15.379740+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379740+0000","last_undegraded":"2024-09-05T06:41:15.379740+0000","last_fullsized":"2024-09-05T06:41:15.379740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390118+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390118+0000","last_peered":"2024-09-05T06:41:15.390118+0000","last_clean":"2024-09-05T06:41:15.390118+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390118+0000","last_undegraded":"2024-09-05T06:41:15.390118+0000","last_fullsized":"2024-09-05T06:41:15.390118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391961+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.391961+0000","last_peered":"2024-09-05T06:41:15.391961+0000","last_clean":"2024-09-05T06:41:15.391961+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.391961+0000","last_undegraded":"2024-09-05T06:41:15.391961+0000","last_fullsized":"2024-09-05T06:41:15.391961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386428+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386428+0000","last_peered":"2024-09-05T06:41:15.386428+0000","last_clean":"2024-09-05T06:41:15.386428+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386428+0000","last_undegraded":"2024-09-05T06:41:15.386428+0000","last_fullsized":"2024-09-05T06:41:15.386428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386857+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386857+0000","last_peered":"2024-09-05T06:41:15.386857+0000","last_clean":"2024-09-05T06:41:15.386857+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386857+0000","last_undegraded":"2024-09-05T06:41:15.386857+0000","last_fullsized":"2024-09-05T06:41:15.386857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393044+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.393044+0000","last_peered":"2024-09-05T06:41:15.393044+0000","last_clean":"2024-09-05T06:41:15.393044+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.393044+0000","last_undegraded":"2024-09-05T06:41:15.393044+0000","last_fullsized":"2024-09-05T06:41:15.393044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386116+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386116+0000","last_peered":"2024-09-05T06:41:15.386116+0000","last_clean":"2024-09-05T06:41:15.386116+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386116+0000","last_undegraded":"2024-09-05T06:41:15.386116+0000","last_fullsized":"2024-09-05T06:41:15.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383635+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383635+0000","last_peered":"2024-09-05T06:41:15.383635+0000","last_clean":"2024-09-05T06:41:15.383635+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383635+0000","last_undegraded":"2024-09-05T06:41:15.383635+0000","last_fullsized":"2024-09-05T06:41:15.383635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383994+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.383994+0000","last_peered":"2024-09-05T06:41:15.383994+0000","last_clean":"2024-09-05T06:41:15.383994+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.383994+0000","last_undegraded":"2024-09-05T06:41:15.383994+0000","last_fullsized":"2024-09-05T06:41:15.383994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.376779+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.376779+0000","last_peered":"2024-09-05T06:41:15.376779+0000","last_clean":"2024-09-05T06:41:15.376779+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.376779+0000","last_undegraded":"2024-09-05T06:41:15.376779+0000","last_fullsized":"2024-09-05T06:41:15.376779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385732+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.385732+0000","last_peered":"2024-09-05T06:41:15.385732+0000","last_clean":"2024-09-05T06:41:15.385732+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.385732+0000","last_undegraded":"2024-09-05T06:41:15.385732+0000","last_fullsized":"2024-09-05T06:41:15.385732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600666,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1182,"num_read_kb":707,"num_write":383,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1801998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841582,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1744896,"data_stored":1173233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":524606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1119547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1810432,"data_stored":1245598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1130496,"data_stored":586711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1040538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":583133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":199959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:20.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T06:41:21.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- 172.21.15.86:0/2445250634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 msgr2=0x7f9c4c10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 --2- 172.21.15.86:0/2445250634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c10d860 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009930 tx=0x7f9c3c02a420 comp rx=0 tx=0).stop 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- 172.21.15.86:0/2445250634 shutdown_connections 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 --2- 172.21.15.86:0/2445250634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c10d860 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 --2- 172.21.15.86:0/2445250634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 --2- 172.21.15.86:0/2445250634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- 172.21.15.86:0/2445250634 >> 172.21.15.86:0/2445250634 conn(0x7f9c4c0fc6d0 msgr2=0x7f9c4c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- 172.21.15.86:0/2445250634 shutdown_connections 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- 172.21.15.86:0/2445250634 wait complete. 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 Processor -- start 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.022+0000 7f9c513c4640 1 -- start start 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c110c90 con 0x7f9c4c101330 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c110b10 con 0x7f9c4c100990 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c110e10 con 0x7f9c4c1069a0 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59114/0 (socket says 172.21.15.86:59114) 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 -- 172.21.15.86:0/1278023462 learned_addr learned my addr 172.21.15.86:0/1278023462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4b7fe640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4affd640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 -- 172.21.15.86:0/1278023462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 msgr2=0x7f9c4c19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 -- 172.21.15.86:0/1278023462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 msgr2=0x7f9c4c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 -- 172.21.15.86:0/1278023462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c19c1b0 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4affd640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4b7fe640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c4a7fc640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007b50 tx=0x7f9c3400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34007c40 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c34007de0 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c19c4a0 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400ad40 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.023+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c06a400 con 0x7f9c4c101330 2024-09-05T06:41:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.024+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c100df0 con 0x7f9c4c101330 2024-09-05T06:41:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.025+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f9c3400eb30 con 0x7f9c4c101330 2024-09-05T06:41:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.026+0000 7f9c43fff640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 0x7f9c380a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.026+0000 7f9c4affd640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 0x7f9c380a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.026+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9c340d10b0 con 0x7f9c4c101330 2024-09-05T06:41:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.027+0000 7f9c4affd640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 0x7f9c380a52d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007eb0 tx=0x7f9c440073c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:21.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.031+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c34023060 con 0x7f9c4c101330 2024-09-05T06:41:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.194+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c4c06ae80 con 0x7f9c380a2e10 2024-09-05T06:41:21.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.202+0000 7f9c43fff640 1 -- 172.21.15.86:0/1278023462 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339593 (secure 0 0 0) 0x7f9c4c06ae80 con 0x7f9c380a2e10 2024-09-05T06:41:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 msgr2=0x7f9c380a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 0x7f9c380a52d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007eb0 tx=0x7f9c440073c0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 msgr2=0x7f9c4c197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007b50 tx=0x7f9c3400e420 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 shutdown_connections 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c380a2e10 0x7f9c380a52d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c4c1069a0 0x7f9c4c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c101330 0x7f9c4c197700 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 --2- 172.21.15.86:0/1278023462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c4c100990 0x7f9c4c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 >> 172.21.15.86:0/1278023462 conn(0x7f9c4c0fc6d0 msgr2=0x7f9c4c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 shutdown_connections 2024-09-05T06:41:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.206+0000 7f9c513c4640 1 -- 172.21.15.86:0/1278023462 wait complete. 2024-09-05T06:41:21.217 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-05T06:41:19.349648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191268,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1318,"num_read_kb":826,"num_write":566,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389708,"kb_used_data":15464,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552116,"statfs":{"total":1151940427776,"available":1151541366784,"internally_reserved":0,"allocated":15835136,"data_stored":9262593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.993319"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375225+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:15.375225+0000","last_peered":"2024-09-05T06:41:15.375225+0000","last_clean":"2024-09-05T06:41:15.375225+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:15.375225+0000","last_undegraded":"2024-09-05T06:41:15.375225+0000","last_fullsized":"2024-09-05T06:41:15.375225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354898+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354898+0000","last_peered":"2024-09-05T06:41:15.354898+0000","last_clean":"2024-09-05T06:41:15.354898+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354898+0000","last_undegraded":"2024-09-05T06:41:15.354898+0000","last_fullsized":"2024-09-05T06:41:15.354898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383888+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.383888+0000","last_peered":"2024-09-05T06:41:15.383888+0000","last_clean":"2024-09-05T06:41:15.383888+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.383888+0000","last_undegraded":"2024-09-05T06:41:15.383888+0000","last_fullsized":"2024-09-05T06:41:15.383888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350604+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350604+0000","last_peered":"2024-09-05T06:41:15.350604+0000","last_clean":"2024-09-05T06:41:15.350604+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350604+0000","last_undegraded":"2024-09-05T06:41:15.350604+0000","last_fullsized":"2024-09-05T06:41:15.350604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385607+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385607+0000","last_peered":"2024-09-05T06:41:15.385607+0000","last_clean":"2024-09-05T06:41:15.385607+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385607+0000","last_undegraded":"2024-09-05T06:41:15.385607+0000","last_fullsized":"2024-09-05T06:41:15.385607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345930+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345930+0000","last_peered":"2024-09-05T06:41:15.345930+0000","last_clean":"2024-09-05T06:41:15.345930+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345930+0000","last_undegraded":"2024-09-05T06:41:15.345930+0000","last_fullsized":"2024-09-05T06:41:15.345930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391262+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391262+0000","last_peered":"2024-09-05T06:41:15.391262+0000","last_clean":"2024-09-05T06:41:15.391262+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391262+0000","last_undegraded":"2024-09-05T06:41:15.391262+0000","last_fullsized":"2024-09-05T06:41:15.391262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345709+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345709+0000","last_peered":"2024-09-05T06:41:15.345709+0000","last_clean":"2024-09-05T06:41:15.345709+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345709+0000","last_undegraded":"2024-09-05T06:41:15.345709+0000","last_fullsized":"2024-09-05T06:41:15.345709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346004+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346004+0000","last_peered":"2024-09-05T06:41:15.346004+0000","last_clean":"2024-09-05T06:41:15.346004+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346004+0000","last_undegraded":"2024-09-05T06:41:15.346004+0000","last_fullsized":"2024-09-05T06:41:15.346004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'50","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.569551+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:15.569551+0000","last_peered":"2024-09-05T06:41:15.569551+0000","last_clean":"2024-09-05T06:41:15.569551+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:15.569551+0000","last_undegraded":"2024-09-05T06:41:15.569551+0000","last_fullsized":"2024-09-05T06:41:15.569551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.004241+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:16.004241+0000","last_peered":"2024-09-05T06:41:16.004241+0000","last_clean":"2024-09-05T06:41:16.004241+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:16.004241+0000","last_undegraded":"2024-09-05T06:41:16.004241+0000","last_fullsized":"2024-09-05T06:41:16.004241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336294+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:15.336294+0000","last_peered":"2024-09-05T06:41:15.336294+0000","last_clean":"2024-09-05T06:41:15.336294+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:15.336294+0000","last_undegraded":"2024-09-05T06:41:15.336294+0000","last_fullsized":"2024-09-05T06:41:15.336294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338545+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:15.338545+0000","last_peered":"2024-09-05T06:41:15.338545+0000","last_clean":"2024-09-05T06:41:15.338545+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:15.338545+0000","last_undegraded":"2024-09-05T06:41:15.338545+0000","last_fullsized":"2024-09-05T06:41:15.338545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.062661+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:16.062661+0000","last_peered":"2024-09-05T06:41:16.062661+0000","last_clean":"2024-09-05T06:41:16.062661+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:16.062661+0000","last_undegraded":"2024-09-05T06:41:16.062661+0000","last_fullsized":"2024-09-05T06:41:16.062661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.566938+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:15.566938+0000","last_peered":"2024-09-05T06:41:15.566938+0000","last_clean":"2024-09-05T06:41:15.566938+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:15.566938+0000","last_undegraded":"2024-09-05T06:41:15.566938+0000","last_fullsized":"2024-09-05T06:41:15.566938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336610+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336610+0000","last_peered":"2024-09-05T06:41:15.336610+0000","last_clean":"2024-09-05T06:41:15.336610+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336610+0000","last_undegraded":"2024-09-05T06:41:15.336610+0000","last_fullsized":"2024-09-05T06:41:15.336610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.994711+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:15.994711+0000","last_peered":"2024-09-05T06:41:15.994711+0000","last_clean":"2024-09-05T06:41:15.994711+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:15.994711+0000","last_undegraded":"2024-09-05T06:41:15.994711+0000","last_fullsized":"2024-09-05T06:41:15.994711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390252+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390252+0000","last_peered":"2024-09-05T06:41:15.390252+0000","last_clean":"2024-09-05T06:41:15.390252+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390252+0000","last_undegraded":"2024-09-05T06:41:15.390252+0000","last_fullsized":"2024-09-05T06:41:15.390252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353154+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353154+0000","last_peered":"2024-09-05T06:41:15.353154+0000","last_clean":"2024-09-05T06:41:15.353154+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353154+0000","last_undegraded":"2024-09-05T06:41:15.353154+0000","last_fullsized":"2024-09-05T06:41:15.353154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381394+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381394+0000","last_peered":"2024-09-05T06:41:15.381394+0000","last_clean":"2024-09-05T06:41:15.381394+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381394+0000","last_undegraded":"2024-09-05T06:41:15.381394+0000","last_fullsized":"2024-09-05T06:41:15.381394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336176+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336176+0000","last_peered":"2024-09-05T06:41:15.336176+0000","last_clean":"2024-09-05T06:41:15.336176+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336176+0000","last_undegraded":"2024-09-05T06:41:15.336176+0000","last_fullsized":"2024-09-05T06:41:15.336176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368864+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368864+0000","last_peered":"2024-09-05T06:41:15.368864+0000","last_clean":"2024-09-05T06:41:15.368864+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368864+0000","last_undegraded":"2024-09-05T06:41:15.368864+0000","last_fullsized":"2024-09-05T06:41:15.368864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":317,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.994479+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:15.994479+0000","last_peered":"2024-09-05T06:41:15.994479+0000","last_clean":"2024-09-05T06:41:15.994479+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:15.994479+0000","last_undegraded":"2024-09-05T06:41:15.994479+0000","last_fullsized":"2024-09-05T06:41:15.994479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65201,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357230+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357230+0000","last_peered":"2024-09-05T06:41:15.357230+0000","last_clean":"2024-09-05T06:41:15.357230+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357230+0000","last_undegraded":"2024-09-05T06:41:15.357230+0000","last_fullsized":"2024-09-05T06:41:15.357230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384938+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.384938+0000","last_peered":"2024-09-05T06:41:15.384938+0000","last_clean":"2024-09-05T06:41:15.384938+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.384938+0000","last_undegraded":"2024-09-05T06:41:15.384938+0000","last_fullsized":"2024-09-05T06:41:15.384938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346385+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346385+0000","last_peered":"2024-09-05T06:41:15.346385+0000","last_clean":"2024-09-05T06:41:15.346385+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346385+0000","last_undegraded":"2024-09-05T06:41:15.346385+0000","last_fullsized":"2024-09-05T06:41:15.346385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392388+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392388+0000","last_peered":"2024-09-05T06:41:15.392388+0000","last_clean":"2024-09-05T06:41:15.392388+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392388+0000","last_undegraded":"2024-09-05T06:41:15.392388+0000","last_fullsized":"2024-09-05T06:41:15.392388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356999+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.356999+0000","last_peered":"2024-09-05T06:41:15.356999+0000","last_clean":"2024-09-05T06:41:15.356999+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.356999+0000","last_undegraded":"2024-09-05T06:41:15.356999+0000","last_fullsized":"2024-09-05T06:41:15.356999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381534+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.381534+0000","last_peered":"2024-09-05T06:41:15.381534+0000","last_clean":"2024-09-05T06:41:15.381534+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.381534+0000","last_undegraded":"2024-09-05T06:41:15.381534+0000","last_fullsized":"2024-09-05T06:41:15.381534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":254,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.993502+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:15.993502+0000","last_peered":"2024-09-05T06:41:15.993502+0000","last_clean":"2024-09-05T06:41:15.993502+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:15.993502+0000","last_undegraded":"2024-09-05T06:41:15.993502+0000","last_fullsized":"2024-09-05T06:41:15.993502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351143+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351143+0000","last_peered":"2024-09-05T06:41:15.351143+0000","last_clean":"2024-09-05T06:41:15.351143+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351143+0000","last_undegraded":"2024-09-05T06:41:15.351143+0000","last_fullsized":"2024-09-05T06:41:15.351143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388662+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388662+0000","last_peered":"2024-09-05T06:41:15.388662+0000","last_clean":"2024-09-05T06:41:15.388662+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388662+0000","last_undegraded":"2024-09-05T06:41:15.388662+0000","last_fullsized":"2024-09-05T06:41:15.388662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335966+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.335966+0000","last_peered":"2024-09-05T06:41:15.335966+0000","last_clean":"2024-09-05T06:41:15.335966+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.335966+0000","last_undegraded":"2024-09-05T06:41:15.335966+0000","last_fullsized":"2024-09-05T06:41:15.335966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.378926+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.378926+0000","last_peered":"2024-09-05T06:41:15.378926+0000","last_clean":"2024-09-05T06:41:15.378926+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.378926+0000","last_undegraded":"2024-09-05T06:41:15.378926+0000","last_fullsized":"2024-09-05T06:41:15.378926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336108+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336108+0000","last_peered":"2024-09-05T06:41:15.336108+0000","last_clean":"2024-09-05T06:41:15.336108+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336108+0000","last_undegraded":"2024-09-05T06:41:15.336108+0000","last_fullsized":"2024-09-05T06:41:15.336108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346029+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346029+0000","last_peered":"2024-09-05T06:41:15.346029+0000","last_clean":"2024-09-05T06:41:15.346029+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346029+0000","last_undegraded":"2024-09-05T06:41:15.346029+0000","last_fullsized":"2024-09-05T06:41:15.346029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.993957+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:15.993957+0000","last_peered":"2024-09-05T06:41:15.993957+0000","last_clean":"2024-09-05T06:41:15.993957+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:15.993957+0000","last_undegraded":"2024-09-05T06:41:15.993957+0000","last_fullsized":"2024-09-05T06:41:15.993957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379740+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379740+0000","last_peered":"2024-09-05T06:41:15.379740+0000","last_clean":"2024-09-05T06:41:15.379740+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379740+0000","last_undegraded":"2024-09-05T06:41:15.379740+0000","last_fullsized":"2024-09-05T06:41:15.379740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390118+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390118+0000","last_peered":"2024-09-05T06:41:15.390118+0000","last_clean":"2024-09-05T06:41:15.390118+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390118+0000","last_undegraded":"2024-09-05T06:41:15.390118+0000","last_fullsized":"2024-09-05T06:41:15.390118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391961+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.391961+0000","last_peered":"2024-09-05T06:41:15.391961+0000","last_clean":"2024-09-05T06:41:15.391961+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.391961+0000","last_undegraded":"2024-09-05T06:41:15.391961+0000","last_fullsized":"2024-09-05T06:41:15.391961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386428+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386428+0000","last_peered":"2024-09-05T06:41:15.386428+0000","last_clean":"2024-09-05T06:41:15.386428+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386428+0000","last_undegraded":"2024-09-05T06:41:15.386428+0000","last_fullsized":"2024-09-05T06:41:15.386428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386857+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386857+0000","last_peered":"2024-09-05T06:41:15.386857+0000","last_clean":"2024-09-05T06:41:15.386857+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386857+0000","last_undegraded":"2024-09-05T06:41:15.386857+0000","last_fullsized":"2024-09-05T06:41:15.386857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393044+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.393044+0000","last_peered":"2024-09-05T06:41:15.393044+0000","last_clean":"2024-09-05T06:41:15.393044+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.393044+0000","last_undegraded":"2024-09-05T06:41:15.393044+0000","last_fullsized":"2024-09-05T06:41:15.393044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386116+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386116+0000","last_peered":"2024-09-05T06:41:15.386116+0000","last_clean":"2024-09-05T06:41:15.386116+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386116+0000","last_undegraded":"2024-09-05T06:41:15.386116+0000","last_fullsized":"2024-09-05T06:41:15.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383635+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383635+0000","last_peered":"2024-09-05T06:41:15.383635+0000","last_clean":"2024-09-05T06:41:15.383635+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383635+0000","last_undegraded":"2024-09-05T06:41:15.383635+0000","last_fullsized":"2024-09-05T06:41:15.383635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383994+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.383994+0000","last_peered":"2024-09-05T06:41:15.383994+0000","last_clean":"2024-09-05T06:41:15.383994+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.383994+0000","last_undegraded":"2024-09-05T06:41:15.383994+0000","last_fullsized":"2024-09-05T06:41:15.383994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.376779+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.376779+0000","last_peered":"2024-09-05T06:41:15.376779+0000","last_clean":"2024-09-05T06:41:15.376779+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.376779+0000","last_undegraded":"2024-09-05T06:41:15.376779+0000","last_fullsized":"2024-09-05T06:41:15.376779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385732+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.385732+0000","last_peered":"2024-09-05T06:41:15.385732+0000","last_clean":"2024-09-05T06:41:15.385732+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.385732+0000","last_undegraded":"2024-09-05T06:41:15.385732+0000","last_fullsized":"2024-09-05T06:41:15.385732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600666,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1182,"num_read_kb":707,"num_write":383,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1801998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841582,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1744896,"data_stored":1173233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":524606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1119547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1810432,"data_stored":1245598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1130496,"data_stored":586711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1040538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":583133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":199959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:21.274 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-05T06:41:21.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T06:41:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 -- 172.21.15.86:0/1422352852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 msgr2=0x7f11d0106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 --2- 172.21.15.86:0/1422352852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d0106d70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009930 tx=0x7f11c002a420 comp rx=0 tx=0).stop 2024-09-05T06:41:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 -- 172.21.15.86:0/1422352852 shutdown_connections 2024-09-05T06:41:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 --2- 172.21.15.86:0/1422352852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 0x7f11d010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 --2- 172.21.15.86:0/1422352852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 0x7f11d0100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 --2- 172.21.15.86:0/1422352852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d0106d70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 -- 172.21.15.86:0/1422352852 >> 172.21.15.86:0/1422352852 conn(0x7f11d0077a10 msgr2=0x7f11d00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 -- 172.21.15.86:0/1422352852 shutdown_connections 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.376+0000 7f11d5465640 1 -- 172.21.15.86:0/1422352852 wait complete. 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 Processor -- start 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 -- start start 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 0x7f11d0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 0x7f11d01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0110c50 con 0x7f11d0106990 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0110ad0 con 0x7f11d0100980 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11d5465640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0110dd0 con 0x7f11d0101320 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59124/0 (socket says 172.21.15.86:59124) 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11cf7fe640 1 -- 172.21.15.86:0/3171516548 learned_addr learned my addr 172.21.15.86:0/3171516548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11ce7fc640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 0x7f11d01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11ceffd640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 0x7f11d0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11cf7fe640 1 -- 172.21.15.86:0/3171516548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 msgr2=0x7f11d01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.377+0000 7f11cf7fe640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 0x7f11d01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11cf7fe640 1 -- 172.21.15.86:0/3171516548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 msgr2=0x7f11d0199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11cf7fe640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 0x7f11d0199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11cf7fe640 1 -- 172.21.15.86:0/3171516548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a4720 con 0x7f11d0106990 2024-09-05T06:41:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11cf7fe640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f11c800eaa0 tx=0x7f11c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800cdb0 con 0x7f11d0106990 2024-09-05T06:41:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11c8005020 con 0x7f11d0106990 2024-09-05T06:41:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a48f0 con 0x7f11d0106990 2024-09-05T06:41:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c80116d0 con 0x7f11d0106990 2024-09-05T06:41:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.378+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a4df0 con 0x7f11d0106990 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.379+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0100de0 con 0x7f11d0106990 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.380+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f11c80051d0 con 0x7f11d0106990 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.381+0000 7f11b7fff640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 0x7f11b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.381+0000 7f11ceffd640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 0x7f11b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.381+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f11c80cfad0 con 0x7f11d0106990 2024-09-05T06:41:21.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.382+0000 7f11ceffd640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 0x7f11b80a5250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c0000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.385+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11c8016030 con 0x7f11d0106990 2024-09-05T06:41:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.548+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11d0062630 con 0x7f11b80a2d90 2024-09-05T06:41:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.556+0000 7f11b7fff640 1 -- 172.21.15.86:0/3171516548 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339534 (secure 0 0 0) 0x7f11d0062630 con 0x7f11b80a2d90 2024-09-05T06:41:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:21.561 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 msgr2=0x7f11b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 0x7f11b80a5250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c0000bc0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 msgr2=0x7f11d01a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f11c800eaa0 tx=0x7f11c800ef60 comp rx=0 tx=0).stop 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 shutdown_connections 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11b80a2d90 0x7f11b80a5250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0106990 0x7f11d01a41e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0101320 0x7f11d01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 --2- 172.21.15.86:0/3171516548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d0100980 0x7f11d0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:21.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 >> 172.21.15.86:0/3171516548 conn(0x7f11d0077a10 msgr2=0x7f11d0103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:21.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 shutdown_connections 2024-09-05T06:41:21.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:21.560+0000 7f11d5465640 1 -- 172.21.15.86:0/3171516548 wait complete. 2024-09-05T06:41:21.570 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-05T06:41:21.350034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195582,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1344,"num_read_kb":846,"num_write":567,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":948,"ondisk_log_size":948,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389844,"kb_used_data":15604,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551980,"statfs":{"total":1151940427776,"available":1151541227520,"internally_reserved":0,"allocated":15978496,"data_stored":9403607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.993704"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375225+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:15.375225+0000","last_peered":"2024-09-05T06:41:15.375225+0000","last_clean":"2024-09-05T06:41:15.375225+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:15.375225+0000","last_undegraded":"2024-09-05T06:41:15.375225+0000","last_fullsized":"2024-09-05T06:41:15.375225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354898+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354898+0000","last_peered":"2024-09-05T06:41:15.354898+0000","last_clean":"2024-09-05T06:41:15.354898+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354898+0000","last_undegraded":"2024-09-05T06:41:15.354898+0000","last_fullsized":"2024-09-05T06:41:15.354898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383888+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.383888+0000","last_peered":"2024-09-05T06:41:15.383888+0000","last_clean":"2024-09-05T06:41:15.383888+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.383888+0000","last_undegraded":"2024-09-05T06:41:15.383888+0000","last_fullsized":"2024-09-05T06:41:15.383888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350604+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350604+0000","last_peered":"2024-09-05T06:41:15.350604+0000","last_clean":"2024-09-05T06:41:15.350604+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350604+0000","last_undegraded":"2024-09-05T06:41:15.350604+0000","last_fullsized":"2024-09-05T06:41:15.350604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385607+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385607+0000","last_peered":"2024-09-05T06:41:15.385607+0000","last_clean":"2024-09-05T06:41:15.385607+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385607+0000","last_undegraded":"2024-09-05T06:41:15.385607+0000","last_fullsized":"2024-09-05T06:41:15.385607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345930+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345930+0000","last_peered":"2024-09-05T06:41:15.345930+0000","last_clean":"2024-09-05T06:41:15.345930+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345930+0000","last_undegraded":"2024-09-05T06:41:15.345930+0000","last_fullsized":"2024-09-05T06:41:15.345930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391262+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391262+0000","last_peered":"2024-09-05T06:41:15.391262+0000","last_clean":"2024-09-05T06:41:15.391262+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391262+0000","last_undegraded":"2024-09-05T06:41:15.391262+0000","last_fullsized":"2024-09-05T06:41:15.391262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345709+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345709+0000","last_peered":"2024-09-05T06:41:15.345709+0000","last_clean":"2024-09-05T06:41:15.345709+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345709+0000","last_undegraded":"2024-09-05T06:41:15.345709+0000","last_fullsized":"2024-09-05T06:41:15.345709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346004+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346004+0000","last_peered":"2024-09-05T06:41:15.346004+0000","last_clean":"2024-09-05T06:41:15.346004+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346004+0000","last_undegraded":"2024-09-05T06:41:15.346004+0000","last_fullsized":"2024-09-05T06:41:15.346004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'50","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.569551+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:15.569551+0000","last_peered":"2024-09-05T06:41:15.569551+0000","last_clean":"2024-09-05T06:41:15.569551+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:15.569551+0000","last_undegraded":"2024-09-05T06:41:15.569551+0000","last_fullsized":"2024-09-05T06:41:15.569551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.004241+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:16.004241+0000","last_peered":"2024-09-05T06:41:16.004241+0000","last_clean":"2024-09-05T06:41:16.004241+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:16.004241+0000","last_undegraded":"2024-09-05T06:41:16.004241+0000","last_fullsized":"2024-09-05T06:41:16.004241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336294+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:15.336294+0000","last_peered":"2024-09-05T06:41:15.336294+0000","last_clean":"2024-09-05T06:41:15.336294+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:15.336294+0000","last_undegraded":"2024-09-05T06:41:15.336294+0000","last_fullsized":"2024-09-05T06:41:15.336294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338545+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:15.338545+0000","last_peered":"2024-09-05T06:41:15.338545+0000","last_clean":"2024-09-05T06:41:15.338545+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:15.338545+0000","last_undegraded":"2024-09-05T06:41:15.338545+0000","last_fullsized":"2024-09-05T06:41:15.338545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:16.062661+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:16.062661+0000","last_peered":"2024-09-05T06:41:16.062661+0000","last_clean":"2024-09-05T06:41:16.062661+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:16.062661+0000","last_undegraded":"2024-09-05T06:41:16.062661+0000","last_fullsized":"2024-09-05T06:41:16.062661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.566938+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:15.566938+0000","last_peered":"2024-09-05T06:41:15.566938+0000","last_clean":"2024-09-05T06:41:15.566938+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:15.566938+0000","last_undegraded":"2024-09-05T06:41:15.566938+0000","last_fullsized":"2024-09-05T06:41:15.566938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336610+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336610+0000","last_peered":"2024-09-05T06:41:15.336610+0000","last_clean":"2024-09-05T06:41:15.336610+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336610+0000","last_undegraded":"2024-09-05T06:41:15.336610+0000","last_fullsized":"2024-09-05T06:41:15.336610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.994711+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:15.994711+0000","last_peered":"2024-09-05T06:41:15.994711+0000","last_clean":"2024-09-05T06:41:15.994711+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:15.994711+0000","last_undegraded":"2024-09-05T06:41:15.994711+0000","last_fullsized":"2024-09-05T06:41:15.994711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390252+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390252+0000","last_peered":"2024-09-05T06:41:15.390252+0000","last_clean":"2024-09-05T06:41:15.390252+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390252+0000","last_undegraded":"2024-09-05T06:41:15.390252+0000","last_fullsized":"2024-09-05T06:41:15.390252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353154+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353154+0000","last_peered":"2024-09-05T06:41:15.353154+0000","last_clean":"2024-09-05T06:41:15.353154+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353154+0000","last_undegraded":"2024-09-05T06:41:15.353154+0000","last_fullsized":"2024-09-05T06:41:15.353154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381394+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381394+0000","last_peered":"2024-09-05T06:41:15.381394+0000","last_clean":"2024-09-05T06:41:15.381394+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381394+0000","last_undegraded":"2024-09-05T06:41:15.381394+0000","last_fullsized":"2024-09-05T06:41:15.381394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336176+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336176+0000","last_peered":"2024-09-05T06:41:15.336176+0000","last_clean":"2024-09-05T06:41:15.336176+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336176+0000","last_undegraded":"2024-09-05T06:41:15.336176+0000","last_fullsized":"2024-09-05T06:41:15.336176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368864+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368864+0000","last_peered":"2024-09-05T06:41:15.368864+0000","last_clean":"2024-09-05T06:41:15.368864+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368864+0000","last_undegraded":"2024-09-05T06:41:15.368864+0000","last_fullsized":"2024-09-05T06:41:15.368864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":329,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:21.000874+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:21.000874+0000","last_peered":"2024-09-05T06:41:21.000874+0000","last_clean":"2024-09-05T06:41:21.000874+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:21.000874+0000","last_undegraded":"2024-09-05T06:41:21.000874+0000","last_fullsized":"2024-09-05T06:41:21.000874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69515,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":184,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357230+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357230+0000","last_peered":"2024-09-05T06:41:15.357230+0000","last_clean":"2024-09-05T06:41:15.357230+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357230+0000","last_undegraded":"2024-09-05T06:41:15.357230+0000","last_fullsized":"2024-09-05T06:41:15.357230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385754+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.385754+0000","last_peered":"2024-09-05T06:41:15.385754+0000","last_clean":"2024-09-05T06:41:15.385754+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.385754+0000","last_undegraded":"2024-09-05T06:41:15.385754+0000","last_fullsized":"2024-09-05T06:41:15.385754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346385+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346385+0000","last_peered":"2024-09-05T06:41:15.346385+0000","last_clean":"2024-09-05T06:41:15.346385+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346385+0000","last_undegraded":"2024-09-05T06:41:15.346385+0000","last_fullsized":"2024-09-05T06:41:15.346385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392388+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392388+0000","last_peered":"2024-09-05T06:41:15.392388+0000","last_clean":"2024-09-05T06:41:15.392388+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392388+0000","last_undegraded":"2024-09-05T06:41:15.392388+0000","last_fullsized":"2024-09-05T06:41:15.392388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356999+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.356999+0000","last_peered":"2024-09-05T06:41:15.356999+0000","last_clean":"2024-09-05T06:41:15.356999+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.356999+0000","last_undegraded":"2024-09-05T06:41:15.356999+0000","last_fullsized":"2024-09-05T06:41:15.356999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":265,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:21.069083+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:21.069083+0000","last_peered":"2024-09-05T06:41:21.069083+0000","last_clean":"2024-09-05T06:41:21.069083+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:21.069083+0000","last_undegraded":"2024-09-05T06:41:21.069083+0000","last_fullsized":"2024-09-05T06:41:21.069083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351201+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351201+0000","last_peered":"2024-09-05T06:41:15.351201+0000","last_clean":"2024-09-05T06:41:15.351201+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351201+0000","last_undegraded":"2024-09-05T06:41:15.351201+0000","last_fullsized":"2024-09-05T06:41:15.351201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388662+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388662+0000","last_peered":"2024-09-05T06:41:15.388662+0000","last_clean":"2024-09-05T06:41:15.388662+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388662+0000","last_undegraded":"2024-09-05T06:41:15.388662+0000","last_fullsized":"2024-09-05T06:41:15.388662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336013+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.336013+0000","last_peered":"2024-09-05T06:41:15.336013+0000","last_clean":"2024-09-05T06:41:15.336013+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.336013+0000","last_undegraded":"2024-09-05T06:41:15.336013+0000","last_fullsized":"2024-09-05T06:41:15.336013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379081+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.379081+0000","last_peered":"2024-09-05T06:41:15.379081+0000","last_clean":"2024-09-05T06:41:15.379081+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.379081+0000","last_undegraded":"2024-09-05T06:41:15.379081+0000","last_fullsized":"2024-09-05T06:41:15.379081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336108+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336108+0000","last_peered":"2024-09-05T06:41:15.336108+0000","last_clean":"2024-09-05T06:41:15.336108+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336108+0000","last_undegraded":"2024-09-05T06:41:15.336108+0000","last_fullsized":"2024-09-05T06:41:15.336108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346070+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346070+0000","last_peered":"2024-09-05T06:41:15.346070+0000","last_clean":"2024-09-05T06:41:15.346070+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346070+0000","last_undegraded":"2024-09-05T06:41:15.346070+0000","last_fullsized":"2024-09-05T06:41:15.346070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":164,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:19.999447+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:19.999447+0000","last_peered":"2024-09-05T06:41:19.999447+0000","last_clean":"2024-09-05T06:41:19.999447+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:19.999447+0000","last_undegraded":"2024-09-05T06:41:19.999447+0000","last_fullsized":"2024-09-05T06:41:19.999447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379740+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379740+0000","last_peered":"2024-09-05T06:41:15.379740+0000","last_clean":"2024-09-05T06:41:15.379740+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379740+0000","last_undegraded":"2024-09-05T06:41:15.379740+0000","last_fullsized":"2024-09-05T06:41:15.379740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390219+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390219+0000","last_peered":"2024-09-05T06:41:15.390219+0000","last_clean":"2024-09-05T06:41:15.390219+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390219+0000","last_undegraded":"2024-09-05T06:41:15.390219+0000","last_fullsized":"2024-09-05T06:41:15.390219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392054+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.392054+0000","last_peered":"2024-09-05T06:41:15.392054+0000","last_clean":"2024-09-05T06:41:15.392054+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.392054+0000","last_undegraded":"2024-09-05T06:41:15.392054+0000","last_fullsized":"2024-09-05T06:41:15.392054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386428+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386428+0000","last_peered":"2024-09-05T06:41:15.386428+0000","last_clean":"2024-09-05T06:41:15.386428+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386428+0000","last_undegraded":"2024-09-05T06:41:15.386428+0000","last_fullsized":"2024-09-05T06:41:15.386428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386857+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386857+0000","last_peered":"2024-09-05T06:41:15.386857+0000","last_clean":"2024-09-05T06:41:15.386857+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386857+0000","last_undegraded":"2024-09-05T06:41:15.386857+0000","last_fullsized":"2024-09-05T06:41:15.386857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393044+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.393044+0000","last_peered":"2024-09-05T06:41:15.393044+0000","last_clean":"2024-09-05T06:41:15.393044+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.393044+0000","last_undegraded":"2024-09-05T06:41:15.393044+0000","last_fullsized":"2024-09-05T06:41:15.393044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386116+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386116+0000","last_peered":"2024-09-05T06:41:15.386116+0000","last_clean":"2024-09-05T06:41:15.386116+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386116+0000","last_undegraded":"2024-09-05T06:41:15.386116+0000","last_fullsized":"2024-09-05T06:41:15.386116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383635+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383635+0000","last_peered":"2024-09-05T06:41:15.383635+0000","last_clean":"2024-09-05T06:41:15.383635+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383635+0000","last_undegraded":"2024-09-05T06:41:15.383635+0000","last_fullsized":"2024-09-05T06:41:15.383635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383994+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.383994+0000","last_peered":"2024-09-05T06:41:15.383994+0000","last_clean":"2024-09-05T06:41:15.383994+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.383994+0000","last_undegraded":"2024-09-05T06:41:15.383994+0000","last_fullsized":"2024-09-05T06:41:15.383994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.376779+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.376779+0000","last_peered":"2024-09-05T06:41:15.376779+0000","last_clean":"2024-09-05T06:41:15.376779+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.376779+0000","last_undegraded":"2024-09-05T06:41:15.376779+0000","last_fullsized":"2024-09-05T06:41:15.376779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385732+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.385732+0000","last_peered":"2024-09-05T06:41:15.385732+0000","last_clean":"2024-09-05T06:41:15.385732+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.385732+0000","last_undegraded":"2024-09-05T06:41:15.385732+0000","last_fullsized":"2024-09-05T06:41:15.385732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604980,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1208,"num_read_kb":727,"num_write":384,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2170880,"data_stored":1810626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841583,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1884160,"data_stored":1309933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":524606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1119547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1810432,"data_stored":1245598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1130496,"data_stored":586711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1040538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":587447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:21.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:41:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:22.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 -- 172.21.15.86:0/452272254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c102d40 msgr2=0x7efe4c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 --2- 172.21.15.86:0/452272254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c102d40 0x7efe4c103120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009900 tx=0x7efe3c02a480 comp rx=0 tx=0).stop 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 -- 172.21.15.86:0/452272254 shutdown_connections 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 --2- 172.21.15.86:0/452272254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 --2- 172.21.15.86:0/452272254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c103660 0x7efe4c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 --2- 172.21.15.86:0/452272254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c102d40 0x7efe4c103120 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 -- 172.21.15.86:0/452272254 >> 172.21.15.86:0/452272254 conn(0x7efe4c0fc7a0 msgr2=0x7efe4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 -- 172.21.15.86:0/452272254 shutdown_connections 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.368+0000 7efe50886640 1 -- 172.21.15.86:0/452272254 wait complete. 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 Processor -- start 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 -- start start 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c102d40 0x7efe4c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c111e60 con 0x7efe4c103660 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c111ce0 con 0x7efe4c102d40 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe50886640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c111fe0 con 0x7efe4c10c170 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59132/0 (socket says 172.21.15.86:59132) 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe49d9b640 1 -- 172.21.15.86:0/1390475659 learned_addr learned my addr 172.21.15.86:0/1390475659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe4a59c640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c102d40 0x7efe4c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.369+0000 7efe4ad9d640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 -- 172.21.15.86:0/1390475659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 msgr2=0x7efe4c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 -- 172.21.15.86:0/1390475659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c102d40 msgr2=0x7efe4c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c102d40 0x7efe4c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 -- 172.21.15.86:0/1390475659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c19d2f0 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe4ad9d640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe49d9b640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c970 tx=0x7efe4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40007bb0 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c19d5e0 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe40007d50 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40018780 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.370+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a4ec0 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.371+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c103120 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.372+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7efe4000f450 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.373+0000 7efe3b7fe640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 0x7efe340a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.373+0000 7efe4a59c640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 0x7efe340a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.373+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efe400cfd40 con 0x7efe4c103660 2024-09-05T06:41:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.373+0000 7efe4a59c640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 0x7efe340a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe3c031440 tx=0x7efe3c005e80 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.376+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe40014030 con 0x7efe4c103660 2024-09-05T06:41:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.578+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe4c1996a0 con 0x7efe4c103660 2024-09-05T06:41:22.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:22 smithi086 ceph-mon[30943]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:22 smithi086 ceph-mon[30943]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:22 smithi086 ceph-mon[30943]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-09-05T06:41:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.578+0000 7efe3b7fe640 1 -- 172.21.15.86:0/1390475659 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7efe4c1996a0 con 0x7efe4c103660 2024-09-05T06:41:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:22.579 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:41:22.579 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:41:22.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.580+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 msgr2=0x7efe340a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.580+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 0x7efe340a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe3c031440 tx=0x7efe3c005e80 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.580+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 msgr2=0x7efe4c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.580+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c970 tx=0x7efe4000ce30 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 shutdown_connections 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe340a2d80 0x7efe340a5240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c10c170 0x7efe4c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4c103660 0x7efe4c198880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 --2- 172.21.15.86:0/1390475659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe4c102d40 0x7efe4c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 >> 172.21.15.86:0/1390475659 conn(0x7efe4c0fc7a0 msgr2=0x7efe4c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 shutdown_connections 2024-09-05T06:41:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:22.581+0000 7efe50886640 1 -- 172.21.15.86:0/1390475659 wait complete. 2024-09-05T06:41:22.608 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:22 smithi096 ceph-mon[34468]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.608 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:22 smithi096 ceph-mon[34468]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.609 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:22 smithi096 ceph-mon[34468]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-09-05T06:41:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:22 smithi115 ceph-mon[34791]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:22 smithi115 ceph-mon[34791]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:22.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:22 smithi115 ceph-mon[34791]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-09-05T06:41:23.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:41:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:23.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:23 smithi096 ceph-mon[34468]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:23.621 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:23 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1390475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:23.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:23 smithi086 ceph-mon[30943]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:23.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:23 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1390475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:23 smithi115 ceph-mon[34791]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:23 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1390475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- 172.21.15.86:0/4037456129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc10c180 msgr2=0x7f3efc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 --2- 172.21.15.86:0/4037456129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc10c180 0x7f3efc10e600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009ca0 tx=0x7f3ef402a950 comp rx=0 tx=0).stop 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- 172.21.15.86:0/4037456129 shutdown_connections 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 --2- 172.21.15.86:0/4037456129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc10c180 0x7f3efc10e600 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 --2- 172.21.15.86:0/4037456129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc1036c0 0x7f3efc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 --2- 172.21.15.86:0/4037456129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 0x7f3efc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- 172.21.15.86:0/4037456129 >> 172.21.15.86:0/4037456129 conn(0x7f3efc0fc7a0 msgr2=0x7f3efc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- 172.21.15.86:0/4037456129 shutdown_connections 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- 172.21.15.86:0/4037456129 wait complete. 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 Processor -- start 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.034+0000 7f3f02e18640 1 -- start start 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 0x7f3efc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc10c180 0x7f3efc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111e10 con 0x7f3efc1036c0 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111c90 con 0x7f3efc102da0 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111f90 con 0x7f3efc10c180 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44108/0 (socket says 172.21.15.86:44108) 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 -- 172.21.15.86:0/1897197275 learned_addr learned my addr 172.21.15.86:0/1897197275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f00bb4640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 0x7f3efc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 -- 172.21.15.86:0/1897197275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc10c180 msgr2=0x7f3efc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f013b5640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc10c180 0x7f3efc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc10c180 0x7f3efc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 -- 172.21.15.86:0/1897197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 msgr2=0x7f3efc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 0x7f3efc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 -- 172.21.15.86:0/1897197275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19d290 con 0x7f3efc1036c0 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3efbfff640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c970 tx=0x7f3ee800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8007bb0 con 0x7f3efc1036c0 2024-09-05T06:41:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc19d580 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ee8007d50 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.035+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee801d060 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.036+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a4e60 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.037+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc103180 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.038+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3ee8004e70 con 0x7f3efc1036c0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.038+0000 7f3ef9ffb640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 0x7f3eec0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.038+0000 7f3f00bb4640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 0x7f3eec0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.038+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3ee80cf370 con 0x7f3efc1036c0 2024-09-05T06:41:24.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.039+0000 7f3f00bb4640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 0x7f3eec0a5240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005fd0 tx=0x7f3ef000b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:24.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.042+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee8014030 con 0x7f3efc1036c0 2024-09-05T06:41:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.209+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3efc199640 con 0x7f3efc1036c0 2024-09-05T06:41:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.210+0000 7f3ef9ffb640 1 -- 172.21.15.86:0/1897197275 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25345 (secure 0 0 0) 0x7f3efc199640 con 0x7f3efc1036c0 2024-09-05T06:41:24.211 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:24.211 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:41:15.328284+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4062552821":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/2959599766":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/3708924769":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4040819651":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3625555544":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4279196951":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6800/3708924769":"2024-09-06T06:41:15.328251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 msgr2=0x7f3eec0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 0x7f3eec0a5240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005fd0 tx=0x7f3ef000b390 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 msgr2=0x7f3efc198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800c970 tx=0x7f3ee800ce30 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 shutdown_connections 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eec0a2d80 0x7f3eec0a5240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc10c180 0x7f3efc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1036c0 0x7f3efc198820 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 --2- 172.21.15.86:0/1897197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3efc102da0 0x7f3efc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 >> 172.21.15.86:0/1897197275 conn(0x7f3efc0fc7a0 msgr2=0x7f3efc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 shutdown_connections 2024-09-05T06:41:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.213+0000 7f3f02e18640 1 -- 172.21.15.86:0/1897197275 wait complete. 2024-09-05T06:41:24.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:24 smithi086 ceph-mon[30943]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 878 B/s wr, 25 op/s 2024-09-05T06:41:24.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1897197275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:24 smithi115 ceph-mon[34791]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 878 B/s wr, 25 op/s 2024-09-05T06:41:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1897197275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:24.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:24 smithi096 ceph-mon[34468]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 878 B/s wr, 25 op/s 2024-09-05T06:41:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1897197275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:24.863 DEBUG:tasks.fwd_scrub:Yielding 2024-09-05T06:41:24.863 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T06:41:24.873 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T06:41:24.873 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T06:41:24.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-05T06:41:24.873 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-05T06:41:24.873 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:41:24.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:41:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.977+0000 7fa5c790a640 1 -- 172.21.15.86:0/1202589866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01009b0 msgr2=0x7fa5c0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.977+0000 7fa5c790a640 1 --2- 172.21.15.86:0/1202589866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01009b0 0x7fa5c0100e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009930 tx=0x7fa5b402a420 comp rx=0 tx=0).stop 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- 172.21.15.86:0/1202589866 shutdown_connections 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- 172.21.15.86:0/1202589866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c0101350 0x7fa5c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- 172.21.15.86:0/1202589866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c01009b0 0x7fa5c0100e10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- 172.21.15.86:0/1202589866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 0x7fa5c0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- 172.21.15.86:0/1202589866 >> 172.21.15.86:0/1202589866 conn(0x7fa5c0077990 msgr2=0x7fa5c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- 172.21.15.86:0/1202589866 shutdown_connections 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- 172.21.15.86:0/1202589866 wait complete. 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 Processor -- start 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- start start 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 0x7fa5c01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 0x7fa5c019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0110cb0 con 0x7fa5c0101350 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0110b30 con 0x7fa5c01009b0 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.978+0000 7fa5c790a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0110e30 con 0x7fa5c01069c0 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44120/0 (socket says 172.21.15.86:44120) 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 -- 172.21.15.86:0/3412216926 learned_addr learned my addr 172.21.15.86:0/3412216926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c5ea7640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 0x7fa5c019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c56a6640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 0x7fa5c01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 -- 172.21.15.86:0/3412216926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 msgr2=0x7fa5c019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 0x7fa5c019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 -- 172.21.15.86:0/3412216926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 msgr2=0x7fa5c01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 0x7fa5c01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 -- 172.21.15.86:0/3412216926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c019c190 con 0x7fa5c0101350 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c56a6640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 0x7fa5c01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c4ea5640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005940 tx=0x7fa5b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b403f030 con 0x7fa5c0101350 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b40318a0 con 0x7fa5c0101350 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4035070 con 0x7fa5c0101350 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c019c420 con 0x7fa5c0101350 2024-09-05T06:41:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.979+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c01a3d10 con 0x7fa5c0101350 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.980+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c0100e10 con 0x7fa5c0101350 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.982+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fa5b4042030 con 0x7fa5c0101350 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.982+0000 7fa5b27fc640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 0x7fa5ac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.982+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa5b40f03a0 con 0x7fa5c0101350 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.982+0000 7fa5c56a6640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 0x7fa5ac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.983+0000 7fa5c56a6640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 0x7fa5ac0a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ba0 tx=0x7fa5bc005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:24.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:24.986+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b40bca70 con 0x7fa5c0101350 2024-09-05T06:41:25.136 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:25.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.177+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fa5c002d7c0 con 0x7fa5c0101350 2024-09-05T06:41:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.181+0000 7fa5b27fc640 1 -- 172.21.15.86:0/3412216926 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fa5b40bc370 con 0x7fa5c0101350 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 msgr2=0x7fa5ac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 0x7fa5ac0a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ba0 tx=0x7fa5bc005f90 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 msgr2=0x7fa5c01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005940 tx=0x7fa5b40042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 shutdown_connections 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa5ac0a2e40 0x7fa5ac0a5300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c01069c0 0x7fa5c019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101350 0x7fa5c01976e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 --2- 172.21.15.86:0/3412216926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c01009b0 0x7fa5c01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 >> 172.21.15.86:0/3412216926 conn(0x7fa5c0077990 msgr2=0x7fa5c0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 shutdown_connections 2024-09-05T06:41:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.183+0000 7fa5c790a640 1 -- 172.21.15.86:0/3412216926 wait complete. 2024-09-05T06:41:25.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-05T06:41:25.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 -- 172.21.15.86:0/1490757809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881069c0 msgr2=0x7f2288106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 --2- 172.21.15.86:0/1490757809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881069c0 0x7f2288106da0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2278009930 tx=0x7f227802a420 comp rx=0 tx=0).stop 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 -- 172.21.15.86:0/1490757809 shutdown_connections 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 --2- 172.21.15.86:0/1490757809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2288101350 0x7f228810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 --2- 172.21.15.86:0/1490757809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22881009b0 0x7f2288100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 --2- 172.21.15.86:0/1490757809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881069c0 0x7f2288106da0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 -- 172.21.15.86:0/1490757809 >> 172.21.15.86:0/1490757809 conn(0x7f2288077990 msgr2=0x7f22880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 -- 172.21.15.86:0/1490757809 shutdown_connections 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.299+0000 7f228d423640 1 -- 172.21.15.86:0/1490757809 wait complete. 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 Processor -- start 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 -- start start 2024-09-05T06:41:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2288101350 0x7f22881987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 0x7f228819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111da0 con 0x7f22881009b0 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111c20 con 0x7f2288101350 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f228d423640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111f20 con 0x7f22881069c0 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44124/0 (socket says 172.21.15.86:44124) 2024-09-05T06:41:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 -- 172.21.15.86:0/60459363 learned_addr learned my addr 172.21.15.86:0/60459363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f227ffff640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2288101350 0x7f22881987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f22877fe640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 0x7f228819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 -- 172.21.15.86:0/60459363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 msgr2=0x7f228819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 0x7f228819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 -- 172.21.15.86:0/60459363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2288101350 msgr2=0x7f22881987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2288101350 0x7f22881987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f2286ffd640 1 -- 172.21.15.86:0/60459363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819d260 con 0x7f22881009b0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.300+0000 7f22877fe640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 0x7f228819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f2286ffd640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2278004770 tx=0x7f22780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227803f070 con 0x7f22881009b0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f227803b460 con 0x7f22881009b0 2024-09-05T06:41:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819d4f0 con 0x7f22881009b0 2024-09-05T06:41:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278035070 con 0x7f22881009b0 2024-09-05T06:41:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.301+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a4de0 con 0x7f22881009b0 2024-09-05T06:41:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.302+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2288100e10 con 0x7f22881009b0 2024-09-05T06:41:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.303+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f227803b610 con 0x7f22881009b0 2024-09-05T06:41:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.303+0000 7f2284ff9640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 0x7f22740a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.304+0000 7f227ffff640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 0x7f22740a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.304+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f22780f0020 con 0x7f22881009b0 2024-09-05T06:41:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.304+0000 7f227ffff640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 0x7f22740a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f22880fe980 tx=0x7f227000b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.307+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22780bc6f0 con 0x7f22881009b0 2024-09-05T06:41:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.487+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f2288108f20 con 0x7f22881009b0 2024-09-05T06:41:25.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.499+0000 7f2284ff9640 1 -- 172.21.15.86:0/60459363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f22780bbff0 con 0x7f22881009b0 2024-09-05T06:41:25.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 msgr2=0x7f22740a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 0x7f22740a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f22880fe980 tx=0x7f227000b390 comp rx=0 tx=0).stop 2024-09-05T06:41:25.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 msgr2=0x7f2288198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2278004770 tx=0x7f22780042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 shutdown_connections 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f22740a2e40 0x7f22740a5300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22881069c0 0x7f228819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2288101350 0x7f22881987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 --2- 172.21.15.86:0/60459363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22881009b0 0x7f2288198270 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 >> 172.21.15.86:0/60459363 conn(0x7f2288077990 msgr2=0x7f22880ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 shutdown_connections 2024-09-05T06:41:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.501+0000 7f228d423640 1 -- 172.21.15.86:0/60459363 wait complete. 2024-09-05T06:41:25.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- 172.21.15.86:0/3346775221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781069c0 msgr2=0x7f5978106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- 172.21.15.86:0/3346775221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781069c0 0x7f5978106da0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5968009930 tx=0x7f596802a420 comp rx=0 tx=0).stop 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- 172.21.15.86:0/3346775221 shutdown_connections 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- 172.21.15.86:0/3346775221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5978101350 0x7f597810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- 172.21.15.86:0/3346775221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- 172.21.15.86:0/3346775221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781069c0 0x7f5978106da0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- 172.21.15.86:0/3346775221 >> 172.21.15.86:0/3346775221 conn(0x7f5978077990 msgr2=0x7f59780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- 172.21.15.86:0/3346775221 shutdown_connections 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- 172.21.15.86:0/3346775221 wait complete. 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 Processor -- start 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- start start 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59781069c0 0x7f59781a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5978110cb0 con 0x7f5978101350 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5978110b30 con 0x7f59781069c0 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.619+0000 7f597db4b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5978110e30 con 0x7f59781009b0 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44140/0 (socket says 172.21.15.86:44140) 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 -- 172.21.15.86:0/2433894408 learned_addr learned my addr 172.21.15.86:0/2433894408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f59777fe640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 -- 172.21.15.86:0/2433894408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 msgr2=0x7f5978199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 -- 172.21.15.86:0/2433894408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59781069c0 msgr2=0x7f59781a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59781069c0 0x7f59781a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 -- 172.21.15.86:0/2433894408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59781a4720 con 0x7f5978101350 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f59777fe640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f596ffff640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596000cd80 con 0x7f5978101350 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59600054d0 con 0x7f5978101350 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59781a48f0 con 0x7f5978101350 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960010490 con 0x7f5978101350 2024-09-05T06:41:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.620+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59781a4e10 con 0x7f5978101350 2024-09-05T06:41:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.621+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978100e10 con 0x7f5978101350 2024-09-05T06:41:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.623+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f5960005000 con 0x7f5978101350 2024-09-05T06:41:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.623+0000 7f59757fa640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 0x7f59640a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.623+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f59600cffe0 con 0x7f5978101350 2024-09-05T06:41:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.623+0000 7f59777fe640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 0x7f59640a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.624+0000 7f59777fe640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 0x7f59640a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968009610 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.627+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59600541b0 con 0x7f5978101350 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.667+0000 7f406203a640 1 -- 172.21.15.86:0/2544389271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102570 msgr2=0x7f405c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.667+0000 7f406203a640 1 --2- 172.21.15.86:0/2544389271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102570 0x7f405c1029b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404402a480 comp rx=0 tx=0).stop 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 -- 172.21.15.86:0/2544389271 shutdown_connections 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 --2- 172.21.15.86:0/2544389271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c102ef0 0x7f405c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 --2- 172.21.15.86:0/2544389271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102570 0x7f405c1029b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 --2- 172.21.15.86:0/2544389271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c100e20 0x7f405c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 -- 172.21.15.86:0/2544389271 >> 172.21.15.86:0/2544389271 conn(0x7f405c0fc7d0 msgr2=0x7f405c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 -- 172.21.15.86:0/2544389271 shutdown_connections 2024-09-05T06:41:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 -- 172.21.15.86:0/2544389271 wait complete. 2024-09-05T06:41:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 Processor -- start 2024-09-05T06:41:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.668+0000 7f406203a640 1 -- start start 2024-09-05T06:41:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c100e20 0x7f405c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c102570 0x7f405c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f405c110e30 con 0x7f405c102ef0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f405c110cb0 con 0x7f405c100e20 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f406203a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f405c110fb0 con 0x7f405c102570 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44166/0 (socket says 172.21.15.86:44166) 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 -- 172.21.15.86:0/746223709 learned_addr learned my addr 172.21.15.86:0/746223709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405affd640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c102570 0x7f405c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405b7fe640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c100e20 0x7f405c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 -- 172.21.15.86:0/746223709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c102570 msgr2=0x7f405c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c102570 0x7f405c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 -- 172.21.15.86:0/746223709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c100e20 msgr2=0x7f405c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c100e20 0x7f405c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 -- 172.21.15.86:0/746223709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c19f580 con 0x7f405c102ef0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.669+0000 7f405bfff640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e970 tx=0x7f404c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.670+0000 7f4058ff9640 1 -- 172.21.15.86:0/746223709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00ccc0 con 0x7f405c102ef0 2024-09-05T06:41:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.670+0000 7f4058ff9640 1 -- 172.21.15.86:0/746223709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f404c00ce60 con 0x7f405c102ef0 2024-09-05T06:41:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.670+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c19f870 con 0x7f405c102ef0 2024-09-05T06:41:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.670+0000 7f4058ff9640 1 -- 172.21.15.86:0/746223709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c010430 con 0x7f405c102ef0 2024-09-05T06:41:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.670+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c1a7150 con 0x7f405c102ef0 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.672+0000 7f4058ff9640 1 -- 172.21.15.86:0/746223709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f404c004ff0 con 0x7f405c102ef0 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.672+0000 7f4064413740 1 Processor -- start 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.672+0000 7f4064413740 1 -- start start 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db911f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db91c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db91f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2db877280 con 0x55d2db91cd40 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2db81eb10 con 0x55d2db911b60 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4064413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d2db873240 con 0x55d2db912480 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db91f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db91f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44182/0 (socket says 172.21.15.86:44182) 2024-09-05T06:41:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405affd640 1 -- 172.21.15.86:0/2597052468 learned_addr learned my addr 172.21.15.86:0/2597052468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db91c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4058ff9640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 0x7f40480a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405bfff640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 0x7f40480a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 -- 172.21.15.86:0/2597052468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 msgr2=0x55d2db911f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db911f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 -- 172.21.15.86:0/2597052468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 msgr2=0x55d2db91f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db91f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 -- 172.21.15.86:0/2597052468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2db82b1f0 con 0x55d2db912480 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f4058ff9640 1 -- 172.21.15.86:0/746223709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f404c0cfd60 con 0x7f405c102ef0 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.673+0000 7f405b7fe640 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db91c800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4054009f80 tx=0x7f40540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f57e431892ffb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4051ffb640 1 -- 172.21.15.86:0/2597052468 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054004d20 con 0x55d2db912480 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4051ffb640 1 -- 172.21.15.86:0/2597052468 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054004050 con 0x55d2db912480 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 -- 172.21.15.86:0/2597052468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 msgr2=0x55d2db91c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db91c800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4054009f80 tx=0x7f40540118d0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f405bfff640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 0x7f40480a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f404c0cd070 tx=0x7f404c01d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 -- 172.21.15.86:0/2597052468 shutdown_connections 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db91f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db91c800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 --2- 172.21.15.86:0/2597052468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db911f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.674+0000 7f4064413740 1 -- 172.21.15.86:0/2597052468 >> 172.21.15.86:0/2597052468 conn(0x55d2db90e0f0 msgr2=0x55d2db90e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.675+0000 7f4064413740 1 -- 172.21.15.86:0/2597052468 shutdown_connections 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.675+0000 7f4064413740 1 -- 172.21.15.86:0/2597052468 wait complete. 2024-09-05T06:41:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.675+0000 7f4064413740 1 -- start start 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.675+0000 7f4064413740 1 Processor -- start 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db919b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db91cd40 0x55d2db91a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2db91f960 con 0x55d2db911b60 2024-09-05T06:41:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2db831f10 con 0x55d2db91cd40 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d2db91fae0 con 0x55d2db912480 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f4064413740 10 client.0 _ll_register_callbacks cb 0x55d2db7faa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44184/0 (socket says 172.21.15.86:44184) 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 -- 172.21.15.86:0/1183926996 learned_addr learned my addr 172.21.15.86:0/1183926996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405b7fe640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db919b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 -- 172.21.15.86:0/1183926996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 msgr2=0x55d2db919b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db919b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 -- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db91cd40 msgr2=0x55d2db91a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db91cd40 0x55d2db91a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 -- 172.21.15.86:0/1183926996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2db5e86a0 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f405affd640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4044005940 tx=0x7f4044031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddd9de3e9dca660 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.676+0000 7f403affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044035040 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4044031840 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4054004050 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f404404f050 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2db92b4c0 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054004050 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4044003c00 con 0x55d2db911b60 2024-09-05T06:41:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 20 client.15450 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db92c980 0x55d2db92ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 4 client.15450 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.677+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2db8847f0 con 0x55d2db92c980 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.678+0000 7f405b7fe640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db92c980 0x55d2db92ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.678+0000 7f403affd640 10 client.15450 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.678+0000 7f405b7fe640 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db92c980 0x55d2db92ed70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f941fb90d0d1bcba server_cookie=6f294dbd44ba2b8c in_seq=0 out_seq=0 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f403affd640 1 -- 172.21.15.86:0/1183926996 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2db8847f0 con 0x55d2db92c980 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f403affd640 10 client.15450 handle_command_reply: tid=0 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 client.15450 shutdown 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 msgr2=0x55d2db91c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4044005940 tx=0x7f4044031d20 comp rx=0 tx=0).stop 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 shutdown_connections 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db92c980 0x55d2db92ed70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db91cd40 0x55d2db91a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db919b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 --2- 172.21.15.86:0/1183926996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db91c7d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 >> 172.21.15.86:0/1183926996 conn(0x55d2db90e0f0 msgr2=0x55d2db913dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f403affd640 0 client.15450 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 shutdown_connections 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 1 -- 172.21.15.86:0/1183926996 wait complete. 2024-09-05T06:41:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.679+0000 7f4064413740 20 client.15450 trim_cache size 0 max 16384 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 Processor -- start 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 -- start start 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db918b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db9345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1b11b0 con 0x55d2db91cd40 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1b2cf0 con 0x55d2db911b60 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f4064413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1b1880 con 0x55d2db912480 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db9345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f405affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f405affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37496/0 (socket says 172.21.15.86:37496) 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f405affd640 1 -- 172.21.15.86:0/1528846432 learned_addr learned my addr 172.21.15.86:0/1528846432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.713+0000 7f405b7fe640 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db918b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 -- 172.21.15.86:0/1528846432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 msgr2=0x55d2db918b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db918b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 -- 172.21.15.86:0/1528846432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 msgr2=0x55d2db9345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db9345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 -- 172.21.15.86:0/1528846432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2db909830 con 0x55d2db911b60 2024-09-05T06:41:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f405affd640 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4044000c00 tx=0x7f4044005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3462c1cbae400c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f40517fa640 1 -- 172.21.15.86:0/1528846432 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044046760 con 0x55d2db911b60 2024-09-05T06:41:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f40517fa640 1 -- 172.21.15.86:0/1528846432 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4044041070 con 0x55d2db911b60 2024-09-05T06:41:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.714+0000 7f40517fa640 1 -- 172.21.15.86:0/1528846432 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40440043a0 con 0x55d2db911b60 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- 172.21.15.86:0/1528846432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 msgr2=0x55d2db9145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4044000c00 tx=0x7f4044005fd0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- 172.21.15.86:0/1528846432 shutdown_connections 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db91cd40 0x55d2db9345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db912480 0x55d2db918b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 --2- 172.21.15.86:0/1528846432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db911b60 0x55d2db9145c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- 172.21.15.86:0/1528846432 >> 172.21.15.86:0/1528846432 conn(0x55d2db90e0f0 msgr2=0x55d2db9183f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- 172.21.15.86:0/1528846432 shutdown_connections 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- 172.21.15.86:0/1528846432 wait complete. 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.715+0000 7f4064413740 1 -- start start 2024-09-05T06:41:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 Processor -- start 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db9254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db91cd40 0x55d2db916e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1b2620 con 0x55d2db911b60 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1b1f50 con 0x55d2db912480 2024-09-05T06:41:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1af190 con 0x55d2db91cd40 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f4064413740 10 client.0 _ll_register_callbacks cb 0x7f40540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db9254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db9254e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44208/0 (socket says 172.21.15.86:44208) 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37504/0 (socket says 172.21.15.86:37504) 2024-09-05T06:41:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405affd640 1 -- 172.21.15.86:0/3277980062 learned_addr learned my addr 172.21.15.86:0/3277980062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405affd640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db91cd40 0x55d2db916e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 -- 172.21.15.86:0/3277980062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db91cd40 msgr2=0x55d2db916e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db91cd40 0x55d2db916e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 -- 172.21.15.86:0/3277980062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 msgr2=0x55d2db9254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db9254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.716+0000 7f405b7fe640 1 -- 172.21.15.86:0/3277980062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2db9122b0 con 0x55d2db912480 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f405b7fe640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4054011dd0 tx=0x7f4054009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde57eb954325e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054005810 con 0x55d2db912480 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4054004790 con 0x55d2db912480 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4044041070 con 0x55d2db912480 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f405403c050 con 0x55d2db912480 2024-09-05T06:41:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f4064413740 10 client.25022 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2db9975a0 con 0x55d2db912480 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f405402f070 con 0x55d2db912480 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.717+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4054003ed0 con 0x55d2db912480 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 10 client.25022 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 10 client.25022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 20 client.25022 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db997ac0 0x55d2db999ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 4 client.25022 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4044041070 con 0x55d2db997ac0 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f405affd640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db997ac0 0x55d2db999ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f403affd640 10 client.25022 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.718+0000 7f405affd640 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db997ac0 0x55d2db999ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0b8eda0f71dc94 server_cookie=627e6536a655d287 in_seq=0 out_seq=0 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f403affd640 1 -- 172.21.15.86:0/3277980062 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4044041070 con 0x55d2db997ac0 2024-09-05T06:41:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f403affd640 10 client.25022 handle_command_reply: tid=0 2024-09-05T06:41:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 client.25022 shutdown 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 msgr2=0x55d2db9168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4054011dd0 tx=0x7f4054009280 comp rx=0 tx=0).stop 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 shutdown_connections 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d2db997ac0 0x55d2db999ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d2db91cd40 0x55d2db916e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2db912480 0x55d2db9168e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 --2- 172.21.15.86:0/3277980062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2db911b60 0x55d2db9254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 >> 172.21.15.86:0/3277980062 conn(0x55d2db90e0f0 msgr2=0x55d2db918c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.719+0000 7f403affd640 0 client.25022 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 shutdown_connections 2024-09-05T06:41:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f4064413740 1 -- 172.21.15.86:0/3277980062 wait complete. 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f4064413740 20 client.25022 trim_cache size 0 max 16384 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6adfb96f-0db8-4fdc-8a53-e6735345dc14", 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:41:25.726 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 msgr2=0x7f40480a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 0x7f40480a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f404c0cd070 tx=0x7f404c01d000 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 msgr2=0x7f405c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.720+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e970 tx=0x7f404c00ed70 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 shutdown_connections 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40480a2e00 0x7f40480a52c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c102ef0 0x7f405c19eea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c102570 0x7f405c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 --2- 172.21.15.86:0/746223709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f405c100e20 0x7f405c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 >> 172.21.15.86:0/746223709 conn(0x7f405c0fc7d0 msgr2=0x7f405c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 shutdown_connections 2024-09-05T06:41:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.721+0000 7f406203a640 1 -- 172.21.15.86:0/746223709 wait complete. 2024-09-05T06:41:25.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.824+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f5978108990 con 0x7f5978101350 2024-09-05T06:41:25.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.830+0000 7f59757fa640 1 -- 172.21.15.86:0/2433894408 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f59600052f0 con 0x7f5978101350 2024-09-05T06:41:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 msgr2=0x7f59640a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 0x7f59640a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968009610 comp rx=0 tx=0).stop 2024-09-05T06:41:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 msgr2=0x7f59781996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000edf0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 shutdown_connections 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f59640a2ec0 0x7f59640a5380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59781069c0 0x7f59781a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978101350 0x7f59781996a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 --2- 172.21.15.86:0/2433894408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59781009b0 0x7f5978199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 >> 172.21.15.86:0/2433894408 conn(0x7f5978077990 msgr2=0x7f5978103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 shutdown_connections 2024-09-05T06:41:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.832+0000 7f597db4b640 1 -- 172.21.15.86:0/2433894408 wait complete. 2024-09-05T06:41:25.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- 172.21.15.86:0/2696231741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c0103600 msgr2=0x7f44c010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 --2- 172.21.15.86:0/2696231741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c0103600 0x7f44c010bae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402a480 comp rx=0 tx=0).stop 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- 172.21.15.86:0/2696231741 shutdown_connections 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 --2- 172.21.15.86:0/2696231741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c010c0b0 0x7f44c010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 --2- 172.21.15.86:0/2696231741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c0103600 0x7f44c010bae0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 --2- 172.21.15.86:0/2696231741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 0x7f44c01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- 172.21.15.86:0/2696231741 >> 172.21.15.86:0/2696231741 conn(0x7f44c0077990 msgr2=0x7f44c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- 172.21.15.86:0/2696231741 shutdown_connections 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- 172.21.15.86:0/2696231741 wait complete. 2024-09-05T06:41:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 Processor -- start 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.950+0000 7f44c84e4640 1 -- start start 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 0x7f44c01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c0103600 0x7f44c0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0111db0 con 0x7f44c010c0b0 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0111c30 con 0x7f44c0103600 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0111f30 con 0x7f44c0102ce0 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44224/0 (socket says 172.21.15.86:44224) 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 -- 172.21.15.86:0/223261878 learned_addr learned my addr 172.21.15.86:0/223261878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6280640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 0x7f44c01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c5a7f640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c0103600 0x7f44c0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 -- 172.21.15.86:0/223261878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 msgr2=0x7f44c01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 0x7f44c01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 -- 172.21.15.86:0/223261878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c0103600 msgr2=0x7f44c0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c0103600 0x7f44c0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 -- 172.21.15.86:0/223261878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c019d2d0 con 0x7f44c010c0b0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c6a81640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017030 con 0x7f44c010c0b0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b8004020 con 0x7f44c010c0b0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c019d5c0 con 0x7f44c010c0b0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8003b20 con 0x7f44c010c0b0 2024-09-05T06:41:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.951+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c01a4e50 con 0x7f44c010c0b0 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.952+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c01030c0 con 0x7f44c010c0b0 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.953+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f44b801a070 con 0x7f44c010c0b0 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.954+0000 7f44bf7fe640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 0x7f44a80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.954+0000 7f44c6280640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 0x7f44a80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.955+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f44b80cf060 con 0x7f44c010c0b0 2024-09-05T06:41:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.955+0000 7f44c6280640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 0x7f44a80a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005fd0 tx=0x7f44b0009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:25.959+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44b8021070 con 0x7f44c010c0b0 2024-09-05T06:41:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.150+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f44c0062630 con 0x7f44a80a2ec0 2024-09-05T06:41:26.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.212+0000 7f44bf7fe640 1 -- 172.21.15.86:0/223261878 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f44c0062630 con 0x7f44a80a2ec0 2024-09-05T06:41:26.212 INFO:teuthology.orchestra.run.smithi086.stdout:Schedule set for path / 2024-09-05T06:41:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 msgr2=0x7f44a80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 0x7f44a80a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44b0005fd0 tx=0x7f44b0009500 comp rx=0 tx=0).stop 2024-09-05T06:41:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 msgr2=0x7f44c019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).stop 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 shutdown_connections 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44a80a2ec0 0x7f44a80a5380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44c010c0b0 0x7f44c019cba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44c0103600 0x7f44c0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 --2- 172.21.15.86:0/223261878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c0102ce0 0x7f44c01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 >> 172.21.15.86:0/223261878 conn(0x7f44c0077990 msgr2=0x7f44c00fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 shutdown_connections 2024-09-05T06:41:26.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.214+0000 7f44c84e4640 1 -- 172.21.15.86:0/223261878 wait complete. 2024-09-05T06:41:26.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 -- 172.21.15.86:0/4066423553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 msgr2=0x7f7ed4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 --2- 172.21.15.86:0/4066423553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed4100e30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009930 tx=0x7f7ec002a420 comp rx=0 tx=0).stop 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 -- 172.21.15.86:0/4066423553 shutdown_connections 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 --2- 172.21.15.86:0/4066423553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 0x7f7ed410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 --2- 172.21.15.86:0/4066423553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed4100e30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 --2- 172.21.15.86:0/4066423553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 0x7f7ed4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 -- 172.21.15.86:0/4066423553 >> 172.21.15.86:0/4066423553 conn(0x7f7ed40fc6f0 msgr2=0x7f7ed40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 -- 172.21.15.86:0/4066423553 shutdown_connections 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 -- 172.21.15.86:0/4066423553 wait complete. 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.330+0000 7f7edad86640 1 Processor -- start 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 -- start start 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 0x7f7ed4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 0x7f7ed41a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4110d20 con 0x7f7ed41009d0 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4110ba0 con 0x7f7ed4101370 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed4110ea0 con 0x7f7ed41069e0 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44234/0 (socket says 172.21.15.86:44234) 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 -- 172.21.15.86:0/3758562453 learned_addr learned my addr 172.21.15.86:0/3758562453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed3fff640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 0x7f7ed4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed9323640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 0x7f7ed41a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 -- 172.21.15.86:0/3758562453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 msgr2=0x7f7ed41a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 0x7f7ed41a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 -- 172.21.15.86:0/3758562453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 msgr2=0x7f7ed4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 0x7f7ed4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 -- 172.21.15.86:0/3758562453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed41a4790 con 0x7f7ed41009d0 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed8b22640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b4a0 tx=0x7f7ec800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8014070 con 0x7f7ed41009d0 2024-09-05T06:41:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ec8010ac0 con 0x7f7ed41009d0 2024-09-05T06:41:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed41a4960 con 0x7f7ed41009d0 2024-09-05T06:41:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.331+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017c30 con 0x7f7ed41009d0 2024-09-05T06:41:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.332+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed41a4e80 con 0x7f7ed41009d0 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.333+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed4100e30 con 0x7f7ed41009d0 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.334+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ed41009d0 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.334+0000 7f7ed1ffb640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 0x7f7ec40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.335+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f7ec80cebe0 con 0x7f7ed41009d0 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.335+0000 7f7ed3fff640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 0x7f7ec40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.335+0000 7f7ed3fff640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 0x7f7ec40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0031440 tx=0x7f7ec0005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.339+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ec809b3c0 con 0x7f7ed41009d0 2024-09-05T06:41:26.420 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3412216926' entity='client.admin' 2024-09-05T06:41:26.420 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 19 op/s 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/60459363' entity='client.admin' 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2433894408' entity='client.admin' 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3412216926' entity='client.admin' 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 19 op/s 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/60459363' entity='client.admin' 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2433894408' entity='client.admin' 2024-09-05T06:41:26.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:41:26.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.533+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7ed4062630 con 0x7f7ec40a2ec0 2024-09-05T06:41:26.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3412216926' entity='client.admin' 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 683 B/s wr, 19 op/s 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/60459363' entity='client.admin' 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2433894408' entity='client.admin' 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:41:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:41:26.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:41:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.594+0000 7f7ed1ffb640 1 -- 172.21.15.86:0/3758562453 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7ed4062630 con 0x7f7ec40a2ec0 2024-09-05T06:41:26.594 INFO:teuthology.orchestra.run.smithi086.stdout:Retention added to path / 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 msgr2=0x7f7ec40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 0x7f7ec40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0031440 tx=0x7f7ec0005a20 comp rx=0 tx=0).stop 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 msgr2=0x7f7ed41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b4a0 tx=0x7f7ec800b960 comp rx=0 tx=0).stop 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 shutdown_connections 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ec40a2ec0 0x7f7ec40a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ed41069e0 0x7f7ed41a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed4101370 0x7f7ed4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 --2- 172.21.15.86:0/3758562453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed41009d0 0x7f7ed41991d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 >> 172.21.15.86:0/3758562453 conn(0x7f7ed40fc6f0 msgr2=0x7f7ed40fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 shutdown_connections 2024-09-05T06:41:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.596+0000 7f7edad86640 1 -- 172.21.15.86:0/3758562453 wait complete. 2024-09-05T06:41:26.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-05T06:41:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 -- 172.21.15.86:0/3011644707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 msgr2=0x7f3bbc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 --2- 172.21.15.86:0/3011644707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc1021b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009900 tx=0x7f3bb002a410 comp rx=0 tx=0).stop 2024-09-05T06:41:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 -- 172.21.15.86:0/3011644707 shutdown_connections 2024-09-05T06:41:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 --2- 172.21.15.86:0/3011644707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 0x7f3bbc10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 --2- 172.21.15.86:0/3011644707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc1021b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 --2- 172.21.15.86:0/3011644707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 0x7f3bbc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 -- 172.21.15.86:0/3011644707 >> 172.21.15.86:0/3011644707 conn(0x7f3bbc0770f0 msgr2=0x7f3bbc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 -- 172.21.15.86:0/3011644707 shutdown_connections 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 -- 172.21.15.86:0/3011644707 wait complete. 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.715+0000 7f3bc219f640 1 Processor -- start 2024-09-05T06:41:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 -- start start 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 0x7f3bbc10e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 0x7f3bbc10f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc111e80 con 0x7f3bbc101d70 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc111d00 con 0x7f3bbc100490 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bc219f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc112000 con 0x7f3bbc1026f0 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44248/0 (socket says 172.21.15.86:44248) 2024-09-05T06:41:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 -- 172.21.15.86:0/171833673 learned_addr learned my addr 172.21.15.86:0/171833673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbbfff640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 0x7f3bbc10f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbb7fe640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 0x7f3bbc10e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 -- 172.21.15.86:0/171833673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 msgr2=0x7f3bbc10f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 0x7f3bbc10f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 -- 172.21.15.86:0/171833673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 msgr2=0x7f3bbc10e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 0x7f3bbc10e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.716+0000 7f3bbaffd640 1 -- 172.21.15.86:0/171833673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc1a67a0 con 0x7f3bbc101d70 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bbaffd640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005940 tx=0x7f3bb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb003f030 con 0x7f3bbc101d70 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb0003cb0 con 0x7f3bbc101d70 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc1a6a30 con 0x7f3bbc101d70 2024-09-05T06:41:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0035070 con 0x7f3bbc101d70 2024-09-05T06:41:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.717+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc1a6f20 con 0x7f3bbc101d70 2024-09-05T06:41:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.718+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bbc100870 con 0x7f3bbc101d70 2024-09-05T06:41:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.719+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3bb0042030 con 0x7f3bbc101d70 2024-09-05T06:41:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.720+0000 7f3bb8ff9640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 0x7f3ba40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.720+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3bb00f0140 con 0x7f3bbc101d70 2024-09-05T06:41:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.720+0000 7f3bbb7fe640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 0x7f3ba40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.721+0000 7f3bbb7fe640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 0x7f3ba40a5300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009bd0 tx=0x7f3bac005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.725+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bb00bc1c0 con 0x7f3bbc101d70 2024-09-05T06:41:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.914+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3bbc1087d0 con 0x7f3ba40a2e40 2024-09-05T06:41:26.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.920+0000 7f3bb8ff9640 1 -- 172.21.15.86:0/171833673 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3bac01b3f0 con 0x7f3ba40a2e40 2024-09-05T06:41:26.920 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-05T00:00:00", "created": "2024-09-05T06:41:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 msgr2=0x7f3ba40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 0x7f3ba40a5300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009bd0 tx=0x7f3bac005f90 comp rx=0 tx=0).stop 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 msgr2=0x7f3bbc10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005940 tx=0x7f3bb00042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 shutdown_connections 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3ba40a2e40 0x7f3ba40a5300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bbc1026f0 0x7f3bbc10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bbc101d70 0x7f3bbc10eef0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 --2- 172.21.15.86:0/171833673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bbc100490 0x7f3bbc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 >> 172.21.15.86:0/171833673 conn(0x7f3bbc0770f0 msgr2=0x7f3bbc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 shutdown_connections 2024-09-05T06:41:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:26.922+0000 7f3bc219f640 1 -- 172.21.15.86:0/171833673 wait complete. 2024-09-05T06:41:26.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-05T06:41:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- 172.21.15.86:0/578285028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381069c0 msgr2=0x7f3838106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 --2- 172.21.15.86:0/578285028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381069c0 0x7f3838106da0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3828009930 tx=0x7f382802a420 comp rx=0 tx=0).stop 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- 172.21.15.86:0/578285028 shutdown_connections 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 --2- 172.21.15.86:0/578285028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3838101350 0x7f383810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 --2- 172.21.15.86:0/578285028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38381009b0 0x7f3838100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 --2- 172.21.15.86:0/578285028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381069c0 0x7f3838106da0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- 172.21.15.86:0/578285028 >> 172.21.15.86:0/578285028 conn(0x7f3838077990 msgr2=0x7f38380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- 172.21.15.86:0/578285028 shutdown_connections 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- 172.21.15.86:0/578285028 wait complete. 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 Processor -- start 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.038+0000 7f383dfc4640 1 -- start start 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 0x7f3838199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38381069c0 0x7f38381a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3838110cb0 con 0x7f38381009b0 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3838110b30 con 0x7f3838101350 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3838110e30 con 0x7f38381069c0 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44268/0 (socket says 172.21.15.86:44268) 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 -- 172.21.15.86:0/315462109 learned_addr learned my addr 172.21.15.86:0/315462109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f3837fff640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38381069c0 0x7f38381a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f3836ffd640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 0x7f3838199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 -- 172.21.15.86:0/315462109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38381069c0 msgr2=0x7f38381a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38381069c0 0x7f38381a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 -- 172.21.15.86:0/315462109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 msgr2=0x7f3838199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 0x7f3838199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 -- 172.21.15.86:0/315462109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a4680 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f3836ffd640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 0x7f3838199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f38377fe640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3828005940 tx=0x7f38280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382803f030 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3828003cb0 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.039+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a4850 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.040+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828035070 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.040+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a4d10 con 0x7f38381009b0 2024-09-05T06:41:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.041+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3838100e10 con 0x7f38381009b0 2024-09-05T06:41:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.042+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3828042030 con 0x7f38381009b0 2024-09-05T06:41:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.043+0000 7f3834ff9640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 0x7f38200a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.043+0000 7f3836ffd640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 0x7f38200a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.043+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f38280efe80 con 0x7f38381009b0 2024-09-05T06:41:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.043+0000 7f3836ffd640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 0x7f38200a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f383806a3f0 tx=0x7f382c005d90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.047+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38280bc550 con 0x7f38381009b0 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 -- 172.21.15.86:0/2795046611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c100e20 msgr2=0x7fdc5c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/2795046611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c100e20 0x7fdc5c101200 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c02a480 comp rx=0 tx=0).stop 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 -- 172.21.15.86:0/2795046611 shutdown_connections 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/2795046611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/2795046611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c102570 0x7fdc5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/2795046611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c100e20 0x7fdc5c101200 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 -- 172.21.15.86:0/2795046611 >> 172.21.15.86:0/2795046611 conn(0x7fdc5c0fc7d0 msgr2=0x7fdc5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 -- 172.21.15.86:0/2795046611 shutdown_connections 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 -- 172.21.15.86:0/2795046611 wait complete. 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.208+0000 7fdc61cc1640 1 Processor -- start 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 -- start start 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c100e20 0x7fdc5c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c110e30 con 0x7fdc5c102570 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c110cb0 con 0x7fdc5c102ef0 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc61cc1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c110fb0 con 0x7fdc5c100e20 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44284/0 (socket says 172.21.15.86:44284) 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 -- 172.21.15.86:0/1595864161 learned_addr learned my addr 172.21.15.86:0/1595864161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c100e20 0x7fdc5c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 -- 172.21.15.86:0/1595864161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c100e20 msgr2=0x7fdc5c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c100e20 0x7fdc5c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 -- 172.21.15.86:0/1595864161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 msgr2=0x7fdc5c1a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c1a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5affd640 1 -- 172.21.15.86:0/1595864161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c1a48a0 con 0x7fdc5c102570 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.209+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000d950 tx=0x7fdc5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc58ff9640 1 -- 172.21.15.86:0/1595864161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50017070 con 0x7fdc5c102570 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc58ff9640 1 -- 172.21.15.86:0/1595864161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc50009c30 con 0x7fdc5c102570 2024-09-05T06:41:27.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c1a4a70 con 0x7fdc5c102570 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc58ff9640 1 -- 172.21.15.86:0/1595864161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000f620 con 0x7fdc5c102570 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.210+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c1a4f80 con 0x7fdc5c102570 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.212+0000 7fdc58ff9640 1 -- 172.21.15.86:0/1595864161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fdc500109a0 con 0x7fdc5c102570 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 Processor -- start 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 -- start start 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc58ff9640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 0x7fdc440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc58ff9640 1 -- 172.21.15.86:0/1595864161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc500cf6d0 con 0x7fdc5c102570 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 0x7fdc440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x561065582760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065591a90 0x561065593e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610654e82e0 con 0x561065582ca0 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56106548fce0 con 0x561065582380 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc6409a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5610654e42a0 con 0x561065591a90 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x561065582760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44300/0 (socket says 172.21.15.86:44300) 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 -- 172.21.15.86:0/1042998887 learned_addr learned my addr 172.21.15.86:0/1042998887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 -- 172.21.15.86:0/1042998887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065591a90 msgr2=0x561065593e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065591a90 0x561065593e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 0x7fdc440a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005fd0 tx=0x7fdc54009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 -- 172.21.15.86:0/1042998887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 msgr2=0x561065582760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x561065582760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5affd640 1 -- 172.21.15.86:0/1042998887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561065474440 con 0x561065582ca0 2024-09-05T06:41:27.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.213+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x561065582760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.214+0000 7fdc5affd640 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdc5009b630 tx=0x7fdc50009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99234de65513e0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.214+0000 7fdc41ffb640 1 -- 172.21.15.86:0/1042998887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50020040 con 0x561065582ca0 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.214+0000 7fdc41ffb640 1 -- 172.21.15.86:0/1042998887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5009b6a0 con 0x561065582ca0 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.214+0000 7fdc41ffb640 1 -- 172.21.15.86:0/1042998887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500cf2d0 con 0x561065582ca0 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- 172.21.15.86:0/1042998887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 msgr2=0x561065591550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdc5009b630 tx=0x7fdc50009de0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- 172.21.15.86:0/1042998887 shutdown_connections 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065591a90 0x561065593e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582ca0 0x561065591550 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- 172.21.15.86:0/1042998887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x561065582760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- 172.21.15.86:0/1042998887 >> 172.21.15.86:0/1042998887 conn(0x56106557ea30 msgr2=0x56106557ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- 172.21.15.86:0/1042998887 shutdown_connections 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- 172.21.15.86:0/1042998887 wait complete. 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- start start 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 Processor -- start 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106558e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 0x56106558ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561065594660 con 0x561065582380 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610654a3050 con 0x561065591a90 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.215+0000 7fdc6409a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5610655947e0 con 0x561065582ca0 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc6409a740 10 client.0 _ll_register_callbacks cb 0x5610654e3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44316/0 (socket says 172.21.15.86:44316) 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 -- 172.21.15.86:0/2658212956 learned_addr learned my addr 172.21.15.86:0/2658212956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5affd640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106558e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 0x56106558ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 -- 172.21.15.86:0/2658212956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 msgr2=0x56106558e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106558e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 -- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 msgr2=0x56106558ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 0x56106558ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 -- 172.21.15.86:0/2658212956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610652593c0 con 0x561065582380 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 0x56106558ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c0317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819ac7402ab5a8fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc3affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c035040 con 0x561065582380 2024-09-05T06:41:27.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc4c031ca0 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.216+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5009b6a0 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c004040 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610655a00d0 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5009b6a0 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc4c004880 con 0x561065582380 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 20 client.15492 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5610655a1590 0x5610655a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 4 client.15492 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610654f5850 con 0x5610655a1590 2024-09-05T06:41:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.217+0000 7fdc5affd640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5610655a1590 0x5610655a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.218+0000 7fdc5affd640 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5610655a1590 0x5610655a3980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650c45a07d2ef1ca server_cookie=6f513aeebea4cede in_seq=0 out_seq=0 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.218+0000 7fdc3affd640 10 client.15492 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.218+0000 7fdc3affd640 1 -- 172.21.15.86:0/2658212956 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610654f5850 con 0x5610655a1590 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.218+0000 7fdc3affd640 10 client.15492 handle_command_reply: tid=0 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.218+0000 7fdc6409a740 1 client.15492 shutdown 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 msgr2=0x561065591460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c0317a0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 shutdown_connections 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5610655a1590 0x5610655a3980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065591a90 0x56106558ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106558e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 --2- 172.21.15.86:0/2658212956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065582380 0x561065591460 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 >> 172.21.15.86:0/2658212956 conn(0x56106557ea30 msgr2=0x561065584790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc3affd640 0 client.15492 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 shutdown_connections 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 1 -- 172.21.15.86:0/2658212956 wait complete. 2024-09-05T06:41:27.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.219+0000 7fdc6409a740 20 client.15492 trim_cache size 0 max 16384 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 Processor -- start 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 -- start start 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x56106558e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106559d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1aced0 con 0x561065591a90 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1aea10 con 0x561065582380 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc6409a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1ad5a0 con 0x561065582ca0 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44332/0 (socket says 172.21.15.86:44332) 2024-09-05T06:41:27.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 -- 172.21.15.86:0/627034040 learned_addr learned my addr 172.21.15.86:0/627034040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5affd640 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106559d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x56106558e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 -- 172.21.15.86:0/627034040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 msgr2=0x56106559d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106559d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 -- 172.21.15.86:0/627034040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 msgr2=0x56106558e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x56106558e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.255+0000 7fdc5bfff640 1 -- 172.21.15.86:0/627034040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106558f070 con 0x561065591a90 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.256+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc540117f0 tx=0x7fdc54013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bacc16486eeb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.256+0000 7fdc417fa640 1 -- 172.21.15.86:0/627034040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5401e940 con 0x561065591a90 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.256+0000 7fdc417fa640 1 -- 172.21.15.86:0/627034040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5401f430 con 0x561065591a90 2024-09-05T06:41:27.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.256+0000 7fdc417fa640 1 -- 172.21.15.86:0/627034040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54027630 con 0x561065591a90 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- 172.21.15.86:0/627034040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 msgr2=0x5610655a9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc540117f0 tx=0x7fdc54013070 comp rx=0 tx=0).stop 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- 172.21.15.86:0/627034040 shutdown_connections 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x5610655a9250 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582ca0 0x56106559d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 --2- 172.21.15.86:0/627034040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582380 0x56106558e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- 172.21.15.86:0/627034040 >> 172.21.15.86:0/627034040 conn(0x56106557ea30 msgr2=0x56106558dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- 172.21.15.86:0/627034040 shutdown_connections 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- 172.21.15.86:0/627034040 wait complete. 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 -- start start 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 Processor -- start 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.257+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582380 0x561065599a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 0x561065599fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1ae340 con 0x561065591a90 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1adc70 con 0x561065582ca0 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1aaeb0 con 0x561065582380 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 10 client.0 _ll_register_callbacks cb 0x7fdc4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44346/0 (socket says 172.21.15.86:44346) 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 -- 172.21.15.86:0/4291926824 learned_addr learned my addr 172.21.15.86:0/4291926824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582380 0x561065599a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5affd640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 0x561065599fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 -- 172.21.15.86:0/4291926824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582380 msgr2=0x561065599a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582380 0x561065599a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 -- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 msgr2=0x561065599fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 0x561065599fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 -- 172.21.15.86:0/4291926824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106558d6c0 con 0x561065591a90 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5affd640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 0x561065599fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc5bfff640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009600 tx=0x7fdc54011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63611c25eb7b0185 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc3affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54036060 con 0x561065591a90 2024-09-05T06:41:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc5401f7f0 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.258+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5401f430 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc5403d050 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56106560c620 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5401f430 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc54005050 con 0x561065591a90 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 20 client.15498 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56106560cc60 0x56106560f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 4 client.15498 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610654efd60 con 0x56106560cc60 2024-09-05T06:41:27.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.259+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56106560cc60 0x56106560f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc5b7fe640 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56106560cc60 0x56106560f080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3890f41906d0178c server_cookie=45d88aefffb2479e in_seq=0 out_seq=0 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc3affd640 10 client.15498 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc3affd640 1 -- 172.21.15.86:0/4291926824 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5610654efd60 con 0x56106560cc60 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc3affd640 10 client.15498 handle_command_reply: tid=0 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 client.15498 shutdown 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 msgr2=0x561065596a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009600 tx=0x7fdc54011550 comp rx=0 tx=0).stop 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 shutdown_connections 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56106560cc60 0x56106560f080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561065591a90 0x561065596a40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561065582ca0 0x561065599fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 --2- 172.21.15.86:0/4291926824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561065582380 0x561065599a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 >> 172.21.15.86:0/4291926824 conn(0x56106557ea30 msgr2=0x56106557d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc3affd640 0 client.15498 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 shutdown_connections 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.260+0000 7fdc6409a740 1 -- 172.21.15.86:0/4291926824 wait complete. 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc6409a740 20 client.15498 trim_cache size 0 max 16384 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "6adfb96f-0db8-4fdc-8a53-e6735345dc14": { 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:41:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6adfb96f-0db8-4fdc-8a53-e6735345dc14", 2024-09-05T06:41:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:41:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:41:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:41:27.267 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 msgr2=0x7fdc440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 0x7fdc440a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005fd0 tx=0x7fdc54009500 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 msgr2=0x7fdc5c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000d950 tx=0x7fdc5000de10 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 shutdown_connections 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc440a2d80 0x7fdc440a5240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc5c102ef0 0x7fdc5c1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c102570 0x7fdc5c199820 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 --2- 172.21.15.86:0/1595864161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc5c100e20 0x7fdc5c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 >> 172.21.15.86:0/1595864161 conn(0x7fdc5c0fc7d0 msgr2=0x7fdc5c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 shutdown_connections 2024-09-05T06:41:27.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.261+0000 7fdc61cc1640 1 -- 172.21.15.86:0/1595864161 wait complete. 2024-09-05T06:41:27.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.263+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3838062630 con 0x7f38200a2ec0 2024-09-05T06:41:27.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.269+0000 7f3834ff9640 1 -- 172.21.15.86:0/315462109 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3838062630 con 0x7f38200a2ec0 2024-09-05T06:41:27.269 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-09-05T06:41:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 msgr2=0x7f38200a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 0x7f38200a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f383806a3f0 tx=0x7f382c005d90 comp rx=0 tx=0).stop 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 msgr2=0x7f38381990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3828005940 tx=0x7f38280042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 shutdown_connections 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38200a2ec0 0x7f38200a5380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38381069c0 0x7f38381a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101350 0x7f3838199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 --2- 172.21.15.86:0/315462109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38381009b0 0x7f38381990c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 >> 172.21.15.86:0/315462109 conn(0x7f3838077990 msgr2=0x7f38381037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 shutdown_connections 2024-09-05T06:41:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:27.272+0000 7f383dfc4640 1 -- 172.21.15.86:0/315462109 wait complete. 2024-09-05T06:41:27.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-05T06:41:27.312 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-05T06:41:27.322 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-05T06:41:27.350 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T06:41:27.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:41:27.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:27 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:41:27.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:27 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:27 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:41:27.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:27 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:27 smithi086 ceph-mon[30943]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:27 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:41:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:27 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:27 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:41:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:27 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:27 smithi096 ceph-mon[34468]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:27 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:41:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:27 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:27 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:41:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:27 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:41:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:27 smithi115 ceph-mon[34791]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:28.071 INFO:tasks.cephfs.filesystem:scrub status for tag:6adfb96f-0db8-4fdc-8a53-e6735345dc14 - {'path': '/', 'tag': '6adfb96f-0db8-4fdc-8a53-e6735345dc14', 'options': 'recursive,force'} 2024-09-05T06:41:28.362 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:28 smithi086 ceph-mon[30943]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.362 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:28 smithi086 ceph-mon[30943]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.362 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:28 smithi086 ceph-mon[30943]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.362 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:28 smithi086 ceph-mon[30943]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-09-05T06:41:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:28 smithi096 ceph-mon[34468]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:28 smithi096 ceph-mon[34468]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:28 smithi096 ceph-mon[34468]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:28 smithi096 ceph-mon[34468]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-09-05T06:41:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 -- 172.21.15.86:0/3309631013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 msgr2=0x7f2c3c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/3309631013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c10ea90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009900 tx=0x7f2c2c02a410 comp rx=0 tx=0).stop 2024-09-05T06:41:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 -- 172.21.15.86:0/3309631013 shutdown_connections 2024-09-05T06:41:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/3309631013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c10ea90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/3309631013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 0x7f2c3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/3309631013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 0x7f2c3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 -- 172.21.15.86:0/3309631013 >> 172.21.15.86:0/3309631013 conn(0x7f2c3c0fd8c0 msgr2=0x7f2c3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 -- 172.21.15.86:0/3309631013 shutdown_connections 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.596+0000 7f2c43cc2640 1 -- 172.21.15.86:0/3309631013 wait complete. 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 Processor -- start 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 -- start start 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 0x7f2c3c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 0x7f2c3c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c111e20 con 0x7f2c3c106f10 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c111ca0 con 0x7f2c3c101de0 2024-09-05T06:41:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c43cc2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c111fa0 con 0x7f2c3c102790 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c4225f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c4225f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44358/0 (socket says 172.21.15.86:44358) 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c4225f640 1 -- 172.21.15.86:0/791317630 learned_addr learned my addr 172.21.15.86:0/791317630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c41a5e640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 0x7f2c3c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.597+0000 7f2c4125d640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 0x7f2c3c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 -- 172.21.15.86:0/791317630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 msgr2=0x7f2c3c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 0x7f2c3c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 -- 172.21.15.86:0/791317630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 msgr2=0x7f2c3c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 0x7f2c3c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 -- 172.21.15.86:0/791317630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c19d250 con 0x7f2c3c106f10 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c4225f640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009230 tx=0x7f2c2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c03f030 con 0x7f2c3c106f10 2024-09-05T06:41:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c2c003aa0 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c19d4e0 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c035070 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.598+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1a4e20 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.599+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c1021c0 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.600+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f2c2c042030 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.601+0000 7f2c3affd640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 0x7f2c240a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.601+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2c2c0f04f0 con 0x7f2c3c106f10 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.601+0000 7f2c41a5e640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 0x7f2c240a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.601+0000 7f2c41a5e640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 0x7f2c240a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fd0 tx=0x7f2c34009680 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.604+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c2c0bcbc0 con 0x7f2c3c106f10 2024-09-05T06:41:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:28 smithi115 ceph-mon[34791]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:28 smithi115 ceph-mon[34791]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:28 smithi115 ceph-mon[34791]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T06:41:28.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:28 smithi115 ceph-mon[34791]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-09-05T06:41:28.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.769+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c3c10a1c0 con 0x7f2c240a2e00 2024-09-05T06:41:28.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.777+0000 7f2c3affd640 1 -- 172.21.15.86:0/791317630 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339454 (secure 0 0 0) 0x7f2c34009c20 con 0x7f2c240a2e00 2024-09-05T06:41:28.780 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.780+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 msgr2=0x7f2c240a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.780+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 0x7f2c240a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fd0 tx=0x7f2c34009680 comp rx=0 tx=0).stop 2024-09-05T06:41:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.780+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 msgr2=0x7f2c3c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.780+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009230 tx=0x7f2c2c004270 comp rx=0 tx=0).stop 2024-09-05T06:41:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 shutdown_connections 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2c240a2e00 0x7f2c240a52c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c106f10 0x7f2c3c19cb70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c3c102790 0x7f2c3c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 --2- 172.21.15.86:0/791317630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c3c101de0 0x7f2c3c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 >> 172.21.15.86:0/791317630 conn(0x7f2c3c0fd8c0 msgr2=0x7f2c3c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 shutdown_connections 2024-09-05T06:41:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:28.781+0000 7f2c43cc2640 1 -- 172.21.15.86:0/791317630 wait complete. 2024-09-05T06:41:29.468 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-05T06:41:27.351338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392214,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1632,"num_read_kb":1194,"num_write":653,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1003,"ondisk_log_size":1003,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390224,"kb_used_data":15992,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551600,"statfs":{"total":1151940427776,"available":1151540838400,"internally_reserved":0,"allocated":16375808,"data_stored":9802316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200946,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":314,"num_read_kb":368,"num_write":87,"num_write_kb":658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.995008"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.720102+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:25.720102+0000","last_peered":"2024-09-05T06:41:25.720102+0000","last_clean":"2024-09-05T06:41:25.720102+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:25.720102+0000","last_undegraded":"2024-09-05T06:41:25.720102+0000","last_fullsized":"2024-09-05T06:41:25.720102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354945+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354945+0000","last_peered":"2024-09-05T06:41:15.354945+0000","last_clean":"2024-09-05T06:41:15.354945+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354945+0000","last_undegraded":"2024-09-05T06:41:15.354945+0000","last_fullsized":"2024-09-05T06:41:15.354945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384091+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.384091+0000","last_peered":"2024-09-05T06:41:15.384091+0000","last_clean":"2024-09-05T06:41:15.384091+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.384091+0000","last_undegraded":"2024-09-05T06:41:15.384091+0000","last_fullsized":"2024-09-05T06:41:15.384091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350654+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350654+0000","last_peered":"2024-09-05T06:41:15.350654+0000","last_clean":"2024-09-05T06:41:15.350654+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350654+0000","last_undegraded":"2024-09-05T06:41:15.350654+0000","last_fullsized":"2024-09-05T06:41:15.350654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385703+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385703+0000","last_peered":"2024-09-05T06:41:15.385703+0000","last_clean":"2024-09-05T06:41:15.385703+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385703+0000","last_undegraded":"2024-09-05T06:41:15.385703+0000","last_fullsized":"2024-09-05T06:41:15.385703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345978+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345978+0000","last_peered":"2024-09-05T06:41:15.345978+0000","last_clean":"2024-09-05T06:41:15.345978+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345978+0000","last_undegraded":"2024-09-05T06:41:15.345978+0000","last_fullsized":"2024-09-05T06:41:15.345978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391359+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391359+0000","last_peered":"2024-09-05T06:41:15.391359+0000","last_clean":"2024-09-05T06:41:15.391359+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391359+0000","last_undegraded":"2024-09-05T06:41:15.391359+0000","last_fullsized":"2024-09-05T06:41:15.391359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345763+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345763+0000","last_peered":"2024-09-05T06:41:15.345763+0000","last_clean":"2024-09-05T06:41:15.345763+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345763+0000","last_undegraded":"2024-09-05T06:41:15.345763+0000","last_fullsized":"2024-09-05T06:41:15.345763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346048+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346048+0000","last_peered":"2024-09-05T06:41:15.346048+0000","last_clean":"2024-09-05T06:41:15.346048+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346048+0000","last_undegraded":"2024-09-05T06:41:15.346048+0000","last_fullsized":"2024-09-05T06:41:15.346048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'76","reported_seq":180,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.210943+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:26.210943+0000","last_peered":"2024-09-05T06:41:26.210943+0000","last_clean":"2024-09-05T06:41:26.210943+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:26.210943+0000","last_undegraded":"2024-09-05T06:41:26.210943+0000","last_fullsized":"2024-09-05T06:41:26.210943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":136,"num_write_kb":1158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.015271+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:25.015271+0000","last_peered":"2024-09-05T06:41:25.015271+0000","last_clean":"2024-09-05T06:41:25.015271+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:25.015271+0000","last_undegraded":"2024-09-05T06:41:25.015271+0000","last_fullsized":"2024-09-05T06:41:25.015271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006131+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:25.006131+0000","last_peered":"2024-09-05T06:41:25.006131+0000","last_clean":"2024-09-05T06:41:25.006131+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:25.006131+0000","last_undegraded":"2024-09-05T06:41:25.006131+0000","last_fullsized":"2024-09-05T06:41:25.006131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":406,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:17.464338+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:17.464338+0000","last_peered":"2024-09-05T06:41:17.464338+0000","last_clean":"2024-09-05T06:41:17.464338+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:17.464338+0000","last_undegraded":"2024-09-05T06:41:17.464338+0000","last_fullsized":"2024-09-05T06:41:17.464338+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.075812+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:25.075812+0000","last_peered":"2024-09-05T06:41:25.075812+0000","last_clean":"2024-09-05T06:41:25.075812+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:25.075812+0000","last_undegraded":"2024-09-05T06:41:25.075812+0000","last_fullsized":"2024-09-05T06:41:25.075812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'59","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.206209+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:26.206209+0000","last_peered":"2024-09-05T06:41:26.206209+0000","last_clean":"2024-09-05T06:41:26.206209+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:26.206209+0000","last_undegraded":"2024-09-05T06:41:26.206209+0000","last_fullsized":"2024-09-05T06:41:26.206209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336654+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336654+0000","last_peered":"2024-09-05T06:41:15.336654+0000","last_clean":"2024-09-05T06:41:15.336654+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336654+0000","last_undegraded":"2024-09-05T06:41:15.336654+0000","last_fullsized":"2024-09-05T06:41:15.336654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006632+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:25.006632+0000","last_peered":"2024-09-05T06:41:25.006632+0000","last_clean":"2024-09-05T06:41:25.006632+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:25.006632+0000","last_undegraded":"2024-09-05T06:41:25.006632+0000","last_fullsized":"2024-09-05T06:41:25.006632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390371+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390371+0000","last_peered":"2024-09-05T06:41:15.390371+0000","last_clean":"2024-09-05T06:41:15.390371+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390371+0000","last_undegraded":"2024-09-05T06:41:15.390371+0000","last_fullsized":"2024-09-05T06:41:15.390371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353200+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353200+0000","last_peered":"2024-09-05T06:41:15.353200+0000","last_clean":"2024-09-05T06:41:15.353200+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353200+0000","last_undegraded":"2024-09-05T06:41:15.353200+0000","last_fullsized":"2024-09-05T06:41:15.353200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381695+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381695+0000","last_peered":"2024-09-05T06:41:15.381695+0000","last_clean":"2024-09-05T06:41:15.381695+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381695+0000","last_undegraded":"2024-09-05T06:41:15.381695+0000","last_fullsized":"2024-09-05T06:41:15.381695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336222+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336222+0000","last_peered":"2024-09-05T06:41:15.336222+0000","last_clean":"2024-09-05T06:41:15.336222+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336222+0000","last_undegraded":"2024-09-05T06:41:15.336222+0000","last_fullsized":"2024-09-05T06:41:15.336222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368981+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368981+0000","last_peered":"2024-09-05T06:41:15.368981+0000","last_clean":"2024-09-05T06:41:15.368981+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368981+0000","last_undegraded":"2024-09-05T06:41:15.368981+0000","last_fullsized":"2024-09-05T06:41:15.368981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.007425+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:26.007425+0000","last_peered":"2024-09-05T06:41:26.007425+0000","last_clean":"2024-09-05T06:41:26.007425+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:26.007425+0000","last_undegraded":"2024-09-05T06:41:26.007425+0000","last_fullsized":"2024-09-05T06:41:26.007425+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69515,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":190,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357275+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357275+0000","last_peered":"2024-09-05T06:41:15.357275+0000","last_clean":"2024-09-05T06:41:15.357275+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357275+0000","last_undegraded":"2024-09-05T06:41:15.357275+0000","last_fullsized":"2024-09-05T06:41:15.357275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385754+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.385754+0000","last_peered":"2024-09-05T06:41:15.385754+0000","last_clean":"2024-09-05T06:41:15.385754+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.385754+0000","last_undegraded":"2024-09-05T06:41:15.385754+0000","last_fullsized":"2024-09-05T06:41:15.385754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346427+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346427+0000","last_peered":"2024-09-05T06:41:15.346427+0000","last_clean":"2024-09-05T06:41:15.346427+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346427+0000","last_undegraded":"2024-09-05T06:41:15.346427+0000","last_fullsized":"2024-09-05T06:41:15.346427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392670+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392670+0000","last_peered":"2024-09-05T06:41:15.392670+0000","last_clean":"2024-09-05T06:41:15.392670+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392670+0000","last_undegraded":"2024-09-05T06:41:15.392670+0000","last_fullsized":"2024-09-05T06:41:15.392670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357061+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.357061+0000","last_peered":"2024-09-05T06:41:15.357061+0000","last_clean":"2024-09-05T06:41:15.357061+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.357061+0000","last_undegraded":"2024-09-05T06:41:15.357061+0000","last_fullsized":"2024-09-05T06:41:15.357061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":275,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.007416+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:26.007416+0000","last_peered":"2024-09-05T06:41:26.007416+0000","last_clean":"2024-09-05T06:41:26.007416+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:26.007416+0000","last_undegraded":"2024-09-05T06:41:26.007416+0000","last_fullsized":"2024-09-05T06:41:26.007416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351201+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351201+0000","last_peered":"2024-09-05T06:41:15.351201+0000","last_clean":"2024-09-05T06:41:15.351201+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351201+0000","last_undegraded":"2024-09-05T06:41:15.351201+0000","last_fullsized":"2024-09-05T06:41:15.351201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388863+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388863+0000","last_peered":"2024-09-05T06:41:15.388863+0000","last_clean":"2024-09-05T06:41:15.388863+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388863+0000","last_undegraded":"2024-09-05T06:41:15.388863+0000","last_fullsized":"2024-09-05T06:41:15.388863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336013+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.336013+0000","last_peered":"2024-09-05T06:41:15.336013+0000","last_clean":"2024-09-05T06:41:15.336013+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.336013+0000","last_undegraded":"2024-09-05T06:41:15.336013+0000","last_fullsized":"2024-09-05T06:41:15.336013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379081+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.379081+0000","last_peered":"2024-09-05T06:41:15.379081+0000","last_clean":"2024-09-05T06:41:15.379081+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.379081+0000","last_undegraded":"2024-09-05T06:41:15.379081+0000","last_fullsized":"2024-09-05T06:41:15.379081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336163+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336163+0000","last_peered":"2024-09-05T06:41:15.336163+0000","last_clean":"2024-09-05T06:41:15.336163+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336163+0000","last_undegraded":"2024-09-05T06:41:15.336163+0000","last_fullsized":"2024-09-05T06:41:15.336163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346070+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346070+0000","last_peered":"2024-09-05T06:41:15.346070+0000","last_clean":"2024-09-05T06:41:15.346070+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346070+0000","last_undegraded":"2024-09-05T06:41:15.346070+0000","last_fullsized":"2024-09-05T06:41:15.346070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.008167+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:26.008167+0000","last_peered":"2024-09-05T06:41:26.008167+0000","last_clean":"2024-09-05T06:41:26.008167+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:26.008167+0000","last_undegraded":"2024-09-05T06:41:26.008167+0000","last_fullsized":"2024-09-05T06:41:26.008167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379840+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379840+0000","last_peered":"2024-09-05T06:41:15.379840+0000","last_clean":"2024-09-05T06:41:15.379840+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379840+0000","last_undegraded":"2024-09-05T06:41:15.379840+0000","last_fullsized":"2024-09-05T06:41:15.379840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390219+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390219+0000","last_peered":"2024-09-05T06:41:15.390219+0000","last_clean":"2024-09-05T06:41:15.390219+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390219+0000","last_undegraded":"2024-09-05T06:41:15.390219+0000","last_fullsized":"2024-09-05T06:41:15.390219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392054+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.392054+0000","last_peered":"2024-09-05T06:41:15.392054+0000","last_clean":"2024-09-05T06:41:15.392054+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.392054+0000","last_undegraded":"2024-09-05T06:41:15.392054+0000","last_fullsized":"2024-09-05T06:41:15.392054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386510+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386510+0000","last_peered":"2024-09-05T06:41:15.386510+0000","last_clean":"2024-09-05T06:41:15.386510+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386510+0000","last_undegraded":"2024-09-05T06:41:15.386510+0000","last_fullsized":"2024-09-05T06:41:15.386510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386966+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386966+0000","last_peered":"2024-09-05T06:41:15.386966+0000","last_clean":"2024-09-05T06:41:15.386966+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386966+0000","last_undegraded":"2024-09-05T06:41:15.386966+0000","last_fullsized":"2024-09-05T06:41:15.386966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394223+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.394223+0000","last_peered":"2024-09-05T06:41:15.394223+0000","last_clean":"2024-09-05T06:41:15.394223+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.394223+0000","last_undegraded":"2024-09-05T06:41:15.394223+0000","last_fullsized":"2024-09-05T06:41:15.394223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386226+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386226+0000","last_peered":"2024-09-05T06:41:15.386226+0000","last_clean":"2024-09-05T06:41:15.386226+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386226+0000","last_undegraded":"2024-09-05T06:41:15.386226+0000","last_fullsized":"2024-09-05T06:41:15.386226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383778+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383778+0000","last_peered":"2024-09-05T06:41:15.383778+0000","last_clean":"2024-09-05T06:41:15.383778+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383778+0000","last_undegraded":"2024-09-05T06:41:15.383778+0000","last_fullsized":"2024-09-05T06:41:15.383778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384549+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.384549+0000","last_peered":"2024-09-05T06:41:15.384549+0000","last_clean":"2024-09-05T06:41:15.384549+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.384549+0000","last_undegraded":"2024-09-05T06:41:15.384549+0000","last_fullsized":"2024-09-05T06:41:15.384549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379776+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.379776+0000","last_peered":"2024-09-05T06:41:15.379776+0000","last_clean":"2024-09-05T06:41:15.379776+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.379776+0000","last_undegraded":"2024-09-05T06:41:15.379776+0000","last_fullsized":"2024-09-05T06:41:15.379776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385732+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.385732+0000","last_peered":"2024-09-05T06:41:15.385732+0000","last_clean":"2024-09-05T06:41:15.385732+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.385732+0000","last_undegraded":"2024-09-05T06:41:15.385732+0000","last_fullsized":"2024-09-05T06:41:15.385732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801612,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1395,"num_read_kb":864,"num_write":450,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841584,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1884160,"data_stored":1309933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005110,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1470464,"data_stored":917822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1120726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1810432,"data_stored":1245598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":591025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1040538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":587447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:29.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:41:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 -- 172.21.15.86:0/1112658091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 msgr2=0x7f82501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 --2- 172.21.15.86:0/1112658091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501029b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8238009900 tx=0x7f823802a480 comp rx=0 tx=0).stop 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 -- 172.21.15.86:0/1112658091 shutdown_connections 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 --2- 172.21.15.86:0/1112658091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250102ef0 0x7f825010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 --2- 172.21.15.86:0/1112658091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501029b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 --2- 172.21.15.86:0/1112658091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250100e20 0x7f8250101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 -- 172.21.15.86:0/1112658091 >> 172.21.15.86:0/1112658091 conn(0x7f82500fc7d0 msgr2=0x7f82500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:30.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 -- 172.21.15.86:0/1112658091 shutdown_connections 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.189+0000 7f8254b8c640 1 -- 172.21.15.86:0/1112658091 wait complete. 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 Processor -- start 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 -- start start 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250100e20 0x7f82501983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250102ef0 0x7f825019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8250110e30 con 0x7f8250102570 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8250110cb0 con 0x7f8250100e20 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8254b8c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8250110fb0 con 0x7f8250102ef0 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44378/0 (socket says 172.21.15.86:44378) 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 -- 172.21.15.86:0/2214715049 learned_addr learned my addr 172.21.15.86:0/2214715049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f824ed9d640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250102ef0 0x7f825019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f824e59c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250100e20 0x7f82501983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 -- 172.21.15.86:0/2214715049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250102ef0 msgr2=0x7f825019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250102ef0 0x7f825019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 -- 172.21.15.86:0/2214715049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250100e20 msgr2=0x7f82501983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250100e20 0x7f82501983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 -- 172.21.15.86:0/2214715049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825019d360 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.190+0000 7f8247fff640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f82380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.191+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823803f030 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.191+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8238003cb0 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.191+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825019d5f0 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.191+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238035070 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.191+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82501a4f30 con 0x7f8250102570 2024-09-05T06:41:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.192+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82501021b0 con 0x7f8250102570 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.193+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8238042030 con 0x7f8250102570 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.193+0000 7f82477fe640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 0x7f823c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.193+0000 7f824e59c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 0x7f823c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.193+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f82380efe00 con 0x7f8250102570 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.194+0000 7f824e59c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 0x7f823c0a5240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8240005fd0 tx=0x7f824000b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.198+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82380bbe80 con 0x7f8250102570 2024-09-05T06:41:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.366+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8250062630 con 0x7f823c0a2d80 2024-09-05T06:41:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.374+0000 7f82477fe640 1 -- 172.21.15.86:0/2214715049 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339455 (secure 0 0 0) 0x7f8250062630 con 0x7f823c0a2d80 2024-09-05T06:41:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 msgr2=0x7f823c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 0x7f823c0a5240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8240005fd0 tx=0x7f824000b390 comp rx=0 tx=0).stop 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 msgr2=0x7f82501988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8238006010 tx=0x7f82380042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 shutdown_connections 2024-09-05T06:41:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f823c0a2d80 0x7f823c0a5240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8250102ef0 0x7f825019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8250102570 0x7f82501988f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 --2- 172.21.15.86:0/2214715049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8250100e20 0x7f82501983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 >> 172.21.15.86:0/2214715049 conn(0x7f82500fc7d0 msgr2=0x7f825006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 shutdown_connections 2024-09-05T06:41:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:30.377+0000 7f8254b8c640 1 -- 172.21.15.86:0/2214715049 wait complete. 2024-09-05T06:41:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:30 smithi115 ceph-mon[34791]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:30.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:30 smithi115 ceph-mon[34791]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T06:41:30.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:30 smithi086 ceph-mon[30943]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:30.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:30 smithi086 ceph-mon[30943]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T06:41:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:30 smithi096 ceph-mon[34468]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:30 smithi096 ceph-mon[34468]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T06:41:31.228 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-05T06:41:29.351911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392214,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1632,"num_read_kb":1194,"num_write":653,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1003,"ondisk_log_size":1003,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390224,"kb_used_data":15992,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551600,"statfs":{"total":1151940427776,"available":1151540838400,"internally_reserved":0,"allocated":16375808,"data_stored":9802316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200946,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":314,"num_read_kb":368,"num_write":87,"num_write_kb":658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002682"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.720102+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:25.720102+0000","last_peered":"2024-09-05T06:41:25.720102+0000","last_clean":"2024-09-05T06:41:25.720102+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:25.720102+0000","last_undegraded":"2024-09-05T06:41:25.720102+0000","last_fullsized":"2024-09-05T06:41:25.720102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354945+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354945+0000","last_peered":"2024-09-05T06:41:15.354945+0000","last_clean":"2024-09-05T06:41:15.354945+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354945+0000","last_undegraded":"2024-09-05T06:41:15.354945+0000","last_fullsized":"2024-09-05T06:41:15.354945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384091+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.384091+0000","last_peered":"2024-09-05T06:41:15.384091+0000","last_clean":"2024-09-05T06:41:15.384091+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.384091+0000","last_undegraded":"2024-09-05T06:41:15.384091+0000","last_fullsized":"2024-09-05T06:41:15.384091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350654+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350654+0000","last_peered":"2024-09-05T06:41:15.350654+0000","last_clean":"2024-09-05T06:41:15.350654+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350654+0000","last_undegraded":"2024-09-05T06:41:15.350654+0000","last_fullsized":"2024-09-05T06:41:15.350654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385703+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385703+0000","last_peered":"2024-09-05T06:41:15.385703+0000","last_clean":"2024-09-05T06:41:15.385703+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385703+0000","last_undegraded":"2024-09-05T06:41:15.385703+0000","last_fullsized":"2024-09-05T06:41:15.385703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345978+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345978+0000","last_peered":"2024-09-05T06:41:15.345978+0000","last_clean":"2024-09-05T06:41:15.345978+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345978+0000","last_undegraded":"2024-09-05T06:41:15.345978+0000","last_fullsized":"2024-09-05T06:41:15.345978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391359+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391359+0000","last_peered":"2024-09-05T06:41:15.391359+0000","last_clean":"2024-09-05T06:41:15.391359+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391359+0000","last_undegraded":"2024-09-05T06:41:15.391359+0000","last_fullsized":"2024-09-05T06:41:15.391359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345763+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345763+0000","last_peered":"2024-09-05T06:41:15.345763+0000","last_clean":"2024-09-05T06:41:15.345763+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345763+0000","last_undegraded":"2024-09-05T06:41:15.345763+0000","last_fullsized":"2024-09-05T06:41:15.345763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346048+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346048+0000","last_peered":"2024-09-05T06:41:15.346048+0000","last_clean":"2024-09-05T06:41:15.346048+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346048+0000","last_undegraded":"2024-09-05T06:41:15.346048+0000","last_fullsized":"2024-09-05T06:41:15.346048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'76","reported_seq":180,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.210943+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:26.210943+0000","last_peered":"2024-09-05T06:41:26.210943+0000","last_clean":"2024-09-05T06:41:26.210943+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:26.210943+0000","last_undegraded":"2024-09-05T06:41:26.210943+0000","last_fullsized":"2024-09-05T06:41:26.210943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":136,"num_write_kb":1158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.015271+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:25.015271+0000","last_peered":"2024-09-05T06:41:25.015271+0000","last_clean":"2024-09-05T06:41:25.015271+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:25.015271+0000","last_undegraded":"2024-09-05T06:41:25.015271+0000","last_fullsized":"2024-09-05T06:41:25.015271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006131+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:25.006131+0000","last_peered":"2024-09-05T06:41:25.006131+0000","last_clean":"2024-09-05T06:41:25.006131+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:25.006131+0000","last_undegraded":"2024-09-05T06:41:25.006131+0000","last_fullsized":"2024-09-05T06:41:25.006131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":406,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:17.464338+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:17.464338+0000","last_peered":"2024-09-05T06:41:17.464338+0000","last_clean":"2024-09-05T06:41:17.464338+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:17.464338+0000","last_undegraded":"2024-09-05T06:41:17.464338+0000","last_fullsized":"2024-09-05T06:41:17.464338+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.075812+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:25.075812+0000","last_peered":"2024-09-05T06:41:25.075812+0000","last_clean":"2024-09-05T06:41:25.075812+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:25.075812+0000","last_undegraded":"2024-09-05T06:41:25.075812+0000","last_fullsized":"2024-09-05T06:41:25.075812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'59","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.206209+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:26.206209+0000","last_peered":"2024-09-05T06:41:26.206209+0000","last_clean":"2024-09-05T06:41:26.206209+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:26.206209+0000","last_undegraded":"2024-09-05T06:41:26.206209+0000","last_fullsized":"2024-09-05T06:41:26.206209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336654+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336654+0000","last_peered":"2024-09-05T06:41:15.336654+0000","last_clean":"2024-09-05T06:41:15.336654+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336654+0000","last_undegraded":"2024-09-05T06:41:15.336654+0000","last_fullsized":"2024-09-05T06:41:15.336654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006632+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:25.006632+0000","last_peered":"2024-09-05T06:41:25.006632+0000","last_clean":"2024-09-05T06:41:25.006632+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:25.006632+0000","last_undegraded":"2024-09-05T06:41:25.006632+0000","last_fullsized":"2024-09-05T06:41:25.006632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390371+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390371+0000","last_peered":"2024-09-05T06:41:15.390371+0000","last_clean":"2024-09-05T06:41:15.390371+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390371+0000","last_undegraded":"2024-09-05T06:41:15.390371+0000","last_fullsized":"2024-09-05T06:41:15.390371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353200+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353200+0000","last_peered":"2024-09-05T06:41:15.353200+0000","last_clean":"2024-09-05T06:41:15.353200+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353200+0000","last_undegraded":"2024-09-05T06:41:15.353200+0000","last_fullsized":"2024-09-05T06:41:15.353200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381695+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381695+0000","last_peered":"2024-09-05T06:41:15.381695+0000","last_clean":"2024-09-05T06:41:15.381695+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381695+0000","last_undegraded":"2024-09-05T06:41:15.381695+0000","last_fullsized":"2024-09-05T06:41:15.381695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336222+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336222+0000","last_peered":"2024-09-05T06:41:15.336222+0000","last_clean":"2024-09-05T06:41:15.336222+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336222+0000","last_undegraded":"2024-09-05T06:41:15.336222+0000","last_fullsized":"2024-09-05T06:41:15.336222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368981+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368981+0000","last_peered":"2024-09-05T06:41:15.368981+0000","last_clean":"2024-09-05T06:41:15.368981+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368981+0000","last_undegraded":"2024-09-05T06:41:15.368981+0000","last_fullsized":"2024-09-05T06:41:15.368981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":340,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.007425+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:26.007425+0000","last_peered":"2024-09-05T06:41:26.007425+0000","last_clean":"2024-09-05T06:41:26.007425+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:26.007425+0000","last_undegraded":"2024-09-05T06:41:26.007425+0000","last_fullsized":"2024-09-05T06:41:26.007425+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69515,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":190,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357275+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357275+0000","last_peered":"2024-09-05T06:41:15.357275+0000","last_clean":"2024-09-05T06:41:15.357275+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357275+0000","last_undegraded":"2024-09-05T06:41:15.357275+0000","last_fullsized":"2024-09-05T06:41:15.357275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385754+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.385754+0000","last_peered":"2024-09-05T06:41:15.385754+0000","last_clean":"2024-09-05T06:41:15.385754+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.385754+0000","last_undegraded":"2024-09-05T06:41:15.385754+0000","last_fullsized":"2024-09-05T06:41:15.385754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346427+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346427+0000","last_peered":"2024-09-05T06:41:15.346427+0000","last_clean":"2024-09-05T06:41:15.346427+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346427+0000","last_undegraded":"2024-09-05T06:41:15.346427+0000","last_fullsized":"2024-09-05T06:41:15.346427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392670+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392670+0000","last_peered":"2024-09-05T06:41:15.392670+0000","last_clean":"2024-09-05T06:41:15.392670+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392670+0000","last_undegraded":"2024-09-05T06:41:15.392670+0000","last_fullsized":"2024-09-05T06:41:15.392670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357061+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.357061+0000","last_peered":"2024-09-05T06:41:15.357061+0000","last_clean":"2024-09-05T06:41:15.357061+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.357061+0000","last_undegraded":"2024-09-05T06:41:15.357061+0000","last_fullsized":"2024-09-05T06:41:15.357061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":275,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.007416+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:26.007416+0000","last_peered":"2024-09-05T06:41:26.007416+0000","last_clean":"2024-09-05T06:41:26.007416+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:26.007416+0000","last_undegraded":"2024-09-05T06:41:26.007416+0000","last_fullsized":"2024-09-05T06:41:26.007416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351201+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351201+0000","last_peered":"2024-09-05T06:41:15.351201+0000","last_clean":"2024-09-05T06:41:15.351201+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351201+0000","last_undegraded":"2024-09-05T06:41:15.351201+0000","last_fullsized":"2024-09-05T06:41:15.351201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388863+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388863+0000","last_peered":"2024-09-05T06:41:15.388863+0000","last_clean":"2024-09-05T06:41:15.388863+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388863+0000","last_undegraded":"2024-09-05T06:41:15.388863+0000","last_fullsized":"2024-09-05T06:41:15.388863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336013+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.336013+0000","last_peered":"2024-09-05T06:41:15.336013+0000","last_clean":"2024-09-05T06:41:15.336013+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.336013+0000","last_undegraded":"2024-09-05T06:41:15.336013+0000","last_fullsized":"2024-09-05T06:41:15.336013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379081+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.379081+0000","last_peered":"2024-09-05T06:41:15.379081+0000","last_clean":"2024-09-05T06:41:15.379081+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.379081+0000","last_undegraded":"2024-09-05T06:41:15.379081+0000","last_fullsized":"2024-09-05T06:41:15.379081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336163+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336163+0000","last_peered":"2024-09-05T06:41:15.336163+0000","last_clean":"2024-09-05T06:41:15.336163+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336163+0000","last_undegraded":"2024-09-05T06:41:15.336163+0000","last_fullsized":"2024-09-05T06:41:15.336163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346070+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346070+0000","last_peered":"2024-09-05T06:41:15.346070+0000","last_clean":"2024-09-05T06:41:15.346070+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346070+0000","last_undegraded":"2024-09-05T06:41:15.346070+0000","last_fullsized":"2024-09-05T06:41:15.346070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.008167+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:26.008167+0000","last_peered":"2024-09-05T06:41:26.008167+0000","last_clean":"2024-09-05T06:41:26.008167+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:26.008167+0000","last_undegraded":"2024-09-05T06:41:26.008167+0000","last_fullsized":"2024-09-05T06:41:26.008167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379840+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379840+0000","last_peered":"2024-09-05T06:41:15.379840+0000","last_clean":"2024-09-05T06:41:15.379840+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379840+0000","last_undegraded":"2024-09-05T06:41:15.379840+0000","last_fullsized":"2024-09-05T06:41:15.379840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390219+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390219+0000","last_peered":"2024-09-05T06:41:15.390219+0000","last_clean":"2024-09-05T06:41:15.390219+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390219+0000","last_undegraded":"2024-09-05T06:41:15.390219+0000","last_fullsized":"2024-09-05T06:41:15.390219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392054+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.392054+0000","last_peered":"2024-09-05T06:41:15.392054+0000","last_clean":"2024-09-05T06:41:15.392054+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.392054+0000","last_undegraded":"2024-09-05T06:41:15.392054+0000","last_fullsized":"2024-09-05T06:41:15.392054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386510+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386510+0000","last_peered":"2024-09-05T06:41:15.386510+0000","last_clean":"2024-09-05T06:41:15.386510+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386510+0000","last_undegraded":"2024-09-05T06:41:15.386510+0000","last_fullsized":"2024-09-05T06:41:15.386510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386966+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386966+0000","last_peered":"2024-09-05T06:41:15.386966+0000","last_clean":"2024-09-05T06:41:15.386966+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386966+0000","last_undegraded":"2024-09-05T06:41:15.386966+0000","last_fullsized":"2024-09-05T06:41:15.386966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394223+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.394223+0000","last_peered":"2024-09-05T06:41:15.394223+0000","last_clean":"2024-09-05T06:41:15.394223+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.394223+0000","last_undegraded":"2024-09-05T06:41:15.394223+0000","last_fullsized":"2024-09-05T06:41:15.394223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386226+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386226+0000","last_peered":"2024-09-05T06:41:15.386226+0000","last_clean":"2024-09-05T06:41:15.386226+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386226+0000","last_undegraded":"2024-09-05T06:41:15.386226+0000","last_fullsized":"2024-09-05T06:41:15.386226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383778+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383778+0000","last_peered":"2024-09-05T06:41:15.383778+0000","last_clean":"2024-09-05T06:41:15.383778+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383778+0000","last_undegraded":"2024-09-05T06:41:15.383778+0000","last_fullsized":"2024-09-05T06:41:15.383778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384549+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.384549+0000","last_peered":"2024-09-05T06:41:15.384549+0000","last_clean":"2024-09-05T06:41:15.384549+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.384549+0000","last_undegraded":"2024-09-05T06:41:15.384549+0000","last_fullsized":"2024-09-05T06:41:15.384549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379776+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.379776+0000","last_peered":"2024-09-05T06:41:15.379776+0000","last_clean":"2024-09-05T06:41:15.379776+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.379776+0000","last_undegraded":"2024-09-05T06:41:15.379776+0000","last_fullsized":"2024-09-05T06:41:15.379776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385732+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.385732+0000","last_peered":"2024-09-05T06:41:15.385732+0000","last_clean":"2024-09-05T06:41:15.385732+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.385732+0000","last_undegraded":"2024-09-05T06:41:15.385732+0000","last_fullsized":"2024-09-05T06:41:15.385732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801612,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1395,"num_read_kb":864,"num_write":450,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841584,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1884160,"data_stored":1309933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005110,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1470464,"data_stored":917822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1120726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1810432,"data_stored":1245598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":591025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1040538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":587447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:31.233 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T06:41:31.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T06:41:31.485 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:31.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:31 smithi086 ceph-mon[30943]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:31 smithi115 ceph-mon[34791]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:31 smithi096 ceph-mon[34468]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 -- 172.21.15.86:0/381667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 msgr2=0x7f39441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 --2- 172.21.15.86:0/381667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441029b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c02a480 comp rx=0 tx=0).stop 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 -- 172.21.15.86:0/381667519 shutdown_connections 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 --2- 172.21.15.86:0/381667519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944102ef0 0x7f394410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 --2- 172.21.15.86:0/381667519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441029b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 --2- 172.21.15.86:0/381667519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944100e20 0x7f3944101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 -- 172.21.15.86:0/381667519 >> 172.21.15.86:0/381667519 conn(0x7f39440fc7d0 msgr2=0x7f39440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:32.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 -- 172.21.15.86:0/381667519 shutdown_connections 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.005+0000 7f39492a3640 1 -- 172.21.15.86:0/381667519 wait complete. 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 Processor -- start 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 -- start start 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944100e20 0x7f3944198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 0x7f394419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110e30 con 0x7f3944102570 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110cb0 con 0x7f3944102ef0 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39492a3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110fb0 con 0x7f3944100e20 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.006+0000 7f39427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44396/0 (socket says 172.21.15.86:44396) 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 -- 172.21.15.86:0/331063225 learned_addr learned my addr 172.21.15.86:0/331063225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:32.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f3942ffd640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944100e20 0x7f3944198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39437fe640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 0x7f394419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 -- 172.21.15.86:0/331063225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944100e20 msgr2=0x7f3944198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944100e20 0x7f3944198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 -- 172.21.15.86:0/331063225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 msgr2=0x7f394419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 0x7f394419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 -- 172.21.15.86:0/331063225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394419d420 con 0x7f3944102570 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39437fe640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 0x7f394419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39427fc640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c03f030 con 0x7f3944102570 2024-09-05T06:41:32.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394419d6b0 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39441a4ff0 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392c003cb0 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.007+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c049060 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.008+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924005290 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.009+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f392c042030 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.010+0000 7f393bfff640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 0x7f39300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.010+0000 7f3942ffd640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 0x7f39300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.010+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f392c0efef0 con 0x7f3944102570 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.011+0000 7f3942ffd640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 0x7f39300a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3934005fd0 tx=0x7f3934009870 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:32.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.015+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f392c0bc5c0 con 0x7f3944102570 2024-09-05T06:41:32.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.178+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3924000bc0 con 0x7f39300a2d80 2024-09-05T06:41:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.188+0000 7f393bfff640 1 -- 172.21.15.86:0/331063225 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339456 (secure 0 0 0) 0x7f3924000bc0 con 0x7f39300a2d80 2024-09-05T06:41:32.192 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:32.194 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T06:41:32.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 msgr2=0x7f39300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 0x7f39300a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3934005fd0 tx=0x7f3934009870 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 msgr2=0x7f39441989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c0042f0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 shutdown_connections 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2d80 0x7f39300a5240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3944102ef0 0x7f394419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3944102570 0x7f39441989b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 --2- 172.21.15.86:0/331063225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3944100e20 0x7f3944198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 >> 172.21.15.86:0/331063225 conn(0x7f39440fc7d0 msgr2=0x7f394406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 shutdown_connections 2024-09-05T06:41:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:32.191+0000 7f39492a3640 1 -- 172.21.15.86:0/331063225 wait complete. 2024-09-05T06:41:32.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:32 smithi086 ceph-mon[30943]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-09-05T06:41:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:32 smithi115 ceph-mon[34791]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-09-05T06:41:32.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:32 smithi096 ceph-mon[34468]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-09-05T06:41:33.011 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-05T06:41:31.352249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392214,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1670,"num_read_kb":1222,"num_write":661,"num_write_kb":5202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182370,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1011,"ondisk_log_size":1011,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391160,"kb_used_data":16568,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124550664,"statfs":{"total":1151940427776,"available":1151539879936,"internally_reserved":0,"allocated":16965632,"data_stored":10392212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200946,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":352,"num_read_kb":396,"num_write":95,"num_write_kb":792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002601"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.720102+0000","last_change":"2024-09-05T06:38:53.364218+0000","last_active":"2024-09-05T06:41:25.720102+0000","last_peered":"2024-09-05T06:41:25.720102+0000","last_clean":"2024-09-05T06:41:25.720102+0000","last_became_active":"2024-09-05T06:38:53.363964+0000","last_became_peered":"2024-09-05T06:38:53.363964+0000","last_unstale":"2024-09-05T06:41:25.720102+0000","last_undegraded":"2024-09-05T06:41:25.720102+0000","last_fullsized":"2024-09-05T06:41:25.720102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:45:12.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381567+0000","last_change":"2024-09-05T06:38:55.370951+0000","last_active":"2024-09-05T06:41:15.381567+0000","last_peered":"2024-09-05T06:41:15.381567+0000","last_clean":"2024-09-05T06:41:15.381567+0000","last_became_active":"2024-09-05T06:38:55.370669+0000","last_became_peered":"2024-09-05T06:38:55.370669+0000","last_unstale":"2024-09-05T06:41:15.381567+0000","last_undegraded":"2024-09-05T06:41:15.381567+0000","last_fullsized":"2024-09-05T06:41:15.381567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:53:36.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339053+0000","last_change":"2024-09-05T06:40:13.500473+0000","last_active":"2024-09-05T06:41:15.339053+0000","last_peered":"2024-09-05T06:41:15.339053+0000","last_clean":"2024-09-05T06:41:15.339053+0000","last_became_active":"2024-09-05T06:40:13.499935+0000","last_became_peered":"2024-09-05T06:40:13.499935+0000","last_unstale":"2024-09-05T06:41:15.339053+0000","last_undegraded":"2024-09-05T06:41:15.339053+0000","last_fullsized":"2024-09-05T06:41:15.339053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.131558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351748+0000","last_change":"2024-09-05T06:38:53.398268+0000","last_active":"2024-09-05T06:41:15.351748+0000","last_peered":"2024-09-05T06:41:15.351748+0000","last_clean":"2024-09-05T06:41:15.351748+0000","last_became_active":"2024-09-05T06:38:53.397900+0000","last_became_peered":"2024-09-05T06:38:53.397900+0000","last_unstale":"2024-09-05T06:41:15.351748+0000","last_undegraded":"2024-09-05T06:41:15.351748+0000","last_fullsized":"2024-09-05T06:41:15.351748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:12.508905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395548+0000","last_change":"2024-09-05T06:38:55.372203+0000","last_active":"2024-09-05T06:41:15.395548+0000","last_peered":"2024-09-05T06:41:15.395548+0000","last_clean":"2024-09-05T06:41:15.395548+0000","last_became_active":"2024-09-05T06:38:55.371754+0000","last_became_peered":"2024-09-05T06:38:55.371754+0000","last_unstale":"2024-09-05T06:41:15.395548+0000","last_undegraded":"2024-09-05T06:41:15.395548+0000","last_fullsized":"2024-09-05T06:41:15.395548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:41:29.473348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345200+0000","last_change":"2024-09-05T06:39:02.871045+0000","last_active":"2024-09-05T06:41:15.345200+0000","last_peered":"2024-09-05T06:41:15.345200+0000","last_clean":"2024-09-05T06:41:15.345200+0000","last_became_active":"2024-09-05T06:39:02.870564+0000","last_became_peered":"2024-09-05T06:39:02.870564+0000","last_unstale":"2024-09-05T06:41:15.345200+0000","last_undegraded":"2024-09-05T06:41:15.345200+0000","last_fullsized":"2024-09-05T06:41:15.345200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:36:43.201085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354945+0000","last_change":"2024-09-05T06:38:53.364467+0000","last_active":"2024-09-05T06:41:15.354945+0000","last_peered":"2024-09-05T06:41:15.354945+0000","last_clean":"2024-09-05T06:41:15.354945+0000","last_became_active":"2024-09-05T06:38:53.364119+0000","last_became_peered":"2024-09-05T06:38:53.364119+0000","last_unstale":"2024-09-05T06:41:15.354945+0000","last_undegraded":"2024-09-05T06:41:15.354945+0000","last_fullsized":"2024-09-05T06:41:15.354945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:22.364996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384091+0000","last_change":"2024-09-05T06:38:55.380463+0000","last_active":"2024-09-05T06:41:15.384091+0000","last_peered":"2024-09-05T06:41:15.384091+0000","last_clean":"2024-09-05T06:41:15.384091+0000","last_became_active":"2024-09-05T06:38:55.380101+0000","last_became_peered":"2024-09-05T06:38:55.380101+0000","last_unstale":"2024-09-05T06:41:15.384091+0000","last_undegraded":"2024-09-05T06:41:15.384091+0000","last_fullsized":"2024-09-05T06:41:15.384091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:49:47.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346275+0000","last_change":"2024-09-05T06:39:02.871391+0000","last_active":"2024-09-05T06:41:15.346275+0000","last_peered":"2024-09-05T06:41:15.346275+0000","last_clean":"2024-09-05T06:41:15.346275+0000","last_became_active":"2024-09-05T06:39:02.870869+0000","last_became_peered":"2024-09-05T06:39:02.870869+0000","last_unstale":"2024-09-05T06:41:15.346275+0000","last_undegraded":"2024-09-05T06:41:15.346275+0000","last_fullsized":"2024-09-05T06:41:15.346275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:33:45.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.350654+0000","last_change":"2024-09-05T06:38:53.363093+0000","last_active":"2024-09-05T06:41:15.350654+0000","last_peered":"2024-09-05T06:41:15.350654+0000","last_clean":"2024-09-05T06:41:15.350654+0000","last_became_active":"2024-09-05T06:38:53.362684+0000","last_became_peered":"2024-09-05T06:38:53.362684+0000","last_unstale":"2024-09-05T06:41:15.350654+0000","last_undegraded":"2024-09-05T06:41:15.350654+0000","last_fullsized":"2024-09-05T06:41:15.350654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:30:52.543584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385703+0000","last_change":"2024-09-05T06:38:55.380479+0000","last_active":"2024-09-05T06:41:15.385703+0000","last_peered":"2024-09-05T06:41:15.385703+0000","last_clean":"2024-09-05T06:41:15.385703+0000","last_became_active":"2024-09-05T06:38:55.380142+0000","last_became_peered":"2024-09-05T06:38:55.380142+0000","last_unstale":"2024-09-05T06:41:15.385703+0000","last_undegraded":"2024-09-05T06:41:15.385703+0000","last_fullsized":"2024-09-05T06:41:15.385703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:43:43.988690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345343+0000","last_change":"2024-09-05T06:39:02.875280+0000","last_active":"2024-09-05T06:41:15.345343+0000","last_peered":"2024-09-05T06:41:15.345343+0000","last_clean":"2024-09-05T06:41:15.345343+0000","last_became_active":"2024-09-05T06:39:02.874736+0000","last_became_peered":"2024-09-05T06:39:02.874736+0000","last_unstale":"2024-09-05T06:41:15.345343+0000","last_undegraded":"2024-09-05T06:41:15.345343+0000","last_fullsized":"2024-09-05T06:41:15.345343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:44.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393247+0000","last_change":"2024-09-05T06:38:55.386668+0000","last_active":"2024-09-05T06:41:15.393247+0000","last_peered":"2024-09-05T06:41:15.393247+0000","last_clean":"2024-09-05T06:41:15.393247+0000","last_became_active":"2024-09-05T06:38:55.386274+0000","last_became_peered":"2024-09-05T06:38:55.386274+0000","last_unstale":"2024-09-05T06:41:15.393247+0000","last_undegraded":"2024-09-05T06:41:15.393247+0000","last_fullsized":"2024-09-05T06:41:15.393247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:15:25.567706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354972+0000","last_change":"2024-09-05T06:38:53.363549+0000","last_active":"2024-09-05T06:41:15.354972+0000","last_peered":"2024-09-05T06:41:15.354972+0000","last_clean":"2024-09-05T06:41:15.354972+0000","last_became_active":"2024-09-05T06:38:53.363219+0000","last_became_peered":"2024-09-05T06:38:53.363219+0000","last_unstale":"2024-09-05T06:41:15.354972+0000","last_undegraded":"2024-09-05T06:41:15.354972+0000","last_fullsized":"2024-09-05T06:41:15.354972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:54:40.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345978+0000","last_change":"2024-09-05T06:39:02.961650+0000","last_active":"2024-09-05T06:41:15.345978+0000","last_peered":"2024-09-05T06:41:15.345978+0000","last_clean":"2024-09-05T06:41:15.345978+0000","last_became_active":"2024-09-05T06:39:02.961204+0000","last_became_peered":"2024-09-05T06:39:02.961204+0000","last_unstale":"2024-09-05T06:41:15.345978+0000","last_undegraded":"2024-09-05T06:41:15.345978+0000","last_fullsized":"2024-09-05T06:41:15.345978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:05:21.404750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391359+0000","last_change":"2024-09-05T06:39:12.929955+0000","last_active":"2024-09-05T06:41:15.391359+0000","last_peered":"2024-09-05T06:41:15.391359+0000","last_clean":"2024-09-05T06:41:15.391359+0000","last_became_active":"2024-09-05T06:39:12.929586+0000","last_became_peered":"2024-09-05T06:39:12.929586+0000","last_unstale":"2024-09-05T06:41:15.391359+0000","last_undegraded":"2024-09-05T06:41:15.391359+0000","last_fullsized":"2024-09-05T06:41:15.391359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:41:27.474778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.863237+0000","last_change":"2024-09-05T06:38:53.362246+0000","last_active":"2024-09-05T06:41:15.863237+0000","last_peered":"2024-09-05T06:41:15.863237+0000","last_clean":"2024-09-05T06:41:15.863237+0000","last_became_active":"2024-09-05T06:38:53.361912+0000","last_became_peered":"2024-09-05T06:38:53.361912+0000","last_unstale":"2024-09-05T06:41:15.863237+0000","last_undegraded":"2024-09-05T06:41:15.863237+0000","last_fullsized":"2024-09-05T06:41:15.863237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:21:16.411342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.345763+0000","last_change":"2024-09-05T06:39:02.870972+0000","last_active":"2024-09-05T06:41:15.345763+0000","last_peered":"2024-09-05T06:41:15.345763+0000","last_clean":"2024-09-05T06:41:15.345763+0000","last_became_active":"2024-09-05T06:39:02.870493+0000","last_became_peered":"2024-09-05T06:39:02.870493+0000","last_unstale":"2024-09-05T06:41:15.345763+0000","last_undegraded":"2024-09-05T06:41:15.345763+0000","last_fullsized":"2024-09-05T06:41:15.345763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:51:19.510744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395872+0000","last_change":"2024-09-05T06:38:55.377513+0000","last_active":"2024-09-05T06:41:15.395872+0000","last_peered":"2024-09-05T06:41:15.395872+0000","last_clean":"2024-09-05T06:41:15.395872+0000","last_became_active":"2024-09-05T06:38:55.377132+0000","last_became_peered":"2024-09-05T06:38:55.377132+0000","last_unstale":"2024-09-05T06:41:15.395872+0000","last_undegraded":"2024-09-05T06:41:15.395872+0000","last_fullsized":"2024-09-05T06:41:15.395872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:44:24.630084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.356067+0000","last_change":"2024-09-05T06:38:53.360236+0000","last_active":"2024-09-05T06:41:15.356067+0000","last_peered":"2024-09-05T06:41:15.356067+0000","last_clean":"2024-09-05T06:41:15.356067+0000","last_became_active":"2024-09-05T06:38:53.359870+0000","last_became_peered":"2024-09-05T06:38:53.359870+0000","last_unstale":"2024-09-05T06:41:15.356067+0000","last_undegraded":"2024-09-05T06:41:15.356067+0000","last_fullsized":"2024-09-05T06:41:15.356067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:46.677566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351668+0000","last_change":"2024-09-05T06:40:13.422027+0000","last_active":"2024-09-05T06:41:15.351668+0000","last_peered":"2024-09-05T06:41:15.351668+0000","last_clean":"2024-09-05T06:41:15.351668+0000","last_became_active":"2024-09-05T06:40:13.421526+0000","last_became_peered":"2024-09-05T06:40:13.421526+0000","last_unstale":"2024-09-05T06:41:15.351668+0000","last_undegraded":"2024-09-05T06:41:15.351668+0000","last_fullsized":"2024-09-05T06:41:15.351668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:11:12.210094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.373107+0000","last_change":"2024-09-05T06:38:55.379105+0000","last_active":"2024-09-05T06:41:15.373107+0000","last_peered":"2024-09-05T06:41:15.373107+0000","last_clean":"2024-09-05T06:41:15.373107+0000","last_became_active":"2024-09-05T06:38:55.377982+0000","last_became_peered":"2024-09-05T06:38:55.377982+0000","last_unstale":"2024-09-05T06:41:15.373107+0000","last_undegraded":"2024-09-05T06:41:15.373107+0000","last_fullsized":"2024-09-05T06:41:15.373107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:50.362104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337538+0000","last_change":"2024-09-05T06:38:53.398565+0000","last_active":"2024-09-05T06:41:15.337538+0000","last_peered":"2024-09-05T06:41:15.337538+0000","last_clean":"2024-09-05T06:41:15.337538+0000","last_became_active":"2024-09-05T06:38:53.398257+0000","last_became_peered":"2024-09-05T06:38:53.398257+0000","last_unstale":"2024-09-05T06:41:15.337538+0000","last_undegraded":"2024-09-05T06:41:15.337538+0000","last_fullsized":"2024-09-05T06:41:15.337538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.466618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346048+0000","last_change":"2024-09-05T06:40:13.423201+0000","last_active":"2024-09-05T06:41:15.346048+0000","last_peered":"2024-09-05T06:41:15.346048+0000","last_clean":"2024-09-05T06:41:15.346048+0000","last_became_active":"2024-09-05T06:40:13.422737+0000","last_became_peered":"2024-09-05T06:40:13.422737+0000","last_unstale":"2024-09-05T06:41:15.346048+0000","last_undegraded":"2024-09-05T06:41:15.346048+0000","last_fullsized":"2024-09-05T06:41:15.346048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:38:50.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394197+0000","last_change":"2024-09-05T06:39:12.930785+0000","last_active":"2024-09-05T06:41:15.394197+0000","last_peered":"2024-09-05T06:41:15.394197+0000","last_clean":"2024-09-05T06:41:15.394197+0000","last_became_active":"2024-09-05T06:39:12.930436+0000","last_became_peered":"2024-09-05T06:39:12.930436+0000","last_unstale":"2024-09-05T06:41:15.394197+0000","last_undegraded":"2024-09-05T06:41:15.394197+0000","last_fullsized":"2024-09-05T06:41:15.394197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:23.774063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346508+0000","last_change":"2024-09-05T06:38:53.357125+0000","last_active":"2024-09-05T06:41:15.346508+0000","last_peered":"2024-09-05T06:41:15.346508+0000","last_clean":"2024-09-05T06:41:15.346508+0000","last_became_active":"2024-09-05T06:38:53.356358+0000","last_became_peered":"2024-09-05T06:38:53.356358+0000","last_unstale":"2024-09-05T06:41:15.346508+0000","last_undegraded":"2024-09-05T06:41:15.346508+0000","last_fullsized":"2024-09-05T06:41:15.346508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:10.003413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.339196+0000","last_change":"2024-09-05T06:39:02.870908+0000","last_active":"2024-09-05T06:41:15.339196+0000","last_peered":"2024-09-05T06:41:15.339196+0000","last_clean":"2024-09-05T06:41:15.339196+0000","last_became_active":"2024-09-05T06:39:02.870424+0000","last_became_peered":"2024-09-05T06:39:02.870424+0000","last_unstale":"2024-09-05T06:41:15.339196+0000","last_undegraded":"2024-09-05T06:41:15.339196+0000","last_fullsized":"2024-09-05T06:41:15.339196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:27:17.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.393329+0000","last_change":"2024-09-05T06:38:55.386019+0000","last_active":"2024-09-05T06:41:15.393329+0000","last_peered":"2024-09-05T06:41:15.393329+0000","last_clean":"2024-09-05T06:41:15.393329+0000","last_became_active":"2024-09-05T06:38:55.385735+0000","last_became_peered":"2024-09-05T06:38:55.385735+0000","last_unstale":"2024-09-05T06:41:15.393329+0000","last_undegraded":"2024-09-05T06:41:15.393329+0000","last_fullsized":"2024-09-05T06:41:15.393329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:01.903745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'76","reported_seq":180,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.210943+0000","last_change":"2024-09-05T06:39:12.935877+0000","last_active":"2024-09-05T06:41:26.210943+0000","last_peered":"2024-09-05T06:41:26.210943+0000","last_clean":"2024-09-05T06:41:26.210943+0000","last_became_active":"2024-09-05T06:39:12.930845+0000","last_became_peered":"2024-09-05T06:39:12.930845+0000","last_unstale":"2024-09-05T06:41:26.210943+0000","last_undegraded":"2024-09-05T06:41:26.210943+0000","last_fullsized":"2024-09-05T06:41:26.210943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:50:07.536163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":136,"num_write_kb":1158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353026+0000","last_change":"2024-09-05T06:39:02.961502+0000","last_active":"2024-09-05T06:41:15.353026+0000","last_peered":"2024-09-05T06:41:15.353026+0000","last_clean":"2024-09-05T06:41:15.353026+0000","last_became_active":"2024-09-05T06:39:02.961126+0000","last_became_peered":"2024-09-05T06:39:02.961126+0000","last_unstale":"2024-09-05T06:41:15.353026+0000","last_undegraded":"2024-09-05T06:41:15.353026+0000","last_fullsized":"2024-09-05T06:41:15.353026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:32:25.816772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388449+0000","last_change":"2024-09-05T06:38:55.371669+0000","last_active":"2024-09-05T06:41:15.388449+0000","last_peered":"2024-09-05T06:41:15.388449+0000","last_clean":"2024-09-05T06:41:15.388449+0000","last_became_active":"2024-09-05T06:38:55.371313+0000","last_became_peered":"2024-09-05T06:38:55.371313+0000","last_unstale":"2024-09-05T06:41:15.388449+0000","last_undegraded":"2024-09-05T06:41:15.388449+0000","last_fullsized":"2024-09-05T06:41:15.388449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:33.266676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354903+0000","last_change":"2024-09-05T06:38:53.399640+0000","last_active":"2024-09-05T06:41:15.354903+0000","last_peered":"2024-09-05T06:41:15.354903+0000","last_clean":"2024-09-05T06:41:15.354903+0000","last_became_active":"2024-09-05T06:38:53.399274+0000","last_became_peered":"2024-09-05T06:38:53.399274+0000","last_unstale":"2024-09-05T06:41:15.354903+0000","last_undegraded":"2024-09-05T06:41:15.354903+0000","last_fullsized":"2024-09-05T06:41:15.354903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:34:41.916621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353029+0000","last_change":"2024-09-05T06:39:02.875223+0000","last_active":"2024-09-05T06:41:15.353029+0000","last_peered":"2024-09-05T06:41:15.353029+0000","last_clean":"2024-09-05T06:41:15.353029+0000","last_became_active":"2024-09-05T06:39:02.874805+0000","last_became_peered":"2024-09-05T06:39:02.874805+0000","last_unstale":"2024-09-05T06:41:15.353029+0000","last_undegraded":"2024-09-05T06:41:15.353029+0000","last_fullsized":"2024-09-05T06:41:15.353029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:04:57.903357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382141+0000","last_change":"2024-09-05T06:38:55.381521+0000","last_active":"2024-09-05T06:41:15.382141+0000","last_peered":"2024-09-05T06:41:15.382141+0000","last_clean":"2024-09-05T06:41:15.382141+0000","last_became_active":"2024-09-05T06:38:55.381204+0000","last_became_peered":"2024-09-05T06:38:55.381204+0000","last_unstale":"2024-09-05T06:41:15.382141+0000","last_undegraded":"2024-09-05T06:41:15.382141+0000","last_fullsized":"2024-09-05T06:41:15.382141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:27:42.325864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.364717+0000","last_change":"2024-09-05T06:38:53.399423+0000","last_active":"2024-09-05T06:41:15.364717+0000","last_peered":"2024-09-05T06:41:15.364717+0000","last_clean":"2024-09-05T06:41:15.364717+0000","last_became_active":"2024-09-05T06:38:53.399135+0000","last_became_peered":"2024-09-05T06:38:53.399135+0000","last_unstale":"2024-09-05T06:41:15.364717+0000","last_undegraded":"2024-09-05T06:41:15.364717+0000","last_fullsized":"2024-09-05T06:41:15.364717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:32:22.797170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.349954+0000","last_change":"2024-09-05T06:39:02.962180+0000","last_active":"2024-09-05T06:41:15.349954+0000","last_peered":"2024-09-05T06:41:15.349954+0000","last_clean":"2024-09-05T06:41:15.349954+0000","last_became_active":"2024-09-05T06:39:02.961783+0000","last_became_peered":"2024-09-05T06:39:02.961783+0000","last_unstale":"2024-09-05T06:41:15.349954+0000","last_undegraded":"2024-09-05T06:41:15.349954+0000","last_fullsized":"2024-09-05T06:41:15.349954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:01:16.234981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383366+0000","last_change":"2024-09-05T06:38:55.377895+0000","last_active":"2024-09-05T06:41:15.383366+0000","last_peered":"2024-09-05T06:41:15.383366+0000","last_clean":"2024-09-05T06:41:15.383366+0000","last_became_active":"2024-09-05T06:38:55.377337+0000","last_became_peered":"2024-09-05T06:38:55.377337+0000","last_unstale":"2024-09-05T06:41:15.383366+0000","last_undegraded":"2024-09-05T06:41:15.383366+0000","last_fullsized":"2024-09-05T06:41:15.383366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:08:04.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.015271+0000","last_change":"2024-09-05T06:38:53.359507+0000","last_active":"2024-09-05T06:41:25.015271+0000","last_peered":"2024-09-05T06:41:25.015271+0000","last_clean":"2024-09-05T06:41:25.015271+0000","last_became_active":"2024-09-05T06:38:53.358992+0000","last_became_peered":"2024-09-05T06:38:53.358992+0000","last_unstale":"2024-09-05T06:41:25.015271+0000","last_undegraded":"2024-09-05T06:41:25.015271+0000","last_fullsized":"2024-09-05T06:41:25.015271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:17.126019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336376+0000","last_change":"2024-09-05T06:39:02.877116+0000","last_active":"2024-09-05T06:41:15.336376+0000","last_peered":"2024-09-05T06:41:15.336376+0000","last_clean":"2024-09-05T06:41:15.336376+0000","last_became_active":"2024-09-05T06:39:02.876737+0000","last_became_peered":"2024-09-05T06:39:02.876737+0000","last_unstale":"2024-09-05T06:41:15.336376+0000","last_undegraded":"2024-09-05T06:41:15.336376+0000","last_fullsized":"2024-09-05T06:41:15.336376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:29.893333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383507+0000","last_change":"2024-09-05T06:38:55.378997+0000","last_active":"2024-09-05T06:41:15.383507+0000","last_peered":"2024-09-05T06:41:15.383507+0000","last_clean":"2024-09-05T06:41:15.383507+0000","last_became_active":"2024-09-05T06:38:55.378685+0000","last_became_peered":"2024-09-05T06:38:55.378685+0000","last_unstale":"2024-09-05T06:41:15.383507+0000","last_undegraded":"2024-09-05T06:41:15.383507+0000","last_fullsized":"2024-09-05T06:41:15.383507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:06.221407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006131+0000","last_change":"2024-09-05T06:38:53.398183+0000","last_active":"2024-09-05T06:41:25.006131+0000","last_peered":"2024-09-05T06:41:25.006131+0000","last_clean":"2024-09-05T06:41:25.006131+0000","last_became_active":"2024-09-05T06:38:53.397851+0000","last_became_peered":"2024-09-05T06:38:53.397851+0000","last_unstale":"2024-09-05T06:41:25.006131+0000","last_undegraded":"2024-09-05T06:41:25.006131+0000","last_fullsized":"2024-09-05T06:41:25.006131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:06:16.167708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336734+0000","last_change":"2024-09-05T06:39:02.961577+0000","last_active":"2024-09-05T06:41:15.336734+0000","last_peered":"2024-09-05T06:41:15.336734+0000","last_clean":"2024-09-05T06:41:15.336734+0000","last_became_active":"2024-09-05T06:39:02.961185+0000","last_became_peered":"2024-09-05T06:39:02.961185+0000","last_unstale":"2024-09-05T06:41:15.336734+0000","last_undegraded":"2024-09-05T06:41:15.336734+0000","last_fullsized":"2024-09-05T06:41:15.336734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:38.775795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388493+0000","last_change":"2024-09-05T06:38:55.376543+0000","last_active":"2024-09-05T06:41:15.388493+0000","last_peered":"2024-09-05T06:41:15.388493+0000","last_clean":"2024-09-05T06:41:15.388493+0000","last_became_active":"2024-09-05T06:38:55.376255+0000","last_became_peered":"2024-09-05T06:38:55.376255+0000","last_unstale":"2024-09-05T06:41:15.388493+0000","last_undegraded":"2024-09-05T06:41:15.388493+0000","last_fullsized":"2024-09-05T06:41:15.388493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:12:36.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338433+0000","last_change":"2024-09-05T06:38:53.363996+0000","last_active":"2024-09-05T06:41:15.338433+0000","last_peered":"2024-09-05T06:41:15.338433+0000","last_clean":"2024-09-05T06:41:15.338433+0000","last_became_active":"2024-09-05T06:38:53.363660+0000","last_became_peered":"2024-09-05T06:38:53.363660+0000","last_unstale":"2024-09-05T06:41:15.338433+0000","last_undegraded":"2024-09-05T06:41:15.338433+0000","last_fullsized":"2024-09-05T06:41:15.338433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:17:10.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336319+0000","last_change":"2024-09-05T06:40:13.423422+0000","last_active":"2024-09-05T06:41:15.336319+0000","last_peered":"2024-09-05T06:41:15.336319+0000","last_clean":"2024-09-05T06:41:15.336319+0000","last_became_active":"2024-09-05T06:40:13.423001+0000","last_became_peered":"2024-09-05T06:40:13.423001+0000","last_unstale":"2024-09-05T06:41:15.336319+0000","last_undegraded":"2024-09-05T06:41:15.336319+0000","last_fullsized":"2024-09-05T06:41:15.336319+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:52:33.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381432+0000","last_change":"2024-09-05T06:38:55.372232+0000","last_active":"2024-09-05T06:41:15.381432+0000","last_peered":"2024-09-05T06:41:15.381432+0000","last_clean":"2024-09-05T06:41:15.381432+0000","last_became_active":"2024-09-05T06:38:55.371614+0000","last_became_peered":"2024-09-05T06:38:55.371614+0000","last_unstale":"2024-09-05T06:41:15.381432+0000","last_undegraded":"2024-09-05T06:41:15.381432+0000","last_fullsized":"2024-09-05T06:41:15.381432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:13:40.301925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346591+0000","last_change":"2024-09-05T06:38:53.359236+0000","last_active":"2024-09-05T06:41:15.346591+0000","last_peered":"2024-09-05T06:41:15.346591+0000","last_clean":"2024-09-05T06:41:15.346591+0000","last_became_active":"2024-09-05T06:38:53.357680+0000","last_became_peered":"2024-09-05T06:38:53.357680+0000","last_unstale":"2024-09-05T06:41:15.346591+0000","last_undegraded":"2024-09-05T06:41:15.346591+0000","last_fullsized":"2024-09-05T06:41:15.346591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:33:07.313343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":406,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:17.464338+0000","last_change":"2024-09-05T06:37:41.874804+0000","last_active":"2024-09-05T06:41:17.464338+0000","last_peered":"2024-09-05T06:41:17.464338+0000","last_clean":"2024-09-05T06:41:17.464338+0000","last_became_active":"2024-09-05T06:37:41.874312+0000","last_became_peered":"2024-09-05T06:37:41.874312+0000","last_unstale":"2024-09-05T06:41:17.464338+0000","last_undegraded":"2024-09-05T06:41:17.464338+0000","last_fullsized":"2024-09-05T06:41:17.464338+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:34:21.378004+0000","last_clean_scrub_stamp":"2024-09-05T06:34:21.378004+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:59:16.692221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338813+0000","last_change":"2024-09-05T06:39:02.874029+0000","last_active":"2024-09-05T06:41:15.338813+0000","last_peered":"2024-09-05T06:41:15.338813+0000","last_clean":"2024-09-05T06:41:15.338813+0000","last_became_active":"2024-09-05T06:39:02.873644+0000","last_became_peered":"2024-09-05T06:39:02.873644+0000","last_unstale":"2024-09-05T06:41:15.338813+0000","last_undegraded":"2024-09-05T06:41:15.338813+0000","last_fullsized":"2024-09-05T06:41:15.338813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:45:33.292257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394409+0000","last_change":"2024-09-05T06:38:55.390641+0000","last_active":"2024-09-05T06:41:15.394409+0000","last_peered":"2024-09-05T06:41:15.394409+0000","last_clean":"2024-09-05T06:41:15.394409+0000","last_became_active":"2024-09-05T06:38:55.389946+0000","last_became_peered":"2024-09-05T06:38:55.389946+0000","last_unstale":"2024-09-05T06:41:15.394409+0000","last_undegraded":"2024-09-05T06:41:15.394409+0000","last_fullsized":"2024-09-05T06:41:15.394409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:05.797989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.075812+0000","last_change":"2024-09-05T06:38:53.398821+0000","last_active":"2024-09-05T06:41:25.075812+0000","last_peered":"2024-09-05T06:41:25.075812+0000","last_clean":"2024-09-05T06:41:25.075812+0000","last_became_active":"2024-09-05T06:38:53.398299+0000","last_became_peered":"2024-09-05T06:38:53.398299+0000","last_unstale":"2024-09-05T06:41:25.075812+0000","last_undegraded":"2024-09-05T06:41:25.075812+0000","last_fullsized":"2024-09-05T06:41:25.075812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:38.301451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1665,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.337537+0000","last_change":"2024-09-05T06:39:02.872515+0000","last_active":"2024-09-05T06:41:15.337537+0000","last_peered":"2024-09-05T06:41:15.337537+0000","last_clean":"2024-09-05T06:41:15.337537+0000","last_became_active":"2024-09-05T06:39:02.872119+0000","last_became_peered":"2024-09-05T06:39:02.872119+0000","last_unstale":"2024-09-05T06:41:15.337537+0000","last_undegraded":"2024-09-05T06:41:15.337537+0000","last_fullsized":"2024-09-05T06:41:15.337537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:00.444332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391875+0000","last_change":"2024-09-05T06:38:55.388308+0000","last_active":"2024-09-05T06:41:15.391875+0000","last_peered":"2024-09-05T06:41:15.391875+0000","last_clean":"2024-09-05T06:41:15.391875+0000","last_became_active":"2024-09-05T06:38:55.387919+0000","last_became_peered":"2024-09-05T06:38:55.387919+0000","last_unstale":"2024-09-05T06:41:15.391875+0000","last_undegraded":"2024-09-05T06:41:15.391875+0000","last_fullsized":"2024-09-05T06:41:15.391875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:18.429179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'67","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:26.918976+0000","last_change":"2024-09-05T06:38:53.360629+0000","last_active":"2024-09-05T06:41:26.918976+0000","last_peered":"2024-09-05T06:41:26.918976+0000","last_clean":"2024-09-05T06:41:26.918976+0000","last_became_active":"2024-09-05T06:38:53.360289+0000","last_became_peered":"2024-09-05T06:38:53.360289+0000","last_unstale":"2024-09-05T06:41:26.918976+0000","last_undegraded":"2024-09-05T06:41:26.918976+0000","last_fullsized":"2024-09-05T06:41:26.918976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:07.290731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353806+0000","last_change":"2024-09-05T06:39:02.874382+0000","last_active":"2024-09-05T06:41:15.353806+0000","last_peered":"2024-09-05T06:41:15.353806+0000","last_clean":"2024-09-05T06:41:15.353806+0000","last_became_active":"2024-09-05T06:39:02.874001+0000","last_became_peered":"2024-09-05T06:39:02.874001+0000","last_unstale":"2024-09-05T06:41:15.353806+0000","last_undegraded":"2024-09-05T06:41:15.353806+0000","last_fullsized":"2024-09-05T06:41:15.353806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:26:39.520225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394254+0000","last_change":"2024-09-05T06:38:55.390364+0000","last_active":"2024-09-05T06:41:15.394254+0000","last_peered":"2024-09-05T06:41:15.394254+0000","last_clean":"2024-09-05T06:41:15.394254+0000","last_became_active":"2024-09-05T06:38:55.389624+0000","last_became_peered":"2024-09-05T06:38:55.389624+0000","last_unstale":"2024-09-05T06:41:15.394254+0000","last_undegraded":"2024-09-05T06:41:15.394254+0000","last_fullsized":"2024-09-05T06:41:15.394254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:43:58.809011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336686+0000","last_change":"2024-09-05T06:39:12.937598+0000","last_active":"2024-09-05T06:41:15.336686+0000","last_peered":"2024-09-05T06:41:15.336686+0000","last_clean":"2024-09-05T06:41:15.336686+0000","last_became_active":"2024-09-05T06:39:12.933684+0000","last_became_peered":"2024-09-05T06:39:12.933684+0000","last_unstale":"2024-09-05T06:41:15.336686+0000","last_undegraded":"2024-09-05T06:41:15.336686+0000","last_fullsized":"2024-09-05T06:41:15.336686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:29:05.027626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335683+0000","last_change":"2024-09-05T06:39:02.874771+0000","last_active":"2024-09-05T06:41:15.335683+0000","last_peered":"2024-09-05T06:41:15.335683+0000","last_clean":"2024-09-05T06:41:15.335683+0000","last_became_active":"2024-09-05T06:39:02.873107+0000","last_became_peered":"2024-09-05T06:39:02.873107+0000","last_unstale":"2024-09-05T06:41:15.335683+0000","last_undegraded":"2024-09-05T06:41:15.335683+0000","last_fullsized":"2024-09-05T06:41:15.335683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:24.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374756+0000","last_change":"2024-09-05T06:38:55.380803+0000","last_active":"2024-09-05T06:41:15.374756+0000","last_peered":"2024-09-05T06:41:15.374756+0000","last_clean":"2024-09-05T06:41:15.374756+0000","last_became_active":"2024-09-05T06:38:55.380449+0000","last_became_peered":"2024-09-05T06:38:55.380449+0000","last_unstale":"2024-09-05T06:41:15.374756+0000","last_undegraded":"2024-09-05T06:41:15.374756+0000","last_fullsized":"2024-09-05T06:41:15.374756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:08:24.165284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354546+0000","last_change":"2024-09-05T06:39:12.937689+0000","last_active":"2024-09-05T06:41:15.354546+0000","last_peered":"2024-09-05T06:41:15.354546+0000","last_clean":"2024-09-05T06:41:15.354546+0000","last_became_active":"2024-09-05T06:39:12.933609+0000","last_became_peered":"2024-09-05T06:39:12.933609+0000","last_unstale":"2024-09-05T06:41:15.354546+0000","last_undegraded":"2024-09-05T06:41:15.354546+0000","last_fullsized":"2024-09-05T06:41:15.354546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:14.313903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.335845+0000","last_change":"2024-09-05T06:39:02.876380+0000","last_active":"2024-09-05T06:41:15.335845+0000","last_peered":"2024-09-05T06:41:15.335845+0000","last_clean":"2024-09-05T06:41:15.335845+0000","last_became_active":"2024-09-05T06:39:02.875956+0000","last_became_peered":"2024-09-05T06:39:02.875956+0000","last_unstale":"2024-09-05T06:41:15.335845+0000","last_undegraded":"2024-09-05T06:41:15.335845+0000","last_fullsized":"2024-09-05T06:41:15.335845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:10:16.861972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385062+0000","last_change":"2024-09-05T06:38:55.377952+0000","last_active":"2024-09-05T06:41:15.385062+0000","last_peered":"2024-09-05T06:41:15.385062+0000","last_clean":"2024-09-05T06:41:15.385062+0000","last_became_active":"2024-09-05T06:38:55.375488+0000","last_became_peered":"2024-09-05T06:38:55.375488+0000","last_unstale":"2024-09-05T06:41:15.385062+0000","last_undegraded":"2024-09-05T06:41:15.385062+0000","last_fullsized":"2024-09-05T06:41:15.385062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:27:26.121605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353497+0000","last_change":"2024-09-05T06:38:53.399810+0000","last_active":"2024-09-05T06:41:15.353497+0000","last_peered":"2024-09-05T06:41:15.353497+0000","last_clean":"2024-09-05T06:41:15.353497+0000","last_became_active":"2024-09-05T06:38:53.399398+0000","last_became_peered":"2024-09-05T06:38:53.399398+0000","last_unstale":"2024-09-05T06:41:15.353497+0000","last_undegraded":"2024-09-05T06:41:15.353497+0000","last_fullsized":"2024-09-05T06:41:15.353497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:31:26.106062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336654+0000","last_change":"2024-09-05T06:39:02.873774+0000","last_active":"2024-09-05T06:41:15.336654+0000","last_peered":"2024-09-05T06:41:15.336654+0000","last_clean":"2024-09-05T06:41:15.336654+0000","last_became_active":"2024-09-05T06:39:02.873423+0000","last_became_peered":"2024-09-05T06:39:02.873423+0000","last_unstale":"2024-09-05T06:41:15.336654+0000","last_undegraded":"2024-09-05T06:41:15.336654+0000","last_fullsized":"2024-09-05T06:41:15.336654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:06:12.799863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394022+0000","last_change":"2024-09-05T06:38:55.384833+0000","last_active":"2024-09-05T06:41:15.394022+0000","last_peered":"2024-09-05T06:41:15.394022+0000","last_clean":"2024-09-05T06:41:15.394022+0000","last_became_active":"2024-09-05T06:38:55.384426+0000","last_became_peered":"2024-09-05T06:38:55.384426+0000","last_unstale":"2024-09-05T06:41:15.394022+0000","last_undegraded":"2024-09-05T06:41:15.394022+0000","last_fullsized":"2024-09-05T06:41:15.394022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:03.002713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:25.006632+0000","last_change":"2024-09-05T06:38:53.361870+0000","last_active":"2024-09-05T06:41:25.006632+0000","last_peered":"2024-09-05T06:41:25.006632+0000","last_clean":"2024-09-05T06:41:25.006632+0000","last_became_active":"2024-09-05T06:38:53.361556+0000","last_became_peered":"2024-09-05T06:38:53.361556+0000","last_unstale":"2024-09-05T06:41:25.006632+0000","last_undegraded":"2024-09-05T06:41:25.006632+0000","last_fullsized":"2024-09-05T06:41:25.006632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:55:14.963894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.363849+0000","last_change":"2024-09-05T06:39:02.876517+0000","last_active":"2024-09-05T06:41:15.363849+0000","last_peered":"2024-09-05T06:41:15.363849+0000","last_clean":"2024-09-05T06:41:15.363849+0000","last_became_active":"2024-09-05T06:39:02.876088+0000","last_became_peered":"2024-09-05T06:39:02.876088+0000","last_unstale":"2024-09-05T06:41:15.363849+0000","last_undegraded":"2024-09-05T06:41:15.363849+0000","last_fullsized":"2024-09-05T06:41:15.363849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:47:36.940148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390371+0000","last_change":"2024-09-05T06:38:55.389838+0000","last_active":"2024-09-05T06:41:15.390371+0000","last_peered":"2024-09-05T06:41:15.390371+0000","last_clean":"2024-09-05T06:41:15.390371+0000","last_became_active":"2024-09-05T06:38:55.389392+0000","last_became_peered":"2024-09-05T06:38:55.389392+0000","last_unstale":"2024-09-05T06:41:15.390371+0000","last_undegraded":"2024-09-05T06:41:15.390371+0000","last_fullsized":"2024-09-05T06:41:15.390371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:36.114696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336066+0000","last_change":"2024-09-05T06:38:53.359584+0000","last_active":"2024-09-05T06:41:15.336066+0000","last_peered":"2024-09-05T06:41:15.336066+0000","last_clean":"2024-09-05T06:41:15.336066+0000","last_became_active":"2024-09-05T06:38:53.359281+0000","last_became_peered":"2024-09-05T06:38:53.359281+0000","last_unstale":"2024-09-05T06:41:15.336066+0000","last_undegraded":"2024-09-05T06:41:15.336066+0000","last_fullsized":"2024-09-05T06:41:15.336066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:25.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.353200+0000","last_change":"2024-09-05T06:39:02.879414+0000","last_active":"2024-09-05T06:41:15.353200+0000","last_peered":"2024-09-05T06:41:15.353200+0000","last_clean":"2024-09-05T06:41:15.353200+0000","last_became_active":"2024-09-05T06:39:02.878948+0000","last_became_peered":"2024-09-05T06:39:02.878948+0000","last_unstale":"2024-09-05T06:41:15.353200+0000","last_undegraded":"2024-09-05T06:41:15.353200+0000","last_fullsized":"2024-09-05T06:41:15.353200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:10.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.381695+0000","last_change":"2024-09-05T06:38:55.388148+0000","last_active":"2024-09-05T06:41:15.381695+0000","last_peered":"2024-09-05T06:41:15.381695+0000","last_clean":"2024-09-05T06:41:15.381695+0000","last_became_active":"2024-09-05T06:38:55.387767+0000","last_became_peered":"2024-09-05T06:38:55.387767+0000","last_unstale":"2024-09-05T06:41:15.381695+0000","last_undegraded":"2024-09-05T06:41:15.381695+0000","last_fullsized":"2024-09-05T06:41:15.381695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:42:43.890638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340918+0000","last_change":"2024-09-05T06:38:53.359347+0000","last_active":"2024-09-05T06:41:15.340918+0000","last_peered":"2024-09-05T06:41:15.340918+0000","last_clean":"2024-09-05T06:41:15.340918+0000","last_became_active":"2024-09-05T06:38:53.359022+0000","last_became_peered":"2024-09-05T06:38:53.359022+0000","last_unstale":"2024-09-05T06:41:15.340918+0000","last_undegraded":"2024-09-05T06:41:15.340918+0000","last_fullsized":"2024-09-05T06:41:15.340918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:08:06.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336222+0000","last_change":"2024-09-05T06:39:02.876080+0000","last_active":"2024-09-05T06:41:15.336222+0000","last_peered":"2024-09-05T06:41:15.336222+0000","last_clean":"2024-09-05T06:41:15.336222+0000","last_became_active":"2024-09-05T06:39:02.875700+0000","last_became_peered":"2024-09-05T06:39:02.875700+0000","last_unstale":"2024-09-05T06:41:15.336222+0000","last_undegraded":"2024-09-05T06:41:15.336222+0000","last_fullsized":"2024-09-05T06:41:15.336222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:18:17.420597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.368981+0000","last_change":"2024-09-05T06:38:55.388315+0000","last_active":"2024-09-05T06:41:15.368981+0000","last_peered":"2024-09-05T06:41:15.368981+0000","last_clean":"2024-09-05T06:41:15.368981+0000","last_became_active":"2024-09-05T06:38:55.387939+0000","last_became_peered":"2024-09-05T06:38:55.387939+0000","last_unstale":"2024-09-05T06:41:15.368981+0000","last_undegraded":"2024-09-05T06:41:15.368981+0000","last_fullsized":"2024-09-05T06:41:15.368981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:30.950912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:31.014833+0000","last_change":"2024-09-05T06:39:12.944454+0000","last_active":"2024-09-05T06:41:31.014833+0000","last_peered":"2024-09-05T06:41:31.014833+0000","last_clean":"2024-09-05T06:41:31.014833+0000","last_became_active":"2024-09-05T06:39:12.933625+0000","last_became_peered":"2024-09-05T06:39:12.933625+0000","last_unstale":"2024-09-05T06:41:31.014833+0000","last_undegraded":"2024-09-05T06:41:31.014833+0000","last_fullsized":"2024-09-05T06:41:31.014833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:33:37.073623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69515,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":195,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357275+0000","last_change":"2024-09-05T06:39:02.872213+0000","last_active":"2024-09-05T06:41:15.357275+0000","last_peered":"2024-09-05T06:41:15.357275+0000","last_clean":"2024-09-05T06:41:15.357275+0000","last_became_active":"2024-09-05T06:39:02.871743+0000","last_became_peered":"2024-09-05T06:39:02.871743+0000","last_unstale":"2024-09-05T06:41:15.357275+0000","last_undegraded":"2024-09-05T06:41:15.357275+0000","last_fullsized":"2024-09-05T06:41:15.357275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:31:45.566669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385754+0000","last_change":"2024-09-05T06:38:55.377395+0000","last_active":"2024-09-05T06:41:15.385754+0000","last_peered":"2024-09-05T06:41:15.385754+0000","last_clean":"2024-09-05T06:41:15.385754+0000","last_became_active":"2024-09-05T06:38:55.377013+0000","last_became_peered":"2024-09-05T06:38:55.377013+0000","last_unstale":"2024-09-05T06:41:15.385754+0000","last_undegraded":"2024-09-05T06:41:15.385754+0000","last_fullsized":"2024-09-05T06:41:15.385754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:55:21.164321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336476+0000","last_change":"2024-09-05T06:38:53.398961+0000","last_active":"2024-09-05T06:41:15.336476+0000","last_peered":"2024-09-05T06:41:15.336476+0000","last_clean":"2024-09-05T06:41:15.336476+0000","last_became_active":"2024-09-05T06:38:53.398586+0000","last_became_peered":"2024-09-05T06:38:53.398586+0000","last_unstale":"2024-09-05T06:41:15.336476+0000","last_undegraded":"2024-09-05T06:41:15.336476+0000","last_fullsized":"2024-09-05T06:41:15.336476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:30:50.590069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346427+0000","last_change":"2024-09-05T06:39:02.873113+0000","last_active":"2024-09-05T06:41:15.346427+0000","last_peered":"2024-09-05T06:41:15.346427+0000","last_clean":"2024-09-05T06:41:15.346427+0000","last_became_active":"2024-09-05T06:39:02.872335+0000","last_became_peered":"2024-09-05T06:39:02.872335+0000","last_unstale":"2024-09-05T06:41:15.346427+0000","last_undegraded":"2024-09-05T06:41:15.346427+0000","last_fullsized":"2024-09-05T06:41:15.346427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:38:52.043134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392670+0000","last_change":"2024-09-05T06:39:12.931561+0000","last_active":"2024-09-05T06:41:15.392670+0000","last_peered":"2024-09-05T06:41:15.392670+0000","last_clean":"2024-09-05T06:41:15.392670+0000","last_became_active":"2024-09-05T06:39:12.931221+0000","last_became_peered":"2024-09-05T06:39:12.931221+0000","last_unstale":"2024-09-05T06:41:15.392670+0000","last_undegraded":"2024-09-05T06:41:15.392670+0000","last_fullsized":"2024-09-05T06:41:15.392670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:05:47.399570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.338779+0000","last_change":"2024-09-05T06:38:53.360637+0000","last_active":"2024-09-05T06:41:15.338779+0000","last_peered":"2024-09-05T06:41:15.338779+0000","last_clean":"2024-09-05T06:41:15.338779+0000","last_became_active":"2024-09-05T06:38:53.360258+0000","last_became_peered":"2024-09-05T06:38:53.360258+0000","last_unstale":"2024-09-05T06:41:15.338779+0000","last_undegraded":"2024-09-05T06:41:15.338779+0000","last_fullsized":"2024-09-05T06:41:15.338779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:48:57.668568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.357061+0000","last_change":"2024-09-05T06:39:02.876549+0000","last_active":"2024-09-05T06:41:15.357061+0000","last_peered":"2024-09-05T06:41:15.357061+0000","last_clean":"2024-09-05T06:41:15.357061+0000","last_became_active":"2024-09-05T06:39:02.876189+0000","last_became_peered":"2024-09-05T06:39:02.876189+0000","last_unstale":"2024-09-05T06:41:15.357061+0000","last_undegraded":"2024-09-05T06:41:15.357061+0000","last_fullsized":"2024-09-05T06:41:15.357061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:49.773398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.377550+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.377024+0000","last_became_peered":"2024-09-05T06:38:55.377024+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:40:35.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":285,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:31.014825+0000","last_change":"2024-09-05T06:38:53.361654+0000","last_active":"2024-09-05T06:41:31.014825+0000","last_peered":"2024-09-05T06:41:31.014825+0000","last_clean":"2024-09-05T06:41:31.014825+0000","last_became_active":"2024-09-05T06:38:53.360797+0000","last_became_peered":"2024-09-05T06:38:53.360797+0000","last_unstale":"2024-09-05T06:41:31.014825+0000","last_undegraded":"2024-09-05T06:41:31.014825+0000","last_fullsized":"2024-09-05T06:41:31.014825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:59:48.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.351201+0000","last_change":"2024-09-05T06:39:02.873380+0000","last_active":"2024-09-05T06:41:15.351201+0000","last_peered":"2024-09-05T06:41:15.351201+0000","last_clean":"2024-09-05T06:41:15.351201+0000","last_became_active":"2024-09-05T06:39:02.872994+0000","last_became_peered":"2024-09-05T06:39:02.872994+0000","last_unstale":"2024-09-05T06:41:15.351201+0000","last_undegraded":"2024-09-05T06:41:15.351201+0000","last_fullsized":"2024-09-05T06:41:15.351201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:53:05.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.352686+0000","last_change":"2024-09-05T06:39:02.875791+0000","last_active":"2024-09-05T06:41:15.352686+0000","last_peered":"2024-09-05T06:41:15.352686+0000","last_clean":"2024-09-05T06:41:15.352686+0000","last_became_active":"2024-09-05T06:39:02.875345+0000","last_became_peered":"2024-09-05T06:39:02.875345+0000","last_unstale":"2024-09-05T06:41:15.352686+0000","last_undegraded":"2024-09-05T06:41:15.352686+0000","last_fullsized":"2024-09-05T06:41:15.352686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:28:12.539755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.388863+0000","last_change":"2024-09-05T06:38:55.375136+0000","last_active":"2024-09-05T06:41:15.388863+0000","last_peered":"2024-09-05T06:41:15.388863+0000","last_clean":"2024-09-05T06:41:15.388863+0000","last_became_active":"2024-09-05T06:38:55.374840+0000","last_became_peered":"2024-09-05T06:38:55.374840+0000","last_unstale":"2024-09-05T06:41:15.388863+0000","last_undegraded":"2024-09-05T06:41:15.388863+0000","last_fullsized":"2024-09-05T06:41:15.388863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:21:10.281175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336013+0000","last_change":"2024-09-05T06:38:53.362198+0000","last_active":"2024-09-05T06:41:15.336013+0000","last_peered":"2024-09-05T06:41:15.336013+0000","last_clean":"2024-09-05T06:41:15.336013+0000","last_became_active":"2024-09-05T06:38:53.361875+0000","last_became_peered":"2024-09-05T06:38:53.361875+0000","last_unstale":"2024-09-05T06:41:15.336013+0000","last_undegraded":"2024-09-05T06:41:15.336013+0000","last_fullsized":"2024-09-05T06:41:15.336013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:32.470641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.354268+0000","last_change":"2024-09-05T06:39:02.874514+0000","last_active":"2024-09-05T06:41:15.354268+0000","last_peered":"2024-09-05T06:41:15.354268+0000","last_clean":"2024-09-05T06:41:15.354268+0000","last_became_active":"2024-09-05T06:39:02.874068+0000","last_became_peered":"2024-09-05T06:39:02.874068+0000","last_unstale":"2024-09-05T06:41:15.354268+0000","last_undegraded":"2024-09-05T06:41:15.354268+0000","last_fullsized":"2024-09-05T06:41:15.354268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:03:51.459513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379081+0000","last_change":"2024-09-05T06:38:55.371133+0000","last_active":"2024-09-05T06:41:15.379081+0000","last_peered":"2024-09-05T06:41:15.379081+0000","last_clean":"2024-09-05T06:41:15.379081+0000","last_became_active":"2024-09-05T06:38:55.370710+0000","last_became_peered":"2024-09-05T06:38:55.370710+0000","last_unstale":"2024-09-05T06:41:15.379081+0000","last_undegraded":"2024-09-05T06:41:15.379081+0000","last_fullsized":"2024-09-05T06:41:15.379081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:54:32.061130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.336163+0000","last_change":"2024-09-05T06:38:53.360759+0000","last_active":"2024-09-05T06:41:15.336163+0000","last_peered":"2024-09-05T06:41:15.336163+0000","last_clean":"2024-09-05T06:41:15.336163+0000","last_became_active":"2024-09-05T06:38:53.360459+0000","last_became_peered":"2024-09-05T06:38:53.360459+0000","last_unstale":"2024-09-05T06:41:15.336163+0000","last_undegraded":"2024-09-05T06:41:15.336163+0000","last_fullsized":"2024-09-05T06:41:15.336163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:47:37.090218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.346070+0000","last_change":"2024-09-05T06:38:53.361393+0000","last_active":"2024-09-05T06:41:15.346070+0000","last_peered":"2024-09-05T06:41:15.346070+0000","last_clean":"2024-09-05T06:41:15.346070+0000","last_became_active":"2024-09-05T06:38:53.360911+0000","last_became_peered":"2024-09-05T06:38:53.360911+0000","last_unstale":"2024-09-05T06:41:15.346070+0000","last_undegraded":"2024-09-05T06:41:15.346070+0000","last_fullsized":"2024-09-05T06:41:15.346070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:56:33.908256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385711+0000","last_change":"2024-09-05T06:39:12.932076+0000","last_active":"2024-09-05T06:41:15.385711+0000","last_peered":"2024-09-05T06:41:15.385711+0000","last_clean":"2024-09-05T06:41:15.385711+0000","last_became_active":"2024-09-05T06:39:12.931702+0000","last_became_peered":"2024-09-05T06:39:12.931702+0000","last_unstale":"2024-09-05T06:41:15.385711+0000","last_undegraded":"2024-09-05T06:41:15.385711+0000","last_fullsized":"2024-09-05T06:41:15.385711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:14:41.979541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.340040+0000","last_change":"2024-09-05T06:39:02.871349+0000","last_active":"2024-09-05T06:41:15.340040+0000","last_peered":"2024-09-05T06:41:15.340040+0000","last_clean":"2024-09-05T06:41:15.340040+0000","last_became_active":"2024-09-05T06:39:02.870811+0000","last_became_peered":"2024-09-05T06:39:02.870811+0000","last_unstale":"2024-09-05T06:41:15.340040+0000","last_undegraded":"2024-09-05T06:41:15.340040+0000","last_fullsized":"2024-09-05T06:41:15.340040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:09:41.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'9","reported_seq":175,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:30.013812+0000","last_change":"2024-09-05T06:38:53.363328+0000","last_active":"2024-09-05T06:41:30.013812+0000","last_peered":"2024-09-05T06:41:30.013812+0000","last_clean":"2024-09-05T06:41:30.013812+0000","last_became_active":"2024-09-05T06:38:53.362986+0000","last_became_peered":"2024-09-05T06:38:53.362986+0000","last_unstale":"2024-09-05T06:41:30.013812+0000","last_undegraded":"2024-09-05T06:41:30.013812+0000","last_fullsized":"2024-09-05T06:41:30.013812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:52.337320+0000","last_clean_scrub_stamp":"2024-09-05T06:38:52.337320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:48.114843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379840+0000","last_change":"2024-09-05T06:38:55.372388+0000","last_active":"2024-09-05T06:41:15.379840+0000","last_peered":"2024-09-05T06:41:15.379840+0000","last_clean":"2024-09-05T06:41:15.379840+0000","last_became_active":"2024-09-05T06:38:55.371974+0000","last_became_peered":"2024-09-05T06:38:55.371974+0000","last_unstale":"2024-09-05T06:41:15.379840+0000","last_undegraded":"2024-09-05T06:41:15.379840+0000","last_fullsized":"2024-09-05T06:41:15.379840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:44:09.674906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.341065+0000","last_change":"2024-09-05T06:39:02.871572+0000","last_active":"2024-09-05T06:41:15.341065+0000","last_peered":"2024-09-05T06:41:15.341065+0000","last_clean":"2024-09-05T06:41:15.341065+0000","last_became_active":"2024-09-05T06:39:02.871141+0000","last_became_peered":"2024-09-05T06:39:02.871141+0000","last_unstale":"2024-09-05T06:41:15.341065+0000","last_undegraded":"2024-09-05T06:41:15.341065+0000","last_fullsized":"2024-09-05T06:41:15.341065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:39:01.851042+0000","last_clean_scrub_stamp":"2024-09-05T06:39:01.851042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:25.516857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385030+0000","last_change":"2024-09-05T06:38:55.382455+0000","last_active":"2024-09-05T06:41:15.385030+0000","last_peered":"2024-09-05T06:41:15.385030+0000","last_clean":"2024-09-05T06:41:15.385030+0000","last_became_active":"2024-09-05T06:38:55.381614+0000","last_became_peered":"2024-09-05T06:38:55.381614+0000","last_unstale":"2024-09-05T06:41:15.385030+0000","last_undegraded":"2024-09-05T06:41:15.385030+0000","last_fullsized":"2024-09-05T06:41:15.385030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:44:27.033746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386323+0000","last_change":"2024-09-05T06:39:12.933654+0000","last_active":"2024-09-05T06:41:15.386323+0000","last_peered":"2024-09-05T06:41:15.386323+0000","last_clean":"2024-09-05T06:41:15.386323+0000","last_became_active":"2024-09-05T06:39:12.933276+0000","last_became_peered":"2024-09-05T06:39:12.933276+0000","last_unstale":"2024-09-05T06:41:15.386323+0000","last_undegraded":"2024-09-05T06:41:15.386323+0000","last_fullsized":"2024-09-05T06:41:15.386323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:43.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383695+0000","last_change":"2024-09-05T06:38:55.374571+0000","last_active":"2024-09-05T06:41:15.383695+0000","last_peered":"2024-09-05T06:41:15.383695+0000","last_clean":"2024-09-05T06:41:15.383695+0000","last_became_active":"2024-09-05T06:38:55.374158+0000","last_became_peered":"2024-09-05T06:38:55.374158+0000","last_unstale":"2024-09-05T06:41:15.383695+0000","last_undegraded":"2024-09-05T06:41:15.383695+0000","last_fullsized":"2024-09-05T06:41:15.383695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:51:37.043006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392707+0000","last_change":"2024-09-05T06:38:55.380359+0000","last_active":"2024-09-05T06:41:15.392707+0000","last_peered":"2024-09-05T06:41:15.392707+0000","last_clean":"2024-09-05T06:41:15.392707+0000","last_became_active":"2024-09-05T06:38:55.380066+0000","last_became_peered":"2024-09-05T06:38:55.380066+0000","last_unstale":"2024-09-05T06:41:15.392707+0000","last_undegraded":"2024-09-05T06:41:15.392707+0000","last_fullsized":"2024-09-05T06:41:15.392707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:41:23.919901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391243+0000","last_change":"2024-09-05T06:40:13.500615+0000","last_active":"2024-09-05T06:41:15.391243+0000","last_peered":"2024-09-05T06:41:15.391243+0000","last_clean":"2024-09-05T06:41:15.391243+0000","last_became_active":"2024-09-05T06:40:13.500246+0000","last_became_peered":"2024-09-05T06:40:13.500246+0000","last_unstale":"2024-09-05T06:41:15.391243+0000","last_undegraded":"2024-09-05T06:41:15.391243+0000","last_fullsized":"2024-09-05T06:41:15.391243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:59:53.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.375909+0000","last_change":"2024-09-05T06:38:55.381227+0000","last_active":"2024-09-05T06:41:15.375909+0000","last_peered":"2024-09-05T06:41:15.375909+0000","last_clean":"2024-09-05T06:41:15.375909+0000","last_became_active":"2024-09-05T06:38:55.380788+0000","last_became_peered":"2024-09-05T06:38:55.380788+0000","last_unstale":"2024-09-05T06:41:15.375909+0000","last_undegraded":"2024-09-05T06:41:15.375909+0000","last_fullsized":"2024-09-05T06:41:15.375909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:33:14.584565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386445+0000","last_change":"2024-09-05T06:38:55.376865+0000","last_active":"2024-09-05T06:41:15.386445+0000","last_peered":"2024-09-05T06:41:15.386445+0000","last_clean":"2024-09-05T06:41:15.386445+0000","last_became_active":"2024-09-05T06:38:55.376365+0000","last_became_peered":"2024-09-05T06:38:55.376365+0000","last_unstale":"2024-09-05T06:41:15.386445+0000","last_undegraded":"2024-09-05T06:41:15.386445+0000","last_fullsized":"2024-09-05T06:41:15.386445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:19.930421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382401+0000","last_change":"2024-09-05T06:38:55.374771+0000","last_active":"2024-09-05T06:41:15.382401+0000","last_peered":"2024-09-05T06:41:15.382401+0000","last_clean":"2024-09-05T06:41:15.382401+0000","last_became_active":"2024-09-05T06:38:55.374421+0000","last_became_peered":"2024-09-05T06:38:55.374421+0000","last_unstale":"2024-09-05T06:41:15.382401+0000","last_undegraded":"2024-09-05T06:41:15.382401+0000","last_fullsized":"2024-09-05T06:41:15.382401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:25:39.527325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.385555+0000","last_change":"2024-09-05T06:38:55.371502+0000","last_active":"2024-09-05T06:41:15.385555+0000","last_peered":"2024-09-05T06:41:15.385555+0000","last_clean":"2024-09-05T06:41:15.385555+0000","last_became_active":"2024-09-05T06:38:55.371141+0000","last_became_peered":"2024-09-05T06:38:55.371141+0000","last_unstale":"2024-09-05T06:41:15.385555+0000","last_undegraded":"2024-09-05T06:41:15.385555+0000","last_fullsized":"2024-09-05T06:41:15.385555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:15.485373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386806+0000","last_change":"2024-09-05T06:38:55.372364+0000","last_active":"2024-09-05T06:41:15.386806+0000","last_peered":"2024-09-05T06:41:15.386806+0000","last_clean":"2024-09-05T06:41:15.386806+0000","last_became_active":"2024-09-05T06:38:55.372097+0000","last_became_peered":"2024-09-05T06:38:55.372097+0000","last_unstale":"2024-09-05T06:41:15.386806+0000","last_undegraded":"2024-09-05T06:41:15.386806+0000","last_fullsized":"2024-09-05T06:41:15.386806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:24:48.474929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394725+0000","last_change":"2024-09-05T06:38:55.389857+0000","last_active":"2024-09-05T06:41:15.394725+0000","last_peered":"2024-09-05T06:41:15.394725+0000","last_clean":"2024-09-05T06:41:15.394725+0000","last_became_active":"2024-09-05T06:38:55.389481+0000","last_became_peered":"2024-09-05T06:38:55.389481+0000","last_unstale":"2024-09-05T06:41:15.394725+0000","last_undegraded":"2024-09-05T06:41:15.394725+0000","last_fullsized":"2024-09-05T06:41:15.394725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:07:03.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.391085+0000","last_change":"2024-09-05T06:38:55.388067+0000","last_active":"2024-09-05T06:41:15.391085+0000","last_peered":"2024-09-05T06:41:15.391085+0000","last_clean":"2024-09-05T06:41:15.391085+0000","last_became_active":"2024-09-05T06:38:55.387729+0000","last_became_peered":"2024-09-05T06:38:55.387729+0000","last_unstale":"2024-09-05T06:41:15.391085+0000","last_undegraded":"2024-09-05T06:41:15.391085+0000","last_fullsized":"2024-09-05T06:41:15.391085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:10:12.117544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.374549+0000","last_change":"2024-09-05T06:38:55.377432+0000","last_active":"2024-09-05T06:41:15.374549+0000","last_peered":"2024-09-05T06:41:15.374549+0000","last_clean":"2024-09-05T06:41:15.374549+0000","last_became_active":"2024-09-05T06:38:55.376852+0000","last_became_peered":"2024-09-05T06:38:55.376852+0000","last_unstale":"2024-09-05T06:41:15.374549+0000","last_undegraded":"2024-09-05T06:41:15.374549+0000","last_fullsized":"2024-09-05T06:41:15.374549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:12.141345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.387694+0000","last_change":"2024-09-05T06:38:55.371504+0000","last_active":"2024-09-05T06:41:15.387694+0000","last_peered":"2024-09-05T06:41:15.387694+0000","last_clean":"2024-09-05T06:41:15.387694+0000","last_became_active":"2024-09-05T06:38:55.371135+0000","last_became_peered":"2024-09-05T06:38:55.371135+0000","last_unstale":"2024-09-05T06:41:15.387694+0000","last_undegraded":"2024-09-05T06:41:15.387694+0000","last_fullsized":"2024-09-05T06:41:15.387694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:18:11.388759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.389011+0000","last_change":"2024-09-05T06:38:55.380898+0000","last_active":"2024-09-05T06:41:15.389011+0000","last_peered":"2024-09-05T06:41:15.389011+0000","last_clean":"2024-09-05T06:41:15.389011+0000","last_became_active":"2024-09-05T06:38:55.380591+0000","last_became_peered":"2024-09-05T06:38:55.380591+0000","last_unstale":"2024-09-05T06:41:15.389011+0000","last_undegraded":"2024-09-05T06:41:15.389011+0000","last_fullsized":"2024-09-05T06:41:15.389011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:44.497587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396214+0000","last_change":"2024-09-05T06:38:55.387161+0000","last_active":"2024-09-05T06:41:15.396214+0000","last_peered":"2024-09-05T06:41:15.396214+0000","last_clean":"2024-09-05T06:41:15.396214+0000","last_became_active":"2024-09-05T06:38:55.386356+0000","last_became_peered":"2024-09-05T06:38:55.386356+0000","last_unstale":"2024-09-05T06:41:15.396214+0000","last_undegraded":"2024-09-05T06:41:15.396214+0000","last_fullsized":"2024-09-05T06:41:15.396214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:46:33.051471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395691+0000","last_change":"2024-09-05T06:38:55.388203+0000","last_active":"2024-09-05T06:41:15.395691+0000","last_peered":"2024-09-05T06:41:15.395691+0000","last_clean":"2024-09-05T06:41:15.395691+0000","last_became_active":"2024-09-05T06:38:55.387888+0000","last_became_peered":"2024-09-05T06:38:55.387888+0000","last_unstale":"2024-09-05T06:41:15.395691+0000","last_undegraded":"2024-09-05T06:41:15.395691+0000","last_fullsized":"2024-09-05T06:41:15.395691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:15:43.116748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.390219+0000","last_change":"2024-09-05T06:38:55.379482+0000","last_active":"2024-09-05T06:41:15.390219+0000","last_peered":"2024-09-05T06:41:15.390219+0000","last_clean":"2024-09-05T06:41:15.390219+0000","last_became_active":"2024-09-05T06:38:55.379203+0000","last_became_peered":"2024-09-05T06:38:55.379203+0000","last_unstale":"2024-09-05T06:41:15.390219+0000","last_undegraded":"2024-09-05T06:41:15.390219+0000","last_fullsized":"2024-09-05T06:41:15.390219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:01:00.530543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.392054+0000","last_change":"2024-09-05T06:38:55.375962+0000","last_active":"2024-09-05T06:41:15.392054+0000","last_peered":"2024-09-05T06:41:15.392054+0000","last_clean":"2024-09-05T06:41:15.392054+0000","last_became_active":"2024-09-05T06:38:55.375678+0000","last_became_peered":"2024-09-05T06:38:55.375678+0000","last_unstale":"2024-09-05T06:41:15.392054+0000","last_undegraded":"2024-09-05T06:41:15.392054+0000","last_fullsized":"2024-09-05T06:41:15.392054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:40:04.125207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386510+0000","last_change":"2024-09-05T06:38:55.375070+0000","last_active":"2024-09-05T06:41:15.386510+0000","last_peered":"2024-09-05T06:41:15.386510+0000","last_clean":"2024-09-05T06:41:15.386510+0000","last_became_active":"2024-09-05T06:38:55.374785+0000","last_became_peered":"2024-09-05T06:38:55.374785+0000","last_unstale":"2024-09-05T06:41:15.386510+0000","last_undegraded":"2024-09-05T06:41:15.386510+0000","last_fullsized":"2024-09-05T06:41:15.386510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:13:26.337023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386966+0000","last_change":"2024-09-05T06:38:55.378011+0000","last_active":"2024-09-05T06:41:15.386966+0000","last_peered":"2024-09-05T06:41:15.386966+0000","last_clean":"2024-09-05T06:41:15.386966+0000","last_became_active":"2024-09-05T06:38:55.377619+0000","last_became_peered":"2024-09-05T06:38:55.377619+0000","last_unstale":"2024-09-05T06:41:15.386966+0000","last_undegraded":"2024-09-05T06:41:15.386966+0000","last_fullsized":"2024-09-05T06:41:15.386966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:20.936771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394223+0000","last_change":"2024-09-05T06:38:55.391425+0000","last_active":"2024-09-05T06:41:15.394223+0000","last_peered":"2024-09-05T06:41:15.394223+0000","last_clean":"2024-09-05T06:41:15.394223+0000","last_became_active":"2024-09-05T06:38:55.391059+0000","last_became_peered":"2024-09-05T06:38:55.391059+0000","last_unstale":"2024-09-05T06:41:15.394223+0000","last_undegraded":"2024-09-05T06:41:15.394223+0000","last_fullsized":"2024-09-05T06:41:15.394223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:26:26.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386226+0000","last_change":"2024-09-05T06:38:55.379703+0000","last_active":"2024-09-05T06:41:15.386226+0000","last_peered":"2024-09-05T06:41:15.386226+0000","last_clean":"2024-09-05T06:41:15.386226+0000","last_became_active":"2024-09-05T06:38:55.379277+0000","last_became_peered":"2024-09-05T06:38:55.379277+0000","last_unstale":"2024-09-05T06:41:15.386226+0000","last_undegraded":"2024-09-05T06:41:15.386226+0000","last_fullsized":"2024-09-05T06:41:15.386226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:06:50.675937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.383778+0000","last_change":"2024-09-05T06:38:55.383817+0000","last_active":"2024-09-05T06:41:15.383778+0000","last_peered":"2024-09-05T06:41:15.383778+0000","last_clean":"2024-09-05T06:41:15.383778+0000","last_became_active":"2024-09-05T06:38:55.382697+0000","last_became_peered":"2024-09-05T06:38:55.382697+0000","last_unstale":"2024-09-05T06:41:15.383778+0000","last_undegraded":"2024-09-05T06:41:15.383778+0000","last_fullsized":"2024-09-05T06:41:15.383778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:40:53.086017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.384549+0000","last_change":"2024-09-05T06:38:55.376919+0000","last_active":"2024-09-05T06:41:15.384549+0000","last_peered":"2024-09-05T06:41:15.384549+0000","last_clean":"2024-09-05T06:41:15.384549+0000","last_became_active":"2024-09-05T06:38:55.376609+0000","last_became_peered":"2024-09-05T06:38:55.376609+0000","last_unstale":"2024-09-05T06:41:15.384549+0000","last_undegraded":"2024-09-05T06:41:15.384549+0000","last_fullsized":"2024-09-05T06:41:15.384549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:09:23.301412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.379776+0000","last_change":"2024-09-05T06:38:55.387282+0000","last_active":"2024-09-05T06:41:15.379776+0000","last_peered":"2024-09-05T06:41:15.379776+0000","last_clean":"2024-09-05T06:41:15.379776+0000","last_became_active":"2024-09-05T06:38:55.386977+0000","last_became_peered":"2024-09-05T06:38:55.386977+0000","last_unstale":"2024-09-05T06:41:15.379776+0000","last_undegraded":"2024-09-05T06:41:15.379776+0000","last_fullsized":"2024-09-05T06:41:15.379776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.924024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.386221+0000","last_change":"2024-09-05T06:38:55.376102+0000","last_active":"2024-09-05T06:41:15.386221+0000","last_peered":"2024-09-05T06:41:15.386221+0000","last_clean":"2024-09-05T06:41:15.386221+0000","last_became_active":"2024-09-05T06:38:55.375519+0000","last_became_peered":"2024-09-05T06:38:55.375519+0000","last_unstale":"2024-09-05T06:41:15.386221+0000","last_undegraded":"2024-09-05T06:41:15.386221+0000","last_fullsized":"2024-09-05T06:41:15.386221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:59.830114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.369287+0000","last_change":"2024-09-05T06:38:55.374663+0000","last_active":"2024-09-05T06:41:15.369287+0000","last_peered":"2024-09-05T06:41:15.369287+0000","last_clean":"2024-09-05T06:41:15.369287+0000","last_became_active":"2024-09-05T06:38:55.374309+0000","last_became_peered":"2024-09-05T06:38:55.374309+0000","last_unstale":"2024-09-05T06:41:15.369287+0000","last_undegraded":"2024-09-05T06:41:15.369287+0000","last_fullsized":"2024-09-05T06:41:15.369287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:20.173031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.382415+0000","last_change":"2024-09-05T06:38:55.381583+0000","last_active":"2024-09-05T06:41:15.382415+0000","last_peered":"2024-09-05T06:41:15.382415+0000","last_clean":"2024-09-05T06:41:15.382415+0000","last_became_active":"2024-09-05T06:38:55.381239+0000","last_became_peered":"2024-09-05T06:38:55.381239+0000","last_unstale":"2024-09-05T06:41:15.382415+0000","last_undegraded":"2024-09-05T06:41:15.382415+0000","last_fullsized":"2024-09-05T06:41:15.382415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:58:28.812322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.395537+0000","last_change":"2024-09-05T06:38:55.381756+0000","last_active":"2024-09-05T06:41:15.395537+0000","last_peered":"2024-09-05T06:41:15.395537+0000","last_clean":"2024-09-05T06:41:15.395537+0000","last_became_active":"2024-09-05T06:38:55.381225+0000","last_became_peered":"2024-09-05T06:38:55.381225+0000","last_unstale":"2024-09-05T06:41:15.395537+0000","last_undegraded":"2024-09-05T06:41:15.395537+0000","last_fullsized":"2024-09-05T06:41:15.395537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:41.072841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.394857+0000","last_change":"2024-09-05T06:38:55.382348+0000","last_active":"2024-09-05T06:41:15.394857+0000","last_peered":"2024-09-05T06:41:15.394857+0000","last_clean":"2024-09-05T06:41:15.394857+0000","last_became_active":"2024-09-05T06:38:55.382098+0000","last_became_peered":"2024-09-05T06:38:55.382098+0000","last_unstale":"2024-09-05T06:41:15.394857+0000","last_undegraded":"2024-09-05T06:41:15.394857+0000","last_fullsized":"2024-09-05T06:41:15.394857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:48:13.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T06:41:15.396142+0000","last_change":"2024-09-05T06:38:55.383701+0000","last_active":"2024-09-05T06:41:15.396142+0000","last_peered":"2024-09-05T06:41:15.396142+0000","last_clean":"2024-09-05T06:41:15.396142+0000","last_became_active":"2024-09-05T06:38:55.383198+0000","last_became_peered":"2024-09-05T06:38:55.383198+0000","last_unstale":"2024-09-05T06:41:15.396142+0000","last_undegraded":"2024-09-05T06:41:15.396142+0000","last_fullsized":"2024-09-05T06:41:15.396142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T06:38:54.344711+0000","last_clean_scrub_stamp":"2024-09-05T06:38:54.344711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:32.117654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801612,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1433,"num_read_kb":892,"num_write":458,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1634,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":361,"ondisk_log_size":361,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1445888,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841585,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29176,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715976,"statfs":{"total":95995035648,"available":95965159424,"internally_reserved":0,"allocated":2080768,"data_stored":1506565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005110,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1470464,"data_stored":917822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1667072,"data_stored":1120726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":516902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855434,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29104,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716048,"statfs":{"total":95995035648,"available":95965233152,"internally_reserved":0,"allocated":2007040,"data_stored":1442230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1134592,"data_stored":591025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37208,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707944,"statfs":{"total":95995035648,"available":95956934656,"internally_reserved":0,"allocated":1785856,"data_stored":1237170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":519181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":587447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":517965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":394925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T06:41:33.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:41:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:33.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:33 smithi115 ceph-mon[34791]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:33.695 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:33 smithi086 ceph-mon[30943]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 -- 172.21.15.86:0/1175366023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 msgr2=0x7fa190106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 --2- 172.21.15.86:0/1175366023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa190106840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa178009930 tx=0x7fa17802a420 comp rx=0 tx=0).stop 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 -- 172.21.15.86:0/1175366023 shutdown_connections 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 --2- 172.21.15.86:0/1175366023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa190106f50 0x7fa19010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 --2- 172.21.15.86:0/1175366023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa190106840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 --2- 172.21.15.86:0/1175366023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1901002f0 0x7fa1901006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 -- 172.21.15.86:0/1175366023 >> 172.21.15.86:0/1175366023 conn(0x7fa190077210 msgr2=0x7fa1900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 -- 172.21.15.86:0/1175366023 shutdown_connections 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.774+0000 7fa194dde640 1 -- 172.21.15.86:0/1175366023 wait complete. 2024-09-05T06:41:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 Processor -- start 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 -- start start 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 0x7fa190198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa190106f50 0x7fa19019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111e50 con 0x7fa190100ca0 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111cd0 con 0x7fa1901002f0 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa194dde640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa190111fd0 con 0x7fa190106f50 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.775+0000 7fa18dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37612/0 (socket says 172.21.15.86:37612) 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 -- 172.21.15.86:0/1209241234 learned_addr learned my addr 172.21.15.86:0/1209241234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18ed9d640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa190106f50 0x7fa19019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18e59c640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 0x7fa190198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 -- 172.21.15.86:0/1209241234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa190106f50 msgr2=0x7fa19019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa190106f50 0x7fa19019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 -- 172.21.15.86:0/1209241234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 msgr2=0x7fa190198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 0x7fa190198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 -- 172.21.15.86:0/1209241234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19019d440 con 0x7fa190100ca0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18e59c640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 0x7fa190198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa18dd9b640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa178031440 tx=0x7fa178004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17803f030 con 0x7fa190100ca0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa178003aa0 con 0x7fa190100ca0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19019d6d0 con 0x7fa190100ca0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178035070 con 0x7fa190100ca0 2024-09-05T06:41:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.776+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a5010 con 0x7fa190100ca0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.777+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1901033b0 con 0x7fa190100ca0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.778+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fa178042030 con 0x7fa190100ca0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.779+0000 7fa1877fe640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 0x7fa17c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.779+0000 7fa18e59c640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 0x7fa17c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.779+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa1780effe0 con 0x7fa190100ca0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.780+0000 7fa18e59c640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 0x7fa17c0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa180005fd0 tx=0x7fa180009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.783+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1780bc6b0 con 0x7fa190100ca0 2024-09-05T06:41:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:33 smithi096 ceph-mon[34468]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T06:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.982+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa19010a140 con 0x7fa190100ca0 2024-09-05T06:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.983+0000 7fa1877fe640 1 -- 172.21.15.86:0/1209241234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7fa1780bbfb0 con 0x7fa190100ca0 2024-09-05T06:41:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:33.983 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 msgr2=0x7fa17c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 0x7fa17c0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa180005fd0 tx=0x7fa180009500 comp rx=0 tx=0).stop 2024-09-05T06:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 msgr2=0x7fa1901989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa178031440 tx=0x7fa178004270 comp rx=0 tx=0).stop 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 shutdown_connections 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa17c0a2d80 0x7fa17c0a5240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa190106f50 0x7fa19019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa190100ca0 0x7fa1901989d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 --2- 172.21.15.86:0/1209241234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1901002f0 0x7fa190198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 >> 172.21.15.86:0/1209241234 conn(0x7fa190077210 msgr2=0x7fa1901052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 shutdown_connections 2024-09-05T06:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:33.985+0000 7fa194dde640 1 -- 172.21.15.86:0/1209241234 wait complete. 2024-09-05T06:41:34.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:34 smithi086 ceph-mon[30943]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-09-05T06:41:34.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:34 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1209241234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:34.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:41:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:34 smithi115 ceph-mon[34791]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-09-05T06:41:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:34 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1209241234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:34.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:34 smithi096 ceph-mon[34468]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-09-05T06:41:34.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:34 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1209241234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 -- 172.21.15.86:0/1679213094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102570 msgr2=0x7f6b141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 --2- 172.21.15.86:0/1679213094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102570 0x7f6b141029b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc02a480 comp rx=0 tx=0).stop 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 -- 172.21.15.86:0/1679213094 shutdown_connections 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 --2- 172.21.15.86:0/1679213094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102ef0 0x7f6b1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 --2- 172.21.15.86:0/1679213094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102570 0x7f6b141029b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 --2- 172.21.15.86:0/1679213094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 0x7f6b14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 -- 172.21.15.86:0/1679213094 >> 172.21.15.86:0/1679213094 conn(0x7f6b140fc7d0 msgr2=0x7f6b140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 -- 172.21.15.86:0/1679213094 shutdown_connections 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.332+0000 7f6b19e38640 1 -- 172.21.15.86:0/1679213094 wait complete. 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 Processor -- start 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 -- start start 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110e30 con 0x7f6b14102ef0 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110cb0 con 0x7f6b14100e20 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b19e38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110fb0 con 0x7f6b14102570 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b13fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b13fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37620/0 (socket says 172.21.15.86:37620) 2024-09-05T06:41:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b13fff640 1 -- 172.21.15.86:0/794430222 learned_addr learned my addr 172.21.15.86:0/794430222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b12ffd640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.333+0000 7f6b137fe640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 -- 172.21.15.86:0/794430222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102570 msgr2=0x7f6b140fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 -- 172.21.15.86:0/794430222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 msgr2=0x7f6b140fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 -- 172.21.15.86:0/794430222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141a6960 con 0x7f6b14102ef0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b13fff640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eaa0 tx=0x7f6b0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cdb0 con 0x7f6b14102ef0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b0c005020 con 0x7f6b14102ef0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0116d0 con 0x7f6b14102ef0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141a6c50 con 0x7f6b14102ef0 2024-09-05T06:41:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.334+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a7110 con 0x7f6b14102ef0 2024-09-05T06:41:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.335+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14101200 con 0x7f6b14102ef0 2024-09-05T06:41:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.336+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6b0c0051d0 con 0x7f6b14102ef0 2024-09-05T06:41:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.337+0000 7f6b10ff9640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 0x7f6b000a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.337+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6b0c0cfe50 con 0x7f6b14102ef0 2024-09-05T06:41:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.337+0000 7f6b137fe640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 0x7f6b000a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.338+0000 7f6b137fe640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 0x7f6b000a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400bbe0 tx=0x7f6b0400b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.340+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b0c016030 con 0x7f6b14102ef0 2024-09-05T06:41:35.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.543+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b140fe1b0 con 0x7f6b14102ef0 2024-09-05T06:41:35.544 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:35 smithi086 ceph-mon[30943]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-05T06:41:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.544+0000 7f6b10ff9640 1 -- 172.21.15.86:0/794430222 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f6b140fe1b0 con 0x7f6b14102ef0 2024-09-05T06:41:35.545 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:35.545 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:41:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 msgr2=0x7f6b000a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 0x7f6b000a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400bbe0 tx=0x7f6b0400b390 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 msgr2=0x7f6b140fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eaa0 tx=0x7f6b0c00ef60 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 shutdown_connections 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6b000a2d80 0x7f6b000a5240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 --2- 172.21.15.86:0/794430222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 >> 172.21.15.86:0/794430222 conn(0x7f6b140fc7d0 msgr2=0x7f6b1406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 shutdown_connections 2024-09-05T06:41:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:35.547+0000 7f6b19e38640 1 -- 172.21.15.86:0/794430222 wait complete. 2024-09-05T06:41:35.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:35 smithi115 ceph-mon[34791]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-05T06:41:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:35 smithi096 ceph-mon[34468]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-05T06:41:36.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T06:41:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:36.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:36 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/794430222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:36.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:36 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/794430222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:36.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:36 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/794430222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:37.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 -- 172.21.15.86:0/4125584120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780102ef0 msgr2=0x7fb78010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:37.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 --2- 172.21.15.86:0/4125584120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780102ef0 0x7fb78010da40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb778009ca0 tx=0x7fb77802a950 comp rx=0 tx=0).stop 2024-09-05T06:41:37.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 -- 172.21.15.86:0/4125584120 shutdown_connections 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 --2- 172.21.15.86:0/4125584120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780102ef0 0x7fb78010da40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 --2- 172.21.15.86:0/4125584120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102570 0x7fb7801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 --2- 172.21.15.86:0/4125584120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780100e20 0x7fb780101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 -- 172.21.15.86:0/4125584120 >> 172.21.15.86:0/4125584120 conn(0x7fb7800fc7d0 msgr2=0x7fb7800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 -- 172.21.15.86:0/4125584120 shutdown_connections 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.062+0000 7fb786d1d640 1 -- 172.21.15.86:0/4125584120 wait complete. 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 Processor -- start 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 -- start start 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 0x7fb7801988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 0x7fb78019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb780110e30 con 0x7fb780100e20 2024-09-05T06:41:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb780110cb0 con 0x7fb780102ef0 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb786d1d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb780110fb0 con 0x7fb780102570 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb784ab9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb784ab9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37634/0 (socket says 172.21.15.86:37634) 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.063+0000 7fb784ab9640 1 -- 172.21.15.86:0/2031681387 learned_addr learned my addr 172.21.15.86:0/2031681387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb777fff640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 0x7fb7801988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb7852ba640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 0x7fb78019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 -- 172.21.15.86:0/2031681387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 msgr2=0x7fb7801988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 0x7fb7801988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 -- 172.21.15.86:0/2031681387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 msgr2=0x7fb78019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 0x7fb78019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 -- 172.21.15.86:0/2031681387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78019d330 con 0x7fb780100e20 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb777fff640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 0x7fb7801988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb7852ba640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 0x7fb78019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb784ab9640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e9f0 tx=0x7fb77c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00cd80 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb77c0054d0 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78019d620 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c010490 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.064+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801a4f00 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.065+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780101200 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.066+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb77c005020 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.067+0000 7fb775ffb640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.067+0000 7fb777fff640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.067+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb77c0cfeb0 con 0x7fb780100e20 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.068+0000 7fb777fff640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 0x7fb7680a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7700058c0 tx=0x7fb770005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.071+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb77c09c580 con 0x7fb780100e20 2024-09-05T06:41:37.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.245+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb78010a0a0 con 0x7fb780100e20 2024-09-05T06:41:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.246+0000 7fb775ffb640 1 -- 172.21.15.86:0/2031681387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25345 (secure 0 0 0) 0x7fb77c09be80 con 0x7fb780100e20 2024-09-05T06:41:37.247 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:37.247 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T06:41:15.328284+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4062552821":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/2959599766":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/3708924769":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4040819651":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3625555544":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4279196951":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6800/3708924769":"2024-09-06T06:41:15.328251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 msgr2=0x7fb7680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 0x7fb7680a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7700058c0 tx=0x7fb770005850 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 msgr2=0x7fb780198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e9f0 tx=0x7fb77c00edf0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 shutdown_connections 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb780102ef0 0x7fb78019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb780102570 0x7fb7801988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 --2- 172.21.15.86:0/2031681387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb780100e20 0x7fb780198380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 >> 172.21.15.86:0/2031681387 conn(0x7fb7800fc7d0 msgr2=0x7fb78006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 shutdown_connections 2024-09-05T06:41:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:37.249+0000 7fb786d1d640 1 -- 172.21.15.86:0/2031681387 wait complete. 2024-09-05T06:41:37.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:37 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2031681387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:37.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:37 smithi115 ceph-mon[34791]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-05T06:41:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:37 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2031681387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:37 smithi086 ceph-mon[30943]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-05T06:41:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:37 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2031681387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T06:41:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:37 smithi096 ceph-mon[34468]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-05T06:41:37.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-05T06:41:38.201 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:39.372 INFO:teuthology.orchestra.run.smithi086.stdout:{"run_dir":"/var/run/ceph"} 2024-09-05T06:41:39.373 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-05T06:41:39.373 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1743 2024-09-05T06:41:39.373 DEBUG:tasks.quiescer:Yielding 2024-09-05T06:41:39.373 INFO:teuthology.run_tasks:Running task workunit... 2024-09-05T06:41:39.387 INFO:tasks.workunit:Pulling workunits from ref 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:41:39.387 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-05T06:41:39.387 INFO:tasks.workunit:timeout=3h 2024-09-05T06:41:39.387 INFO:tasks.workunit:cleanup=True 2024-09-05T06:41:39.388 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:39.388 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 79h/121d Inode: 1 Links: 2 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-05T06:41:39.407 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 06:39:37.298214344 +0000 2024-09-05T06:41:39.408 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 06:39:37.298214344 +0000 2024-09-05T06:41:39.408 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 06:41:16.230123434 +0000 2024-09-05T06:41:39.408 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-09-05T06:41:39.408 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-05T06:41:39.409 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-05T06:41:39.479 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T06:41:39.535 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-05T06:41:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:40.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.167+0000 7febf512a640 1 -- 172.21.15.86:0/1439935400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102ef0 msgr2=0x7febf010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:40.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.167+0000 7febf512a640 1 --2- 172.21.15.86:0/1439935400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102ef0 0x7febf010da40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7febe8009ca0 tx=0x7febe802a950 comp rx=0 tx=0).stop 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 -- 172.21.15.86:0/1439935400 shutdown_connections 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 --2- 172.21.15.86:0/1439935400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102ef0 0x7febf010da40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 --2- 172.21.15.86:0/1439935400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0102570 0x7febf01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 --2- 172.21.15.86:0/1439935400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0100e20 0x7febf0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 -- 172.21.15.86:0/1439935400 >> 172.21.15.86:0/1439935400 conn(0x7febf00fc7d0 msgr2=0x7febf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 -- 172.21.15.86:0/1439935400 shutdown_connections 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 -- 172.21.15.86:0/1439935400 wait complete. 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 Processor -- start 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.168+0000 7febf512a640 1 -- start start 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0100e20 0x7febf01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0102ef0 0x7febf01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febf0110e30 con 0x7febf0102570 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febf0110cb0 con 0x7febf0102ef0 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febf512a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7febf0110fb0 con 0x7febf0100e20 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37666/0 (socket says 172.21.15.86:37666) 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 -- 172.21.15.86:0/3318668626 learned_addr learned my addr 172.21.15.86:0/3318668626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febeed9d640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0100e20 0x7febf01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febef59e640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0102ef0 0x7febf01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 -- 172.21.15.86:0/3318668626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0100e20 msgr2=0x7febf01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0100e20 0x7febf01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 -- 172.21.15.86:0/3318668626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0102ef0 msgr2=0x7febf01a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0102ef0 0x7febf01a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 -- 172.21.15.86:0/3318668626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf01a4870 con 0x7febf0102570 2024-09-05T06:41:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.169+0000 7febe7fff640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7febd800d770 tx=0x7febd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.170+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8017070 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.170+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd80054d0 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.170+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004cf0 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.170+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf01a4a40 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.170+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01a4f80 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.171+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf0101200 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.172+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7febd8004e90 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.172+0000 7febecd99640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 0x7febdc0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.173+0000 7febeed9d640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 0x7febdc0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.173+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7febd80cf290 con 0x7febf0102570 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.173+0000 7febeed9d640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 0x7febdc0a52c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7febe000bbe0 tx=0x7febe000b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.177+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febd809b310 con 0x7febf0102570 2024-09-05T06:41:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.382+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febf0109090 con 0x7febf0102570 2024-09-05T06:41:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.383+0000 7febecd99640 1 -- 172.21.15.86:0/3318668626 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7febd809f610 con 0x7febf0102570 2024-09-05T06:41:40.383 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:41:40.383 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:41:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:41:40.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 msgr2=0x7febdc0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:40.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 0x7febdc0a52c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7febe000bbe0 tx=0x7febe000b390 comp rx=0 tx=0).stop 2024-09-05T06:41:40.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 msgr2=0x7febf01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7febd800d770 tx=0x7febd800dc30 comp rx=0 tx=0).stop 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 shutdown_connections 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7febdc0a2e00 0x7febdc0a52c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febf0102ef0 0x7febf01a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102570 0x7febf01997f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 --2- 172.21.15.86:0/3318668626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febf0100e20 0x7febf01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 >> 172.21.15.86:0/3318668626 conn(0x7febf00fc7d0 msgr2=0x7febf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 shutdown_connections 2024-09-05T06:41:40.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:40.385+0000 7febf512a640 1 -- 172.21.15.86:0/3318668626 wait complete. 2024-09-05T06:41:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:40 smithi115 ceph-mon[34791]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-05T06:41:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:40 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:41:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:40 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3318668626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:40.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:40 smithi086 ceph-mon[30943]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-05T06:41:40.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:40 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:41:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:40 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3318668626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:40.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:40 smithi096 ceph-mon[34468]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-05T06:41:40.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:40 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:41:40.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:40 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3318668626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:41:41.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T06:38:55.927385+0000', 'modified': '2024-09-05T06:41:16.604262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24466, 'mds_2': 24461, 'mds_3': 24422, 'mds_4': 24403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3075373024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3075373024}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3075373024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/1507644375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 1507644375}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 1507644375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.115:6833/3498674074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3498674074}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3498674074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.96:6833/1668283475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 1668283475}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 1668283475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/1272008759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 1272008759}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 1272008759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/3823127248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3823127248}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3823127248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6837/2087354694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 2087354694}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2087354694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.96:6837/1007288434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 1007288434}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 1007288434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.115:6839/4249007973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4249007973}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4249007973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/2996609527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2996609527}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2996609527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24403, 24422, 24461, 24466, 24479]} 2024-09-05T06:41:41.262 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T06:41:41.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.l status 2024-09-05T06:41:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 -- 172.21.15.86:0/246247200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 msgr2=0x7fab7c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 --2- 172.21.15.86:0/246247200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c10d130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab7402a410 comp rx=0 tx=0).stop 2024-09-05T06:41:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 -- 172.21.15.86:0/246247200 shutdown_connections 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 --2- 172.21.15.86:0/246247200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c10d130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 --2- 172.21.15.86:0/246247200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 --2- 172.21.15.86:0/246247200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 0x7fab7c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.992+0000 7fab8356b640 1 -- 172.21.15.86:0/246247200 >> 172.21.15.86:0/246247200 conn(0x7fab7c077210 msgr2=0x7fab7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- 172.21.15.86:0/246247200 shutdown_connections 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- 172.21.15.86:0/246247200 wait complete. 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 Processor -- start 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- start start 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 0x7fab7c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10fc40 con 0x7fab7c1055b0 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10fac0 con 0x7fab7c100e30 2024-09-05T06:41:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.993+0000 7fab8356b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10fdc0 con 0x7fab7c100480 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37676/0 (socket says 172.21.15.86:37676) 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 -- 172.21.15.86:0/777373733 learned_addr learned my addr 172.21.15.86:0/777373733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81307640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 0x7fab7c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab80b06640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 -- 172.21.15.86:0/777373733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 msgr2=0x7fab7c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 0x7fab7c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 -- 172.21.15.86:0/777373733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 msgr2=0x7fab7c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 -- 172.21.15.86:0/777373733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c19d290 con 0x7fab7c1055b0 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab80b06640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab81b08640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab74009230 tx=0x7fab74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab7a7fc640 1 -- 172.21.15.86:0/777373733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7403f030 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c19d520 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab7a7fc640 1 -- 172.21.15.86:0/777373733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab74003aa0 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab7a7fc640 1 -- 172.21.15.86:0/777373733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035070 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.994+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a4e60 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.997+0000 7fab7a7fc640 1 -- 172.21.15.86:0/777373733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fab74042030 con 0x7fab7c1055b0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.997+0000 7fab85944740 1 Processor -- start 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.997+0000 7fab85944740 1 -- start start 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.997+0000 7fab85944740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d793564db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab85944740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab85944740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d793571fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab85944740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7934cb090 con 0x55d7935652f0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab85944740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d793472a70 con 0x55d7935649d0 2024-09-05T06:41:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab85944740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d7934c7050 con 0x55d79356fbb0 2024-09-05T06:41:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab7a7fc640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 0x7fab640a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37692/0 (socket says 172.21.15.86:37692) 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 -- 172.21.15.86:0/4156558700 learned_addr learned my addr 172.21.15.86:0/4156558700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab80b06640 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d793564db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81b08640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 0x7fab640a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab80b06640 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d793571fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 -- 172.21.15.86:0/4156558700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 msgr2=0x55d793571fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d793571fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 -- 172.21.15.86:0/4156558700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 msgr2=0x55d793564db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d793564db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 -- 172.21.15.86:0/4156558700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79347f130 con 0x55d7935652f0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab7a7fc640 1 -- 172.21.15.86:0/777373733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fab740f01d0 con 0x7fab7c1055b0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81307640 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00f4e0 tx=0x7fab6c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d35063e94664a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab81b08640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 0x7fab640a5240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab740ed4c0 tx=0x7fab740b8500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab637fe640 1 -- 172.21.15.86:0/4156558700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c004d20 con 0x55d7935652f0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab637fe640 1 -- 172.21.15.86:0/4156558700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab6c00d9b0 con 0x55d7935652f0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.998+0000 7fab637fe640 1 -- 172.21.15.86:0/4156558700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c005590 con 0x55d7935652f0 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 -- 172.21.15.86:0/4156558700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 msgr2=0x55d79356f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00f4e0 tx=0x7fab6c00af90 comp rx=0 tx=0).stop 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 -- 172.21.15.86:0/4156558700 shutdown_connections 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d793571fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935652f0 0x55d79356f670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 --2- 172.21.15.86:0/4156558700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d793564db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 -- 172.21.15.86:0/4156558700 >> 172.21.15.86:0/4156558700 conn(0x55d793560f60 msgr2=0x55d793561340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 -- 172.21.15.86:0/4156558700 shutdown_connections 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:41.999+0000 7fab85944740 1 -- 172.21.15.86:0/4156558700 wait complete. 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 -- start start 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 Processor -- start 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d79356c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d79356cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d793572780 con 0x55d7935649d0 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d793485f20 con 0x55d7935652f0 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d793572900 con 0x55d79356fbb0 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab85944740 10 client.0 _ll_register_callbacks cb 0x55d79344f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37694/0 (socket says 172.21.15.86:37694) 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.000+0000 7fab80b06640 1 -- 172.21.15.86:0/1436246928 learned_addr learned my addr 172.21.15.86:0/1436246928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab81307640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d79356c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 -- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 msgr2=0x55d79356cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d79356cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 -- 172.21.15.86:0/1436246928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 msgr2=0x55d79356c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d79356c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 -- 172.21.15.86:0/1436246928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79323c2b0 con 0x55d7935649d0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab80b06640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab700129c0 tx=0x7fab70012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc16a6c476210ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000fd10 con 0x55d7935649d0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fab700047a0 con 0x55d7935649d0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6c00d9b0 con 0x55d7935649d0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab70027050 con 0x55d7935649d0 2024-09-05T06:41:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab85944740 10 client.15552 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d79357e030 con 0x55d7935649d0 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab6c00d9b0 con 0x55d7935649d0 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fab700104f0 con 0x55d7935649d0 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.001+0000 7fab85944740 10 client.15552 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab85944740 10 client.15552 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab85944740 20 client.15552 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d793580700 0x55d793581b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab85944740 4 client.15552 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7934d8600 con 0x55d793580700 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab81307640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d793580700 0x55d793581b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab60ff9640 10 client.15552 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.002+0000 7fab81307640 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d793580700 0x55d793581b70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6a8c956cbd4d2c4 server_cookie=e930dc75e8f3dd71 in_seq=0 out_seq=0 2024-09-05T06:41:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab60ff9640 1 -- 172.21.15.86:0/1436246928 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7934d8600 con 0x55d793580700 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab60ff9640 10 client.15552 handle_command_reply: tid=0 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 client.15552 shutdown 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 msgr2=0x55d79356f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab700129c0 tx=0x7fab70012e80 comp rx=0 tx=0).stop 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 shutdown_connections 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d793580700 0x55d793581b70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d79356fbb0 0x55d79356cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab60ff9640 0 client.15552 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d79356c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 --2- 172.21.15.86:0/1436246928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7935649d0 0x55d79356f3e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 >> 172.21.15.86:0/1436246928 conn(0x55d793560f60 msgr2=0x55d793566c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 shutdown_connections 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.003+0000 7fab85944740 1 -- 172.21.15.86:0/1436246928 wait complete. 2024-09-05T06:41:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.004+0000 7fab85944740 20 client.15552 trim_cache size 0 max 16384 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 Processor -- start 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 -- start start 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d79356bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935652f0 0x55d79358a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d79357feb0 con 0x55d79356fbb0 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d793580580 con 0x55d7935649d0 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab85944740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d793580c50 con 0x55d7935652f0 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.021+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37710/0 (socket says 172.21.15.86:37710) 2024-09-05T06:41:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 -- 172.21.15.86:0/2683140717 learned_addr learned my addr 172.21.15.86:0/2683140717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab81307640 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935652f0 0x55d79358a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d79356bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 -- 172.21.15.86:0/2683140717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935652f0 msgr2=0x55d79358a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935652f0 0x55d79358a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 -- 172.21.15.86:0/2683140717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 msgr2=0x55d79356bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d79356bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 -- 172.21.15.86:0/2683140717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79356f750 con 0x55d79356fbb0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab80b06640 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab70005940 tx=0x7fab700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeb632f03cf7fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab62ffd640 1 -- 172.21.15.86:0/2683140717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70033070 con 0x55d79356fbb0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab62ffd640 1 -- 172.21.15.86:0/2683140717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab700105d0 con 0x55d79356fbb0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.022+0000 7fab62ffd640 1 -- 172.21.15.86:0/2683140717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70005030 con 0x55d79356fbb0 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- 172.21.15.86:0/2683140717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 msgr2=0x55d79356b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab70005940 tx=0x7fab700046c0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- 172.21.15.86:0/2683140717 shutdown_connections 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d79356b000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935652f0 0x55d79358a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 --2- 172.21.15.86:0/2683140717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935649d0 0x55d79356bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- 172.21.15.86:0/2683140717 >> 172.21.15.86:0/2683140717 conn(0x55d793560f60 msgr2=0x55d79356c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- 172.21.15.86:0/2683140717 shutdown_connections 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- 172.21.15.86:0/2683140717 wait complete. 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 -- start start 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.023+0000 7fab85944740 1 Processor -- start 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935649d0 0x55d793587460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d7935879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7935819f0 con 0x55d79356fbb0 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d793581320 con 0x55d7935652f0 2024-09-05T06:41:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d7935820c0 con 0x55d7935649d0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 10 client.0 _ll_register_callbacks cb 0x7fab70012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37720/0 (socket says 172.21.15.86:37720) 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 -- 172.21.15.86:0/3022248759 learned_addr learned my addr 172.21.15.86:0/3022248759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab81307640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d7935879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935649d0 0x55d793587460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 -- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935649d0 msgr2=0x55d793587460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935649d0 0x55d793587460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 -- 172.21.15.86:0/3022248759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 msgr2=0x55d7935879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d7935879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 -- 172.21.15.86:0/3022248759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7935e7430 con 0x55d79356fbb0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab80b06640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab700109e0 tx=0x7fab70010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feb7f1dbc1f7b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70033070 con 0x55d79356fbb0 2024-09-05T06:41:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fab70004930 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.024+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab700105d0 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab70045050 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 10 client.15558 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7935ee7e0 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab700105d0 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fab7001cbc0 con 0x55d79356fbb0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 10 client.15558 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 10 client.15558 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 20 client.15558 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d7935f2f30 0x55d7935f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 4 client.15558 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "status"}) -- 0x55d7934d2b10 con 0x55d7935f2f30 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.025+0000 7fab81307640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d7935f2f30 0x55d7935f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab81307640 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d7935f2f30 0x55d7935f1c40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14f9c9fee29237e1 server_cookie=564c475bfa9213b6 in_seq=0 out_seq=0 2024-09-05T06:41:42.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab60ff9640 10 client.15558 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab60ff9640 1 -- 172.21.15.86:0/3022248759 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d7934d2b10 con 0x55d7935f2f30 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab60ff9640 10 client.15558 handle_command_reply: tid=0 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 client.15558 shutdown 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 msgr2=0x55d7935e6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab700109e0 tx=0x7fab70010780 comp rx=0 tx=0).stop 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 shutdown_connections 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55d7935f2f30 0x55d7935f1c40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d79356fbb0 0x55d7935e6e70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7935652f0 0x55d7935879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 --2- 172.21.15.86:0/3022248759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7935649d0 0x55d793587460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 >> 172.21.15.86:0/3022248759 conn(0x55d793560f60 msgr2=0x55d79356c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab60ff9640 0 client.15558 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 shutdown_connections 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 1 -- 172.21.15.86:0/3022248759 wait complete. 2024-09-05T06:41:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.026+0000 7fab85944740 20 client.15558 trim_cache size 0 max 16384 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24403, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 129.17671439899999, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T06:41:42.037 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 207.43855411499999 2024-09-05T06:41:42.038 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:42.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 msgr2=0x7fab640a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 0x7fab640a5240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab740ed4c0 tx=0x7fab740b8500 comp rx=0 tx=0).stop 2024-09-05T06:41:42.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 msgr2=0x7fab7c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:42.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab74009230 tx=0x7fab74004270 comp rx=0 tx=0).stop 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 shutdown_connections 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fab640a2d80 0x7fab640a5240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab7c1055b0 0x7fab7c19cbb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab7c100e30 0x7fab7c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 --2- 172.21.15.86:0/777373733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab7c100480 0x7fab7c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 >> 172.21.15.86:0/777373733 conn(0x7fab7c077210 msgr2=0x7fab7c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 shutdown_connections 2024-09-05T06:41:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:42.027+0000 7fab8356b640 1 -- 172.21.15.86:0/777373733 wait complete. 2024-09-05T06:41:42.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:42 smithi086 ceph-mon[30943]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-05T06:41:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:42 smithi115 ceph-mon[34791]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-05T06:41:42.738 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.k status 2024-09-05T06:41:42.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:42 smithi096 ceph-mon[34468]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-05T06:41:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.461+0000 7fda7ad02640 1 -- 172.21.15.86:0/3100809141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74100e20 msgr2=0x7fda74101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.461+0000 7fda7ad02640 1 --2- 172.21.15.86:0/3100809141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74100e20 0x7fda74101200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda64009900 tx=0x7fda6402a480 comp rx=0 tx=0).stop 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- 172.21.15.86:0/3100809141 shutdown_connections 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- 172.21.15.86:0/3100809141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74102ef0 0x7fda7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- 172.21.15.86:0/3100809141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102570 0x7fda741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- 172.21.15.86:0/3100809141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74100e20 0x7fda74101200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- 172.21.15.86:0/3100809141 >> 172.21.15.86:0/3100809141 conn(0x7fda740fc7d0 msgr2=0x7fda740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- 172.21.15.86:0/3100809141 shutdown_connections 2024-09-05T06:41:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- 172.21.15.86:0/3100809141 wait complete. 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 Processor -- start 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- start start 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74100e20 0x7fda74197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 0x7fda7419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda74110e00 con 0x7fda74102570 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda74110c80 con 0x7fda74102ef0 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda7ad02640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda74110f80 con 0x7fda74100e20 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.462+0000 7fda73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47884/0 (socket says 172.21.15.86:47884) 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 -- 172.21.15.86:0/2159867410 learned_addr learned my addr 172.21.15.86:0/2159867410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda78a9e640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74100e20 0x7fda74197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda7929f640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 0x7fda7419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 -- 172.21.15.86:0/2159867410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74100e20 msgr2=0x7fda74197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74100e20 0x7fda74197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 -- 172.21.15.86:0/2159867410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 msgr2=0x7fda7419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 0x7fda7419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 -- 172.21.15.86:0/2159867410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7419c1f0 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda7929f640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 0x7fda7419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda73fff640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c920 tx=0x7fda5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda71ffb640 1 -- 172.21.15.86:0/2159867410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c007c70 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda71ffb640 1 -- 172.21.15.86:0/2159867410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda5c007e10 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7419c4e0 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda71ffb640 1 -- 172.21.15.86:0/2159867410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c018820 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.463+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda741a3dc0 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.465+0000 7fda71ffb640 1 -- 172.21.15.86:0/2159867410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fda5c0189c0 con 0x7fda74102570 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 Processor -- start 2024-09-05T06:41:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 -- start start 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda71ffb640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 0x7fda600a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a99b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a99e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a8f5090 con 0x55ed2a98e850 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a89ca70 con 0x55ed2a98f170 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7d0db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a8f1050 con 0x55ed2a99bdc0 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda71ffb640 1 -- 172.21.15.86:0/2159867410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fda5c0cfbd0 con 0x7fda74102570 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda78a9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda78a9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47900/0 (socket says 172.21.15.86:47900) 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda78a9e640 1 -- 172.21.15.86:0/299543843 learned_addr learned my addr 172.21.15.86:0/299543843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.466+0000 7fda7929f640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 0x7fda600a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 -- 172.21.15.86:0/299543843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 msgr2=0x55ed2a99e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a99e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda73fff640 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a99b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 -- 172.21.15.86:0/299543843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 msgr2=0x55ed2a99b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a99b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 -- 172.21.15.86:0/299543843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2a8a9130 con 0x55ed2a98e850 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda73fff640 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a99b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda78a9e640 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda64009900 tx=0x7fda64031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b789ce4093da96 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda7929f640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 0x7fda600a52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55ed2a98c050 tx=0x7fda6c00a420 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda6affd640 1 -- 172.21.15.86:0/299543843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6403f070 con 0x55ed2a98e850 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda6affd640 1 -- 172.21.15.86:0/299543843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda64031e50 con 0x55ed2a98e850 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.467+0000 7fda6affd640 1 -- 172.21.15.86:0/299543843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64035070 con 0x55ed2a98e850 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- 172.21.15.86:0/299543843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 msgr2=0x55ed2a98ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda64009900 tx=0x7fda64031c60 comp rx=0 tx=0).stop 2024-09-05T06:41:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- 172.21.15.86:0/299543843 shutdown_connections 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a99e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a99b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 --2- 172.21.15.86:0/299543843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98e850 0x55ed2a98ec30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- 172.21.15.86:0/299543843 >> 172.21.15.86:0/299543843 conn(0x55ed2a98af60 msgr2=0x55ed2a98b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- 172.21.15.86:0/299543843 shutdown_connections 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- 172.21.15.86:0/299543843 wait complete. 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 -- start start 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.468+0000 7fda7d0db740 1 Processor -- start 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a99b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a998e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a99e990 con 0x55ed2a98f170 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a8aff20 con 0x55ed2a98e850 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a99eb10 con 0x55ed2a99bdc0 2024-09-05T06:41:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 10 client.0 _ll_register_callbacks cb 0x55ed2a879030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47906/0 (socket says 172.21.15.86:47906) 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 -- 172.21.15.86:0/3152337453 learned_addr learned my addr 172.21.15.86:0/3152337453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7929f640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a998e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda78a9e640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a99b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 -- 172.21.15.86:0/3152337453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 msgr2=0x55ed2a998e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a998e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 -- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 msgr2=0x55ed2a99b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a99b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 -- 172.21.15.86:0/3152337453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2a6662b0 con 0x55ed2a98f170 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda78a9e640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a99b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda73fff640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c8f0 tx=0x7fda5c018d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561722fdb9997654 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda53fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:43.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0cd2d0 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fda5c0a4320 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.469+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda64031e50 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda5c09bb40 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed2a9aa230 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda64031e50 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fda5c097a90 con 0x55ed2a98f170 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 10 client.15570 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 20 client.15570 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2a9ac850 0x55ed2a9adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 4 client.15570 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed2a902600 con 0x55ed2a9ac850 2024-09-05T06:41:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.470+0000 7fda78a9e640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2a9ac850 0x55ed2a9adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda78a9e640 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2a9ac850 0x55ed2a9adcc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d34e27ca130e5c0 server_cookie=1fd98de5087fd72a in_seq=0 out_seq=0 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda53fff640 10 client.15570 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda53fff640 1 -- 172.21.15.86:0/3152337453 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed2a902600 con 0x55ed2a9ac850 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda53fff640 10 client.15570 handle_command_reply: tid=0 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 client.15570 shutdown 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 msgr2=0x55ed2a998910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c8f0 tx=0x7fda5c018d70 comp rx=0 tx=0).stop 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 shutdown_connections 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2a9ac850 0x55ed2a9adcc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a99bdc0 0x55ed2a998e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a98f170 0x55ed2a998910 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.471+0000 7fda53fff640 0 client.15570 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.472+0000 7fda7d0db740 1 --2- 172.21.15.86:0/3152337453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a99b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.472+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 >> 172.21.15.86:0/3152337453 conn(0x55ed2a98af60 msgr2=0x55ed2a990be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.472+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 shutdown_connections 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.472+0000 7fda7d0db740 1 -- 172.21.15.86:0/3152337453 wait complete. 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.472+0000 7fda7d0db740 20 client.15570 trim_cache size 0 max 16384 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 Processor -- start 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 -- start start 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 0x55ed2a9a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a9b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9ac000 con 0x55ed2a99bdc0 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9ac6d0 con 0x55ed2a98f170 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7d0db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9acda0 con 0x55ed2a98e850 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7929f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7929f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47910/0 (socket says 172.21.15.86:47910) 2024-09-05T06:41:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.489+0000 7fda7929f640 1 -- 172.21.15.86:0/1759034191 learned_addr learned my addr 172.21.15.86:0/1759034191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda78a9e640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 0x55ed2a9a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda73fff640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a9b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 -- 172.21.15.86:0/1759034191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 msgr2=0x55ed2a9a8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 0x55ed2a9a8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 -- 172.21.15.86:0/1759034191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 msgr2=0x55ed2a9b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a9b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-09-05T06:41:43.508 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24422, 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 131.65054584200001, 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 206.289389779 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 -- 172.21.15.86:0/1759034191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2a98efa0 con 0x55ed2a99bdc0 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda78a9e640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 0x55ed2a9a8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda7929f640 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fda6c011790 tx=0x7fda6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadb781131e4245c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda6a7fc640 1 -- 172.21.15.86:0/1759034191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01ca30 con 0x55ed2a99bdc0 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda6a7fc640 1 -- 172.21.15.86:0/1759034191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda6c01cbd0 con 0x55ed2a99bdc0 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.490+0000 7fda6a7fc640 1 -- 172.21.15.86:0/1759034191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c01d6b0 con 0x55ed2a99bdc0 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- 172.21.15.86:0/1759034191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 msgr2=0x55ed2a9971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fda6c011790 tx=0x7fda6c012000 comp rx=0 tx=0).stop 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- 172.21.15.86:0/1759034191 shutdown_connections 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9971e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98f170 0x55ed2a9b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- 172.21.15.86:0/1759034191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98e850 0x55ed2a9a8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- 172.21.15.86:0/1759034191 >> 172.21.15.86:0/1759034191 conn(0x55ed2a98af60 msgr2=0x55ed2a985ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- 172.21.15.86:0/1759034191 shutdown_connections 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- 172.21.15.86:0/1759034191 wait complete. 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 -- start start 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 Processor -- start 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a993c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 0x55ed2a994150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.491+0000 7fda7d0db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9adb40 con 0x55ed2a99bdc0 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9ad470 con 0x55ed2a98e850 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ed2a9ae210 con 0x55ed2a98f170 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 10 client.0 _ll_register_callbacks cb 0x7fda5c086040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47924/0 (socket says 172.21.15.86:47924) 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 -- 172.21.15.86:0/4004215212 learned_addr learned my addr 172.21.15.86:0/4004215212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda73fff640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 0x55ed2a994150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda78a9e640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a993c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 -- 172.21.15.86:0/4004215212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 msgr2=0x55ed2a994150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 0x55ed2a994150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 -- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 msgr2=0x55ed2a993c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a993c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 -- 172.21.15.86:0/4004215212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2a9b6420 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda78a9e640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a993c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda73fff640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 0x55ed2a994150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7929f640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda6c011760 tx=0x7fda6c01cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619edafc01fbed39 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda53fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c036060 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fda6c004a40 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6c01cbd0 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda6c03d050 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 10 client.15576 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.492+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed2aa18b70 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda6c01cbd0 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fda6c005160 con 0x55ed2a99bdc0 2024-09-05T06:41:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 10 client.15576 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 20 client.15576 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2aa1d3c0 0x55ed2aa1c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 4 client.15576 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "status"}) -- 0x55ed2a8fcb10 con 0x55ed2aa1d3c0 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda78a9e640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2aa1d3c0 0x55ed2aa1c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda53fff640 10 client.15576 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.493+0000 7fda78a9e640 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2aa1d3c0 0x55ed2aa1c0d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b54eb0952d1e52e2 server_cookie=323aae6c8c38a7af in_seq=0 out_seq=0 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda53fff640 1 -- 172.21.15.86:0/4004215212 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ed2a8fcb10 con 0x55ed2aa1d3c0 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda53fff640 10 client.15576 handle_command_reply: tid=0 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 client.15576 shutdown 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 msgr2=0x55ed2a9b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda6c011760 tx=0x7fda6c01cfb0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 shutdown_connections 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ed2aa1d3c0 0x55ed2aa1c0d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2a99bdc0 0x55ed2a9b5c20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ed2a98f170 0x55ed2a994150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 --2- 172.21.15.86:0/4004215212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2a98e850 0x55ed2a993c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda53fff640 0 client.15576 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 >> 172.21.15.86:0/4004215212 conn(0x55ed2a98af60 msgr2=0x55ed2a997160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 shutdown_connections 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 1 -- 172.21.15.86:0/4004215212 wait complete. 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7d0db740 20 client.15576 trim_cache size 0 max 16384 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 msgr2=0x7fda600a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 0x7fda600a52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55ed2a98c050 tx=0x7fda6c00a420 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 msgr2=0x7fda74197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.494+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c920 tx=0x7fda5c00cde0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 shutdown_connections 2024-09-05T06:41:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda600a2e00 0x7fda600a52c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda74102ef0 0x7fda7419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda74102570 0x7fda74197780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 --2- 172.21.15.86:0/2159867410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda74100e20 0x7fda74197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 >> 172.21.15.86:0/2159867410 conn(0x7fda740fc7d0 msgr2=0x7fda74106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 shutdown_connections 2024-09-05T06:41:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:43.495+0000 7fda7ad02640 1 -- 172.21.15.86:0/2159867410 wait complete. 2024-09-05T06:41:44.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T06:41:44.434 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:44.679 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:44 smithi086 ceph-mon[30943]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-09-05T06:41:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:44 smithi115 ceph-mon[34791]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-09-05T06:41:44.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:44 smithi096 ceph-mon[34468]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 -- 172.21.15.86:0/4268234647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 msgr2=0x7f6f6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 --2- 172.21.15.86:0/4268234647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c10da40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009ca0 tx=0x7f6f6402a950 comp rx=0 tx=0).stop 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 -- 172.21.15.86:0/4268234647 shutdown_connections 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 --2- 172.21.15.86:0/4268234647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c10da40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 --2- 172.21.15.86:0/4268234647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 0x7f6f6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 --2- 172.21.15.86:0/4268234647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 0x7f6f6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 -- 172.21.15.86:0/4268234647 >> 172.21.15.86:0/4268234647 conn(0x7f6f6c0fc7d0 msgr2=0x7f6f6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 -- 172.21.15.86:0/4268234647 shutdown_connections 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 -- 172.21.15.86:0/4268234647 wait complete. 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.937+0000 7f6f717f4640 1 Processor -- start 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 -- start start 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 0x7f6f6c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 0x7f6f6c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c110e30 con 0x7f6f6c102ef0 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c110cb0 con 0x7f6f6c102570 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f717f4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c110fb0 con 0x7f6f6c100e20 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 0x7f6f6c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 0x7f6f6c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47944/0 (socket says 172.21.15.86:47944) 2024-09-05T06:41:44.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 -- 172.21.15.86:0/3258992683 learned_addr learned my addr 172.21.15.86:0/3258992683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 -- 172.21.15.86:0/3258992683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 msgr2=0x7f6f6c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 0x7f6f6c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 -- 172.21.15.86:0/3258992683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 msgr2=0x7f6f6c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 0x7f6f6c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.938+0000 7f6f6b7fe640 1 -- 172.21.15.86:0/3258992683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c19d350 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f6b7fe640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402ae70 tx=0x7f6f6402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f53fff640 1 -- 172.21.15.86:0/3258992683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64034040 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c19d5e0 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f53fff640 1 -- 172.21.15.86:0/3258992683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f640308f0 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f53fff640 1 -- 172.21.15.86:0/3258992683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64004780 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.939+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a4f20 con 0x7f6f6c102ef0 2024-09-05T06:41:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.941+0000 7f6f53fff640 1 -- 172.21.15.86:0/3258992683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6f64037020 con 0x7f6f6c102ef0 2024-09-05T06:41:44.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.941+0000 7f6f73bcd740 1 Processor -- start 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.941+0000 7f6f73bcd740 1 -- start start 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a263c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a272a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a1ca090 con 0x55ab9a272fc0 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a171a70 con 0x55ab9a263890 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a1c6050 con 0x55ab9a2641b0 2024-09-05T06:41:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f53fff640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 0x7f6f540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47956/0 (socket says 172.21.15.86:47956) 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/552723103 learned_addr learned my addr 172.21.15.86:0/552723103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6b7fe640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 0x7f6f540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a272a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a263c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f53fff640 1 -- 172.21.15.86:0/3258992683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6f640f08d0 con 0x7f6f6c102ef0 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 -- 172.21.15.86:0/552723103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 msgr2=0x55ab9a263c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a263c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 -- 172.21.15.86:0/552723103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 msgr2=0x55ab9a2753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 -- 172.21.15.86:0/552723103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9a17e130 con 0x55ab9a2641b0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6affd640 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a272a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00ab40 tx=0x7f6f5c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ace43c3cc4f4cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.942+0000 7f6f6b7fe640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 0x7f6f540a5240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f640edcf0 tx=0x7f6f6408e040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f517fa640 1 -- 172.21.15.86:0/552723103 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c004020 con 0x55ab9a2641b0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f517fa640 1 -- 172.21.15.86:0/552723103 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f5c0041c0 con 0x55ab9a2641b0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f517fa640 1 -- 172.21.15.86:0/552723103 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0186a0 con 0x55ab9a2641b0 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 -- 172.21.15.86:0/552723103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 msgr2=0x55ab9a272a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a272a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00ab40 tx=0x7f6f5c00f3c0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 -- 172.21.15.86:0/552723103 shutdown_connections 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a272a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/552723103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a263c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 -- 172.21.15.86:0/552723103 >> 172.21.15.86:0/552723103 conn(0x55ab9a25ff60 msgr2=0x55ab9a260340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 -- 172.21.15.86:0/552723103 shutdown_connections 2024-09-05T06:41:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.943+0000 7f6f73bcd740 1 -- 172.21.15.86:0/552723103 wait complete. 2024-09-05T06:41:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 -- start start 2024-09-05T06:41:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 Processor -- start 2024-09-05T06:41:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a2641b0 0x55ab9a26fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a272fc0 0x55ab9a270050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a275b90 con 0x55ab9a263890 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a184f20 con 0x55ab9a2641b0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a275d10 con 0x55ab9a272fc0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f73bcd740 10 client.0 _ll_register_callbacks cb 0x55ab9a14e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47966/0 (socket says 172.21.15.86:47966) 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/32845747 learned_addr learned my addr 172.21.15.86:0/32845747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a272fc0 0x55ab9a270050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6affd640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a2641b0 0x55ab9a26fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/32845747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a272fc0 msgr2=0x55ab9a270050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a272fc0 0x55ab9a270050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.944+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a2641b0 msgr2=0x55ab9a26fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a2641b0 0x55ab9a26fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/32845747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab99f3b2b0 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f60003660 tx=0x7f6f60014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0d87afb45b14e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60012930 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6f60012ad0 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c0041c0 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f6002a050 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab9a281510 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f5c0041c0 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6f6000f3f0 con 0x55ab9a263890 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 10 client.15588 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:41:44.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 20 client.15588 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a283b10 0x55ab9a284f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 4 client.15588 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.945+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab9a1d7600 con 0x55ab9a283b10 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.946+0000 7f6f6affd640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a283b10 0x55ab9a284f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.946+0000 7f6f6affd640 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a283b10 0x55ab9a284f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6f6d4dc5249782b server_cookie=132e0a151a77fe65 in_seq=0 out_seq=0 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.946+0000 7f6f4eff5640 10 client.15588 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f4eff5640 1 -- 172.21.15.86:0/32845747 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab9a1d7600 con 0x55ab9a283b10 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f4eff5640 10 client.15588 handle_command_reply: tid=0 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 client.15588 shutdown 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 msgr2=0x55ab9a2727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f60003660 tx=0x7f6f60014040 comp rx=0 tx=0).stop 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 shutdown_connections 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a283b10 0x55ab9a284f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a272fc0 0x55ab9a270050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a2641b0 0x55ab9a26fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/32845747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a263890 0x55ab9a2727c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 >> 172.21.15.86:0/32845747 conn(0x55ab9a25ff60 msgr2=0x55ab9a265cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f4eff5640 0 client.15588 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 shutdown_connections 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 1 -- 172.21.15.86:0/32845747 wait complete. 2024-09-05T06:41:44.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.947+0000 7f6f73bcd740 20 client.15588 trim_cache size 0 max 16384 2024-09-05T06:41:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.964+0000 7f6f73bcd740 1 Processor -- start 2024-09-05T06:41:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.964+0000 7f6f73bcd740 1 -- start start 2024-09-05T06:41:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a2841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a26e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a2832c0 con 0x55ab9a272fc0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a283990 con 0x55ab9a263890 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a284060 con 0x55ab9a2641b0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47982/0 (socket says 172.21.15.86:47982) 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1050261719 learned_addr learned my addr 172.21.15.86:0/1050261719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6affd640 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a26e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1050261719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 msgr2=0x55ab9a26e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a26e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1050261719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 msgr2=0x55ab9a2841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a2841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1050261719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9a270630 con 0x55ab9a272fc0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b820 tx=0x7f6f60012560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671638f0ab913901 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f50ff9640 1 -- 172.21.15.86:0/1050261719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60033070 con 0x55ab9a272fc0 2024-09-05T06:41:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f50ff9640 1 -- 172.21.15.86:0/1050261719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f6000b390 con 0x55ab9a272fc0 2024-09-05T06:41:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.965+0000 7f6f50ff9640 1 -- 172.21.15.86:0/1050261719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000f9e0 con 0x55ab9a272fc0 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1050261719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 msgr2=0x55ab9a26e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b820 tx=0x7f6f60012560 comp rx=0 tx=0).stop 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1050261719 shutdown_connections 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a26e920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a26e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1050261719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a2841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1050261719 >> 172.21.15.86:0/1050261719 conn(0x55ab9a25ff60 msgr2=0x55ab9a26fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1050261719 shutdown_connections 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1050261719 wait complete. 2024-09-05T06:41:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 -- start start 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.966+0000 7f6f73bcd740 1 Processor -- start 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a28d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a28d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a284e00 con 0x55ab9a272fc0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a284730 con 0x55ab9a263890 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a2854d0 con 0x55ab9a2641b0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 10 client.0 _ll_register_callbacks cb 0x7f6f6000bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47984/0 (socket says 172.21.15.86:47984) 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1757713500 learned_addr learned my addr 172.21.15.86:0/1757713500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6affd640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a28d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a28d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1757713500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 msgr2=0x55ab9a28d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a28d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 msgr2=0x55ab9a28d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a28d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 -- 172.21.15.86:0/1757713500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9a286d90 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f6a7fc640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f60000c00 tx=0x7f6f600125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61fcd856b5f8118 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60033070 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6f6000fbe0 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.967+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6000b390 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f60042050 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 10 client.15594 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab9a28b740 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f6000b390 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6f6002e430 con 0x55ab9a272fc0 2024-09-05T06:41:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 10 client.15594 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 20 client.15594 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a2f22e0 0x55ab9a2f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 4 client.15594 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "status"}) -- 0x55ab9a1d1b10 con 0x55ab9a2f22e0 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.968+0000 7f6f6affd640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a2f22e0 0x55ab9a2f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f4eff5640 10 client.15594 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f6affd640 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a2f22e0 0x55ab9a2f0ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9269def57fb47531 server_cookie=4a98b821b662003 in_seq=0 out_seq=0 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f4eff5640 1 -- 172.21.15.86:0/1757713500 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ab9a1d1b10 con 0x55ab9a2f22e0 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f4eff5640 10 client.15594 handle_command_reply: tid=0 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 client.15594 shutdown 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 msgr2=0x55ab9a2866f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f60000c00 tx=0x7f6f600125d0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 shutdown_connections 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab9a2f22e0 0x55ab9a2f0ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab9a272fc0 0x55ab9a2866f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab9a2641b0 0x55ab9a28d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 --2- 172.21.15.86:0/1757713500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab9a263890 0x55ab9a28d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f4eff5640 0 client.15594 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 >> 172.21.15.86:0/1757713500 conn(0x55ab9a25ff60 msgr2=0x55ab9a26e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 shutdown_connections 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 1 -- 172.21.15.86:0/1757713500 wait complete. 2024-09-05T06:41:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f73bcd740 20 client.15594 trim_cache size 0 max 16384 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24461, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 134.10214303800001, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T06:41:44.975 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 190.82056371100001 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 msgr2=0x7f6f540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 0x7f6f540a5240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f640edcf0 tx=0x7f6f6408e040 comp rx=0 tx=0).stop 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 msgr2=0x7f6f6c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402ae70 tx=0x7f6f6402f5d0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 shutdown_connections 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f540a2d80 0x7f6f540a5240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6c102ef0 0x7f6f6c19cc70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6c102570 0x7f6f6c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 --2- 172.21.15.86:0/3258992683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f6c100e20 0x7f6f6c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 >> 172.21.15.86:0/3258992683 conn(0x7f6f6c0fc7d0 msgr2=0x7f6f6c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 shutdown_connections 2024-09-05T06:41:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:44.969+0000 7f6f717f4640 1 -- 172.21.15.86:0/3258992683 wait complete. 2024-09-05T06:41:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:45 smithi115 ceph-mon[34791]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:45.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:45 smithi086 ceph-mon[30943]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:45 smithi096 ceph-mon[34468]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:45.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T06:41:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:46.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 -- 172.21.15.86:0/3828119320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08105f40 msgr2=0x7fba081063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 --2- 172.21.15.86:0/3828119320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08105f40 0x7fba081063d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba00009ca0 tx=0x7fba0002a940 comp rx=0 tx=0).stop 2024-09-05T06:41:46.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 -- 172.21.15.86:0/3828119320 shutdown_connections 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 --2- 172.21.15.86:0/3828119320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08105f40 0x7fba081063d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 --2- 172.21.15.86:0/3828119320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 0x7fba08105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 --2- 172.21.15.86:0/3828119320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08100b30 0x7fba08100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 -- 172.21.15.86:0/3828119320 >> 172.21.15.86:0/3828119320 conn(0x7fba080fc7a0 msgr2=0x7fba080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 -- 172.21.15.86:0/3828119320 shutdown_connections 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.589+0000 7fba0d915640 1 -- 172.21.15.86:0/3828119320 wait complete. 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 Processor -- start 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 -- start start 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 0x7fba081a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08105f40 0x7fba0819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba08118420 con 0x7fba08100b30 2024-09-05T06:41:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba081182a0 con 0x7fba08101450 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba0d915640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba081185a0 con 0x7fba08105f40 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48004/0 (socket says 172.21.15.86:48004) 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 0x7fba081a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455394046 learned_addr learned my addr 172.21.15.86:0/3455394046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.590+0000 7fba077fe640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08105f40 0x7fba0819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455394046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08105f40 msgr2=0x7fba0819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08105f40 0x7fba0819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455394046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 msgr2=0x7fba081a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 0x7fba081a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455394046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0819ef50 con 0x7fba08100b30 2024-09-05T06:41:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c970 tx=0x7fb9f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba0c913640 1 -- 172.21.15.86:0/3455394046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8007bb0 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba0c913640 1 -- 172.21.15.86:0/3455394046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f8007d50 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0819f240 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba0c913640 1 -- 172.21.15.86:0/3455394046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800f450 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.591+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081ab490 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.593+0000 7fba0c913640 1 -- 172.21.15.86:0/3455394046 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb9f8004f20 con 0x7fba08100b30 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 Processor -- start 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 -- start start 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1de8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0c913640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 0x7fb9f40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1deafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581e1d44090 con 0x5581e1ddd9d0 2024-09-05T06:41:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581e1ceba70 con 0x5581e1dde2f0 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0fcee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5581e1d40050 con 0x5581e1de8bb0 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48010/0 (socket says 172.21.15.86:48010) 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba077fe640 1 -- 172.21.15.86:0/2741124263 learned_addr learned my addr 172.21.15.86:0/2741124263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 0x7fb9f40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba0c913640 1 -- 172.21.15.86:0/3455394046 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb9f80cf6a0 con 0x7fba08100b30 2024-09-05T06:41:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba06ffd640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1de8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.594+0000 7fba077fe640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1deafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 -- 172.21.15.86:0/2741124263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 msgr2=0x5581e1deafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1deafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 -- 172.21.15.86:0/2741124263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 msgr2=0x5581e1de8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1de8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 -- 172.21.15.86:0/2741124263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e1cf8130 con 0x5581e1ddd9d0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba06ffd640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1de8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 0x7fb9f40a5240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba080ffe70 tx=0x7fb9f0005f70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fba077fe640 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fba00009a60 tx=0x7fba00009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cebda34cceecd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fb9fdffb640 1 -- 172.21.15.86:0/2741124263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00034040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fb9fdffb640 1 -- 172.21.15.86:0/2741124263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba00044040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.595+0000 7fb9fdffb640 1 -- 172.21.15.86:0/2741124263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00030840 con 0x5581e1ddd9d0 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- 172.21.15.86:0/2741124263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 msgr2=0x5581e1ddddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fba00009a60 tx=0x7fba00009900 comp rx=0 tx=0).stop 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- 172.21.15.86:0/2741124263 shutdown_connections 2024-09-05T06:41:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1deafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1de8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 --2- 172.21.15.86:0/2741124263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1ddddb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- 172.21.15.86:0/2741124263 >> 172.21.15.86:0/2741124263 conn(0x5581e1dd9f60 msgr2=0x5581e1dda340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- 172.21.15.86:0/2741124263 shutdown_connections 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- 172.21.15.86:0/2741124263 wait complete. 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 -- start start 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.596+0000 7fba0fcee740 1 Processor -- start 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1dde2f0 0x5581e1de57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1de8bb0 0x5581e1de5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581e1deb780 con 0x5581e1ddd9d0 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581e1cfef20 con 0x5581e1de8bb0 2024-09-05T06:41:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5581e1deb900 con 0x5581e1dde2f0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 10 client.0 _ll_register_callbacks cb 0x5581e1cc8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48016/0 (socket says 172.21.15.86:48016) 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455471815 learned_addr learned my addr 172.21.15.86:0/3455471815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1dde2f0 0x5581e1de57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba077fe640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1de8bb0 0x5581e1de5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1dde2f0 msgr2=0x5581e1de57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1dde2f0 0x5581e1de57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455471815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1de8bb0 msgr2=0x5581e1de5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1de8bb0 0x5581e1de5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 -- 172.21.15.86:0/3455471815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e1ab52b0 con 0x5581e1ddd9d0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9f809b420 tx=0x7fb9f809b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37353ebc34a796c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fb9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80cf040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb9f80b02c0 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00044040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9f80b04c0 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 10 client.15606 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.597+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581e1df70a0 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba00044040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb9f80a3e50 con 0x5581e1ddd9d0 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 10 client.15606 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 20 client.15606 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1df9690 0x5581e1dfab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 4 client.15606 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e1d51600 con 0x5581e1df9690 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.598+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1df9690 0x5581e1dfab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.599+0000 7fb9e6ffd640 10 client.15606 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.599+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1df9690 0x5581e1dfab00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dba10424e1ad9378 server_cookie=11a7778bcc95863b in_seq=0 out_seq=0 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.599+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/3455471815 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581e1d51600 con 0x5581e1df9690 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.599+0000 7fb9e6ffd640 10 client.15606 handle_command_reply: tid=0 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.599+0000 7fba0fcee740 1 client.15606 shutdown 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 msgr2=0x5581e1de8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9f809b420 tx=0x7fb9f809b260 comp rx=0 tx=0).stop 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 shutdown_connections 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1df9690 0x5581e1dfab00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1de8bb0 0x5581e1de5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1dde2f0 0x5581e1de57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fb9e6ffd640 0 client.15606 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3455471815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1de8450 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 >> 172.21.15.86:0/3455471815 conn(0x5581e1dd9f60 msgr2=0x5581e1ddfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 shutdown_connections 2024-09-05T06:41:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 1 -- 172.21.15.86:0/3455471815 wait complete. 2024-09-05T06:41:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.600+0000 7fba0fcee740 20 client.15606 trim_cache size 0 max 16384 2024-09-05T06:41:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.617+0000 7fba0fcee740 1 Processor -- start 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.617+0000 7fba0fcee740 1 -- start start 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1ddd9d0 0x5581e1de5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1de2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581e1df8e40 con 0x5581e1dde2f0 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581e1df9510 con 0x5581e1ddd9d0 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba0fcee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5581e1df9be0 con 0x5581e1de8bb0 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48022/0 (socket says 172.21.15.86:48022) 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 -- 172.21.15.86:0/3955912462 learned_addr learned my addr 172.21.15.86:0/3955912462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba077fe640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1de2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba06ffd640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1ddd9d0 0x5581e1de5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 -- 172.21.15.86:0/3955912462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 msgr2=0x5581e1de2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1de2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 -- 172.21.15.86:0/3955912462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1ddd9d0 msgr2=0x5581e1de5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1ddd9d0 0x5581e1de5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 -- 172.21.15.86:0/3955912462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e1deb890 con 0x5581e1dde2f0 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fba077fe640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1de2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000a470 tx=0x7fb9f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b0082ab113835c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.618+0000 7fb9fd7fa640 1 -- 172.21.15.86:0/3955912462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0016070 con 0x5581e1dde2f0 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fb9fd7fa640 1 -- 172.21.15.86:0/3955912462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f0005020 con 0x5581e1dde2f0 2024-09-05T06:41:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fb9fd7fa640 1 -- 172.21.15.86:0/3955912462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004b50 con 0x5581e1dde2f0 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 -- 172.21.15.86:0/3955912462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 msgr2=0x5581e1de2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000a470 tx=0x7fb9f0012000 comp rx=0 tx=0).stop 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 -- 172.21.15.86:0/3955912462 shutdown_connections 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1de2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1dde2f0 0x5581e1de2a00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 --2- 172.21.15.86:0/3955912462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1ddd9d0 0x5581e1de5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 -- 172.21.15.86:0/3955912462 >> 172.21.15.86:0/3955912462 conn(0x5581e1dd9f60 msgr2=0x5581e1dda340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 -- 172.21.15.86:0/3955912462 shutdown_connections 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.619+0000 7fba0fcee740 1 -- 172.21.15.86:0/3955912462 wait complete. 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 -- start start 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 Processor -- start 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1dfe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1e5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581e1dfa980 con 0x5581e1ddd9d0 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581e1dfa2b0 con 0x5581e1dde2f0 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5581e1dfb050 con 0x5581e1de8bb0 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba0fcee740 10 client.0 _ll_register_callbacks cb 0x7fb9f80979c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48036/0 (socket says 172.21.15.86:48036) 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba06ffd640 1 -- 172.21.15.86:0/719776366 learned_addr learned my addr 172.21.15.86:0/719776366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1dfe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba077fe640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1e5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.620+0000 7fba06ffd640 1 -- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 msgr2=0x5581e1e5bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba06ffd640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1e5bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba06ffd640 1 -- 172.21.15.86:0/719776366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 msgr2=0x5581e1dfe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba06ffd640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1dfe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba06ffd640 1 -- 172.21.15.86:0/719776366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e1e5c470 con 0x5581e1ddd9d0 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba077fe640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1e5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1dfe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba06ffd640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8097b80 tx=0x7fb9f80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3140afcc56e0d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801d040 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb9f80cf230 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0005020 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9f809b080 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581e1e63800 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f0005020 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb9f8097470 con 0x5581e1ddd9d0 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 10 client.15612 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.621+0000 7fba0fcee740 20 client.15612 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1e68200 0x5581e1e66f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fba0fcee740 4 client.15612 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "status"}) -- 0x5581e1d4bb10 con 0x5581e1e68200 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1e68200 0x5581e1e66f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fb9e6ffd640 10 client.15612 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fb9fe7fc640 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1e68200 0x5581e1e66f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92c8c56aab10f264 server_cookie=2e0d2b048cd26f56 in_seq=0 out_seq=0 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fb9e6ffd640 1 -- 172.21.15.86:0/719776366 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5581e1d4bb10 con 0x5581e1e68200 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fb9e6ffd640 10 client.15612 handle_command_reply: tid=0 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.622+0000 7fba0fcee740 1 client.15612 shutdown 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 msgr2=0x5581e1dfe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8097b80 tx=0x7fb9f80076c0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 shutdown_connections 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5581e1e68200 0x5581e1e66f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5581e1de8bb0 0x5581e1e5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581e1dde2f0 0x5581e1dfe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fb9e6ffd640 0 client.15612 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:41:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 --2- 172.21.15.86:0/719776366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581e1ddd9d0 0x5581e1dfe410 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 >> 172.21.15.86:0/719776366 conn(0x5581e1dd9f60 msgr2=0x5581e1de4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 shutdown_connections 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 1 -- 172.21.15.86:0/719776366 wait complete. 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0fcee740 20 client.15612 trim_cache size 0 max 16384 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24466, 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:46.630 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 136.76335943500001, 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 189.912368214 2024-09-05T06:41:46.631 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 msgr2=0x7fb9f40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 0x7fb9f40a5240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba080ffe70 tx=0x7fb9f0005f70 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 msgr2=0x7fba081a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c970 tx=0x7fb9f800ce30 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 shutdown_connections 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb9f40a2d80 0x7fb9f40a5240 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba08105f40 0x7fba0819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba08101450 0x7fba081a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 --2- 172.21.15.86:0/3455394046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08100b30 0x7fba081a4630 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 >> 172.21.15.86:0/3455394046 conn(0x7fba080fc7a0 msgr2=0x7fba0810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 shutdown_connections 2024-09-05T06:41:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:46.623+0000 7fba0d915640 1 -- 172.21.15.86:0/3455394046 wait complete. 2024-09-05T06:41:47.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T06:41:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/354469799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc100a80 msgr2=0x7fbbcc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/354469799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc100a80 0x7fbbcc100ee0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009930 tx=0x7fbbbc02a420 comp rx=0 tx=0).stop 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/354469799 shutdown_connections 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/354469799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc101420 0x7fbbcc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/354469799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc100a80 0x7fbbcc100ee0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/354469799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc106a80 0x7fbbcc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/354469799 >> 172.21.15.86:0/354469799 conn(0x7fbbcc0fc7a0 msgr2=0x7fbbcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/354469799 shutdown_connections 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.223+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/354469799 wait complete. 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 Processor -- start 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 -- start start 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc100a80 0x7fbbcc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 0x7fbbcc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc110d30 con 0x7fbbcc106a80 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc110bb0 con 0x7fbbcc101420 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd3f0c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc110eb0 con 0x7fbbcc100a80 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48056/0 (socket says 172.21.15.86:48056) 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 -- 172.21.15.86:0/2465680038 learned_addr learned my addr 172.21.15.86:0/2465680038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 0x7fbbcc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc100a80 0x7fbbcc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 -- 172.21.15.86:0/2465680038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc100a80 msgr2=0x7fbbcc199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc100a80 0x7fbbcc199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 -- 172.21.15.86:0/2465680038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 msgr2=0x7fbbcc199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 0x7fbbcc199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd24a9640 1 -- 172.21.15.86:0/2465680038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc1a47c0 con 0x7fbbcc106a80 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.224+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 0x7fbbcc199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbd24a9640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400cad0 tx=0x7fbbc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbc2ffd640 1 -- 172.21.15.86:0/2465680038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400ae20 con 0x7fbbcc106a80 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc1a4990 con 0x7fbbcc106a80 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbc2ffd640 1 -- 172.21.15.86:0/2465680038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc4005020 con 0x7fbbcc106a80 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbc2ffd640 1 -- 172.21.15.86:0/2465680038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400f6b0 con 0x7fbbcc106a80 2024-09-05T06:41:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.225+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc1a4e50 con 0x7fbbcc106a80 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 Processor -- start 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbc2ffd640 1 -- 172.21.15.86:0/2465680038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fbbc40051d0 con 0x7fbbcc106a80 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 -- start start 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560076917090 con 0x5600769b12f0 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600768bea70 con 0x5600769b09d0 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560076913050 con 0x5600769bbbb0 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd1ca8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd1ca8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48064/0 (socket says 172.21.15.86:48064) 2024-09-05T06:41:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.228+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/3453954459 learned_addr learned my addr 172.21.15.86:0/3453954459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbc2ffd640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 0x7fbbb80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/3453954459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 msgr2=0x5600769bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/3453954459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 msgr2=0x5600769b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/3453954459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600768cb130 con 0x5600769b12f0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd24a9640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 0x7fbbb80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009c00 tx=0x7fbbc80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cf31bffc5b0438 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbc2ffd640 1 -- 172.21.15.86:0/2465680038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbbc40cfb20 con 0x7fbbcc106a80 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbb7fff640 1 -- 172.21.15.86:0/3453954459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8012450 con 0x5600769b12f0 2024-09-05T06:41:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbb7fff640 1 -- 172.21.15.86:0/3453954459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc8012aa0 con 0x5600769b12f0 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.229+0000 7fbbb7fff640 1 -- 172.21.15.86:0/3453954459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800fa40 con 0x5600769b12f0 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- 172.21.15.86:0/3453954459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 msgr2=0x5600769bb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd24a9640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 0x7fbbb80a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40cce30 tx=0x7fbbc4097e10 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009c00 tx=0x7fbbc80117b0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- 172.21.15.86:0/3453954459 shutdown_connections 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769bb670 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/3453954459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- 172.21.15.86:0/3453954459 >> 172.21.15.86:0/3453954459 conn(0x5600769acf60 msgr2=0x5600769ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- 172.21.15.86:0/3453954459 shutdown_connections 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- 172.21.15.86:0/3453954459 wait complete. 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.230+0000 7fbbd62e5740 1 -- start start 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 Processor -- start 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769b09d0 0x5600769bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600769be780 con 0x5600769bbbb0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600768d1f20 con 0x5600769b12f0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5600769be900 con 0x5600769b09d0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd62e5740 10 client.0 _ll_register_callbacks cb 0x56007689b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48068/0 (socket says 172.21.15.86:48068) 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 -- 172.21.15.86:0/1230187555 learned_addr learned my addr 172.21.15.86:0/1230187555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 -- 172.21.15.86:0/1230187555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769b09d0 msgr2=0x5600769bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769b09d0 0x5600769bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 -- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 msgr2=0x5600769b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd14a7640 1 -- 172.21.15.86:0/1230187555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600766882b0 con 0x5600769bbbb0 2024-09-05T06:41:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.231+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0096c0 tx=0x7fbbbc031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3552981cb0b5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc044070 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbbbc004450 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc8012aa0 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc050050 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 10 client.15624 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600769ca010 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc8012aa0 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbbbc03a040 con 0x5600769bbbb0 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 10 client.15624 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 20 client.15624 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600769cc600 0x5600769cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 4 client.15624 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.232+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560076924600 con 0x5600769cc600 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.233+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600769cc600 0x5600769cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.233+0000 7fbbb57fa640 10 client.15624 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.233+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600769cc600 0x5600769cda70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b99ca42ccf52ebe server_cookie=b9ad013a33a6160f in_seq=0 out_seq=0 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbb57fa640 1 -- 172.21.15.86:0/1230187555 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560076924600 con 0x5600769cc600 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbb57fa640 10 client.15624 handle_command_reply: tid=0 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 client.15624 shutdown 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 msgr2=0x5600769b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0096c0 tx=0x7fbbbc031740 comp rx=0 tx=0).stop 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 shutdown_connections 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600769cc600 0x5600769cda70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769bbbb0 0x5600769b8cf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/1230187555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769b09d0 0x5600769bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbb57fa640 0 client.15624 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 >> 172.21.15.86:0/1230187555 conn(0x5600769acf60 msgr2=0x5600769b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 shutdown_connections 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 1 -- 172.21.15.86:0/1230187555 wait complete. 2024-09-05T06:41:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.234+0000 7fbbd62e5740 20 client.15624 trim_cache size 0 max 16384 2024-09-05T06:41:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.252+0000 7fbbd62e5740 1 Processor -- start 2024-09-05T06:41:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.252+0000 7fbbd62e5740 1 -- start start 2024-09-05T06:41:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600769cbdb0 con 0x5600769b12f0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600769cc480 con 0x5600769b09d0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5600769ccb50 con 0x5600769bbbb0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48078/0 (socket says 172.21.15.86:48078) 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/2791818802 learned_addr learned my addr 172.21.15.86:0/2791818802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/2791818802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 msgr2=0x5600769d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/2791818802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 msgr2=0x5600769cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 -- 172.21.15.86:0/2791818802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600769be890 con 0x5600769b12f0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80099c0 tx=0x7fbbc800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e9bcd39f34887a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbb77fe640 1 -- 172.21.15.86:0/2791818802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800f4b0 con 0x5600769b12f0 2024-09-05T06:41:48.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbb77fe640 1 -- 172.21.15.86:0/2791818802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc80054f0 con 0x5600769b12f0 2024-09-05T06:41:48.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.253+0000 7fbbb77fe640 1 -- 172.21.15.86:0/2791818802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800fe80 con 0x5600769b12f0 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 -- 172.21.15.86:0/2791818802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 msgr2=0x5600769a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80099c0 tx=0x7fbbc800f720 comp rx=0 tx=0).stop 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 -- 172.21.15.86:0/2791818802 shutdown_connections 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b12f0 0x5600769a7ce0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/2791818802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b09d0 0x5600769cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 -- 172.21.15.86:0/2791818802 >> 172.21.15.86:0/2791818802 conn(0x5600769acf60 msgr2=0x5600769b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 -- 172.21.15.86:0/2791818802 shutdown_connections 2024-09-05T06:41:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.254+0000 7fbbd62e5740 1 -- 172.21.15.86:0/2791818802 wait complete. 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 -- start start 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 Processor -- start 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600769cd8f0 con 0x5600769b09d0 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600769cd220 con 0x5600769b12f0 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5600769cdfc0 con 0x5600769bbbb0 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd62e5740 10 client.0 _ll_register_callbacks cb 0x7fbbc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd14a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd14a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48088/0 (socket says 172.21.15.86:48088) 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.255+0000 7fbbd14a7640 1 -- 172.21.15.86:0/4046183538 learned_addr learned my addr 172.21.15.86:0/4046183538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 -- 172.21.15.86:0/4046183538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 msgr2=0x5600769d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 -- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 msgr2=0x5600769d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 -- 172.21.15.86:0/4046183538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600769d4570 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd14a7640 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001eb0 tx=0x7fbbbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87272152179693a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc059060 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc80054f0 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbbbc004250 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc060050 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560076a3acd0 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc80054f0 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbbbc0035f0 con 0x5600769b09d0 2024-09-05T06:41:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 10 client.15630 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.256+0000 7fbbd62e5740 20 client.15630 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560076a3f950 0x560076a3e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 4 client.15630 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "status"}) -- 0x56007691eb10 con 0x560076a3f950 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560076a3f950 0x560076a3e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd1ca8640 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560076a3f950 0x560076a3e660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0d8d6c7f4ccab6 server_cookie=75bcfd61be5a41 in_seq=0 out_seq=0 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbb57fa640 10 client.15630 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbb57fa640 1 -- 172.21.15.86:0/4046183538 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56007691eb10 con 0x560076a3f950 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbb57fa640 10 client.15630 handle_command_reply: tid=0 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 1 client.15630 shutdown 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 msgr2=0x5600769d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.257+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001eb0 tx=0x7fbbbc009780 comp rx=0 tx=0).stop 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 shutdown_connections 2024-09-05T06:41:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560076a3f950 0x560076a3e660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600769bbbb0 0x5600769d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600769b12f0 0x5600769d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbb57fa640 0 client.15630 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 --2- 172.21.15.86:0/4046183538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600769b09d0 0x5600769d6410 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 >> 172.21.15.86:0/4046183538 conn(0x5600769acf60 msgr2=0x5600769b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 shutdown_connections 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 1 -- 172.21.15.86:0/4046183538 wait complete. 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd62e5740 20 client.15630 trim_cache size 0 max 16384 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "5a055634-6b50-11ef-bcd6-c7b262605968", 2024-09-05T06:41:48.264 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24479, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 171.89253447900001, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 90, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 182.88243631700001 2024-09-05T06:41:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 msgr2=0x7fbbb80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 0x7fbbb80a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40cce30 tx=0x7fbbc4097e10 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 msgr2=0x7fbbcc1a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400cad0 tx=0x7fbbc400cf90 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 shutdown_connections 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbbb80a2d80 0x7fbbb80a5240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc106a80 0x7fbbcc1a4280 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbcc101420 0x7fbbcc199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 --2- 172.21.15.86:0/2465680038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbcc100a80 0x7fbbcc199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 >> 172.21.15.86:0/2465680038 conn(0x7fbbcc0fc7a0 msgr2=0x7fbbcc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 shutdown_connections 2024-09-05T06:41:48.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:48.258+0000 7fbbd3f0c640 1 -- 172.21.15.86:0/2465680038 wait complete. 2024-09-05T06:41:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:48 smithi115 ceph-mon[34791]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T06:41:48.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:48 smithi086 ceph-mon[30943]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T06:41:48.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:48 smithi096 ceph-mon[34468]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T06:41:48.963 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-05T06:41:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:50 smithi115 ceph-mon[34791]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:50.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:50 smithi086 ceph-mon[30943]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:50.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:50 smithi096 ceph-mon[34468]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-05T06:41:51.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:51 smithi115 ceph-mon[34791]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-05T06:41:51.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:51 smithi086 ceph-mon[30943]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-05T06:41:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:51 smithi096 ceph-mon[34468]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-05T06:41:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:54 smithi115 ceph-mon[34791]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:41:54.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:54 smithi086 ceph-mon[30943]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:41:54.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:54 smithi096 ceph-mon[34468]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T06:41:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:56 smithi115 ceph-mon[34791]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:41:56.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:56 smithi086 ceph-mon[30943]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:41:56.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:56 smithi096 ceph-mon[34468]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:41:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:41:57 smithi115 ceph-mon[34791]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-05T06:41:57.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:41:57 smithi086 ceph-mon[30943]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-05T06:41:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:41:57 smithi096 ceph-mon[34468]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-05T06:41:58.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:41:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:41:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 -- 172.21.15.86:0/1523342492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e01049a0 msgr2=0x7f53e01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 --2- 172.21.15.86:0/1523342492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e01049a0 0x7f53e01089f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009ca0 tx=0x7f53d802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 -- 172.21.15.86:0/1523342492 shutdown_connections 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 --2- 172.21.15.86:0/1523342492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e0109120 0x7f53e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 --2- 172.21.15.86:0/1523342492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e01049a0 0x7f53e01089f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 --2- 172.21.15.86:0/1523342492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 0x7f53e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 -- 172.21.15.86:0/1523342492 >> 172.21.15.86:0/1523342492 conn(0x7f53e00ffab0 msgr2=0x7f53e0101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 -- 172.21.15.86:0/1523342492 shutdown_connections 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.881+0000 7f53e823e640 1 -- 172.21.15.86:0/1523342492 wait complete. 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 Processor -- start 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 -- start start 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 0x7f53e019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 0x7f53e019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0113fd0 con 0x7f53e0109120 2024-09-05T06:41:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0113e50 con 0x7f53e01049a0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e823e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0114150 con 0x7f53e0103ff0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e67db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e67db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58872/0 (socket says 172.21.15.86:58872) 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e67db640 1 -- 172.21.15.86:0/363754281 learned_addr learned my addr 172.21.15.86:0/363754281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e57d9640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 0x7f53e019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.882+0000 7f53e5fda640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 0x7f53e019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 -- 172.21.15.86:0/363754281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 msgr2=0x7f53e019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 0x7f53e019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 -- 172.21.15.86:0/363754281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 msgr2=0x7f53e019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 0x7f53e019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 -- 172.21.15.86:0/363754281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e019f4e0 con 0x7f53e0109120 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e57d9640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 0x7f53e019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e67db640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d750 tx=0x7f53d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53d6ffd640 1 -- 172.21.15.86:0/363754281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0017070 con 0x7f53e0109120 2024-09-05T06:41:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e019f7d0 con 0x7f53e0109120 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e01a70b0 con 0x7f53e0109120 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53d6ffd640 1 -- 172.21.15.86:0/363754281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d00054d0 con 0x7f53e0109120 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.883+0000 7f53d6ffd640 1 -- 172.21.15.86:0/363754281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004b90 con 0x7f53e0109120 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.886+0000 7f53d6ffd640 1 -- 172.21.15.86:0/363754281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f53d0009bd0 con 0x7f53e0109120 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.886+0000 7f53ea617740 1 Processor -- start 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.886+0000 7f53ea617740 1 -- start start 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d62141c2e0 con 0x55d6214c1680 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6213c3ce0 con 0x55d6214b6dc0 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53ea617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6214182a0 con 0x55d6214b64a0 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58886/0 (socket says 172.21.15.86:58886) 2024-09-05T06:41:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 -- 172.21.15.86:0/2180812865 learned_addr learned my addr 172.21.15.86:0/2180812865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e5fda640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53d6ffd640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 0x7f53cc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e67db640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 0x7f53cc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 -- 172.21.15.86:0/2180812865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 msgr2=0x55d6214b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 -- 172.21.15.86:0/2180812865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 msgr2=0x55d6214c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214c1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 -- 172.21.15.86:0/2180812865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6213a8440 con 0x55d6214c1680 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e5fda640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.887+0000 7f53e57d9640 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009ca0 tx=0x7f53d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9cf67a23805c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53cbfff640 1 -- 172.21.15.86:0/2180812865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8031a40 con 0x55d6214c1680 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53d6ffd640 1 -- 172.21.15.86:0/363754281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f53d00cf340 con 0x7f53e0109120 2024-09-05T06:41:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53cbfff640 1 -- 172.21.15.86:0/2180812865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d8036040 con 0x55d6214c1680 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53cbfff640 1 -- 172.21.15.86:0/2180812865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8004330 con 0x55d6214c1680 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53e67db640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 0x7f53cc0a5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53d00cc650 tx=0x7f53d00976b0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53ea617740 1 -- 172.21.15.86:0/2180812865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 msgr2=0x55d6214c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.888+0000 7f53ea617740 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009ca0 tx=0x7f53d802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 -- 172.21.15.86:0/2180812865 shutdown_connections 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214c3a70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 --2- 172.21.15.86:0/2180812865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 -- 172.21.15.86:0/2180812865 >> 172.21.15.86:0/2180812865 conn(0x55d6214b2a30 msgr2=0x55d6214b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 -- 172.21.15.86:0/2180812865 shutdown_connections 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 -- 172.21.15.86:0/2180812865 wait complete. 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 -- start start 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.889+0000 7f53ea617740 1 Processor -- start 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6214c40d0 con 0x55d6214c1680 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6213d7050 con 0x55d6214b6dc0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6214c4250 con 0x55d6214b64a0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53ea617740 10 client.0 _ll_register_callbacks cb 0x55d621417160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214be6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58896/0 (socket says 172.21.15.86:58896) 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e57d9640 1 -- 172.21.15.86:0/4282485814 learned_addr learned my addr 172.21.15.86:0/4282485814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e57d9640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 -- 172.21.15.86:0/4282485814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 msgr2=0x55d6214c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 -- 172.21.15.86:0/4282485814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 msgr2=0x55d6214be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e5fda640 1 -- 172.21.15.86:0/4282485814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62118d3c0 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.890+0000 7f53e57d9640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53e5fda640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214be1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0126d0 tx=0x7f53dc012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f711d6e88c24e846 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004240 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d8036040 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f53dc0043e0 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53dc025050 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 10 client.25145 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6214cfb20 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d8036040 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f53dc00f460 con 0x55d6214b6dc0 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 10 client.25145 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 10 client.25145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 20 client.25145 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d6214d0fd0 0x55d6214d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 4 client.25145 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.891+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d621429850 con 0x55d6214d0fd0 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.892+0000 7f53e57d9640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d6214d0fd0 0x55d6214d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.892+0000 7f53e57d9640 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d6214d0fd0 0x55d6214d33f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bbec3310311d3c server_cookie=e856fb8babe0d948 in_seq=0 out_seq=0 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.892+0000 7f53c97fa640 10 client.25145 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53c97fa640 1 -- 172.21.15.86:0/4282485814 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d621429850 con 0x55d6214d0fd0 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53c97fa640 10 client.25145 handle_command_reply: tid=0 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 client.25145 shutdown 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 msgr2=0x55d6214be1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214be1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0126d0 tx=0x7f53dc012b90 comp rx=0 tx=0).stop 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 shutdown_connections 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d6214d0fd0 0x55d6214d33f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214c1680 0x55d6214be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214be1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53c97fa640 0 client.25145 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 --2- 172.21.15.86:0/4282485814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 >> 172.21.15.86:0/4282485814 conn(0x55d6214b2a30 msgr2=0x55d6214b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 shutdown_connections 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 1 -- 172.21.15.86:0/4282485814 wait complete. 2024-09-05T06:41:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.893+0000 7f53ea617740 20 client.25145 trim_cache size 0 max 16384 2024-09-05T06:41:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 Processor -- start 2024-09-05T06:41:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 -- start start 2024-09-05T06:41:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214b9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214c1680 0x55d6214bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4010010 con 0x55d6214b64a0 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4011b50 con 0x55d6214b6dc0 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53ea617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40106e0 con 0x55d6214c1680 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.910+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58910/0 (socket says 172.21.15.86:58910) 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 -- 172.21.15.86:0/1682087534 learned_addr learned my addr 172.21.15.86:0/1682087534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e5fda640 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214c1680 0x55d6214bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 -- 172.21.15.86:0/1682087534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214c1680 msgr2=0x55d6214bd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214c1680 0x55d6214bd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 -- 172.21.15.86:0/1682087534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 msgr2=0x55d6214b9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214b9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 -- 172.21.15.86:0/1682087534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6214d3a20 con 0x55d6214b64a0 2024-09-05T06:41:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009ca0 tx=0x7f53d8043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369b4cb14136f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53cb7fe640 1 -- 172.21.15.86:0/1682087534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8004b40 con 0x55d6214b64a0 2024-09-05T06:41:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53cb7fe640 1 -- 172.21.15.86:0/1682087534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d8036040 con 0x55d6214b64a0 2024-09-05T06:41:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.911+0000 7f53cb7fe640 1 -- 172.21.15.86:0/1682087534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d802f4c0 con 0x55d6214b64a0 2024-09-05T06:41:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- 172.21.15.86:0/1682087534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 msgr2=0x55d6214cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009ca0 tx=0x7f53d8043040 comp rx=0 tx=0).stop 2024-09-05T06:41:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- 172.21.15.86:0/1682087534 shutdown_connections 2024-09-05T06:41:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214c1680 0x55d6214bd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214b6dc0 0x55d6214b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 --2- 172.21.15.86:0/1682087534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b64a0 0x55d6214cd3c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- 172.21.15.86:0/1682087534 >> 172.21.15.86:0/1682087534 conn(0x55d6214b2a30 msgr2=0x55d6214bca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- 172.21.15.86:0/1682087534 shutdown_connections 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- 172.21.15.86:0/1682087534 wait complete. 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.912+0000 7f53ea617740 1 -- start start 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 Processor -- start 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 0x55d6214c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4011480 con 0x55d6214b6dc0 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4010db0 con 0x55d6214c1680 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53c400dff0 con 0x55d6214b64a0 2024-09-05T06:41:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53ea617740 10 client.0 _ll_register_callbacks cb 0x7f53dc009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58920/0 (socket says 172.21.15.86:58920) 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 -- 172.21.15.86:0/1239330569 learned_addr learned my addr 172.21.15.86:0/1239330569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 0x55d6214c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 -- 172.21.15.86:0/1239330569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 msgr2=0x55d6214c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 -- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 msgr2=0x55d6214c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 0x55d6214c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e5fda640 1 -- 172.21.15.86:0/1239330569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6214b2860 con 0x55d6214b6dc0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.913+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 0x55d6214c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53e5fda640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55d6214bda40 tx=0x7f53dc0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe055c662bfbe01 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:41:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00fcb0 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f53dc0119a0 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d8036040 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53dc03c050 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53ea617740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d62153bac0 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53d8036040 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.914+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f53dc028040 con 0x55d6214b6dc0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 20 client.15648 populate_metadata read hostname 'smithi086' 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d62153c240 0x55d62153e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 4 client.15648 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6214bea80 con 0x55d62153c240 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d62153c240 0x55d62153e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53e57d9640 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d62153c240 0x55d62153e660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8f0182651f04cd server_cookie=c6f64cab0c710701 in_seq=0 out_seq=0 2024-09-05T06:41:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53c97fa640 10 client.15648 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53c97fa640 1 -- 172.21.15.86:0/1239330569 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d6214bea80 con 0x55d62153c240 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53c97fa640 10 client.15648 handle_command_reply: tid=0 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.915+0000 7f53ea617740 1 client.15648 shutdown 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 msgr2=0x55d6214c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55d6214bda40 tx=0x7f53dc0054e0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 shutdown_connections 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d62153c240 0x55d62153e660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6214c1680 0x55d6214c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6214b6dc0 0x55d6214c5000 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53c97fa640 0 client.15648 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 --2- 172.21.15.86:0/1239330569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6214b64a0 0x55d6214c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 >> 172.21.15.86:0/1239330569 conn(0x55d6214b2a30 msgr2=0x55d6214b97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 shutdown_connections 2024-09-05T06:41:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 1 -- 172.21.15.86:0/1239330569 wait complete. 2024-09-05T06:41:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53ea617740 20 client.15648 trim_cache size 0 max 16384 2024-09-05T06:41:58.921 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:41:58.921 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:41:58.921 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:41:58.921 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 msgr2=0x7f53cc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 0x7f53cc0a5240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53d00cc650 tx=0x7f53d00976b0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 msgr2=0x7f53e019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.916+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53d000d750 tx=0x7f53d000dc10 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 shutdown_connections 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f53cc0a2d80 0x7f53cc0a5240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e0109120 0x7f53e019ee00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53e01049a0 0x7f53e019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 --2- 172.21.15.86:0/363754281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53e0103ff0 0x7f53e019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 >> 172.21.15.86:0/363754281 conn(0x7f53e00ffab0 msgr2=0x7f53e0100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 shutdown_connections 2024-09-05T06:41:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:41:58.917+0000 7f53e823e640 1 -- 172.21.15.86:0/363754281 wait complete. 2024-09-05T06:41:59.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:41:59.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:42:00.038 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 -- 172.21.15.86:0/3843583434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4101df0 msgr2=0x7f5de41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3843583434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4101df0 0x7f5de41021d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009900 tx=0x7f5dd402a410 comp rx=0 tx=0).stop 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 -- 172.21.15.86:0/3843583434 shutdown_connections 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3843583434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de4106f20 0x7f5de410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3843583434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de41027a0 0x7f5de41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3843583434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4101df0 0x7f5de41021d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 -- 172.21.15.86:0/3843583434 >> 172.21.15.86:0/3843583434 conn(0x7f5de40fd8f0 msgr2=0x7f5de40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 -- 172.21.15.86:0/3843583434 shutdown_connections 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 -- 172.21.15.86:0/3843583434 wait complete. 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.541+0000 7f5de99c3640 1 Processor -- start 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 -- start start 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de4101df0 0x7f5de40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de41027a0 0x7f5de4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4111ea0 con 0x7f5de4106f20 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4111d20 con 0x7f5de4101df0 2024-09-05T06:42:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de99c3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4112020 con 0x7f5de41027a0 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58942/0 (socket says 172.21.15.86:58942) 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 -- 172.21.15.86:0/3788690013 learned_addr learned my addr 172.21.15.86:0/3788690013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de2ffd640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de4101df0 0x7f5de40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de27fc640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de41027a0 0x7f5de4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 -- 172.21.15.86:0/3788690013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de41027a0 msgr2=0x7f5de4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de41027a0 0x7f5de4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 -- 172.21.15.86:0/3788690013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de4101df0 msgr2=0x7f5de40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de4101df0 0x7f5de40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.542+0000 7f5de37fe640 1 -- 172.21.15.86:0/3788690013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41a6860 con 0x7f5de4106f20 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de37fe640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00b970 tx=0x7f5ddc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc019070 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41a6b50 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc014440 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0075d0 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.543+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41a7040 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.544+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de41021d0 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.545+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f5ddc01c070 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.546+0000 7f5de89c1640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 0x7f5dcc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.546+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5ddc0cee90 con 0x7f5de4106f20 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.546+0000 7f5de2ffd640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 0x7f5dcc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.546+0000 7f5de2ffd640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 0x7f5dcc0a5240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005ab0 tx=0x7f5dd4005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.550+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ddc09b020 con 0x7f5de4106f20 2024-09-05T06:42:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:00 smithi115 ceph-mon[34791]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:42:00.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-mon[30943]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:42:00.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:42:00.603+0000 7f8fe4660640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:42:00.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:42:00.603+0000 7f8fe4660640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:42:00.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:42:00.603+0000 7f8fe4660640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:42:00.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:42:00.603+0000 7f8fe4660640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:42:00.737 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 06:42:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T06:42:00.603+0000 7f8fe4660640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T06:42:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.785+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de402d7c0 con 0x7f5de4106f20 2024-09-05T06:42:00.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.786+0000 7f5de89c1640 1 -- 172.21.15.86:0/3788690013 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f5ddc09f320 con 0x7f5de4106f20 2024-09-05T06:42:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:42:00.786 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:42:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:42:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 msgr2=0x7f5dcc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 0x7f5dcc0a5240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005ab0 tx=0x7f5dd4005a20 comp rx=0 tx=0).stop 2024-09-05T06:42:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 msgr2=0x7f5de4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00b970 tx=0x7f5ddc00be30 comp rx=0 tx=0).stop 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 shutdown_connections 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dcc0a2d80 0x7f5dcc0a5240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4106f20 0x7f5de4074f70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de41027a0 0x7f5de4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 --2- 172.21.15.86:0/3788690013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5de4101df0 0x7f5de40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 >> 172.21.15.86:0/3788690013 conn(0x7f5de40fd8f0 msgr2=0x7f5de40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.788+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 shutdown_connections 2024-09-05T06:42:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:00.789+0000 7f5de99c3640 1 -- 172.21.15.86:0/3788690013 wait complete. 2024-09-05T06:42:00.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:00 smithi096 ceph-mon[34468]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-05T06:42:01.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:42:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:01 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3788690013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:01.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:01 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3788690013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:01 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3788690013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:02.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:02 smithi115 ceph-mon[34791]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:02.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:02 smithi115 ceph-mon[34791]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-05T06:42:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:02 smithi086 ceph-mon[30943]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:02.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:02 smithi086 ceph-mon[30943]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-05T06:42:02.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:02 smithi096 ceph-mon[34468]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:02.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:02 smithi096 ceph-mon[34468]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-05T06:42:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:03 smithi115 ceph-mon[34791]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:03.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:03 smithi086 ceph-mon[30943]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:03.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:03 smithi096 ceph-mon[34468]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:03.964 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.2 2024-09-05T06:42:03.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:42:04.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1153078973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 msgr2=0x7f2dd4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1153078973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd4106e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009930 tx=0x7f2dd002a420 comp rx=0 tx=0).stop 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1153078973 shutdown_connections 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1153078973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4101410 0x7f2dd410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1153078973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4100a70 0x7f2dd4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1153078973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd4106e50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1153078973 >> 172.21.15.86:0/1153078973 conn(0x7f2dd40fc790 msgr2=0x7f2dd40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1153078973 shutdown_connections 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.862+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1153078973 wait complete. 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 Processor -- start 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 -- start start 2024-09-05T06:42:04.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 0x7f2dd41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 0x7f2dd4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4110d60 con 0x7f2dd4106a70 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4110be0 con 0x7f2dd4101410 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2ddbe1e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4110ee0 con 0x7f2dd4100a70 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2dda3bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2dda3bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43718/0 (socket says 172.21.15.86:43718) 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.863+0000 7f2dda3bb640 1 -- 172.21.15.86:0/1381190735 learned_addr learned my addr 172.21.15.86:0/1381190735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 0x7f2dd41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 0x7f2dd4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 -- 172.21.15.86:0/1381190735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 msgr2=0x7f2dd41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 0x7f2dd41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 -- 172.21.15.86:0/1381190735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 msgr2=0x7f2dd4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 0x7f2dd4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 -- 172.21.15.86:0/1381190735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd419d3a0 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 0x7f2dd4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 0x7f2dd41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dda3bb640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00eaa0 tx=0x7f2dcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dcaffd640 1 -- 172.21.15.86:0/1381190735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00cdb0 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dcaffd640 1 -- 172.21.15.86:0/1381190735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dcc005020 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2dcaffd640 1 -- 172.21.15.86:0/1381190735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0116d0 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd419d690 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.864+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a4f70 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.867+0000 7f2dde1f7740 1 Processor -- start 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.867+0000 7f2dcaffd640 1 -- 172.21.15.86:0/1381190735 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f2dcc0051d0 con 0x7f2dd4106a70 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 -- start start 2024-09-05T06:42:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471242db40 0x56471242c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 0x56471243db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647123922f0 con 0x56471243b780 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564712339cf0 con 0x56471242c5d0 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56471238e2b0 con 0x56471242db40 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd93b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 0x56471243db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34784/0 (socket says 172.21.15.86:34784) 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 -- 172.21.15.86:0/3590594598 learned_addr learned my addr 172.21.15.86:0/3590594598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471242db40 0x56471242c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dcaffd640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 0x7f2dc00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 -- 172.21.15.86:0/3590594598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471242db40 msgr2=0x56471242c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471242db40 0x56471242c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.868+0000 7f2dd9bba640 1 -- 172.21.15.86:0/3590594598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 msgr2=0x56471243db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 0x56471243db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dd9bba640 1 -- 172.21.15.86:0/3590594598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471231e3a0 con 0x56471242c5d0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dda3bb640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 0x7f2dc00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dcaffd640 1 -- 172.21.15.86:0/1381190735 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dcc0cfcd0 con 0x7f2dd4106a70 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 0x56471243db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0031440 tx=0x7f2dd0032e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83041ce879fda6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dda3bb640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 0x7f2dc00a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0ccf20 tx=0x7f2dcc0cffd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dbffff640 1 -- 172.21.15.86:0/3590594598 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd003f070 con 0x56471242c5d0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dbffff640 1 -- 172.21.15.86:0/3590594598 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd0031760 con 0x56471242c5d0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.869+0000 7f2dbffff640 1 -- 172.21.15.86:0/3590594598 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0035070 con 0x56471242c5d0 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3590594598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 msgr2=0x56471242ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0031440 tx=0x7f2dd0032e70 comp rx=0 tx=0).stop 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3590594598 shutdown_connections 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471243b780 0x56471243db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x56471242ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3590594598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471242db40 0x56471242c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3590594598 >> 172.21.15.86:0/3590594598 conn(0x5647123f5310 msgr2=0x564712428c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3590594598 shutdown_connections 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3590594598 wait complete. 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 -- start start 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.870+0000 7f2dde1f7740 1 Processor -- start 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x564712438440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712438a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56471243e420 con 0x56471242c5d0 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56471234d060 con 0x56471242db40 2024-09-05T06:42:04.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56471243e5a0 con 0x56471243b780 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dde1f7740 10 client.0 _ll_register_callbacks cb 0x5647120a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd9bba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x564712438440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43730/0 (socket says 172.21.15.86:43730) 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 -- 172.21.15.86:0/3826552288 learned_addr learned my addr 172.21.15.86:0/3826552288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712438a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 -- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 msgr2=0x564712438a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712438a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 -- 172.21.15.86:0/3826552288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 msgr2=0x564712438440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x564712438440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd93b9640 1 -- 172.21.15.86:0/3826552288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647121033c0 con 0x56471242c5d0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.871+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x564712438440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400ce30 tx=0x7f2dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aa738490659f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc401c070 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dc4014c80 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd0031760 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dc4028050 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564712449fa0 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd0031760 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2dc4015040 con 0x56471242c5d0 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 10 client.15672 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 20 client.15672 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.872+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56471244b3b0 0x56471244d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.873+0000 7f2dde1f7740 4 client.15672 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:04.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.873+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471239f860 con 0x56471244b3b0 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.873+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56471244b3b0 0x56471244d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.873+0000 7f2dbd7fa640 10 client.15672 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.873+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56471244b3b0 0x56471244d7c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e32631263434621e server_cookie=a0defd159bf9daeb in_seq=0 out_seq=0 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/3826552288 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56471239f860 con 0x56471244b3b0 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dbd7fa640 10 client.15672 handle_command_reply: tid=0 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 client.15672 shutdown 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 msgr2=0x56471243b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400ce30 tx=0x7f2dc40046c0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 shutdown_connections 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56471244b3b0 0x56471244d7c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712438a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x564712438440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dbd7fa640 0 client.15672 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/3826552288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x56471243b190 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 >> 172.21.15.86:0/3826552288 conn(0x5647123f5310 msgr2=0x56471242e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 shutdown_connections 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 1 -- 172.21.15.86:0/3826552288 wait complete. 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.874+0000 7f2dde1f7740 20 client.15672 trim_cache size 0 max 16384 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 Processor -- start 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 -- start start 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x5647124476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712453090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41acdf0 con 0x56471242c5d0 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41ae930 con 0x56471242db40 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.887+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41ad4c0 con 0x56471243b780 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43744/0 (socket says 172.21.15.86:43744) 2024-09-05T06:42:04.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 -- 172.21.15.86:0/2584637475 learned_addr learned my addr 172.21.15.86:0/2584637475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712453090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x5647124476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 -- 172.21.15.86:0/2584637475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 msgr2=0x564712453090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712453090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 -- 172.21.15.86:0/2584637475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 msgr2=0x5647124476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x5647124476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 -- 172.21.15.86:0/2584637475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647124393e0 con 0x56471242c5d0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x5647124476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4017ac0 tx=0x7f2dc400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169665567d54e753 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dbf7fe640 1 -- 172.21.15.86:0/2584637475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc401c070 con 0x56471242c5d0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dbf7fe640 1 -- 172.21.15.86:0/2584637475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc400a070 con 0x56471242c5d0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.888+0000 7f2dbf7fe640 1 -- 172.21.15.86:0/2584637475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400f040 con 0x56471242c5d0 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- 172.21.15.86:0/2584637475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 msgr2=0x564712428e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4017ac0 tx=0x7f2dc400c6a0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- 172.21.15.86:0/2584637475 shutdown_connections 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712453090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242db40 0x5647124476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/2584637475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242c5d0 0x564712428e10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- 172.21.15.86:0/2584637475 >> 172.21.15.86:0/2584637475 conn(0x5647123f5310 msgr2=0x564712428700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- 172.21.15.86:0/2584637475 shutdown_connections 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- 172.21.15.86:0/2584637475 wait complete. 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 -- start start 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.889+0000 7f2dde1f7740 1 Processor -- start 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x5647124355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712436070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41ae260 con 0x56471242db40 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41adb90 con 0x56471242c5d0 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41aadd0 con 0x56471243b780 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dde1f7740 10 client.0 _ll_register_callbacks cb 0x7f2dc400c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43748/0 (socket says 172.21.15.86:43748) 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 -- 172.21.15.86:0/1156084789 learned_addr learned my addr 172.21.15.86:0/1156084789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x5647124355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 -- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 msgr2=0x564712436070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712436070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712436070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 -- 172.21.15.86:0/1156084789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 msgr2=0x5647124355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x5647124355f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 -- 172.21.15.86:0/1156084789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647124284f0 con 0x56471242db40 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712436070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dd9bba640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0031440 tx=0x7f2dd0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5676f9c56035a2e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.890+0000 7f2dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd003f040 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dd0031680 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc400a070 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dd005b050 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647124b6300 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc400a070 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2dd00038e0 con 0x56471242db40 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 10 client.15678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 20 client.15678 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5647124b6640 0x5647124b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 4 client.15678 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.891+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564712399d70 con 0x5647124b6640 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5647124b6640 0x5647124b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dbd7fa640 10 client.15678 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dd93b9640 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5647124b6640 0x5647124b8a30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20c75dc9fea01f50 server_cookie=a5c033e3441edfd3 in_seq=0 out_seq=0 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dbd7fa640 1 -- 172.21.15.86:0/1156084789 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564712399d70 con 0x5647124b6640 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dbd7fa640 10 client.15678 handle_command_reply: tid=0 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 client.15678 shutdown 2024-09-05T06:42:04.732 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 msgr2=0x564712435b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0031440 tx=0x7f2dd0005a20 comp rx=0 tx=0).stop 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 shutdown_connections 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5647124b6640 0x5647124b8a30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56471243b780 0x564712436070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56471242db40 0x564712435b30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 --2- 172.21.15.86:0/1156084789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56471242c5d0 0x5647124355f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dbd7fa640 0 client.15678 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.892+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 >> 172.21.15.86:0/1156084789 conn(0x5647123f5310 msgr2=0x5647124274e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 shutdown_connections 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2dde1f7740 1 -- 172.21.15.86:0/1156084789 wait complete. 2024-09-05T06:42:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2dde1f7740 20 client.15678 trim_cache size 0 max 16384 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 msgr2=0x7f2dc00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 0x7f2dc00a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0ccf20 tx=0x7f2dcc0cffd0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 msgr2=0x7f2dd419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00eaa0 tx=0x7f2dcc00ef60 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 shutdown_connections 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc00a2e90 0x7f2dc00a5350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd4106a70 0x7f2dd419ccc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd4101410 0x7f2dd4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 --2- 172.21.15.86:0/1381190735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd4100a70 0x7f2dd41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 >> 172.21.15.86:0/1381190735 conn(0x7f2dd40fc790 msgr2=0x7f2dd41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 shutdown_connections 2024-09-05T06:42:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:03.893+0000 7f2ddbe1e640 1 -- 172.21.15.86:0/1381190735 wait complete. 2024-09-05T06:42:06.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:42:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:06.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:06 smithi086 ceph-mon[30943]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:06 smithi115 ceph-mon[34791]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:06 smithi096 ceph-mon[34468]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-05T06:42:06.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 -- 172.21.15.86:0/1757716036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a41082e0 msgr2=0x7f13a410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/1757716036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a41082e0 0x7f13a410ec50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f139c009cd0 tx=0x7f139c02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 -- 172.21.15.86:0/1757716036 shutdown_connections 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/1757716036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a41082e0 0x7f13a410ec50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/1757716036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a4103760 0x7f13a4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/1757716036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4102db0 0x7f13a4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.975+0000 7f13a8a1f640 1 -- 172.21.15.86:0/1757716036 >> 172.21.15.86:0/1757716036 conn(0x7f13a40fc7d0 msgr2=0x7f13a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.976+0000 7f13a8a1f640 1 -- 172.21.15.86:0/1757716036 shutdown_connections 2024-09-05T06:42:06.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.976+0000 7f13a8a1f640 1 -- 172.21.15.86:0/1757716036 wait complete. 2024-09-05T06:42:06.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 Processor -- start 2024-09-05T06:42:06.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 -- start start 2024-09-05T06:42:06.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:06.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4103760 0x7f13a419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 0x7f13a419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4101590 con 0x7f13a4102db0 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4101410 con 0x7f13a4103760 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a8a1f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4101710 con 0x7f13a41082e0 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43760/0 (socket says 172.21.15.86:43760) 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a259c640 1 -- 172.21.15.86:0/2428121349 learned_addr learned my addr 172.21.15.86:0/2428121349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:06.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.977+0000 7f13a1d9b640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4103760 0x7f13a419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a2d9d640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 0x7f13a419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 -- 172.21.15.86:0/2428121349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 msgr2=0x7f13a419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 0x7f13a419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 -- 172.21.15.86:0/2428121349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4103760 msgr2=0x7f13a419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4103760 0x7f13a419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 -- 172.21.15.86:0/2428121349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a419f6f0 con 0x7f13a4102db0 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a2d9d640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 0x7f13a419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:06.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13a259c640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f139400d950 tx=0x7f139400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394017030 con 0x7f13a4102db0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.978+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1394004020 con 0x7f13a4102db0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.979+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a419f9e0 con 0x7f13a4102db0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.979+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41a72c0 con 0x7f13a4102db0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.979+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394005570 con 0x7f13a4102db0 2024-09-05T06:42:06.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.980+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4102660 con 0x7f13a4102db0 2024-09-05T06:42:06.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.981+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f13940041d0 con 0x7f13a4102db0 2024-09-05T06:42:06.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.982+0000 7f13937fe640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 0x7f138c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:06.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.982+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f13940cfb10 con 0x7f13a4102db0 2024-09-05T06:42:06.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.982+0000 7f13a1d9b640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 0x7f138c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:06.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.983+0000 7f13a1d9b640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 0x7f138c0a52d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13a4107e30 tx=0x7f139800a2d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:06.989+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1394024330 con 0x7f13a4102db0 2024-09-05T06:42:07.036 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:07.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.239+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13a41a74f0 con 0x7f13a4102db0 2024-09-05T06:42:07.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.240+0000 7f13937fe640 1 -- 172.21.15.86:0/2428121349 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f139409c100 con 0x7f13a4102db0 2024-09-05T06:42:07.240 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:42:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:42:07.240 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:42:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 msgr2=0x7f138c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 0x7f138c0a52d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13a4107e30 tx=0x7f139800a2d0 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 msgr2=0x7f13a419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f139400d950 tx=0x7f139400dd50 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 shutdown_connections 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f138c0a2e10 0x7f138c0a52d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a41082e0 0x7f13a419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4103760 0x7f13a419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 --2- 172.21.15.86:0/2428121349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4102db0 0x7f13a419a740 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 >> 172.21.15.86:0/2428121349 conn(0x7f13a40fc7d0 msgr2=0x7f13a4107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 shutdown_connections 2024-09-05T06:42:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:07.243+0000 7f13a8a1f640 1 -- 172.21.15.86:0/2428121349 wait complete. 2024-09-05T06:42:07.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:07 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2428121349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:07.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:07 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2428121349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:07 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2428121349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:08.055 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0' 2024-09-05T06:42:08.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0 2024-09-05T06:42:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 -- 172.21.15.86:0/1947800679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102570 msgr2=0x7f1ccc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/1947800679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102570 0x7f1ccc1029b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009900 tx=0x7f1cb402a480 comp rx=0 tx=0).stop 2024-09-05T06:42:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 -- 172.21.15.86:0/1947800679 shutdown_connections 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/1947800679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/1947800679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102570 0x7f1ccc1029b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/1947800679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 0x7f1ccc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.318+0000 7f1cd1dea640 1 -- 172.21.15.86:0/1947800679 >> 172.21.15.86:0/1947800679 conn(0x7f1ccc0fc7d0 msgr2=0x7f1ccc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 -- 172.21.15.86:0/1947800679 shutdown_connections 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 -- 172.21.15.86:0/1947800679 wait complete. 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 Processor -- start 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 -- start start 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 0x7f1ccc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102570 0x7f1ccc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.319+0000 7f1cd1dea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc110e50 con 0x7f1ccc102ef0 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cd1dea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc110cd0 con 0x7f1ccc102570 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cd1dea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc110fd0 con 0x7f1ccc100e20 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43768/0 (socket says 172.21.15.86:43768) 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 -- 172.21.15.86:0/201579058 learned_addr learned my addr 172.21.15.86:0/201579058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102570 0x7f1ccc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 0x7f1ccc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 -- 172.21.15.86:0/201579058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 msgr2=0x7f1ccc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 0x7f1ccc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 -- 172.21.15.86:0/201579058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102570 msgr2=0x7f1ccc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102570 0x7f1ccc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 -- 172.21.15.86:0/201579058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc19d2d0 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c970 tx=0x7f1cc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cc97fa640 1 -- 172.21.15.86:0/201579058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4007bb0 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc19d5c0 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cc97fa640 1 -- 172.21.15.86:0/201579058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc4007d50 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.320+0000 7f1cc97fa640 1 -- 172.21.15.86:0/201579058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4018700 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.321+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a4ea0 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.323+0000 7f1cc97fa640 1 -- 172.21.15.86:0/201579058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f1cc400fa20 con 0x7f1ccc102ef0 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 Processor -- start 2024-09-05T06:42:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cc97fa640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 0x7f1cb80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 -- start start 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 0x7f1cb80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 0x558c2d991a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d99c370 0x558c2d99e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c2d8f72f0 con 0x558c2d992b40 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c2d89ecf0 con 0x558c2d99c370 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c2d8f32b0 con 0x558c2d9915d0 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43774/0 (socket says 172.21.15.86:43774) 2024-09-05T06:42:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/1678991443 learned_addr learned my addr 172.21.15.86:0/1678991443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cc97fa640 1 -- 172.21.15.86:0/201579058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f1cc40cfb80 con 0x7f1ccc102ef0 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 0x558c2d991a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/1678991443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 msgr2=0x558c2d991a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 0x558c2d991a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/1678991443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d99c370 msgr2=0x558c2d99e760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d99c370 0x558c2d99e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.324+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/1678991443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2d8833a0 con 0x558c2d992b40 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 0x558c2d991a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 0x7f1cb80a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40042c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc007f90 tx=0x7f1cbc00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951bcb0952f0f1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1cc1ffb640 1 -- 172.21.15.86:0/1678991443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004d20 con 0x558c2d992b40 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1cc1ffb640 1 -- 172.21.15.86:0/1678991443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cbc010bb0 con 0x558c2d992b40 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.325+0000 7f1cc1ffb640 1 -- 172.21.15.86:0/1678991443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc005720 con 0x558c2d992b40 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1678991443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 msgr2=0x558c2d991090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc007f90 tx=0x7f1cbc00f850 comp rx=0 tx=0).stop 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1678991443 shutdown_connections 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d99c370 0x558c2d99e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d9915d0 0x558c2d991a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1678991443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d991090 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1678991443 >> 172.21.15.86:0/1678991443 conn(0x558c2d95a310 msgr2=0x558c2d98dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1678991443 shutdown_connections 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1678991443 wait complete. 2024-09-05T06:42:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 -- start start 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.326+0000 7f1cd41c3740 1 Processor -- start 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d999060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c2d99f070 con 0x558c2d9915d0 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c2d8b2060 con 0x558c2d992b40 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c2d99f1f0 con 0x558c2d99c370 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cd41c3740 10 client.0 _ll_register_callbacks cb 0x558c2d6058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43788/0 (socket says 172.21.15.86:43788) 2024-09-05T06:42:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/2016616483 learned_addr learned my addr 172.21.15.86:0/2016616483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d999060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/2016616483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 msgr2=0x558c2d9997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 msgr2=0x558c2d999060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d999060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/2016616483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2d6683c0 con 0x558c2d9915d0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00fd50 tx=0x7f1cbc005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119167affc175c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.327+0000 7f1caaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc026020 con 0x558c2d9915d0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f1cbc005850 con 0x558c2d9915d0 2024-09-05T06:42:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc010bb0 con 0x558c2d9915d0 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc031050 con 0x558c2d9915d0 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c2d9aeee0 con 0x558c2d9915d0 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cbc010bb0 con 0x558c2d9915d0 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1cbc022840 con 0x558c2d9915d0 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 10 client.15696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 20 client.15696 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2d9b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 4 client.15696 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.328+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c2d904860 con 0x558c2d9b02e0 2024-09-05T06:42:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.329+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2d9b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.329+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2d9b26d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89230f4094f2c25b server_cookie=272a035c51ddb810 in_seq=0 out_seq=0 2024-09-05T06:42:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.329+0000 7f1caaffd640 10 client.15696 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1caaffd640 1 -- 172.21.15.86:0/2016616483 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c2d904860 con 0x558c2d9b02e0 2024-09-05T06:42:08.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1caaffd640 10 client.15696 handle_command_reply: tid=0 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 client.15696 shutdown 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 msgr2=0x558c2d99bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00fd50 tx=0x7f1cbc005420 comp rx=0 tx=0).stop 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 shutdown_connections 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2d9b26d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1caaffd640 0 client.15696 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d999060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/2016616483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d99bdb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 >> 172.21.15.86:0/2016616483 conn(0x558c2d95a310 msgr2=0x558c2d993960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 shutdown_connections 2024-09-05T06:42:08.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.330+0000 7f1cd41c3740 1 -- 172.21.15.86:0/2016616483 wait complete. 2024-09-05T06:42:08.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.331+0000 7f1cd41c3740 20 client.15696 trim_cache size 0 max 16384 2024-09-05T06:42:08.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 Processor -- start 2024-09-05T06:42:08.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 -- start start 2024-09-05T06:42:08.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d9915d0 0x558c2d994120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1acd10 con 0x558c2d992b40 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1ae850 con 0x558c2d9915d0 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.344+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1ad3e0 con 0x558c2d99c370 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43794/0 (socket says 172.21.15.86:43794) 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 -- 172.21.15.86:0/1723474259 learned_addr learned my addr 172.21.15.86:0/1723474259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d9915d0 0x558c2d994120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 -- 172.21.15.86:0/1723474259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 msgr2=0x558c2d9b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 -- 172.21.15.86:0/1723474259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d9915d0 msgr2=0x558c2d994120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d9915d0 0x558c2d994120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 -- 172.21.15.86:0/1723474259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2d992760 con 0x558c2d992b40 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009230 tx=0x7f1cb403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693f7b24e85f3be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc17fa640 1 -- 172.21.15.86:0/1723474259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4035040 con 0x558c2d992b40 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc17fa640 1 -- 172.21.15.86:0/1723474259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cb403c9e0 con 0x558c2d992b40 2024-09-05T06:42:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.345+0000 7f1cc17fa640 1 -- 172.21.15.86:0/1723474259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40528a0 con 0x558c2d992b40 2024-09-05T06:42:08.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1723474259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 msgr2=0x558c2d994660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009230 tx=0x7f1cb403f000 comp rx=0 tx=0).stop 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1723474259 shutdown_connections 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d9b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d992b40 0x558c2d994660 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/1723474259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d9915d0 0x558c2d994120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1723474259 >> 172.21.15.86:0/1723474259 conn(0x558c2d95a310 msgr2=0x558c2d998150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1723474259 shutdown_connections 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.346+0000 7f1cd41c3740 1 -- 172.21.15.86:0/1723474259 wait complete. 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 -- start start 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 Processor -- start 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d995d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d996270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1ae180 con 0x558c2d9915d0 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1adab0 con 0x558c2d992b40 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1aacf0 con 0x558c2d99c370 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1cd41c3740 10 client.0 _ll_register_callbacks cb 0x7f1cbc007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43804/0 (socket says 172.21.15.86:43804) 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/403870032 learned_addr learned my addr 172.21.15.86:0/403870032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.347+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d996270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d995d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/403870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 msgr2=0x558c2d996270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d996270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 msgr2=0x558c2d995d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d995d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 -- 172.21.15.86:0/403870032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2d996a70 con 0x558c2d9915d0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccbfff640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d996270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d995d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1ccb7fe640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc007760 tx=0x7f1cbc018630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4e2069692d22bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1caaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc039020 con 0x558c2d9915d0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f1cbc005850 con 0x558c2d9915d0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.348+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb403c9e0 con 0x558c2d9915d0 2024-09-05T06:42:08.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc044050 con 0x558c2d9915d0 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 10 client.15702 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c2da1ac40 con 0x558c2d9915d0 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cb403c9e0 con 0x558c2d9915d0 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1cbc018720 con 0x558c2d9915d0 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 10 client.15702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 20 client.15702 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2da1d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 4 client.15702 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.349+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c2d8fed70 con 0x558c2d9b02e0 2024-09-05T06:42:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2da1d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1caaffd640 10 client.15702 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cc2ffd640 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2da1d420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c65111b54f82193b server_cookie=11f20b4e0bc0829b in_seq=0 out_seq=0 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1caaffd640 1 -- 172.21.15.86:0/403870032 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c2d8fed70 con 0x558c2d9b02e0 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1caaffd640 10 client.15702 handle_command_reply: tid=0 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 client.15702 shutdown 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 msgr2=0x558c2d9a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc007760 tx=0x7f1cbc018630 comp rx=0 tx=0).stop 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 shutdown_connections 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x558c2d9b02e0 0x558c2da1d420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c2d99c370 0x558c2d996270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c2d992b40 0x558c2d995d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 --2- 172.21.15.86:0/403870032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c2d9915d0 0x558c2d9a4b90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 >> 172.21.15.86:0/403870032 conn(0x558c2d95a310 msgr2=0x558c2d98c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.350+0000 7f1caaffd640 0 client.15702 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 shutdown_connections 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd41c3740 1 -- 172.21.15.86:0/403870032 wait complete. 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd41c3740 20 client.15702 trim_cache size 0 max 16384 2024-09-05T06:42:08.357 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 msgr2=0x7f1cb80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 0x7f1cb80a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40042c0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 msgr2=0x7f1ccc19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c970 tx=0x7f1cc400ce30 comp rx=0 tx=0).stop 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.351+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 shutdown_connections 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1cb80a2e10 0x7f1cb80a52d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ccc102ef0 0x7f1ccc19cbf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ccc102570 0x7f1ccc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 --2- 172.21.15.86:0/201579058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ccc100e20 0x7f1ccc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:08.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 >> 172.21.15.86:0/201579058 conn(0x7f1ccc0fc7d0 msgr2=0x7f1ccc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:08.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 shutdown_connections 2024-09-05T06:42:08.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:08.352+0000 7f1cd1dea640 1 -- 172.21.15.86:0/201579058 wait complete. 2024-09-05T06:42:08.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:08 smithi086 ceph-mon[30943]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:08 smithi115 ceph-mon[34791]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:08.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:08 smithi096 ceph-mon[34468]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:09.166 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:09.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:42:09.425 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:09 smithi086 ceph-mon[30943]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-05T06:42:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:09 smithi115 ceph-mon[34791]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-05T06:42:09.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 -- 172.21.15.86:0/3752465641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4100a90 msgr2=0x7fe3c4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/3752465641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4100a90 0x7fe3c4100ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009930 tx=0x7fe3b802a420 comp rx=0 tx=0).stop 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 -- 172.21.15.86:0/3752465641 shutdown_connections 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/3752465641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/3752465641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4100a90 0x7fe3c4100ef0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/3752465641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4106a90 0x7fe3c4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 -- 172.21.15.86:0/3752465641 >> 172.21.15.86:0/3752465641 conn(0x7fe3c40fc7d0 msgr2=0x7fe3c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 -- 172.21.15.86:0/3752465641 shutdown_connections 2024-09-05T06:42:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.763+0000 7fe3c9db1640 1 -- 172.21.15.86:0/3752465641 wait complete. 2024-09-05T06:42:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.764+0000 7fe3c9db1640 1 Processor -- start 2024-09-05T06:42:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.764+0000 7fe3c9db1640 1 -- start start 2024-09-05T06:42:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 0x7fe3c41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 0x7fe3c419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 0x7fe3c41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 0x7fe3c41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34904/0 (socket says 172.21.15.86:34904) 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 0x7fe3c419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c37fe640 1 -- 172.21.15.86:0/868355704 learned_addr learned my addr 172.21.15.86:0/868355704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110d50 con 0x7fe3c4106a90 2024-09-05T06:42:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110bd0 con 0x7fe3c4100a90 2024-09-05T06:42:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.765+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110ed0 con 0x7fe3c4101430 2024-09-05T06:42:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 -- 172.21.15.86:0/868355704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 msgr2=0x7fe3c41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 -- 172.21.15.86:0/868355704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 msgr2=0x7fe3c41972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 0x7fe3c41972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419c260 con 0x7fe3c4106a90 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c3fff640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 0x7fe3c419bb80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00e940 tx=0x7fe3bc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c0ff9640 1 -- 172.21.15.86:0/868355704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc004d20 con 0x7fe3c4106a90 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c0ff9640 1 -- 172.21.15.86:0/868355704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3bc005020 con 0x7fe3c4106a90 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.766+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419c550 con 0x7fe3c4106a90 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.767+0000 7fe3c0ff9640 1 -- 172.21.15.86:0/868355704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0116d0 con 0x7fe3c4106a90 2024-09-05T06:42:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.767+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a3e30 con 0x7fe3c4106a90 2024-09-05T06:42:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.770+0000 7fe3cc18a740 1 Processor -- start 2024-09-05T06:42:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3c0ff9640 1 -- 172.21.15.86:0/868355704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fe3bc00cb20 con 0x7fe3c4106a90 2024-09-05T06:42:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 -- start start 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3e762a0 con 0x55e0f3f11f70 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3e1db10 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3e72260 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43842/0 (socket says 172.21.15.86:43842) 2024-09-05T06:42:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.771+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/3124167777 learned_addr learned my addr 172.21.15.86:0/3124167777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/3124167777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 msgr2=0x55e0f3f267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/3124167777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 msgr2=0x55e0f3f0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/3124167777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f3be7b90 con 0x55e0f3f11f70 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c0ff9640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 0x7fe3ac0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006010 tx=0x7fe3b8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6979a2aa3b67b793 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 0x7fe3ac0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3c0ff9640 1 -- 172.21.15.86:0/868355704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe3bc0cfbb0 con 0x7fe3c4106a90 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.772+0000 7fe3a9ffb640 1 -- 172.21.15.86:0/3124167777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8035040 con 0x55e0f3f11f70 2024-09-05T06:42:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.773+0000 7fe3a9ffb640 1 -- 172.21.15.86:0/3124167777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b8004e20 con 0x55e0f3f11f70 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.773+0000 7fe3a9ffb640 1 -- 172.21.15.86:0/3124167777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8003e00 con 0x55e0f3f11f70 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3124167777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 msgr2=0x55e0f3f12350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006010 tx=0x7fe3b8005e50 comp rx=0 tx=0).stop 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 0x7fe3ac0a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e0f3f0e210 tx=0x7fe3b400b480 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3124167777 shutdown_connections 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3124167777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f11f70 0x55e0f3f12350 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3124167777 >> 172.21.15.86:0/3124167777 conn(0x55e0f3f0c7e0 msgr2=0x55e0f3f0cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3124167777 shutdown_connections 2024-09-05T06:42:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3124167777 wait complete. 2024-09-05T06:42:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 -- start start 2024-09-05T06:42:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.774+0000 7fe3cc18a740 1 Processor -- start 2024-09-05T06:42:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f24290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3f26d00 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3dfe6e0 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0f3f26e80 con 0x55e0f3f11f70 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3cc18a740 10 client.0 _ll_register_callbacks cb 0x55e0f3e22a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43844/0 (socket says 172.21.15.86:43844) 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3c37fe640 1 -- 172.21.15.86:0/1522575423 learned_addr learned my addr 172.21.15.86:0/1522575423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.775+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f24290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c3fff640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 -- 172.21.15.86:0/1522575423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 msgr2=0x55e0f3f247d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f247d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 -- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0f2b0 msgr2=0x55e0f3f24290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f24290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 -- 172.21.15.86:0/1522575423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f3f30360 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003df0 tx=0x7fe3b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40748aa244dbb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4005020 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe3b4017030 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.776+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b8004e20 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3b4031050 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 10 client.15714 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e0f3f32d30 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b8004e20 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b4027410 con 0x55e0f3f0e8f0 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 10 client.15714 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:42:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 20 client.15714 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.777+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3f341f0 0x55e0f3f36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.778+0000 7fe3cc18a740 4 client.15714 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.778+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f3e839d0 con 0x55e0f3f341f0 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.778+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3f341f0 0x55e0f3f36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.778+0000 7fe3a2ffd640 10 client.15714 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.778+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3f341f0 0x55e0f3f36690 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b13d4f659e7bfa server_cookie=29b93bbbd91d2671 in_seq=0 out_seq=0 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.779+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/1522575423 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0f3e839d0 con 0x55e0f3f341f0 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.779+0000 7fe3a2ffd640 10 client.15714 handle_command_reply: tid=0 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.779+0000 7fe3cc18a740 1 client.15714 shutdown 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 msgr2=0x55e0f3f23d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003df0 tx=0x7fe3b4004730 comp rx=0 tx=0).stop 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 shutdown_connections 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3f341f0 0x55e0f3f36690 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f247d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3a2ffd640 0 client.15714 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/1522575423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0e8f0 0x55e0f3f23d50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 >> 172.21.15.86:0/1522575423 conn(0x55e0f3f0c7e0 msgr2=0x55e0f3f0be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 shutdown_connections 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 1 -- 172.21.15.86:0/1522575423 wait complete. 2024-09-05T06:42:09.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.780+0000 7fe3cc18a740 20 client.15714 trim_cache size 0 max 16384 2024-09-05T06:42:09.788 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:09 smithi096 ceph-mon[34468]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-05T06:42:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 Processor -- start 2024-09-05T06:42:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 -- start start 2024-09-05T06:42:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f13bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f11f70 0x55e0f3f21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 0x55e0f3f10bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41acda0 con 0x55e0f3f25c70 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ae8e0 con 0x55e0f3f11f70 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ad470 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 0x55e0f3f10bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 0x55e0f3f10bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43860/0 (socket says 172.21.15.86:43860) 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.811+0000 7fe3c3fff640 1 -- 172.21.15.86:0/3282476158 learned_addr learned my addr 172.21.15.86:0/3282476158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f13bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f11f70 0x55e0f3f21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 -- 172.21.15.86:0/3282476158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f11f70 msgr2=0x55e0f3f21780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f11f70 0x55e0f3f21780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 -- 172.21.15.86:0/3282476158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 msgr2=0x55e0f3f10bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 0x55e0f3f10bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 -- 172.21.15.86:0/3282476158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f3f36d10 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.812+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f13bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3b401c000 tx=0x7fe3b4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a10a1940b8ae14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.813+0000 7fe3a97fa640 1 -- 172.21.15.86:0/3282476158 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4003a10 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.813+0000 7fe3a97fa640 1 -- 172.21.15.86:0/3282476158 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b4027ca0 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.813+0000 7fe3a97fa640 1 -- 172.21.15.86:0/3282476158 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4004ad0 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.813+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3282476158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 msgr2=0x55e0f3f13bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.813+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f13bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3b401c000 tx=0x7fe3b4005fd0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3282476158 shutdown_connections 2024-09-05T06:42:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f25c70 0x55e0f3f10bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f11f70 0x55e0f3f21780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/3282476158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f13bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3282476158 >> 172.21.15.86:0/3282476158 conn(0x55e0f3f0c7e0 msgr2=0x55e0f3f25560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3282476158 shutdown_connections 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 -- 172.21.15.86:0/3282476158 wait complete. 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 -- start start 2024-09-05T06:42:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.814+0000 7fe3cc18a740 1 Processor -- start 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f2c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f25c70 0x55e0f3f2c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ae210 con 0x55e0f3f0f2b0 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41adb40 con 0x55e0f3f25c70 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41aad80 con 0x55e0f3f11f70 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3cc18a740 10 client.0 _ll_register_callbacks cb 0x7fe3b4003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43872/0 (socket says 172.21.15.86:43872) 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c37fe640 1 -- 172.21.15.86:0/2947137112 learned_addr learned my addr 172.21.15.86:0/2947137112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f2c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c3fff640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f25c70 0x55e0f3f2c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f25c70 msgr2=0x55e0f3f2c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.815+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f25c70 0x55e0f3f2c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/2947137112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 msgr2=0x55e0f3f3c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3c2ffd640 1 -- 172.21.15.86:0/2947137112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f3f3afa0 con 0x55e0f3f11f70 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3c2ffd640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f2c0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2da88b4271fe49c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8058060 con 0x55e0f3f11f70 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe3b8004b80 con 0x55e0f3f11f70 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b4027ca0 con 0x55e0f3f11f70 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3b805f050 con 0x55e0f3f11f70 2024-09-05T06:42:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3cc18a740 10 client.25177 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.816+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e0f3fa0130 con 0x55e0f3f11f70 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b8005850 con 0x55e0f3f11f70 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b8053350 con 0x55e0f3f11f70 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3cc18a740 10 client.25177 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3cc18a740 10 client.25177 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.817+0000 7fe3cc18a740 20 client.25177 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3fa03c0 0x55e0f3fa27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3cc18a740 4 client.25177 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}] 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}) -- 0x7fe3b4027ca0 con 0x55e0f3fa03c0 2024-09-05T06:42:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3fa03c0 0x55e0f3fa27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3c37fe640 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3fa03c0 0x55e0f3fa27b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e75845370d79e6 server_cookie=5c86b2dbac78b3c in_seq=0 out_seq=0 2024-09-05T06:42:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.818+0000 7fe3a2ffd640 10 client.25177 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3a2ffd640 1 -- 172.21.15.86:0/2947137112 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fe3b4027ca0 con 0x55e0f3fa03c0 2024-09-05T06:42:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3a2ffd640 10 client.25177 handle_command_reply: tid=0 2024-09-05T06:42:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 client.25177 shutdown 2024-09-05T06:42:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 msgr2=0x55e0f3f2c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f2c0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8009880 comp rx=0 tx=0).stop 2024-09-05T06:42:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 shutdown_connections 2024-09-05T06:42:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e0f3fa03c0 0x55e0f3fa27b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f3f25c70 0x55e0f3f2c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0f3f11f70 0x55e0f3f2c0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 --2- 172.21.15.86:0/2947137112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e0f3f0f2b0 0x55e0f3f3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.827+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 >> 172.21.15.86:0/2947137112 conn(0x55e0f3f0c7e0 msgr2=0x55e0f3f0b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.828+0000 7fe3a2ffd640 0 client.25177 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.828+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 shutdown_connections 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.828+0000 7fe3cc18a740 1 -- 172.21.15.86:0/2947137112 wait complete. 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.828+0000 7fe3cc18a740 20 client.25177 trim_cache size 0 max 16384 2024-09-05T06:42:09.829 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 28, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 2, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "c2f055af": { 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 148.2, 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:42:09.830 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:42:09.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 msgr2=0x7fe3ac0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 0x7fe3ac0a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e0f3f0e210 tx=0x7fe3b400b480 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 msgr2=0x7fe3c419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 0x7fe3c419bb80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00e940 tx=0x7fe3bc00ee00 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 shutdown_connections 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3ac0a2f10 0x7fe3ac0a53d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c4106a90 0x7fe3c419bb80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c4101430 0x7fe3c41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 --2- 172.21.15.86:0/868355704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3c4100a90 0x7fe3c41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 >> 172.21.15.86:0/868355704 conn(0x7fe3c40fc7d0 msgr2=0x7fe3c40fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 shutdown_connections 2024-09-05T06:42:09.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:09.829+0000 7fe3c9db1640 1 -- 172.21.15.86:0/868355704 wait complete. 2024-09-05T06:42:10.941 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c2f055af', quiesce took 7.0 seconds. Will release after: 21.2 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 -- 172.21.15.86:0/3007552601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c102570 msgr2=0x7f565c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 --2- 172.21.15.86:0/3007552601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c102570 0x7f565c1029b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002a480 comp rx=0 tx=0).stop 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 -- 172.21.15.86:0/3007552601 shutdown_connections 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 --2- 172.21.15.86:0/3007552601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 0x7f565c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 --2- 172.21.15.86:0/3007552601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c102570 0x7f565c1029b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 --2- 172.21.15.86:0/3007552601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 -- 172.21.15.86:0/3007552601 >> 172.21.15.86:0/3007552601 conn(0x7f565c0fc7d0 msgr2=0x7f565c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 -- 172.21.15.86:0/3007552601 shutdown_connections 2024-09-05T06:42:11.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.124+0000 7f56620ab640 1 -- 172.21.15.86:0/3007552601 wait complete. 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 Processor -- start 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 -- start start 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 0x7f565c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 0x7f565c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110e30 con 0x7f565c19ced0 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110cb0 con 0x7f565c100e20 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f56620ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110fb0 con 0x7f565c102ef0 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f565bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 0x7f565c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34958/0 (socket says 172.21.15.86:34958) 2024-09-05T06:42:11.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.125+0000 7f565b7fe640 1 -- 172.21.15.86:0/632195847 learned_addr learned my addr 172.21.15.86:0/632195847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565affd640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 0x7f565c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 -- 172.21.15.86:0/632195847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 msgr2=0x7f565c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 0x7f565c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 -- 172.21.15.86:0/632195847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 msgr2=0x7f565c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 -- 172.21.15.86:0/632195847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c19d990 con 0x7f565c19ced0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565b7fe640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f565bfff640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 0x7f565c19d380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f565400e940 tx=0x7f565400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f5658ff9640 1 -- 172.21.15.86:0/632195847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654004d20 con 0x7f565c19ced0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f5658ff9640 1 -- 172.21.15.86:0/632195847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5654005020 con 0x7f565c19ced0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c19dc80 con 0x7f565c19ced0 2024-09-05T06:42:11.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f5658ff9640 1 -- 172.21.15.86:0/632195847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56540116d0 con 0x7f565c19ced0 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.126+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c10bd60 con 0x7f565c19ced0 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.130+0000 7f5658ff9640 1 -- 172.21.15.86:0/632195847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f565400cb20 con 0x7f565c19ced0 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.130+0000 7f5664484740 1 Processor -- start 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.130+0000 7f5658ff9640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 0x7f56440a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 -- start start 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565affd640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 0x7f56440a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea119a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccea07f2f0 con 0x55ccea11ab40 2024-09-05T06:42:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccea026cf0 con 0x55ccea1195d0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5664484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ccea07b2b0 con 0x55ccea12cb80 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43904/0 (socket says 172.21.15.86:43904) 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565b7fe640 1 -- 172.21.15.86:0/1808780785 learned_addr learned my addr 172.21.15.86:0/1808780785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f5658ff9640 1 -- 172.21.15.86:0/632195847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f56540cfaf0 con 0x7f565c19ced0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565bfff640 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565affd640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 0x7f56440a53d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5650005940 tx=0x7f56500058d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.131+0000 7f565affd640 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea119a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 -- 172.21.15.86:0/1808780785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 msgr2=0x55ccea12ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea12ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 -- 172.21.15.86:0/1808780785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 msgr2=0x55ccea119a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea119a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 -- 172.21.15.86:0/1808780785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccea00b3a0 con 0x55ccea11ab40 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f564c0098f0 tx=0x7f564c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b026747498c30ff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f5641ffb640 1 -- 172.21.15.86:0/1808780785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c00e930 con 0x55ccea11ab40 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f5641ffb640 1 -- 172.21.15.86:0/1808780785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f564c019420 con 0x55ccea11ab40 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.132+0000 7f5641ffb640 1 -- 172.21.15.86:0/1808780785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c018620 con 0x55ccea11ab40 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 -- 172.21.15.86:0/1808780785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 msgr2=0x55ccea119090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f564c0098f0 tx=0x7f564c005f90 comp rx=0 tx=0).stop 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 -- 172.21.15.86:0/1808780785 shutdown_connections 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea12ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea119a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 --2- 172.21.15.86:0/1808780785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea119090 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 -- 172.21.15.86:0/1808780785 >> 172.21.15.86:0/1808780785 conn(0x55ccea0e2310 msgr2=0x55ccea115c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 -- 172.21.15.86:0/1808780785 shutdown_connections 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.133+0000 7f5664484740 1 -- 172.21.15.86:0/1808780785 wait complete. 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 -- start start 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 Processor -- start 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 0x55ccea12c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea11ab40 0x55ccea129820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccea12f880 con 0x55ccea12cb80 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccea03a060 con 0x55ccea11ab40 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ccea12fa00 con 0x55ccea1195d0 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f5664484740 10 client.0 _ll_register_callbacks cb 0x55cce9d8d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f565bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f565bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43912/0 (socket says 172.21.15.86:43912) 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f565bfff640 1 -- 172.21.15.86:0/3260849951 learned_addr learned my addr 172.21.15.86:0/3260849951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.134+0000 7f565affd640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea11ab40 0x55ccea129820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 -- 172.21.15.86:0/3260849951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 msgr2=0x55ccea12c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565b7fe640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 0x55ccea12c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 0x55ccea12c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 -- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea11ab40 msgr2=0x55ccea129820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea11ab40 0x55ccea129820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 -- 172.21.15.86:0/3260849951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce9df03c0 con 0x55ccea12cb80 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565b7fe640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 0x55ccea12c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f565bfff640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f565400c640 tx=0x7f565409b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92f6144067b58ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f563affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565401f070 con 0x55ccea12cb80 2024-09-05T06:42:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c019420 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f56540051d0 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f565400ce10 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f5664484740 10 client.15732 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.135+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccea13b370 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f564c019420 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f565400f040 con 0x55ccea12cb80 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 20 client.15732 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea11c790 0x55ccea11ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 4 client.15732 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccea08c860 con 0x55ccea11c790 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.136+0000 7f565b7fe640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea11c790 0x55ccea11ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.137+0000 7f565b7fe640 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea11c790 0x55ccea11ec20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=555df8b1c36e579e server_cookie=f921cdcbd018b2a7 in_seq=0 out_seq=0 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.137+0000 7f563affd640 10 client.15732 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.137+0000 7f563affd640 1 -- 172.21.15.86:0/3260849951 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccea08c860 con 0x55ccea11c790 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.137+0000 7f563affd640 10 client.15732 handle_command_reply: tid=0 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.137+0000 7f5664484740 1 client.15732 shutdown 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 msgr2=0x55ccea129e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f565400c640 tx=0x7f565409b660 comp rx=0 tx=0).stop 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 shutdown_connections 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea11c790 0x55ccea11ec20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea12cb80 0x55ccea129e10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea11ab40 0x55ccea129820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 --2- 172.21.15.86:0/3260849951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea1195d0 0x55ccea12c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 >> 172.21.15.86:0/3260849951 conn(0x55ccea0e2310 msgr2=0x55ccea11b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f563affd640 0 client.15732 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:11.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 shutdown_connections 2024-09-05T06:42:11.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 1 -- 172.21.15.86:0/3260849951 wait complete. 2024-09-05T06:42:11.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.138+0000 7f5664484740 20 client.15732 trim_cache size 0 max 16384 2024-09-05T06:42:11.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 Processor -- start 2024-09-05T06:42:11.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 -- start start 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea115e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea152db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f565c073f30 con 0x55ccea11ab40 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1a67d0 con 0x55ccea1195d0 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f5664484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1a5360 con 0x55ccea12cb80 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43924/0 (socket says 172.21.15.86:43924) 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565affd640 1 -- 172.21.15.86:0/1118696128 learned_addr learned my addr 172.21.15.86:0/1118696128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565bfff640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.156+0000 7f565affd640 1 -- 172.21.15.86:0/1118696128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 msgr2=0x55ccea152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565affd640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565affd640 1 -- 172.21.15.86:0/1118696128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 msgr2=0x55ccea115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565affd640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea115e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565affd640 1 -- 172.21.15.86:0/1118696128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccea12c8e0 con 0x55ccea11ab40 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565bfff640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f565affd640 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f565c101be0 tx=0x7f565003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5fdaf22cbfbe4e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f56417fa640 1 -- 172.21.15.86:0/1118696128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650035040 con 0x55ccea11ab40 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f56417fa640 1 -- 172.21.15.86:0/1118696128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565004e070 con 0x55ccea11ab40 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.157+0000 7f56417fa640 1 -- 172.21.15.86:0/1118696128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650004cc0 con 0x55ccea11ab40 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 -- 172.21.15.86:0/1118696128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 msgr2=0x55ccea152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f565c101be0 tx=0x7f565003d000 comp rx=0 tx=0).stop 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 -- 172.21.15.86:0/1118696128 shutdown_connections 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea12cb80 0x55ccea152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea11ab40 0x55ccea152870 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 --2- 172.21.15.86:0/1118696128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea1195d0 0x55ccea115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 -- 172.21.15.86:0/1118696128 >> 172.21.15.86:0/1118696128 conn(0x55ccea0e2310 msgr2=0x55ccea115700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 -- 172.21.15.86:0/1118696128 shutdown_connections 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.158+0000 7f5664484740 1 -- 172.21.15.86:0/1118696128 wait complete. 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 -- start start 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 Processor -- start 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea11ab40 0x55ccea120b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 0x55ccea1210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1a6100 con 0x55ccea1195d0 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1a5a30 con 0x55ccea12cb80 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f565c071f10 con 0x55ccea11ab40 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f5664484740 10 client.0 _ll_register_callbacks cb 0x7f56540057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43930/0 (socket says 172.21.15.86:43930) 2024-09-05T06:42:11.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.159+0000 7f565b7fe640 1 -- 172.21.15.86:0/1555992545 learned_addr learned my addr 172.21.15.86:0/1555992545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 0x55ccea1210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565affd640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea11ab40 0x55ccea120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 -- 172.21.15.86:0/1555992545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea11ab40 msgr2=0x55ccea120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea11ab40 0x55ccea120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 -- 172.21.15.86:0/1555992545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 msgr2=0x55ccea120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 -- 172.21.15.86:0/1555992545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccea1ae970 con 0x55ccea12cb80 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f565bfff640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 0x55ccea1210c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5654098400 tx=0x7f5654004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92029bb78a9cdc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f563affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565401f070 con 0x55ccea12cb80 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f565409b090 con 0x55ccea12cb80 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.160+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565004e070 con 0x55ccea12cb80 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f565409ba30 con 0x55ccea12cb80 2024-09-05T06:42:11.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 10 client.25226 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccea1afb60 con 0x55ccea12cb80 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565409fd30 con 0x55ccea12cb80 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f56540b4b80 con 0x55ccea12cb80 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 10 client.25226 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 10 client.25226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 20 client.25226 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea1b02d0 0x55ccea1b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 4 client.25226 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.161+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f565004e070 con 0x55ccea1b02d0 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.162+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea1b02d0 0x55ccea1b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.162+0000 7f563affd640 10 client.25226 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.162+0000 7f565b7fe640 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea1b02d0 0x55ccea1b26c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e1cb5544518c398 server_cookie=bd1f8051c9c7dce3 in_seq=0 out_seq=0 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f563affd640 1 -- 172.21.15.86:0/1555992545 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f565004e070 con 0x55ccea1b02d0 2024-09-05T06:42:11.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f563affd640 10 client.25226 handle_command_reply: tid=0 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 client.25226 shutdown 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 msgr2=0x55ccea1210c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 0x55ccea1210c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5654098400 tx=0x7f5654004ec0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 shutdown_connections 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ccea1b02d0 0x55ccea1b26c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccea12cb80 0x55ccea1210c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccea11ab40 0x55ccea120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 --2- 172.21.15.86:0/1555992545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccea1195d0 0x55ccea120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 >> 172.21.15.86:0/1555992545 conn(0x55ccea0e2310 msgr2=0x55ccea12b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f563affd640 0 client.25226 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 shutdown_connections 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 1 -- 172.21.15.86:0/1555992545 wait complete. 2024-09-05T06:42:11.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.163+0000 7f5664484740 20 client.25226 trim_cache size 0 max 16384 2024-09-05T06:42:11.171 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:11.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 msgr2=0x7f56440a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 0x7f56440a53d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5650005940 tx=0x7f56500058d0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 msgr2=0x7f565c19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 0x7f565c19d380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f565400e940 tx=0x7f565400ee00 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 shutdown_connections 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f56440a2f10 0x7f56440a53d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f565c19ced0 0x7f565c19d380 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f565c102ef0 0x7f565c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 --2- 172.21.15.86:0/632195847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565c100e20 0x7f565c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 >> 172.21.15.86:0/632195847 conn(0x7f565c0fc7d0 msgr2=0x7f565c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.164+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 shutdown_connections 2024-09-05T06:42:11.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:11.165+0000 7f56620ab640 1 -- 172.21.15.86:0/632195847 wait complete. 2024-09-05T06:42:12.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:42:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:12 smithi115 ceph-mon[34791]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:12.700 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:12 smithi086 ceph-mon[30943]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:12.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:12 smithi096 ceph-mon[34468]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 -- 172.21.15.86:0/674379387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8101012d0 msgr2=0x7fc81010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 --2- 172.21.15.86:0/674379387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8101012d0 0x7fc81010df90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc800009900 tx=0x7fc80002a410 comp rx=0 tx=0).stop 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 -- 172.21.15.86:0/674379387 shutdown_connections 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 --2- 172.21.15.86:0/674379387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc81010e4d0 0x7fc8101108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 --2- 172.21.15.86:0/674379387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8101012d0 0x7fc81010df90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 --2- 172.21.15.86:0/674379387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc810100920 0x7fc810100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.799+0000 7fc8174fc640 1 -- 172.21.15.86:0/674379387 >> 172.21.15.86:0/674379387 conn(0x7fc8100fc790 msgr2=0x7fc8100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.800+0000 7fc8174fc640 1 -- 172.21.15.86:0/674379387 shutdown_connections 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.800+0000 7fc8174fc640 1 -- 172.21.15.86:0/674379387 wait complete. 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.800+0000 7fc8174fc640 1 Processor -- start 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 -- start start 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc810100920 0x7fc81019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 0x7fc81019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc810114000 con 0x7fc81010e4d0 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc810113e80 con 0x7fc810100920 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41904/0 (socket says 172.21.15.86:41904) 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc8174fc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc810114180 con 0x7fc8101012d0 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 -- 172.21.15.86:0/3713016493 learned_addr learned my addr 172.21.15.86:0/3713016493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc814a97640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 0x7fc81019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 -- 172.21.15.86:0/3713016493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 msgr2=0x7fc81019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 0x7fc81019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 -- 172.21.15.86:0/3713016493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc810100920 msgr2=0x7fc81019a5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc810100920 0x7fc81019a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.801+0000 7fc815a99640 1 -- 172.21.15.86:0/3713016493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81019f570 con 0x7fc81010e4d0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc814a97640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 0x7fc81019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc815a99640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9d0 tx=0x7fc80800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc8067fc640 1 -- 172.21.15.86:0/3713016493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80800cd80 con 0x7fc81010e4d0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc8067fc640 1 -- 172.21.15.86:0/3713016493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc808005020 con 0x7fc81010e4d0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc8067fc640 1 -- 172.21.15.86:0/3713016493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080116d0 con 0x7fc81010e4d0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81019f860 con 0x7fc81010e4d0 2024-09-05T06:42:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.802+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a7140 con 0x7fc81010e4d0 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.805+0000 7fc8067fc640 1 -- 172.21.15.86:0/3713016493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fc8080051d0 con 0x7fc81010e4d0 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.805+0000 7fc8198d5740 1 Processor -- start 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 -- start start 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8067fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 0x7fc7fc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559694eea2f0 con 0x559694f845d0 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559694e91cf0 con 0x559694f8f370 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8198d5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559694ee62b0 con 0x559694f85b40 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc8067fc640 1 -- 172.21.15.86:0/3713016493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fc8080cfce0 con 0x7fc81010e4d0 2024-09-05T06:42:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc814a97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc814a97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41914/0 (socket says 172.21.15.86:41914) 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc814a97640 1 -- 172.21.15.86:0/1941201165 learned_addr learned my addr 172.21.15.86:0/1941201165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc815298640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 0x7fc7fc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc814a97640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.806+0000 7fc815a99640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 -- 172.21.15.86:0/1941201165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 msgr2=0x559694f84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 -- 172.21.15.86:0/1941201165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 msgr2=0x559694f91760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f91760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 -- 172.21.15.86:0/1941201165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559694e763a0 con 0x559694f845d0 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc815a99640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f91760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc814a97640 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc800005ec0 tx=0x7fc800005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d9d63dfd9c6149 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc815298640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 0x7fc7fc0a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b990 tx=0x7fc80c00b370 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc7fb7fe640 1 -- 172.21.15.86:0/1941201165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800035040 con 0x559694f845d0 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc7fb7fe640 1 -- 172.21.15.86:0/1941201165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8000035f0 con 0x559694f845d0 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.807+0000 7fc7fb7fe640 1 -- 172.21.15.86:0/1941201165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000044e0 con 0x559694f845d0 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 -- 172.21.15.86:0/1941201165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 msgr2=0x559694f84a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc800005ec0 tx=0x7fc800005b50 comp rx=0 tx=0).stop 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 -- 172.21.15.86:0/1941201165 shutdown_connections 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f91760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f84a10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 --2- 172.21.15.86:0/1941201165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 -- 172.21.15.86:0/1941201165 >> 172.21.15.86:0/1941201165 conn(0x559694f4d310 msgr2=0x559694f80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 -- 172.21.15.86:0/1941201165 shutdown_connections 2024-09-05T06:42:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.808+0000 7fc8198d5740 1 -- 172.21.15.86:0/1941201165 wait complete. 2024-09-05T06:42:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 -- start start 2024-09-05T06:42:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 Processor -- start 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559694f92010 con 0x559694f845d0 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559694ea5060 con 0x559694f8f370 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.809+0000 7fc8198d5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559694f92190 con 0x559694f85b40 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc8198d5740 10 client.0 _ll_register_callbacks cb 0x559694bf88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41926/0 (socket says 172.21.15.86:41926) 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 -- 172.21.15.86:0/3035554723 learned_addr learned my addr 172.21.15.86:0/3035554723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc814a97640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815a99640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 -- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 msgr2=0x559694f8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 -- 172.21.15.86:0/3035554723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 msgr2=0x559694f8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f8c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 -- 172.21.15.86:0/3035554723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559694c5b3c0 con 0x559694f845d0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc814a97640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815a99640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc815298640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc80c010710 tx=0x7fc80c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883da265ebed7ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c021020 con 0x559694f845d0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fc80c00f680 con 0x559694f845d0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.810+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8000035f0 con 0x559694f845d0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc80c02c050 con 0x559694f845d0 2024-09-05T06:42:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559694f9dba0 con 0x559694f845d0 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc80c016030 con 0x559694f845d0 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc80c01d9b0 con 0x559694f845d0 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 20 client.15750 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559694f9f060 0x559694fa1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 4 client.15750 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.811+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8000035f0 con 0x559694f9f060 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.812+0000 7fc814a97640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559694f9f060 0x559694fa1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.812+0000 7fc814a97640 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559694f9f060 0x559694fa1450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44f03b87dbe8d8fa server_cookie=d5cb9eea7f315b13 in_seq=0 out_seq=0 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.812+0000 7fc7f8ff9640 10 client.15750 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/3035554723 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc8000035f0 con 0x559694f9f060 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc7f8ff9640 10 client.15750 handle_command_reply: tid=0 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 client.15750 shutdown 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 msgr2=0x559694f8ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc80c010710 tx=0x7fc80c004730 comp rx=0 tx=0).stop 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 shutdown_connections 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559694f9f060 0x559694fa1450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f8f370 0x559694f8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 --2- 172.21.15.86:0/3035554723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f8ede0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 >> 172.21.15.86:0/3035554723 conn(0x559694f4d310 msgr2=0x559694f86960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc7f8ff9640 0 client.15750 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 shutdown_connections 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 1 -- 172.21.15.86:0/3035554723 wait complete. 2024-09-05T06:42:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.813+0000 7fc8198d5740 20 client.15750 trim_cache size 0 max 16384 2024-09-05T06:42:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 Processor -- start 2024-09-05T06:42:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 -- start start 2024-09-05T06:42:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f85b40 0x559694f87660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 0x559694fa6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101b1300 con 0x559694f845d0 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101b2e40 con 0x559694f85b40 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc8198d5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101b19d0 con 0x559694f8f370 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41934/0 (socket says 172.21.15.86:41934) 2024-09-05T06:42:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 -- 172.21.15.86:0/384378126 learned_addr learned my addr 172.21.15.86:0/384378126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc814a97640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f85b40 0x559694f87660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815a99640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 0x559694fa6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 -- 172.21.15.86:0/384378126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 msgr2=0x559694fa6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 0x559694fa6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 -- 172.21.15.86:0/384378126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f85b40 msgr2=0x559694f87660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f85b40 0x559694f87660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.827+0000 7fc815298640 1 -- 172.21.15.86:0/384378126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559694fa1ab0 con 0x559694f845d0 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.828+0000 7fc815a99640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 0x559694fa6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.828+0000 7fc815298640 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c00dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c280af9415b55f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.828+0000 7fc7faffd640 1 -- 172.21.15.86:0/384378126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c01d640 con 0x559694f845d0 2024-09-05T06:42:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.828+0000 7fc7faffd640 1 -- 172.21.15.86:0/384378126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc80c037070 con 0x559694f845d0 2024-09-05T06:42:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.828+0000 7fc7faffd640 1 -- 172.21.15.86:0/384378126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c00fdc0 con 0x559694f845d0 2024-09-05T06:42:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- 172.21.15.86:0/384378126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 msgr2=0x559694f87120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c00dfd0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- 172.21.15.86:0/384378126 shutdown_connections 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f8f370 0x559694fa6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f85b40 0x559694f87660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 --2- 172.21.15.86:0/384378126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f845d0 0x559694f87120 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- 172.21.15.86:0/384378126 >> 172.21.15.86:0/384378126 conn(0x559694f4d310 msgr2=0x559694f7fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- 172.21.15.86:0/384378126 shutdown_connections 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- 172.21.15.86:0/384378126 wait complete. 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.829+0000 7fc8198d5740 1 -- start start 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 Processor -- start 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f93de0 0x559695008cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101b2770 con 0x559694f8f370 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc814a97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101b20a0 con 0x559694f93de0 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101af2e0 con 0x559694f85b40 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc8198d5740 10 client.0 _ll_register_callbacks cb 0x7fc80c005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc814a97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41940/0 (socket says 172.21.15.86:41940) 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.830+0000 7fc814a97640 1 -- 172.21.15.86:0/826164041 learned_addr learned my addr 172.21.15.86:0/826164041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc815a99640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f93de0 0x559695008cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc815298640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 -- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 msgr2=0x559694f93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 -- 172.21.15.86:0/826164041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f93de0 msgr2=0x559695008cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f93de0 0x559695008cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 -- 172.21.15.86:0/826164041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596950093c0 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc814a97640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc800031600 tx=0x7fc800003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbe1fe74f490d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800035070 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc80c037070 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.831+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fc800003fa0 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc800060050 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55969500a600 con 0x559694f8f370 2024-09-05T06:42:12.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc800057070 con 0x559694f8f370 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc80003a040 con 0x559694f8f370 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 20 client.15756 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55969500a8f0 0x55969500cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 4 client.15756 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.832+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc80c037070 con 0x55969500a8f0 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc815298640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55969500a8f0 0x55969500cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc7f8ff9640 10 client.15756 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc815298640 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55969500a8f0 0x55969500cce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5365aab0a1bd04a9 server_cookie=5722a406e9f45f00 in_seq=0 out_seq=0 2024-09-05T06:42:12.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc7f8ff9640 1 -- 172.21.15.86:0/826164041 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc80c037070 con 0x55969500a8f0 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc7f8ff9640 10 client.15756 handle_command_reply: tid=0 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.833+0000 7fc8198d5740 1 client.15756 shutdown 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 msgr2=0x559694f938a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc800031600 tx=0x7fc800003f30 comp rx=0 tx=0).stop 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 shutdown_connections 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55969500a8f0 0x55969500cce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559694f93de0 0x559695008cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559694f8f370 0x559694f938a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 --2- 172.21.15.86:0/826164041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559694f85b40 0x559694f93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 >> 172.21.15.86:0/826164041 conn(0x559694f4d310 msgr2=0x559694f8d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc7f8ff9640 0 client.15756 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 shutdown_connections 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 1 -- 172.21.15.86:0/826164041 wait complete. 2024-09-05T06:42:12.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.834+0000 7fc8198d5740 20 client.15756 trim_cache size 0 max 16384 2024-09-05T06:42:12.839 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:12.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 msgr2=0x7fc7fc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 0x7fc7fc0a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b990 tx=0x7fc80c00b370 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 msgr2=0x7fc81019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9d0 tx=0x7fc80800ee90 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 shutdown_connections 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc7fc0a2e90 0x7fc7fc0a5350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81010e4d0 0x7fc81019ee90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8101012d0 0x7fc81019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 --2- 172.21.15.86:0/3713016493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc810100920 0x7fc81019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 >> 172.21.15.86:0/3713016493 conn(0x7fc8100fc790 msgr2=0x7fc8100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 shutdown_connections 2024-09-05T06:42:12.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:12.835+0000 7fc8174fc640 1 -- 172.21.15.86:0/3713016493 wait complete. 2024-09-05T06:42:13.707 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:13 smithi086 ceph-mon[30943]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:13.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:42:13.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:13 smithi096 ceph-mon[34468]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:13 smithi115 ceph-mon[34791]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 -- 172.21.15.86:0/3286687391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 msgr2=0x7f695c1108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 --2- 172.21.15.86:0/3286687391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c1108a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6954009ca0 tx=0x7f695402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 -- 172.21.15.86:0/3286687391 shutdown_connections 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 --2- 172.21.15.86:0/3286687391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c1108a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 --2- 172.21.15.86:0/3286687391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c074f50 0x7f695c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 --2- 172.21.15.86:0/3286687391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c0745a0 0x7f695c074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 -- 172.21.15.86:0/3286687391 >> 172.21.15.86:0/3286687391 conn(0x7f695c0fd8d0 msgr2=0x7f695c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.471+0000 7f696161f640 1 -- 172.21.15.86:0/3286687391 shutdown_connections 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.472+0000 7f696161f640 1 -- 172.21.15.86:0/3286687391 wait complete. 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.472+0000 7f696161f640 1 Processor -- start 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.472+0000 7f696161f640 1 -- start start 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.472+0000 7f696161f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c0745a0 0x7f695c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.472+0000 7f696161f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f696161f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f696161f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c114090 con 0x7f695c10e470 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f696161f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c113f10 con 0x7f695c074f50 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f696161f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c114210 con 0x7f695c0745a0 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46246/0 (socket says 172.21.15.86:46246) 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 -- 172.21.15.86:0/3607859370 learned_addr learned my addr 172.21.15.86:0/3607859370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695affd640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c0745a0 0x7f695c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 -- 172.21.15.86:0/3607859370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c0745a0 msgr2=0x7f695c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c0745a0 0x7f695c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 -- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 msgr2=0x7f695c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 -- 172.21.15.86:0/3607859370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c19f3e0 con 0x7f695c074f50 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.473+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f695000c920 tx=0x7f695000cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.474+0000 7f6943fff640 1 -- 172.21.15.86:0/3607859370 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950007bb0 con 0x7f695c074f50 2024-09-05T06:42:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.474+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c19f6d0 con 0x7f695c074f50 2024-09-05T06:42:14.523 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.474+0000 7f6943fff640 1 -- 172.21.15.86:0/3607859370 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950007d50 con 0x7f695c074f50 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.474+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1a6fb0 con 0x7f695c074f50 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.474+0000 7f6943fff640 1 -- 172.21.15.86:0/3607859370 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950018800 con 0x7f695c074f50 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.476+0000 7f6943fff640 1 -- 172.21.15.86:0/3607859370 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f695000f450 con 0x7f695c074f50 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f6943fff640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 0x7f69440a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 Processor -- start 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 -- start start 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f6943fff640 1 -- 172.21.15.86:0/3607859370 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f69500cfb30 con 0x7f695c074f50 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f695affd640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 0x7f69440a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f32fd962f0 con 0x55f32fe305d0 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f32fd3dcf0 con 0x55f32fe31b40 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.477+0000 7f69639f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f32fd922b0 con 0x55f32fe3b370 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41968/0 (socket says 172.21.15.86:41968) 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 -- 172.21.15.86:0/3841744487 learned_addr learned my addr 172.21.15.86:0/3841744487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695affd640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 -- 172.21.15.86:0/3841744487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 msgr2=0x55f32fe3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 -- 172.21.15.86:0/3841744487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 msgr2=0x55f32fe30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 -- 172.21.15.86:0/3841744487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32fd223a0 con 0x55f32fe305d0 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695affd640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 0x7f69440a5350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f694c00abe0 tx=0x7f694c00a2d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.478+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6950097e80 tx=0x7f695000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2485d4430a4e4d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.479+0000 7f69417fa640 1 -- 172.21.15.86:0/3841744487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500a4690 con 0x55f32fe305d0 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.479+0000 7f69417fa640 1 -- 172.21.15.86:0/3841744487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695000f740 con 0x55f32fe305d0 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.479+0000 7f69417fa640 1 -- 172.21.15.86:0/3841744487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500cd000 con 0x55f32fe305d0 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.479+0000 7f69639f8740 1 -- 172.21.15.86:0/3841744487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 msgr2=0x55f32fe30a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.479+0000 7f69639f8740 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6950097e80 tx=0x7f695000fbb0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 -- 172.21.15.86:0/3841744487 shutdown_connections 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe30a10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 --2- 172.21.15.86:0/3841744487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 -- 172.21.15.86:0/3841744487 >> 172.21.15.86:0/3841744487 conn(0x55f32fdf9310 msgr2=0x55f32fe2cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 -- 172.21.15.86:0/3841744487 shutdown_connections 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.480+0000 7f69639f8740 1 -- 172.21.15.86:0/3841744487 wait complete. 2024-09-05T06:42:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.481+0000 7f69639f8740 1 -- start start 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.481+0000 7f69639f8740 1 Processor -- start 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe38040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 0x55f32fe38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f32fe467c0 con 0x55f32fe3b370 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f32fd51060 con 0x55f32fe31b40 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f32fe46940 con 0x55f32fe305d0 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f69639f8740 10 client.0 _ll_register_callbacks cb 0x55f32faa48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48242/0 (socket says 172.21.15.86:48242) 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695affd640 1 -- 172.21.15.86:0/3560840245 learned_addr learned my addr 172.21.15.86:0/3560840245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe38040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 0x55f32fe38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 -- 172.21.15.86:0/3560840245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 msgr2=0x55f32fe3ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 -- 172.21.15.86:0/3560840245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 msgr2=0x55f32fe38650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 0x55f32fe38650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695a7fc640 1 -- 172.21.15.86:0/3560840245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32fb073c0 con 0x55f32fe31b40 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695affd640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.482+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 0x55f32fe38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe38040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6950097e80 tx=0x7f6950097cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de445d113b9e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f693eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500a4690 con 0x55f32fe31b40 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695000f740 con 0x55f32fe31b40 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6950098090 con 0x55f32fe31b40 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69500860f0 con 0x55f32fe31b40 2024-09-05T06:42:14.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f69639f8740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f32fe52290 con 0x55f32fe31b40 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.483+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695000f740 con 0x55f32fe31b40 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6950097890 con 0x55f32fe31b40 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 10 client.25247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 20 client.25247 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32fe535e0 0x55f32fe559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 4 client.25247 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32fda3860 con 0x55f32fe535e0 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f695affd640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32fe535e0 0x55f32fe559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f695affd640 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32fe535e0 0x55f32fe559d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0604e5fc3f489dc server_cookie=71265eb5578ffd46 in_seq=0 out_seq=0 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.484+0000 7f693eff5640 10 client.25247 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f693eff5640 1 -- 172.21.15.86:0/3560840245 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f32fda3860 con 0x55f32fe535e0 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f693eff5640 10 client.25247 handle_command_reply: tid=0 2024-09-05T06:42:14.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 client.25247 shutdown 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 msgr2=0x55f32fe38040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe38040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6950097e80 tx=0x7f6950097cc0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 shutdown_connections 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32fe535e0 0x55f32fe559d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f693eff5640 0 client.25247 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe3b370 0x55f32fe38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe31b40 0x55f32fe38040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 --2- 172.21.15.86:0/3560840245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe305d0 0x55f32fe3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.485+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 >> 172.21.15.86:0/3560840245 conn(0x55f32fdf9310 msgr2=0x55f32fe32990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.486+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 shutdown_connections 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.486+0000 7f69639f8740 1 -- 172.21.15.86:0/3560840245 wait complete. 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.486+0000 7f69639f8740 20 client.25247 trim_cache size 0 max 16384 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.498+0000 7f69639f8740 1 Processor -- start 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.498+0000 7f69639f8740 1 -- start start 2024-09-05T06:42:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe305d0 0x55f32fe4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe58f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1accb0 con 0x55f32fe31b40 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ae7f0 con 0x55f32fe305d0 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f69639f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ad380 con 0x55f32fe3b370 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41988/0 (socket says 172.21.15.86:41988) 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 -- 172.21.15.86:0/3405575716 learned_addr learned my addr 172.21.15.86:0/3405575716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695affd640 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe305d0 0x55f32fe4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 -- 172.21.15.86:0/3405575716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 msgr2=0x55f32fe58f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe58f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 -- 172.21.15.86:0/3405575716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe305d0 msgr2=0x55f32fe4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe305d0 0x55f32fe4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 -- 172.21.15.86:0/3405575716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32fe2d730 con 0x55f32fe31b40 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.499+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69500983f0 tx=0x7f6950005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aece52161bf4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f6940ff9640 1 -- 172.21.15.86:0/3405575716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500a4690 con 0x55f32fe31b40 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f6940ff9640 1 -- 172.21.15.86:0/3405575716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950098090 con 0x55f32fe31b40 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f6940ff9640 1 -- 172.21.15.86:0/3405575716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500ba330 con 0x55f32fe31b40 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 -- 172.21.15.86:0/3405575716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 msgr2=0x55f32fe589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69500983f0 tx=0x7f6950005e40 comp rx=0 tx=0).stop 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 -- 172.21.15.86:0/3405575716 shutdown_connections 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe3b370 0x55f32fe58f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe31b40 0x55f32fe589f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 --2- 172.21.15.86:0/3405575716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe305d0 0x55f32fe4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.500+0000 7f69639f8740 1 -- 172.21.15.86:0/3405575716 >> 172.21.15.86:0/3405575716 conn(0x55f32fdf9310 msgr2=0x55f32fe37000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 -- 172.21.15.86:0/3405575716 shutdown_connections 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 -- 172.21.15.86:0/3405575716 wait complete. 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 -- start start 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 Processor -- start 2024-09-05T06:42:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.501+0000 7f69639f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 0x55f32fe566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f69639f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f69639f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ae120 con 0x55f32fe305d0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f69639f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ada50 con 0x55f32fe3b370 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f69639f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1aacd0 con 0x55f32fe535e0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f69639f8740 10 client.0 _ll_register_callbacks cb 0x7f694c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 0x55f32fe566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48260/0 (socket says 172.21.15.86:48260) 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 -- 172.21.15.86:0/3894850600 learned_addr learned my addr 172.21.15.86:0/3894850600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695affd640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 -- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 msgr2=0x55f32fe566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 0x55f32fe566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 -- 172.21.15.86:0/3894850600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 msgr2=0x55f32fe561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695b7fe640 1 -- 172.21.15.86:0/3894850600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32feb5570 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695a7fc640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 0x55f32fe566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.502+0000 7f695affd640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f695b7fe640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f695402ade0 tx=0x7f6954004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0fa83ccf9df150c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f693eff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695404a060 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6950098090 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6954004360 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6954051050 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f69639f8740 10 client.25219 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f32febdae0 con 0x55f32fe535e0 2024-09-05T06:42:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.503+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950098090 con 0x55f32fe535e0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f695403c410 con 0x55f32fe535e0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 10 client.25219 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 10 client.25219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 20 client.25219 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32febe1b0 0x55f32fec05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 4 client.25219 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.504+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f32fd9dd70 con 0x55f32febe1b0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f695affd640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32febe1b0 0x55f32fec05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f693eff5640 10 client.25219 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f695affd640 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32febe1b0 0x55f32fec05a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917f99f05aaaf1cf server_cookie=2fd4b174aa4a9d89 in_seq=0 out_seq=0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f693eff5640 1 -- 172.21.15.86:0/3894850600 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f32fd9dd70 con 0x55f32febe1b0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f693eff5640 10 client.25219 handle_command_reply: tid=0 2024-09-05T06:42:14.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.505+0000 7f69639f8740 1 client.25219 shutdown 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 msgr2=0x55f32feb4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f695402ade0 tx=0x7f6954004270 comp rx=0 tx=0).stop 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 shutdown_connections 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f32febe1b0 0x55f32fec05a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f32fe535e0 0x55f32feb4ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f693eff5640 0 client.25219 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f32fe3b370 0x55f32fe566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 --2- 172.21.15.86:0/3894850600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32fe305d0 0x55f32fe561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 >> 172.21.15.86:0/3894850600 conn(0x55f32fdf9310 msgr2=0x55f32fe367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 shutdown_connections 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 1 -- 172.21.15.86:0/3894850600 wait complete. 2024-09-05T06:42:14.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f69639f8740 20 client.25219 trim_cache size 0 max 16384 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 msgr2=0x7f69440a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 0x7f69440a5350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f694c00abe0 tx=0x7f694c00a2d0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 msgr2=0x7f695c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.506+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f695000c920 tx=0x7f695000cd20 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 shutdown_connections 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2e90 0x7f69440a5350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c10e470 0x7f695c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c074f50 0x7f695c19a970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 --2- 172.21.15.86:0/3607859370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c0745a0 0x7f695c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 >> 172.21.15.86:0/3607859370 conn(0x7f695c0fd8d0 msgr2=0x7f695c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 shutdown_connections 2024-09-05T06:42:14.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:14.507+0000 7f696161f640 1 -- 172.21.15.86:0/3607859370 wait complete. 2024-09-05T06:42:16.227 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:42:16.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:42:16.548 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:16.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:16 smithi086 ceph-mon[30943]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:16 smithi115 ceph-mon[34791]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:16.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:16 smithi096 ceph-mon[34468]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-05T06:42:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.106+0000 7f9c432cf640 1 -- 172.21.15.86:0/685416288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 msgr2=0x7f9c3c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.106+0000 7f9c432cf640 1 --2- 172.21.15.86:0/685416288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c10bb60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009900 tx=0x7f9c2c02a480 comp rx=0 tx=0).stop 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 -- 172.21.15.86:0/685416288 shutdown_connections 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 --2- 172.21.15.86:0/685416288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 0x7f9c3c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 --2- 172.21.15.86:0/685416288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c10bb60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 --2- 172.21.15.86:0/685416288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 -- 172.21.15.86:0/685416288 >> 172.21.15.86:0/685416288 conn(0x7f9c3c0fc7d0 msgr2=0x7f9c3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 -- 172.21.15.86:0/685416288 shutdown_connections 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 -- 172.21.15.86:0/685416288 wait complete. 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 Processor -- start 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.107+0000 7f9c432cf640 1 -- start start 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 0x7f9c3c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c111e30 con 0x7f9c3c103670 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c111cb0 con 0x7f9c3c102d50 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c432cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c111fb0 con 0x7f9c3c10c180 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42018/0 (socket says 172.21.15.86:42018) 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 -- 172.21.15.86:0/3432391315 learned_addr learned my addr 172.21.15.86:0/3432391315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4186c640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 0x7f9c3c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 -- 172.21.15.86:0/3432391315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 msgr2=0x7f9c3c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 0x7f9c3c06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 -- 172.21.15.86:0/3432391315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 msgr2=0x7f9c3c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4086a640 1 -- 172.21.15.86:0/3432391315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c06ca40 con 0x7f9c3c103670 2024-09-05T06:42:17.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.108+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c327fc640 1 -- 172.21.15.86:0/3432391315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c03f030 con 0x7f9c3c103670 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3c1ad0e0 con 0x7f9c3c103670 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c327fc640 1 -- 172.21.15.86:0/3432391315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2c003800 con 0x7f9c3c103670 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c327fc640 1 -- 172.21.15.86:0/3432391315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035070 con 0x7f9c3c103670 2024-09-05T06:42:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.109+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3c1ad610 con 0x7f9c3c103670 2024-09-05T06:42:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.112+0000 7f9c456a8740 1 Processor -- start 2024-09-05T06:42:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.112+0000 7f9c456a8740 1 -- start start 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c327fc640 1 -- 172.21.15.86:0/3432391315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f9c2c031af0 con 0x7f9c3c103670 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a51c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a54540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55baa89a8280 con 0x55baa8a42a40 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55baa894fb10 con 0x55baa8a43360 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c456a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55baa89a4240 con 0x55baa8a52150 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4186c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4186c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a42e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42026/0 (socket says 172.21.15.86:42026) 2024-09-05T06:42:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4186c640 1 -- 172.21.15.86:0/990846292 learned_addr learned my addr 172.21.15.86:0/990846292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a51c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4086a640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a54540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 -- 172.21.15.86:0/990846292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 msgr2=0x55baa8a54540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a54540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 -- 172.21.15.86:0/990846292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 msgr2=0x55baa8a42e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a42e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4106b640 1 -- 172.21.15.86:0/990846292 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa895c1f0 con 0x55baa8a43360 2024-09-05T06:42:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.113+0000 7f9c4086a640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a54540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c327fc640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 0x7f9c280a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c327fc640 1 -- 172.21.15.86:0/3432391315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9c2c0f0260 con 0x7f9c3c103670 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c4106b640 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a51c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800bab0 tx=0x7f9c3800bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df8fca713452c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c4186c640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 0x7f9c280a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c277fe640 1 -- 172.21.15.86:0/990846292 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38017420 con 0x55baa8a43360 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c277fe640 1 -- 172.21.15.86:0/990846292 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c38017a70 con 0x55baa8a43360 2024-09-05T06:42:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.114+0000 7f9c277fe640 1 -- 172.21.15.86:0/990846292 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800fa80 con 0x55baa8a43360 2024-09-05T06:42:17.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c4186c640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 0x7f9c280a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007fd0 tx=0x7f9c34007520 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 -- 172.21.15.86:0/990846292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 msgr2=0x55baa8a51c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a51c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800bab0 tx=0x7f9c3800bf70 comp rx=0 tx=0).stop 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 -- 172.21.15.86:0/990846292 shutdown_connections 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a54540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a51c10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 --2- 172.21.15.86:0/990846292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a42e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 -- 172.21.15.86:0/990846292 >> 172.21.15.86:0/990846292 conn(0x55baa8a3f0f0 msgr2=0x55baa8a3f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 -- 172.21.15.86:0/990846292 shutdown_connections 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.115+0000 7f9c456a8740 1 -- 172.21.15.86:0/990846292 wait complete. 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 -- start start 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 Processor -- start 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55baa8a54d20 con 0x55baa8a43360 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55baa8962f10 con 0x55baa8a52150 2024-09-05T06:42:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.116+0000 7f9c456a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55baa8a54ea0 con 0x55baa8a42a40 2024-09-05T06:42:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c456a8740 10 client.0 _ll_register_callbacks cb 0x55baa892ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4106b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46322/0 (socket says 172.21.15.86:46322) 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 -- 172.21.15.86:0/860615746 learned_addr learned my addr 172.21.15.86:0/860615746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4086a640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 -- 172.21.15.86:0/860615746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 msgr2=0x55baa8a4f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 -- 172.21.15.86:0/860615746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 msgr2=0x55baa8a4f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 -- 172.21.15.86:0/860615746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa87196a0 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4186c640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c340042f0 tx=0x7f9c34004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f74bacd93573488 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c4086a640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.117+0000 7f9c24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400fa00 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c38017a70 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9c3400fba0 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c3400fde0 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c456a8740 10 client.25259 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55baa8a60a50 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.118+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c38017a70 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9c34026430 con 0x55baa8a52150 2024-09-05T06:42:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 10 client.25259 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 10 client.25259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 20 client.25259 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a622d0 0x55baa8a64720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 4 client.25259 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.119+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa89b57f0 con 0x55baa8a622d0 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.120+0000 7f9c4106b640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a622d0 0x55baa8a64720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.120+0000 7f9c4106b640 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a622d0 0x55baa8a64720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7447f088ece3621f server_cookie=c1a8065875d43491 in_seq=0 out_seq=0 2024-09-05T06:42:17.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.120+0000 7f9c24ff9640 10 client.25259 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c24ff9640 1 -- 172.21.15.86:0/860615746 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55baa89b57f0 con 0x55baa8a622d0 2024-09-05T06:42:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c24ff9640 10 client.25259 handle_command_reply: tid=0 2024-09-05T06:42:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 client.25259 shutdown 2024-09-05T06:42:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 msgr2=0x55baa8a4fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c340042f0 tx=0x7f9c34004a40 comp rx=0 tx=0).stop 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 shutdown_connections 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a622d0 0x55baa8a64720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a4fb10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 --2- 172.21.15.86:0/860615746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 >> 172.21.15.86:0/860615746 conn(0x55baa8a3f0f0 msgr2=0x55baa8a44e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c24ff9640 0 client.25259 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 shutdown_connections 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 1 -- 172.21.15.86:0/860615746 wait complete. 2024-09-05T06:42:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.121+0000 7f9c456a8740 20 client.25259 trim_cache size 0 max 16384 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 Processor -- start 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 -- start start 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b5620 con 0x55baa8a43360 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b7160 con 0x55baa8a52150 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c456a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b5cf0 con 0x55baa8a42a40 2024-09-05T06:42:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4086a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4086a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42040/0 (socket says 172.21.15.86:42040) 2024-09-05T06:42:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4086a640 1 -- 172.21.15.86:0/3143861016 learned_addr learned my addr 172.21.15.86:0/3143861016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4186c640 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 -- 172.21.15.86:0/3143861016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 msgr2=0x55baa8a6a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a6a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 -- 172.21.15.86:0/3143861016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 msgr2=0x55baa8a4fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.139+0000 7f9c4106b640 1 -- 172.21.15.86:0/3143861016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa8a43150 con 0x55baa8a42a40 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.140+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c38019040 tx=0x7f9c3800ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c5aeb71b8dca3a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.140+0000 7f9c26ffd640 1 -- 172.21.15.86:0/3143861016 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3802e070 con 0x55baa8a42a40 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.140+0000 7f9c26ffd640 1 -- 172.21.15.86:0/3143861016 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c3800fb40 con 0x55baa8a42a40 2024-09-05T06:42:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.140+0000 7f9c26ffd640 1 -- 172.21.15.86:0/3143861016 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38028780 con 0x55baa8a42a40 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- 172.21.15.86:0/3143861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 msgr2=0x55baa8a4f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c38019040 tx=0x7f9c3800ba30 comp rx=0 tx=0).stop 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- 172.21.15.86:0/3143861016 shutdown_connections 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a52150 0x55baa8a6a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a43360 0x55baa8a4fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3143861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a42a40 0x55baa8a4f5e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- 172.21.15.86:0/3143861016 >> 172.21.15.86:0/3143861016 conn(0x55baa8a3f0f0 msgr2=0x55baa8a51000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- 172.21.15.86:0/3143861016 shutdown_connections 2024-09-05T06:42:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- 172.21.15.86:0/3143861016 wait complete. 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 -- start start 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 Processor -- start 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.141+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a55cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42046/0 (socket says 172.21.15.86:42046) 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 -- 172.21.15.86:0/3049155923 learned_addr learned my addr 172.21.15.86:0/3049155923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a56280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b6a90 con 0x55baa8a42a40 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b63c0 con 0x55baa8a43360 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b3600 con 0x55baa8a52150 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c456a8740 10 client.0 _ll_register_callbacks cb 0x7f9c3c074370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a55cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4186c640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a56280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 -- 172.21.15.86:0/3049155923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 msgr2=0x55baa8a56280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a56280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 -- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 msgr2=0x55baa8a55cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a55cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa8a56ab0 con 0x55baa8a42a40 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c4106b640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800ba00 tx=0x7f9c3800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46f4481d83bbdf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.142+0000 7f9c24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3802e070 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9c3800f8a0 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c3800fb40 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c3803d050 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 10 client.15789 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55baa8a45a30 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c3800fb40 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9c380175e0 con 0x55baa8a42a40 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 20 client.15789 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a45f40 0x55baa8a48360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 4 client.15789 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:42:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.143+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55baa89afd00 con 0x55baa8a45f40 2024-09-05T06:42:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.144+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a45f40 0x55baa8a48360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.144+0000 7f9c4086a640 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a45f40 0x55baa8a48360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1180c43bf4da4e server_cookie=8a82c95276d5c2f8 in_seq=0 out_seq=0 2024-09-05T06:42:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.144+0000 7f9c24ff9640 10 client.15789 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.145+0000 7f9c24ff9640 1 -- 172.21.15.86:0/3049155923 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55baa89afd00 con 0x55baa8a45f40 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.145+0000 7f9c24ff9640 10 client.15789 handle_command_reply: tid=0 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.145+0000 7f9c456a8740 1 client.15789 shutdown 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.145+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 msgr2=0x55baa8a5a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.145+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800ba00 tx=0x7f9c3800b590 comp rx=0 tx=0).stop 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 shutdown_connections 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baa8a45f40 0x55baa8a48360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baa8a52150 0x55baa8a56280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baa8a43360 0x55baa8a55cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 --2- 172.21.15.86:0/3049155923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baa8a42a40 0x55baa8a5a980 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 >> 172.21.15.86:0/3049155923 conn(0x55baa8a3f0f0 msgr2=0x55baa8a50770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c24ff9640 0 client.15789 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 shutdown_connections 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 1 -- 172.21.15.86:0/3049155923 wait complete. 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.146+0000 7f9c456a8740 20 client.15789 trim_cache size 0 max 16384 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "389dee40-80ba-424e-9520-f2ce4c92e81f", 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:42:17.149 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 msgr2=0x7f9c280a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 0x7f9c280a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007fd0 tx=0x7f9c34007520 comp rx=0 tx=0).stop 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 msgr2=0x7f9c3c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c0042f0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 shutdown_connections 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9c280a2e90 0x7f9c280a5350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c3c10c180 0x7f9c3c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c3c103670 0x7f9c3c06bce0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 --2- 172.21.15.86:0/3432391315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c3c102d50 0x7f9c3c073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 >> 172.21.15.86:0/3432391315 conn(0x7f9c3c0fc7d0 msgr2=0x7f9c3c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 shutdown_connections 2024-09-05T06:42:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:17.147+0000 7f9c432cf640 1 -- 172.21.15.86:0/3432391315 wait complete. 2024-09-05T06:42:17.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:17 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:42:17.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:42:17.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:17 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:42:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:42:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:17 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:42:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:42:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:42:17.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:42:18.195 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:18.606 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:18 smithi086 ceph-mon[30943]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-05T06:42:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:18 smithi115 ceph-mon[34791]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.732+0000 7fbed8285640 1 -- 172.21.15.86:0/2363369828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0106a60 msgr2=0x7fbed0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.732+0000 7fbed8285640 1 --2- 172.21.15.86:0/2363369828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0106a60 0x7fbed0106e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009930 tx=0x7fbecc02a420 comp rx=0 tx=0).stop 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 -- 172.21.15.86:0/2363369828 shutdown_connections 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 --2- 172.21.15.86:0/2363369828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0101400 0x7fbed010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 --2- 172.21.15.86:0/2363369828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0100a60 0x7fbed0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 --2- 172.21.15.86:0/2363369828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0106a60 0x7fbed0106e40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 -- 172.21.15.86:0/2363369828 >> 172.21.15.86:0/2363369828 conn(0x7fbed00fc780 msgr2=0x7fbed00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 -- 172.21.15.86:0/2363369828 shutdown_connections 2024-09-05T06:42:18.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.733+0000 7fbed8285640 1 -- 172.21.15.86:0/2363369828 wait complete. 2024-09-05T06:42:18.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 Processor -- start 2024-09-05T06:42:18.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 -- start start 2024-09-05T06:42:18.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0101400 0x7fbed010e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0106a60 0x7fbed010ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0111e30 con 0x7fbed0100a60 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0111cb0 con 0x7fbed0106a60 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed8285640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0111fb0 con 0x7fbed0101400 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed6021640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed6021640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42066/0 (socket says 172.21.15.86:42066) 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.734+0000 7fbed6021640 1 -- 172.21.15.86:0/134696641 learned_addr learned my addr 172.21.15.86:0/134696641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6822640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0106a60 0x7fbed010ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed5820640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0101400 0x7fbed010e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 -- 172.21.15.86:0/134696641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0101400 msgr2=0x7fbed010e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0101400 0x7fbed010e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 -- 172.21.15.86:0/134696641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0106a60 msgr2=0x7fbed010ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0106a60 0x7fbed010ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 -- 172.21.15.86:0/134696641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01a6610 con 0x7fbed0100a60 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbed6021640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02a940 tx=0x7fbecc030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.735+0000 7fbec77fe640 1 -- 172.21.15.86:0/134696641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc03f070 con 0x7fbed0100a60 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.736+0000 7fbec77fe640 1 -- 172.21.15.86:0/134696641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbecc004460 con 0x7fbed0100a60 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.736+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a68a0 con 0x7fbed0100a60 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.736+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a6d10 con 0x7fbed0100a60 2024-09-05T06:42:18.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.737+0000 7fbec77fe640 1 -- 172.21.15.86:0/134696641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc035030 con 0x7fbed0100a60 2024-09-05T06:42:18.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.739+0000 7fbec77fe640 1 -- 172.21.15.86:0/134696641 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fbecc042070 con 0x7fbed0100a60 2024-09-05T06:42:18.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.739+0000 7fbeda65e740 1 Processor -- start 2024-09-05T06:42:18.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.739+0000 7fbeda65e740 1 -- start start 2024-09-05T06:42:18.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802affdc0 0x55a802b0a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b0ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a802a652e0 con 0x55a802b0a680 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a802a0cce0 con 0x55a802affdc0 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbeda65e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a802a612a0 con 0x55a802aff4a0 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbed6822640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbed6822640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48340/0 (socket says 172.21.15.86:48340) 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbed6822640 1 -- 172.21.15.86:0/3902386785 learned_addr learned my addr 172.21.15.86:0/3902386785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbed6021640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802affdc0 0x55a802b0a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.740+0000 7fbed5820640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b0ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 -- 172.21.15.86:0/3902386785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802affdc0 msgr2=0x55a802b0a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802affdc0 0x55a802b0a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 -- 172.21.15.86:0/3902386785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 msgr2=0x55a802b0ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b0ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 -- 172.21.15.86:0/3902386785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8029f1440 con 0x55a802aff4a0 2024-09-05T06:42:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed5820640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b0ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbec77fe640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 0x7fbebc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6822640 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009f70 tx=0x7fbec8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c0d6b4e39f126c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbed6021640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 0x7fbebc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbec4dea640 1 -- 172.21.15.86:0/3902386785 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8016070 con 0x55a802aff4a0 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbec4dea640 1 -- 172.21.15.86:0/3902386785 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec8004d20 con 0x55a802aff4a0 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.741+0000 7fbec4dea640 1 -- 172.21.15.86:0/3902386785 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec80127d0 con 0x55a802aff4a0 2024-09-05T06:42:18.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.742+0000 7fbec77fe640 1 -- 172.21.15.86:0/134696641 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbecc0f06d0 con 0x7fbed0100a60 2024-09-05T06:42:18.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.742+0000 7fbeda65e740 1 -- 172.21.15.86:0/3902386785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 msgr2=0x55a802aff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.742+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009f70 tx=0x7fbec8010800 comp rx=0 tx=0).stop 2024-09-05T06:42:18.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.742+0000 7fbed6021640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 0x7fbebc0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ed920 tx=0x7fbecc0f6510 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 -- 172.21.15.86:0/3902386785 shutdown_connections 2024-09-05T06:42:18.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b0ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802affdc0 0x55a802b0a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3902386785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802aff4a0 0x55a802aff880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 -- 172.21.15.86:0/3902386785 >> 172.21.15.86:0/3902386785 conn(0x55a802afba30 msgr2=0x55a802afbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 -- 172.21.15.86:0/3902386785 shutdown_connections 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 -- 172.21.15.86:0/3902386785 wait complete. 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 -- start start 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.743+0000 7fbeda65e740 1 Processor -- start 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b07840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a802b0d250 con 0x55a802affdc0 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a802a20050 con 0x55a802aff4a0 2024-09-05T06:42:18.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a802b0d3d0 con 0x55a802b0a680 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbed6822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbeda65e740 10 client.0 _ll_register_callbacks cb 0x55a802a60160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbed6822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46364/0 (socket says 172.21.15.86:46364) 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbed6822640 1 -- 172.21.15.86:0/1975426630 learned_addr learned my addr 172.21.15.86:0/1975426630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbed6822640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b07840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.744+0000 7fbed5820640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 -- 172.21.15.86:0/1975426630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 msgr2=0x55a802b07840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b07840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 -- 172.21.15.86:0/1975426630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 msgr2=0x55a802b072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 -- 172.21.15.86:0/1975426630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8027d63c0 con 0x55a802aff4a0 2024-09-05T06:42:18.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed5820640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbed6822640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009d30 tx=0x7fbec8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2df5b06a03178c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbeb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec801e020 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec8004d20 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.745+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbec8012c00 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbec8016070 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 10 client.25277 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a802b18d30 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec80096f0 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbec800f040 con 0x55a802aff4a0 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 10 client.25277 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:18.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 10 client.25277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 20 client.25277 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b1a1f0 0x55a802b1c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 4 client.25277 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.746+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbec8004d20 con 0x55a802b1a1f0 2024-09-05T06:42:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.747+0000 7fbed5820640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b1a1f0 0x55a802b1c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.747+0000 7fbed5820640 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b1a1f0 0x55a802b1c5e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c888dc76007da27 server_cookie=1d454aeceb5f4f2f in_seq=0 out_seq=0 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.747+0000 7fbeb9ffb640 10 client.25277 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/1975426630 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbec8004d20 con 0x55a802b1a1f0 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeb9ffb640 10 client.25277 handle_command_reply: tid=0 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 client.25277 shutdown 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 msgr2=0x55a802b0a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009d30 tx=0x7fbec8005830 comp rx=0 tx=0).stop 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 shutdown_connections 2024-09-05T06:42:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b1a1f0 0x55a802b1c5e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b07840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 --2- 172.21.15.86:0/1975426630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b0a010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 >> 172.21.15.86:0/1975426630 conn(0x55a802afba30 msgr2=0x55a802b01710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeb9ffb640 0 client.25277 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 shutdown_connections 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 1 -- 172.21.15.86:0/1975426630 wait complete. 2024-09-05T06:42:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.748+0000 7fbeda65e740 20 client.25277 trim_cache size 0 max 16384 2024-09-05T06:42:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 Processor -- start 2024-09-05T06:42:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 -- start start 2024-09-05T06:42:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802afaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb400ffc0 con 0x55a802affdc0 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4011b00 con 0x55a802aff4a0 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbeda65e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4010690 con 0x55a802b0a680 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbed5820640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbed5820640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42094/0 (socket says 172.21.15.86:42094) 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbed5820640 1 -- 172.21.15.86:0/3602930973 learned_addr learned my addr 172.21.15.86:0/3602930973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.782+0000 7fbed6822640 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802afaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed6822640 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 -- 172.21.15.86:0/3602930973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 msgr2=0x55a802b221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 -- 172.21.15.86:0/3602930973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 msgr2=0x55a802afaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802afaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 -- 172.21.15.86:0/3602930973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a802b07e40 con 0x55a802affdc0 2024-09-05T06:42:18.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbed5820640 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbed010fe50 tx=0x7fbec0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96d540f4ba1aa7f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbebbfff640 1 -- 172.21.15.86:0/3602930973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec00120d0 con 0x55a802affdc0 2024-09-05T06:42:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.783+0000 7fbebbfff640 1 -- 172.21.15.86:0/3602930973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec0012270 con 0x55a802affdc0 2024-09-05T06:42:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbebbfff640 1 -- 172.21.15.86:0/3602930973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec00107b0 con 0x55a802affdc0 2024-09-05T06:42:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 -- 172.21.15.86:0/3602930973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 msgr2=0x55a802b21c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbed010fe50 tx=0x7fbec0005da0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 -- 172.21.15.86:0/3602930973 shutdown_connections 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0a680 0x55a802b221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802affdc0 0x55a802b21c70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.784+0000 7fbeda65e740 1 --2- 172.21.15.86:0/3602930973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802afaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.785+0000 7fbeda65e740 1 -- 172.21.15.86:0/3602930973 >> 172.21.15.86:0/3602930973 conn(0x55a802afba30 msgr2=0x55a802afa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.785+0000 7fbeda65e740 1 -- 172.21.15.86:0/3602930973 shutdown_connections 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.785+0000 7fbeda65e740 1 -- 172.21.15.86:0/3602930973 wait complete. 2024-09-05T06:42:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.785+0000 7fbeda65e740 1 -- start start 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.785+0000 7fbeda65e740 1 Processor -- start 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 0x55a802b10360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4011430 con 0x55a802b0a680 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4010d60 con 0x55a802aff4a0 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb400dfa0 con 0x55a802b0fee0 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbeda65e740 10 client.0 _ll_register_callbacks cb 0x7fbecc0f63e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed5820640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed5820640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42108/0 (socket says 172.21.15.86:42108) 2024-09-05T06:42:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed5820640 1 -- 172.21.15.86:0/786085819 learned_addr learned my addr 172.21.15.86:0/786085819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 0x55a802b10360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 -- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 msgr2=0x55a802b12a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b12a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 -- 172.21.15.86:0/786085819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 msgr2=0x55a802b12f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed6822640 1 -- 172.21.15.86:0/786085819 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a802b10b90 con 0x55a802b0fee0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.786+0000 7fbed5820640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbed6822640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 0x55a802b10360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000f80 tx=0x7fbec8005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40c50e738f953d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012c00 con 0x55a802b0fee0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbec8023040 con 0x55a802b0fee0 2024-09-05T06:42:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec0012270 con 0x55a802b0fee0 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbec8004c50 con 0x55a802b0fee0 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeda65e740 10 client.25243 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.787+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a802b84cd0 con 0x55a802b0fee0 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec0012270 con 0x55a802b0fee0 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbec8029630 con 0x55a802b0fee0 2024-09-05T06:42:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 10 client.25243 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 10 client.25243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 20 client.25243 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b852e0 0x55a802b876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 4 client.25243 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a802a72850 con 0x55a802b852e0 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.788+0000 7fbed5820640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b852e0 0x55a802b876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeb9ffb640 10 client.25243 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbed5820640 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b852e0 0x55a802b876d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d19b427909ff1a9 server_cookie=a2cd8161b89efede in_seq=0 out_seq=0 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeb9ffb640 1 -- 172.21.15.86:0/786085819 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a802a72850 con 0x55a802b852e0 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeb9ffb640 10 client.25243 handle_command_reply: tid=0 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeda65e740 1 client.25243 shutdown 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 msgr2=0x55a802b10360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 0x55a802b10360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000f80 tx=0x7fbec8005a40 comp rx=0 tx=0).stop 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 shutdown_connections 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.789+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a802b852e0 0x55a802b876d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a802b0fee0 0x55a802b10360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeb9ffb640 0 client.25243 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a802b0a680 0x55a802b12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 --2- 172.21.15.86:0/786085819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a802aff4a0 0x55a802b12a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 >> 172.21.15.86:0/786085819 conn(0x55a802afba30 msgr2=0x55a802afab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "389dee40-80ba-424e-9520-f2ce4c92e81f": { 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "389dee40-80ba-424e-9520-f2ce4c92e81f", 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:18.793 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 shutdown_connections 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 1 -- 172.21.15.86:0/786085819 wait complete. 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbeda65e740 20 client.25243 trim_cache size 0 max 16384 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 msgr2=0x7fbebc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 0x7fbebc0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ed920 tx=0x7fbecc0f6510 comp rx=0 tx=0).stop 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 msgr2=0x7fbed010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.790+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02a940 tx=0x7fbecc030650 comp rx=0 tx=0).stop 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 shutdown_connections 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbebc0a2e90 0x7fbebc0a5350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbed0106a60 0x7fbed010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed0101400 0x7fbed010e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 --2- 172.21.15.86:0/134696641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed0100a60 0x7fbed010e3c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 >> 172.21.15.86:0/134696641 conn(0x7fbed00fc780 msgr2=0x7fbed00fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 shutdown_connections 2024-09-05T06:42:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:18.791+0000 7fbed8285640 1 -- 172.21.15.86:0/134696641 wait complete. 2024-09-05T06:42:18.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:18 smithi096 ceph-mon[34468]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-05T06:42:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:19 smithi086 ceph-mon[30943]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-05T06:42:19.593 INFO:tasks.cephfs.filesystem:scrub status for tag:389dee40-80ba-424e-9520-f2ce4c92e81f - {'path': '/', 'tag': '389dee40-80ba-424e-9520-f2ce4c92e81f', 'options': 'recursive,force'} 2024-09-05T06:42:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:19 smithi115 ceph-mon[34791]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-05T06:42:19.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:19 smithi096 ceph-mon[34468]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-05T06:42:20.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:20 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:42:20.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:20 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:42:20.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:20 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:42:21.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:21 smithi086 ceph-mon[30943]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-05T06:42:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:21 smithi096 ceph-mon[34468]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-05T06:42:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:21 smithi115 ceph-mon[34791]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-05T06:42:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:24 smithi115 ceph-mon[34791]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:24 smithi086 ceph-mon[30943]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:24.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:24 smithi096 ceph-mon[34468]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:26 smithi115 ceph-mon[34791]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:26.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:26 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:42:26.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:26 smithi086 ceph-mon[30943]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:26.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:26 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:42:26.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:26 smithi096 ceph-mon[34468]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:26 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:42:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:42:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:42:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:27 smithi115 ceph-mon[34791]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:27 smithi086 ceph-mon[30943]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:42:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:42:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:42:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:27 smithi096 ceph-mon[34468]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:30 smithi115 ceph-mon[34791]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:30.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:30 smithi086 ceph-mon[30943]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:30 smithi096 ceph-mon[34468]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-05T06:42:32.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:42:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:32.681 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:32 smithi086 ceph-mon[30943]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:32 smithi115 ceph-mon[34791]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:32.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:32 smithi096 ceph-mon[34468]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-05T06:42:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 -- 172.21.15.86:0/914963389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 msgr2=0x7f550410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 --2- 172.21.15.86:0/914963389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550410ec50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009cd0 tx=0x7f54fc02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 -- 172.21.15.86:0/914963389 shutdown_connections 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 --2- 172.21.15.86:0/914963389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550410ec50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 --2- 172.21.15.86:0/914963389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 0x7f5504107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 --2- 172.21.15.86:0/914963389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 -- 172.21.15.86:0/914963389 >> 172.21.15.86:0/914963389 conn(0x7f55040fc7d0 msgr2=0x7f55040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 -- 172.21.15.86:0/914963389 shutdown_connections 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.915+0000 7f550b425640 1 -- 172.21.15.86:0/914963389 wait complete. 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.916+0000 7f550b425640 1 Processor -- start 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.916+0000 7f550b425640 1 -- start start 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.916+0000 7f550b425640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.916+0000 7f550b425640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 0x7f5504198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f550b425640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f550b425640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111f60 con 0x7f55041082e0 2024-09-05T06:42:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f550b425640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111de0 con 0x7f5504102db0 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f550b425640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f55041120e0 con 0x7f5504103760 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47458/0 (socket says 172.21.15.86:47458) 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 -- 172.21.15.86:0/3931599913 learned_addr learned my addr 172.21.15.86:0/3931599913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55089c0640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 0x7f5504198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55091c1640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 -- 172.21.15.86:0/3931599913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 msgr2=0x7f5504198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 0x7f5504198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 -- 172.21.15.86:0/3931599913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 msgr2=0x7f5504198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 -- 172.21.15.86:0/3931599913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550419d4b0 con 0x7f55041082e0 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55091c1640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f55099c2640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0097b0 tx=0x7f54fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.917+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc004240 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.918+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550419d740 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.918+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54fc0305a0 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.918+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc045650 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.918+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55041a5080 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.919+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5504103190 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.921+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f54fc043020 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.922+0000 7f54fa7fc640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 0x7f54f00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.922+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f54fc02e030 con 0x7f55041082e0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.922+0000 7f55091c1640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 0x7f54f00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.923+0000 7f55091c1640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 0x7f54f00a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f550000abe0 tx=0x7f550000a5c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:32.925+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54fc04a060 con 0x7f55041082e0 2024-09-05T06:42:33.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.313+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5504199860 con 0x7f55041082e0 2024-09-05T06:42:33.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.314+0000 7f54fa7fc640 1 -- 172.21.15.86:0/3931599913 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f5504199860 con 0x7f55041082e0 2024-09-05T06:42:33.315 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:42:33.315 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:42:33.316 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:42:33.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 msgr2=0x7f54f00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:33.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 0x7f54f00a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f550000abe0 tx=0x7f550000a5c0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 msgr2=0x7f550419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:33.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0097b0 tx=0x7f54fc003ba0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 shutdown_connections 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2e90 0x7f54f00a5350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55041082e0 0x7f550419cdd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504103760 0x7f5504198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 --2- 172.21.15.86:0/3931599913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102db0 0x7f5504198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.316+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 >> 172.21.15.86:0/3931599913 conn(0x7f55040fc7d0 msgr2=0x7f5504105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.317+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 shutdown_connections 2024-09-05T06:42:33.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:33.317+0000 7f550b425640 1 -- 172.21.15.86:0/3931599913 wait complete. 2024-09-05T06:42:33.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:33 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3931599913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:33.695 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3931599913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:33 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3931599913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:34.058 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id c2f055af --release --await' 2024-09-05T06:42:34.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id c2f055af --release --await 2024-09-05T06:42:34.431 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:34 smithi086 ceph-mon[30943]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:34.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:34 smithi115 ceph-mon[34791]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:34.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:34 smithi096 ceph-mon[34468]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.955+0000 7faf4c6e9640 1 -- 172.21.15.86:0/637180177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf44102d30 msgr2=0x7faf44103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.955+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/637180177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf44102d30 0x7faf44103110 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faf40009930 tx=0x7faf4002a420 comp rx=0 tx=0).stop 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- 172.21.15.86:0/637180177 shutdown_connections 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/637180177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441082a0 0x7faf4410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/637180177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf441036e0 0x7faf44107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/637180177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf44102d30 0x7faf44103110 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- 172.21.15.86:0/637180177 >> 172.21.15.86:0/637180177 conn(0x7faf440fc7d0 msgr2=0x7faf440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- 172.21.15.86:0/637180177 shutdown_connections 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- 172.21.15.86:0/637180177 wait complete. 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 Processor -- start 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- start start 2024-09-05T06:42:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 0x7faf44100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 0x7faf44100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf44111f50 con 0x7faf441082a0 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf44111dd0 con 0x7faf44102d30 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.956+0000 7faf4c6e9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf441120d0 con 0x7faf441036e0 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47466/0 (socket says 172.21.15.86:47466) 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 -- 172.21.15.86:0/296192552 learned_addr learned my addr 172.21.15.86:0/296192552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf49c84640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 0x7faf44100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4a485640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 0x7faf44100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 -- 172.21.15.86:0/296192552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 msgr2=0x7faf44100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 0x7faf44100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 -- 172.21.15.86:0/296192552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 msgr2=0x7faf44100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 0x7faf44100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 -- 172.21.15.86:0/296192552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf441a6990 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf49c84640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 0x7faf44100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4a485640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 0x7faf44100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4ac86640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00eb10 tx=0x7faf3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf3b7fe640 1 -- 172.21.15.86:0/296192552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c004b20 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf441a6c80 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf3b7fe640 1 -- 172.21.15.86:0/296192552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf3c004cc0 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf3b7fe640 1 -- 172.21.15.86:0/296192552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c010690 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.957+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441a7210 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf3b7fe640 1 -- 172.21.15.86:0/296192552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7faf3c005020 con 0x7faf441082a0 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 Processor -- start 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 -- start start 2024-09-05T06:42:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0060 0x55b4905e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b490545620 con 0x55b4905eb240 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b4904ece50 con 0x55b4905e0060 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.960+0000 7faf4eac2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4905415e0 con 0x55b4905e0980 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf3b7fe640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 0x7faf300a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47480/0 (socket says 172.21.15.86:47480) 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf49c84640 1 -- 172.21.15.86:0/977439282 learned_addr learned my addr 172.21.15.86:0/977439282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf49c84640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0060 0x55b4905e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4ac86640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 0x7faf300a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 -- 172.21.15.86:0/977439282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0060 msgr2=0x55b4905e0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0060 0x55b4905e0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 -- 172.21.15.86:0/977439282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 msgr2=0x55b4905ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 -- 172.21.15.86:0/977439282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4904ddd40 con 0x55b4905e0980 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf3b7fe640 1 -- 172.21.15.86:0/296192552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf3c0d0280 con 0x7faf441082a0 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf49c84640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4a485640 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905ead00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf40000c00 tx=0x7faf40032c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9aa795c8784d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.961+0000 7faf4ac86640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 0x7faf300a52d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0cd4d0 tx=0x7faf3c005b40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf38ff9640 1 -- 172.21.15.86:0/977439282 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4003f070 con 0x55b4905e0980 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf38ff9640 1 -- 172.21.15.86:0/977439282 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf40004030 con 0x55b4905e0980 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf38ff9640 1 -- 172.21.15.86:0/977439282 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40035070 con 0x55b4905e0980 2024-09-05T06:42:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 -- 172.21.15.86:0/977439282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 msgr2=0x55b4905ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905ead00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf40000c00 tx=0x7faf40032c10 comp rx=0 tx=0).stop 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 -- 172.21.15.86:0/977439282 shutdown_connections 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905eb240 0x55b4905ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905ead00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 --2- 172.21.15.86:0/977439282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0060 0x55b4905e0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.962+0000 7faf4eac2740 1 -- 172.21.15.86:0/977439282 >> 172.21.15.86:0/977439282 conn(0x55b4905dc5f0 msgr2=0x55b4905dc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- 172.21.15.86:0/977439282 shutdown_connections 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- 172.21.15.86:0/977439282 wait complete. 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- start start 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 Processor -- start 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 0x55b4905e8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b4905ede10 con 0x55b4905e0980 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b4904cdf10 con 0x55b4905eb240 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4905edf90 con 0x55b4905e0060 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4eac2740 10 client.0 _ll_register_callbacks cb 0x55b4904f1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4a485640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4a485640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47494/0 (socket says 172.21.15.86:47494) 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf49c84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 0x55b4905e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.963+0000 7faf4a485640 1 -- 172.21.15.86:0/2185635122 learned_addr learned my addr 172.21.15.86:0/2185635122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf49c84640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 -- 172.21.15.86:0/2185635122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 msgr2=0x55b4905e8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 0x55b4905e8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 -- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 msgr2=0x55b4905e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 -- 172.21.15.86:0/2185635122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b490254580 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf49c84640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 0x55b4905e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4a485640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faf400092a0 tx=0x7faf40004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a72b100f5d5010 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4004b020 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf400051e0 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf40004030 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf40056050 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4eac2740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b49031e960 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.964+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf40004030 con 0x55b4905e0980 2024-09-05T06:42:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faf400035f0 con 0x55b4905e0980 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 10 client.15825 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 20 client.15825 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4905fab70 0x55b4905fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 4 client.15825 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b490552b90 con 0x55b4905fab70 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.965+0000 7faf49c84640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4905fab70 0x55b4905fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf2dffb640 10 client.15825 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf49c84640 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4905fab70 0x55b4905fcf60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bbaeabe8a493c4 server_cookie=ddb51fb6c59943ce in_seq=0 out_seq=0 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf2dffb640 1 -- 172.21.15.86:0/2185635122 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b490552b90 con 0x55b4905fab70 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf2dffb640 10 client.15825 handle_command_reply: tid=0 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf4eac2740 1 client.15825 shutdown 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 msgr2=0x55b4905e85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faf400092a0 tx=0x7faf40004490 comp rx=0 tx=0).stop 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 shutdown_connections 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4905fab70 0x55b4905fcf60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.966+0000 7faf2dffb640 0 client.15825 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0980 0x55b4905e85a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2185635122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0060 0x55b4905e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 >> 172.21.15.86:0/2185635122 conn(0x55b4905dc5f0 msgr2=0x55b4905e22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 shutdown_connections 2024-09-05T06:42:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 1 -- 172.21.15.86:0/2185635122 wait complete. 2024-09-05T06:42:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.967+0000 7faf4eac2740 20 client.15825 trim_cache size 0 max 16384 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 Processor -- start 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 -- start start 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 0x55b490602560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905eb240 0x55b490602aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf441b12f0 con 0x55b4905e0060 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf441b2e30 con 0x55b4905e0980 2024-09-05T06:42:34.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4eac2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf441b19c0 con 0x55b4905eb240 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47508/0 (socket says 172.21.15.86:47508) 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 -- 172.21.15.86:0/2316082219 learned_addr learned my addr 172.21.15.86:0/2316082219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 -- 172.21.15.86:0/2316082219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905eb240 msgr2=0x55b490602aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4a485640 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 0x55b490602560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905eb240 0x55b490602aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 -- 172.21.15.86:0/2316082219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 msgr2=0x55b490602560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 0x55b490602560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf49c84640 1 -- 172.21.15.86:0/2316082219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4905dc420 con 0x55b4905e0060 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.989+0000 7faf4a485640 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 0x55b490602560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf49c84640 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55b4905fd500 tx=0x7faf34005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8ba898cba56836 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf2ffff640 1 -- 172.21.15.86:0/2316082219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340140b0 con 0x55b4905e0060 2024-09-05T06:42:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf2ffff640 1 -- 172.21.15.86:0/2316082219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf34026020 con 0x55b4905e0060 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf2ffff640 1 -- 172.21.15.86:0/2316082219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34012950 con 0x55b4905e0060 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf4eac2740 1 -- 172.21.15.86:0/2316082219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 msgr2=0x55b4905db640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.990+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55b4905fd500 tx=0x7faf34005850 comp rx=0 tx=0).stop 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- 172.21.15.86:0/2316082219 shutdown_connections 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905eb240 0x55b490602aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905e0980 0x55b490602560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- 172.21.15.86:0/2316082219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905db640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- 172.21.15.86:0/2316082219 >> 172.21.15.86:0/2316082219 conn(0x55b4905dc5f0 msgr2=0x55b4905daf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- 172.21.15.86:0/2316082219 shutdown_connections 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- 172.21.15.86:0/2316082219 wait complete. 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- start start 2024-09-05T06:42:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 Processor -- start 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf441b2760 con 0x55b4905e0060 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf441b2090 con 0x55b4905eb240 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf441af2d0 con 0x55b4905e0980 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf4eac2740 10 client.0 _ll_register_callbacks cb 0x7faf40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.991+0000 7faf49c84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47512/0 (socket says 172.21.15.86:47512) 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 -- 172.21.15.86:0/1731885152 learned_addr learned my addr 172.21.15.86:0/1731885152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf4a485640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 -- 172.21.15.86:0/1731885152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 msgr2=0x55b4905f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 -- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 msgr2=0x55b4905ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 -- 172.21.15.86:0/1731885152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4905ee6e0 con 0x55b4905e0060 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf4a485640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf49c84640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf340143d0 tx=0x7faf3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8aab4ce9c0ab7c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34003ca0 con 0x55b4905e0060 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf34022e80 con 0x55b4905e0060 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf34026020 con 0x55b4905e0060 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf34012450 con 0x55b4905e0060 2024-09-05T06:42:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf4eac2740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.992+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b490665b90 con 0x55b4905e0060 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf34026020 con 0x55b4905e0060 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faf3402d030 con 0x55b4905e0060 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 10 client.15831 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 20 client.15831 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4906661d0 0x55b4906685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 4 client.15831 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2f055af", "release": true, "await": true}] 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2f055af", "release": true, "await": true}) -- 0x55b49054d0a0 con 0x55b4906661d0 2024-09-05T06:42:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4a485640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4906661d0 0x55b4906685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf2dffb640 10 client.15831 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.993+0000 7faf4a485640 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4906661d0 0x55b4906685f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c7b9313f819dde server_cookie=b6c4327b3a8e92dd in_seq=0 out_seq=0 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.995+0000 7faf2dffb640 1 -- 172.21.15.86:0/1731885152 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b49054d0a0 con 0x55b4906661d0 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.995+0000 7faf2dffb640 10 client.15831 handle_command_reply: tid=0 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.995+0000 7faf4eac2740 1 client.15831 shutdown 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 msgr2=0x55b4905f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faf340143d0 tx=0x7faf3400cbf0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 shutdown_connections 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b4906661d0 0x55b4906685f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf2dffb640 0 client.15831 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b4905eb240 0x55b4905ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4905e0980 0x55b4905f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 --2- 172.21.15.86:0/1731885152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4905e0060 0x55b4905f31f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 >> 172.21.15.86:0/1731885152 conn(0x55b4905dc5f0 msgr2=0x55b4905e8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 shutdown_connections 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 1 -- 172.21.15.86:0/1731885152 wait complete. 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4eac2740 20 client.15831 trim_cache size 0 max 16384 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 28, 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 4, 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:42:35.002 INFO:teuthology.orchestra.run.smithi086.stdout: "c2f055af": { 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 148.2, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:42:35.003 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:42:35.004 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 msgr2=0x7faf300a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 0x7faf300a52d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0cd4d0 tx=0x7faf3c005b40 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 msgr2=0x7faf44101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00eb10 tx=0x7faf3c00c2d0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 shutdown_connections 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf300a2e10 0x7faf300a52d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf441082a0 0x7faf44101410 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf441036e0 0x7faf44100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 --2- 172.21.15.86:0/296192552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf44102d30 0x7faf44100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.996+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 >> 172.21.15.86:0/296192552 conn(0x7faf440fc7d0 msgr2=0x7faf44105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.997+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 shutdown_connections 2024-09-05T06:42:35.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:34.997+0000 7faf4c6e9640 1 -- 172.21.15.86:0/296192552 wait complete. 2024-09-05T06:42:35.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:35 smithi086 ceph-mon[30943]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:35 smithi096 ceph-mon[34468]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:35.843 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c2f055af', total seconds elapsed: 31.9 2024-09-05T06:42:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:35 smithi115 ceph-mon[34791]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:38.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:38 smithi115 ceph-mon[34791]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T06:42:38.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:38 smithi086 ceph-mon[30943]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T06:42:38.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:38 smithi096 ceph-mon[34468]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T06:42:39.342 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'. 2024-09-05T06:42:39.342 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-09-05T06:42:39.343 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.344 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-05T06:42:39.344 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-09-05T06:42:39.344 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at 91c8a7e56f7 Merge pull request #59582 from zdover23/wip-doc-2024-09-04-backport-59546-to-squid 2024-09-05T06:42:39.347 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-05T06:42:39.373 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-05T06:42:39.376 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T06:42:39.376 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-05T06:42:40.022 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-05T06:42:40.090 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-05T06:42:40.131 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T06:42:40.133 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T06:42:40.133 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-05T06:42:40.171 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T06:42:40.175 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T06:42:40.175 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-05T06:42:40.231 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-05T06:42:40.232 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-05T06:42:40.232 DEBUG:teuthology.orchestra.run.smithi086:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-05T06:42:40.301 INFO:tasks.workunit.client.0.smithi086.stderr:+ dbench 1 2024-09-05T06:42:40.403 INFO:tasks.workunit.client.0.smithi086.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-05T06:42:40.403 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T06:42:40.403 INFO:tasks.workunit.client.0.smithi086.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-05T06:42:40.404 INFO:tasks.workunit.client.0.smithi086.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-05T06:42:40.504 INFO:tasks.workunit.client.0.smithi086.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-05T06:42:40.504 INFO:tasks.workunit.client.0.smithi086.stdout:releasing clients 2024-09-05T06:42:40.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:40 smithi115 ceph-mon[34791]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:40.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:40 smithi086 ceph-mon[30943]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:40.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:40 smithi096 ceph-mon[34468]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T06:42:41.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 970 21.82 MB/sec warmup 1 sec latency 12.808 ms 2024-09-05T06:42:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:41 smithi115 ceph-mon[34791]: pgmap v46: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-09-05T06:42:41.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:41 smithi086 ceph-mon[30943]: pgmap v46: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-09-05T06:42:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:41 smithi096 ceph-mon[34468]: pgmap v46: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.7 MiB/s wr, 25 op/s 2024-09-05T06:42:42.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 1826 13.17 MB/sec warmup 2 sec latency 14.089 ms 2024-09-05T06:42:43.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 2582 9.65 MB/sec warmup 3 sec latency 16.049 ms 2024-09-05T06:42:44.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 3395 8.87 MB/sec warmup 4 sec latency 16.307 ms 2024-09-05T06:42:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:44 smithi115 ceph-mon[34791]: pgmap v47: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:44.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:44 smithi086 ceph-mon[30943]: pgmap v47: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:44.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:44 smithi096 ceph-mon[34468]: pgmap v47: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:45.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 4070 8.02 MB/sec warmup 5 sec latency 12.864 ms 2024-09-05T06:42:46.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 4756 7.03 MB/sec warmup 6 sec latency 13.049 ms 2024-09-05T06:42:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:46 smithi115 ceph-mon[34791]: pgmap v48: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:46.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:46 smithi086 ceph-mon[30943]: pgmap v48: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:46 smithi096 ceph-mon[34468]: pgmap v48: 129 pgs: 129 active+clean; 22 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-09-05T06:42:47.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 5613 6.53 MB/sec warmup 7 sec latency 18.814 ms 2024-09-05T06:42:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:47 smithi086 ceph-mon[30943]: pgmap v49: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.6 MiB/s wr, 62 op/s 2024-09-05T06:42:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:47 smithi086 ceph-mon[30943]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-05T06:42:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:47 smithi096 ceph-mon[34468]: pgmap v49: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.6 MiB/s wr, 62 op/s 2024-09-05T06:42:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:47 smithi096 ceph-mon[34468]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-05T06:42:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:47 smithi115 ceph-mon[34791]: pgmap v49: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.6 MiB/s wr, 62 op/s 2024-09-05T06:42:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:47 smithi115 ceph-mon[34791]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-05T06:42:48.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 6477 6.58 MB/sec warmup 8 sec latency 25.768 ms 2024-09-05T06:42:49.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 7231 6.43 MB/sec warmup 9 sec latency 11.765 ms 2024-09-05T06:42:49.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:42:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.393+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/1639416268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94101fc0 msgr2=0x7fbb94105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.393+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/1639416268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94101fc0 0x7fbb94105710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009900 tx=0x7fbb8402a410 comp rx=0 tx=0).stop 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/1639416268 shutdown_connections 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/1639416268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94105e40 0x7fbb9410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/1639416268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94101fc0 0x7fbb94105710 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/1639416268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101610 0x7fbb941019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/1639416268 >> 172.21.15.86:0/1639416268 conn(0x7fbb940fc5c0 msgr2=0x7fbb940fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/1639416268 shutdown_connections 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/1639416268 wait complete. 2024-09-05T06:42:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 Processor -- start 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- start start 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94101610 0x7fbb941983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101fc0 0x7fbb941988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94110d50 con 0x7fbb94105e40 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94110bd0 con 0x7fbb94101fc0 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.394+0000 7fbb9c3c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94110ed0 con 0x7fbb94101610 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36792/0 (socket says 172.21.15.86:36792) 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94101610 0x7fbb941983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 -- 172.21.15.86:0/272951562 learned_addr learned my addr 172.21.15.86:0/272951562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb99964640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101fc0 0x7fbb941988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 -- 172.21.15.86:0/272951562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94101610 msgr2=0x7fbb941983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94101610 0x7fbb941983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 -- 172.21.15.86:0/272951562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101fc0 msgr2=0x7fbb941988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101fc0 0x7fbb941988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 -- 172.21.15.86:0/272951562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419d360 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9a966640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d750 tx=0x7fbb8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb8b7fe640 1 -- 172.21.15.86:0/272951562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c017070 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb8b7fe640 1 -- 172.21.15.86:0/272951562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8c0054d0 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419d650 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb8b7fe640 1 -- 172.21.15.86:0/272951562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004c60 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.395+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a4f30 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.398+0000 7fbb8b7fe640 1 -- 172.21.15.86:0/272951562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbb8c004e00 con 0x7fbb94105e40 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.398+0000 7fbb9e7a2740 1 Processor -- start 2024-09-05T06:42:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 -- start start 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x563405906140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563405906680 0x563405908a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634058612e0 con 0x563405906680 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563405808ce0 con 0x5634058fb4a0 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56340585d2a0 con 0x5634058fbdc0 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb8b7fe640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 0x7fbb800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563405906680 0x563405908a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x563405906140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42134/0 (socket says 172.21.15.86:42134) 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 -- 172.21.15.86:0/3725140315 learned_addr learned my addr 172.21.15.86:0/3725140315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb9a966640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 0x7fbb800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb8b7fe640 1 -- 172.21.15.86:0/272951562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbb8c0cf030 con 0x7fbb94105e40 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 -- 172.21.15.86:0/3725140315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 msgr2=0x563405906140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x563405906140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 -- 172.21.15.86:0/3725140315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563405906680 msgr2=0x563405908a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563405906680 0x563405908a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.399+0000 7fbb99964640 1 -- 172.21.15.86:0/3725140315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634057ed440 con 0x5634058fb4a0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb99964640 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406a100 tx=0x7fbb84031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45506d4b312e4660 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb9a966640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 0x7fbb800a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc450 tx=0x7fbb8c0bb040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb88dea640 1 -- 172.21.15.86:0/3725140315 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84035070 con 0x5634058fb4a0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb88dea640 1 -- 172.21.15.86:0/3725140315 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb84004030 con 0x5634058fb4a0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb88dea640 1 -- 172.21.15.86:0/3725140315 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84003a10 con 0x5634058fb4a0 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/3725140315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 msgr2=0x5634058fb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.400+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406a100 tx=0x7fbb84031400 comp rx=0 tx=0).stop 2024-09-05T06:42:50.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/3725140315 shutdown_connections 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563405906680 0x563405908a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x563405906140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/3725140315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x5634058fb880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/3725140315 >> 172.21.15.86:0/3725140315 conn(0x5634058f7a30 msgr2=0x5634058f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/3725140315 shutdown_connections 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/3725140315 wait complete. 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- start start 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 Processor -- start 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 0x563405903260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563405906680 0x5634059037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563405909250 con 0x5634058fbdc0 2024-09-05T06:42:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56340581c050 con 0x5634058fb4a0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5634059093d0 con 0x563405906680 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.401+0000 7fbb9e7a2740 10 client.0 _ll_register_callbacks cb 0x56340585c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 0x563405903260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42150/0 (socket says 172.21.15.86:42150) 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb99964640 1 -- 172.21.15.86:0/1916500297 learned_addr learned my addr 172.21.15.86:0/1916500297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb99964640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563405906680 0x5634059037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 -- 172.21.15.86:0/1916500297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563405906680 msgr2=0x5634059037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563405906680 0x5634059037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 -- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 msgr2=0x563405905fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 -- 172.21.15.86:0/1916500297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634055d23c0 con 0x5634058fbdc0 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb99964640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 0x563405903260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbb90005fd0 tx=0x7fbb9000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd27c46e071f85e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90021030 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbb90004b20 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb84004030 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb9002b050 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9e7a2740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.402+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563405914ce0 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb84004030 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbb9001d590 con 0x5634058fbdc0 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 20 client.15843 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405916240 0x563405918680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 4 client.15843 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56340586e850 con 0x563405916240 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb99964640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405916240 0x563405918680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb7dffb640 10 client.15843 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.403+0000 7fbb99964640 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405916240 0x563405918680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d9a3fd05d79b03 server_cookie=e56a51ddb8a3bffc in_seq=0 out_seq=0 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1916500297 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56340586e850 con 0x563405916240 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb7dffb640 10 client.15843 handle_command_reply: tid=0 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 client.15843 shutdown 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 msgr2=0x563405903260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 0x563405903260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbb90005fd0 tx=0x7fbb9000e780 comp rx=0 tx=0).stop 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 shutdown_connections 2024-09-05T06:42:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405916240 0x563405918680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563405906680 0x5634059037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb7dffb640 0 client.15843 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fbdc0 0x563405903260 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1916500297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fb4a0 0x563405905fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 >> 172.21.15.86:0/1916500297 conn(0x5634058f7a30 msgr2=0x5634058f6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.405+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 shutdown_connections 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.406+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1916500297 wait complete. 2024-09-05T06:42:50.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.406+0000 7fbb9e7a2740 20 client.15843 trim_cache size 0 max 16384 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 Processor -- start 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 -- start start 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058f7a30 0x5634059124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 0x56340591e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ace70 con 0x5634058fb4a0 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ae9b0 con 0x5634058fbdc0 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ad540 con 0x5634058f7a30 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36816/0 (socket says 172.21.15.86:36816) 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 0x56340591e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9a165640 1 -- 172.21.15.86:0/1242733076 learned_addr learned my addr 172.21.15.86:0/1242733076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb99964640 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058f7a30 0x5634059124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.425+0000 7fbb9a165640 1 -- 172.21.15.86:0/1242733076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058f7a30 msgr2=0x5634059124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058f7a30 0x5634059124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb9a165640 1 -- 172.21.15.86:0/1242733076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 msgr2=0x56340591e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 0x56340591e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb9a165640 1 -- 172.21.15.86:0/1242733076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634058f8ae0 con 0x5634058fb4a0 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb99964640 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 0x56340591e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbb9001d980 tx=0x7fbb9001de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8927c0e77d3691a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb7ffff640 1 -- 172.21.15.86:0/1242733076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90033070 con 0x5634058fb4a0 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb7ffff640 1 -- 172.21.15.86:0/1242733076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb90003e20 con 0x5634058fb4a0 2024-09-05T06:42:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.426+0000 7fbb7ffff640 1 -- 172.21.15.86:0/1242733076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90021040 con 0x5634058fb4a0 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1242733076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 msgr2=0x56340591df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbb9001d980 tx=0x7fbb9001de00 comp rx=0 tx=0).stop 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1242733076 shutdown_connections 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058fbdc0 0x56340591e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340591df50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1242733076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058f7a30 0x5634059124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1242733076 >> 172.21.15.86:0/1242733076 conn(0x563405903900 msgr2=0x563405902350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1242733076 shutdown_connections 2024-09-05T06:42:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1242733076 wait complete. 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 -- start start 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.427+0000 7fbb9e7a2740 1 Processor -- start 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x5634058ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ae2e0 con 0x5634058fb4a0 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941adc10 con 0x5634058f7a30 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941aae50 con 0x5634058fbdc0 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9e7a2740 10 client.0 _ll_register_callbacks cb 0x7fbb90013c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9a165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36824/0 (socket says 172.21.15.86:36824) 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9a165640 1 -- 172.21.15.86:0/1451082333 learned_addr learned my addr 172.21.15.86:0/1451082333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42172/0 (socket says 172.21.15.86:42172) 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x5634058ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 -- 172.21.15.86:0/1451082333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 msgr2=0x5634058ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x5634058ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 -- 172.21.15.86:0/1451082333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 msgr2=0x56340590ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 -- 172.21.15.86:0/1451082333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634058fb2d0 con 0x5634058f7a30 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb99964640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x563405906650 tx=0x7fbb90009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8621af2207a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.428+0000 7fbb7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84035040 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fbb84056040 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb90003e20 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb84005300 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 10 client.25316 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563405980950 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb90003e20 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbb8403b690 con 0x5634058f7a30 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 10 client.25316 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 20 client.25316 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405906680 0x563405982b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 4 client.25316 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.429+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563405868d60 con 0x563405906680 2024-09-05T06:42:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405906680 0x563405982b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb7dffb640 10 client.25316 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9a165640 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405906680 0x563405982b80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e0e00fc3ac6394 server_cookie=7cc1de9fc4f80277 in_seq=0 out_seq=0 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb7dffb640 1 -- 172.21.15.86:0/1451082333 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563405868d60 con 0x563405906680 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb7dffb640 10 client.25316 handle_command_reply: tid=0 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 client.25316 shutdown 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 msgr2=0x56340590e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x563405906650 tx=0x7fbb90009500 comp rx=0 tx=0).stop 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 shutdown_connections 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563405906680 0x563405982b80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634058fbdc0 0x5634058ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634058fb4a0 0x56340590ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb7dffb640 0 client.25316 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 --2- 172.21.15.86:0/1451082333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634058f7a30 0x56340590e900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 >> 172.21.15.86:0/1451082333 conn(0x563405903900 msgr2=0x5634058fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.430+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 shutdown_connections 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9e7a2740 1 -- 172.21.15.86:0/1451082333 wait complete. 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9e7a2740 20 client.25316 trim_cache size 0 max 16384 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:42:50.437 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 msgr2=0x7fbb800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 0x7fbb800a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc450 tx=0x7fbb8c0bb040 comp rx=0 tx=0).stop 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 msgr2=0x7fbb9419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d750 tx=0x7fbb8c00dc10 comp rx=0 tx=0).stop 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 shutdown_connections 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb800a2d70 0x7fbb800a5230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb94105e40 0x7fbb9419cc80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb94101fc0 0x7fbb941988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 --2- 172.21.15.86:0/272951562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb94101610 0x7fbb941983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 >> 172.21.15.86:0/272951562 conn(0x7fbb940fc5c0 msgr2=0x7fbb941003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 shutdown_connections 2024-09-05T06:42:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:50.431+0000 7fbb9c3c9640 1 -- 172.21.15.86:0/272951562 wait complete. 2024-09-05T06:42:50.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 7714 5.92 MB/sec warmup 10 sec latency 43.709 ms 2024-09-05T06:42:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:50 smithi115 ceph-mon[34791]: pgmap v50: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 58 op/s 2024-09-05T06:42:50.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:50 smithi086 ceph-mon[30943]: pgmap v50: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 58 op/s 2024-09-05T06:42:50.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:50 smithi096 ceph-mon[34468]: pgmap v50: 129 pgs: 129 active+clean; 26 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 58 op/s 2024-09-05T06:42:51.201 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:42:51.201 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:42:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:51.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 8485 5.84 MB/sec warmup 11 sec latency 13.810 ms 2024-09-05T06:42:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3035266856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 msgr2=0x7f1d701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3035266856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d701021d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009900 tx=0x7f1d6c02a410 comp rx=0 tx=0).stop 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3035266856 shutdown_connections 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3035266856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70106f20 0x7f1d7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3035266856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d701027a0 0x7f1d701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3035266856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d701021d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3035266856 >> 172.21.15.86:0/3035266856 conn(0x7f1d700fd8f0 msgr2=0x7f1d700ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3035266856 shutdown_connections 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.988+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3035266856 wait complete. 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.989+0000 7f1d77d5e640 1 Processor -- start 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.989+0000 7f1d77d5e640 1 -- start start 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.989+0000 7f1d77d5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.989+0000 7f1d77d5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 0x7f1d70074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.989+0000 7f1d77d5e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 0x7f1d70074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d77d5e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70111e80 con 0x7f1d70101df0 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d77d5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70111d00 con 0x7f1d701027a0 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d77d5e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70112000 con 0x7f1d70106f20 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d75afa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d75afa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36842/0 (socket says 172.21.15.86:36842) 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d75afa640 1 -- 172.21.15.86:0/3025739087 learned_addr learned my addr 172.21.15.86:0/3025739087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 0x7f1d70074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d752f9640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 0x7f1d70074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 -- 172.21.15.86:0/3025739087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 msgr2=0x7f1d70074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 0x7f1d70074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 -- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 msgr2=0x7f1d700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 -- 172.21.15.86:0/3025739087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d701a6860 con 0x7f1d70106f20 2024-09-05T06:42:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d752f9640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 0x7f1d70074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d762fb640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 0x7f1d70074f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009fd0 tx=0x7f1d6800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800ce20 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.990+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d68005020 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.991+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701a6b50 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.991+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68010690 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.991+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d701a7040 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.991+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d701021d0 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.993+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1d680051d0 con 0x7f1d70106f20 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.994+0000 7f1d66ffd640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 0x7f1d5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.994+0000 7f1d75afa640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 0x7f1d5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.994+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f1d680cff80 con 0x7f1d70106f20 2024-09-05T06:42:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.994+0000 7f1d75afa640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 0x7f1d5c0a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0096c0 tx=0x7f1d6c005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:51.996+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d6809c760 con 0x7f1d70106f20 2024-09-05T06:42:52.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.229+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d70075a20 con 0x7f1d70106f20 2024-09-05T06:42:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.230+0000 7f1d66ffd640 1 -- 172.21.15.86:0/3025739087 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f1d70075a20 con 0x7f1d70106f20 2024-09-05T06:42:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:42:52.230 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"btime":"2024-09-05T06:41:17:336484+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:41:16.604262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:42:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-09-05T06:42:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 msgr2=0x7f1d5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 0x7f1d5c0a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0096c0 tx=0x7f1d6c005a20 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 msgr2=0x7f1d70074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 0x7f1d70074f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009fd0 tx=0x7f1d6800ef10 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 shutdown_connections 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1d5c0a2cf0 0x7f1d5c0a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d70106f20 0x7f1d70074f70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d701027a0 0x7f1d70074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 --2- 172.21.15.86:0/3025739087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d70101df0 0x7f1d700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 >> 172.21.15.86:0/3025739087 conn(0x7f1d700fd8f0 msgr2=0x7f1d700fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 shutdown_connections 2024-09-05T06:42:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:52.233+0000 7f1d77d5e640 1 -- 172.21.15.86:0/3025739087 wait complete. 2024-09-05T06:42:52.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:52 smithi086 ceph-mon[30943]: pgmap v51: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-05T06:42:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:52 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3025739087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:52.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 9187 5.41 MB/sec warmup 12 sec latency 22.057 ms 2024-09-05T06:42:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:52 smithi115 ceph-mon[34791]: pgmap v51: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-05T06:42:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:52 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3025739087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:52.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:52 smithi096 ceph-mon[34468]: pgmap v51: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-05T06:42:52.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:52 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3025739087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:42:52.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:42:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:53.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 10072 5.52 MB/sec warmup 13 sec latency 18.906 ms 2024-09-05T06:42:53.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:53 smithi086 ceph-mon[30943]: pgmap v52: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:53 smithi115 ceph-mon[34791]: pgmap v52: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 -- 172.21.15.86:0/2424017521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c102ef0 msgr2=0x7fef6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 --2- 172.21.15.86:0/2424017521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c102ef0 0x7fef6c10da40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fef64009900 tx=0x7fef6402a480 comp rx=0 tx=0).stop 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 -- 172.21.15.86:0/2424017521 shutdown_connections 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 --2- 172.21.15.86:0/2424017521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c102ef0 0x7fef6c10da40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 --2- 172.21.15.86:0/2424017521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102570 0x7fef6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 --2- 172.21.15.86:0/2424017521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c100e20 0x7fef6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 -- 172.21.15.86:0/2424017521 >> 172.21.15.86:0/2424017521 conn(0x7fef6c0fc7d0 msgr2=0x7fef6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 -- 172.21.15.86:0/2424017521 shutdown_connections 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.739+0000 7fef7457b640 1 -- 172.21.15.86:0/2424017521 wait complete. 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 Processor -- start 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 -- start start 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c102570 0x7fef6c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 0x7fef6c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c110e50 con 0x7fef6c100e20 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c110cd0 con 0x7fef6c102ef0 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef7457b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c110fd0 con 0x7fef6c102570 2024-09-05T06:42:53.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef71b16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c102570 0x7fef6c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54240/0 (socket says 172.21.15.86:54240) 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef72317640 1 -- 172.21.15.86:0/518456847 learned_addr learned my addr 172.21.15.86:0/518456847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.740+0000 7fef72b18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 0x7fef6c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 -- 172.21.15.86:0/518456847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c102570 msgr2=0x7fef6c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c102570 0x7fef6c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 -- 172.21.15.86:0/518456847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 msgr2=0x7fef6c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 0x7fef6c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 -- 172.21.15.86:0/518456847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c19c2a0 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72b18640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 0x7fef6c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef72317640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fef6800df40 tx=0x7fef6800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef637fe640 1 -- 172.21.15.86:0/518456847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800ea60 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef637fe640 1 -- 172.21.15.86:0/518456847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef6800ec00 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef637fe640 1 -- 172.21.15.86:0/518456847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68016670 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c19c530 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.741+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c069f70 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.743+0000 7fef637fe640 1 -- 172.21.15.86:0/518456847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef68009580 con 0x7fef6c100e20 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 Processor -- start 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 -- start start 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 0x562424f52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562424eab2f0 con 0x562424f50370 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562424e52cf0 con 0x562424f46b40 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef76954740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562424ea72b0 con 0x562424f455d0 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef72b18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef637fe640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 0x7fef580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef71b16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 0x562424f52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef72b18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42194/0 (socket says 172.21.15.86:42194) 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.744+0000 7fef72b18640 1 -- 172.21.15.86:0/777336750 learned_addr learned my addr 172.21.15.86:0/777336750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef72317640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef72b18640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 0x7fef580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 -- 172.21.15.86:0/777336750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 msgr2=0x562424f45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef637fe640 1 -- 172.21.15.86:0/518456847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fef680d0d60 con 0x7fef6c100e20 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 -- 172.21.15.86:0/777336750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 msgr2=0x562424f45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 -- 172.21.15.86:0/777336750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562424e373a0 con 0x562424f50370 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef72b18640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef72317640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef71b16640 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 0x562424f52760 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009bf0 tx=0x7fef5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7de5256ea6a2289 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef72b18640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 0x7fef580a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef64005c40 tx=0x7fef6402a480 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef60ff9640 1 -- 172.21.15.86:0/777336750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009da0 con 0x562424f50370 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.745+0000 7fef60ff9640 1 -- 172.21.15.86:0/777336750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef5c005020 con 0x562424f50370 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef60ff9640 1 -- 172.21.15.86:0/777336750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004a30 con 0x562424f50370 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 -- 172.21.15.86:0/777336750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 msgr2=0x562424f52760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 0x562424f52760 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009bf0 tx=0x7fef5c011810 comp rx=0 tx=0).stop 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 -- 172.21.15.86:0/777336750 shutdown_connections 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f50370 0x562424f52760 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 --2- 172.21.15.86:0/777336750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 -- 172.21.15.86:0/777336750 >> 172.21.15.86:0/777336750 conn(0x562424f0e310 msgr2=0x562424f41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 -- 172.21.15.86:0/777336750 shutdown_connections 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.746+0000 7fef76954740 1 -- 172.21.15.86:0/777336750 wait complete. 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 -- start start 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 Processor -- start 2024-09-05T06:42:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424f4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562424f57480 con 0x562424f46b40 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562424e66060 con 0x562424f50370 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562424f57600 con 0x562424f455d0 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef76954740 10 client.0 _ll_register_callbacks cb 0x562424bb98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef71b16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef71b16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54254/0 (socket says 172.21.15.86:54254) 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef71b16640 1 -- 172.21.15.86:0/4089148314 learned_addr learned my addr 172.21.15.86:0/4089148314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef72317640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.747+0000 7fef72b18640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424f4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 -- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 msgr2=0x562424f4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 -- 172.21.15.86:0/4089148314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 msgr2=0x562424f4d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424f4d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 -- 172.21.15.86:0/4089148314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562424c1c3c0 con 0x562424f46b40 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef72317640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef71b16640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01b040 tx=0x7fef5c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ac3d60d133d0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c029020 con 0x562424f46b40 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5c005020 con 0x562424f46b40 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fef5c004aa0 con 0x562424f46b40 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef5c034050 con 0x562424f46b40 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef76954740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:53.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562424f62e00 con 0x562424f46b40 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.748+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef5c005020 con 0x562424f46b40 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fef5c023420 con 0x562424f46b40 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 10 client.15867 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 20 client.15867 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424f66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 4 client.15867 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562424eb8860 con 0x562424f64210 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef72317640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424f66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef72317640 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424f66600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a85d38c55f02337b server_cookie=3439f914c33877f1 in_seq=0 out_seq=0 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.749+0000 7fef55ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef55ffb640 1 -- 172.21.15.86:0/4089148314 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562424eb8860 con 0x562424f64210 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef55ffb640 10 client.15867 handle_command_reply: tid=0 2024-09-05T06:42:53.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 client.15867 shutdown 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 msgr2=0x562424f4cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01b040 tx=0x7fef5c009850 comp rx=0 tx=0).stop 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 shutdown_connections 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424f66600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef55ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424f4d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f46b40 0x562424f4cf80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 --2- 172.21.15.86:0/4089148314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f455d0 0x562424f4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 >> 172.21.15.86:0/4089148314 conn(0x562424f0e310 msgr2=0x562424f47960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 shutdown_connections 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 1 -- 172.21.15.86:0/4089148314 wait complete. 2024-09-05T06:42:53.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.750+0000 7fef76954740 20 client.15867 trim_cache size 0 max 16384 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 Processor -- start 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 -- start start 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f4f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f50370 0x562424f48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1ace50 con 0x562424f455d0 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1ae990 con 0x562424f46b40 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef76954740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1ad520 con 0x562424f50370 2024-09-05T06:42:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef71b16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f4f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54258/0 (socket says 172.21.15.86:54258) 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef72317640 1 -- 172.21.15.86:0/2081270431 learned_addr learned my addr 172.21.15.86:0/2081270431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.764+0000 7fef72b18640 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f50370 0x562424f48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 -- 172.21.15.86:0/2081270431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f50370 msgr2=0x562424f48120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f50370 0x562424f48120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 -- 172.21.15.86:0/2081270431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 msgr2=0x562424f4f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f4f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 -- 172.21.15.86:0/2081270431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562424f50040 con 0x562424f455d0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef71b16640 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f4f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef72317640 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef68009fd0 tx=0x7fef68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec65cf786ec4682 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef57fff640 1 -- 172.21.15.86:0/2081270431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68024040 con 0x562424f455d0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef57fff640 1 -- 172.21.15.86:0/2081270431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef680a13f0 con 0x562424f455d0 2024-09-05T06:42:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.765+0000 7fef57fff640 1 -- 172.21.15.86:0/2081270431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680ce350 con 0x562424f455d0 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- 172.21.15.86:0/2081270431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 msgr2=0x562424f46300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef68009fd0 tx=0x7fef68004a40 comp rx=0 tx=0).stop 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- 172.21.15.86:0/2081270431 shutdown_connections 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f50370 0x562424f48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f46b40 0x562424f4f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- 172.21.15.86:0/2081270431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f46300 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- 172.21.15.86:0/2081270431 >> 172.21.15.86:0/2081270431 conn(0x562424f0e310 msgr2=0x562424f3ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- 172.21.15.86:0/2081270431 shutdown_connections 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- 172.21.15.86:0/2081270431 wait complete. 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- start start 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 Processor -- start 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 0x562424f57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424fcce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1ae2c0 con 0x562424f455d0 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1adbf0 con 0x562424f50370 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.766+0000 7fef76954740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1aae30 con 0x562424f46b40 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef76954740 10 client.0 _ll_register_callbacks cb 0x7fef64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54274/0 (socket says 172.21.15.86:54274) 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 -- 172.21.15.86:0/695644485 learned_addr learned my addr 172.21.15.86:0/695644485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72b18640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424fcce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef71b16640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 0x562424f57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 -- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 msgr2=0x562424f57f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 0x562424f57f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 -- 172.21.15.86:0/695644485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 msgr2=0x562424fcce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424fcce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 -- 172.21.15.86:0/695644485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562424fcd380 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72b18640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424fcce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef71b16640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 0x562424f57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef72317640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fef68099ac0 tx=0x7fef680d1150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c508fda48e6714c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6809b210 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fef680a10d0 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.767+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef680a13f0 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef680ce350 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562424fce5c0 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef680a13f0 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fef68098e40 con 0x562424f455d0 2024-09-05T06:42:53.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 10 client.15873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 20 client.15873 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424fd0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 4 client.15873 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562424eb2d70 con 0x562424f64210 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.768+0000 7fef71b16640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424fd0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef71b16640 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424fd0cd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2cad2f8d899e019 server_cookie=eeaf57d838a6b367 in_seq=0 out_seq=0 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef55ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef55ffb640 1 -- 172.21.15.86:0/695644485 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562424eb2d70 con 0x562424f64210 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef55ffb640 10 client.15873 handle_command_reply: tid=0 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 client.15873 shutdown 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 msgr2=0x562424f57a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fef68099ac0 tx=0x7fef680d1150 comp rx=0 tx=0).stop 2024-09-05T06:42:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 shutdown_connections 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x562424f64210 0x562424fd0cd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562424f50370 0x562424fcce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562424f46b40 0x562424f57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 --2- 172.21.15.86:0/695644485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562424f455d0 0x562424f57a10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 >> 172.21.15.86:0/695644485 conn(0x562424f0e310 msgr2=0x562424f5ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef55ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 shutdown_connections 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 1 -- 172.21.15.86:0/695644485 wait complete. 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.769+0000 7fef76954740 20 client.15873 trim_cache size 0 max 16384 2024-09-05T06:42:53.778 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 msgr2=0x7fef580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 0x7fef580a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef64005c40 tx=0x7fef6402a480 comp rx=0 tx=0).stop 2024-09-05T06:42:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 msgr2=0x7fef6c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fef6800df40 tx=0x7fef6800df70 comp rx=0 tx=0).stop 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 shutdown_connections 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef580a2cf0 0x7fef580a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef6c102ef0 0x7fef6c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef6c102570 0x7fef6c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 --2- 172.21.15.86:0/518456847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c100e20 0x7fef6c1972f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 >> 172.21.15.86:0/518456847 conn(0x7fef6c0fc7d0 msgr2=0x7fef6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 shutdown_connections 2024-09-05T06:42:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:53.770+0000 7fef7457b640 1 -- 172.21.15.86:0/518456847 wait complete. 2024-09-05T06:42:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:53 smithi096 ceph-mon[34468]: pgmap v52: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:54.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 10733 5.49 MB/sec warmup 14 sec latency 16.353 ms 2024-09-05T06:42:54.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:42:54.781 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 -- 172.21.15.86:0/1047193497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 msgr2=0x7f82c010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 --2- 172.21.15.86:0/1047193497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c010c9f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009900 tx=0x7f82ac02a410 comp rx=0 tx=0).stop 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 -- 172.21.15.86:0/1047193497 shutdown_connections 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 --2- 172.21.15.86:0/1047193497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 0x7f82c0119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 --2- 172.21.15.86:0/1047193497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c010c9f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 --2- 172.21.15.86:0/1047193497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 0x7f82c0112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 -- 172.21.15.86:0/1047193497 >> 172.21.15.86:0/1047193497 conn(0x7f82c006cdc0 msgr2=0x7f82c006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 -- 172.21.15.86:0/1047193497 shutdown_connections 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.246+0000 7f82c6e62640 1 -- 172.21.15.86:0/1047193497 wait complete. 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 Processor -- start 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 -- start start 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 0x7f82c01a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 0x7f82c01a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82c011c850 con 0x7f82c010c590 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82c011c6d0 con 0x7f82c010cf30 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c6e62640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82c011c9d0 con 0x7f82c0112640 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54300/0 (socket says 172.21.15.86:54300) 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c4bfe640 1 -- 172.21.15.86:0/932893949 learned_addr learned my addr 172.21.15.86:0/932893949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82c53ff640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 0x7f82c01a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.247+0000 7f82bffff640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 0x7f82c01a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 -- 172.21.15.86:0/932893949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 msgr2=0x7f82c01a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 0x7f82c01a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 -- 172.21.15.86:0/932893949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 msgr2=0x7f82c01a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 0x7f82c01a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 -- 172.21.15.86:0/932893949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c01a3390 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d950 tx=0x7f82b400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82bdffb640 1 -- 172.21.15.86:0/932893949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4017030 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c01a3680 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82bdffb640 1 -- 172.21.15.86:0/932893949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b4004020 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82bdffb640 1 -- 172.21.15.86:0/932893949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4005570 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.248+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c01af9d0 con 0x7f82c010c590 2024-09-05T06:42:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.251+0000 7f82bdffb640 1 -- 172.21.15.86:0/932893949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f82b40041d0 con 0x7f82c010c590 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.251+0000 7f82c923b740 1 Processor -- start 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 -- start start 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bdffb640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 0x7f82b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b97706a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977077760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b976fd02f0 con 0x55b977075370 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b976f77cf0 con 0x55b97706a5d0 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c923b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b976fcc2b0 con 0x55b97706bb40 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c53ff640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 0x7f82b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bdffb640 1 -- 172.21.15.86:0/932893949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f82b40cf130 con 0x7f82c010c590 2024-09-05T06:42:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c53ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42264/0 (socket says 172.21.15.86:42264) 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 -- 172.21.15.86:0/3413294307 learned_addr learned my addr 172.21.15.86:0/3413294307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 -- 172.21.15.86:0/3413294307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 msgr2=0x55b97706a090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b97706a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 -- 172.21.15.86:0/3413294307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 msgr2=0x55b977077760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977077760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 -- 172.21.15.86:0/3413294307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b976f5c3a0 con 0x55b97706a5d0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82bffff640 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009900 tx=0x7f82ac005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e52753b75ec966 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.252+0000 7f82c53ff640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 0x7f82b00a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82b8005fd0 tx=0x7f82b8009f30 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82aaffd640 1 -- 172.21.15.86:0/3413294307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac040070 con 0x55b97706a5d0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82aaffd640 1 -- 172.21.15.86:0/3413294307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac004030 con 0x55b97706a5d0 2024-09-05T06:42:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82aaffd640 1 -- 172.21.15.86:0/3413294307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac035040 con 0x55b97706a5d0 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 -- 172.21.15.86:0/3413294307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 msgr2=0x55b97706aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009900 tx=0x7f82ac005c00 comp rx=0 tx=0).stop 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 -- 172.21.15.86:0/3413294307 shutdown_connections 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977077760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b97706aa10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 --2- 172.21.15.86:0/3413294307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b97706a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 -- 172.21.15.86:0/3413294307 >> 172.21.15.86:0/3413294307 conn(0x55b977033310 msgr2=0x55b977066c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 -- 172.21.15.86:0/3413294307 shutdown_connections 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.253+0000 7f82c923b740 1 -- 172.21.15.86:0/3413294307 wait complete. 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 -- start start 2024-09-05T06:42:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 Processor -- start 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 0x55b977072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b977078010 con 0x55b97706bb40 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b976f8b060 con 0x55b97706a5d0 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b977078190 con 0x55b977075370 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c923b740 10 client.0 _ll_register_callbacks cb 0x55b976cde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c53ff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 0x55b977072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42266/0 (socket says 172.21.15.86:42266) 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54312/0 (socket says 172.21.15.86:54312) 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.254+0000 7f82c4bfe640 1 -- 172.21.15.86:0/3264054444 learned_addr learned my addr 172.21.15.86:0/3264054444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 -- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 msgr2=0x55b977074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 -- 172.21.15.86:0/3264054444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 msgr2=0x55b977072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 -- 172.21.15.86:0/3264054444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b976d413c0 con 0x55b977075370 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82bffff640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c53ff640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 0x55b977072600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f82b80099a0 tx=0x7f82b801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c86635163d022b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f829ffff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8024070 con 0x55b977075370 2024-09-05T06:42:55.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ac004030 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f82b8004710 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82b8033050 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c923b740 10 client.25309 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b977083b60 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.255+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac004030 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82b8028400 con 0x55b977075370 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 10 client.25309 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 10 client.25309 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 20 client.25309 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b977084fc0 0x55b9770873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 4 client.25309 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:55.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b976fdd860 con 0x55b977084fc0 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b977084fc0 0x55b9770873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b977084fc0 0x55b9770873b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d1dcb5129a9e6c4 server_cookie=68711c4a122dbec4 in_seq=0 out_seq=0 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.256+0000 7f829ffff640 10 client.25309 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f829ffff640 1 -- 172.21.15.86:0/3264054444 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b976fdd860 con 0x55b977084fc0 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f829ffff640 10 client.25309 handle_command_reply: tid=0 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 client.25309 shutdown 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 msgr2=0x55b977072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 0x55b977072600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f82b80099a0 tx=0x7f82b801d000 comp rx=0 tx=0).stop 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 shutdown_connections 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b977084fc0 0x55b9770873b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b977075370 0x55b977072600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f829ffff640 0 client.25309 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:55.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 --2- 172.21.15.86:0/3264054444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 >> 172.21.15.86:0/3264054444 conn(0x55b977033310 msgr2=0x55b97706c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 shutdown_connections 2024-09-05T06:42:55.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 1 -- 172.21.15.86:0/3264054444 wait complete. 2024-09-05T06:42:55.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.257+0000 7f82c923b740 20 client.25309 trim_cache size 0 max 16384 2024-09-05T06:42:55.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 Processor -- start 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 -- start start 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b977072560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01b9960 con 0x55b977075370 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01bb4a0 con 0x55b97706a5d0 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.271+0000 7f82c923b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01ba030 con 0x55b97706bb40 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c53ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c53ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54328/0 (socket says 172.21.15.86:54328) 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c53ff640 1 -- 172.21.15.86:0/350948964 learned_addr learned my addr 172.21.15.86:0/350948964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82bffff640 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b977072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 -- 172.21.15.86:0/350948964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 msgr2=0x55b977072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b977072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 -- 172.21.15.86:0/350948964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 msgr2=0x55b977081310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 -- 172.21.15.86:0/350948964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97706d120 con 0x55b97706a5d0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c53ff640 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977066e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82b40cc500 tx=0x7f82b400df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d22acde8ab303c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82aa7fc640 1 -- 172.21.15.86:0/350948964 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b409f620 con 0x55b97706a5d0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82aa7fc640 1 -- 172.21.15.86:0/350948964 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b40a3ee0 con 0x55b97706a5d0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.272+0000 7f82aa7fc640 1 -- 172.21.15.86:0/350948964 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40cc8b0 con 0x55b97706a5d0 2024-09-05T06:42:55.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- 172.21.15.86:0/350948964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 msgr2=0x55b977066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977066e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82b40cc500 tx=0x7f82b400df90 comp rx=0 tx=0).stop 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- 172.21.15.86:0/350948964 shutdown_connections 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b977075370 0x55b977081310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706bb40 0x55b977072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 --2- 172.21.15.86:0/350948964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b97706a5d0 0x55b977066e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- 172.21.15.86:0/350948964 >> 172.21.15.86:0/350948964 conn(0x55b977033310 msgr2=0x55b977066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- 172.21.15.86:0/350948964 shutdown_connections 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- 172.21.15.86:0/350948964 wait complete. 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.273+0000 7f82c923b740 1 -- start start 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 Processor -- start 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 0x55b977078440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 0x55b977078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01badd0 con 0x55b97706bb40 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01ba700 con 0x55b977075370 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01b7940 con 0x55b97706a5d0 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c923b740 10 client.0 _ll_register_callbacks cb 0x7f82b8011630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54342/0 (socket says 172.21.15.86:54342) 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 0x55b977078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c4bfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 0x55b977078440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82bffff640 1 -- 172.21.15.86:0/2613026371 learned_addr learned my addr 172.21.15.86:0/2613026371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 -- 172.21.15.86:0/2613026371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 msgr2=0x55b977078440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 0x55b977078440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 -- 172.21.15.86:0/2613026371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 msgr2=0x55b977078980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c53ff640 1 -- 172.21.15.86:0/2613026371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9770f6450 con 0x55b977075370 2024-09-05T06:42:55.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82bffff640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.274+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 0x55b977078440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f82c53ff640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 0x55b977078ec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b977072c40 tx=0x7f82b8028cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e1a2ff5da2369 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f829ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b803b020 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b40a3ee0 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f82b8004710 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82b8046050 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f82c923b740 10 client.25358 fetch_fsmap learned FSMap version 28 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9770f7730 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.275+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b40a3ee0 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82b8004910 con 0x55b977075370 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 10 client.25358 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T06:42:55.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 10 client.25358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 20 client.25358 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b9770f7de0 0x55b9770fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 4 client.25358 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b976fd7d70 con 0x55b9770f7de0 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b9770f7de0 0x55b9770fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f82c4bfe640 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b9770f7de0 0x55b9770fa200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5289c56bdbdab07b server_cookie=c07e96cbebc50f43 in_seq=0 out_seq=0 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.276+0000 7f829ffff640 10 client.25358 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f829ffff640 1 -- 172.21.15.86:0/2613026371 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b976fd7d70 con 0x55b9770f7de0 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f829ffff640 10 client.25358 handle_command_reply: tid=0 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 client.25358 shutdown 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 msgr2=0x55b977078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 0x55b977078ec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b977072c40 tx=0x7f82b8028cc0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 shutdown_connections 2024-09-05T06:42:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b9770f7de0 0x55b9770fa200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b977075370 0x55b977078ec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b97706bb40 0x55b977078980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 --2- 172.21.15.86:0/2613026371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b97706a5d0 0x55b977078440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 >> 172.21.15.86:0/2613026371 conn(0x55b977033310 msgr2=0x55b977074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f829ffff640 0 client.25358 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 shutdown_connections 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 1 -- 172.21.15.86:0/2613026371 wait complete. 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.277+0000 7f82c923b740 20 client.25358 trim_cache size 0 max 16384 2024-09-05T06:42:55.286 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 msgr2=0x7f82b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 0x7f82b00a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82b8005fd0 tx=0x7f82b8009f30 comp rx=0 tx=0).stop 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 msgr2=0x7f82c01a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d950 tx=0x7f82b400dd50 comp rx=0 tx=0).stop 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 shutdown_connections 2024-09-05T06:42:55.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82b00a2cf0 0x7f82b00a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c0112640 0x7f82c01a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82c010cf30 0x7f82c01a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 --2- 172.21.15.86:0/932893949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c010c590 0x7f82c01a8a50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 >> 172.21.15.86:0/932893949 conn(0x7f82c006cdc0 msgr2=0x7f82c010b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 shutdown_connections 2024-09-05T06:42:55.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:55.278+0000 7f82c6e62640 1 -- 172.21.15.86:0/932893949 wait complete. 2024-09-05T06:42:55.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 11148 5.22 MB/sec warmup 15 sec latency 16.190 ms 2024-09-05T06:42:55.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:42:56.258 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:56.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 11794 5.02 MB/sec warmup 16 sec latency 12.186 ms 2024-09-05T06:42:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:56 smithi115 ceph-mon[34791]: pgmap v53: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:56 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:42:56.691 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:56 smithi086 ceph-mon[30943]: pgmap v53: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:56.691 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:56 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 -- 172.21.15.86:0/3106395075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 msgr2=0x7faa3010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 --2- 172.21.15.86:0/3106395075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa3010d950 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faa28009cd0 tx=0x7faa2802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 -- 172.21.15.86:0/3106395075 shutdown_connections 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 --2- 172.21.15.86:0/3106395075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa3010d950 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 --2- 172.21.15.86:0/3106395075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 0x7faa30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 --2- 172.21.15.86:0/3106395075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 0x7faa30106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 -- 172.21.15.86:0/3106395075 >> 172.21.15.86:0/3106395075 conn(0x7faa300fc7a0 msgr2=0x7faa300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 -- 172.21.15.86:0/3106395075 shutdown_connections 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.781+0000 7faa36616640 1 -- 172.21.15.86:0/3106395075 wait complete. 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 Processor -- start 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 -- start start 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 0x7faa300fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 0x7faa300fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa30110d80 con 0x7faa30101420 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa30110c00 con 0x7faa30106a80 2024-09-05T06:42:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa36616640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa30110f00 con 0x7faa30100a80 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 0x7faa300fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54362/0 (socket says 172.21.15.86:54362) 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 -- 172.21.15.86:0/2670661010 learned_addr learned my addr 172.21.15.86:0/2670661010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa34bb3640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 0x7faa300fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 -- 172.21.15.86:0/2670661010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 msgr2=0x7faa300fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 0x7faa300fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 -- 172.21.15.86:0/2670661010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 msgr2=0x7faa300fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 0x7faa300fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.782+0000 7faa27fff640 1 -- 172.21.15.86:0/2670661010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301a67f0 con 0x7faa30101420 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa27fff640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9f0 tx=0x7faa1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa2dffb640 1 -- 172.21.15.86:0/2670661010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1800cd80 con 0x7faa30101420 2024-09-05T06:42:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa301a6ae0 con 0x7faa30101420 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa2dffb640 1 -- 172.21.15.86:0/2670661010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa180054d0 con 0x7faa30101420 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa2dffb640 1 -- 172.21.15.86:0/2670661010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18010490 con 0x7faa30101420 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.783+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa301a7020 con 0x7faa30101420 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.785+0000 7faa2dffb640 1 -- 172.21.15.86:0/2670661010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faa18005020 con 0x7faa30101420 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 Processor -- start 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 -- start start 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2dffb640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 0x7faa1c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bc4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce34b2a2f0 con 0x55ce34bcf370 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ce34ad1cf0 con 0x55ce34bc5b40 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa389ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce34b262b0 con 0x55ce34bc45d0 2024-09-05T06:42:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa34bb3640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 0x7faa1c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2dffb640 1 -- 172.21.15.86:0/2670661010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faa180cfb10 con 0x7faa30101420 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42322/0 (socket says 172.21.15.86:42322) 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 -- 172.21.15.86:0/1144066070 learned_addr learned my addr 172.21.15.86:0/1144066070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa27fff640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bc4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa34bb3640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 -- 172.21.15.86:0/1144066070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 msgr2=0x55ce34bc4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bc4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 -- 172.21.15.86:0/1144066070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 msgr2=0x55ce34bd1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bd1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa2ffff640 1 -- 172.21.15.86:0/1144066070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34ab63a0 con 0x55ce34bc5b40 2024-09-05T06:42:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.786+0000 7faa34bb3640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bd1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.787+0000 7faa2ffff640 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa20009c70 tx=0x7faa20011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b28b02cec076ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.787+0000 7faa34bb3640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 0x7faa1c0a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa28004af0 tx=0x7faa28005ef0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.787+0000 7faa267fc640 1 -- 172.21.15.86:0/1144066070 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20004230 con 0x55ce34bc5b40 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.787+0000 7faa267fc640 1 -- 172.21.15.86:0/1144066070 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa200043d0 con 0x55ce34bc5b40 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.787+0000 7faa267fc640 1 -- 172.21.15.86:0/1144066070 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa200197a0 con 0x55ce34bc5b40 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- 172.21.15.86:0/1144066070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 msgr2=0x55ce34bc4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa20009c70 tx=0x7faa20011810 comp rx=0 tx=0).stop 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- 172.21.15.86:0/1144066070 shutdown_connections 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bd1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bc4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- 172.21.15.86:0/1144066070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bc4090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- 172.21.15.86:0/1144066070 >> 172.21.15.86:0/1144066070 conn(0x55ce34b8d310 msgr2=0x55ce34bc0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- 172.21.15.86:0/1144066070 shutdown_connections 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- 172.21.15.86:0/1144066070 wait complete. 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- start start 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 Processor -- start 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bcbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce34bd2070 con 0x55ce34bcf370 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ce34ae5060 con 0x55ce34bc5b40 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.788+0000 7faa389ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce34bd21f0 con 0x55ce34bc45d0 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa389ef740 10 client.0 _ll_register_callbacks cb 0x55ce348388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa34bb3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa34bb3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54378/0 (socket says 172.21.15.86:54378) 2024-09-05T06:42:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa34bb3640 1 -- 172.21.15.86:0/4130780999 learned_addr learned my addr 172.21.15.86:0/4130780999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bcbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 -- 172.21.15.86:0/4130780999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 msgr2=0x55ce34bced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 -- 172.21.15.86:0/4130780999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 msgr2=0x55ce34bcc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 -- 172.21.15.86:0/4130780999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3489b3c0 con 0x55ce34bc5b40 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa34bb3640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa27fff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bcbfb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa180cce80 tx=0x7faa1805c1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f74a82d49c84792 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1809c350 con 0x55ce34bc5b40 2024-09-05T06:42:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa200043d0 con 0x55ce34bc5b40 2024-09-05T06:42:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.789+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faa180512f0 con 0x55ce34bc5b40 2024-09-05T06:42:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa180b1030 con 0x55ce34bc5b40 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 10 client.25370 fetch_fsmap learned FSMap version 29 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce34bddaa0 con 0x55ce34bc5b40 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa200043d0 con 0x55ce34bc5b40 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7faa1805e440 con 0x55ce34bc5b40 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 10 client.25370 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 10 client.25370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 20 client.25370 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34bdeeb0 0x55ce34be12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 4 client.25370 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce34b37860 con 0x55ce34bdeeb0 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.790+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34bdeeb0 0x55ce34be12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.791+0000 7faa0f7fe640 10 client.25370 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.791+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34bdeeb0 0x55ce34be12f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=157f9cd18422e0c3 server_cookie=c64e938d883b6b4e in_seq=0 out_seq=0 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.791+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4130780999 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce34b37860 con 0x55ce34bdeeb0 2024-09-05T06:42:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.791+0000 7faa0f7fe640 10 client.25370 handle_command_reply: tid=0 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.791+0000 7faa389ef740 1 client.25370 shutdown 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 msgr2=0x55ce34bcbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bcbfb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa180cce80 tx=0x7faa1805c1f0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 shutdown_connections 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34bdeeb0 0x55ce34be12f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bcf370 0x55ce34bcc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bcbfb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 --2- 172.21.15.86:0/4130780999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bc45d0 0x55ce34bced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa0f7fe640 0 client.25370 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 >> 172.21.15.86:0/4130780999 conn(0x55ce34b8d310 msgr2=0x55ce34bc6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 shutdown_connections 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 1 -- 172.21.15.86:0/4130780999 wait complete. 2024-09-05T06:42:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.793+0000 7faa389ef740 20 client.25370 trim_cache size 0 max 16384 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.806+0000 7faa389ef740 1 Processor -- start 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.806+0000 7faa389ef740 1 -- start start 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 0x55ce34bdb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34be7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b11e0 con 0x55ce34bc5b40 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b2d50 con 0x55ce34bc45d0 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa389ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b18b0 con 0x55ce34bcf370 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54394/0 (socket says 172.21.15.86:54394) 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 -- 172.21.15.86:0/2859312072 learned_addr learned my addr 172.21.15.86:0/2859312072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa2ffff640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 0x55ce34bdb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa34bb3640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34be7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:56 smithi096 ceph-mon[34468]: pgmap v53: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.6 MiB/s wr, 83 op/s 2024-09-05T06:42:56.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:56 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 -- 172.21.15.86:0/2859312072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 msgr2=0x55ce34be7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34be7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 -- 172.21.15.86:0/2859312072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 msgr2=0x55ce34bdb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 0x55ce34bdb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 -- 172.21.15.86:0/2859312072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34bc7120 con 0x55ce34bc5b40 2024-09-05T06:42:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa34bb3640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34be7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa2ffff640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 0x55ce34bdb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.807+0000 7faa27fff640 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa180a4390 tx=0x7faa180097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a51c3f292e970e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa25ffb640 1 -- 172.21.15.86:0/2859312072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1805a070 con 0x55ce34bc5b40 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa25ffb640 1 -- 172.21.15.86:0/2859312072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa180b4e50 con 0x55ce34bc5b40 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa25ffb640 1 -- 172.21.15.86:0/2859312072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1809c350 con 0x55ce34bc5b40 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 -- 172.21.15.86:0/2859312072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 msgr2=0x55ce34be6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa180a4390 tx=0x7faa180097a0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 -- 172.21.15.86:0/2859312072 shutdown_connections 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34be7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc5b40 0x55ce34be6c10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 --2- 172.21.15.86:0/2859312072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc45d0 0x55ce34bdb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.808+0000 7faa389ef740 1 -- 172.21.15.86:0/2859312072 >> 172.21.15.86:0/2859312072 conn(0x55ce34b8d310 msgr2=0x55ce34bc7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- 172.21.15.86:0/2859312072 shutdown_connections 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- 172.21.15.86:0/2859312072 wait complete. 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- start start 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 Processor -- start 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34bd3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b2650 con 0x55ce34bc45d0 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b1f80 con 0x55ce34bc5b40 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.809+0000 7faa389ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa301af1c0 con 0x55ce34bcf370 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa389ef740 10 client.0 _ll_register_callbacks cb 0x7faa1809ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42350/0 (socket says 172.21.15.86:42350) 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 -- 172.21.15.86:0/4002168264 learned_addr learned my addr 172.21.15.86:0/4002168264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54410/0 (socket says 172.21.15.86:54410) 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa34bb3640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34bd3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 -- 172.21.15.86:0/4002168264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 msgr2=0x55ce34bd3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34bd3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 -- 172.21.15.86:0/4002168264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 msgr2=0x55ce34bd32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 -- 172.21.15.86:0/4002168264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34c48ce0 con 0x55ce34bc5b40 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa27fff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa18099f00 tx=0x7faa18004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5a384ce6e78345 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1805a070 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa180b4e50 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faa1805e5c0 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.810+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa180af050 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 10 client.25376 fetch_fsmap learned FSMap version 29 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce34c49fa0 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa180b4e50 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7faa18052070 con 0x55ce34bc5b40 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 10 client.25376 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 10 client.25376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 20 client.25376 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34c4a3b0 0x55ce34c4c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 4 client.25376 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce34b31d70 con 0x55ce34c4a3b0 2024-09-05T06:42:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.811+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34c4a3b0 0x55ce34c4c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa0f7fe640 10 client.25376 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa2ffff640 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34c4a3b0 0x55ce34c4c7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd2c7242ed541f9a server_cookie=a75283f9176549cc in_seq=0 out_seq=0 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa0f7fe640 1 -- 172.21.15.86:0/4002168264 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce34b31d70 con 0x55ce34c4a3b0 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa0f7fe640 10 client.25376 handle_command_reply: tid=0 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 client.25376 shutdown 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 msgr2=0x55ce34bd3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa18099f00 tx=0x7faa18004b10 comp rx=0 tx=0).stop 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 shutdown_connections 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ce34c4a3b0 0x55ce34c4c7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce34bcf370 0x55ce34bd3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ce34bc5b40 0x55ce34bd3800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 --2- 172.21.15.86:0/4002168264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce34bc45d0 0x55ce34bd32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 >> 172.21.15.86:0/4002168264 conn(0x55ce34b8d310 msgr2=0x55ce34bcd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa0f7fe640 0 client.25376 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 shutdown_connections 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 1 -- 172.21.15.86:0/4002168264 wait complete. 2024-09-05T06:42:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.812+0000 7faa389ef740 20 client.25376 trim_cache size 0 max 16384 2024-09-05T06:42:56.819 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 msgr2=0x7faa1c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 0x7faa1c0a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa28004af0 tx=0x7faa28005ef0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 msgr2=0x7faa300fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9f0 tx=0x7faa1800edf0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 shutdown_connections 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa1c0a2d70 0x7faa1c0a5230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa30106a80 0x7faa300fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa30101420 0x7faa300fd2a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 --2- 172.21.15.86:0/2670661010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa30100a80 0x7faa300fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 >> 172.21.15.86:0/2670661010 conn(0x7faa300fc7a0 msgr2=0x7faa3006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 shutdown_connections 2024-09-05T06:42:56.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:56.813+0000 7faa36616640 1 -- 172.21.15.86:0/2670661010 wait complete. 2024-09-05T06:42:57.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 12518 4.92 MB/sec warmup 17 sec latency 17.126 ms 2024-09-05T06:42:57.692 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:42:57 smithi086 ceph-mon[30943]: pgmap v54: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T06:42:57.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:42:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:42:57 smithi096 ceph-mon[34468]: pgmap v54: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T06:42:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:42:57 smithi115 ceph-mon[34791]: pgmap v54: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T06:42:57.977 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:42:58.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 13158 4.78 MB/sec warmup 18 sec latency 21.548 ms 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 -- 172.21.15.86:0/4237382474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 msgr2=0x7f2dd8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 --2- 172.21.15.86:0/4237382474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8107b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009930 tx=0x7f2dc402a420 comp rx=0 tx=0).stop 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 -- 172.21.15.86:0/4237382474 shutdown_connections 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 --2- 172.21.15.86:0/4237382474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 0x7f2dd810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 --2- 172.21.15.86:0/4237382474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8107b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 --2- 172.21.15.86:0/4237382474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.733+0000 7f2dded3b640 1 -- 172.21.15.86:0/4237382474 >> 172.21.15.86:0/4237382474 conn(0x7f2dd80fc7d0 msgr2=0x7f2dd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 -- 172.21.15.86:0/4237382474 shutdown_connections 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 -- 172.21.15.86:0/4237382474 wait complete. 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 Processor -- start 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 -- start start 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 0x7f2dd819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd8111f60 con 0x7f2dd81036e0 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.734+0000 7f2dded3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd8111de0 con 0x7f2dd8102d30 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dded3b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81120e0 con 0x7f2dd81082a0 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42362/0 (socket says 172.21.15.86:42362) 2024-09-05T06:42:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 -- 172.21.15.86:0/2004506896 learned_addr learned my addr 172.21.15.86:0/2004506896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 0x7f2dd819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 -- 172.21.15.86:0/2004506896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 msgr2=0x7f2dd819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 0x7f2dd819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 -- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 msgr2=0x7f2dd8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 -- 172.21.15.86:0/2004506896 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd819d4d0 con 0x7f2dd8102d30 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9f0 tx=0x7f2dcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dd5ffb640 1 -- 172.21.15.86:0/2004506896 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00cd80 con 0x7f2dd8102d30 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd819d7c0 con 0x7f2dd8102d30 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dd5ffb640 1 -- 172.21.15.86:0/2004506896 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dcc0108e0 con 0x7f2dd8102d30 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.735+0000 7f2dd5ffb640 1 -- 172.21.15.86:0/2004506896 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc018920 con 0x7f2dd8102d30 2024-09-05T06:42:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.736+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd81a50a0 con 0x7f2dd8102d30 2024-09-05T06:42:58.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2dd5ffb640 1 -- 172.21.15.86:0/2004506896 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2dcc010a90 con 0x7f2dd8102d30 2024-09-05T06:42:58.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 Processor -- start 2024-09-05T06:42:58.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 -- start start 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571d2dab2f0 con 0x5571d2e455d0 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571d2d52cf0 con 0x5571d2e54780 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2de1114740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571d2da72b0 con 0x5571d2e46b40 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42364/0 (socket says 172.21.15.86:42364) 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2ddd2d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2dd7fff640 1 -- 172.21.15.86:0/3175895290 learned_addr learned my addr 172.21.15.86:0/3175895290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.739+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 -- 172.21.15.86:0/3175895290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 msgr2=0x5571d2e45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 -- 172.21.15.86:0/3175895290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 msgr2=0x5571d2e45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 -- 172.21.15.86:0/3175895290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d2d373a0 con 0x5571d2e54780 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd5ffb640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 0x7f2dc80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009900 tx=0x7f2dc4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8259a0700b0401a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 0x7f2dc80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dc2ffd640 1 -- 172.21.15.86:0/3175895290 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4035040 con 0x5571d2e54780 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dd5ffb640 1 -- 172.21.15.86:0/2004506896 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dcc0cfb00 con 0x7f2dd8102d30 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dc2ffd640 1 -- 172.21.15.86:0/3175895290 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc4004430 con 0x5571d2e54780 2024-09-05T06:42:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.740+0000 7f2dc2ffd640 1 -- 172.21.15.86:0/3175895290 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40035f0 con 0x5571d2e54780 2024-09-05T06:42:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- 172.21.15.86:0/3175895290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 msgr2=0x5571d2e56b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 0x7f2dc80a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd0009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009900 tx=0x7f2dc4004040 comp rx=0 tx=0).stop 2024-09-05T06:42:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- 172.21.15.86:0/3175895290 shutdown_connections 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e56b70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 --2- 172.21.15.86:0/3175895290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- 172.21.15.86:0/3175895290 >> 172.21.15.86:0/3175895290 conn(0x5571d2e0e310 msgr2=0x5571d2e41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- 172.21.15.86:0/3175895290 shutdown_connections 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- 172.21.15.86:0/3175895290 wait complete. 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 -- start start 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.741+0000 7f2de1114740 1 Processor -- start 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e51440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e51a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571d2e57420 con 0x5571d2e455d0 2024-09-05T06:42:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571d2d66060 con 0x5571d2e54780 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571d2e575a0 con 0x5571d2e46b40 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2de1114740 10 client.0 _ll_register_callbacks cb 0x5571d2ab98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54436/0 (socket says 172.21.15.86:54436) 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 -- 172.21.15.86:0/3369189755 learned_addr learned my addr 172.21.15.86:0/3369189755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e51a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e51440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 -- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 msgr2=0x5571d2e51440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e51440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 -- 172.21.15.86:0/3369189755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 msgr2=0x5571d2e51a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e51a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddcad7640 1 -- 172.21.15.86:0/3369189755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d2b1c3c0 con 0x5571d2e455d0 2024-09-05T06:42:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.742+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e51a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9c0 tx=0x7f2dcc09bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ef7c7f61efe8a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc09bcf0 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dcc010430 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc4004430 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2dcc09fff0 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2de1114740 10 client.15921 fetch_fsmap learned FSMap version 29 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5571d2e62f40 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc4004430 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.743+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f2dcc097bd0 con 0x5571d2e455d0 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 10 client.15921 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:42:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 10 client.15921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 20 client.15921 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2e64400 0x5571d2e66810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 4 client.15921 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d2db8860 con 0x5571d2e64400 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2e64400 0x5571d2e66810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2db7fff640 10 client.15921 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.744+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2e64400 0x5571d2e66810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9e74b4df868709c server_cookie=208cc74372a9870d in_seq=0 out_seq=0 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2db7fff640 1 -- 172.21.15.86:0/3369189755 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571d2db8860 con 0x5571d2e64400 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2db7fff640 10 client.15921 handle_command_reply: tid=0 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 client.15921 shutdown 2024-09-05T06:42:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 msgr2=0x5571d2e54190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9c0 tx=0x7f2dcc09bbb0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 shutdown_connections 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2e64400 0x5571d2e66810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e54780 0x5571d2e51a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e51440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 --2- 172.21.15.86:0/3369189755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e455d0 0x5571d2e54190 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 >> 172.21.15.86:0/3369189755 conn(0x5571d2e0e310 msgr2=0x5571d2e47960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2db7fff640 0 client.15921 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 shutdown_connections 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.745+0000 7f2de1114740 1 -- 172.21.15.86:0/3369189755 wait complete. 2024-09-05T06:42:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.746+0000 7f2de1114740 20 client.15921 trim_cache size 0 max 16384 2024-09-05T06:42:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.759+0000 7f2de1114740 1 Processor -- start 2024-09-05T06:42:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.759+0000 7f2de1114740 1 -- start start 2024-09-05T06:42:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e455d0 0x5571d2e41e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 0x5571d2e606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81aceb0 con 0x5571d2e54780 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81aea20 con 0x5571d2e46b40 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2de1114740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81ad580 con 0x5571d2e455d0 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2ddd2d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 0x5571d2e606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2ddd2d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e50d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54440/0 (socket says 172.21.15.86:54440) 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2ddd2d8640 1 -- 172.21.15.86:0/584296580 learned_addr learned my addr 172.21.15.86:0/584296580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e455d0 0x5571d2e41e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 -- 172.21.15.86:0/584296580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e455d0 msgr2=0x5571d2e41e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e455d0 0x5571d2e41e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 -- 172.21.15.86:0/584296580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 msgr2=0x5571d2e50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 -- 172.21.15.86:0/584296580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d2e50430 con 0x5571d2e46b40 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 0x5571d2e606c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2dc402a940 tx=0x7f2dc40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829e1d012a8e36d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.760+0000 7f2dc27fc640 1 -- 172.21.15.86:0/584296580 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4035040 con 0x5571d2e46b40 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2dc27fc640 1 -- 172.21.15.86:0/584296580 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40043f0 con 0x5571d2e46b40 2024-09-05T06:42:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2dc27fc640 1 -- 172.21.15.86:0/584296580 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40035f0 con 0x5571d2e46b40 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 -- 172.21.15.86:0/584296580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 msgr2=0x5571d2e606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 0x5571d2e606c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2dc402a940 tx=0x7f2dc40045e0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 -- 172.21.15.86:0/584296580 shutdown_connections 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e46b40 0x5571d2e606c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 --2- 172.21.15.86:0/584296580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e455d0 0x5571d2e41e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.761+0000 7f2de1114740 1 -- 172.21.15.86:0/584296580 >> 172.21.15.86:0/584296580 conn(0x5571d2e0e310 msgr2=0x5571d2e41700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 -- 172.21.15.86:0/584296580 shutdown_connections 2024-09-05T06:42:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 -- 172.21.15.86:0/584296580 wait complete. 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 -- start start 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 Processor -- start 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e455d0 0x5571d2e49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.762+0000 7f2de1114740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2de1114740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81ae320 con 0x5571d2e54780 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2de1114740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81adc50 con 0x5571d2e455d0 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2de1114740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81aae90 con 0x5571d2e46b40 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddcad7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e455d0 0x5571d2e49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2de1114740 10 client.0 _ll_register_callbacks cb 0x7f2dcc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54446/0 (socket says 172.21.15.86:54446) 2024-09-05T06:42:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 -- 172.21.15.86:0/117519056 learned_addr learned my addr 172.21.15.86:0/117519056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2dd7fff640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 -- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 msgr2=0x5571d2e49660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e49660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 -- 172.21.15.86:0/117519056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e455d0 msgr2=0x5571d2e49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e455d0 0x5571d2e49120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 -- 172.21.15.86:0/117519056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d2e5bd60 con 0x5571d2e54780 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2ddd2d8640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0003fd0 tx=0x7f2dd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4207f1da270fc158 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0004b20 con 0x5571d2e54780 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2dd0017030 con 0x5571d2e54780 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.763+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc40043f0 con 0x5571d2e54780 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2dd0036050 con 0x5571d2e54780 2024-09-05T06:42:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 10 client.15927 fetch_fsmap learned FSMap version 29 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5571d2e5d020 con 0x5571d2e54780 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40043f0 con 0x5571d2e54780 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f2dd0005020 con 0x5571d2e54780 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 10 client.15927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 20 client.15927 populate_metadata read hostname 'smithi086' 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2ed3a00 0x5571d2ed5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 4 client.15927 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571d2db2d70 con 0x5571d2ed3a00 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.764+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2ed3a00 0x5571d2ed5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2ddcad7640 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2ed3a00 0x5571d2ed5e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3515a84d6d9b31b server_cookie=b76ae34ab737de03 in_seq=0 out_seq=0 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2db7fff640 10 client.15927 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2db7fff640 1 -- 172.21.15.86:0/117519056 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571d2db2d70 con 0x5571d2ed3a00 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2db7fff640 10 client.15927 handle_command_reply: tid=0 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 client.15927 shutdown 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 msgr2=0x5571d2e5b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0003fd0 tx=0x7f2dd0004ab0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 shutdown_connections 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5571d2ed3a00 0x5571d2ed5e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571d2e54780 0x5571d2e5b820 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571d2e46b40 0x5571d2e49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 --2- 172.21.15.86:0/117519056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571d2e455d0 0x5571d2e49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 >> 172.21.15.86:0/117519056 conn(0x5571d2e0e310 msgr2=0x5571d2e404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2db7fff640 0 client.15927 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 shutdown_connections 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 1 -- 172.21.15.86:0/117519056 wait complete. 2024-09-05T06:42:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.765+0000 7f2de1114740 20 client.15927 trim_cache size 0 max 16384 2024-09-05T06:42:58.774 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:42:58.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 msgr2=0x7f2dc80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 0x7f2dc80a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd0009500 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 msgr2=0x7f2dd8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9f0 tx=0x7f2dcc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 shutdown_connections 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2dc80a2cf0 0x7f2dc80a51b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd81082a0 0x7f2dd819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd81036e0 0x7f2dd8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 --2- 172.21.15.86:0/2004506896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd8102d30 0x7f2dd8198520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 >> 172.21.15.86:0/2004506896 conn(0x7f2dd80fc7d0 msgr2=0x7f2dd8105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 shutdown_connections 2024-09-05T06:42:58.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:42:58.766+0000 7f2dded3b640 1 -- 172.21.15.86:0/2004506896 wait complete. 2024-09-05T06:42:59.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 13962 4.88 MB/sec warmup 19 sec latency 67.009 ms 2024-09-05T06:42:59.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:42:59.847 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.350+0000 7fac82318640 1 -- 172.21.15.86:0/2587242199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c101de0 msgr2=0x7fac7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.350+0000 7fac82318640 1 --2- 172.21.15.86:0/2587242199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c101de0 0x7fac7c1021c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009900 tx=0x7fac6c02a410 comp rx=0 tx=0).stop 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 -- 172.21.15.86:0/2587242199 shutdown_connections 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 --2- 172.21.15.86:0/2587242199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 --2- 172.21.15.86:0/2587242199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c102790 0x7fac7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 --2- 172.21.15.86:0/2587242199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c101de0 0x7fac7c1021c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 -- 172.21.15.86:0/2587242199 >> 172.21.15.86:0/2587242199 conn(0x7fac7c0fd880 msgr2=0x7fac7c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 -- 172.21.15.86:0/2587242199 shutdown_connections 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 -- 172.21.15.86:0/2587242199 wait complete. 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 Processor -- start 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.351+0000 7fac82318640 1 -- start start 2024-09-05T06:43:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 0x7fac7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 0x7fac7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c111e20 con 0x7fac7c106f10 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c111ca0 con 0x7fac7c102790 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac82318640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c111fa0 con 0x7fac7c101de0 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac808b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac808b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54468/0 (socket says 172.21.15.86:54468) 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac808b5640 1 -- 172.21.15.86:0/1288938801 learned_addr learned my addr 172.21.15.86:0/1288938801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 0x7fac7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7bfff640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 0x7fac7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1288938801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 msgr2=0x7fac7c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 0x7fac7c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 msgr2=0x7fac7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.352+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1288938801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c1a6850 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac7bfff640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 0x7fac7c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac808b5640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 0x7fac7c074a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9f0 tx=0x7fac7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac797fa640 1 -- 172.21.15.86:0/1288938801 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000ccc0 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac797fa640 1 -- 172.21.15.86:0/1288938801 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac700108e0 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac797fa640 1 -- 172.21.15.86:0/1288938801 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70018880 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c1a6b40 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.353+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c1a70d0 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.355+0000 7fac797fa640 1 -- 172.21.15.86:0/1288938801 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fac70010a90 con 0x7fac7c102790 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac846f1740 1 Processor -- start 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac846f1740 1 -- start start 2024-09-05T06:43:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac797fa640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 0x7fac640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac846f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac846f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2b6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.356+0000 7fac846f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac846f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ba2ad12f0 con 0x557ba2b76370 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac846f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ba2a78cf0 con 0x557ba2b6b5d0 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac846f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557ba2acd2b0 con 0x557ba2b6cb40 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac808b5640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 0x7fac640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac797fa640 1 -- 172.21.15.86:0/1288938801 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fac700cfb20 con 0x7fac7c102790 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55294/0 (socket says 172.21.15.86:55294) 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 -- 172.21.15.86:0/2473574391 learned_addr learned my addr 172.21.15.86:0/2473574391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2b6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac808b5640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 -- 172.21.15.86:0/2473574391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 msgr2=0x557ba2b6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2b6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 -- 172.21.15.86:0/2473574391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 msgr2=0x557ba2b78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b78760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 -- 172.21.15.86:0/2473574391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba2a5d3a0 con 0x557ba2b6cb40 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac808b5640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac808b5640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 0x7fac640a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac740049d0 tx=0x7fac7400fef0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.357+0000 7fac7bfff640 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009900 tx=0x7fac6c005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b7f42db7219984 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.358+0000 7fac627fc640 1 -- 172.21.15.86:0/2473574391 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c040070 con 0x557ba2b6cb40 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.358+0000 7fac627fc640 1 -- 172.21.15.86:0/2473574391 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac6c031860 con 0x557ba2b6cb40 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.358+0000 7fac627fc640 1 -- 172.21.15.86:0/2473574391 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c035040 con 0x557ba2b6cb40 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- 172.21.15.86:0/2473574391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 msgr2=0x557ba2b6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009900 tx=0x7fac6c005d90 comp rx=0 tx=0).stop 2024-09-05T06:43:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- 172.21.15.86:0/2473574391 shutdown_connections 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2b6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 --2- 172.21.15.86:0/2473574391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b6b090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- 172.21.15.86:0/2473574391 >> 172.21.15.86:0/2473574391 conn(0x557ba2b34310 msgr2=0x557ba2b67c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- 172.21.15.86:0/2473574391 shutdown_connections 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- 172.21.15.86:0/2473574391 wait complete. 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.359+0000 7fac846f1740 1 -- start start 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 Processor -- start 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 0x557ba2b75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ba2b79010 con 0x557ba2b76370 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ba2a8c060 con 0x557ba2b6cb40 2024-09-05T06:43:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557ba2b79190 con 0x557ba2b6b5d0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac846f1740 10 client.0 _ll_register_callbacks cb 0x557ba27df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 0x557ba2b75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac808b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42434/0 (socket says 172.21.15.86:42434) 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1326445322 learned_addr learned my addr 172.21.15.86:0/1326445322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1326445322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 msgr2=0x557ba2b75cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 0x557ba2b75cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1326445322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 msgr2=0x557ba2b73520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b73520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7b7fe640 1 -- 172.21.15.86:0/1326445322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba28423c0 con 0x557ba2b6cb40 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.360+0000 7fac7bfff640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 0x557ba2b75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac7009b9b0 tx=0x7fac7009bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f341f35d147a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700a4870 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fac70010430 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6c031860 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac70010630 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 10 client.25406 fetch_fsmap learned FSMap version 29 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ba2b84a40 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac6c031860 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fac7007acd0 con 0x557ba2b6cb40 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 10 client.25406 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 10 client.25406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 20 client.25406 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2b85f00 0x557ba2b88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 4 client.25406 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.361+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ba2ade860 con 0x557ba2b85f00 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.362+0000 7fac7bfff640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2b85f00 0x557ba2b88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.362+0000 7fac5b7fe640 10 client.25406 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.362+0000 7fac7bfff640 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2b85f00 0x557ba2b88330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698f419551f9e897 server_cookie=3a2b2b5d67ac207e in_seq=0 out_seq=0 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac5b7fe640 1 -- 172.21.15.86:0/1326445322 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ba2ade860 con 0x557ba2b85f00 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac5b7fe640 10 client.25406 handle_command_reply: tid=0 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 client.25406 shutdown 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 msgr2=0x557ba2b72fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac7009b9b0 tx=0x7fac7009bd70 comp rx=0 tx=0).stop 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 shutdown_connections 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2b85f00 0x557ba2b88330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6cb40 0x557ba2b72fa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 --2- 172.21.15.86:0/1326445322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6b5d0 0x557ba2b75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac5b7fe640 0 client.25406 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 >> 172.21.15.86:0/1326445322 conn(0x557ba2b34310 msgr2=0x557ba2b6d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 shutdown_connections 2024-09-05T06:43:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 1 -- 172.21.15.86:0/1326445322 wait complete. 2024-09-05T06:43:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.364+0000 7fac846f1740 20 client.25406 trim_cache size 0 max 16384 2024-09-05T06:43:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 Processor -- start 2024-09-05T06:43:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 -- start start 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 0x557ba2b822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b72860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 0x557ba2b6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1b1210 con 0x557ba2b6b5d0 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1b2d80 con 0x557ba2b76370 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac846f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1b18e0 con 0x557ba2b6cb40 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac808b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 0x557ba2b6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.378+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 0x557ba2b822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac808b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 0x557ba2b6e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42444/0 (socket says 172.21.15.86:42444) 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac808b5640 1 -- 172.21.15.86:0/747005139 learned_addr learned my addr 172.21.15.86:0/747005139 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b72860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 -- 172.21.15.86:0/747005139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 msgr2=0x557ba2b72860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b72860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 -- 172.21.15.86:0/747005139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 msgr2=0x557ba2b6e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 0x557ba2b6e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 -- 172.21.15.86:0/747005139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba2b76060 con 0x557ba2b6b5d0 2024-09-05T06:43:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b72860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac7bfff640 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 0x557ba2b822b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac6c031a70 tx=0x7fac6c005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28703ed8209b5f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac61ffb640 1 -- 172.21.15.86:0/747005139 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c035040 con 0x557ba2b6b5d0 2024-09-05T06:43:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac61ffb640 1 -- 172.21.15.86:0/747005139 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac6c04e920 con 0x557ba2b6b5d0 2024-09-05T06:43:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.379+0000 7fac61ffb640 1 -- 172.21.15.86:0/747005139 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c040040 con 0x557ba2b6b5d0 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 -- 172.21.15.86:0/747005139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 msgr2=0x557ba2b822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 0x557ba2b822b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fac6c031a70 tx=0x7fac6c005460 comp rx=0 tx=0).stop 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 -- 172.21.15.86:0/747005139 shutdown_connections 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b76370 0x557ba2b6e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2b72860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 --2- 172.21.15.86:0/747005139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b6b5d0 0x557ba2b822b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 -- 172.21.15.86:0/747005139 >> 172.21.15.86:0/747005139 conn(0x557ba2b34310 msgr2=0x557ba2b6abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 -- 172.21.15.86:0/747005139 shutdown_connections 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.380+0000 7fac846f1740 1 -- 172.21.15.86:0/747005139 wait complete. 2024-09-05T06:43:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 -- start start 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 Processor -- start 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2be9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1b2680 con 0x557ba2b76370 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1b1fb0 con 0x557ba2b6b5d0 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1af1f0 con 0x557ba2b6cb40 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac846f1740 10 client.0 _ll_register_callbacks cb 0x7fac7000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.381+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42448/0 (socket says 172.21.15.86:42448) 2024-09-05T06:43:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 -- 172.21.15.86:0/481530124 learned_addr learned my addr 172.21.15.86:0/481530124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac808b5640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2be9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 -- 172.21.15.86:0/481530124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 msgr2=0x557ba2be9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2be9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 -- 172.21.15.86:0/481530124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 msgr2=0x557ba2b799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b799e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 -- 172.21.15.86:0/481530124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba2b7a0d0 con 0x557ba2b6b5d0 2024-09-05T06:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac7bfff640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c02a930 tx=0x7fac6c0314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b269ba7bad0647a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c035030 con 0x557ba2b6b5d0 2024-09-05T06:43:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fac6c04e680 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.382+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6c04e920 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac6c066050 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 10 client.25412 fetch_fsmap learned FSMap version 29 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ba2bf0590 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac6c05d070 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fac6c058990 con 0x557ba2b6b5d0 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 10 client.25412 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 20 client.25412 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2bf07b0 0x557ba2bf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 4 client.25412 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.383+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fac6c04e920 con 0x557ba2bf07b0 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.384+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2bf07b0 0x557ba2bf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.384+0000 7fac5b7fe640 10 client.25412 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.384+0000 7fac7b7fe640 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2bf07b0 0x557ba2bf2ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52220473a61dbebb server_cookie=99100f597681a4eb in_seq=0 out_seq=0 2024-09-05T06:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac5b7fe640 1 -- 172.21.15.86:0/481530124 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fac6c04e920 con 0x557ba2bf07b0 2024-09-05T06:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac5b7fe640 10 client.25412 handle_command_reply: tid=0 2024-09-05T06:43:00.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 client.25412 shutdown 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 msgr2=0x557ba2be8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c02a930 tx=0x7fac6c0314a0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 shutdown_connections 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557ba2bf07b0 0x557ba2bf2ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ba2b76370 0x557ba2b799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2b6cb40 0x557ba2be9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac5b7fe640 0 client.25412 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 --2- 172.21.15.86:0/481530124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ba2b6b5d0 0x557ba2be8e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.385+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 >> 172.21.15.86:0/481530124 conn(0x557ba2b34310 msgr2=0x557ba2b74800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 shutdown_connections 2024-09-05T06:43:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac846f1740 1 -- 172.21.15.86:0/481530124 wait complete. 2024-09-05T06:43:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac846f1740 20 client.25412 trim_cache size 0 max 16384 2024-09-05T06:43:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 msgr2=0x7fac640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 0x7fac640a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac740049d0 tx=0x7fac7400fef0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 msgr2=0x7fac7c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 0x7fac7c074a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9f0 tx=0x7fac7000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 shutdown_connections 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac640a2d70 0x7fac640a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7c106f10 0x7fac7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7c102790 0x7fac7c074a30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 --2- 172.21.15.86:0/1288938801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac7c101de0 0x7fac7c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:00.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.386+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 >> 172.21.15.86:0/1288938801 conn(0x7fac7c0fd880 msgr2=0x7fac7c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:00.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.387+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 shutdown_connections 2024-09-05T06:43:00.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:00.387+0000 7fac82318640 1 -- 172.21.15.86:0/1288938801 wait complete. 2024-09-05T06:43:00.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 14487 4.86 MB/sec warmup 20 sec latency 24.163 ms 2024-09-05T06:43:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:00 smithi115 ceph-mon[34791]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-09-05T06:43:00.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:00 smithi086 ceph-mon[30943]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-09-05T06:43:00.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:00 smithi096 ceph-mon[34468]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-09-05T06:43:01.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:01 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:43:01.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:01 smithi086 ceph-mon[30943]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-09-05T06:43:01.504 INFO:tasks.workunit.client.0.smithi086.stdout: 1 14890 4.65 MB/sec warmup 21 sec latency 13.813 ms 2024-09-05T06:43:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:01 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:43:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:01 smithi096 ceph-mon[34468]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-09-05T06:43:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:01 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:43:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:01 smithi115 ceph-mon[34791]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-09-05T06:43:02.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:43:02.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:43:02.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 15451 4.53 MB/sec warmup 22 sec latency 13.645 ms 2024-09-05T06:43:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 -- 172.21.15.86:0/999046811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102ef0 msgr2=0x7f370810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 --2- 172.21.15.86:0/999046811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102ef0 0x7f370810da40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009900 tx=0x7f36f802a480 comp rx=0 tx=0).stop 2024-09-05T06:43:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 -- 172.21.15.86:0/999046811 shutdown_connections 2024-09-05T06:43:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 --2- 172.21.15.86:0/999046811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102ef0 0x7f370810da40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 --2- 172.21.15.86:0/999046811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102570 0x7f37081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 --2- 172.21.15.86:0/999046811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 0x7f3708101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 -- 172.21.15.86:0/999046811 >> 172.21.15.86:0/999046811 conn(0x7f37080fc7d0 msgr2=0x7f37080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 -- 172.21.15.86:0/999046811 shutdown_connections 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.135+0000 7f371040a640 1 -- 172.21.15.86:0/999046811 wait complete. 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 Processor -- start 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 -- start start 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 0x7f37081972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102ef0 0x7f370819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110e30 con 0x7f3708102570 2024-09-05T06:43:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110cb0 con 0x7f3708102ef0 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f371040a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110fb0 con 0x7f3708100e20 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34272/0 (socket says 172.21.15.86:34272) 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 -- 172.21.15.86:0/4251896373 learned_addr learned my addr 172.21.15.86:0/4251896373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370e9a7640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102ef0 0x7f370819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370e1a6640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 0x7f37081972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 -- 172.21.15.86:0/4251896373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 msgr2=0x7f37081972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 0x7f37081972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 -- 172.21.15.86:0/4251896373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102ef0 msgr2=0x7f370819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102ef0 0x7f370819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.136+0000 7f370d9a5640 1 -- 172.21.15.86:0/4251896373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819c260 con 0x7f3708102570 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f370d9a5640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ebc0 tx=0x7f36fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f37077fe640 1 -- 172.21.15.86:0/4251896373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc010630 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f37077fe640 1 -- 172.21.15.86:0/4251896373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36fc010c80 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f37077fe640 1 -- 172.21.15.86:0/4251896373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc018b30 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819c550 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.137+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a3e30 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.139+0000 7f37077fe640 1 -- 172.21.15.86:0/4251896373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36fc018cd0 con 0x7f3708102570 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 Processor -- start 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 -- start start 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9f02340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fa9e58280 con 0x559fa9ef29a0 2024-09-05T06:43:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559fa9dffb10 con 0x559fa9efff50 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37127e3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fa9e54240 con 0x559fa9ef32c0 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.140+0000 7f37077fe640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 0x7f36f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42810/0 (socket says 172.21.15.86:42810) 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 -- 172.21.15.86:0/3658727302 learned_addr learned my addr 172.21.15.86:0/3658727302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370d9a5640 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9f02340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e9a7640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 0x7f36f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f37077fe640 1 -- 172.21.15.86:0/4251896373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f36fc0cfd50 con 0x7f3708102570 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 -- 172.21.15.86:0/3658727302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 msgr2=0x559fa9f02340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9f02340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 -- 172.21.15.86:0/3658727302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 msgr2=0x559fa9ef2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 -- 172.21.15.86:0/3658727302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa9e0c1f0 con 0x559fa9ef32c0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e9a7640 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3700009d20 tx=0x7f3700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d2f16396826785 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f370e9a7640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 0x7f36f00a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005ec0 tx=0x7f36f8009260 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f3704dea640 1 -- 172.21.15.86:0/3658727302 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370001e070 con 0x559fa9ef32c0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f3704dea640 1 -- 172.21.15.86:0/3658727302 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3700019ae0 con 0x559fa9ef32c0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.141+0000 7f3704dea640 1 -- 172.21.15.86:0/3658727302 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000e9f0 con 0x559fa9ef32c0 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 -- 172.21.15.86:0/3658727302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 msgr2=0x559fa9effa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3700009d20 tx=0x7f3700011810 comp rx=0 tx=0).stop 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 -- 172.21.15.86:0/3658727302 shutdown_connections 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9f02340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9effa10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 --2- 172.21.15.86:0/3658727302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 -- 172.21.15.86:0/3658727302 >> 172.21.15.86:0/3658727302 conn(0x559fa9eef0f0 msgr2=0x559fa9eef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 -- 172.21.15.86:0/3658727302 shutdown_connections 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.142+0000 7f37127e3740 1 -- 172.21.15.86:0/3658727302 wait complete. 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 -- start start 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 Processor -- start 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef29a0 0x559fa9efcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9efd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fa9f02b20 con 0x559fa9efff50 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559fa9e12f10 con 0x559fa9ef29a0 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fa9f02ca0 con 0x559fa9ef32c0 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f37127e3740 10 client.0 _ll_register_callbacks cb 0x559fa9ddba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f370e9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f370e9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34290/0 (socket says 172.21.15.86:34290) 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f370e9a7640 1 -- 172.21.15.86:0/586263308 learned_addr learned my addr 172.21.15.86:0/586263308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f370d9a5640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9efd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.143+0000 7f370e1a6640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef29a0 0x559fa9efcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 -- 172.21.15.86:0/586263308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 msgr2=0x559fa9efd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9efd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 -- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef29a0 msgr2=0x559fa9efcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef29a0 0x559fa9efcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 -- 172.21.15.86:0/586263308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa9bc96a0 con 0x559fa9efff50 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370d9a5640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9efd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f370e9a7640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36f8032fc0 tx=0x7f36f802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e3a112cd3c44a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:03.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8035070 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700019ae0 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f36f8004320 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36f8058050 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f37127e3740 10 client.15957 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fa9f0e860 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3700019ae0 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.144+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f36f8004b10 con 0x559fa9efff50 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 10 client.15957 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 20 client.15957 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 4 client.15957 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa9e657f0 con 0x559fa9f0fcc0 2024-09-05T06:43:03.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f370e1a6640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f370e1a6640 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f12130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2b446ebc8768a server_cookie=6b7c43f728b4c0e5 in_seq=0 out_seq=0 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.145+0000 7f36edffb640 10 client.15957 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f36edffb640 1 -- 172.21.15.86:0/586263308 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fa9e657f0 con 0x559fa9f0fcc0 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f36edffb640 10 client.15957 handle_command_reply: tid=0 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 client.15957 shutdown 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 msgr2=0x559fa9efd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36f8032fc0 tx=0x7f36f802a480 comp rx=0 tx=0).stop 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 shutdown_connections 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f12130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9efd960 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f36edffb640 0 client.15957 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9efd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 --2- 172.21.15.86:0/586263308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef29a0 0x559fa9efcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 >> 172.21.15.86:0/586263308 conn(0x559fa9eef0f0 msgr2=0x559fa9eee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 shutdown_connections 2024-09-05T06:43:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 1 -- 172.21.15.86:0/586263308 wait complete. 2024-09-05T06:43:03.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.146+0000 7f37127e3740 20 client.15957 trim_cache size 0 max 16384 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.165+0000 7f37127e3740 1 Processor -- start 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.165+0000 7f37127e3740 1 -- start start 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 0x559fa9efed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef32c0 0x559fa9f17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ace50 con 0x559fa9efff50 2024-09-05T06:43:03.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ae9f0 con 0x559fa9ef32c0 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f37127e3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ad520 con 0x559fa9ef29a0 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e1a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 0x559fa9efed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34306/0 (socket says 172.21.15.86:34306) 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 -- 172.21.15.86:0/2097792336 learned_addr learned my addr 172.21.15.86:0/2097792336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370d9a5640 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef32c0 0x559fa9f17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 -- 172.21.15.86:0/2097792336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 msgr2=0x559fa9efed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 0x559fa9efed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 -- 172.21.15.86:0/2097792336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef32c0 msgr2=0x559fa9f17a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef32c0 0x559fa9f17a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 -- 172.21.15.86:0/2097792336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa9efbe90 con 0x559fa9efff50 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e1a6640 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 0x559fa9efed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.166+0000 7f370e9a7640 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f36f8033000 tx=0x7f36f8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468bb3ede1761bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.167+0000 7f36effff640 1 -- 172.21.15.86:0/2097792336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8060070 con 0x559fa9efff50 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.167+0000 7f36effff640 1 -- 172.21.15.86:0/2097792336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36f8031640 con 0x559fa9efff50 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.167+0000 7f36effff640 1 -- 172.21.15.86:0/2097792336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8035040 con 0x559fa9efff50 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.167+0000 7f37127e3740 1 -- 172.21.15.86:0/2097792336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 msgr2=0x559fa9f17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.167+0000 7f37127e3740 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f36f8033000 tx=0x7f36f8005b50 comp rx=0 tx=0).stop 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- 172.21.15.86:0/2097792336 shutdown_connections 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9efff50 0x559fa9f17f40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9ef32c0 0x559fa9f17a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- 172.21.15.86:0/2097792336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef29a0 0x559fa9efed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- 172.21.15.86:0/2097792336 >> 172.21.15.86:0/2097792336 conn(0x559fa9eef0f0 msgr2=0x559fa9efe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- 172.21.15.86:0/2097792336 shutdown_connections 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- 172.21.15.86:0/2097792336 wait complete. 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- start start 2024-09-05T06:43:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 Processor -- start 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9ef5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9ef6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ae2c0 con 0x559fa9ef29a0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37081adbf0 con 0x559fa9efff50 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37081aae30 con 0x559fa9ef32c0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.168+0000 7f37127e3740 10 client.0 _ll_register_callbacks cb 0x7f36f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34322/0 (socket says 172.21.15.86:34322) 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 -- 172.21.15.86:0/3596466967 learned_addr learned my addr 172.21.15.86:0/3596466967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370d9a5640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9ef5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e9a7640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9ef6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 -- 172.21.15.86:0/3596466967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 msgr2=0x559fa9ef5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9ef5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 -- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 msgr2=0x559fa9ef6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9ef6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 -- 172.21.15.86:0/3596466967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa9f08b60 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f370e1a6640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3700009800 tx=0x7f370001aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e324d0dd7f8973a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f36edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370001e040 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3700005450 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f8031640 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.169+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3700036050 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 10 client.15963 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fa9f812f0 con 0x559fa9ef29a0 2024-09-05T06:43:03.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36f8031640 con 0x559fa9ef29a0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3700025040 con 0x559fa9ef29a0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 20 client.15963 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 4 client.15963 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559fa9e5fd00 con 0x559fa9f0fcc0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.170+0000 7f370d9a5640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.171+0000 7f36edffb640 10 client.15963 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.171+0000 7f370d9a5640 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f83910 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0555af086380cac server_cookie=5293304530409e3e in_seq=0 out_seq=0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f36edffb640 1 -- 172.21.15.86:0/3596466967 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559fa9e5fd00 con 0x559fa9f0fcc0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f36edffb640 10 client.15963 handle_command_reply: tid=0 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 client.15963 shutdown 2024-09-05T06:43:03.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 msgr2=0x559fa9ef57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3700009800 tx=0x7f370001aed0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 shutdown_connections 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559fa9f0fcc0 0x559fa9f83910 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559fa9efff50 0x559fa9ef6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fa9ef32c0 0x559fa9ef5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 --2- 172.21.15.86:0/3596466967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fa9ef29a0 0x559fa9ef57a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 >> 172.21.15.86:0/3596466967 conn(0x559fa9eef0f0 msgr2=0x559fa9efd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f36edffb640 0 client.15963 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 shutdown_connections 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 1 -- 172.21.15.86:0/3596466967 wait complete. 2024-09-05T06:43:03.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.172+0000 7f37127e3740 20 client.15963 trim_cache size 0 max 16384 2024-09-05T06:43:03.180 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:03.180 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:43:03.180 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "89aea890-5c9c-4aec-ab62-1b6720e6dc3a", 2024-09-05T06:43:03.180 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:43:03.180 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 msgr2=0x7f36f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 0x7f36f00a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005ec0 tx=0x7f36f8009260 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 msgr2=0x7f37081977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ebc0 tx=0x7f36fc00c6a0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 shutdown_connections 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f36f00a2d70 0x7f36f00a5230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3708102ef0 0x7f370819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3708102570 0x7f37081977f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 --2- 172.21.15.86:0/4251896373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3708100e20 0x7f37081972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 >> 172.21.15.86:0/4251896373 conn(0x7f37080fc7d0 msgr2=0x7f37080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 shutdown_connections 2024-09-05T06:43:03.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:03.173+0000 7f371040a640 1 -- 172.21.15.86:0/4251896373 wait complete. 2024-09-05T06:43:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:03.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:03.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 16110 4.48 MB/sec warmup 23 sec latency 25.048 ms 2024-09-05T06:43:03.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:03.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:04 smithi086 ceph-mon[30943]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:04.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 16746 4.39 MB/sec warmup 24 sec latency 14.143 ms 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:04 smithi096 ceph-mon[34468]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:43:04.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:04.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:04 smithi115 ceph-mon[34791]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:05.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 17491 4.47 MB/sec warmup 25 sec latency 63.456 ms 2024-09-05T06:43:06.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 17953 4.47 MB/sec warmup 26 sec latency 59.516 ms 2024-09-05T06:43:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:06 smithi115 ceph-mon[34791]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:06.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:06 smithi086 ceph-mon[30943]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:06 smithi096 ceph-mon[34468]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-09-05T06:43:07.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 18210 4.32 MB/sec warmup 27 sec latency 31.844 ms 2024-09-05T06:43:07.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:07 smithi086 ceph-mon[30943]: pgmap v59: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T06:43:07.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:43:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:07 smithi096 ceph-mon[34468]: pgmap v59: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T06:43:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:07 smithi115 ceph-mon[34791]: pgmap v59: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-09-05T06:43:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:08.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 18657 4.19 MB/sec warmup 28 sec latency 34.634 ms 2024-09-05T06:43:09.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- 172.21.15.86:0/3176670438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 msgr2=0x7f35fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 --2- 172.21.15.86:0/3176670438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc101200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009900 tx=0x7f35ec02a480 comp rx=0 tx=0).stop 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- 172.21.15.86:0/3176670438 shutdown_connections 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 --2- 172.21.15.86:0/3176670438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102ef0 0x7f35fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 --2- 172.21.15.86:0/3176670438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102570 0x7f35fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 --2- 172.21.15.86:0/3176670438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc101200 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- 172.21.15.86:0/3176670438 >> 172.21.15.86:0/3176670438 conn(0x7f35fc0fc7d0 msgr2=0x7f35fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- 172.21.15.86:0/3176670438 shutdown_connections 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- 172.21.15.86:0/3176670438 wait complete. 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 Processor -- start 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.458+0000 7f3602919640 1 -- start start 2024-09-05T06:43:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102570 0x7f35fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 0x7f35fc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc110e30 con 0x7f35fc100e20 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc110cb0 con 0x7f35fc102ef0 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc110fb0 con 0x7f35fc102570 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34346/0 (socket says 172.21.15.86:34346) 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 -- 172.21.15.86:0/1027243419 learned_addr learned my addr 172.21.15.86:0/1027243419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35f3fff640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102570 0x7f35fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3600eb6640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 0x7f35fc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 -- 172.21.15.86:0/1027243419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102570 msgr2=0x7f35fc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102570 0x7f35fc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 -- 172.21.15.86:0/1027243419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 msgr2=0x7f35fc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 0x7f35fc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 -- 172.21.15.86:0/1027243419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19d380 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3600eb6640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 0x7f35fc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35fbfff640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006010 tx=0x7f35ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35f9ffb640 1 -- 172.21.15.86:0/1027243419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec03f030 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35f9ffb640 1 -- 172.21.15.86:0/1027243419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ec003aa0 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc19d610 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.459+0000 7f35f9ffb640 1 -- 172.21.15.86:0/1027243419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec035070 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.460+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a4f50 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f35f9ffb640 1 -- 172.21.15.86:0/1027243419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f35ec042030 con 0x7f35fc100e20 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 Processor -- start 2024-09-05T06:43:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 -- start start 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563688ffe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.462+0000 7f3604cf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563688f642e0 con 0x56368900da90 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f3604cf2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563688f0bce0 con 0x563688ffeca0 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f3604cf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563688f602a0 con 0x563688ffe380 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f3600eb6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563688ffe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43998/0 (socket says 172.21.15.86:43998) 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34360/0 (socket says 172.21.15.86:34360) 2024-09-05T06:43:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 -- 172.21.15.86:0/3471969117 learned_addr learned my addr 172.21.15.86:0/3471969117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35f9ffb640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 -- 172.21.15.86:0/3471969117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 msgr2=0x563688ffe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563688ffe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 -- 172.21.15.86:0/3471969117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 msgr2=0x56368900fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 -- 172.21.15.86:0/3471969117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563688ef0440 con 0x563688ffeca0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f3600eb6640 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563688ffe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f3600eb6640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35f9ffb640 1 -- 172.21.15.86:0/1027243419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f35ec0efb50 con 0x7f35fc100e20 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.463+0000 7f35fbfff640 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0ece30 tx=0x7f35ec0868c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b81d2ea82a9744 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3600eb6640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 0x7f35e80a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005fd0 tx=0x7f35f4004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f35f27fc640 1 -- 172.21.15.86:0/3471969117 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec049020 con 0x563688ffeca0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f35f27fc640 1 -- 172.21.15.86:0/3471969117 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ec0bc390 con 0x563688ffeca0 2024-09-05T06:43:09.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f35f27fc640 1 -- 172.21.15.86:0/3471969117 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0bb7d0 con 0x563688ffeca0 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 -- 172.21.15.86:0/3471969117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 msgr2=0x56368900d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0ece30 tx=0x7f35ec0868c0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 -- 172.21.15.86:0/3471969117 shutdown_connections 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900d550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 --2- 172.21.15.86:0/3471969117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563688ffe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 -- 172.21.15.86:0/3471969117 >> 172.21.15.86:0/3471969117 conn(0x563688ffaa30 msgr2=0x563688ffae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 -- 172.21.15.86:0/3471969117 shutdown_connections 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.464+0000 7f3604cf2740 1 -- 172.21.15.86:0/3471969117 wait complete. 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 -- start start 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 Processor -- start 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x56368900d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563689010420 con 0x56368900da90 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563688f1f050 con 0x563688ffeca0 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5636890105a0 con 0x563688ffe380 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3604cf2740 10 client.0 _ll_register_callbacks cb 0x563688f5f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3600eb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x56368900d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3600eb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34372/0 (socket says 172.21.15.86:34372) 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44006/0 (socket says 172.21.15.86:44006) 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.465+0000 7f3600eb6640 1 -- 172.21.15.86:0/4260648195 learned_addr learned my addr 172.21.15.86:0/4260648195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 -- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 msgr2=0x56368900a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 -- 172.21.15.86:0/4260648195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 msgr2=0x56368900aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 -- 172.21.15.86:0/4260648195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563688cd53c0 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35f3fff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35fbfff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x56368900d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005d00 tx=0x7f35ec0efe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ac911b5c256a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec049040 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec0bc390 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f35ec0b7b60 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35ec0bb5f0 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f3604cf2740 10 client.25396 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.466+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563689010420 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ec0bc390 con 0x563688ffe380 2024-09-05T06:43:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35ec0ed140 con 0x563688ffe380 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 10 client.25396 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 20 client.25396 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368901f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 4 client.25396 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563688f71850 con 0x56368901cf90 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f35f3fff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368901f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f35db7fe640 10 client.25396 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.467+0000 7f35f3fff640 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368901f3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604b8c397e995e7e server_cookie=c459824f0f9171bb in_seq=0 out_seq=0 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.468+0000 7f35db7fe640 1 -- 172.21.15.86:0/4260648195 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563688f71850 con 0x56368901cf90 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.468+0000 7f35db7fe640 10 client.25396 handle_command_reply: tid=0 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.468+0000 7f3604cf2740 1 client.25396 shutdown 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 msgr2=0x56368900d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x56368900d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005d00 tx=0x7f35ec0efe50 comp rx=0 tx=0).stop 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 shutdown_connections 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368901f3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368900da90 0x56368900aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f35db7fe640 0 client.25396 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563688ffeca0 0x56368900a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 --2- 172.21.15.86:0/4260648195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x56368900d3b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 >> 172.21.15.86:0/4260648195 conn(0x563688ffaa30 msgr2=0x563689000790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 shutdown_connections 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 1 -- 172.21.15.86:0/4260648195 wait complete. 2024-09-05T06:43:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.469+0000 7f3604cf2740 20 client.25396 trim_cache size 0 max 16384 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.487+0000 7f3604cf2740 1 Processor -- start 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.487+0000 7f3604cf2740 1 -- start start 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689008f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x5636890251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1acd90 con 0x563688ffeca0 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1ae930 con 0x56368900da90 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3604cf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1ad460 con 0x563688ffe380 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34388/0 (socket says 172.21.15.86:34388) 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 -- 172.21.15.86:0/258214446 learned_addr learned my addr 172.21.15.86:0/258214446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689008f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f3600eb6640 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x5636890251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 -- 172.21.15.86:0/258214446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 msgr2=0x563689008f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689008f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 -- 172.21.15.86:0/258214446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 msgr2=0x5636890251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x5636890251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 -- 172.21.15.86:0/258214446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56368900d720 con 0x563688ffeca0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35fbfff640 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689008f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f3fff640 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0fff70 tx=0x7f35e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30eae84958c57739 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.488+0000 7f35f1ffb640 1 -- 172.21.15.86:0/258214446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e400fac0 con 0x563688ffeca0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f35f1ffb640 1 -- 172.21.15.86:0/258214446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e400fc60 con 0x563688ffeca0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f35f1ffb640 1 -- 172.21.15.86:0/258214446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4026600 con 0x563688ffeca0 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 -- 172.21.15.86:0/258214446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 msgr2=0x563689024ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0fff70 tx=0x7f35e4005f90 comp rx=0 tx=0).stop 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 -- 172.21.15.86:0/258214446 shutdown_connections 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x5636890251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689024ca0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 --2- 172.21.15.86:0/258214446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689008f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 -- 172.21.15.86:0/258214446 >> 172.21.15.86:0/258214446 conn(0x563688ffaa30 msgr2=0x563688ff9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.489+0000 7f3604cf2740 1 -- 172.21.15.86:0/258214446 shutdown_connections 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 -- 172.21.15.86:0/258214446 wait complete. 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 -- start start 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 Processor -- start 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689011680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x563689012100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1ae200 con 0x563688ffeca0 2024-09-05T06:43:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1adb30 con 0x56368900da90 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1aad70 con 0x563688ffe380 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.490+0000 7f3604cf2740 10 client.0 _ll_register_callbacks cb 0x7f35e40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689011680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34404/0 (socket says 172.21.15.86:34404) 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35f3fff640 1 -- 172.21.15.86:0/282630490 learned_addr learned my addr 172.21.15.86:0/282630490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f3600eb6640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x563689012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 -- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 msgr2=0x563689012100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x563689012100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 -- 172.21.15.86:0/282630490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 msgr2=0x563689011bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 -- 172.21.15.86:0/282630490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563689086dd0 con 0x563688ffe380 2024-09-05T06:43:09.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35f3fff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35fbfff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689011680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0c49f0 tx=0x7f35ec0b8510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee95e0d52963c36f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec049070 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f35ec0b7b60 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e400fc60 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.491+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35ec0bb7d0 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 10 client.25402 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563689086dd0 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35ec0bc390 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35ec003c50 con 0x563688ffe380 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 10 client.25402 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 20 client.25402 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368908a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 4 client.25402 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f35e400fc60 con 0x56368901cf90 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f35f3fff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368908a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f35db7fe640 10 client.25402 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.492+0000 7f35f3fff640 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368908a440 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c9ec9eac763683 server_cookie=d1d8cc0b191803eb in_seq=0 out_seq=0 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f35db7fe640 1 -- 172.21.15.86:0/282630490 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f35e400fc60 con 0x56368901cf90 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f35db7fe640 10 client.25402 handle_command_reply: tid=0 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 client.25402 shutdown 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 msgr2=0x563689011680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689011680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0c49f0 tx=0x7f35ec0b8510 comp rx=0 tx=0).stop 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 shutdown_connections 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56368901cf90 0x56368908a440 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56368900da90 0x563689012100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563688ffeca0 0x563689011bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 --2- 172.21.15.86:0/282630490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563688ffe380 0x563689011680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 >> 172.21.15.86:0/282630490 conn(0x563688ffaa30 msgr2=0x563689009ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f35db7fe640 0 client.25402 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 shutdown_connections 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.493+0000 7f3604cf2740 1 -- 172.21.15.86:0/282630490 wait complete. 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3604cf2740 20 client.25402 trim_cache size 0 max 16384 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:43:09.507 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: "89aea890-5c9c-4aec-ab62-1b6720e6dc3a": { 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "89aea890-5c9c-4aec-ab62-1b6720e6dc3a", 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:09.508 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 msgr2=0x7f35e80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 0x7f35e80a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005fd0 tx=0x7f35f4004a40 comp rx=0 tx=0).stop 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 msgr2=0x7f35fc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006010 tx=0x7f35ec004270 comp rx=0 tx=0).stop 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 shutdown_connections 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35fc102ef0 0x7f35fc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc102570 0x7f35fc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 --2- 172.21.15.86:0/1027243419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc100e20 0x7f35fc1983d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.494+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 >> 172.21.15.86:0/1027243419 conn(0x7f35fc0fc7d0 msgr2=0x7f35fc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.495+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 shutdown_connections 2024-09-05T06:43:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:09.495+0000 7f3602919640 1 -- 172.21.15.86:0/1027243419 wait complete. 2024-09-05T06:43:09.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 19255 4.19 MB/sec warmup 29 sec latency 56.105 ms 2024-09-05T06:43:10.320 INFO:tasks.cephfs.filesystem:scrub status for tag:89aea890-5c9c-4aec-ab62-1b6720e6dc3a - {'path': '/', 'tag': '89aea890-5c9c-4aec-ab62-1b6720e6dc3a', 'options': 'recursive,force'} 2024-09-05T06:43:10.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 19991 4.08 MB/sec warmup 30 sec latency 22.973 ms 2024-09-05T06:43:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:10 smithi115 ceph-mon[34791]: pgmap v60: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-05T06:43:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:10 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:43:10.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:10 smithi086 ceph-mon[30943]: pgmap v60: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-05T06:43:10.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:10 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:43:10.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:10 smithi096 ceph-mon[34468]: pgmap v60: 129 pgs: 129 active+clean; 35 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-09-05T06:43:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:10 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:43:11.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 20557 4.12 MB/sec warmup 31 sec latency 79.753 ms 2024-09-05T06:43:11.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:43:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:43:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:11 smithi086 ceph-mon[30943]: pgmap v61: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-05T06:43:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:43:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:43:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:11 smithi096 ceph-mon[34468]: pgmap v61: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-05T06:43:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:43:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:43:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:11 smithi115 ceph-mon[34791]: pgmap v61: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-05T06:43:12.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 21169 4.10 MB/sec warmup 32 sec latency 13.779 ms 2024-09-05T06:43:13.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 21666 4.11 MB/sec warmup 33 sec latency 14.538 ms 2024-09-05T06:43:14.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 21998 4.00 MB/sec warmup 34 sec latency 12.509 ms 2024-09-05T06:43:14.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:14 smithi115 ceph-mon[34791]: pgmap v62: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:14.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:14 smithi086 ceph-mon[30943]: pgmap v62: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:14.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:14 smithi096 ceph-mon[34468]: pgmap v62: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:15.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 22555 3.94 MB/sec warmup 35 sec latency 16.356 ms 2024-09-05T06:43:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:15 smithi115 ceph-mon[34791]: pgmap v63: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:15.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:15 smithi086 ceph-mon[30943]: pgmap v63: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:15 smithi096 ceph-mon[34468]: pgmap v63: 129 pgs: 129 active+clean; 36 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:43:16.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 23250 3.92 MB/sec warmup 36 sec latency 17.609 ms 2024-09-05T06:43:17.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 23877 3.88 MB/sec warmup 37 sec latency 13.719 ms 2024-09-05T06:43:17.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:17 smithi086 ceph-mon[30943]: pgmap v64: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T06:43:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:17 smithi096 ceph-mon[34468]: pgmap v64: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T06:43:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:17 smithi115 ceph-mon[34791]: pgmap v64: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T06:43:18.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 24636 3.95 MB/sec warmup 38 sec latency 14.443 ms 2024-09-05T06:43:19.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 25127 3.96 MB/sec warmup 39 sec latency 13.813 ms 2024-09-05T06:43:19.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:19 smithi086 ceph-mon[30943]: pgmap v65: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:43:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:19 smithi096 ceph-mon[34468]: pgmap v65: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:43:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:19 smithi115 ceph-mon[34791]: pgmap v65: 129 pgs: 129 active+clean; 37 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:43:20.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 25430 3.87 MB/sec warmup 40 sec latency 15.418 ms 2024-09-05T06:43:21.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 25933 3.82 MB/sec warmup 41 sec latency 48.921 ms 2024-09-05T06:43:21.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:21 smithi086 ceph-mon[30943]: pgmap v66: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-05T06:43:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:21 smithi096 ceph-mon[34468]: pgmap v66: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-05T06:43:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:21 smithi115 ceph-mon[34791]: pgmap v66: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-05T06:43:22.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 26567 3.81 MB/sec warmup 42 sec latency 14.998 ms 2024-09-05T06:43:23.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 27267 3.76 MB/sec warmup 43 sec latency 17.210 ms 2024-09-05T06:43:23.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:23 smithi086 ceph-mon[30943]: pgmap v67: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:23 smithi096 ceph-mon[34468]: pgmap v67: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:23 smithi115 ceph-mon[34791]: pgmap v67: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:24.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28075 3.82 MB/sec warmup 44 sec latency 13.387 ms 2024-09-05T06:43:25.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28557 3.84 MB/sec warmup 45 sec latency 10.011 ms 2024-09-05T06:43:25.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:25 smithi086 ceph-mon[30943]: pgmap v68: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:25 smithi096 ceph-mon[34468]: pgmap v68: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:25 smithi115 ceph-mon[34791]: pgmap v68: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:26.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28875 3.78 MB/sec warmup 46 sec latency 15.150 ms 2024-09-05T06:43:27.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29307 3.72 MB/sec warmup 47 sec latency 15.222 ms 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: pgmap v69: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:43:28.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:27 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.2 2024-09-05T06:43:28.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: pgmap v69: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:43:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:27 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: pgmap v69: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:43:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:27 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:43:28.494 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:28.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29914 3.73 MB/sec warmup 48 sec latency 11.182 ms 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 -- 172.21.15.86:0/1245192842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 msgr2=0x7fe61410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/1245192842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61410eaa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009ca0 tx=0x7fe60c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 -- 172.21.15.86:0/1245192842 shutdown_connections 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/1245192842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61410eaa0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/1245192842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6141027a0 0x7fe6141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/1245192842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe614101df0 0x7fe6141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 -- 172.21.15.86:0/1245192842 >> 172.21.15.86:0/1245192842 conn(0x7fe6140fd8f0 msgr2=0x7fe6140ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 -- 172.21.15.86:0/1245192842 shutdown_connections 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.023+0000 7fe61bbb7640 1 -- 172.21.15.86:0/1245192842 wait complete. 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 Processor -- start 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 -- start start 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 0x7fe614198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 0x7fe6141988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe614111e30 con 0x7fe614106f20 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe614111cb0 con 0x7fe6141027a0 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61bbb7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe614111fb0 con 0x7fe614101df0 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619152640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 0x7fe6141988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61a154640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619152640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 0x7fe6141988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57916/0 (socket says 172.21.15.86:57916) 2024-09-05T06:43:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619152640 1 -- 172.21.15.86:0/2711628812 learned_addr learned my addr 172.21.15.86:0/2711628812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 0x7fe614198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 -- 172.21.15.86:0/2711628812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 msgr2=0x7fe6141988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 0x7fe6141988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 -- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 msgr2=0x7fe61419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe619953640 1 -- 172.21.15.86:0/2711628812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61419d320 con 0x7fe614101df0 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.024+0000 7fe61a154640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe619953640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 0x7fe614198370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e9f0 tx=0x7fe61000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000ccc0 con 0x7fe614101df0 2024-09-05T06:43:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6100108e0 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610018880 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61419d610 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.025+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6141a4ef0 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.026+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6141021d0 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.027+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe610010a90 con 0x7fe614101df0 2024-09-05T06:43:29.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.028+0000 7fe60affd640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 0x7fe6000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:29.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.028+0000 7fe619152640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 0x7fe6000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:29.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.028+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe6100cfbe0 con 0x7fe614101df0 2024-09-05T06:43:29.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.028+0000 7fe619152640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 0x7fe6000a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6140fdda0 tx=0x7fe604005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.033+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe61009bd70 con 0x7fe614101df0 2024-09-05T06:43:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.272+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6141996d0 con 0x7fe614101df0 2024-09-05T06:43:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.273+0000 7fe60affd640 1 -- 172.21.15.86:0/2711628812 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7fe6100a0070 con 0x7fe614101df0 2024-09-05T06:43:29.273 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:43:29.273 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:43:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 msgr2=0x7fe6000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 0x7fe6000a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6140fdda0 tx=0x7fe604005f90 comp rx=0 tx=0).stop 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 msgr2=0x7fe614198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 0x7fe614198370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe61000e9f0 tx=0x7fe61000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 shutdown_connections 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe6000a2cf0 0x7fe6000a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe614106f20 0x7fe61419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6141027a0 0x7fe6141988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 --2- 172.21.15.86:0/2711628812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe614101df0 0x7fe614198370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 >> 172.21.15.86:0/2711628812 conn(0x7fe6140fd8f0 msgr2=0x7fe6140fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 shutdown_connections 2024-09-05T06:43:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:29.275+0000 7fe61bbb7640 1 -- 172.21.15.86:0/2711628812 wait complete. 2024-09-05T06:43:29.505 INFO:tasks.workunit.client.0.smithi086.stdout: 1 30559 3.67 MB/sec warmup 49 sec latency 24.365 ms 2024-09-05T06:43:29.679 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2711628812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2711628812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2711628812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:30.056 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0' 2024-09-05T06:43:30.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0 2024-09-05T06:43:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:30.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31185 3.70 MB/sec warmup 50 sec latency 15.926 ms 2024-09-05T06:43:30.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:30 smithi115 ceph-mon[34791]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-05T06:43:30.698 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:30 smithi086 ceph-mon[30943]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-05T06:43:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:30 smithi096 ceph-mon[34468]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 -- 172.21.15.86:0/1295316661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 msgr2=0x7f4aa81108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 --2- 172.21.15.86:0/1295316661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa81108e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009ca0 tx=0x7f4aa002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 -- 172.21.15.86:0/1295316661 shutdown_connections 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 --2- 172.21.15.86:0/1295316661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa81108e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 --2- 172.21.15.86:0/1295316661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa8101360 0x7f4aa810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 --2- 172.21.15.86:0/1295316661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa81009b0 0x7f4aa8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 -- 172.21.15.86:0/1295316661 >> 172.21.15.86:0/1295316661 conn(0x7f4aa80fc7d0 msgr2=0x7f4aa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 -- 172.21.15.86:0/1295316661 shutdown_connections 2024-09-05T06:43:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.931+0000 7f4aaf423640 1 -- 172.21.15.86:0/1295316661 wait complete. 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 Processor -- start 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 -- start start 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa81009b0 0x7f4aa819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 0x7f4aa819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81140b0 con 0x7f4aa810e4d0 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8113f30 con 0x7f4aa8101360 2024-09-05T06:43:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.932+0000 7f4aaf423640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8114230 con 0x7f4aa81009b0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54204/0 (socket says 172.21.15.86:54204) 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 -- 172.21.15.86:0/2580143664 learned_addr learned my addr 172.21.15.86:0/2580143664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aac9be640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 0x7f4aa819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa81009b0 0x7f4aa819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 -- 172.21.15.86:0/2580143664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa81009b0 msgr2=0x7f4aa819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa81009b0 0x7f4aa819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 -- 172.21.15.86:0/2580143664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 msgr2=0x7f4aa819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 0x7f4aa819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 -- 172.21.15.86:0/2580143664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa819f580 con 0x7f4aa810e4d0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aac9be640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 0x7f4aa819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aad9c0640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009dc0 tx=0x7f4aa0032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4a9e7fc640 1 -- 172.21.15.86:0/2580143664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0003c30 con 0x7f4aa810e4d0 2024-09-05T06:43:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4a9e7fc640 1 -- 172.21.15.86:0/2580143664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aa0003dd0 con 0x7f4aa810e4d0 2024-09-05T06:43:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa819f810 con 0x7f4aa810e4d0 2024-09-05T06:43:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4a9e7fc640 1 -- 172.21.15.86:0/2580143664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0030aa0 con 0x7f4aa810e4d0 2024-09-05T06:43:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.933+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa81a7150 con 0x7f4aa810e4d0 2024-09-05T06:43:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.936+0000 7f4a9e7fc640 1 -- 172.21.15.86:0/2580143664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4aa0030ce0 con 0x7f4aa810e4d0 2024-09-05T06:43:30.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.936+0000 7f4ab17fc740 1 Processor -- start 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 -- start start 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 0x561527695b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x5615276a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x5615276a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615275fb2a0 con 0x561527696080 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5615275a2b10 con 0x561527695760 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5615275f7260 con 0x5615276a0940 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aad1bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x5615276a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aad1bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x5615276a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54214/0 (socket says 172.21.15.86:54214) 2024-09-05T06:43:30.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aad1bf640 1 -- 172.21.15.86:0/447360234 learned_addr learned my addr 172.21.15.86:0/447360234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 0x561527695b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x5615276a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4a9e7fc640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 0x7f4a940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 -- 172.21.15.86:0/447360234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 msgr2=0x5615276a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x5615276a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 -- 172.21.15.86:0/447360234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 msgr2=0x5615276a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x5615276a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.937+0000 7f4aac9be640 1 -- 172.21.15.86:0/447360234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152736cb90 con 0x561527695760 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4a9e7fc640 1 -- 172.21.15.86:0/2580143664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4aa00f0b80 con 0x7f4aa810e4d0 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4aad9c0640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 0x7f4a940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4aac9be640 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 0x561527695b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4aa810e8f0 tx=0x7f4a98005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0995678e39660d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4a937fe640 1 -- 172.21.15.86:0/447360234 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98013110 con 0x561527695760 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4a937fe640 1 -- 172.21.15.86:0/447360234 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a980132b0 con 0x561527695760 2024-09-05T06:43:30.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.938+0000 7f4a937fe640 1 -- 172.21.15.86:0/447360234 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98005280 con 0x561527695760 2024-09-05T06:43:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- 172.21.15.86:0/447360234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 msgr2=0x561527695b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4aad9c0640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 0x7f4a940a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00b92c0 tx=0x7f4aa00095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 0x561527695b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4aa810e8f0 tx=0x7f4a98005f90 comp rx=0 tx=0).stop 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- 172.21.15.86:0/447360234 shutdown_connections 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x5615276a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x5615276a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/447360234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527695760 0x561527695b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- 172.21.15.86:0/447360234 >> 172.21.15.86:0/447360234 conn(0x561527691cf0 msgr2=0x5615276920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- 172.21.15.86:0/447360234 shutdown_connections 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- 172.21.15.86:0/447360234 wait complete. 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 -- start start 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.939+0000 7f4ab17fc740 1 Processor -- start 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527695760 0x56152769d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x56152769e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615276a3510 con 0x561527696080 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5615275836e0 con 0x5615276a0940 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5615276a3690 con 0x561527695760 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4ab17fc740 10 client.0 _ll_register_callbacks cb 0x5615275a7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54224/0 (socket says 172.21.15.86:54224) 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 -- 172.21.15.86:0/4280386183 learned_addr learned my addr 172.21.15.86:0/4280386183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aac9be640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527695760 0x56152769d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aac9be640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x56152769e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 -- 172.21.15.86:0/4280386183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527695760 msgr2=0x56152769d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527695760 0x56152769d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 -- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 msgr2=0x56152769e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x56152769e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 -- 172.21.15.86:0/4280386183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615276ac9d0 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4012870 tx=0x7f4aa4012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f884450587f052 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.940+0000 7f4a90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004050 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4aa40041f0 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a980132b0 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4025050 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 10 client.15999 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615276af3c0 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a980132b0 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4aa4003990 con 0x561527696080 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 10 client.15999 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 20 client.15999 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5615276b0920 0x5615276b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 4 client.15999 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615276089d0 con 0x5615276b0920 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.941+0000 7f4aac9be640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5615276b0920 0x5615276b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.942+0000 7f4aac9be640 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5615276b0920 0x5615276b2d40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaabb8e83545ede9 server_cookie=87df71eb9de4c77d in_seq=0 out_seq=0 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.942+0000 7f4a90ff9640 10 client.15999 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.942+0000 7f4a90ff9640 1 -- 172.21.15.86:0/4280386183 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615276089d0 con 0x5615276b0920 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.942+0000 7f4a90ff9640 10 client.15999 handle_command_reply: tid=0 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.942+0000 7f4ab17fc740 1 client.15999 shutdown 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 msgr2=0x56152769dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4012870 tx=0x7f4aa4012d30 comp rx=0 tx=0).stop 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 shutdown_connections 2024-09-05T06:43:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5615276b0920 0x5615276b2d40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x56152769e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527696080 0x56152769dd30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/4280386183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527695760 0x56152769d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 >> 172.21.15.86:0/4280386183 conn(0x561527691cf0 msgr2=0x561527690e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4a90ff9640 0 client.15999 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 shutdown_connections 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 1 -- 172.21.15.86:0/4280386183 wait complete. 2024-09-05T06:43:30.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.943+0000 7f4ab17fc740 20 client.15999 trim_cache size 0 max 16384 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 Processor -- start 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 -- start start 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527696080 0x5615276b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x5615276b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81b11b0 con 0x561527695760 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.960+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81b2d50 con 0x5615276a0940 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81b1880 con 0x561527696080 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54232/0 (socket says 172.21.15.86:54232) 2024-09-05T06:43:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 -- 172.21.15.86:0/3675490003 learned_addr learned my addr 172.21.15.86:0/3675490003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527696080 0x5615276b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x5615276b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 -- 172.21.15.86:0/3675490003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527696080 msgr2=0x5615276b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527696080 0x5615276b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 -- 172.21.15.86:0/3675490003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 msgr2=0x5615276b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x5615276b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 -- 172.21.15.86:0/3675490003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152768d260 con 0x561527695760 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800e780 tx=0x7f4a98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1322cc5e56e0cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4a92ffd640 1 -- 172.21.15.86:0/3675490003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800ec70 con 0x561527695760 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4a92ffd640 1 -- 172.21.15.86:0/3675490003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a98005520 con 0x561527695760 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.961+0000 7f4a92ffd640 1 -- 172.21.15.86:0/3675490003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004c10 con 0x561527695760 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3675490003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 msgr2=0x56152769e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800e780 tx=0x7f4a98005f90 comp rx=0 tx=0).stop 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3675490003 shutdown_connections 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5615276a0940 0x5615276b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561527696080 0x5615276b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3675490003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561527695760 0x56152769e0c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3675490003 >> 172.21.15.86:0/3675490003 conn(0x561527691cf0 msgr2=0x56152769c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3675490003 shutdown_connections 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.962+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3675490003 wait complete. 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 -- start start 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 Processor -- start 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x56152769ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152769b370 0x56152771a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81b2620 con 0x56152769b370 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81b1f50 con 0x561527696080 2024-09-05T06:43:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81af190 con 0x5615276a0940 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.963+0000 7f4ab17fc740 10 client.0 _ll_register_callbacks cb 0x7f4aa4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aac9be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aac9be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57970/0 (socket says 172.21.15.86:57970) 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aac9be640 1 -- 172.21.15.86:0/3242871598 learned_addr learned my addr 172.21.15.86:0/3242871598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x56152769ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152769b370 0x56152771a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 -- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 msgr2=0x56152769a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 -- 172.21.15.86:0/3242871598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152769b370 msgr2=0x56152771a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152769b370 0x56152771a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 -- 172.21.15.86:0/3242871598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152771ab90 con 0x5615276a0940 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4aad1bf640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x56152769ae30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4003db0 tx=0x7f4aa4004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2055480afcaf270 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4a90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004440 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4aa4018040 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a98005520 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.964+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4038050 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 10 client.25426 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56152771be30 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a98005520 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4aa4005020 con 0x5615276a0940 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 10 client.25426 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 10 client.25426 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 20 client.25426 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56152771c670 0x56152771ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 4 client.25426 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}] 2024-09-05T06:43:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}) -- 0x561527602d20 con 0x56152771c670 2024-09-05T06:43:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.965+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56152771c670 0x56152771ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.966+0000 7f4aac9be640 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56152771c670 0x56152771ea90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e0362356843d5e server_cookie=577b5fa56b9cf86 in_seq=0 out_seq=0 2024-09-05T06:43:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:30.966+0000 7f4a90ff9640 10 client.25426 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4a90ff9640 1 -- 172.21.15.86:0/3242871598 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561527602d20 con 0x56152771c670 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4a90ff9640 10 client.25426 handle_command_reply: tid=0 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 client.25426 shutdown 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 msgr2=0x56152769ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x56152769ae30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4003db0 tx=0x7f4aa4004250 comp rx=0 tx=0).stop 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 shutdown_connections 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56152771c670 0x56152771ea90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152769b370 0x56152771a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4a90ff9640 0 client.25426 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615276a0940 0x56152769ae30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 --2- 172.21.15.86:0/3242871598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561527696080 0x56152769a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 >> 172.21.15.86:0/3242871598 conn(0x561527691cf0 msgr2=0x561527690650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.035+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 shutdown_connections 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4ab17fc740 1 -- 172.21.15.86:0/3242871598 wait complete. 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4ab17fc740 20 client.25426 trim_cache size 0 max 16384 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 30, 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:43:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 6, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "17bd525e": { 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 6, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 144.2, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:43:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:31.040 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:31.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 msgr2=0x7f4a940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:31.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 0x7f4a940a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00b92c0 tx=0x7f4aa00095c0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 msgr2=0x7f4aa819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:31.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009dc0 tx=0x7f4aa0032ff0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 shutdown_connections 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4a940a2cf0 0x7f4a940a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aa810e4d0 0x7f4aa819eea0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa8101360 0x7f4aa819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 --2- 172.21.15.86:0/2580143664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aa81009b0 0x7f4aa819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.036+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 >> 172.21.15.86:0/2580143664 conn(0x7f4aa80fc7d0 msgr2=0x7f4aa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.037+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 shutdown_connections 2024-09-05T06:43:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:31.037+0000 7f4aaf423640 1 -- 172.21.15.86:0/2580143664 wait complete. 2024-09-05T06:43:31.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.67 MB/sec warmup 51 sec latency 459.114 ms 2024-09-05T06:43:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:31 smithi086 ceph-mon[30943]: pgmap v71: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-05T06:43:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:31 smithi096 ceph-mon[34468]: pgmap v71: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-05T06:43:31.883 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '17bd525e', quiesce took 3.7 seconds. Will release after: 20.5 2024-09-05T06:43:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:31 smithi115 ceph-mon[34791]: pgmap v71: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-09-05T06:43:32.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.60 MB/sec warmup 52 sec latency 1459.212 ms 2024-09-05T06:43:33.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.54 MB/sec warmup 53 sec latency 2459.257 ms 2024-09-05T06:43:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:33 smithi086 ceph-mon[30943]: pgmap v72: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:33 smithi096 ceph-mon[34468]: pgmap v72: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:33 smithi115 ceph-mon[34791]: pgmap v72: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:34.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.47 MB/sec warmup 54 sec latency 3459.360 ms 2024-09-05T06:43:35.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.41 MB/sec warmup 55 sec latency 4459.440 ms 2024-09-05T06:43:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:35 smithi086 ceph-mon[30943]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:35 smithi096 ceph-mon[34468]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:35 smithi115 ceph-mon[34791]: pgmap v73: 129 pgs: 129 active+clean; 44 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:43:36.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.35 MB/sec warmup 56 sec latency 5459.479 ms 2024-09-05T06:43:37.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.29 MB/sec warmup 57 sec latency 6459.578 ms 2024-09-05T06:43:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:37 smithi086 ceph-mon[30943]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:43:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:37 smithi096 ceph-mon[34468]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:43:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:37 smithi115 ceph-mon[34791]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:43:38.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.23 MB/sec warmup 58 sec latency 7459.689 ms 2024-09-05T06:43:39.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.18 MB/sec warmup 59 sec latency 8459.785 ms 2024-09-05T06:43:39.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:39 smithi086 ceph-mon[30943]: pgmap v75: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-09-05T06:43:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:39 smithi096 ceph-mon[34468]: pgmap v75: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-09-05T06:43:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:39 smithi115 ceph-mon[34791]: pgmap v75: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-09-05T06:43:40.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:43:40.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.12 MB/sec warmup 60 sec latency 9459.850 ms 2024-09-05T06:43:40.581 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:41.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.07 MB/sec warmup 61 sec latency 10459.883 ms 2024-09-05T06:43:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:41 smithi086 ceph-mon[30943]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 598 KiB/s wr, 83 op/s 2024-09-05T06:43:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:41 smithi096 ceph-mon[34468]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 598 KiB/s wr, 83 op/s 2024-09-05T06:43:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:41 smithi115 ceph-mon[34791]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 598 KiB/s wr, 83 op/s 2024-09-05T06:43:42.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 3.02 MB/sec warmup 62 sec latency 11459.963 ms 2024-09-05T06:43:42.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 -- 172.21.15.86:0/889501503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708106a80 msgr2=0x7f6708106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 --2- 172.21.15.86:0/889501503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708106a80 0x7f6708106e60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009930 tx=0x7f66fc02a420 comp rx=0 tx=0).stop 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 -- 172.21.15.86:0/889501503 shutdown_connections 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 --2- 172.21.15.86:0/889501503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708101420 0x7f670810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 --2- 172.21.15.86:0/889501503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708100a80 0x7f6708100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 --2- 172.21.15.86:0/889501503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708106a80 0x7f6708106e60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 -- 172.21.15.86:0/889501503 >> 172.21.15.86:0/889501503 conn(0x7f67080fc7a0 msgr2=0x7f67080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.777+0000 7f670ef4d640 1 -- 172.21.15.86:0/889501503 shutdown_connections 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.778+0000 7f670ef4d640 1 -- 172.21.15.86:0/889501503 wait complete. 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.778+0000 7f670ef4d640 1 Processor -- start 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.778+0000 7f670ef4d640 1 -- start start 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 0x7f67081982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 0x7f670819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111e50 con 0x7f6708101420 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111cd0 con 0x7f6708100a80 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111fd0 con 0x7f6708106a80 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38134/0 (socket says 172.21.15.86:38134) 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 -- 172.21.15.86:0/2649241881 learned_addr learned my addr 172.21.15.86:0/2649241881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 0x7f670819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670cce9640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 0x7f67081982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 -- 172.21.15.86:0/2649241881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 msgr2=0x7f670819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 0x7f670819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 -- 172.21.15.86:0/2649241881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 msgr2=0x7f67081982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 0x7f67081982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 -- 172.21.15.86:0/2649241881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670819d2a0 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 0x7f670819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670cce9640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 0x7f67081982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6707fff640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d950 tx=0x7f66f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6705ffb640 1 -- 172.21.15.86:0/2649241881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4017070 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6705ffb640 1 -- 172.21.15.86:0/2649241881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f4009c30 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670819d590 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.779+0000 7f6705ffb640 1 -- 172.21.15.86:0/2649241881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f400f620 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.780+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a4e70 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.782+0000 7f6705ffb640 1 -- 172.21.15.86:0/2649241881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f66f40109a0 con 0x7f6708101420 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.782+0000 7f6711326740 1 Processor -- start 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.782+0000 7f6711326740 1 -- start start 2024-09-05T06:43:42.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac9542e0 con 0x55e9ac9eedc0 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac8fbce0 con 0x55e9ac9ee4a0 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6711326740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac9502a0 con 0x55e9ac9f9680 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6705ffb640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 0x7f66f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38146/0 (socket says 172.21.15.86:38146) 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 -- 172.21.15.86:0/2098684625 learned_addr learned my addr 172.21.15.86:0/2098684625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 0x7f66f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6707fff640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f6705ffb640 1 -- 172.21.15.86:0/2649241881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f66f40cf530 con 0x7f6708101420 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 -- 172.21.15.86:0/2098684625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 msgr2=0x55e9ac9fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 -- 172.21.15.86:0/2098684625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 msgr2=0x55e9ac9ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 -- 172.21.15.86:0/2098684625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ac8e0440 con 0x55e9ac9eedc0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670cce9640 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f66fc031440 tx=0x7f66fc032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627d2986846f81a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 0x7f66f80a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6700007310 tx=0x7f6700007450 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f66f2ffd640 1 -- 172.21.15.86:0/2098684625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc03f070 con 0x55e9ac9eedc0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.783+0000 7f66f2ffd640 1 -- 172.21.15.86:0/2098684625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66fc031760 con 0x55e9ac9eedc0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f66f2ffd640 1 -- 172.21.15.86:0/2098684625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc035070 con 0x55e9ac9eedc0 2024-09-05T06:43:42.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 -- 172.21.15.86:0/2098684625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 msgr2=0x55e9ac9f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f66fc031440 tx=0x7f66fc032e70 comp rx=0 tx=0).stop 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 -- 172.21.15.86:0/2098684625 shutdown_connections 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f9140 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 --2- 172.21.15.86:0/2098684625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 -- 172.21.15.86:0/2098684625 >> 172.21.15.86:0/2098684625 conn(0x55e9ac9eaa30 msgr2=0x55e9ac9eae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 -- 172.21.15.86:0/2098684625 shutdown_connections 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.784+0000 7f6711326740 1 -- 172.21.15.86:0/2098684625 wait complete. 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 -- start start 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 Processor -- start 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac9fc250 con 0x55e9ac9eedc0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac90f050 con 0x55e9ac9ee4a0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac9fc3d0 con 0x55e9ac9f9680 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6711326740 10 client.0 _ll_register_callbacks cb 0x55e9ac94f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6707fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6707fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38158/0 (socket says 172.21.15.86:38158) 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6707fff640 1 -- 172.21.15.86:0/4229791666 learned_addr learned my addr 172.21.15.86:0/4229791666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f670cce9640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6707fff640 1 -- 172.21.15.86:0/4229791666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 msgr2=0x55e9ac9f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f670d4ea640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.785+0000 7f6707fff640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6707fff640 1 -- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 msgr2=0x55e9ac9f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6707fff640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6707fff640 1 -- 172.21.15.86:0/4229791666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ac6c53c0 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f670cce9640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6707fff640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f66f409b600 tx=0x7f66f4090b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bae859c2ed3cdea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4020040 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f66f4097ce0 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66fc031760 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66f4086070 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 10 client.16017 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e9aca07b90 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66fc031760 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f66f40cf130 con 0x55e9ac9eedc0 2024-09-05T06:43:42.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 10 client.16017 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 20 client.16017 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca08ee0 0x55e9aca0b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.786+0000 7f6711326740 4 client.16017 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.787+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ac961850 con 0x55e9aca08ee0 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.787+0000 7f670cce9640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca08ee0 0x55e9aca0b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.787+0000 7f670cce9640 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca08ee0 0x55e9aca0b3a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5a7431799eb276 server_cookie=5af0b057f3d16551 in_seq=0 out_seq=0 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.787+0000 7f66e7fff640 10 client.16017 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f66e7fff640 1 -- 172.21.15.86:0/4229791666 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9ac961850 con 0x55e9aca08ee0 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f66e7fff640 10 client.16017 handle_command_reply: tid=0 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 client.16017 shutdown 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 msgr2=0x55e9ac9f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f66f409b600 tx=0x7f66f4090b00 comp rx=0 tx=0).stop 2024-09-05T06:43:42.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 shutdown_connections 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca08ee0 0x55e9aca0b3a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9f6290 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f66e7fff640 0 client.16017 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 --2- 172.21.15.86:0/4229791666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9ee4a0 0x55e9ac9f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 >> 172.21.15.86:0/4229791666 conn(0x55e9ac9eaa30 msgr2=0x55e9ac9f0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 shutdown_connections 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 1 -- 172.21.15.86:0/4229791666 wait complete. 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.788+0000 7f6711326740 20 client.16017 trim_cache size 0 max 16384 2024-09-05T06:43:42.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 Processor -- start 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 -- start start 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca10c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9aca111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67081aef80 con 0x55e9ac9f9680 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67081b0b20 con 0x55e9ac9eedc0 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f6711326740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67081af650 con 0x55e9ac9ee4a0 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f670d4ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f670d4ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38174/0 (socket says 172.21.15.86:38174) 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.805+0000 7f670d4ea640 1 -- 172.21.15.86:0/679805533 learned_addr learned my addr 172.21.15.86:0/679805533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670cce9640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca10c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f6707fff640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9aca111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 -- 172.21.15.86:0/679805533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 msgr2=0x55e9aca10c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca10c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 -- 172.21.15.86:0/679805533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 msgr2=0x55e9aca111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9aca111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 -- 172.21.15.86:0/679805533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ac9f6b60 con 0x55e9ac9f9680 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f6707fff640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9aca111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f670d4ea640 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6700007240 tx=0x7f67000072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6279c6b8c877b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f66f27fc640 1 -- 172.21.15.86:0/679805533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700004d20 con 0x55e9ac9f9680 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f66f27fc640 1 -- 172.21.15.86:0/679805533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f670001bb80 con 0x55e9ac9f9680 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f66f27fc640 1 -- 172.21.15.86:0/679805533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000056e0 con 0x55e9ac9f9680 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f6711326740 1 -- 172.21.15.86:0/679805533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 msgr2=0x55e9ac9f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.806+0000 7f6711326740 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6700007240 tx=0x7f67000072a0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- 172.21.15.86:0/679805533 shutdown_connections 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9f3510 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9aca111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- 172.21.15.86:0/679805533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca10c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- 172.21.15.86:0/679805533 >> 172.21.15.86:0/679805533 conn(0x55e9ac9eaa30 msgr2=0x55e9ac9f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- 172.21.15.86:0/679805533 shutdown_connections 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- 172.21.15.86:0/679805533 wait complete. 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- start start 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 Processor -- start 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67081b03f0 con 0x55e9ac9f9680 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67081afd20 con 0x55e9ac9eedc0 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67081acf60 con 0x55e9ac9ee4a0 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f6711326740 10 client.0 _ll_register_callbacks cb 0x7f66f400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f670d4ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.807+0000 7f670d4ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38186/0 (socket says 172.21.15.86:38186) 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 -- 172.21.15.86:0/2130268129 learned_addr learned my addr 172.21.15.86:0/2130268129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 -- 172.21.15.86:0/2130268129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 msgr2=0x55e9aca01d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 -- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 msgr2=0x55e9ac9ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 -- 172.21.15.86:0/2130268129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ac9ffe20 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f670d4ea640 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f670001bab0 tx=0x7f670001bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f07f1f6c91f20f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700004e10 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6700004fb0 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670001bb80 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f670003b050 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f6711326740 10 client.16023 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e9aca73bc0 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f670001bb80 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.808+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6700036030 con 0x55e9ac9f9680 2024-09-05T06:43:42.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 10 client.16023 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 20 client.16023 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca74330 0x55e9aca767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:43:42.836 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 4 client.16023 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9ac95bd60 con 0x55e9aca74330 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f670cce9640 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca74330 0x55e9aca767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f670cce9640 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca74330 0x55e9aca767b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11edc4525a314b64 server_cookie=d5f713e1489b8fa6 in_seq=0 out_seq=0 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f66e7fff640 10 client.16023 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f66e7fff640 1 -- 172.21.15.86:0/2130268129 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9ac95bd60 con 0x55e9aca74330 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f66e7fff640 10 client.16023 handle_command_reply: tid=0 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.809+0000 7f6711326740 1 client.16023 shutdown 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 msgr2=0x55e9ac9ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f670001bab0 tx=0x7f670001bae0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 shutdown_connections 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e9aca74330 0x55e9aca767b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9ac9f9680 0x55e9ac9ff5f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e9ac9eedc0 0x55e9ac9ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 --2- 172.21.15.86:0/2130268129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9ac9ee4a0 0x55e9aca01d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f66e7fff640 0 client.16023 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 >> 172.21.15.86:0/2130268129 conn(0x55e9ac9eaa30 msgr2=0x55e9ac9e9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 shutdown_connections 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 1 -- 172.21.15.86:0/2130268129 wait complete. 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f6711326740 20 client.16023 trim_cache size 0 max 16384 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 msgr2=0x7f66f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 0x7f66f80a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6700007310 tx=0x7f6700007450 comp rx=0 tx=0).stop 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 msgr2=0x7f6708198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:42.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d950 tx=0x7f66f400de10 comp rx=0 tx=0).stop 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 shutdown_connections 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66f80a2cf0 0x7f66f80a51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708106a80 0x7f670819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708101420 0x7f6708198830 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 --2- 172.21.15.86:0/2649241881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6708100a80 0x7f67081982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.810+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 >> 172.21.15.86:0/2649241881 conn(0x7f67080fc7a0 msgr2=0x7f67080fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.811+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 shutdown_connections 2024-09-05T06:43:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:42.811+0000 7f670ef4d640 1 -- 172.21.15.86:0/2649241881 wait complete. 2024-09-05T06:43:43.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.97 MB/sec warmup 63 sec latency 12460.007 ms 2024-09-05T06:43:43.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:43 smithi086 ceph-mon[30943]: pgmap v77: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:43.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:43:43.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:43:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:43 smithi096 ceph-mon[34468]: pgmap v77: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:43 smithi115 ceph-mon[34791]: pgmap v77: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:44.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.93 MB/sec warmup 64 sec latency 13460.035 ms 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.537+0000 7ffb252d8640 1 -- 172.21.15.86:0/452840757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20106f40 msgr2=0x7ffb2010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.537+0000 7ffb252d8640 1 --2- 172.21.15.86:0/452840757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20106f40 0x7ffb2010eac0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009ca0 tx=0x7ffb1802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- 172.21.15.86:0/452840757 shutdown_connections 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- 172.21.15.86:0/452840757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20106f40 0x7ffb2010eac0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- 172.21.15.86:0/452840757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 0x7ffb20106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- 172.21.15.86:0/452840757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20101e10 0x7ffb201021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- 172.21.15.86:0/452840757 >> 172.21.15.86:0/452840757 conn(0x7ffb200fd8d0 msgr2=0x7ffb200ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- 172.21.15.86:0/452840757 shutdown_connections 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- 172.21.15.86:0/452840757 wait complete. 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 Processor -- start 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- start start 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 0x7ffb20198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 0x7ffb2019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20111e50 con 0x7ffb20101e10 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20111cd0 con 0x7ffb201027c0 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.538+0000 7ffb252d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20111fd0 con 0x7ffb20106f40 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38192/0 (socket says 172.21.15.86:38192) 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 -- 172.21.15.86:0/3880053044 learned_addr learned my addr 172.21.15.86:0/3880053044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1f7fe640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 0x7ffb2019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb17fff640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 0x7ffb20198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 -- 172.21.15.86:0/3880053044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 msgr2=0x7ffb2019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 0x7ffb2019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 -- 172.21.15.86:0/3880053044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 msgr2=0x7ffb20198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 0x7ffb20198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 -- 172.21.15.86:0/3880053044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2019d2e0 con 0x7ffb20101e10 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1f7fe640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 0x7ffb2019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1effd640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d770 tx=0x7ffb1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10017070 con 0x7ffb20101e10 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2019d5d0 con 0x7ffb20101e10 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb100054d0 con 0x7ffb20101e10 2024-09-05T06:43:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004cf0 con 0x7ffb20101e10 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.539+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb201a4eb0 con 0x7ffb20101e10 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.540+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb201021f0 con 0x7ffb20101e10 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.541+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb10004e90 con 0x7ffb20101e10 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.542+0000 7ffb1cff9640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 0x7ffb0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.542+0000 7ffb17fff640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 0x7ffb0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.542+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ffb100cf2a0 con 0x7ffb20101e10 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.542+0000 7ffb17fff640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 0x7ffb0c0a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb200fdcf0 tx=0x7ffb0800b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.546+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb1009bae0 con 0x7ffb20101e10 2024-09-05T06:43:44.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.745+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb2010a1e0 con 0x7ffb20101e10 2024-09-05T06:43:44.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.745+0000 7ffb1cff9640 1 -- 172.21.15.86:0/3880053044 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7ffb1009b3e0 con 0x7ffb20101e10 2024-09-05T06:43:44.745 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:43:44.746 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:43:44.746 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:43:44.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 msgr2=0x7ffb0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:44.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 0x7ffb0c0a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb200fdcf0 tx=0x7ffb0800b390 comp rx=0 tx=0).stop 2024-09-05T06:43:44.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 msgr2=0x7ffb20198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:44.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d770 tx=0x7ffb1000dc30 comp rx=0 tx=0).stop 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 shutdown_connections 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb0c0a2cf0 0x7ffb0c0a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb20106f40 0x7ffb2019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb201027c0 0x7ffb20198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 --2- 172.21.15.86:0/3880053044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb20101e10 0x7ffb20198330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 >> 172.21.15.86:0/3880053044 conn(0x7ffb200fd8d0 msgr2=0x7ffb200fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 shutdown_connections 2024-09-05T06:43:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:44.747+0000 7ffb252d8640 1 -- 172.21.15.86:0/3880053044 wait complete. 2024-09-05T06:43:45.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:44 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3880053044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:45.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:44 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3880053044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:45.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:44 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3880053044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:45.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:43:45.506 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.88 MB/sec warmup 65 sec latency 14460.064 ms 2024-09-05T06:43:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:45 smithi096 ceph-mon[34468]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:45 smithi086 ceph-mon[30943]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:46.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:45 smithi115 ceph-mon[34791]: pgmap v78: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 -- 172.21.15.86:0/2219657807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4100a80 msgr2=0x7f2cd4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/2219657807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4100a80 0x7f2cd4100ee0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009930 tx=0x7f2cd002a420 comp rx=0 tx=0).stop 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 -- 172.21.15.86:0/2219657807 shutdown_connections 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/2219657807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/2219657807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4100a80 0x7f2cd4100ee0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/2219657807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4106a80 0x7f2cd4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 -- 172.21.15.86:0/2219657807 >> 172.21.15.86:0/2219657807 conn(0x7f2cd40fc7a0 msgr2=0x7f2cd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 -- 172.21.15.86:0/2219657807 shutdown_connections 2024-09-05T06:43:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.197+0000 7f2cdc33d640 1 -- 172.21.15.86:0/2219657807 wait complete. 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 Processor -- start 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 -- start start 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 0x7f2cd41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4110d50 con 0x7f2cd4106a80 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4110bd0 con 0x7f2cd4100a80 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cdc33d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4110ed0 con 0x7f2cd4101420 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38214/0 (socket says 172.21.15.86:38214) 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 -- 172.21.15.86:0/4052997315 learned_addr learned my addr 172.21.15.86:0/4052997315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 0x7f2cd41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 -- 172.21.15.86:0/4052997315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 msgr2=0x7f2cd4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd4199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 -- 172.21.15.86:0/4052997315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 msgr2=0x7f2cd41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 0x7f2cd41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 -- 172.21.15.86:0/4052997315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd41a47a0 con 0x7f2cd4106a80 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 0x7f2cd41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.198+0000 7f2cda8da640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b970 tx=0x7f2ccc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.199+0000 7f2cc77fe640 1 -- 172.21.15.86:0/4052997315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc019070 con 0x7f2cd4106a80 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.199+0000 7f2cc77fe640 1 -- 172.21.15.86:0/4052997315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ccc014440 con 0x7f2cd4106a80 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.199+0000 7f2cc77fe640 1 -- 172.21.15.86:0/4052997315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0075d0 con 0x7f2cd4106a80 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.199+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd41a4970 con 0x7f2cd4106a80 2024-09-05T06:43:46.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.199+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd41a4eb0 con 0x7f2cd4106a80 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.201+0000 7f2cc77fe640 1 -- 172.21.15.86:0/4052997315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2ccc01c070 con 0x7f2cd4106a80 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.201+0000 7f2cc77fe640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 0x7f2cc00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cc77fe640 1 -- 172.21.15.86:0/4052997315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2ccc0cecd0 con 0x7f2cd4106a80 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 0x7f2cc00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 Processor -- start 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 0x7f2cc00a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800abe0 tx=0x7f2cc800a5c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 -- start start 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc103745d0 0x55bc10374a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10381760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc102da2f0 con 0x55bc10375b40 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc10281cf0 con 0x55bc103745d0 2024-09-05T06:43:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cde716740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc102d62b0 con 0x55bc1037f370 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38228/0 (socket says 172.21.15.86:38228) 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.202+0000 7f2cda0d9640 1 -- 172.21.15.86:0/2888629286 learned_addr learned my addr 172.21.15.86:0/2888629286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda8da640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10381760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 -- 172.21.15.86:0/2888629286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 msgr2=0x55bc10381760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10381760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 -- 172.21.15.86:0/2888629286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc103745d0 msgr2=0x55bc10374a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc103745d0 0x55bc10374a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc103745d0 0x55bc10374a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 -- 172.21.15.86:0/2888629286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc102663a0 con 0x55bc10375b40 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda8da640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10381760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800ef10 tx=0x7f2cc8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbb86e80e416ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cc4dea640 1 -- 172.21.15.86:0/2888629286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8021070 con 0x55bc10375b40 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cc4dea640 1 -- 172.21.15.86:0/2888629286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cc8010e40 con 0x55bc10375b40 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.203+0000 7f2cc4dea640 1 -- 172.21.15.86:0/2888629286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800fc10 con 0x55bc10375b40 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- 172.21.15.86:0/2888629286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 msgr2=0x55bc10374090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800ef10 tx=0x7f2cc8011000 comp rx=0 tx=0).stop 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- 172.21.15.86:0/2888629286 shutdown_connections 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10381760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc103745d0 0x55bc10374a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- 172.21.15.86:0/2888629286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10374090 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- 172.21.15.86:0/2888629286 >> 172.21.15.86:0/2888629286 conn(0x55bc1033d310 msgr2=0x55bc10370c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- 172.21.15.86:0/2888629286 shutdown_connections 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- 172.21.15.86:0/2888629286 wait complete. 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 -- start start 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 Processor -- start 2024-09-05T06:43:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.204+0000 7f2cde716740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc10375b40 0x55bc1037bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc1037c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc10382070 con 0x55bc103745d0 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc10295060 con 0x55bc1037f370 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc103821f0 con 0x55bc10375b40 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 10 client.0 _ll_register_callbacks cb 0x55bc0ffe88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38244/0 (socket says 172.21.15.86:38244) 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 -- 172.21.15.86:0/10009934 learned_addr learned my addr 172.21.15.86:0/10009934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc10375b40 0x55bc1037bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda8da640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc1037c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 -- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc10375b40 msgr2=0x55bc1037bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc10375b40 0x55bc1037bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 -- 172.21.15.86:0/10009934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 msgr2=0x55bc1037c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc1037c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 -- 172.21.15.86:0/10009934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1004b3c0 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800a360 tx=0x7f2cc800f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c87a992ab16ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8035060 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc102e7860 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2cc801d870 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cc803c050 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 10 client.16041 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.205+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc1038dac0 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bc102e7860 con 0x55bc103745d0 2024-09-05T06:43:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2cc80268e0 con 0x55bc103745d0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 10 client.16041 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 20 client.16041 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc1038eec0 0x55bc103912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 4 client.16041 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc102e1d70 con 0x55bc1038eec0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.206+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc1038eec0 0x55bc103912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.207+0000 7f2cbdffb640 10 client.16041 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.207+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc1038eec0 0x55bc103912b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f25f2e5ac45b327 server_cookie=755acfa5648376f6 in_seq=0 out_seq=0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cbdffb640 1 -- 172.21.15.86:0/10009934 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc102e1d70 con 0x55bc1038eec0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cbdffb640 10 client.16041 handle_command_reply: tid=0 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 client.16041 shutdown 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 msgr2=0x55bc1037ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800a360 tx=0x7f2cc800f5f0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 shutdown_connections 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc1038eec0 0x55bc103912b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc1037c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc10375b40 0x55bc1037bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cbdffb640 0 client.16041 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 --2- 172.21.15.86:0/10009934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc1037ed20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 >> 172.21.15.86:0/10009934 conn(0x55bc1033d310 msgr2=0x55bc10376930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 shutdown_connections 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 1 -- 172.21.15.86:0/10009934 wait complete. 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.208+0000 7f2cde716740 20 client.16041 trim_cache size 0 max 16384 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 Processor -- start 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 -- start start 2024-09-05T06:43:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc103745d0 0x55bc10377120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc10396a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41af100 con 0x55bc10375b40 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b0ca0 con 0x55bc1037f370 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cde716740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41af7d0 con 0x55bc103745d0 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38246/0 (socket says 172.21.15.86:38246) 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 -- 172.21.15.86:0/3922908258 learned_addr learned my addr 172.21.15.86:0/3922908258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc103745d0 0x55bc10377120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 -- 172.21.15.86:0/3922908258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc103745d0 msgr2=0x55bc10377120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc103745d0 0x55bc10377120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cda8da640 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc10396a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 -- 172.21.15.86:0/3922908258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 msgr2=0x55bc10396a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc10396a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 -- 172.21.15.86:0/3922908258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc10377ba0 con 0x55bc10375b40 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.221+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00096c0 tx=0x7f2cd00040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387cb233a33cdc46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cbffff640 1 -- 172.21.15.86:0/3922908258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd004a070 con 0x55bc10375b40 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cbffff640 1 -- 172.21.15.86:0/3922908258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd00054f0 con 0x55bc10375b40 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cbffff640 1 -- 172.21.15.86:0/3922908258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0035040 con 0x55bc10375b40 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- 172.21.15.86:0/3922908258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 msgr2=0x55bc10377660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00096c0 tx=0x7f2cd00040f0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- 172.21.15.86:0/3922908258 shutdown_connections 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc1037f370 0x55bc10396a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc10375b40 0x55bc10377660 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 --2- 172.21.15.86:0/3922908258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc103745d0 0x55bc10377120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- 172.21.15.86:0/3922908258 >> 172.21.15.86:0/3922908258 conn(0x55bc1033d310 msgr2=0x55bc1036fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- 172.21.15.86:0/3922908258 shutdown_connections 2024-09-05T06:43:46.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- 172.21.15.86:0/3922908258 wait complete. 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.222+0000 7f2cde716740 1 -- start start 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 Processor -- start 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 0x55bc10383a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10383ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b0570 con 0x55bc103745d0 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41afea0 con 0x55bc10375b40 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41ad0e0 con 0x55bc1037f370 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 10 client.0 _ll_register_callbacks cb 0x7f2cc800ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38256/0 (socket says 172.21.15.86:38256) 2024-09-05T06:43:46.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 -- 172.21.15.86:0/4162702238 learned_addr learned my addr 172.21.15.86:0/4162702238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 0x55bc10383a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 -- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 msgr2=0x55bc10383ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10383ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 -- 172.21.15.86:0/4162702238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 msgr2=0x55bc10383a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 0x55bc10383a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 -- 172.21.15.86:0/4162702238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc103f8ce0 con 0x55bc103745d0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 0x55bc10383a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cda0d9640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8026cb0 tx=0x7f2cc8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e470cf3442a73d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8014050 con 0x55bc103745d0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2cc801d870 con 0x55bc103745d0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.223+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd00054f0 con 0x55bc103745d0 2024-09-05T06:43:46.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cc8035040 con 0x55bc103745d0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 10 client.16047 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc103f9fe0 con 0x55bc103745d0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd00054f0 con 0x55bc103745d0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2cc80268a0 con 0x55bc103745d0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 10 client.16047 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 10 client.16047 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 20 client.16047 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc103fa690 0x55bc103fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 4 client.16047 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc1025e170 con 0x55bc103fa690 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.224+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc103fa690 0x55bc103fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cd98d8640 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc103fa690 0x55bc103fcac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fc10ef85878d5bd server_cookie=a03c5e9f35e16883 in_seq=0 out_seq=0 2024-09-05T06:43:46.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cbdffb640 10 client.16047 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cbdffb640 1 -- 172.21.15.86:0/4162702238 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc1025e170 con 0x55bc103fa690 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cbdffb640 10 client.16047 handle_command_reply: tid=0 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 client.16047 shutdown 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 msgr2=0x55bc10383530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8026cb0 tx=0x7f2cc8005be0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 shutdown_connections 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55bc103fa690 0x55bc103fcac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cbdffb640 0 client.16047 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc1037f370 0x55bc10383ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc10375b40 0x55bc10383a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 --2- 172.21.15.86:0/4162702238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc103745d0 0x55bc10383530 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 >> 172.21.15.86:0/4162702238 conn(0x55bc1033d310 msgr2=0x55bc1037d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 shutdown_connections 2024-09-05T06:43:46.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 1 -- 172.21.15.86:0/4162702238 wait complete. 2024-09-05T06:43:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.225+0000 7f2cde716740 20 client.16047 trim_cache size 0 max 16384 2024-09-05T06:43:46.242 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:46.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 msgr2=0x7f2cc00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 0x7f2cc00a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800abe0 tx=0x7f2cc800a5c0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 msgr2=0x7f2cd41a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b970 tx=0x7f2ccc00be30 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 shutdown_connections 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2cc00a2d70 0x7f2cc00a5230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd4106a80 0x7f2cd41a4260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cd4101420 0x7f2cd4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 --2- 172.21.15.86:0/4052997315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4100a80 0x7f2cd41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 >> 172.21.15.86:0/4052997315 conn(0x7f2cd40fc7a0 msgr2=0x7f2cd40fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 shutdown_connections 2024-09-05T06:43:46.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:46.226+0000 7f2cdc33d640 1 -- 172.21.15.86:0/4052997315 wait complete. 2024-09-05T06:43:46.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.84 MB/sec warmup 66 sec latency 15460.094 ms 2024-09-05T06:43:46.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:43:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:47.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.80 MB/sec warmup 67 sec latency 16460.139 ms 2024-09-05T06:43:47.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:47 smithi086 ceph-mon[30943]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 -- 172.21.15.86:0/2508709704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 msgr2=0x7faf68100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 --2- 172.21.15.86:0/2508709704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf68100ee0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7faf58009930 tx=0x7faf5802a420 comp rx=0 tx=0).stop 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 -- 172.21.15.86:0/2508709704 shutdown_connections 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 --2- 172.21.15.86:0/2508709704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 --2- 172.21.15.86:0/2508709704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf68100ee0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 --2- 172.21.15.86:0/2508709704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf68106a80 0x7faf68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 -- 172.21.15.86:0/2508709704 >> 172.21.15.86:0/2508709704 conn(0x7faf680fc7a0 msgr2=0x7faf680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 -- 172.21.15.86:0/2508709704 shutdown_connections 2024-09-05T06:43:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.766+0000 7faf70617640 1 -- 172.21.15.86:0/2508709704 wait complete. 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 Processor -- start 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 -- start start 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf68106a80 0x7faf6810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf68111e50 con 0x7faf68100a80 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf68111cd0 con 0x7faf68101420 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf70617640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf68111fd0 con 0x7faf68106a80 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf6e3b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf6e3b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38276/0 (socket says 172.21.15.86:38276) 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.767+0000 7faf6e3b3640 1 -- 172.21.15.86:0/1446364162 learned_addr learned my addr 172.21.15.86:0/1446364162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 -- 172.21.15.86:0/1446364162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf68106a80 msgr2=0x7faf6810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf68106a80 0x7faf6810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 -- 172.21.15.86:0/1446364162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 msgr2=0x7faf6810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 -- 172.21.15.86:0/1446364162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf681a6830 con 0x7faf68100a80 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c920 tx=0x7faf6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf5f7fe640 1 -- 172.21.15.86:0/1446364162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64007c70 con 0x7faf68100a80 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf5f7fe640 1 -- 172.21.15.86:0/1446364162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf64007e10 con 0x7faf68100a80 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf681a6b20 con 0x7faf68100a80 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf5f7fe640 1 -- 172.21.15.86:0/1446364162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64018820 con 0x7faf68100a80 2024-09-05T06:43:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.768+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf681a70b0 con 0x7faf68100a80 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf5f7fe640 1 -- 172.21.15.86:0/1446364162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faf640189c0 con 0x7faf68100a80 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 Processor -- start 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 -- start start 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d949a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d956760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a64d8af2f0 con 0x55a64d94ab40 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a64d856cf0 con 0x55a64d954370 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf729f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a64d8ab2b0 con 0x55a64d9495d0 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.771+0000 7faf6ebb4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38292/0 (socket says 172.21.15.86:38292) 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 -- 172.21.15.86:0/3721486540 learned_addr learned my addr 172.21.15.86:0/3721486540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:47.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d949a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf5f7fe640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 0x7faf540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d956760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 -- 172.21.15.86:0/3721486540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 msgr2=0x55a64d949a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d949a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 -- 172.21.15.86:0/3721486540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 msgr2=0x55a64d956760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d956760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 -- 172.21.15.86:0/3721486540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64d83b3a0 con 0x55a64d94ab40 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf5f7fe640 1 -- 172.21.15.86:0/1446364162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf640cf9a0 con 0x7faf68100a80 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 0x7faf540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ec0 tx=0x7faf6000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebbddeec4e493d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf5cff9640 1 -- 172.21.15.86:0/3721486540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60020640 con 0x55a64d94ab40 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf5cff9640 1 -- 172.21.15.86:0/3721486540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf60020c90 con 0x55a64d94ab40 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.772+0000 7faf5cff9640 1 -- 172.21.15.86:0/3721486540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600149c0 con 0x55a64d94ab40 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 -- 172.21.15.86:0/3721486540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 msgr2=0x55a64d949090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ec0 tx=0x7faf6000b420 comp rx=0 tx=0).stop 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 0x7faf540a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf6001d000 tx=0x7faf6001e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 -- 172.21.15.86:0/3721486540 shutdown_connections 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d956760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d949a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 --2- 172.21.15.86:0/3721486540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d949090 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 -- 172.21.15.86:0/3721486540 >> 172.21.15.86:0/3721486540 conn(0x55a64d912310 msgr2=0x55a64d945c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 -- 172.21.15.86:0/3721486540 shutdown_connections 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.773+0000 7faf729f0740 1 -- 172.21.15.86:0/3721486540 wait complete. 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 -- start start 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 Processor -- start 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d953d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d9515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a64d957070 con 0x55a64d94ab40 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a64d86a060 con 0x55a64d954370 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a64d9571f0 con 0x55a64d9495d0 2024-09-05T06:43:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf729f0740 10 client.0 _ll_register_callbacks cb 0x55a64d5bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38298/0 (socket says 172.21.15.86:38298) 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 -- 172.21.15.86:0/2316615431 learned_addr learned my addr 172.21.15.86:0/2316615431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d9515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d953d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 -- 172.21.15.86:0/2316615431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 msgr2=0x55a64d953d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d953d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 -- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 msgr2=0x55a64d9515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d9515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.774+0000 7faf6dbb2640 1 -- 172.21.15.86:0/2316615431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64d6203c0 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faf58009900 tx=0x7faf580317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e5a4c9c5b81519 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5803f070 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf58004450 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf60020c90 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf5804c050 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 10 client.16059 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a64d962af0 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf60020c90 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faf58004bd0 con 0x55a64d94ab40 2024-09-05T06:43:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 10 client.16059 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 10 client.16059 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 20 client.16059 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d963fb0 0x55a64d966410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 4 client.16059 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.775+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a64d8bc860 con 0x55a64d963fb0 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.776+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d963fb0 0x55a64d966410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.776+0000 7faf51ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.776+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d963fb0 0x55a64d966410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c5153bccd5c4a55 server_cookie=492096c3d092f09e in_seq=0 out_seq=0 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf51ffb640 1 -- 172.21.15.86:0/2316615431 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a64d8bc860 con 0x55a64d963fb0 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf51ffb640 10 client.16059 handle_command_reply: tid=0 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 client.16059 shutdown 2024-09-05T06:43:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 msgr2=0x55a64d951000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faf58009900 tx=0x7faf580317e0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 shutdown_connections 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d963fb0 0x55a64d966410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d954370 0x55a64d9515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d951000 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf51ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 --2- 172.21.15.86:0/2316615431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d9495d0 0x55a64d953d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 >> 172.21.15.86:0/2316615431 conn(0x55a64d912310 msgr2=0x55a64d94b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 shutdown_connections 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 1 -- 172.21.15.86:0/2316615431 wait complete. 2024-09-05T06:43:47.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.777+0000 7faf729f0740 20 client.16059 trim_cache size 0 max 16384 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 Processor -- start 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 -- start start 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d960310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d94c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf681b12b0 con 0x55a64d94ab40 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf681b2e50 con 0x55a64d9495d0 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf729f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf681b1980 con 0x55a64d954370 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.790+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38302/0 (socket says 172.21.15.86:38302) 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 -- 172.21.15.86:0/1832884845 learned_addr learned my addr 172.21.15.86:0/1832884845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d94c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d960310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 -- 172.21.15.86:0/1832884845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 msgr2=0x55a64d94c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d94c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 -- 172.21.15.86:0/1832884845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 msgr2=0x55a64d960310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d960310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 -- 172.21.15.86:0/1832884845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64d94a760 con 0x55a64d94ab40 2024-09-05T06:43:47.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faf5802a940 tx=0x7faf5802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded42ad2fb41cf0c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf53fff640 1 -- 172.21.15.86:0/1832884845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58055070 con 0x55a64d94ab40 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf53fff640 1 -- 172.21.15.86:0/1832884845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58005480 con 0x55a64d94ab40 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.791+0000 7faf53fff640 1 -- 172.21.15.86:0/1832884845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5803f070 con 0x55a64d94ab40 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- 172.21.15.86:0/1832884845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 msgr2=0x55a64d94c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faf5802a940 tx=0x7faf5802a420 comp rx=0 tx=0).stop 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- 172.21.15.86:0/1832884845 shutdown_connections 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d94c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d94c120 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- 172.21.15.86:0/1832884845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d960310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- 172.21.15.86:0/1832884845 >> 172.21.15.86:0/1832884845 conn(0x55a64d912310 msgr2=0x55a64d9500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- 172.21.15.86:0/1832884845 shutdown_connections 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- 172.21.15.86:0/1832884845 wait complete. 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- start start 2024-09-05T06:43:47.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 Processor -- start 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d95cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d958380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf681b2720 con 0x55a64d94ab40 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf681b2050 con 0x55a64d9495d0 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf681af290 con 0x55a64d954370 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.792+0000 7faf729f0740 10 client.0 _ll_register_callbacks cb 0x7faf58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38312/0 (socket says 172.21.15.86:38312) 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 -- 172.21.15.86:0/588719815 learned_addr learned my addr 172.21.15.86:0/588719815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d958380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d95cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 -- 172.21.15.86:0/588719815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 msgr2=0x55a64d958380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d958380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 -- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 msgr2=0x55a64d95cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d95cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 -- 172.21.15.86:0/588719815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64d958b60 con 0x55a64d94ab40 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d95cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6ebb4640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d958380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf6dbb2640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faf580036b0 tx=0x7faf58031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9417c0ea769826bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5805e060 con 0x55a64d94ab40 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7faf58004460 con 0x55a64d94ab40 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf58005480 con 0x55a64d94ab40 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf58065050 con 0x55a64d94ab40 2024-09-05T06:43:47.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf729f0740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.793+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a64d9cec00 con 0x55a64d94ab40 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58005480 con 0x55a64d94ab40 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faf5803b430 con 0x55a64d94ab40 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 10 client.16065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 20 client.16065 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d9cf2b0 0x55a64d9d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 4 client.16065 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a64d8b6d70 con 0x55a64d9cf2b0 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d9cf2b0 0x55a64d9d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf6e3b3640 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d9cf2b0 0x55a64d9d16a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e70f6eebb03c51b8 server_cookie=b05c5fcfbfbde438 in_seq=0 out_seq=0 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf51ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:43:47.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf51ffb640 1 -- 172.21.15.86:0/588719815 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a64d8b6d70 con 0x55a64d9cf2b0 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.794+0000 7faf51ffb640 10 client.16065 handle_command_reply: tid=0 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 client.16065 shutdown 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 msgr2=0x55a64d957e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faf580036b0 tx=0x7faf58031400 comp rx=0 tx=0).stop 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 shutdown_connections 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a64d9cf2b0 0x55a64d9d16a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a64d954370 0x55a64d958380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a64d94ab40 0x55a64d957e40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 --2- 172.21.15.86:0/588719815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a64d9495d0 0x55a64d95cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 >> 172.21.15.86:0/588719815 conn(0x55a64d912310 msgr2=0x55a64d9444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf51ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 shutdown_connections 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 1 -- 172.21.15.86:0/588719815 wait complete. 2024-09-05T06:43:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf729f0740 20 client.16065 trim_cache size 0 max 16384 2024-09-05T06:43:47.810 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:47.811 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:47 smithi096 ceph-mon[34468]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-05T06:43:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 msgr2=0x7faf540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 0x7faf540a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf6001d000 tx=0x7faf6001e000 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 msgr2=0x7faf6810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c920 tx=0x7faf6400cde0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 shutdown_connections 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf540a2d70 0x7faf540a5230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf68106a80 0x7faf6810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf68101420 0x7faf6810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 --2- 172.21.15.86:0/1446364162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf68100a80 0x7faf6810e3e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.795+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 >> 172.21.15.86:0/1446364162 conn(0x7faf680fc7a0 msgr2=0x7faf680fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.796+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 shutdown_connections 2024-09-05T06:43:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:47.796+0000 7faf70617640 1 -- 172.21.15.86:0/1446364162 wait complete. 2024-09-05T06:43:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:47 smithi115 ceph-mon[34791]: pgmap v79: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-05T06:43:48.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.76 MB/sec warmup 68 sec latency 17460.184 ms 2024-09-05T06:43:48.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:43:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:49.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 -- 172.21.15.86:0/2283638644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103660 msgr2=0x7f19c010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 --2- 172.21.15.86:0/2283638644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103660 0x7f19c010bb50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009900 tx=0x7f19a802a480 comp rx=0 tx=0).stop 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 -- 172.21.15.86:0/2283638644 shutdown_connections 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 --2- 172.21.15.86:0/2283638644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c010c170 0x7f19c010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 --2- 172.21.15.86:0/2283638644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103660 0x7f19c010bb50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 --2- 172.21.15.86:0/2283638644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c0102d40 0x7f19c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 -- 172.21.15.86:0/2283638644 >> 172.21.15.86:0/2283638644 conn(0x7f19c00fc7a0 msgr2=0x7f19c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 -- 172.21.15.86:0/2283638644 shutdown_connections 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.350+0000 7f19c502a640 1 -- 172.21.15.86:0/2283638644 wait complete. 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.351+0000 7f19c502a640 1 Processor -- start 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.351+0000 7f19c502a640 1 -- start start 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0103660 0x7f19c0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 0x7f19c019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0111ee0 con 0x7f19c0102d40 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0111d60 con 0x7f19c0103660 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0112060 con 0x7f19c010c170 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38336/0 (socket says 172.21.15.86:38336) 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 -- 172.21.15.86:0/1810364908 learned_addr learned my addr 172.21.15.86:0/1810364908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 0x7f19c019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19b7fff640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0103660 0x7f19c0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 -- 172.21.15.86:0/1810364908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 msgr2=0x7f19c019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 0x7f19c019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 -- 172.21.15.86:0/1810364908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0103660 msgr2=0x7f19c0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0103660 0x7f19c0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 -- 172.21.15.86:0/1810364908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c019d490 con 0x7f19c0102d40 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 0x7f19c019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bed9d640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f19b000b4a0 tx=0x7f19b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bcd99640 1 -- 172.21.15.86:0/1810364908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0014070 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c019d780 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bcd99640 1 -- 172.21.15.86:0/1810364908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b0010ac0 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19bcd99640 1 -- 172.21.15.86:0/1810364908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017c30 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.352+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c01a5060 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19bcd99640 1 -- 172.21.15.86:0/1810364908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19b0017070 con 0x7f19c0102d40 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 Processor -- start 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 -- start start 2024-09-05T06:43:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3276a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3283760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626d31dc2f0 con 0x5626d3281370 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d3183cf0 con 0x5626d32765d0 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.355+0000 7f19c7403740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626d31d82b0 con 0x5626d3277b40 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3283760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49364/0 (socket says 172.21.15.86:49364) 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 -- 172.21.15.86:0/1360883589 learned_addr learned my addr 172.21.15.86:0/1360883589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bcd99640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 0x7f19ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bed9d640 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3276a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 -- 172.21.15.86:0/1360883589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 msgr2=0x5626d3276a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3276a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 -- 172.21.15.86:0/1360883589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 msgr2=0x5626d3283760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3283760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 -- 172.21.15.86:0/1360883589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d31683a0 con 0x5626d3277b40 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bcd99640 1 -- 172.21.15.86:0/1810364908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f19b00ceea0 con 0x7f19c0102d40 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 0x7f19ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c550 tx=0x7f19b800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276721c22a16b3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19b57fa640 1 -- 172.21.15.86:0/1360883589 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8018640 con 0x5626d3277b40 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19bf59e640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 0x7f19ac0a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19b801f000 tx=0x7f19b8018000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19b57fa640 1 -- 172.21.15.86:0/1360883589 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b8007e70 con 0x5626d3277b40 2024-09-05T06:43:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.356+0000 7f19b57fa640 1 -- 172.21.15.86:0/1360883589 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80175a0 con 0x5626d3277b40 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 -- 172.21.15.86:0/1360883589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 msgr2=0x5626d3276090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c550 tx=0x7f19b800ca10 comp rx=0 tx=0).stop 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 -- 172.21.15.86:0/1360883589 shutdown_connections 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3283760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3276a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 --2- 172.21.15.86:0/1360883589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3276090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 -- 172.21.15.86:0/1360883589 >> 172.21.15.86:0/1360883589 conn(0x5626d323f310 msgr2=0x5626d3272c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 -- 172.21.15.86:0/1360883589 shutdown_connections 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.357+0000 7f19c7403740 1 -- 172.21.15.86:0/1360883589 wait complete. 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 -- start start 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 Processor -- start 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3280d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d327e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626d3284070 con 0x5626d3281370 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d3197060 con 0x5626d32765d0 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626d32841f0 con 0x5626d3277b40 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19c7403740 10 client.0 _ll_register_callbacks cb 0x5626d2eea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38358/0 (socket says 172.21.15.86:38358) 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bf59e640 1 -- 172.21.15.86:0/2534513803 learned_addr learned my addr 172.21.15.86:0/2534513803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3280d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d327e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 -- 172.21.15.86:0/2534513803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 msgr2=0x5626d327e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d327e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 -- 172.21.15.86:0/2534513803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 msgr2=0x5626d327e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bed9d640 1 -- 172.21.15.86:0/2534513803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d2f4d3c0 con 0x5626d32765d0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.358+0000 7f19bf59e640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3280d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19b00cc320 tx=0x7f19b009b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290b3dbc9dd943b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001e030 con 0x5626d32765d0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b8007e70 con 0x5626d32765d0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f19b00cc490 con 0x5626d32765d0 2024-09-05T06:43:49.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19b00a3d60 con 0x5626d32765d0 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19c7403740 10 client.25499 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626d328fbf0 con 0x5626d32765d0 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b8007e70 con 0x5626d32765d0 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.359+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f19b00a38d0 con 0x5626d32765d0 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 10 client.25499 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 10 client.25499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 20 client.25499 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d3291000 0x5626d32933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 4 client.25499 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626d31e9860 con 0x5626d3291000 2024-09-05T06:43:49.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d3291000 0x5626d32933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19a67fc640 10 client.25499 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.360+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d3291000 0x5626d32933f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=372dcdbfe3877566 server_cookie=d156755c410fa975 in_seq=0 out_seq=0 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19a67fc640 1 -- 172.21.15.86:0/2534513803 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626d31e9860 con 0x5626d3291000 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19a67fc640 10 client.25499 handle_command_reply: tid=0 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 client.25499 shutdown 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 msgr2=0x5626d3280d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3280d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19b00cc320 tx=0x7f19b009b7f0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 shutdown_connections 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d3291000 0x5626d32933f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d327e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d327e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19a67fc640 0 client.25499 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:43:49.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 --2- 172.21.15.86:0/2534513803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3280d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 >> 172.21.15.86:0/2534513803 conn(0x5626d323f310 msgr2=0x5626d3278960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 shutdown_connections 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 1 -- 172.21.15.86:0/2534513803 wait complete. 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.361+0000 7f19c7403740 20 client.25499 trim_cache size 0 max 16384 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 Processor -- start 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 -- start start 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3279120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3279660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01af100 con 0x5626d3281370 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b0ca0 con 0x5626d32765d0 2024-09-05T06:43:49.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19c7403740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01af7d0 con 0x5626d3277b40 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38368/0 (socket says 172.21.15.86:38368) 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 -- 172.21.15.86:0/3325549300 learned_addr learned my addr 172.21.15.86:0/3325549300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bed9d640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3279120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19b7fff640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3279660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 -- 172.21.15.86:0/3325549300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 msgr2=0x5626d3279660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3279660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.375+0000 7f19bf59e640 1 -- 172.21.15.86:0/3325549300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 msgr2=0x5626d3279120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19bf59e640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3279120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19bf59e640 1 -- 172.21.15.86:0/3325549300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d327cff0 con 0x5626d3281370 2024-09-05T06:43:49.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19bed9d640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3279120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19bf59e640 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c550 tx=0x7f19b800ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280dca0da261fcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19b4ff9640 1 -- 172.21.15.86:0/3325549300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8007e70 con 0x5626d3281370 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19b4ff9640 1 -- 172.21.15.86:0/3325549300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b8009a70 con 0x5626d3281370 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.376+0000 7f19b4ff9640 1 -- 172.21.15.86:0/3325549300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800a730 con 0x5626d3281370 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- 172.21.15.86:0/3325549300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 msgr2=0x5626d3298ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c550 tx=0x7f19b800ae80 comp rx=0 tx=0).stop 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- 172.21.15.86:0/3325549300 shutdown_connections 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d3281370 0x5626d3298ba0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d3279660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- 172.21.15.86:0/3325549300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d32765d0 0x5626d3279120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- 172.21.15.86:0/3325549300 >> 172.21.15.86:0/3325549300 conn(0x5626d323f310 msgr2=0x5626d3271d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- 172.21.15.86:0/3325549300 shutdown_connections 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- 172.21.15.86:0/3325549300 wait complete. 2024-09-05T06:43:49.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- start start 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 Processor -- start 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d32856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 0x5626d3285c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01b0570 con 0x5626d32765d0 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01afea0 con 0x5626d3281370 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01ad0e0 con 0x5626d3277b40 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19c7403740 10 client.0 _ll_register_callbacks cb 0x7f19b00174e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.377+0000 7f19bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38376/0 (socket says 172.21.15.86:38376) 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 -- 172.21.15.86:0/2667968705 learned_addr learned my addr 172.21.15.86:0/2667968705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d32856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bf59e640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 0x5626d3285c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 -- 172.21.15.86:0/2667968705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 msgr2=0x5626d32856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d32856d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 -- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 msgr2=0x5626d3285c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 0x5626d3285c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 -- 172.21.15.86:0/2667968705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d3281100 con 0x5626d32765d0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bf59e640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 0x5626d3285c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d32856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19bed9d640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f19b00993a0 tx=0x7f19b0097390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eda65eaad6c3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001e070 con 0x5626d32765d0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f19b00cc490 con 0x5626d32765d0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b8009a70 con 0x5626d32765d0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19b0010c70 con 0x5626d32765d0 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19c7403740 10 client.16083 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:49.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626d32fbc30 con 0x5626d32765d0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.378+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b8009a70 con 0x5626d32765d0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f19b00b5020 con 0x5626d32765d0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 10 client.16083 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 10 client.16083 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 20 client.16083 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d32fc2f0 0x5626d32fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 4 client.16083 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626d31e3d70 con 0x5626d32fc2f0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d32fc2f0 0x5626d32fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19a67fc640 10 client.16083 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19b7fff640 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d32fc2f0 0x5626d32fe6e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eab99ce254f2c23b server_cookie=a9010b91633b871b in_seq=0 out_seq=0 2024-09-05T06:43:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19a67fc640 1 -- 172.21.15.86:0/2667968705 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626d31e3d70 con 0x5626d32fc2f0 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.379+0000 7f19a67fc640 10 client.16083 handle_command_reply: tid=0 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 client.16083 shutdown 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 msgr2=0x5626d3285190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f19b00993a0 tx=0x7f19b0097390 comp rx=0 tx=0).stop 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 shutdown_connections 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5626d32fc2f0 0x5626d32fe6e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d3281370 0x5626d3285c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626d3277b40 0x5626d32856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 --2- 172.21.15.86:0/2667968705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626d32765d0 0x5626d3285190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 >> 172.21.15.86:0/2667968705 conn(0x5626d323f310 msgr2=0x5626d327f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19a67fc640 0 client.16083 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 shutdown_connections 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 1 -- 172.21.15.86:0/2667968705 wait complete. 2024-09-05T06:43:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c7403740 20 client.16083 trim_cache size 0 max 16384 2024-09-05T06:43:49.398 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 msgr2=0x7f19ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 0x7f19ac0a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19b801f000 tx=0x7f19b8018000 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 msgr2=0x7f19c01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.380+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f19b000b4a0 tx=0x7f19b000b960 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 shutdown_connections 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19ac0a2cf0 0x7f19ac0a51b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19c010c170 0x7f19c019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19c0103660 0x7f19c0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 --2- 172.21.15.86:0/1810364908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0102d40 0x7f19c01984e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 >> 172.21.15.86:0/1810364908 conn(0x7f19c00fc7a0 msgr2=0x7f19c00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 shutdown_connections 2024-09-05T06:43:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:49.381+0000 7f19c502a640 1 -- 172.21.15.86:0/1810364908 wait complete. 2024-09-05T06:43:49.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.72 MB/sec warmup 69 sec latency 18460.238 ms 2024-09-05T06:43:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:49 smithi086 ceph-mon[30943]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-05T06:43:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:49 smithi096 ceph-mon[34468]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-05T06:43:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:49 smithi115 ceph-mon[34791]: pgmap v80: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-05T06:43:50.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:43:50.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.68 MB/sec warmup 70 sec latency 19460.282 ms 2024-09-05T06:43:50.560 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/2931544265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 msgr2=0x7fb8e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/2931544265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e410e610 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009900 tx=0x7fb8dc02a480 comp rx=0 tx=0).stop 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/2931544265 shutdown_connections 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/2931544265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e410e610 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/2931544265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/2931544265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 0x7fb8e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/2931544265 >> 172.21.15.86:0/2931544265 conn(0x7fb8e40fc7a0 msgr2=0x7fb8e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/2931544265 shutdown_connections 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.037+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/2931544265 wait complete. 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.038+0000 7fb8e9dcd640 1 Processor -- start 2024-09-05T06:43:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.038+0000 7fb8e9dcd640 1 -- start start 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 0x7fb8e41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4111e80 con 0x7fb8e410c170 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4111d00 con 0x7fb8e4103660 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4112000 con 0x7fb8e4102d40 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38400/0 (socket says 172.21.15.86:38400) 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 -- 172.21.15.86:0/3425428876 learned_addr learned my addr 172.21.15.86:0/3425428876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 0x7fb8e41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 -- 172.21.15.86:0/3425428876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 msgr2=0x7fb8e41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 0x7fb8e41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 -- 172.21.15.86:0/3425428876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 msgr2=0x7fb8e41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 -- 172.21.15.86:0/3425428876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e419d350 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e3fff640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc004770 tx=0x7fb8dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e0ff9640 1 -- 172.21.15.86:0/3425428876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc03f030 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e0ff9640 1 -- 172.21.15.86:0/3425428876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8dc0318e0 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e419d5e0 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e0ff9640 1 -- 172.21.15.86:0/3425428876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc035070 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.039+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e41a4f20 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.041+0000 7fb8e0ff9640 1 -- 172.21.15.86:0/3425428876 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb8dc042030 con 0x7fb8e410c170 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 Processor -- start 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 -- start start 2024-09-05T06:43:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbccf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632dbc282f0 con 0x5632dbcc25d0 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632dbbcfcf0 con 0x5632dbccd370 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632dbc242b0 con 0x5632dbcc3b40 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38406/0 (socket says 172.21.15.86:38406) 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 -- 172.21.15.86:0/860423904 learned_addr learned my addr 172.21.15.86:0/860423904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e0ff9640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 0x7fb8cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 -- 172.21.15.86:0/860423904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 msgr2=0x5632dbcc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e3fff640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 0x7fb8cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 -- 172.21.15.86:0/860423904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 msgr2=0x5632dbccf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbccf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.042+0000 7fb8e37fe640 1 -- 172.21.15.86:0/860423904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dbbb43a0 con 0x5632dbcc25d0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8e0ff9640 1 -- 172.21.15.86:0/3425428876 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb8dc0efc20 con 0x7fb8e410c170 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4010480 tx=0x7fb8d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6bbc98f8aa93df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8c9ffb640 1 -- 172.21.15.86:0/860423904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4003b70 con 0x5632dbcc25d0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8e3fff640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 0x7fb8cc0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc04a000 tx=0x7fb8dc08edd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8c9ffb640 1 -- 172.21.15.86:0/860423904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d4003d10 con 0x5632dbcc25d0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8c9ffb640 1 -- 172.21.15.86:0/860423904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011b00 con 0x5632dbcc25d0 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/860423904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 msgr2=0x5632dbcc2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.043+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4010480 tx=0x7fb8d4009f90 comp rx=0 tx=0).stop 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/860423904 shutdown_connections 2024-09-05T06:43:51.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbccf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcc2a10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/860423904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/860423904 >> 172.21.15.86:0/860423904 conn(0x5632dbc8b310 msgr2=0x5632dbcbec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/860423904 shutdown_connections 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/860423904 wait complete. 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- start start 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 Processor -- start 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbccce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632dbcd0070 con 0x5632dbccd370 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632dbbe3060 con 0x5632dbcc3b40 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632dbcd01f0 con 0x5632dbcc25d0 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.044+0000 7fb8ec1a6740 10 client.0 _ll_register_callbacks cb 0x5632db9368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:51.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38412/0 (socket says 172.21.15.86:38412) 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/1444389797 learned_addr learned my addr 172.21.15.86:0/1444389797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbccce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 msgr2=0x5632dbccce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbccce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/1444389797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 msgr2=0x5632dbcca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/1444389797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632db9993c0 con 0x5632dbccd370 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800ce50 tx=0x7fb8d800c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828116af47ee6cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800f040 con 0x5632dbccd370 2024-09-05T06:43:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb8d8013a50 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d4003d10 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8025050 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8ec1a6740 10 client.16095 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632dbcdbbe0 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.045+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d4003d10 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb8d8017c90 con 0x5632dbccd370 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 10 client.16095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 20 client.16095 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbcdd0a0 0x5632dbcdf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 4 client.16095 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632dbc35860 con 0x5632dbcdd0a0 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.046+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbcdd0a0 0x5632dbcdf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8c2ffd640 10 client.16095 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:43:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbcdd0a0 0x5632dbcdf4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1feef15880c7432 server_cookie=24511eeb82461b20 in_seq=0 out_seq=0 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/1444389797 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632dbc35860 con 0x5632dbcdd0a0 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8c2ffd640 10 client.16095 handle_command_reply: tid=0 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 client.16095 shutdown 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 msgr2=0x5632dbcca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800ce50 tx=0x7fb8d800c190 comp rx=0 tx=0).stop 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 shutdown_connections 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbcdd0a0 0x5632dbcdf4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.047+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcca870 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/1444389797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbccce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8c2ffd640 0 client.16095 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 >> 172.21.15.86:0/1444389797 conn(0x5632dbc8b310 msgr2=0x5632dbcc4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 shutdown_connections 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/1444389797 wait complete. 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.048+0000 7fb8ec1a6740 20 client.16095 trim_cache size 0 max 16384 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.060+0000 7fb8ec1a6740 1 Processor -- start 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.060+0000 7fb8ec1a6740 1 -- start start 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbcc5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41af020 con 0x5632dbcc25d0 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41b0bc0 con 0x5632dbccd370 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41af6f0 con 0x5632dbcc3b40 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38426/0 (socket says 172.21.15.86:38426) 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/3443111661 learned_addr learned my addr 172.21.15.86:0/3443111661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/3443111661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 msgr2=0x5632dbcc5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/3443111661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 msgr2=0x5632dbcc5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbcc5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/3443111661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dbcc8630 con 0x5632dbcc25d0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8001eb0 tx=0x7fb8d800c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceff6858884e89b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8c97fa640 1 -- 172.21.15.86:0/3443111661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8015e00 con 0x5632dbcc25d0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8c97fa640 1 -- 172.21.15.86:0/3443111661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d801c070 con 0x5632dbcc25d0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.061+0000 7fb8c97fa640 1 -- 172.21.15.86:0/3443111661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8029430 con 0x5632dbcc25d0 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/3443111661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 msgr2=0x5632dbcbee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8001eb0 tx=0x7fb8d800c5b0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/3443111661 shutdown_connections 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbccd370 0x5632dbcc5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc3b40 0x5632dbcc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/3443111661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbcc25d0 0x5632dbcbee10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/3443111661 >> 172.21.15.86:0/3443111661 conn(0x5632dbc8b310 msgr2=0x5632dbcbe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/3443111661 shutdown_connections 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/3443111661 wait complete. 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 -- start start 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.062+0000 7fb8ec1a6740 1 Processor -- start 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbcd5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcd5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41b0490 con 0x5632dbccd370 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41afdc0 con 0x5632dbcc3b40 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41ad000 con 0x5632dbcc25d0 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 10 client.0 _ll_register_callbacks cb 0x7fb8e406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38428/0 (socket says 172.21.15.86:38428) 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/874729800 learned_addr learned my addr 172.21.15.86:0/874729800 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcd5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbcd5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 msgr2=0x5632dbcd5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbcd5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/874729800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 msgr2=0x5632dbcd5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcd5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 -- 172.21.15.86:0/874729800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632dbcc70f0 con 0x5632dbccd370 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcd5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8e2ffd640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5632dbce59e0 tx=0x7fb8d8013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6d998317401e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800c040 con 0x5632dbccd370 2024-09-05T06:43:51.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb8d80209f0 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.063+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d801c070 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8020bf0 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632dbd477e0 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d801c070 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb8d8033410 con 0x5632dbccd370 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 10 client.16101 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 20 client.16101 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbd47da0 0x5632dbd4a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 4 client.16101 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632dbc2fd70 con 0x5632dbd47da0 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.064+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbd47da0 0x5632dbd4a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8c2ffd640 10 client.16101 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:43:51.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8e37fe640 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbd47da0 0x5632dbd4a190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28458f3bbc67864e server_cookie=3eea7723aabea03e in_seq=0 out_seq=0 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8c2ffd640 1 -- 172.21.15.86:0/874729800 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632dbc2fd70 con 0x5632dbd47da0 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8c2ffd640 10 client.16101 handle_command_reply: tid=0 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 client.16101 shutdown 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 msgr2=0x5632dbcc68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5632dbce59e0 tx=0x7fb8d8013be0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 shutdown_connections 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5632dbd47da0 0x5632dbd4a190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632dbccd370 0x5632dbcc68f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632dbcc3b40 0x5632dbcd5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 --2- 172.21.15.86:0/874729800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632dbcc25d0 0x5632dbcd5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 >> 172.21.15.86:0/874729800 conn(0x5632dbc8b310 msgr2=0x5632dbccb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8c2ffd640 0 client.16101 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 shutdown_connections 2024-09-05T06:43:51.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 1 -- 172.21.15.86:0/874729800 wait complete. 2024-09-05T06:43:51.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.065+0000 7fb8ec1a6740 20 client.16101 trim_cache size 0 max 16384 2024-09-05T06:43:51.081 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:51.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 msgr2=0x7fb8cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 0x7fb8cc0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc04a000 tx=0x7fb8dc08edd0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 msgr2=0x7fb8e419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc004770 tx=0x7fb8dc004270 comp rx=0 tx=0).stop 2024-09-05T06:43:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 shutdown_connections 2024-09-05T06:43:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8cc0a2cf0 0x7fb8cc0a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8e410c170 0x7fb8e419cc70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e4103660 0x7fb8e41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 --2- 172.21.15.86:0/3425428876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4102d40 0x7fb8e41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 >> 172.21.15.86:0/3425428876 conn(0x7fb8e40fc7a0 msgr2=0x7fb8e40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 shutdown_connections 2024-09-05T06:43:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:51.066+0000 7fb8e9dcd640 1 -- 172.21.15.86:0/3425428876 wait complete. 2024-09-05T06:43:51.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.64 MB/sec warmup 71 sec latency 20460.363 ms 2024-09-05T06:43:51.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:51 smithi086 ceph-mon[30943]: pgmap v81: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-09-05T06:43:51.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:43:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:51 smithi096 ceph-mon[34468]: pgmap v81: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-09-05T06:43:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:51 smithi115 ceph-mon[34791]: pgmap v81: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-09-05T06:43:52.048 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:52.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:43:52.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.60 MB/sec warmup 72 sec latency 21460.430 ms 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 -- 172.21.15.86:0/2326591960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 msgr2=0x7f0d4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 --2- 172.21.15.86:0/2326591960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c10ea90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009ca0 tx=0x7f0d4402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 -- 172.21.15.86:0/2326591960 shutdown_connections 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 --2- 172.21.15.86:0/2326591960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c10ea90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 --2- 172.21.15.86:0/2326591960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c102790 0x7f0d4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 --2- 172.21.15.86:0/2326591960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c101de0 0x7f0d4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 -- 172.21.15.86:0/2326591960 >> 172.21.15.86:0/2326591960 conn(0x7f0d4c0fd8c0 msgr2=0x7f0d4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 -- 172.21.15.86:0/2326591960 shutdown_connections 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.529+0000 7f0d51432640 1 -- 172.21.15.86:0/2326591960 wait complete. 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 Processor -- start 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 -- start start 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c101de0 0x7f0d4c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c102790 0x7f0d4c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111e40 con 0x7f0d4c106f10 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111cc0 con 0x7f0d4c101de0 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d51432640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111fc0 con 0x7f0d4c102790 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39794/0 (socket says 172.21.15.86:39794) 2024-09-05T06:43:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.530+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/612801407 learned_addr learned my addr 172.21.15.86:0/612801407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4affd640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c101de0 0x7f0d4c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d43fff640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c102790 0x7f0d4c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/612801407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c102790 msgr2=0x7f0d4c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c102790 0x7f0d4c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/612801407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c101de0 msgr2=0x7f0d4c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c101de0 0x7f0d4c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/612801407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c19d330 con 0x7f0d4c106f10 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009dc0 tx=0x7f0d44032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d48ff9640 1 -- 172.21.15.86:0/612801407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44003cf0 con 0x7f0d4c106f10 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d48ff9640 1 -- 172.21.15.86:0/612801407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d440044f0 con 0x7f0d4c106f10 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c19d5c0 con 0x7f0d4c106f10 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d48ff9640 1 -- 172.21.15.86:0/612801407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44030bf0 con 0x7f0d4c106f10 2024-09-05T06:43:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.531+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a4f00 con 0x7f0d4c106f10 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.533+0000 7f0d48ff9640 1 -- 172.21.15.86:0/612801407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0d44004040 con 0x7f0d4c106f10 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.534+0000 7f0d48ff9640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 0x7f0d380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.534+0000 7f0d4affd640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 0x7f0d380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.534+0000 7f0d48ff9640 1 -- 172.21.15.86:0/612801407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0d440f0e80 con 0x7f0d4c106f10 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d4affd640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 0x7f0d380a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009890 tx=0x7f0d3c008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 Processor -- start 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 -- start start 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861485a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861492760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618613eb2f0 con 0x561861486b40 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561861392cf0 con 0x5618614855d0 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d5380b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618613e72b0 con 0x561861490370 2024-09-05T06:43:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d4affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d4affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39806/0 (socket says 172.21.15.86:39806) 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d4affd640 1 -- 172.21.15.86:0/362063022 learned_addr learned my addr 172.21.15.86:0/362063022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.535+0000 7f0d43fff640 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861485a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 -- 172.21.15.86:0/362063022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 msgr2=0x561861492760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861492760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 -- 172.21.15.86:0/362063022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 msgr2=0x561861485a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861485a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 -- 172.21.15.86:0/362063022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618613773a0 con 0x561861486b40 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d43fff640 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861485a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d4affd640 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d830 tx=0x7f0d3c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd92e94979dadb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d417fa640 1 -- 172.21.15.86:0/362063022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c024070 con 0x561861486b40 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d417fa640 1 -- 172.21.15.86:0/362063022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c01f400 con 0x561861486b40 2024-09-05T06:43:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.536+0000 7f0d417fa640 1 -- 172.21.15.86:0/362063022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c014e60 con 0x561861486b40 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- 172.21.15.86:0/362063022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 msgr2=0x561861485090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d830 tx=0x7f0d3c010000 comp rx=0 tx=0).stop 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- 172.21.15.86:0/362063022 shutdown_connections 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861492760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861485a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 --2- 172.21.15.86:0/362063022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861485090 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- 172.21.15.86:0/362063022 >> 172.21.15.86:0/362063022 conn(0x56186144e310 msgr2=0x561861481c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- 172.21.15.86:0/362063022 shutdown_connections 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- 172.21.15.86:0/362063022 wait complete. 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 -- start start 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.537+0000 7f0d5380b740 1 Processor -- start 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x56186148d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861486b40 0x56186148d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561861493070 con 0x561861490370 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5618613a6060 con 0x561861486b40 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618614931f0 con 0x5618614855d0 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d5380b740 10 client.0 _ll_register_callbacks cb 0x5618610f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39812/0 (socket says 172.21.15.86:39812) 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/3897130689 learned_addr learned my addr 172.21.15.86:0/3897130689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4affd640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x56186148d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861486b40 0x56186148d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/3897130689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 msgr2=0x56186148d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x56186148d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861486b40 msgr2=0x56186148d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861486b40 0x56186148d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 -- 172.21.15.86:0/3897130689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186115c3c0 con 0x561861490370 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005e30 tx=0x7f0d440bca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec034599e5c52ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d327fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440bd770 con 0x561861490370 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.538+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0d440046a0 con 0x561861490370 2024-09-05T06:43:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.539+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c01f400 con 0x561861490370 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.539+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d440bd070 con 0x561861490370 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.539+0000 7f0d5380b740 10 client.16113 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.539+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561861493070 con 0x561861490370 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c01f400 con 0x561861490370 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0d440b96e0 con 0x561861490370 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 10 client.16113 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 20 client.16113 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618614a43c0 0x5618614a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 4 client.16113 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618613f8860 con 0x5618614a43c0 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.540+0000 7f0d4affd640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618614a43c0 0x5618614a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.541+0000 7f0d327fc640 10 client.16113 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.541+0000 7f0d4affd640 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618614a43c0 0x5618614a67e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfca0c2314ccfcb server_cookie=6b40937a8c24072b in_seq=0 out_seq=0 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.541+0000 7f0d327fc640 1 -- 172.21.15.86:0/3897130689 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618613f8860 con 0x5618614a43c0 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.541+0000 7f0d327fc640 10 client.16113 handle_command_reply: tid=0 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.541+0000 7f0d5380b740 1 client.16113 shutdown 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 msgr2=0x56186148db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005e30 tx=0x7f0d440bca90 comp rx=0 tx=0).stop 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 shutdown_connections 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618614a43c0 0x5618614a67e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861490370 0x56186148db70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861486b40 0x56186148d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 --2- 172.21.15.86:0/3897130689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x56186148d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 >> 172.21.15.86:0/3897130689 conn(0x56186144e310 msgr2=0x561861487930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d327fc640 0 client.16113 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 shutdown_connections 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 1 -- 172.21.15.86:0/3897130689 wait complete. 2024-09-05T06:43:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.542+0000 7f0d5380b740 20 client.16113 trim_cache size 0 max 16384 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 Processor -- start 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 -- start start 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x5618614ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861490370 0x561861487110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ace60 con 0x561861486b40 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1aea00 con 0x561861490370 2024-09-05T06:43:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d5380b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ad530 con 0x5618614855d0 2024-09-05T06:43:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39820/0 (socket says 172.21.15.86:39820) 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.555+0000 7f0d43fff640 1 -- 172.21.15.86:0/1355450591 learned_addr learned my addr 172.21.15.86:0/1355450591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861490370 0x561861487110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d4affd640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x5618614ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 -- 172.21.15.86:0/1355450591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 msgr2=0x5618614ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x5618614ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 -- 172.21.15.86:0/1355450591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861490370 msgr2=0x561861487110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861490370 0x561861487110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 -- 172.21.15.86:0/1355450591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561861482df0 con 0x561861486b40 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d4affd640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x5618614ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400cae0 tx=0x7f0d340113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12bf64ad5c0a7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d40ff9640 1 -- 172.21.15.86:0/1355450591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34011db0 con 0x561861486b40 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d40ff9640 1 -- 172.21.15.86:0/1355450591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d340035f0 con 0x561861486b40 2024-09-05T06:43:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.556+0000 7f0d40ff9640 1 -- 172.21.15.86:0/1355450591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3401b690 con 0x561861486b40 2024-09-05T06:43:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 -- 172.21.15.86:0/1355450591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 msgr2=0x5618614ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400cae0 tx=0x7f0d340113e0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 -- 172.21.15.86:0/1355450591 shutdown_connections 2024-09-05T06:43:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561861490370 0x561861487110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x5618614ac610 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1355450591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618614855d0 0x5618614ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 -- 172.21.15.86:0/1355450591 >> 172.21.15.86:0/1355450591 conn(0x56186144e310 msgr2=0x56186148c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 -- 172.21.15.86:0/1355450591 shutdown_connections 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.557+0000 7f0d5380b740 1 -- 172.21.15.86:0/1355450591 wait complete. 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 -- start start 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 Processor -- start 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861498a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861494280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ae2d0 con 0x561861486b40 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1adc00 con 0x5618614855d0 2024-09-05T06:43:52.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1aae40 con 0x561861490370 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d5380b740 10 client.0 _ll_register_callbacks cb 0x7f0d440305c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39832/0 (socket says 172.21.15.86:39832) 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 -- 172.21.15.86:0/1611101525 learned_addr learned my addr 172.21.15.86:0/1611101525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d4affd640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861498a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d4b7fe640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861494280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 -- 172.21.15.86:0/1611101525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 msgr2=0x561861494280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861494280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 -- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 msgr2=0x561861498a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861498a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.558+0000 7f0d43fff640 1 -- 172.21.15.86:0/1611101525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618614948c0 con 0x561861486b40 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0d34003a50 tx=0x7f0d34003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41173239b360fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3402c060 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0d34004410 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d340035f0 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d34033050 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 10 client.16119 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56186150ef20 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d3400c6e0 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0d3401b570 con 0x561861486b40 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 10 client.16119 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 20 client.16119 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56186150f680 0x561861511aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 4 client.16119 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:43:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.559+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0d340035f0 con 0x56186150f680 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.560+0000 7f0d4affd640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56186150f680 0x561861511aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.560+0000 7f0d327fc640 10 client.16119 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.560+0000 7f0d4affd640 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56186150f680 0x561861511aa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a67e5e8acb293d server_cookie=2c3a56b9a33eaec9 in_seq=0 out_seq=0 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.560+0000 7f0d327fc640 1 -- 172.21.15.86:0/1611101525 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0d340035f0 con 0x56186150f680 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.560+0000 7f0d327fc640 10 client.16119 handle_command_reply: tid=0 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 client.16119 shutdown 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 msgr2=0x561861493d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0d34003a50 tx=0x7f0d34003a80 comp rx=0 tx=0).stop 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 shutdown_connections 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56186150f680 0x561861511aa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561861490370 0x561861494280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561861486b40 0x561861493d40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 --2- 172.21.15.86:0/1611101525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618614855d0 0x561861498a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 >> 172.21.15.86:0/1611101525 conn(0x56186144e310 msgr2=0x56186148d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d327fc640 0 client.16119 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 shutdown_connections 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 1 -- 172.21.15.86:0/1611101525 wait complete. 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.561+0000 7f0d5380b740 20 client.16119 trim_cache size 0 max 16384 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:43:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 msgr2=0x7f0d380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 0x7f0d380a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009890 tx=0x7f0d3c008040 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 msgr2=0x7f0d4c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009dc0 tx=0x7f0d44032ff0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 shutdown_connections 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0d380a2cf0 0x7f0d380a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4c106f10 0x7f0d4c19cc50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c102790 0x7f0d4c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 --2- 172.21.15.86:0/612801407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d4c101de0 0x7f0d4c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 >> 172.21.15.86:0/612801407 conn(0x7f0d4c0fd8c0 msgr2=0x7f0d4c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 shutdown_connections 2024-09-05T06:43:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:52.562+0000 7f0d51432640 1 -- 172.21.15.86:0/612801407 wait complete. 2024-09-05T06:43:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:53.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.57 MB/sec warmup 73 sec latency 22460.465 ms 2024-09-05T06:43:53.682 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:53 smithi086 ceph-mon[30943]: pgmap v82: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:53 smithi096 ceph-mon[34468]: pgmap v82: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 -- 172.21.15.86:0/2081859536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0102ef0 msgr2=0x7fe6a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/2081859536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0102ef0 0x7fe6a010da40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe698009ca0 tx=0x7fe69802a950 comp rx=0 tx=0).stop 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 -- 172.21.15.86:0/2081859536 shutdown_connections 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/2081859536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0102ef0 0x7fe6a010da40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/2081859536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 0x7fe6a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/2081859536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0100e20 0x7fe6a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 -- 172.21.15.86:0/2081859536 >> 172.21.15.86:0/2081859536 conn(0x7fe6a00fc7d0 msgr2=0x7fe6a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 -- 172.21.15.86:0/2081859536 shutdown_connections 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.910+0000 7fe6a67ff640 1 -- 172.21.15.86:0/2081859536 wait complete. 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 Processor -- start 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 -- start start 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 0x7fe6a01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110e30 con 0x7fe6a0100e20 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110cb0 con 0x7fe6a0102ef0 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe6a67ff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110fb0 con 0x7fe6a0102570 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39852/0 (socket says 172.21.15.86:39852) 2024-09-05T06:43:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.911+0000 7fe69ffff640 1 -- 172.21.15.86:0/279178348 learned_addr learned my addr 172.21.15.86:0/279178348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69f7fe640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 0x7fe6a01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 -- 172.21.15.86:0/279178348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 msgr2=0x7fe6a01978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 0x7fe6a01978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 -- 172.21.15.86:0/279178348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0102ef0 msgr2=0x7fe6a019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe6a4d9c640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 -- 172.21.15.86:0/279178348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a019c330 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69ffff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d950 tx=0x7fe69000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690017070 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a019c620 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe690009c30 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69000f620 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.912+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a3f00 con 0x7fe6a0100e20 2024-09-05T06:43:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.913+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a0101200 con 0x7fe6a0100e20 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.914+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6900109a0 con 0x7fe6a0100e20 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.915+0000 7fe69d7fa640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 0x7fe68c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.915+0000 7fe69f7fe640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 0x7fe68c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.915+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe6900cf570 con 0x7fe6a0100e20 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.915+0000 7fe69f7fe640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 0x7fe68c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe688005fd0 tx=0x7fe68800b510 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:53.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:53.919+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe690009de0 con 0x7fe6a0100e20 2024-09-05T06:43:53.935 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:53 smithi115 ceph-mon[34791]: pgmap v82: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.121+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6a0109090 con 0x7fe6a0100e20 2024-09-05T06:43:54.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.122+0000 7fe69d7fa640 1 -- 172.21.15.86:0/279178348 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7fe69009bd50 con 0x7fe6a0100e20 2024-09-05T06:43:54.122 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:43:54.122 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:43:54.122 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 msgr2=0x7fe68c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 0x7fe68c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe688005fd0 tx=0x7fe68800b510 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 msgr2=0x7fe6a0197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d950 tx=0x7fe69000de10 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 shutdown_connections 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe68c0a2cf0 0x7fe68c0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a0102570 0x7fe6a01978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 --2- 172.21.15.86:0/279178348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a0100e20 0x7fe6a0197380 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 >> 172.21.15.86:0/279178348 conn(0x7fe6a00fc7d0 msgr2=0x7fe6a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:54.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 shutdown_connections 2024-09-05T06:43:54.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:54.124+0000 7fe6a67ff640 1 -- 172.21.15.86:0/279178348 wait complete. 2024-09-05T06:43:54.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:43:54.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:43:54.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.53 MB/sec warmup 74 sec latency 23460.531 ms 2024-09-05T06:43:54.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:54 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/279178348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:54.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:54 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/279178348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:54.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:54 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/279178348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:43:54.950 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 17bd525e --release --await' 2024-09-05T06:43:54.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 17bd525e --release --await 2024-09-05T06:43:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:55.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.50 MB/sec warmup 75 sec latency 24460.581 ms 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 -- 172.21.15.86:0/3619699810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102570 msgr2=0x7f2fcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 --2- 172.21.15.86:0/3619699810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102570 0x7f2fcc1029b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009900 tx=0x7f2fc002a480 comp rx=0 tx=0).stop 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 -- 172.21.15.86:0/3619699810 shutdown_connections 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 --2- 172.21.15.86:0/3619699810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 --2- 172.21.15.86:0/3619699810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102570 0x7f2fcc1029b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 --2- 172.21.15.86:0/3619699810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc100e20 0x7f2fcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 -- 172.21.15.86:0/3619699810 >> 172.21.15.86:0/3619699810 conn(0x7f2fcc0fc7d0 msgr2=0x7f2fcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 -- 172.21.15.86:0/3619699810 shutdown_connections 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.552+0000 7f2fd2892640 1 -- 172.21.15.86:0/3619699810 wait complete. 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 Processor -- start 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 -- start start 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc100e20 0x7f2fcc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 0x7f2fcc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc110e30 con 0x7f2fcc102ef0 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc110cb0 con 0x7f2fcc102570 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd2892640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc110fb0 con 0x7f2fcc100e20 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd0e2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd0e2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39862/0 (socket says 172.21.15.86:39862) 2024-09-05T06:43:55.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fd0e2f640 1 -- 172.21.15.86:0/203292404 learned_addr learned my addr 172.21.15.86:0/203292404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.553+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc100e20 0x7f2fcc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 0x7f2fcc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 -- 172.21.15.86:0/203292404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc100e20 msgr2=0x7f2fcc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc100e20 0x7f2fcc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 -- 172.21.15.86:0/203292404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 msgr2=0x7f2fcc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 0x7f2fcc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 -- 172.21.15.86:0/203292404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1a6980 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 0x7f2fcc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd0e2f640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400eaa0 tx=0x7f2fc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fc97fa640 1 -- 172.21.15.86:0/203292404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400cdb0 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1a6c70 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fc97fa640 1 -- 172.21.15.86:0/203292404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc4005020 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fc97fa640 1 -- 172.21.15.86:0/203292404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40116d0 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.554+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a7200 con 0x7f2fcc102ef0 2024-09-05T06:43:55.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.556+0000 7f2fc97fa640 1 -- 172.21.15.86:0/203292404 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2fc40051d0 con 0x7f2fcc102ef0 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.556+0000 7f2fd4c6b740 1 Processor -- start 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 -- start start 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fb3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e659f19280 con 0x55e659fbed40 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e659ec0b10 con 0x55e659fb3b60 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e659f15240 con 0x55e659fb4480 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fc97fa640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39874/0 (socket says 172.21.15.86:39874) 2024-09-05T06:43:55.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/4240732943 learned_addr learned my addr 172.21.15.86:0/4240732943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fb3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fd0e2f640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fc97fa640 1 -- 172.21.15.86:0/203292404 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2fc40cf940 con 0x7f2fcc102ef0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/4240732943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 msgr2=0x55e659fbe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/4240732943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 msgr2=0x55e659fb3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fb3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/4240732943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e659ecd1f0 con 0x55e659fbed40 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.557+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2fc002a9a0 tx=0x7f2fc0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f31c5b93ccd56b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd0e2f640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40ccd60 tx=0x7f2fc406eaf0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fb27fc640 1 -- 172.21.15.86:0/4240732943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0035040 con 0x55e659fbed40 2024-09-05T06:43:55.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fb27fc640 1 -- 172.21.15.86:0/4240732943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc0005020 con 0x55e659fbed40 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fb27fc640 1 -- 172.21.15.86:0/4240732943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00044e0 con 0x55e659fbed40 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/4240732943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 msgr2=0x55e659fc1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2fc002a9a0 tx=0x7f2fc0005b70 comp rx=0 tx=0).stop 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/4240732943 shutdown_connections 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fc1130 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/4240732943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fb3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/4240732943 >> 172.21.15.86:0/4240732943 conn(0x55e659fb00f0 msgr2=0x55e659fb04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.558+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/4240732943 shutdown_connections 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/4240732943 wait complete. 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 -- start start 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 Processor -- start 2024-09-05T06:43:55.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fbbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e659fc1910 con 0x55e659fbed40 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e659ed3f10 con 0x55e659fb3b60 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e659fc1a90 con 0x55e659fb4480 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.559+0000 7f2fd4c6b740 10 client.0 _ll_register_callbacks cb 0x55e659e9ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39886/0 (socket says 172.21.15.86:39886) 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/540481336 learned_addr learned my addr 172.21.15.86:0/540481336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fbbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/540481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 msgr2=0x55e659fbc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 msgr2=0x55e659fbbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fbbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 -- 172.21.15.86:0/540481336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e659c8a6a0 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0006010 tx=0x7f2fc0032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a93b28fa34e60d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc004d030 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2fc0004260 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc0005020 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2fc0057050 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fd4c6b740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.560+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e659fcd5a0 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc0005020 con 0x55e659fbed40 2024-09-05T06:43:55.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2fc003a040 con 0x55e659fbed40 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 20 client.16137 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e659fce8f0 0x55e659fd0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 4 client.16137 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e659f267f0 con 0x55e659fce8f0 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e659fce8f0 0x55e659fd0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fab7fe640 10 client.16137 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.561+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e659fce8f0 0x55e659fd0ce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bb9fa662ef0ff1 server_cookie=758a43c7bac422 in_seq=0 out_seq=0 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fab7fe640 1 -- 172.21.15.86:0/540481336 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e659f267f0 con 0x55e659fce8f0 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fab7fe640 10 client.16137 handle_command_reply: tid=0 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 client.16137 shutdown 2024-09-05T06:43:55.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 msgr2=0x55e659fbc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0006010 tx=0x7f2fc0032c40 comp rx=0 tx=0).stop 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 shutdown_connections 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e659fce8f0 0x55e659fd0ce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fbed40 0x55e659fbc700 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb4480 0x55e659fbc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fab7fe640 0 client.16137 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/540481336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fb3b60 0x55e659fbbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 >> 172.21.15.86:0/540481336 conn(0x55e659fb00f0 msgr2=0x55e659fb5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 shutdown_connections 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.562+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/540481336 wait complete. 2024-09-05T06:43:55.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.563+0000 7f2fd4c6b740 20 client.16137 trim_cache size 0 max 16384 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 Processor -- start 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 -- start start 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fb5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b1290 con 0x55e659fb4480 2024-09-05T06:43:55.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b2e30 con 0x55e659fbed40 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b1960 con 0x55e659fb3b60 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39900/0 (socket says 172.21.15.86:39900) 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 -- 172.21.15.86:0/3258744948 learned_addr learned my addr 172.21.15.86:0/3258744948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fb5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 -- 172.21.15.86:0/3258744948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 msgr2=0x55e659fb5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fb5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 -- 172.21.15.86:0/3258744948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 msgr2=0x55e659fd4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcbfff640 1 -- 172.21.15.86:0/3258744948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e659fba1f0 con 0x55e659fb4480 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.580+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fb5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.581+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00b620 tx=0x7f2fbc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5a1293bf8bdaf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.581+0000 7f2fb1ffb640 1 -- 172.21.15.86:0/3258744948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc003de0 con 0x55e659fb4480 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.581+0000 7f2fb1ffb640 1 -- 172.21.15.86:0/3258744948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fbc0054d0 con 0x55e659fb4480 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.581+0000 7f2fb1ffb640 1 -- 172.21.15.86:0/3258744948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc004e60 con 0x55e659fb4480 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/3258744948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 msgr2=0x55e659fd3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00b620 tx=0x7f2fbc00b390 comp rx=0 tx=0).stop 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/3258744948 shutdown_connections 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd3d00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/3258744948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fb5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/3258744948 >> 172.21.15.86:0/3258744948 conn(0x55e659fb00f0 msgr2=0x55e659fb55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/3258744948 shutdown_connections 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/3258744948 wait complete. 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 -- start start 2024-09-05T06:43:55.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.582+0000 7f2fd4c6b740 1 Processor -- start 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fd4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b2700 con 0x55e659fb4480 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b2030 con 0x55e659fbed40 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1af270 con 0x55e659fb3b60 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fd4c6b740 10 client.0 _ll_register_callbacks cb 0x7f2fc0032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39916/0 (socket says 172.21.15.86:39916) 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 -- 172.21.15.86:0/1671861830 learned_addr learned my addr 172.21.15.86:0/1671861830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fd4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 -- 172.21.15.86:0/1671861830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 msgr2=0x55e659fd4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fd4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 -- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 msgr2=0x55e659fd2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 -- 172.21.15.86:0/1671861830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e659fd2f60 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fcbfff640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55e659fb3450 tx=0x7f2fbc003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aae4822e54b2023 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.583+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc02a070 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2fbc004730 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fbc0054d0 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2fbc037050 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 10 client.16143 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e65a039ee0 con 0x55e659fb4480 2024-09-05T06:43:55.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fbc0054d0 con 0x55e659fb4480 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2fbc011a80 con 0x55e659fb4480 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 10 client.16143 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 20 client.16143 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e65a03a600 0x55e65a03ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 4 client.16143 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e659f20d00 con 0x55e65a03a600 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.584+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e65a03a600 0x55e65a03ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.585+0000 7f2fcb7fe640 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e65a03a600 0x55e65a03ca20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0252b33afdd3d61 server_cookie=b37cb3fbb228c025 in_seq=0 out_seq=0 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.585+0000 7f2fab7fe640 10 client.16143 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fab7fe640 1 -- 172.21.15.86:0/1671861830 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e659f20d00 con 0x55e65a03a600 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fab7fe640 10 client.16143 handle_command_reply: tid=0 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 client.16143 shutdown 2024-09-05T06:43:55.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 msgr2=0x55e659fd24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55e659fb3450 tx=0x7f2fbc003c50 comp rx=0 tx=0).stop 2024-09-05T06:43:55.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 shutdown_connections 2024-09-05T06:43:55.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e65a03a600 0x55e65a03ca20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e659fbed40 0x55e659fd2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.586+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e659fb4480 0x55e659fd24e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd4c6b740 1 --2- 172.21.15.86:0/1671861830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e659fb3b60 0x55e659fd4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 >> 172.21.15.86:0/1671861830 conn(0x55e659fb00f0 msgr2=0x55e659fbc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fab7fe640 0 client.16143 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 shutdown_connections 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd4c6b740 1 -- 172.21.15.86:0/1671861830 wait complete. 2024-09-05T06:43:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd4c6b740 20 client.16143 trim_cache size 0 max 16384 2024-09-05T06:43:55.601 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:55.601 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:43:55.601 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "fce0a3cf-24ae-4662-b9d1-c2c9fb596d0e", 2024-09-05T06:43:55.601 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:43:55.601 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 msgr2=0x7f2fb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40ccd60 tx=0x7f2fc406eaf0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 msgr2=0x7f2fcc0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.587+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400eaa0 tx=0x7f2fc400ef60 comp rx=0 tx=0).stop 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 shutdown_connections 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc0fd700 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fcc102570 0x7f2fcc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 --2- 172.21.15.86:0/203292404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fcc100e20 0x7f2fcc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 >> 172.21.15.86:0/203292404 conn(0x7f2fcc0fc7d0 msgr2=0x7f2fcc06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 shutdown_connections 2024-09-05T06:43:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:55.588+0000 7f2fd2892640 1 -- 172.21.15.86:0/203292404 wait complete. 2024-09-05T06:43:55.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:55 smithi086 ceph-mon[30943]: pgmap v83: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:55 smithi096 ceph-mon[34468]: pgmap v83: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:55 smithi115 ceph-mon[34791]: pgmap v83: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-05T06:43:56.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:43:56.446 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:56.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31650 2.47 MB/sec warmup 76 sec latency 25460.610 ms 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:56 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:56 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:56 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:43:56.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 -- 172.21.15.86:0/975926047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec541000d0 msgr2=0x7fec54100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 --2- 172.21.15.86:0/975926047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec541000d0 0x7fec54100530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009900 tx=0x7fec3c02a410 comp rx=0 tx=0).stop 2024-09-05T06:43:56.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 -- 172.21.15.86:0/975926047 shutdown_connections 2024-09-05T06:43:56.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 --2- 172.21.15.86:0/975926047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 0x7fec5410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 --2- 172.21.15.86:0/975926047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec541000d0 0x7fec54100530 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 --2- 172.21.15.86:0/975926047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec54106ba0 0x7fec54106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 -- 172.21.15.86:0/975926047 >> 172.21.15.86:0/975926047 conn(0x7fec54077210 msgr2=0x7fec540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 -- 172.21.15.86:0/975926047 shutdown_connections 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.949+0000 7fec59f8a640 1 -- 172.21.15.86:0/975926047 wait complete. 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 Processor -- start 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 -- start start 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec541000d0 0x7fec541012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 0x7fec541017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111f40 con 0x7fec54106ba0 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111dc0 con 0x7fec54100a70 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec59f8a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec541120c0 con 0x7fec541000d0 2024-09-05T06:43:56.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec53fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec53fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39928/0 (socket says 172.21.15.86:39928) 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec53fff640 1 -- 172.21.15.86:0/867440650 learned_addr learned my addr 172.21.15.86:0/867440650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec537fe640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec541000d0 0x7fec541012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.950+0000 7fec4bfff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 0x7fec541017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 -- 172.21.15.86:0/867440650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec541000d0 msgr2=0x7fec541012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec541000d0 0x7fec541012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 -- 172.21.15.86:0/867440650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 msgr2=0x7fec541017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 0x7fec541017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 -- 172.21.15.86:0/867440650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec541a6860 con 0x7fec54106ba0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec53fff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00cce0 tx=0x7fec4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec517fa640 1 -- 172.21.15.86:0/867440650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00f450 con 0x7fec54106ba0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec541a6b50 con 0x7fec54106ba0 2024-09-05T06:43:56.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec517fa640 1 -- 172.21.15.86:0/867440650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec4c00faa0 con 0x7fec54106ba0 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec517fa640 1 -- 172.21.15.86:0/867440650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c018b70 con 0x7fec54106ba0 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.951+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541a7090 con 0x7fec54106ba0 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.953+0000 7fec517fa640 1 -- 172.21.15.86:0/867440650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fec4c018d10 con 0x7fec54106ba0 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.953+0000 7fec5c363740 1 Processor -- start 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.953+0000 7fec5c363740 1 -- start start 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d927440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d931d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162d88c620 con 0x56162d932240 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56162d833e50 con 0x56162d927980 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec5c363740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56162d8885e0 con 0x56162d927060 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39932/0 (socket says 172.21.15.86:39932) 2024-09-05T06:43:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 -- 172.21.15.86:0/1808647439 learned_addr learned my addr 172.21.15.86:0/1808647439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec517fa640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 0x7fec400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d927440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec53fff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 0x7fec400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec537fe640 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d931d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 -- 172.21.15.86:0/1808647439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 msgr2=0x56162d927440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d927440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 -- 172.21.15.86:0/1808647439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 msgr2=0x56162d931d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d931d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 -- 172.21.15.86:0/1808647439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162d824d40 con 0x56162d932240 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec517fa640 1 -- 172.21.15.86:0/867440650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fec4c0cfc50 con 0x7fec54106ba0 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fec3c032fd0 tx=0x7fec3c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55647bc31a63778 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec49ffb640 1 -- 172.21.15.86:0/1808647439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c035040 con 0x56162d932240 2024-09-05T06:43:56.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.954+0000 7fec53fff640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 0x7fec400a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0cd5c0 tx=0x7fec4c06dd50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec49ffb640 1 -- 172.21.15.86:0/1808647439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec3c0035f0 con 0x56162d932240 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec49ffb640 1 -- 172.21.15.86:0/1808647439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0044e0 con 0x56162d932240 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- 172.21.15.86:0/1808647439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 msgr2=0x56162d934630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fec3c032fd0 tx=0x7fec3c0056d0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- 172.21.15.86:0/1808647439 shutdown_connections 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d932240 0x56162d934630 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d931d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 --2- 172.21.15.86:0/1808647439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d927440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- 172.21.15.86:0/1808647439 >> 172.21.15.86:0/1808647439 conn(0x56162d9235f0 msgr2=0x56162d9239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- 172.21.15.86:0/1808647439 shutdown_connections 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- 172.21.15.86:0/1808647439 wait complete. 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.955+0000 7fec5c363740 1 -- start start 2024-09-05T06:43:56.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 Processor -- start 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927980 0x56162d92f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d92fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162d934e10 con 0x56162d927060 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56162d814f10 con 0x56162d932240 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56162d934f90 con 0x56162d927980 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec5c363740 10 client.0 _ll_register_callbacks cb 0x56162d838db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39946/0 (socket says 172.21.15.86:39946) 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 -- 172.21.15.86:0/749868600 learned_addr learned my addr 172.21.15.86:0/749868600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d92fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec537fe640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927980 0x56162d92f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 -- 172.21.15.86:0/749868600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927980 msgr2=0x56162d92f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927980 0x56162d92f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 -- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 msgr2=0x56162d92fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d92fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.956+0000 7fec4bfff640 1 -- 172.21.15.86:0/749868600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162d59b580 con 0x56162d927060 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec4bfff640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f214b91894b8d508 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c050020 con 0x56162d927060 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fec3c0037a0 con 0x56162d927060 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c0035f0 con 0x56162d927060 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec3c035030 con 0x56162d927060 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:56.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56162d665960 con 0x56162d927060 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec3c0035f0 con 0x56162d927060 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fec3c03a040 con 0x56162d927060 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 10 client.16155 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 20 client.16155 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d941c50 0x56162d944040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 4 client.16155 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.957+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162d899b90 con 0x56162d941c50 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.958+0000 7fec537fe640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d941c50 0x56162d944040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.958+0000 7fec32ffd640 10 client.16155 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.958+0000 7fec537fe640 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d941c50 0x56162d944040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a399c08a77fb3c8a server_cookie=2143a9a93d612b45 in_seq=0 out_seq=0 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec32ffd640 1 -- 172.21.15.86:0/749868600 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56162d899b90 con 0x56162d941c50 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec32ffd640 10 client.16155 handle_command_reply: tid=0 2024-09-05T06:43:56.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 client.16155 shutdown 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 msgr2=0x56162d92f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c005a20 comp rx=0 tx=0).stop 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 shutdown_connections 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d941c50 0x56162d944040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d92fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927980 0x56162d92f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec32ffd640 0 client.16155 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 --2- 172.21.15.86:0/749868600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d92f110 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 >> 172.21.15.86:0/749868600 conn(0x56162d9235f0 msgr2=0x56162d9292d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 shutdown_connections 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 1 -- 172.21.15.86:0/749868600 wait complete. 2024-09-05T06:43:56.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.959+0000 7fec5c363740 20 client.16155 trim_cache size 0 max 16384 2024-09-05T06:43:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 Processor -- start 2024-09-05T06:43:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 -- start start 2024-09-05T06:43:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d92c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d932240 0x56162d92c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec541b11e0 con 0x56162d927060 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec541b2d80 con 0x56162d927980 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec5c363740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec541b18b0 con 0x56162d932240 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39952/0 (socket says 172.21.15.86:39952) 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.976+0000 7fec4bfff640 1 -- 172.21.15.86:0/610034188 learned_addr learned my addr 172.21.15.86:0/610034188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 -- 172.21.15.86:0/610034188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d932240 msgr2=0x56162d92c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d932240 0x56162d92c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 -- 172.21.15.86:0/610034188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 msgr2=0x56162d92c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d92c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 -- 172.21.15.86:0/610034188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162d923090 con 0x56162d927060 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec4bfff640 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fec3c049410 tx=0x7fec3c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d989aaa65421f742 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec497fa640 1 -- 172.21.15.86:0/610034188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c050070 con 0x56162d927060 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec497fa640 1 -- 172.21.15.86:0/610034188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec3c031a50 con 0x56162d927060 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec497fa640 1 -- 172.21.15.86:0/610034188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0038c0 con 0x56162d927060 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec5c363740 1 -- 172.21.15.86:0/610034188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 msgr2=0x56162d93d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.977+0000 7fec5c363740 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fec3c049410 tx=0x7fec3c02a410 comp rx=0 tx=0).stop 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- 172.21.15.86:0/610034188 shutdown_connections 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d932240 0x56162d92c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d927980 0x56162d92c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- 172.21.15.86:0/610034188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927060 0x56162d93d920 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- 172.21.15.86:0/610034188 >> 172.21.15.86:0/610034188 conn(0x56162d9235f0 msgr2=0x56162d93d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- 172.21.15.86:0/610034188 shutdown_connections 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- 172.21.15.86:0/610034188 wait complete. 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- start start 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 Processor -- start 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d947080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d945550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec541b2650 con 0x56162d927980 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec541b1f80 con 0x56162d932240 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec541af1c0 con 0x56162d927060 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.978+0000 7fec5c363740 10 client.0 _ll_register_callbacks cb 0x7fec3c031700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39956/0 (socket says 172.21.15.86:39956) 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 -- 172.21.15.86:0/1422739769 learned_addr learned my addr 172.21.15.86:0/1422739769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d947080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d945550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 -- 172.21.15.86:0/1422739769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 msgr2=0x56162d947080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d947080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 -- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 msgr2=0x56162d945550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d945550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 -- 172.21.15.86:0/1422739769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162d945d50 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec537fe640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x56162d930750 tx=0x7fec4400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b412d487b03f089 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44022020 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fec4401d420 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c031a50 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec4402d050 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec5c363740 10 client.16161 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.979+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56162d9ad0b0 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec3c031a50 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fec440117b0 con 0x56162d927980 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 10 client.16161 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:56.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 10 client.16161 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 20 client.16161 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d9ad910 0x56162d9afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 4 client.16161 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "17bd525e", "release": true, "await": true}] 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "17bd525e", "release": true, "await": true}) -- 0x56162d8940a0 con 0x56162d9ad910 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d9ad910 0x56162d9afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec32ffd640 10 client.16161 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.980+0000 7fec4bfff640 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d9ad910 0x56162d9afd30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da264d38a8ab5de server_cookie=8a0183fd2afcf73c in_seq=0 out_seq=0 2024-09-05T06:43:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec32ffd640 1 -- 172.21.15.86:0/1422739769 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56162d8940a0 con 0x56162d9ad910 2024-09-05T06:43:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec32ffd640 10 client.16161 handle_command_reply: tid=0 2024-09-05T06:43:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 client.16161 shutdown 2024-09-05T06:43:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 msgr2=0x56162d9475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x56162d930750 tx=0x7fec4400bdb0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 shutdown_connections 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56162d9ad910 0x56162d9afd30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec32ffd640 0 client.16161 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162d932240 0x56162d945550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162d927980 0x56162d9475c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 --2- 172.21.15.86:0/1422739769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56162d927060 0x56162d947080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 >> 172.21.15.86:0/1422739769 conn(0x56162d9235f0 msgr2=0x56162d93da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 shutdown_connections 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.987+0000 7fec5c363740 1 -- 172.21.15.86:0/1422739769 wait complete. 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec5c363740 20 client.16161 trim_cache size 0 max 16384 2024-09-05T06:43:56.991 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 30, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 8, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "17bd525e": { 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 8, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:43:56.992 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 144.2, 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:56.993 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 msgr2=0x7fec400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 0x7fec400a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0cd5c0 tx=0x7fec4c06dd50 comp rx=0 tx=0).stop 2024-09-05T06:43:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 msgr2=0x7fec54101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00cce0 tx=0x7fec4c00a620 comp rx=0 tx=0).stop 2024-09-05T06:43:56.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 shutdown_connections 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fec400a2cf0 0x7fec400a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec54106ba0 0x7fec54101d30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec54100a70 0x7fec541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 --2- 172.21.15.86:0/867440650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec541000d0 0x7fec541012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.988+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 >> 172.21.15.86:0/867440650 conn(0x7fec54077210 msgr2=0x7fec540fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.989+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 shutdown_connections 2024-09-05T06:43:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:56.989+0000 7fec59f8a640 1 -- 172.21.15.86:0/867440650 wait complete. 2024-09-05T06:43:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:43:57.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 31781 2.45 MB/sec warmup 77 sec latency 25937.391 ms 2024-09-05T06:43:57.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:57 smithi086 ceph-mon[30943]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T06:43:57.713 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '17bd525e', total seconds elapsed: 29.5 2024-09-05T06:43:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:57 smithi096 ceph-mon[34468]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T06:43:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:57 smithi115 ceph-mon[34791]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 -- 172.21.15.86:0/2387182088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 msgr2=0x7f86c010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2387182088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c010bb40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009900 tx=0x7f86b002a480 comp rx=0 tx=0).stop 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 -- 172.21.15.86:0/2387182088 shutdown_connections 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2387182088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c010c160 0x7f86c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2387182088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c010bb40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2387182088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c0102d30 0x7f86c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 -- 172.21.15.86:0/2387182088 >> 172.21.15.86:0/2387182088 conn(0x7f86c00fc790 msgr2=0x7f86c00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 -- 172.21.15.86:0/2387182088 shutdown_connections 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.178+0000 7f86c7a65640 1 -- 172.21.15.86:0/2387182088 wait complete. 2024-09-05T06:43:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 Processor -- start 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 -- start start 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c010c160 0x7f86c019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111e10 con 0x7f86c0102d30 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111c90 con 0x7f86c010c160 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c7a65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111f90 con 0x7f86c0103650 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39974/0 (socket says 172.21.15.86:39974) 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 -- 172.21.15.86:0/2545624855 learned_addr learned my addr 172.21.15.86:0/2545624855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c6002640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c010c160 0x7f86c019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5000640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 -- 172.21.15.86:0/2545624855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 msgr2=0x7f86c0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 -- 172.21.15.86:0/2545624855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c010c160 msgr2=0x7f86c019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c010c160 0x7f86c019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.179+0000 7f86c5801640 1 -- 172.21.15.86:0/2545624855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c019d3c0 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86c5801640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00e9f0 tx=0x7f86bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86b67fc640 1 -- 172.21.15.86:0/2545624855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00cd80 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c019d6b0 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86b67fc640 1 -- 172.21.15.86:0/2545624855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86bc0054d0 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86b67fc640 1 -- 172.21.15.86:0/2545624855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc010490 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.180+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a4f90 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.182+0000 7f86b67fc640 1 -- 172.21.15.86:0/2545624855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86bc005020 con 0x7f86c0102d30 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 Processor -- start 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 -- start start 2024-09-05T06:43:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12b4a90 0x55dfe12b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dfe120b2e0 con 0x55dfe12a5ca0 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfe11b2ce0 con 0x55dfe12b4a90 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dfe12072a0 con 0x55dfe12a5380 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86b67fc640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 0x7f86ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c6002640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39984/0 (socket says 172.21.15.86:39984) 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c6002640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:58044/0 (socket says 172.21.15.86:58044) 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c5801640 1 -- 172.21.15.86:0/3143344456 learned_addr learned my addr 172.21.15.86:0/3143344456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c6002640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 0x7f86ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.183+0000 7f86c5000640 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12b4a90 0x55dfe12b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 -- 172.21.15.86:0/3143344456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12b4a90 msgr2=0x55dfe12b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12b4a90 0x55dfe12b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 -- 172.21.15.86:0/3143344456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 msgr2=0x55dfe12b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 -- 172.21.15.86:0/3143344456 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe1197440 con 0x55dfe12a5380 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86b67fc640 1 -- 172.21.15.86:0/2545624855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f86bc0cfb90 con 0x7f86c0102d30 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86b8011cc0 tx=0x7f86b800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cff83e44267917 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86c6002640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 0x7f86ac0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86b800fc90 tx=0x7f86b8018000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86ab7fe640 1 -- 172.21.15.86:0/3143344456 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8004b20 con 0x55dfe12a5380 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86ab7fe640 1 -- 172.21.15.86:0/3143344456 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86b8009ce0 con 0x55dfe12a5380 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.184+0000 7f86ab7fe640 1 -- 172.21.15.86:0/3143344456 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8005350 con 0x55dfe12a5380 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3143344456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 msgr2=0x55dfe12a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86b8011cc0 tx=0x7f86b800f790 comp rx=0 tx=0).stop 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3143344456 shutdown_connections 2024-09-05T06:43:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12b4a90 0x55dfe12b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3143344456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12a5760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3143344456 >> 172.21.15.86:0/3143344456 conn(0x55dfe12a1a30 msgr2=0x55dfe12a1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3143344456 shutdown_connections 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3143344456 wait complete. 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 -- start start 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.185+0000 7f86c9e3e740 1 Processor -- start 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12b4a90 0x55dfe12b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dfe12b7660 con 0x55dfe12a5380 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfe11c6050 con 0x55dfe12a5ca0 2024-09-05T06:43:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dfe12b77e0 con 0x55dfe12b4a90 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 10 client.0 _ll_register_callbacks cb 0x55dfe1206160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40000/0 (socket says 172.21.15.86:40000) 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 -- 172.21.15.86:0/1910827136 learned_addr learned my addr 172.21.15.86:0/1910827136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c6002640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12b4a90 0x55dfe12b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5000640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 -- 172.21.15.86:0/1910827136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12b4a90 msgr2=0x55dfe12b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12b4a90 0x55dfe12b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 -- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 msgr2=0x55dfe12b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 -- 172.21.15.86:0/1910827136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe0f7c3c0 con 0x55dfe12a5380 2024-09-05T06:43:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c5801640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0986a0 tx=0x7f86bc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39749e5bc8bf98d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc09b810 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f86bc0a44b0 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.186+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b8009ce0 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86bc09b630 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfe12c30d0 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86b8009ce0 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f86bc097a20 con 0x55dfe12a5380 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 20 client.16173 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe12c4600 0x55dfe12c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 4 client.16173 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe1218850 con 0x55dfe12c4600 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.187+0000 7f86c5000640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe12c4600 0x55dfe12c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c5000640 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe12c4600 0x55dfe12c6a10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1c624edc85427c server_cookie=d9c04c49d8a35b3d in_seq=0 out_seq=0 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86a8ff9640 10 client.16173 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86a8ff9640 1 -- 172.21.15.86:0/1910827136 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfe1218850 con 0x55dfe12c4600 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86a8ff9640 10 client.16173 handle_command_reply: tid=0 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 client.16173 shutdown 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 msgr2=0x55dfe12b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0986a0 tx=0x7f86bc009310 comp rx=0 tx=0).stop 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 shutdown_connections 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe12c4600 0x55dfe12c6a10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12b4a90 0x55dfe12b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86a8ff9640 0 client.16173 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/1910827136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12a5380 0x55dfe12b43e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.188+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 >> 172.21.15.86:0/1910827136 conn(0x55dfe12a1a30 msgr2=0x55dfe12a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.189+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 shutdown_connections 2024-09-05T06:43:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.189+0000 7f86c9e3e740 1 -- 172.21.15.86:0/1910827136 wait complete. 2024-09-05T06:43:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.189+0000 7f86c9e3e740 20 client.16173 trim_cache size 0 max 16384 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 Processor -- start 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 -- start start 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01acd90 con 0x55dfe12b4a90 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01ae930 con 0x55dfe12a5ca0 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01ad460 con 0x55dfe12a5380 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c6002640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c6002640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40002/0 (socket says 172.21.15.86:40002) 2024-09-05T06:43:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c6002640 1 -- 172.21.15.86:0/2673793089 learned_addr learned my addr 172.21.15.86:0/2673793089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5000640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 -- 172.21.15.86:0/2673793089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 msgr2=0x55dfe12cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 -- 172.21.15.86:0/2673793089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 msgr2=0x55dfe12cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c5801640 1 -- 172.21.15.86:0/2673793089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe12a1860 con 0x55dfe12a5380 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.209+0000 7f86c6002640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.210+0000 7f86c5801640 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12c0900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86bc09bc10 tx=0x7f86bc0ccfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9788ec52debd590 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.210+0000 7f86aaffd640 1 -- 172.21.15.86:0/2673793089 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0b8030 con 0x55dfe12a5380 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.210+0000 7f86aaffd640 1 -- 172.21.15.86:0/2673793089 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86bc0d3030 con 0x55dfe12a5380 2024-09-05T06:43:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.210+0000 7f86aaffd640 1 -- 172.21.15.86:0/2673793089 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0a4700 con 0x55dfe12a5380 2024-09-05T06:43:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- 172.21.15.86:0/2673793089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 msgr2=0x55dfe12c0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12c0900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86bc09bc10 tx=0x7f86bc0ccfb0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- 172.21.15.86:0/2673793089 shutdown_connections 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5ca0 0x55dfe12cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/2673793089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5380 0x55dfe12c0900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- 172.21.15.86:0/2673793089 >> 172.21.15.86:0/2673793089 conn(0x55dfe12a1a30 msgr2=0x55dfe12acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- 172.21.15.86:0/2673793089 shutdown_connections 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- 172.21.15.86:0/2673793089 wait complete. 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 -- start start 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.211+0000 7f86c9e3e740 1 Processor -- start 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01ae200 con 0x55dfe12b4a90 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01adb30 con 0x55dfe12a5380 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01aad70 con 0x55dfe12a5ca0 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c9e3e740 10 client.0 _ll_register_callbacks cb 0x7f86bc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:56948/0 (socket says 172.21.15.86:56948) 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 -- 172.21.15.86:0/3165985463 learned_addr learned my addr 172.21.15.86:0/3165985463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5000640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c6002640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 -- 172.21.15.86:0/3165985463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5ca0 msgr2=0x55dfe12b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 -- 172.21.15.86:0/3165985463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 msgr2=0x55dfe12b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.212+0000 7f86c5801640 1 -- 172.21.15.86:0/3165985463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe132e500 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c5801640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0a4410 tx=0x7f86bc098570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f15bf2e7d5540e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0a8020 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f86bc0a44b0 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86bc0d3030 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0b8030 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 10 client.25562 fetch_fsmap learned FSMap version 30 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfe132f7f0 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86bc005310 con 0x55dfe12a5380 2024-09-05T06:43:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f86bc09ab90 con 0x55dfe12a5380 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 10 client.25562 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 20 client.25562 populate_metadata read hostname 'smithi086' 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe132fc50 0x55dfe1332070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 4 client.25562 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.213+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f86bc0d3030 con 0x55dfe132fc50 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c5000640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe132fc50 0x55dfe1332070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86a8ff9640 10 client.25562 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c5000640 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe132fc50 0x55dfe1332070 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4e9c6988c3e3be server_cookie=984773279496e7a9 in_seq=0 out_seq=0 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86a8ff9640 1 -- 172.21.15.86:0/3165985463 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f86bc0d3030 con 0x55dfe132fc50 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86a8ff9640 10 client.25562 handle_command_reply: tid=0 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 client.25562 shutdown 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 msgr2=0x55dfe12b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0a4410 tx=0x7f86bc098570 comp rx=0 tx=0).stop 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 shutdown_connections 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dfe132fc50 0x55dfe1332070 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfe12b4a90 0x55dfe12b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dfe12a5ca0 0x55dfe12b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86a8ff9640 0 client.25562 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.214+0000 7f86c9e3e740 1 --2- 172.21.15.86:0/3165985463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfe12a5380 0x55dfe12b8a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.215+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 >> 172.21.15.86:0/3165985463 conn(0x55dfe12a1a30 msgr2=0x55dfe12a0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.215+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 shutdown_connections 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.215+0000 7f86c9e3e740 1 -- 172.21.15.86:0/3165985463 wait complete. 2024-09-05T06:43:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.215+0000 7f86c9e3e740 20 client.25562 trim_cache size 0 max 16384 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "fce0a3cf-24ae-4662-b9d1-c2c9fb596d0e": { 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "fce0a3cf-24ae-4662-b9d1-c2c9fb596d0e", 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:43:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 msgr2=0x7f86ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 0x7f86ac0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86b800fc90 tx=0x7f86b8018000 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 msgr2=0x7f86c0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00e9f0 tx=0x7f86bc00edf0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 shutdown_connections 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86ac0a2cf0 0x7f86ac0a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86c010c160 0x7f86c019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c0103650 0x7f86c0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 --2- 172.21.15.86:0/2545624855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c0102d30 0x7f86c0198410 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:43:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 >> 172.21.15.86:0/2545624855 conn(0x7f86c00fc790 msgr2=0x7f86c00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:43:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 shutdown_connections 2024-09-05T06:43:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:43:58.216+0000 7f86c7a65640 1 -- 172.21.15.86:0/2545624855 wait complete. 2024-09-05T06:43:58.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 32276 2.47 MB/sec warmup 78 sec latency 39.603 ms 2024-09-05T06:43:59.016 INFO:tasks.cephfs.filesystem:scrub status for tag:fce0a3cf-24ae-4662-b9d1-c2c9fb596d0e - {'path': '/', 'tag': 'fce0a3cf-24ae-4662-b9d1-c2c9fb596d0e', 'options': 'recursive,force'} 2024-09-05T06:43:59.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 32586 2.44 MB/sec warmup 79 sec latency 11.331 ms 2024-09-05T06:43:59.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:43:59 smithi115 ceph-mon[34791]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T06:43:59.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:43:59 smithi086 ceph-mon[30943]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T06:43:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:43:59 smithi096 ceph-mon[34468]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T06:44:00.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 33144 2.44 MB/sec warmup 80 sec latency 13.428 ms 2024-09-05T06:44:01.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 33793 2.45 MB/sec warmup 81 sec latency 22.844 ms 2024-09-05T06:44:01.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:01 smithi086 ceph-mon[30943]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 497 KiB/s wr, 82 op/s 2024-09-05T06:44:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:01 smithi096 ceph-mon[34468]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 497 KiB/s wr, 82 op/s 2024-09-05T06:44:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:01 smithi115 ceph-mon[34791]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 497 KiB/s wr, 82 op/s 2024-09-05T06:44:02.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 34420 2.44 MB/sec warmup 82 sec latency 14.946 ms 2024-09-05T06:44:03.507 INFO:tasks.workunit.client.0.smithi086.stdout: 1 35248 2.50 MB/sec warmup 83 sec latency 13.577 ms 2024-09-05T06:44:03.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:03 smithi086 ceph-mon[30943]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:03.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:03 smithi096 ceph-mon[34468]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:03.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:03 smithi115 ceph-mon[34791]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:04.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 35718 2.52 MB/sec warmup 84 sec latency 13.456 ms 2024-09-05T06:44:05.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 35979 2.50 MB/sec warmup 85 sec latency 12.767 ms 2024-09-05T06:44:05.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:05 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:05 smithi086 ceph-mon[30943]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:05 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:05 smithi096 ceph-mon[34468]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:05 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:05 smithi115 ceph-mon[34791]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 495 KiB/s wr, 77 op/s 2024-09-05T06:44:06.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 36449 2.48 MB/sec warmup 86 sec latency 13.730 ms 2024-09-05T06:44:06.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:06.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:06.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:06.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:07.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 37069 2.50 MB/sec warmup 87 sec latency 14.502 ms 2024-09-05T06:44:07.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:07 smithi086 ceph-mon[30943]: pgmap v89: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T06:44:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:07 smithi096 ceph-mon[34468]: pgmap v89: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T06:44:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:07 smithi115 ceph-mon[34791]: pgmap v89: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T06:44:08.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 37758 2.48 MB/sec warmup 88 sec latency 30.405 ms 2024-09-05T06:44:09.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 38436 2.53 MB/sec warmup 89 sec latency 12.212 ms 2024-09-05T06:44:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:09 smithi086 ceph-mon[30943]: pgmap v90: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:44:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:09 smithi096 ceph-mon[34468]: pgmap v90: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:44:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:09 smithi115 ceph-mon[34791]: pgmap v90: 129 pgs: 129 active+clean; 49 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-09-05T06:44:10.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39037 2.55 MB/sec warmup 90 sec latency 13.259 ms 2024-09-05T06:44:11.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39434 2.54 MB/sec warmup 91 sec latency 14.899 ms 2024-09-05T06:44:11.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:11 smithi086 ceph-mon[30943]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-05T06:44:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:11 smithi096 ceph-mon[34468]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-05T06:44:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:11 smithi115 ceph-mon[34791]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-05T06:44:12.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39838 2.52 MB/sec warmup 92 sec latency 11.195 ms 2024-09-05T06:44:13.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 40459 2.54 MB/sec warmup 93 sec latency 13.524 ms 2024-09-05T06:44:13.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:13 smithi086 ceph-mon[30943]: pgmap v92: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:13 smithi096 ceph-mon[34468]: pgmap v92: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:13 smithi115 ceph-mon[34791]: pgmap v92: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:14.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 41120 2.52 MB/sec warmup 94 sec latency 37.423 ms 2024-09-05T06:44:15.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 41675 2.52 MB/sec warmup 95 sec latency 13.317 ms 2024-09-05T06:44:15.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:15 smithi086 ceph-mon[30943]: pgmap v93: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:15 smithi096 ceph-mon[34468]: pgmap v93: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:15 smithi115 ceph-mon[34791]: pgmap v93: 129 pgs: 129 active+clean; 50 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:44:16.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 42387 2.56 MB/sec warmup 96 sec latency 14.293 ms 2024-09-05T06:44:17.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 42896 2.58 MB/sec warmup 97 sec latency 11.843 ms 2024-09-05T06:44:17.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:17 smithi086 ceph-mon[30943]: pgmap v94: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:44:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:17 smithi096 ceph-mon[34468]: pgmap v94: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:44:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:17 smithi115 ceph-mon[34791]: pgmap v94: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:44:18.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43203 2.56 MB/sec warmup 98 sec latency 11.687 ms 2024-09-05T06:44:19.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43772 2.55 MB/sec warmup 99 sec latency 14.689 ms 2024-09-05T06:44:19.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:19 smithi086 ceph-mon[30943]: pgmap v95: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:44:19.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:19 smithi096 ceph-mon[34468]: pgmap v95: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:44:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:19 smithi115 ceph-mon[34791]: pgmap v95: 129 pgs: 129 active+clean; 52 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:44:20.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 44443 2.56 MB/sec warmup 100 sec latency 14.520 ms 2024-09-05T06:44:21.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 45066 2.56 MB/sec warmup 101 sec latency 14.041 ms 2024-09-05T06:44:21.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:21 smithi086 ceph-mon[30943]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-05T06:44:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:21 smithi096 ceph-mon[34468]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-05T06:44:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:21 smithi115 ceph-mon[34791]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-05T06:44:22.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 45887 2.60 MB/sec warmup 102 sec latency 13.749 ms 2024-09-05T06:44:23.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46368 2.61 MB/sec warmup 103 sec latency 12.423 ms 2024-09-05T06:44:23.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:23 smithi086 ceph-mon[30943]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:23 smithi096 ceph-mon[34468]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:23 smithi115 ceph-mon[34791]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:24.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46646 2.59 MB/sec warmup 104 sec latency 12.591 ms 2024-09-05T06:44:25.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 47137 2.58 MB/sec warmup 105 sec latency 13.157 ms 2024-09-05T06:44:25.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:25 smithi086 ceph-mon[30943]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:25 smithi096 ceph-mon[34468]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:25 smithi115 ceph-mon[34791]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-05T06:44:26.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 47794 2.59 MB/sec warmup 106 sec latency 18.928 ms 2024-09-05T06:44:27.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 48490 2.58 MB/sec warmup 107 sec latency 14.160 ms 2024-09-05T06:44:27.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:27 smithi086 ceph-mon[30943]: pgmap v99: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-09-05T06:44:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:27 smithi096 ceph-mon[34468]: pgmap v99: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-09-05T06:44:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:27 smithi115 ceph-mon[34791]: pgmap v99: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-09-05T06:44:28.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 49075 2.61 MB/sec warmup 108 sec latency 98.537 ms 2024-09-05T06:44:28.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:28 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:44:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:28 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:44:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:28 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:44:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:28 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:44:28.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:28 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:44:28.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:28 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:44:28.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:28 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:44:28.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:28 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:44:28.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:28 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:44:28.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:28 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:44:28.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:28 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:44:28.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:28 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:44:29.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:44:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:29.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 49669 2.63 MB/sec warmup 109 sec latency 14.846 ms 2024-09-05T06:44:29.693 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:29 smithi086 ceph-mon[30943]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:44:29.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 -- 172.21.15.86:0/1670910706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978100aa0 msgr2=0x7fd97810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/1670910706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978100aa0 0x7fd97810ea20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd970009be0 tx=0x7fd97002a6c0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 -- 172.21.15.86:0/1670910706 shutdown_connections 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/1670910706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978100aa0 0x7fd97810ea20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/1670910706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd978100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/1670910706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978106b80 0x7fd978106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 -- 172.21.15.86:0/1670910706 >> 172.21.15.86:0/1670910706 conn(0x7fd978077230 msgr2=0x7fd9780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 -- 172.21.15.86:0/1670910706 shutdown_connections 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 -- 172.21.15.86:0/1670910706 wait complete. 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.784+0000 7fd97d5a0640 1 Processor -- start 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 -- start start 2024-09-05T06:44:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd9781012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 0x7fd978101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 0x7fd978101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd978111f00 con 0x7fd978106b80 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd978111d80 con 0x7fd978100100 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd97d5a0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd978112080 con 0x7fd978100aa0 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 0x7fd978101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd9781012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 0x7fd978101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33246/0 (socket says 172.21.15.86:33246) 2024-09-05T06:44:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd9777fe640 1 -- 172.21.15.86:0/3983689111 learned_addr learned my addr 172.21.15.86:0/3983689111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd9767fc640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 0x7fd978101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 -- 172.21.15.86:0/3983689111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 msgr2=0x7fd978101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 0x7fd978101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 -- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 msgr2=0x7fd978101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 0x7fd978101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd976ffd640 1 -- 172.21.15.86:0/3983689111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9781a6880 con 0x7fd978100100 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.785+0000 7fd9767fc640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 0x7fd978101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd976ffd640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd9781012e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd968005fd0 tx=0x7fd96800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd95ffff640 1 -- 172.21.15.86:0/3983689111 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800cd00 con 0x7fd978100100 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9781a6b70 con 0x7fd978100100 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd95ffff640 1 -- 172.21.15.86:0/3983689111 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9680035f0 con 0x7fd978100100 2024-09-05T06:44:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd95ffff640 1 -- 172.21.15.86:0/3983689111 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968010670 con 0x7fd978100100 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.786+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781a70b0 con 0x7fd978100100 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.788+0000 7fd95ffff640 1 -- 172.21.15.86:0/3983689111 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9680037a0 con 0x7fd978100100 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.788+0000 7fd97f979740 1 Processor -- start 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 -- start start 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd95ffff640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 0x7fd9600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589d361f2e0 con 0x5589d36c4680 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589d35c6ce0 con 0x5589d36b94a0 2024-09-05T06:44:29.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd97f979740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5589d361b2a0 con 0x5589d36b9dc0 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd976ffd640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 0x7fd9600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33248/0 (socket says 172.21.15.86:33248) 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9777fe640 1 -- 172.21.15.86:0/1451010846 learned_addr learned my addr 172.21.15.86:0/1451010846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd95ffff640 1 -- 172.21.15.86:0/3983689111 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd9680cfa40 con 0x7fd978100100 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 -- 172.21.15.86:0/1451010846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 msgr2=0x5589d36b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 -- 172.21.15.86:0/1451010846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 msgr2=0x5589d36c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9767fc640 1 -- 172.21.15.86:0/1451010846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d35ab440 con 0x5589d36b9dc0 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.789+0000 7fd9777fe640 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd976ffd640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 0x7fd9600a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9680a4260 tx=0x7fd9680c66b0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd9767fc640 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c4140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009fa0 tx=0x7fd96c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c8d7dab4838994 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd95d7fa640 1 -- 172.21.15.86:0/1451010846 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004030 con 0x5589d36b9dc0 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd95d7fa640 1 -- 172.21.15.86:0/1451010846 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd96c0054d0 con 0x5589d36b9dc0 2024-09-05T06:44:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd95d7fa640 1 -- 172.21.15.86:0/1451010846 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004c70 con 0x5589d36b9dc0 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd97f979740 1 -- 172.21.15.86:0/1451010846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 msgr2=0x5589d36c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd97f979740 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c4140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009fa0 tx=0x7fd96c0118d0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd97f979740 1 -- 172.21.15.86:0/1451010846 shutdown_connections 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.790+0000 7fd97f979740 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c4140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 --2- 172.21.15.86:0/1451010846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- 172.21.15.86:0/1451010846 >> 172.21.15.86:0/1451010846 conn(0x5589d36b5a30 msgr2=0x5589d36b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- 172.21.15.86:0/1451010846 shutdown_connections 2024-09-05T06:44:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- 172.21.15.86:0/1451010846 wait complete. 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- start start 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 Processor -- start 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b94a0 0x5589d36c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 0x5589d36c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589d36c7250 con 0x5589d36c4680 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589d35da050 con 0x5589d36b9dc0 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5589d36c73d0 con 0x5589d36b94a0 2024-09-05T06:44:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd97f979740 10 client.0 _ll_register_callbacks cb 0x5589d361a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33250/0 (socket says 172.21.15.86:33250) 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd9777fe640 1 -- 172.21.15.86:0/2623888357 learned_addr learned my addr 172.21.15.86:0/2623888357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.791+0000 7fd9767fc640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 0x5589d36c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b94a0 0x5589d36c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 -- 172.21.15.86:0/2623888357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b94a0 msgr2=0x5589d36c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b94a0 0x5589d36c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 -- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 msgr2=0x5589d36c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 0x5589d36c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 -- 172.21.15.86:0/2623888357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d33903c0 con 0x5589d36c4680 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9767fc640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 0x5589d36c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd9777fe640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd97002abe0 tx=0x7fd97002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990f0a2df090e3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd95aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970036040 con 0x5589d36c4680 2024-09-05T06:44:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd970003d80 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96c0054d0 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd970052050 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd97f979740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589d36d2c90 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.792+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd96c0054d0 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fd97003c030 con 0x5589d36c4680 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 20 client.16191 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d36d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 4 client.16191 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589d362c850 con 0x5589d36d4150 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd9767fc640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d36d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd95aff5640 10 client.16191 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.793+0000 7fd9767fc640 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d36d6540 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf811cfc6d56420 server_cookie=7cbe9d9919c0e313 in_seq=0 out_seq=0 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd95aff5640 1 -- 172.21.15.86:0/2623888357 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589d362c850 con 0x5589d36d4150 2024-09-05T06:44:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd95aff5640 10 client.16191 handle_command_reply: tid=0 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 client.16191 shutdown 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 msgr2=0x5589d36c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd97002abe0 tx=0x7fd97002a6c0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 shutdown_connections 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d36d6540 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c1870 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b9dc0 0x5589d36c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd95aff5640 0 client.16191 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:29.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 --2- 172.21.15.86:0/2623888357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b94a0 0x5589d36c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 >> 172.21.15.86:0/2623888357 conn(0x5589d36b5a30 msgr2=0x5589d36bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 shutdown_connections 2024-09-05T06:44:29.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 1 -- 172.21.15.86:0/2623888357 wait complete. 2024-09-05T06:44:29.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.794+0000 7fd97f979740 20 client.16191 trim_cache size 0 max 16384 2024-09-05T06:44:29.819 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:29 smithi096 ceph-mon[34468]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:44:29.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 Processor -- start 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 -- start start 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781b1300 con 0x5589d36c4680 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781b2ea0 con 0x5589d36b94a0 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd97f979740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781b19d0 con 0x5589d36b9dc0 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd9767fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57002/0 (socket says 172.21.15.86:57002) 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.815+0000 7fd9777fe640 1 -- 172.21.15.86:0/1371262630 learned_addr learned my addr 172.21.15.86:0/1371262630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 -- 172.21.15.86:0/1371262630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 msgr2=0x5589d36bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36bfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 -- 172.21.15.86:0/1371262630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 msgr2=0x5589d36dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36dbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 -- 172.21.15.86:0/1371262630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d36c0280 con 0x5589d36b94a0 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd9777fe640 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd970006010 tx=0x7fd97002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4275f1ed94814 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd95cff9640 1 -- 172.21.15.86:0/1371262630 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970036030 con 0x5589d36b94a0 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd95cff9640 1 -- 172.21.15.86:0/1371262630 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd970037040 con 0x5589d36b94a0 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.816+0000 7fd95cff9640 1 -- 172.21.15.86:0/1371262630 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970003de0 con 0x5589d36b94a0 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- 172.21.15.86:0/1371262630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 msgr2=0x5589d36d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd970006010 tx=0x7fd97002a6c0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- 172.21.15.86:0/1371262630 shutdown_connections 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 --2- 172.21.15.86:0/1371262630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36d04e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- 172.21.15.86:0/1371262630 >> 172.21.15.86:0/1371262630 conn(0x5589d36b5a30 msgr2=0x5589d36c3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- 172.21.15.86:0/1371262630 shutdown_connections 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- 172.21.15.86:0/1371262630 wait complete. 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 -- start start 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.817+0000 7fd97f979740 1 Processor -- start 2024-09-05T06:44:29.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36c8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781b2770 con 0x5589d36c4680 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781b20a0 con 0x5589d36b94a0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781af2e0 con 0x5589d36b9dc0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd97f979740 10 client.0 _ll_register_callbacks cb 0x7fd968005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33262/0 (socket says 172.21.15.86:33262) 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9777fe640 1 -- 172.21.15.86:0/3290566542 learned_addr learned my addr 172.21.15.86:0/3290566542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 -- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 msgr2=0x5589d36c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 -- 172.21.15.86:0/3290566542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 msgr2=0x5589d36c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 -- 172.21.15.86:0/3290566542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d36d68b0 con 0x5589d36b9dc0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9777fe640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd9767fc640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c8b90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd96c01b040 tx=0x7fd96c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11e60cb9e52cfc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.818+0000 7fd95aff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c02d030 con 0x5589d36b9dc0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd96c016040 con 0x5589d36b9dc0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd970037040 con 0x5589d36b9dc0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd96c03a050 con 0x5589d36b9dc0 2024-09-05T06:44:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 10 client.25570 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589d373f1a0 con 0x5589d36b9dc0 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd970037040 con 0x5589d36b9dc0 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fd96c019a80 con 0x5589d36b9dc0 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 10 client.25570 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 20 client.25570 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d3741790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 4 client.25570 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589d3626d60 con 0x5589d36d4150 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.819+0000 7fd9777fe640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d3741790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd95aff5640 10 client.25570 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd9777fe640 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d3741790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57de22d0123c313 server_cookie=d83e6063ed5e7735 in_seq=0 out_seq=0 2024-09-05T06:44:29.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd95aff5640 1 -- 172.21.15.86:0/3290566542 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5589d3626d60 con 0x5589d36d4150 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd95aff5640 10 client.25570 handle_command_reply: tid=0 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 client.25570 shutdown 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 msgr2=0x5589d36c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c8b90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd96c01b040 tx=0x7fd96c0096c0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 shutdown_connections 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5589d36d4150 0x5589d3741790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589d36c4680 0x5589d36c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5589d36b9dc0 0x5589d36c8b90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd95aff5640 0 client.25570 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 --2- 172.21.15.86:0/3290566542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589d36b94a0 0x5589d36c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 >> 172.21.15.86:0/3290566542 conn(0x5589d36b5a30 msgr2=0x5589d36c3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 shutdown_connections 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 1 -- 172.21.15.86:0/3290566542 wait complete. 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.820+0000 7fd97f979740 20 client.25570 trim_cache size 0 max 16384 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:44:29.833 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:44:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:29 smithi115 ceph-mon[34791]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:44:29.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 msgr2=0x7fd9600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 0x7fd9600a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9680a4260 tx=0x7fd9680c66b0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 msgr2=0x7fd9781012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd9781012e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd968005fd0 tx=0x7fd96800ee50 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 shutdown_connections 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9600a2cf0 0x7fd9600a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd978106b80 0x7fd978101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd978100aa0 0x7fd978101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 --2- 172.21.15.86:0/3983689111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd978100100 0x7fd9781012e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 >> 172.21.15.86:0/3983689111 conn(0x7fd978077230 msgr2=0x7fd978104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 shutdown_connections 2024-09-05T06:44:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:29.821+0000 7fd97d5a0640 1 -- 172.21.15.86:0/3983689111 wait complete. 2024-09-05T06:44:30.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50057 2.62 MB/sec warmup 110 sec latency 20.070 ms 2024-09-05T06:44:30.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:44:30.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:44:30.882 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 -- 172.21.15.86:0/355769337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b74102d30 msgr2=0x7f0b74103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 --2- 172.21.15.86:0/355769337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b74102d30 0x7f0b74103110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009890 tx=0x7f0b6402a380 comp rx=0 tx=0).stop 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 -- 172.21.15.86:0/355769337 shutdown_connections 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 --2- 172.21.15.86:0/355769337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b7410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 --2- 172.21.15.86:0/355769337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b741036e0 0x7f0b74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 --2- 172.21.15.86:0/355769337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b74102d30 0x7f0b74103110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 -- 172.21.15.86:0/355769337 >> 172.21.15.86:0/355769337 conn(0x7f0b740fc7d0 msgr2=0x7f0b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 -- 172.21.15.86:0/355769337 shutdown_connections 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.374+0000 7f0b7948d640 1 -- 172.21.15.86:0/355769337 wait complete. 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 Processor -- start 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 -- start start 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b74101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b74111f50 con 0x7f0b74102d30 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b74111dd0 con 0x7f0b741036e0 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.375+0000 7f0b7948d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741120d0 con 0x7f0b741082a0 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b727fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33274/0 (socket says 172.21.15.86:33274) 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b727fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57018/0 (socket says 172.21.15.86:57018) 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b72ffd640 1 -- 172.21.15.86:0/3721528490 learned_addr learned my addr 172.21.15.86:0/3721528490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b74101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 -- 172.21.15.86:0/3721528490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 msgr2=0x7f0b74100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 -- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 msgr2=0x7f0b74100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 -- 172.21.15.86:0/3721528490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b741a69b0 con 0x7f0b741082a0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b727fc640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b72ffd640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b737fe640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b74101410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009fd0 tx=0x7f0b6c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00ce20 con 0x7f0b741082a0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b6c005020 con 0x7f0b741082a0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c010690 con 0x7f0b741082a0 2024-09-05T06:44:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b741a6ca0 con 0x7f0b741082a0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.376+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b741a7180 con 0x7f0b741082a0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.378+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7406a850 con 0x7f0b741082a0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.379+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0b6c0051d0 con 0x7f0b741082a0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.379+0000 7f0b5bfff640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 0x7f0b5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.379+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0b6c0cfdf0 con 0x7f0b741082a0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.379+0000 7f0b72ffd640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 0x7f0b5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.380+0000 7f0b72ffd640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 0x7f0b5c0a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b640057c0 tx=0x7f0b64009490 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:31.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.385+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b6c09c630 con 0x7f0b741082a0 2024-09-05T06:44:31.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50367 2.60 MB/sec warmup 111 sec latency 28.966 ms 2024-09-05T06:44:31.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.627+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7410a230 con 0x7f0b741082a0 2024-09-05T06:44:31.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:31 smithi086 ceph-mon[30943]: pgmap v101: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-05T06:44:31.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.627+0000 7f0b5bfff640 1 -- 172.21.15.86:0/3721528490 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7f0b6c09bf30 con 0x7f0b741082a0 2024-09-05T06:44:31.628 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:44:31.628 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:44:31.628 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:44:31.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 msgr2=0x7f0b5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:31.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 0x7f0b5c0a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b640057c0 tx=0x7f0b64009490 comp rx=0 tx=0).stop 2024-09-05T06:44:31.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 msgr2=0x7f0b74101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:31.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b74101410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009fd0 tx=0x7f0b6c00ef10 comp rx=0 tx=0).stop 2024-09-05T06:44:31.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 shutdown_connections 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b5c0a2d70 0x7f0b5c0a5230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b741082a0 0x7f0b74101410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b741036e0 0x7f0b74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 --2- 172.21.15.86:0/3721528490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b74102d30 0x7f0b74100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 >> 172.21.15.86:0/3721528490 conn(0x7f0b740fc7d0 msgr2=0x7f0b74105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 shutdown_connections 2024-09-05T06:44:31.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:31.630+0000 7f0b7948d640 1 -- 172.21.15.86:0/3721528490 wait complete. 2024-09-05T06:44:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:31 smithi096 ceph-mon[34468]: pgmap v101: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-05T06:44:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:31 smithi115 ceph-mon[34791]: pgmap v101: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-05T06:44:32.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:44:32.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50922 2.60 MB/sec warmup 112 sec latency 17.523 ms 2024-09-05T06:44:32.735 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:32.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:32 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3721528490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:32.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:32 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3721528490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:32.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:32 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3721528490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:33.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.228+0000 7f6961046640 1 -- 172.21.15.86:0/1643311474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c102d30 msgr2=0x7f695c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.228+0000 7f6961046640 1 --2- 172.21.15.86:0/1643311474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c102d30 0x7f695c103110 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f694c009930 tx=0x7f694c02a420 comp rx=0 tx=0).stop 2024-09-05T06:44:33.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 -- 172.21.15.86:0/1643311474 shutdown_connections 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 --2- 172.21.15.86:0/1643311474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c1082a0 0x7f695c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 --2- 172.21.15.86:0/1643311474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 --2- 172.21.15.86:0/1643311474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c102d30 0x7f695c103110 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 -- 172.21.15.86:0/1643311474 >> 172.21.15.86:0/1643311474 conn(0x7f695c0fc7d0 msgr2=0x7f695c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 -- 172.21.15.86:0/1643311474 shutdown_connections 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 -- 172.21.15.86:0/1643311474 wait complete. 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.229+0000 7f6961046640 1 Processor -- start 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 -- start start 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c102d30 0x7f695c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c111f50 con 0x7f695c1082a0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c111dd0 con 0x7f695c1036e0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f6961046640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1120d0 con 0x7f695c102d30 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55788/0 (socket says 172.21.15.86:55788) 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695b59e640 1 -- 172.21.15.86:0/3181129997 learned_addr learned my addr 172.21.15.86:0/3181129997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c102d30 0x7f695c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 -- 172.21.15.86:0/3181129997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c102d30 msgr2=0x7f695c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c102d30 0x7f695c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 -- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 msgr2=0x7f695c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695a59c640 1 -- 172.21.15.86:0/3181129997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c19d4c0 con 0x7f695c1036e0 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.230+0000 7f695b59e640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:33.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f695a59c640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c198a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f695c0697d0 tx=0x7f695000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f6943fff640 1 -- 172.21.15.86:0/3181129997 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695000cc40 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c19d7b0 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f6943fff640 1 -- 172.21.15.86:0/3181129997 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695000cde0 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f6943fff640 1 -- 172.21.15.86:0/3181129997 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950018760 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.231+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1a5090 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.233+0000 7f6943fff640 1 -- 172.21.15.86:0/3181129997 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6950010430 con 0x7f695c1036e0 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 Processor -- start 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 -- start start 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f6943fff640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 0x7f69440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f1015d0 0x55af6f101a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f10e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af6f0672f0 con 0x55af6f10c370 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af6f00ecf0 con 0x55af6f102b40 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f696341f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af6f0632b0 con 0x55af6f1015d0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52572/0 (socket says 172.21.15.86:52572) 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695b59e640 1 -- 172.21.15.86:0/4101622903 learned_addr learned my addr 172.21.15.86:0/4101622903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 0x7f69440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695b59e640 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f10e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.234+0000 7f695a59c640 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f1015d0 0x55af6f101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f6943fff640 1 -- 172.21.15.86:0/3181129997 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f69500cfb00 con 0x7f695c1036e0 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 -- 172.21.15.86:0/4101622903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f1015d0 msgr2=0x55af6f101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f1015d0 0x55af6f101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 -- 172.21.15.86:0/4101622903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 msgr2=0x55af6f10e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f10e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 -- 172.21.15.86:0/4101622903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6eff33a0 con 0x55af6f102b40 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695b59e640 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6954009370 tx=0x7f6954009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fddfc389e3c1fa70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 0x7f69440a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f69417fa640 1 -- 172.21.15.86:0/4101622903 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954003b70 con 0x55af6f102b40 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f69417fa640 1 -- 172.21.15.86:0/4101622903 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6954003d10 con 0x55af6f102b40 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.235+0000 7f69417fa640 1 -- 172.21.15.86:0/4101622903 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954004530 con 0x55af6f102b40 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- 172.21.15.86:0/4101622903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 msgr2=0x55af6f101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6954009370 tx=0x7f6954009260 comp rx=0 tx=0).stop 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- 172.21.15.86:0/4101622903 shutdown_connections 2024-09-05T06:44:33.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f1015d0 0x55af6f101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 --2- 172.21.15.86:0/4101622903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f101090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- 172.21.15.86:0/4101622903 >> 172.21.15.86:0/4101622903 conn(0x55af6f0ca310 msgr2=0x55af6f0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- 172.21.15.86:0/4101622903 shutdown_connections 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- 172.21.15.86:0/4101622903 wait complete. 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.236+0000 7f696341f740 1 -- start start 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 Processor -- start 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 0x55af6f108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af6f10f010 con 0x55af6f102b40 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af6f022060 con 0x55af6f1015d0 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af6f10f190 con 0x55af6f10c370 2024-09-05T06:44:33.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f696341f740 10 client.0 _ll_register_callbacks cb 0x55af6ed758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 0x55af6f108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52580/0 (socket says 172.21.15.86:52580) 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695ad9d640 1 -- 172.21.15.86:0/1169487120 learned_addr learned my addr 172.21.15.86:0/1169487120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 -- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 msgr2=0x55af6f109570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f109570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 -- 172.21.15.86:0/1169487120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 msgr2=0x55af6f10bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 -- 172.21.15.86:0/1169487120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6edd83c0 con 0x55af6f102b40 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695ad9d640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f695a59c640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 0x55af6f108fd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f69500cce10 tx=0x7f69500b4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54fbc643676a9b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f693eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.237+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950026040 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f69500b51a0 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6954003d10 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69500a4850 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 10 client.16215 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af6f11aa60 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6954003d10 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6950010720 con 0x55af6f102b40 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 10 client.16215 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 10 client.16215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 20 client.16215 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f11e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 4 client.16215 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:33.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af6f074860 con 0x55af6f11bec0 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.238+0000 7f695ad9d640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f11e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.239+0000 7f695ad9d640 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f11e2d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e37189c7d856a59a server_cookie=e259f1cf74c08d26 in_seq=0 out_seq=0 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.239+0000 7f693eff5640 10 client.16215 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.239+0000 7f693eff5640 1 -- 172.21.15.86:0/1169487120 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af6f074860 con 0x55af6f11bec0 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.239+0000 7f693eff5640 10 client.16215 handle_command_reply: tid=0 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.239+0000 7f696341f740 1 client.16215 shutdown 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 msgr2=0x55af6f108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 0x55af6f108fd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f69500cce10 tx=0x7f69500b4ea0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 shutdown_connections 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f11e2d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f693eff5640 0 client.16215 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:44:33.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f102b40 0x55af6f108fd0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 --2- 172.21.15.86:0/1169487120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 >> 172.21.15.86:0/1169487120 conn(0x55af6f0ca310 msgr2=0x55af6f103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 shutdown_connections 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 1 -- 172.21.15.86:0/1169487120 wait complete. 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.240+0000 7f696341f740 20 client.16215 trim_cache size 0 max 16384 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 Processor -- start 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 -- start start 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f102b40 0x55af6f10af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1acf20 con 0x55af6f10c370 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1aeac0 con 0x55af6f1015d0 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f696341f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ad5f0 con 0x55af6f102b40 2024-09-05T06:44:33.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55820/0 (socket says 172.21.15.86:55820) 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 -- 172.21.15.86:0/3434536835 learned_addr learned my addr 172.21.15.86:0/3434536835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695a59c640 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f102b40 0x55af6f10af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 -- 172.21.15.86:0/3434536835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f102b40 msgr2=0x55af6f10af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f102b40 0x55af6f10af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 -- 172.21.15.86:0/3434536835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 msgr2=0x55af6f108890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f108890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 -- 172.21.15.86:0/3434536835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6f0fe960 con 0x55af6f10c370 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.256+0000 7f695b59e640 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6954009e90 tx=0x7f6954016a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acb2e7257871e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f6940ff9640 1 -- 172.21.15.86:0/3434536835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954004c00 con 0x55af6f10c370 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f6940ff9640 1 -- 172.21.15.86:0/3434536835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6954028070 con 0x55af6f10c370 2024-09-05T06:44:33.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f6940ff9640 1 -- 172.21.15.86:0/3434536835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954003970 con 0x55af6f10c370 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 -- 172.21.15.86:0/3434536835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 msgr2=0x55af6f104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6954009e90 tx=0x7f6954016a40 comp rx=0 tx=0).stop 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 -- 172.21.15.86:0/3434536835 shutdown_connections 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f10c370 0x55af6f104120 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f102b40 0x55af6f10af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 --2- 172.21.15.86:0/3434536835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f1015d0 0x55af6f108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 -- 172.21.15.86:0/3434536835 >> 172.21.15.86:0/3434536835 conn(0x55af6f0ca310 msgr2=0x55af6f10a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.257+0000 7f696341f740 1 -- 172.21.15.86:0/3434536835 shutdown_connections 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 -- 172.21.15.86:0/3434536835 wait complete. 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 -- start start 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 Processor -- start 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 0x55af6f110100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f110b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1ae390 con 0x55af6f1015d0 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1adcc0 con 0x55af6f102b40 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1aaf00 con 0x55af6f10c370 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f696341f740 10 client.0 _ll_register_callbacks cb 0x7f695009b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f695ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 0x55af6f110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f695a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f695a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52604/0 (socket says 172.21.15.86:52604) 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f695a59c640 1 -- 172.21.15.86:0/3819221041 learned_addr learned my addr 172.21.15.86:0/3819221041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.258+0000 7f695b59e640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 0x55af6f110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55830/0 (socket says 172.21.15.86:55830) 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 -- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 msgr2=0x55af6f110b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f110b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 -- 172.21.15.86:0/3819221041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 msgr2=0x55af6f110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 0x55af6f110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 -- 172.21.15.86:0/3819221041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6f0fd4f0 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f695a59c640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69500980f0 tx=0x7f695009bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d109c05c6b5eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f693eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500b4ba0 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6954028070 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f695009b0e0 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69500aa050 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f696341f740 10 client.25601 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.259+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af6f186c00 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6954028070 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f69500ad020 con 0x55af6f102b40 2024-09-05T06:44:33.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 10 client.25601 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 10 client.25601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 20 client.25601 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f1893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 4 client.25601 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af6f06ed70 con 0x55af6f11bec0 2024-09-05T06:44:33.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.260+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f1893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.261+0000 7f693eff5640 10 client.25601 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.261+0000 7f695ad9d640 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f1893f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6113271edc8f9622 server_cookie=6a886c129abff45 in_seq=0 out_seq=0 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f693eff5640 1 -- 172.21.15.86:0/3819221041 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af6f06ed70 con 0x55af6f11bec0 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f693eff5640 10 client.25601 handle_command_reply: tid=0 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 client.25601 shutdown 2024-09-05T06:44:33.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 msgr2=0x55af6f110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69500980f0 tx=0x7f695009bb90 comp rx=0 tx=0).stop 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 shutdown_connections 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af6f11bec0 0x55af6f1893f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af6f10c370 0x55af6f110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af6f102b40 0x55af6f110640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 --2- 172.21.15.86:0/3819221041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af6f1015d0 0x55af6f110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f693eff5640 0 client.25601 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 >> 172.21.15.86:0/3819221041 conn(0x55af6f0ca310 msgr2=0x55af6f10b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 shutdown_connections 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 1 -- 172.21.15.86:0/3819221041 wait complete. 2024-09-05T06:44:33.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.262+0000 7f696341f740 20 client.25601 trim_cache size 0 max 16384 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 msgr2=0x7f69440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 0x7f69440a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c000bc0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 msgr2=0x7f695c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c198a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f695c0697d0 tx=0x7f695000ee50 comp rx=0 tx=0).stop 2024-09-05T06:44:33.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 shutdown_connections 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f69440a2cf0 0x7f69440a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f695c1082a0 0x7f695c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f695c1036e0 0x7f695c198a50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 --2- 172.21.15.86:0/3181129997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f695c102d30 0x7f695c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 >> 172.21.15.86:0/3181129997 conn(0x7f695c0fc7d0 msgr2=0x7f695c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 shutdown_connections 2024-09-05T06:44:33.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:33.263+0000 7f6961046640 1 -- 172.21.15.86:0/3181129997 wait complete. 2024-09-05T06:44:33.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 51552 2.60 MB/sec warmup 113 sec latency 54.096 ms 2024-09-05T06:44:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:33 smithi096 ceph-mon[34468]: pgmap v102: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:33.891 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:33 smithi086 ceph-mon[30943]: pgmap v102: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:33 smithi115 ceph-mon[34791]: pgmap v102: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:34.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:44:34.349 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:34.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 52162 2.60 MB/sec warmup 114 sec latency 13.245 ms 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 -- 172.21.15.86:0/3650223134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c0102d30 msgr2=0x7fe2c0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 --2- 172.21.15.86:0/3650223134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c0102d30 0x7fe2c0103110 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009930 tx=0x7fe2bc02a420 comp rx=0 tx=0).stop 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 -- 172.21.15.86:0/3650223134 shutdown_connections 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 --2- 172.21.15.86:0/3650223134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 0x7fe2c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 --2- 172.21.15.86:0/3650223134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c01036e0 0x7fe2c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 --2- 172.21.15.86:0/3650223134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c0102d30 0x7fe2c0103110 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 -- 172.21.15.86:0/3650223134 >> 172.21.15.86:0/3650223134 conn(0x7fe2c00fc7d0 msgr2=0x7fe2c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.860+0000 7fe2c780d640 1 -- 172.21.15.86:0/3650223134 shutdown_connections 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.861+0000 7fe2c780d640 1 -- 172.21.15.86:0/3650223134 wait complete. 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.861+0000 7fe2c780d640 1 Processor -- start 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.861+0000 7fe2c780d640 1 -- start start 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0102d30 0x7fe2c0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 0x7fe2c0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0111f50 con 0x7fe2c01036e0 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0111dd0 con 0x7fe2c0102d30 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01120d0 con 0x7fe2c01082a0 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55856/0 (socket says 172.21.15.86:55856) 2024-09-05T06:44:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 -- 172.21.15.86:0/343916923 learned_addr learned my addr 172.21.15.86:0/343916923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0102d30 0x7fe2c0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 0x7fe2c0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 -- 172.21.15.86:0/343916923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 msgr2=0x7fe2c0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 0x7fe2c0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 -- 172.21.15.86:0/343916923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0102d30 msgr2=0x7fe2c0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0102d30 0x7fe2c0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 -- 172.21.15.86:0/343916923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c01a69b0 con 0x7fe2c01036e0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000d740 tx=0x7fe2b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2b67fc640 1 -- 172.21.15.86:0/343916923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0017070 con 0x7fe2c01036e0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c01a6ca0 con 0x7fe2c01036e0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2b67fc640 1 -- 172.21.15.86:0/343916923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b000de50 con 0x7fe2c01036e0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2b67fc640 1 -- 172.21.15.86:0/343916923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000f620 con 0x7fe2c01036e0 2024-09-05T06:44:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.862+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c01a7190 con 0x7fe2c01036e0 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.865+0000 7fe2b67fc640 1 -- 172.21.15.86:0/343916923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe2b001a070 con 0x7fe2c01036e0 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.866+0000 7fe2c9be6740 1 Processor -- start 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.866+0000 7fe2c9be6740 1 -- start start 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.866+0000 7fe2b67fc640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 0x7fe2ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.866+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.866+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 0x5638abfa1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638abefa2f0 con 0x5638abf9f370 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5638abea1cf0 con 0x5638abf945d0 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5638abef62b0 con 0x5638abf95b40 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2b67fc640 1 -- 172.21.15.86:0/343916923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe2b00cf190 con 0x7fe2c01036e0 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52620/0 (socket says 172.21.15.86:52620) 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 -- 172.21.15.86:0/2264349053 learned_addr learned my addr 172.21.15.86:0/2264349053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 0x7fe2ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 0x5638abfa1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 -- 172.21.15.86:0/2264349053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 msgr2=0x5638abf94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf94090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 -- 172.21.15.86:0/2264349053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 msgr2=0x5638abfa1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 0x5638abfa1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 -- 172.21.15.86:0/2264349053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638abe863a0 con 0x5638abf945d0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 0x5638abfa1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.867+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0097d50 tx=0x7fe2b009ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f088c1ff82d94cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.868+0000 7fe2ab7fe640 1 -- 172.21.15.86:0/2264349053 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0021070 con 0x5638abf945d0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.868+0000 7fe2ab7fe640 1 -- 172.21.15.86:0/2264349053 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b009b380 con 0x5638abf945d0 2024-09-05T06:44:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.868+0000 7fe2ab7fe640 1 -- 172.21.15.86:0/2264349053 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00cc780 con 0x5638abf945d0 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- 172.21.15.86:0/2264349053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 msgr2=0x5638abf94a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 0x7fe2ac0a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5638abf95070 tx=0x7fe2b8009b70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0097d50 tx=0x7fe2b009ae00 comp rx=0 tx=0).stop 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- 172.21.15.86:0/2264349053 shutdown_connections 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf9f370 0x5638abfa1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf945d0 0x5638abf94a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/2264349053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- 172.21.15.86:0/2264349053 >> 172.21.15.86:0/2264349053 conn(0x5638abf5d310 msgr2=0x5638abf90c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- 172.21.15.86:0/2264349053 shutdown_connections 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- 172.21.15.86:0/2264349053 wait complete. 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 -- start start 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.869+0000 7fe2c9be6740 1 Processor -- start 2024-09-05T06:44:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf9ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abf9c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638abfa1fb0 con 0x5638abf95b40 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5638abeb5060 con 0x5638abf9f370 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5638abfa2130 con 0x5638abf945d0 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c9be6740 10 client.0 _ll_register_callbacks cb 0x5638abc088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abf9c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52636/0 (socket says 172.21.15.86:52636) 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c55a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf9ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 -- 172.21.15.86:0/521565206 learned_addr learned my addr 172.21.15.86:0/521565206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 -- 172.21.15.86:0/521565206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 msgr2=0x5638abf9ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf9ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 -- 172.21.15.86:0/521565206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 msgr2=0x5638abf9c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abf9c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 -- 172.21.15.86:0/521565206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638abc6b3c0 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800f970 tx=0x7fe2b8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84b945d52c83a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.870+0000 7fe2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8010dc0 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe2b8004b20 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b009b380 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2b8031050 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 10 client.25613 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638abfada50 con 0x5638abf9f370 2024-09-05T06:44:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b009b380 con 0x5638abf9f370 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe2b8025030 con 0x5638abf9f370 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 10 client.25613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 20 client.25613 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638abfaf380 0x5638abfb17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 4 client.25613 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.871+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638abf07860 con 0x5638abfaf380 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.872+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638abfaf380 0x5638abfb17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.872+0000 7fe2a8ff9640 10 client.25613 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.872+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638abfaf380 0x5638abfb17c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6837a7de9b3a102 server_cookie=d47e04c568e1dc0b in_seq=0 out_seq=0 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/521565206 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638abf07860 con 0x5638abfaf380 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2a8ff9640 10 client.25613 handle_command_reply: tid=0 2024-09-05T06:44:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 client.25613 shutdown 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 msgr2=0x5638abf9c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800f970 tx=0x7fe2b8004a40 comp rx=0 tx=0).stop 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 shutdown_connections 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638abfaf380 0x5638abfb17c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf9c690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abf9c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2a8ff9640 0 client.25613 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/521565206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf9ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 >> 172.21.15.86:0/521565206 conn(0x5638abf5d310 msgr2=0x5638abf969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 shutdown_connections 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 1 -- 172.21.15.86:0/521565206 wait complete. 2024-09-05T06:44:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.873+0000 7fe2c9be6740 20 client.25613 trim_cache size 0 max 16384 2024-09-05T06:44:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.887+0000 7fe2c9be6740 1 Processor -- start 2024-09-05T06:44:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.887+0000 7fe2c9be6740 1 -- start start 2024-09-05T06:44:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf95410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01b1320 con 0x5638abf95b40 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01b2ec0 con 0x5638abf9f370 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01b19f0 con 0x5638abf945d0 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c4da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55898/0 (socket says 172.21.15.86:55898) 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c4da8640 1 -- 172.21.15.86:0/1653011708 learned_addr learned my addr 172.21.15.86:0/1653011708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf95410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1653011708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 msgr2=0x5638abf95410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf95410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1653011708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 msgr2=0x5638abfab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1653011708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638abf9a5c0 con 0x5638abf9f370 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c4da8640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf90700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80059b0 tx=0x7fe2b800f550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93c7002d8cd280 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.888+0000 7fe2aaffd640 1 -- 172.21.15.86:0/1653011708 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b803a070 con 0x5638abf9f370 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2aaffd640 1 -- 172.21.15.86:0/1653011708 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b8010800 con 0x5638abf9f370 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2aaffd640 1 -- 172.21.15.86:0/1653011708 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8005730 con 0x5638abf9f370 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1653011708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 msgr2=0x5638abf90700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf90700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80059b0 tx=0x7fe2b800f550 comp rx=0 tx=0).stop 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1653011708 shutdown_connections 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf90700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf95b40 0x5638abfab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1653011708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf945d0 0x5638abf95410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1653011708 >> 172.21.15.86:0/1653011708 conn(0x5638abf5d310 msgr2=0x5638abf9bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1653011708 shutdown_connections 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.889+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1653011708 wait complete. 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 -- start start 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 Processor -- start 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 0x5638abfa7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf98e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01b2790 con 0x5638abf945d0 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01b20c0 con 0x5638abf9f370 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01af300 con 0x5638abf95b40 2024-09-05T06:44:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c9be6740 10 client.0 _ll_register_callbacks cb 0x7fe2b800f620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.890+0000 7fe2c5daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c55a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 0x5638abfa7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52662/0 (socket says 172.21.15.86:52662) 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1940989439 learned_addr learned my addr 172.21.15.86:0/1940989439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1940989439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 msgr2=0x5638abf98e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf98e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1940989439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 msgr2=0x5638abfa7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 0x5638abfa7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 -- 172.21.15.86:0/1940989439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638abf99c90 con 0x5638abf9f370 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 0x5638abfa7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2c5daa640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8038040 tx=0x7fe2b8010f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153dc9ecc230fcf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.891+0000 7fe2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8042060 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fe2b8005000 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b8010800 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2b8049050 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 10 client.25619 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638ac01a180 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b8010800 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe2b8025040 con 0x5638abf9f370 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 10 client.25619 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 10 client.25619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:44:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 20 client.25619 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638ac01a780 0x5638ac01cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 4 client.25619 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:44:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638abf01d70 con 0x5638ac01a780 2024-09-05T06:44:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.892+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638ac01a780 0x5638ac01cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2a8ff9640 10 client.25619 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c55a9640 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638ac01a780 0x5638ac01cba0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df1169b66426b67 server_cookie=875b0c57b7a4ae2d in_seq=0 out_seq=0 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2a8ff9640 1 -- 172.21.15.86:0/1940989439 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638abf01d70 con 0x5638ac01a780 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2a8ff9640 10 client.25619 handle_command_reply: tid=0 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 client.25619 shutdown 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 msgr2=0x5638abf99460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8038040 tx=0x7fe2b8010f10 comp rx=0 tx=0).stop 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 shutdown_connections 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5638ac01a780 0x5638ac01cba0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5638abf9f370 0x5638abf99460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638abf95b40 0x5638abf98e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 --2- 172.21.15.86:0/1940989439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638abf945d0 0x5638abfa7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2a8ff9640 0 client.25619 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 >> 172.21.15.86:0/1940989439 conn(0x5638abf5d310 msgr2=0x5638abf90700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 shutdown_connections 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 1 -- 172.21.15.86:0/1940989439 wait complete. 2024-09-05T06:44:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.893+0000 7fe2c9be6740 20 client.25619 trim_cache size 0 max 16384 2024-09-05T06:44:34.901 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:44:34.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 msgr2=0x7fe2ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 0x7fe2ac0a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5638abf95070 tx=0x7fe2b8009b70 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 msgr2=0x7fe2c0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000d740 tx=0x7fe2b000dc00 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 shutdown_connections 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe2ac0a2cf0 0x7fe2ac0a51b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2c01082a0 0x7fe2c0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c01036e0 0x7fe2c0100ed0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 --2- 172.21.15.86:0/343916923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0102d30 0x7fe2c0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 >> 172.21.15.86:0/343916923 conn(0x7fe2c00fc7d0 msgr2=0x7fe2c0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 shutdown_connections 2024-09-05T06:44:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:34.894+0000 7fe2c780d640 1 -- 172.21.15.86:0/343916923 wait complete. 2024-09-05T06:44:35.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 52969 2.64 MB/sec warmup 115 sec latency 60.479 ms 2024-09-05T06:44:35.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:35 smithi086 ceph-mon[30943]: pgmap v103: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:35.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:44:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:35 smithi096 ceph-mon[34468]: pgmap v103: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:35 smithi115 ceph-mon[34791]: pgmap v103: 129 pgs: 129 active+clean; 58 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:44:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:36.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 53446 2.65 MB/sec warmup 116 sec latency 14.520 ms 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 -- 172.21.15.86:0/4290744088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 msgr2=0x7f01041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 --2- 172.21.15.86:0/4290744088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041029b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009900 tx=0x7f00f402a480 comp rx=0 tx=0).stop 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 -- 172.21.15.86:0/4290744088 shutdown_connections 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 --2- 172.21.15.86:0/4290744088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 0x7f010410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 --2- 172.21.15.86:0/4290744088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041029b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 --2- 172.21.15.86:0/4290744088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 0x7f0104101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 -- 172.21.15.86:0/4290744088 >> 172.21.15.86:0/4290744088 conn(0x7f01040fc7d0 msgr2=0x7f01040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 -- 172.21.15.86:0/4290744088 shutdown_connections 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.834+0000 7f010b740640 1 -- 172.21.15.86:0/4290744088 wait complete. 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 Processor -- start 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 -- start start 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 0x7f0104199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 0x7f01041a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0104110e30 con 0x7f0104102570 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0104110cb0 con 0x7f0104100e20 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f010b740640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0104110fb0 con 0x7f0104102ef0 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55932/0 (socket says 172.21.15.86:55932) 2024-09-05T06:44:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.835+0000 7f0108cdb640 1 -- 172.21.15.86:0/1579676452 learned_addr learned my addr 172.21.15.86:0/1579676452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0109cdd640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 0x7f01041a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f01094dc640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 0x7f0104199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 -- 172.21.15.86:0/1579676452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 msgr2=0x7f01041a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 0x7f01041a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 -- 172.21.15.86:0/1579676452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 msgr2=0x7f0104199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 0x7f0104199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 -- 172.21.15.86:0/1579676452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01041a4820 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f0108cdb640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006010 tx=0x7f00f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f00fa7fc640 1 -- 172.21.15.86:0/1579676452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f403f030 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f00fa7fc640 1 -- 172.21.15.86:0/1579676452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00f4003aa0 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041a49f0 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f00fa7fc640 1 -- 172.21.15.86:0/1579676452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4035070 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.836+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01041a4ea0 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.838+0000 7f00fa7fc640 1 -- 172.21.15.86:0/1579676452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f00f4042030 con 0x7f0104102570 2024-09-05T06:44:36.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.839+0000 7f010db19740 1 Processor -- start 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.839+0000 7f010db19740 1 -- start start 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.839+0000 7f00fa7fc640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 0x7f00f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.839+0000 7f0109cdd640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 0x7f00f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.839+0000 7f00fa7fc640 1 -- 172.21.15.86:0/1579676452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f00f40efcd0 con 0x7f0104102570 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a42a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a4d370 0x5571c1a4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571c19a82f0 con 0x5571c1a425d0 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571c194fcf0 con 0x5571c1a43b40 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f010db19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571c19a42b0 con 0x5571c1a4d370 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a42a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52688/0 (socket says 172.21.15.86:52688) 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 -- 172.21.15.86:0/326644708 learned_addr learned my addr 172.21.15.86:0/326644708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f0109cdd640 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a4d370 0x5571c1a4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f0109cdd640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 0x7f00f00a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005fd0 tx=0x7f00fc0107a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 -- 172.21.15.86:0/326644708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a4d370 msgr2=0x5571c1a4f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a4d370 0x5571c1a4f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 -- 172.21.15.86:0/326644708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 msgr2=0x5571c1a42a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a42a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 -- 172.21.15.86:0/326644708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c19343a0 con 0x5571c1a43b40 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f0108cdb640 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a42a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f01094dc640 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0100009cf0 tx=0x7f0100011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458136ebb675b077 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f00ef7fe640 1 -- 172.21.15.86:0/326644708 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100004230 con 0x5571c1a43b40 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.840+0000 7f00ef7fe640 1 -- 172.21.15.86:0/326644708 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01000043d0 con 0x5571c1a43b40 2024-09-05T06:44:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f00ef7fe640 1 -- 172.21.15.86:0/326644708 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01000197e0 con 0x5571c1a43b40 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 -- 172.21.15.86:0/326644708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 msgr2=0x5571c1a42090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0100009cf0 tx=0x7f0100011810 comp rx=0 tx=0).stop 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 -- 172.21.15.86:0/326644708 shutdown_connections 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a4d370 0x5571c1a4f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a42a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 --2- 172.21.15.86:0/326644708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a42090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 -- 172.21.15.86:0/326644708 >> 172.21.15.86:0/326644708 conn(0x5571c1a0b310 msgr2=0x5571c1a3ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 -- 172.21.15.86:0/326644708 shutdown_connections 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.841+0000 7f010db19740 1 -- 172.21.15.86:0/326644708 wait complete. 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 -- start start 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 Processor -- start 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a43b40 0x5571c1a49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a4d370 0x5571c1a4a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571c1a50070 con 0x5571c1a425d0 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571c1963060 con 0x5571c1a4d370 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5571c1a501f0 con 0x5571c1a43b40 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f010db19740 10 client.0 _ll_register_callbacks cb 0x5571c16b68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f01094dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f01094dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55942/0 (socket says 172.21.15.86:55942) 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a43b40 0x5571c1a49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f01094dc640 1 -- 172.21.15.86:0/2835605934 learned_addr learned my addr 172.21.15.86:0/2835605934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.842+0000 7f0109cdd640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a4d370 0x5571c1a4a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 -- 172.21.15.86:0/2835605934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a43b40 msgr2=0x5571c1a49f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a43b40 0x5571c1a49f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 -- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a4d370 msgr2=0x5571c1a4a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a4d370 0x5571c1a4a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 -- 172.21.15.86:0/2835605934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c17193c0 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f01094dc640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0100019480 tx=0x7f01000038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88225644ec4bb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100027070 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0100005450 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01000043d0 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0100033050 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f010db19740 10 client.16251 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571c1a5ba60 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01000043d0 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0100019520 con 0x5571c1a425d0 2024-09-05T06:44:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.843+0000 7f010db19740 10 client.16251 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f010db19740 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f010db19740 20 client.16251 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a5ce10 0x5571c1a5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f010db19740 4 client.16251 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c19b5860 con 0x5571c1a5ce10 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f0108cdb640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a5ce10 0x5571c1a5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f00ecff9640 10 client.16251 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.844+0000 7f0108cdb640 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a5ce10 0x5571c1a5f200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d3fe9f6c9347d0f server_cookie=9977733ffc837976 in_seq=0 out_seq=0 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f00ecff9640 1 -- 172.21.15.86:0/2835605934 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c19b5860 con 0x5571c1a5ce10 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f00ecff9640 10 client.16251 handle_command_reply: tid=0 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 client.16251 shutdown 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 msgr2=0x5571c1a4cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0100019480 tx=0x7f01000038a0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 shutdown_connections 2024-09-05T06:44:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a5ce10 0x5571c1a5f200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a4d370 0x5571c1a4a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a43b40 0x5571c1a49f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 --2- 172.21.15.86:0/2835605934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a4cca0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 >> 172.21.15.86:0/2835605934 conn(0x5571c1a0b310 msgr2=0x5571c1a44930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f00ecff9640 0 client.16251 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 shutdown_connections 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 1 -- 172.21.15.86:0/2835605934 wait complete. 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.845+0000 7f010db19740 20 client.16251 trim_cache size 0 max 16384 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 Processor -- start 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 -- start start 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a425d0 0x5571c1a45310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01041af000 con 0x5571c1a43b40 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01041b0ba0 con 0x5571c1a0b310 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f010db19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01041af6d0 con 0x5571c1a425d0 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f0109cdd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f01094dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f0109cdd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55946/0 (socket says 172.21.15.86:55946) 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f01094dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52706/0 (socket says 172.21.15.86:52706) 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f0109cdd640 1 -- 172.21.15.86:0/94786273 learned_addr learned my addr 172.21.15.86:0/94786273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.859+0000 7f0108cdb640 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a425d0 0x5571c1a45310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 -- 172.21.15.86:0/94786273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a425d0 msgr2=0x5571c1a45310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a425d0 0x5571c1a45310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 -- 172.21.15.86:0/94786273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 msgr2=0x5571c1a59310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 -- 172.21.15.86:0/94786273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c1a45120 con 0x5571c1a43b40 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f01094dc640 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f0109cdd640 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f00fc010be0 tx=0x7f00fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db91c86311c270e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f00eeffd640 1 -- 172.21.15.86:0/94786273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0130d0 con 0x5571c1a43b40 2024-09-05T06:44:36.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f00eeffd640 1 -- 172.21.15.86:0/94786273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00fc01fde0 con 0x5571c1a43b40 2024-09-05T06:44:36.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.860+0000 7f00eeffd640 1 -- 172.21.15.86:0/94786273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011bf0 con 0x5571c1a43b40 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- 172.21.15.86:0/94786273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 msgr2=0x5571c1a48f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f00fc010be0 tx=0x7f00fc015000 comp rx=0 tx=0).stop 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- 172.21.15.86:0/94786273 shutdown_connections 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a43b40 0x5571c1a48f70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a425d0 0x5571c1a45310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 --2- 172.21.15.86:0/94786273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a0b310 0x5571c1a59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- 172.21.15.86:0/94786273 >> 172.21.15.86:0/94786273 conn(0x5571c1a4a5f0 msgr2=0x5571c1a48860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- 172.21.15.86:0/94786273 shutdown_connections 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- 172.21.15.86:0/94786273 wait complete. 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 -- start start 2024-09-05T06:44:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.861+0000 7f010db19740 1 Processor -- start 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a0b310 0x5571c1ac3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01041b0470 con 0x5571c1a425d0 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01041afda0 con 0x5571c1a43b40 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01041acfe0 con 0x5571c1a0b310 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f010db19740 10 client.0 _ll_register_callbacks cb 0x7f00f4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55956/0 (socket says 172.21.15.86:55956) 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 -- 172.21.15.86:0/3803959919 learned_addr learned my addr 172.21.15.86:0/3803959919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f01094dc640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a0b310 0x5571c1ac3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0109cdd640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 -- 172.21.15.86:0/3803959919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a0b310 msgr2=0x5571c1ac3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a0b310 0x5571c1ac3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 -- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 msgr2=0x5571c1a559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.862+0000 7f0108cdb640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f0108cdb640 1 -- 172.21.15.86:0/3803959919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c1a55f80 con 0x5571c1a425d0 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f0108cdb640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f00f40bbe00 tx=0x7f00f40c0280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b407406dc0389d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f00ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f409f060 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f00f40b8530 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00fc01fde0 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00f40d9030 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f010db19740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571c1acbde0 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.863+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00fc01fde0 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f00f409ace0 con 0x5571c1a425d0 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 10 client.16257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 20 client.16257 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a4d370 0x5571c1ace5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 4 client.16257 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:44:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571c19afd70 con 0x5571c1a4d370 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f01094dc640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a4d370 0x5571c1ace5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f01094dc640 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a4d370 0x5571c1ace5e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1103d72c62384b78 server_cookie=c0e85ca82a9816ed in_seq=0 out_seq=0 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f00ecff9640 10 client.16257 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f00ecff9640 1 -- 172.21.15.86:0/3803959919 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571c19afd70 con 0x5571c1a4d370 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f00ecff9640 10 client.16257 handle_command_reply: tid=0 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.864+0000 7f010db19740 1 client.16257 shutdown 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 msgr2=0x5571c1a55480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f00f40bbe00 tx=0x7f00f40c0280 comp rx=0 tx=0).stop 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 shutdown_connections 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5571c1a4d370 0x5571c1ace5e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571c1a43b40 0x5571c1a559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571c1a425d0 0x5571c1a55480 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 --2- 172.21.15.86:0/3803959919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5571c1a0b310 0x5571c1ac3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 >> 172.21.15.86:0/3803959919 conn(0x5571c1a4a5f0 msgr2=0x5571c1a49080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f00ecff9640 0 client.16257 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 shutdown_connections 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 1 -- 172.21.15.86:0/3803959919 wait complete. 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.865+0000 7f010db19740 20 client.16257 trim_cache size 0 max 16384 2024-09-05T06:44:36.880 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 msgr2=0x7f00f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 0x7f00f00a5230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005fd0 tx=0x7f00fc0107a0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 msgr2=0x7f01041997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006010 tx=0x7f00f4004270 comp rx=0 tx=0).stop 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 shutdown_connections 2024-09-05T06:44:36.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f00f00a2d70 0x7f00f00a5230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0104102ef0 0x7f01041a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0104102570 0x7f01041997a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 --2- 172.21.15.86:0/1579676452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0104100e20 0x7f0104199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 >> 172.21.15.86:0/1579676452 conn(0x7f01040fc7d0 msgr2=0x7f01040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 shutdown_connections 2024-09-05T06:44:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:36.866+0000 7f010b740640 1 -- 172.21.15.86:0/1579676452 wait complete. 2024-09-05T06:44:37.508 INFO:tasks.workunit.client.0.smithi086.stdout: 1 53700 2.63 MB/sec warmup 117 sec latency 25.618 ms 2024-09-05T06:44:37.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:37 smithi086 ceph-mon[30943]: pgmap v104: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-05T06:44:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:37 smithi096 ceph-mon[34468]: pgmap v104: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-05T06:44:37.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:44:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:37 smithi115 ceph-mon[34791]: pgmap v104: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-09-05T06:44:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:38.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 54109 2.61 MB/sec warmup 118 sec latency 58.229 ms 2024-09-05T06:44:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 -- 172.21.15.86:0/1625103561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14102570 msgr2=0x7fce141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 --2- 172.21.15.86:0/1625103561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14102570 0x7fce141029b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009900 tx=0x7fcdfc02a480 comp rx=0 tx=0).stop 2024-09-05T06:44:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 -- 172.21.15.86:0/1625103561 shutdown_connections 2024-09-05T06:44:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 --2- 172.21.15.86:0/1625103561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102ef0 0x7fce1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 --2- 172.21.15.86:0/1625103561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14102570 0x7fce141029b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 --2- 172.21.15.86:0/1625103561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14100e20 0x7fce14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 -- 172.21.15.86:0/1625103561 >> 172.21.15.86:0/1625103561 conn(0x7fce140fc7d0 msgr2=0x7fce140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.674+0000 7fce189cb640 1 -- 172.21.15.86:0/1625103561 shutdown_connections 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.675+0000 7fce189cb640 1 -- 172.21.15.86:0/1625103561 wait complete. 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.675+0000 7fce189cb640 1 Processor -- start 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.675+0000 7fce189cb640 1 -- start start 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14102ef0 0x7fce1419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce14110e30 con 0x7fce14100e20 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce14110cb0 con 0x7fce14102570 2024-09-05T06:44:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce14110fb0 con 0x7fce14102ef0 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55966/0 (socket says 172.21.15.86:55966) 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52734/0 (socket says 172.21.15.86:52734) 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 -- 172.21.15.86:0/805255373 learned_addr learned my addr 172.21.15.86:0/805255373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce12d9d640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14102ef0 0x7fce1419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 -- 172.21.15.86:0/805255373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14102ef0 msgr2=0x7fce1419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14102ef0 0x7fce1419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 -- 172.21.15.86:0/805255373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 msgr2=0x7fce14198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 -- 172.21.15.86:0/805255373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1419d3b0 con 0x7fce14100e20 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0bfff640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce1259c640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fce04009b40 tx=0x7fce040071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0b7fe640 1 -- 172.21.15.86:0/805255373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04007990 con 0x7fce14100e20 2024-09-05T06:44:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1419d6a0 con 0x7fce14100e20 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0b7fe640 1 -- 172.21.15.86:0/805255373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0400f020 con 0x7fce14100e20 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce140fcbb0 con 0x7fce14100e20 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.676+0000 7fce0b7fe640 1 -- 172.21.15.86:0/805255373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0400ed80 con 0x7fce14100e20 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.679+0000 7fce0b7fe640 1 -- 172.21.15.86:0/805255373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce0400f1d0 con 0x7fce14100e20 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.679+0000 7fce1ada4740 1 Processor -- start 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.679+0000 7fce1ada4740 1 -- start start 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55579030c2f0 con 0x5557903b1370 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557902b3cf0 con 0x5557903a7b40 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1ada4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557903082b0 con 0x5557903a65d0 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce0b7fe640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52748/0 (socket says 172.21.15.86:52748) 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce12d9d640 1 -- 172.21.15.86:0/1397155798 learned_addr learned my addr 172.21.15.86:0/1397155798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce1259c640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce0bfff640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce12d9d640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.680+0000 7fce0b7fe640 1 -- 172.21.15.86:0/805255373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fce040d2e70 con 0x7fce14100e20 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 -- 172.21.15.86:0/1397155798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 msgr2=0x5557903a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 -- 172.21.15.86:0/1397155798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 msgr2=0x5557903b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 -- 172.21.15.86:0/1397155798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557902983a0 con 0x5557903a65d0 2024-09-05T06:44:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce12d9d640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce1259c640 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903a6a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce040d01e0 tx=0x7fce04072020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b37ea1d7b2965c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce12d9d640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 0x7fce000a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005c40 tx=0x7fce0c004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce08ff9640 1 -- 172.21.15.86:0/1397155798 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0409f6b0 con 0x5557903a65d0 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce08ff9640 1 -- 172.21.15.86:0/1397155798 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0409efb0 con 0x5557903a65d0 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.681+0000 7fce08ff9640 1 -- 172.21.15.86:0/1397155798 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04069c50 con 0x5557903a65d0 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- 172.21.15.86:0/1397155798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 msgr2=0x5557903a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903a6a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce040d01e0 tx=0x7fce04072020 comp rx=0 tx=0).stop 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- 172.21.15.86:0/1397155798 shutdown_connections 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903a6a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1397155798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- 172.21.15.86:0/1397155798 >> 172.21.15.86:0/1397155798 conn(0x55579036f310 msgr2=0x5557903a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- 172.21.15.86:0/1397155798 shutdown_connections 2024-09-05T06:44:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- 172.21.15.86:0/1397155798 wait complete. 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 -- start start 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.682+0000 7fce1ada4740 1 Processor -- start 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903b1370 0x5557903ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557903b4010 con 0x5557903a65d0 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557902c7060 con 0x5557903a7b40 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557903b4190 con 0x5557903b1370 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55988/0 (socket says 172.21.15.86:55988) 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 -- 172.21.15.86:0/844472424 learned_addr learned my addr 172.21.15.86:0/844472424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce0bfff640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 10 client.0 _ll_register_callbacks cb 0x55579001a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce12d9d640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903b1370 0x5557903ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 -- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903b1370 msgr2=0x5557903ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903b1370 0x5557903ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 -- 172.21.15.86:0/844472424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 msgr2=0x5557903ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 -- 172.21.15.86:0/844472424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579007d3c0 con 0x5557903a65d0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce0bfff640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1259c640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fce040d07e0 tx=0x7fce0409b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6655e46b88b9c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fcdf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040a3360 con 0x5557903a65d0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fce040a79a0 con 0x5557903a65d0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.683+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0409efb0 con 0x5557903a65d0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce04007b30 con 0x5557903a65d0 2024-09-05T06:44:38.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 10 client.16269 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557903b4010 con 0x5557903a65d0 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0409efb0 con 0x5557903a65d0 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fce0409ad00 con 0x5557903a65d0 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 10 client.16269 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 20 client.16269 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5557903c0c80 0x5557903c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 4 client.16269 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555790319860 con 0x5557903c0c80 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.684+0000 7fce0bfff640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5557903c0c80 0x5557903c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.685+0000 7fce0bfff640 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5557903c0c80 0x5557903c3070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7257fedd14dc806a server_cookie=17c8fbe438ff179a in_seq=0 out_seq=0 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.685+0000 7fcdf9ffb640 10 client.16269 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.685+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/844472424 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555790319860 con 0x5557903c0c80 2024-09-05T06:44:38.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.685+0000 7fcdf9ffb640 10 client.16269 handle_command_reply: tid=0 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.685+0000 7fce1ada4740 1 client.16269 shutdown 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 msgr2=0x5557903b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fce040d07e0 tx=0x7fce0409b8c0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 shutdown_connections 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5557903c0c80 0x5557903c3070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903b1370 0x5557903ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fcdf9ffb640 0 client.16269 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 --2- 172.21.15.86:0/844472424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903b0dc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 >> 172.21.15.86:0/844472424 conn(0x55579036f310 msgr2=0x5557903a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 shutdown_connections 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 1 -- 172.21.15.86:0/844472424 wait complete. 2024-09-05T06:44:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.686+0000 7fce1ada4740 20 client.16269 trim_cache size 0 max 16384 2024-09-05T06:44:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 Processor -- start 2024-09-05T06:44:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 -- start start 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 0x5557903c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 0x5557903c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce141acf40 con 0x5557903a65d0 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce141aeae0 con 0x5557903b1370 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1ada4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce141ad610 con 0x5557903a7b40 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55996/0 (socket says 172.21.15.86:55996) 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce1259c640 1 -- 172.21.15.86:0/570662403 learned_addr learned my addr 172.21.15.86:0/570662403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.701+0000 7fce12d9d640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 0x5557903c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce0bfff640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 0x5557903c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 -- 172.21.15.86:0/570662403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 msgr2=0x5557903c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 0x5557903c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 -- 172.21.15.86:0/570662403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 msgr2=0x5557903c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 0x5557903c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 -- 172.21.15.86:0/570662403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557903b10f0 con 0x5557903a65d0 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce0bfff640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 0x5557903c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce12d9d640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 0x5557903c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fce1259c640 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce040a7de0 tx=0x7fce0409b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbe969f562d94ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fcdfbfff640 1 -- 172.21.15.86:0/570662403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04082070 con 0x5557903a65d0 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fcdfbfff640 1 -- 172.21.15.86:0/570662403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce040a3360 con 0x5557903a65d0 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.702+0000 7fcdfbfff640 1 -- 172.21.15.86:0/570662403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04007cd0 con 0x5557903a65d0 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- 172.21.15.86:0/570662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 msgr2=0x5557903ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce040a7de0 tx=0x7fce0409b380 comp rx=0 tx=0).stop 2024-09-05T06:44:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- 172.21.15.86:0/570662403 shutdown_connections 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903b1370 0x5557903c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a7b40 0x5557903c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 --2- 172.21.15.86:0/570662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903a65d0 0x5557903ad050 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- 172.21.15.86:0/570662403 >> 172.21.15.86:0/570662403 conn(0x55579036f310 msgr2=0x5557903ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- 172.21.15.86:0/570662403 shutdown_connections 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- 172.21.15.86:0/570662403 wait complete. 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.703+0000 7fce1ada4740 1 -- start start 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 Processor -- start 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce141ae3b0 con 0x5557903b1370 2024-09-05T06:44:38.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce141adce0 con 0x5557903a7b40 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce141aaf20 con 0x5557903a65d0 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1ada4740 10 client.0 _ll_register_callbacks cb 0x7fcdfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52766/0 (socket says 172.21.15.86:52766) 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 -- 172.21.15.86:0/1324118944 learned_addr learned my addr 172.21.15.86:0/1324118944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56000/0 (socket says 172.21.15.86:56000) 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce1259c640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 -- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 msgr2=0x5557903b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 -- 172.21.15.86:0/1324118944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 msgr2=0x5557903b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:38.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce0bfff640 1 -- 172.21.15.86:0/1324118944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579042a780 con 0x5557903a7b40 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.704+0000 7fce12d9d640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fce0bfff640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009900 tx=0x7fcdfc032720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880fd0e8ec7bf576 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc035040 con 0x5557903a7b40 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce040a3360 con 0x5557903a7b40 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fcdfc004450 con 0x5557903a7b40 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc059050 con 0x5557903a7b40 2024-09-05T06:44:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fce1ada4740 10 client.25655 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55579042ba60 con 0x5557903a7b40 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce040a3360 con 0x5557903a7b40 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.705+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcdfc004a60 con 0x5557903a7b40 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 10 client.25655 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 10 client.25655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 20 client.25655 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55579042c210 0x55579042e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 4 client.25655 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555790313d70 con 0x55579042c210 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1259c640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55579042c210 0x55579042e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fcdf9ffb640 10 client.25655 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1259c640 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55579042c210 0x55579042e630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f5093228896355d server_cookie=75473f7bdc168e63 in_seq=0 out_seq=0 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fcdf9ffb640 1 -- 172.21.15.86:0/1324118944 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555790313d70 con 0x55579042c210 2024-09-05T06:44:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fcdf9ffb640 10 client.25655 handle_command_reply: tid=0 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 1 client.25655 shutdown 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 msgr2=0x5557903b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.706+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009900 tx=0x7fcdfc032720 comp rx=0 tx=0).stop 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 shutdown_connections 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55579042c210 0x55579042e630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557903b1370 0x5557903b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557903a7b40 0x5557903b5630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 --2- 172.21.15.86:0/1324118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557903a65d0 0x5557903b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 >> 172.21.15.86:0/1324118944 conn(0x55579036f310 msgr2=0x5557903ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fcdf9ffb640 0 client.25655 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:44:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 shutdown_connections 2024-09-05T06:44:38.719 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 1 -- 172.21.15.86:0/1324118944 wait complete. 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce1ada4740 20 client.25655 trim_cache size 0 max 16384 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 msgr2=0x7fce000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 0x7fce000a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005c40 tx=0x7fce0c004ab0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 msgr2=0x7fce14198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.707+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fce04009b40 tx=0x7fce040071c0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 shutdown_connections 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14102ef0 0x7fce1419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce14102570 0x7fce14198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 --2- 172.21.15.86:0/805255373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce14100e20 0x7fce14198400 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:38.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 >> 172.21.15.86:0/805255373 conn(0x7fce140fc7d0 msgr2=0x7fce1406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:38.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 shutdown_connections 2024-09-05T06:44:38.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:38.708+0000 7fce189cb640 1 -- 172.21.15.86:0/805255373 wait complete. 2024-09-05T06:44:39.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 54706 2.63 MB/sec warmup 119 sec latency 13.886 ms 2024-09-05T06:44:39.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:44:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:39 smithi096 ceph-mon[34468]: pgmap v105: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:44:39.837 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:39 smithi115 ceph-mon[34791]: pgmap v105: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:44:39.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:39 smithi086 ceph-mon[30943]: pgmap v105: 129 pgs: 129 active+clean; 59 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/378163905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4102ef0 msgr2=0x7f0ec410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/378163905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4102ef0 0x7f0ec410da40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009ca0 tx=0x7f0ebc02a950 comp rx=0 tx=0).stop 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/378163905 shutdown_connections 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/378163905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4102ef0 0x7f0ec410da40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/378163905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102570 0x7f0ec41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/378163905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4100e20 0x7f0ec4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/378163905 >> 172.21.15.86:0/378163905 conn(0x7f0ec40fc7d0 msgr2=0x7f0ec40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/378163905 shutdown_connections 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.327+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/378163905 wait complete. 2024-09-05T06:44:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 Processor -- start 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 -- start start 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4102570 0x7f0ec419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102ef0 0x7f0ec419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4110e30 con 0x7f0ec4100e20 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4110cb0 con 0x7f0ec4102ef0 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ecb9d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4110fb0 con 0x7f0ec4102570 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56014/0 (socket says 172.21.15.86:56014) 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.328+0000 7f0ec9774640 1 -- 172.21.15.86:0/3496480554 learned_addr learned my addr 172.21.15.86:0/3496480554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4102570 0x7f0ec419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9f75640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102ef0 0x7f0ec419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 -- 172.21.15.86:0/3496480554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4102570 msgr2=0x7f0ec419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4102570 0x7f0ec419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 -- 172.21.15.86:0/3496480554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102ef0 msgr2=0x7f0ec419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102ef0 0x7f0ec419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 -- 172.21.15.86:0/3496480554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec419f590 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ec9774640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000d950 tx=0x7f0ec000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0eba7fc640 1 -- 172.21.15.86:0/3496480554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0017070 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0eba7fc640 1 -- 172.21.15.86:0/3496480554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ec0009c30 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec419f880 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0eba7fc640 1 -- 172.21.15.86:0/3496480554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000f620 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.329+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec41a7160 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0eba7fc640 1 -- 172.21.15.86:0/3496480554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0ec00109a0 con 0x7f0ec4100e20 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 Processor -- start 2024-09-05T06:44:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 -- start start 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b82a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f971ae82f0 con 0x55f971b83b40 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f971a8fcf0 con 0x55f971b8d370 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f971ae42b0 con 0x55f971b825d0 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ec9f75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ec9f75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b82090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56030/0 (socket says 172.21.15.86:56030) 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ec9f75640 1 -- 172.21.15.86:0/4283535330 learned_addr learned my addr 172.21.15.86:0/4283535330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.332+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec9774640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b82a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0eba7fc640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 0x7f0eb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 -- 172.21.15.86:0/4283535330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 msgr2=0x55f971b82a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b82a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 -- 172.21.15.86:0/4283535330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 msgr2=0x55f971b82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 -- 172.21.15.86:0/4283535330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f971a743a0 con 0x55f971b8d370 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec9774640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec9f75640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 0x7f0eb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0eba7fc640 1 -- 172.21.15.86:0/3496480554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0ec00cf3d0 con 0x7f0ec4100e20 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b8f760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009bf0 tx=0x7f0eb4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7e938938186d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0eaf7fe640 1 -- 172.21.15.86:0/4283535330 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4004050 con 0x55f971b8d370 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0eaf7fe640 1 -- 172.21.15.86:0/4283535330 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eb40054d0 con 0x55f971b8d370 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.333+0000 7f0eaf7fe640 1 -- 172.21.15.86:0/4283535330 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4004cc0 con 0x55f971b8d370 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- 172.21.15.86:0/4283535330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 msgr2=0x55f971b8f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b8f760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009bf0 tx=0x7f0eb4011810 comp rx=0 tx=0).stop 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ec9f75640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 0x7f0eb00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009ca0 tx=0x7f0ebc03c040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- 172.21.15.86:0/4283535330 shutdown_connections 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b8f760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/4283535330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- 172.21.15.86:0/4283535330 >> 172.21.15.86:0/4283535330 conn(0x55f971b4b310 msgr2=0x55f971b7ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- 172.21.15.86:0/4283535330 shutdown_connections 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- 172.21.15.86:0/4283535330 wait complete. 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.334+0000 7f0ecddb1740 1 -- start start 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 Processor -- start 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 0x55f971b8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f971b90070 con 0x55f971b8d370 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f971aa3060 con 0x55f971b83b40 2024-09-05T06:44:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f971b901f0 con 0x55f971b825d0 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ecddb1740 10 client.0 _ll_register_callbacks cb 0x55f9717f68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec9f75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 0x55f971b8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec9f75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 0x55f971b8a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56034/0 (socket says 172.21.15.86:56034) 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52800/0 (socket says 172.21.15.86:52800) 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec9f75640 1 -- 172.21.15.86:0/2295172177 learned_addr learned my addr 172.21.15.86:0/2295172177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 -- 172.21.15.86:0/2295172177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 msgr2=0x55f971b8cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b8cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 -- 172.21.15.86:0/2295172177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 msgr2=0x55f971b8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.335+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 0x55f971b8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0ec8f73640 1 -- 172.21.15.86:0/2295172177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9718593c0 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb4009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf76a29b0d836509 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0eacff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4029070 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0eb4003a00 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb40054d0 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0eb4035050 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0ecddb1740 10 client.25667 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f971b9bac0 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.336+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eb40054d0 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0eb40050b0 con 0x55f971b83b40 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 20 client.25667 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971b9ce70 0x55f971b9f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 4 client.25667 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f971af5860 con 0x55f971b9ce70 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971b9ce70 0x55f971b9f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971b9ce70 0x55f971b9f260 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d11077bfddeac2 server_cookie=689641829ef66abf in_seq=0 out_seq=0 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.337+0000 7f0eacff9640 10 client.25667 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0eacff9640 1 -- 172.21.15.86:0/2295172177 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f971af5860 con 0x55f971b9ce70 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0eacff9640 10 client.25667 handle_command_reply: tid=0 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 client.25667 shutdown 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 msgr2=0x55f971b89fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb4009fb0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 shutdown_connections 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971b9ce70 0x55f971b9f260 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b8d370 0x55f971b8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0eacff9640 0 client.25667 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b83b40 0x55f971b89fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2295172177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 >> 172.21.15.86:0/2295172177 conn(0x55f971b4b310 msgr2=0x55f971b84960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 shutdown_connections 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2295172177 wait complete. 2024-09-05T06:44:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.338+0000 7f0ecddb1740 20 client.25667 trim_cache size 0 max 16384 2024-09-05T06:44:40.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 Processor -- start 2024-09-05T06:44:40.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 -- start start 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41b1190 con 0x55f971b83b40 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41b2d30 con 0x55f971b8d370 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41b1860 con 0x55f971b825d0 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ec8f73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35610/0 (socket says 172.21.15.86:35610) 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ec9774640 1 -- 172.21.15.86:0/2435366114 learned_addr learned my addr 172.21.15.86:0/2435366114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.353+0000 7f0ec9f75640 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 -- 172.21.15.86:0/2435366114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 msgr2=0x55f971b85660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b85660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 -- 172.21.15.86:0/2435366114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 msgr2=0x55f971b85120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b85120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 -- 172.21.15.86:0/2435366114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f971b9f890 con 0x55f971b825d0 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0ec9774640 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ec009b2c0 tx=0x7f0ec0086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544debf23e44a8ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0eaeffd640 1 -- 172.21.15.86:0/2435366114 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0020040 con 0x55f971b825d0 2024-09-05T06:44:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0eaeffd640 1 -- 172.21.15.86:0/2435366114 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ec00a4180 con 0x55f971b825d0 2024-09-05T06:44:40.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.354+0000 7f0eaeffd640 1 -- 172.21.15.86:0/2435366114 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0097c30 con 0x55f971b825d0 2024-09-05T06:44:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2435366114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 msgr2=0x55f971b99310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ec009b2c0 tx=0x7f0ec0086040 comp rx=0 tx=0).stop 2024-09-05T06:44:40.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2435366114 shutdown_connections 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b8d370 0x55f971b85660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/2435366114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b825d0 0x55f971b99310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2435366114 >> 172.21.15.86:0/2435366114 conn(0x55f971b4b310 msgr2=0x55f971b8c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2435366114 shutdown_connections 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.355+0000 7f0ecddb1740 1 -- 172.21.15.86:0/2435366114 wait complete. 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 -- start start 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 Processor -- start 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 0x55f971b87330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41b2600 con 0x55f971b83b40 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41b1f30 con 0x55f971b825d0 2024-09-05T06:44:40.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec41af170 con 0x55f971b8d370 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ecddb1740 10 client.0 _ll_register_callbacks cb 0x7f0ebc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ec8f73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ec9774640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52818/0 (socket says 172.21.15.86:52818) 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ec9774640 1 -- 172.21.15.86:0/1976066086 learned_addr learned my addr 172.21.15.86:0/1976066086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.356+0000 7f0ec9f75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 0x55f971b87330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 -- 172.21.15.86:0/1976066086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 msgr2=0x55f971b87330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 0x55f971b87330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 -- 172.21.15.86:0/1976066086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 msgr2=0x55f971b86db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b86db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 -- 172.21.15.86:0/1976066086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f971b87b60 con 0x55f971b825d0 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b86db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:40.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9f75640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 0x55f971b87330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ec9774640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00a3c50 tx=0x7f0ec00978a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff1b60aac09d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0eacff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00b4060 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0ec0097c30 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec00a4180 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ec0020070 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ecddb1740 10 client.25673 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f971c07b30 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.357+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ec00a4180 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0ec0084b00 con 0x55f971b825d0 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 10 client.25673 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:40.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 20 client.25673 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971c08260 0x55f971c0a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 4 client.25673 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f971aefd70 con 0x55f971c08260 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971c08260 0x55f971c0a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0eacff9640 10 client.25673 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ec8f73640 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971c08260 0x55f971c0a680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85dcf668c42d58e server_cookie=a07caae92f815429 in_seq=0 out_seq=0 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0eacff9640 1 -- 172.21.15.86:0/1976066086 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f971aefd70 con 0x55f971c08260 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0eacff9640 10 client.25673 handle_command_reply: tid=0 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.358+0000 7f0ecddb1740 1 client.25673 shutdown 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 msgr2=0x55f971b95c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00a3c50 tx=0x7f0ec00978a0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 shutdown_connections 2024-09-05T06:44:40.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f971c08260 0x55f971c0a680 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f971b8d370 0x55f971b87330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0eacff9640 0 client.25673 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f971b83b40 0x55f971b86db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 --2- 172.21.15.86:0/1976066086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f971b825d0 0x55f971b95c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 >> 172.21.15.86:0/1976066086 conn(0x55f971b4b310 msgr2=0x55f971b88790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 shutdown_connections 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 1 -- 172.21.15.86:0/1976066086 wait complete. 2024-09-05T06:44:40.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecddb1740 20 client.25673 trim_cache size 0 max 16384 2024-09-05T06:44:40.367 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.359+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 msgr2=0x7f0eb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 0x7f0eb00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009ca0 tx=0x7f0ebc03c040 comp rx=0 tx=0).stop 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 msgr2=0x7f0ec419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000d950 tx=0x7f0ec000de10 comp rx=0 tx=0).stop 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 shutdown_connections 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0eb00a2cf0 0x7f0eb00a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec4102ef0 0x7f0ec419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ec4102570 0x7f0ec419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 --2- 172.21.15.86:0/3496480554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec4100e20 0x7f0ec419a5e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:40.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 >> 172.21.15.86:0/3496480554 conn(0x7f0ec40fc7d0 msgr2=0x7f0ec406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:40.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 shutdown_connections 2024-09-05T06:44:40.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:40.360+0000 7f0ecb9d8640 1 -- 172.21.15.86:0/3496480554 wait complete. 2024-09-05T06:44:41.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 55854 2.45 MB/sec execute 1 sec latency 65.814 ms 2024-09-05T06:44:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:41 smithi096 ceph-mon[34468]: pgmap v106: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:44:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:41 smithi115 ceph-mon[34791]: pgmap v106: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:44:41.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:41 smithi086 ceph-mon[30943]: pgmap v106: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:44:42.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:44:42.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:44:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:42.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 56567 4.38 MB/sec execute 2 sec latency 13.567 ms 2024-09-05T06:44:42.657 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T06:44:42.658 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T06:44:42.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:44:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 -- 172.21.15.86:0/881374692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 msgr2=0x7f5f6810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 --2- 172.21.15.86:0/881374692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f6810ea20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009ca0 tx=0x7f5f6002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 -- 172.21.15.86:0/881374692 shutdown_connections 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 --2- 172.21.15.86:0/881374692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f6810ea20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 --2- 172.21.15.86:0/881374692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 0x7f5f68100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 --2- 172.21.15.86:0/881374692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 0x7f5f68106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.972+0000 7f5f6e145640 1 -- 172.21.15.86:0/881374692 >> 172.21.15.86:0/881374692 conn(0x7f5f68077230 msgr2=0x7f5f680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.973+0000 7f5f6e145640 1 -- 172.21.15.86:0/881374692 shutdown_connections 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.973+0000 7f5f6e145640 1 -- 172.21.15.86:0/881374692 wait complete. 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.973+0000 7f5f6e145640 1 Processor -- start 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 -- start start 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 0x7f5f68198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 0x7f5f6819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68111f00 con 0x7f5f68100aa0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68111d80 con 0x7f5f68106b80 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f6e145640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68112080 con 0x7f5f68100100 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58120/0 (socket says 172.21.15.86:58120) 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 -- 172.21.15.86:0/3685741323 learned_addr learned my addr 172.21.15.86:0/3685741323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f677fe640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 0x7f5f68198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f67fff640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 0x7f5f6819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 -- 172.21.15.86:0/3685741323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 msgr2=0x7f5f68198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 0x7f5f68198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 -- 172.21.15.86:0/3685741323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 msgr2=0x7f5f6819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 0x7f5f6819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 -- 172.21.15.86:0/3685741323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6819d3f0 con 0x7f5f68100aa0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.974+0000 7f5f66ffd640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9f0 tx=0x7f5f5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.975+0000 7f5f64ff9640 1 -- 172.21.15.86:0/3685741323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00cd80 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.975+0000 7f5f64ff9640 1 -- 172.21.15.86:0/3685741323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f5c0054d0 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.975+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6819d6e0 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.975+0000 7f5f64ff9640 1 -- 172.21.15.86:0/3685741323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c010490 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.975+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a4fc0 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.977+0000 7f5f64ff9640 1 -- 172.21.15.86:0/3685741323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f5c005020 con 0x7f5f68100aa0 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.977+0000 7f5f7051e740 1 Processor -- start 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.977+0000 7f5f7051e740 1 -- start start 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 0x55a04880ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f64ff9640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 0x7f5f500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a04881c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a048770280 con 0x55a04880b380 2024-09-05T06:44:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a048717b10 con 0x55a04881a150 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f7051e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a04876c240 con 0x55a04880aa60 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58128/0 (socket says 172.21.15.86:58128) 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 -- 172.21.15.86:0/1253655439 learned_addr learned my addr 172.21.15.86:0/1253655439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 0x55a04880ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f66ffd640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 0x7f5f500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a04881c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f64ff9640 1 -- 172.21.15.86:0/3685741323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5f5c0cfb90 con 0x7f5f68100aa0 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 -- 172.21.15.86:0/1253655439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 msgr2=0x55a04880ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 0x55a04880ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 -- 172.21.15.86:0/1253655439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 msgr2=0x55a04881c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a04881c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f677fe640 1 -- 172.21.15.86:0/1253655439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0487241f0 con 0x55a04880b380 2024-09-05T06:44:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 0x55a04880ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.978+0000 7f5f66ffd640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 0x7f5f500a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0cd500 tx=0x7f5f5c0c6860 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.979+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009c70 tx=0x7f5f58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced359676fdb330f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.979+0000 7f5f4dffb640 1 -- 172.21.15.86:0/1253655439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58004230 con 0x55a04880b380 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.979+0000 7f5f4dffb640 1 -- 172.21.15.86:0/1253655439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f580043d0 con 0x55a04880b380 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.979+0000 7f5f4dffb640 1 -- 172.21.15.86:0/1253655439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58019640 con 0x55a04880b380 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- 172.21.15.86:0/1253655439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 msgr2=0x55a048819c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009c70 tx=0x7f5f58011810 comp rx=0 tx=0).stop 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- 172.21.15.86:0/1253655439 shutdown_connections 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a04881c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048819c10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1253655439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880aa60 0x55a04880ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- 172.21.15.86:0/1253655439 >> 172.21.15.86:0/1253655439 conn(0x55a0488070f0 msgr2=0x55a0488074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- 172.21.15.86:0/1253655439 shutdown_connections 2024-09-05T06:44:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- 172.21.15.86:0/1253655439 wait complete. 2024-09-05T06:44:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.980+0000 7f5f7051e740 1 -- start start 2024-09-05T06:44:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 Processor -- start 2024-09-05T06:44:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a048817010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048817550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04881a150 0x55a048817a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a04881cd80 con 0x55a04880b380 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a04872af10 con 0x55a04880aa60 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a04881cf00 con 0x55a04881a150 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f7051e740 10 client.0 _ll_register_callbacks cb 0x55a0486f3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048817550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048817550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58138/0 (socket says 172.21.15.86:58138) 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f677fe640 1 -- 172.21.15.86:0/1956919531 learned_addr learned my addr 172.21.15.86:0/1956919531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a048817010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04881a150 0x55a048817a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 -- 172.21.15.86:0/1956919531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04881a150 msgr2=0x55a048817a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04881a150 0x55a048817a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 -- 172.21.15.86:0/1956919531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 msgr2=0x55a048817550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048817550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.981+0000 7f5f67fff640 1 -- 172.21.15.86:0/1956919531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0484e16a0 con 0x55a04880aa60 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a048817010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f60006010 tx=0x7f5f600095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4252ed8c9b88dabe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600309d0 con 0x55a04880aa60 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f580043d0 con 0x55a04880aa60 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5f60030b70 con 0x55a04880aa60 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f60031840 con 0x55a04880aa60 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 10 client.25685 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0488289a0 con 0x55a04880aa60 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f580043d0 con 0x55a04880aa60 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5f6002f4c0 con 0x55a04880aa60 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 20 client.25685 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048829cf0 0x55a04882c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 4 client.25685 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.982+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04877d7f0 con 0x55a048829cf0 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.983+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048829cf0 0x55a04882c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.983+0000 7f5f46ffd640 10 client.25685 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.983+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048829cf0 0x55a04882c0e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614e207d1b370730 server_cookie=9532ea06ae7b9f9c in_seq=0 out_seq=0 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1956919531 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a04877d7f0 con 0x55a048829cf0 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f46ffd640 10 client.25685 handle_command_reply: tid=0 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 client.25685 shutdown 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 msgr2=0x55a048817010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a048817010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f60006010 tx=0x7f5f600095c0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 shutdown_connections 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048829cf0 0x55a04882c0e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04881a150 0x55a048817a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880b380 0x55a048817550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f46ffd640 0 client.25685 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1956919531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a048817010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 >> 172.21.15.86:0/1956919531 conn(0x55a0488070f0 msgr2=0x55a04880ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 shutdown_connections 2024-09-05T06:44:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 1 -- 172.21.15.86:0/1956919531 wait complete. 2024-09-05T06:44:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:42.984+0000 7f5f7051e740 20 client.25685 trim_cache size 0 max 16384 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.003+0000 7f5f7051e740 1 Processor -- start 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.003+0000 7f5f7051e740 1 -- start start 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a048831ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ace30 con 0x55a04880aa60 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ae9d0 con 0x55a04881a150 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f7051e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ad500 con 0x55a04880b380 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58148/0 (socket says 172.21.15.86:58148) 2024-09-05T06:44:43.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f67fff640 1 -- 172.21.15.86:0/3144329132 learned_addr learned my addr 172.21.15.86:0/3144329132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 -- 172.21.15.86:0/3144329132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 msgr2=0x55a048831ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a048831ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 -- 172.21.15.86:0/3144329132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 msgr2=0x55a048818e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 -- 172.21.15.86:0/3144329132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a048801680 con 0x55a04880b380 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f67fff640 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.004+0000 7f5f677fe640 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488319b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f58003e70 tx=0x7f5f580056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c4d532b0003717 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.005+0000 7f5f4d7fa640 1 -- 172.21.15.86:0/3144329132 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580097a0 con 0x55a04880b380 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.005+0000 7f5f4d7fa640 1 -- 172.21.15.86:0/3144329132 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f58004410 con 0x55a04880b380 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.005+0000 7f5f4d7fa640 1 -- 172.21.15.86:0/3144329132 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58003850 con 0x55a04880b380 2024-09-05T06:44:43.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- 172.21.15.86:0/3144329132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 msgr2=0x55a0488319b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488319b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f58003e70 tx=0x7f5f580056d0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- 172.21.15.86:0/3144329132 shutdown_connections 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04881a150 0x55a048831ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488319b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 --2- 172.21.15.86:0/3144329132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04880aa60 0x55a048818e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- 172.21.15.86:0/3144329132 >> 172.21.15.86:0/3144329132 conn(0x55a0488070f0 msgr2=0x55a048818760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- 172.21.15.86:0/3144329132 shutdown_connections 2024-09-05T06:44:43.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- 172.21.15.86:0/3144329132 wait complete. 2024-09-05T06:44:43.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.006+0000 7f5f7051e740 1 -- start start 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 Processor -- start 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04881a150 0x55a048814c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ae2a0 con 0x55a04881a150 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681adbd0 con 0x55a04880aa60 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681aae10 con 0x55a04880b380 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f7051e740 10 client.0 _ll_register_callbacks cb 0x7f5f600316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-09-05T06:44:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f67fff640 1 -- 172.21.15.86:0/1203515601 learned_addr learned my addr 172.21.15.86:0/1203515601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60762/0 (socket says 172.21.15.86:60762) 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.007+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04881a150 0x55a048814c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 -- 172.21.15.86:0/1203515601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 msgr2=0x55a0488146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488146f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 -- 172.21.15.86:0/1203515601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04881a150 msgr2=0x55a048814c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04881a150 0x55a048814c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 -- 172.21.15.86:0/1203515601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a048808280 con 0x55a04880aa60 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f67fff640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098a0 tx=0x7f5f60009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc7bdcc65dff8a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6005a070 con 0x55a04880aa60 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5f60030be0 con 0x55a04880aa60 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f58004410 con 0x55a04880aa60 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f60067050 con 0x55a04880aa60 2024-09-05T06:44:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f7051e740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.008+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a048894c70 con 0x55a04880aa60 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f58004410 con 0x55a04880aa60 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5f6004dcc0 con 0x55a04880aa60 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 20 client.25688 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048895340 0x55a048897760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 4 client.25688 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a048777d00 con 0x55a048895340 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048895340 0x55a048897760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f46ffd640 10 client.25688 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:43.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.009+0000 7f5f677fe640 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048895340 0x55a048897760 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ecf52043f142a5 server_cookie=9ba538ae89c61923 in_seq=0 out_seq=0 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f46ffd640 1 -- 172.21.15.86:0/1203515601 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a048777d00 con 0x55a048895340 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f46ffd640 10 client.25688 handle_command_reply: tid=0 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 client.25688 shutdown 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 msgr2=0x55a0488141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098a0 tx=0x7f5f60009360 comp rx=0 tx=0).stop 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 shutdown_connections 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a048895340 0x55a048897760 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f46ffd640 0 client.25688 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04881a150 0x55a048814c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a04880b380 0x55a0488146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 --2- 172.21.15.86:0/1203515601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a04880aa60 0x55a0488141b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.011+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 >> 172.21.15.86:0/1203515601 conn(0x55a0488070f0 msgr2=0x55a048806270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 shutdown_connections 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f7051e740 1 -- 172.21.15.86:0/1203515601 wait complete. 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f7051e740 20 client.25688 trim_cache size 0 max 16384 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2042aedd-4e3d-4623-8753-91cc7d95f239", 2024-09-05T06:44:43.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:44:43.016 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:44:43.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 msgr2=0x7f5f500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:43.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 0x7f5f500a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0cd500 tx=0x7f5f5c0c6860 comp rx=0 tx=0).stop 2024-09-05T06:44:43.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 msgr2=0x7f5f68198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.012+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9f0 tx=0x7f5f5c00edf0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 shutdown_connections 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f500a2cf0 0x7f5f500a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f68106b80 0x7f5f6819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68100aa0 0x7f5f68198980 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 --2- 172.21.15.86:0/3685741323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68100100 0x7f5f68198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 >> 172.21.15.86:0/3685741323 conn(0x7f5f68077230 msgr2=0x7f5f68104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 shutdown_connections 2024-09-05T06:44:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:43.013+0000 7f5f6e145640 1 -- 172.21.15.86:0/3685741323 wait complete. 2024-09-05T06:44:43.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57048 4.38 MB/sec execute 3 sec latency 41.390 ms 2024-09-05T06:44:43.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:43 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:44:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:43 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:43 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:44:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:43 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:43.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:43 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:44:43.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:43 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:43.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:43 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:44:43.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:43 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:43 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:44:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:43 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:43 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:44:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:43 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:44:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:44:44.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:44 smithi086 ceph-mon[30943]: pgmap v107: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:44.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57322 3.37 MB/sec execute 4 sec latency 18.841 ms 2024-09-05T06:44:44.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:44 smithi096 ceph-mon[34468]: pgmap v107: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 -- 172.21.15.86:0/2720685825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 msgr2=0x7fb06c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 --2- 172.21.15.86:0/2720685825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c10ec30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb064009cd0 tx=0x7fb06402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 -- 172.21.15.86:0/2720685825 shutdown_connections 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 --2- 172.21.15.86:0/2720685825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c10ec30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 --2- 172.21.15.86:0/2720685825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c103740 0x7fb06c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 --2- 172.21.15.86:0/2720685825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c102d90 0x7fb06c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 -- 172.21.15.86:0/2720685825 >> 172.21.15.86:0/2720685825 conn(0x7fb06c0fc7d0 msgr2=0x7fb06c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.577+0000 7fb071cf3640 1 -- 172.21.15.86:0/2720685825 shutdown_connections 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.578+0000 7fb071cf3640 1 -- 172.21.15.86:0/2720685825 wait complete. 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 Processor -- start 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 -- start start 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 0x7fb06c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 0x7fb06c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c111f40 con 0x7fb06c108290 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c111dc0 con 0x7fb06c102d90 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.579+0000 7fb071cf3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1120c0 con 0x7fb06c103740 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:44.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 0x7fb06c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 0x7fb06c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58170/0 (socket says 172.21.15.86:58170) 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06bfff640 1 -- 172.21.15.86:0/169492824 learned_addr learned my addr 172.21.15.86:0/169492824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 -- 172.21.15.86:0/169492824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 msgr2=0x7fb06c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 0x7fb06c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 -- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 msgr2=0x7fb06c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 -- 172.21.15.86:0/169492824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19d440 con 0x7fb06c102d90 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06bfff640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06affd640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 0x7fb06c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb06b7fe640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 0x7fb06c198490 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9f0 tx=0x7fb05c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00ccc0 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c19d730 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb05c005020 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004a00 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.580+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a5010 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.582+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c103170 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.583+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb05c004ba0 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.583+0000 7fb068ff9640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 0x7fb0540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.583+0000 7fb06affd640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 0x7fb0540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.583+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fb05c0cf820 con 0x7fb06c102d90 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.584+0000 7fb06affd640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 0x7fb0540a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0fee20 tx=0x7fb060005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.588+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb05c09c000 con 0x7fb06c102d90 2024-09-05T06:44:44.659 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:44:44.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:44:44.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:44 smithi115 ceph-mon[34791]: pgmap v107: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.829+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb06c1997f0 con 0x7fb06c102d90 2024-09-05T06:44:44.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.830+0000 7fb068ff9640 1 -- 172.21.15.86:0/169492824 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7fb05c0afe20 con 0x7fb06c102d90 2024-09-05T06:44:44.830 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:44:44.830 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:44:44.830 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 msgr2=0x7fb0540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 0x7fb0540a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0fee20 tx=0x7fb060005f90 comp rx=0 tx=0).stop 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 msgr2=0x7fb06c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 0x7fb06c198490 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9f0 tx=0x7fb05c00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 shutdown_connections 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0540a2cf0 0x7fb0540a51b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c108290 0x7fb06c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb06c103740 0x7fb06c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 --2- 172.21.15.86:0/169492824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102d90 0x7fb06c198490 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:44.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 >> 172.21.15.86:0/169492824 conn(0x7fb06c0fc7d0 msgr2=0x7fb06c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:44.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 shutdown_connections 2024-09-05T06:44:44.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:44.832+0000 7fb071cf3640 1 -- 172.21.15.86:0/169492824 wait complete. 2024-09-05T06:44:45.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57836 3.08 MB/sec execute 5 sec latency 13.602 ms 2024-09-05T06:44:45.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:45 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/169492824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:45 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/169492824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:45.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:45 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/169492824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:44:45.869 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T06:44:45.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.063+0000 7f9753d36640 1 -- 172.21.15.86:0/1397237226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 msgr2=0x7f974c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.063+0000 7f9753d36640 1 --2- 172.21.15.86:0/1397237226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c1029b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f973c009900 tx=0x7f973c02a480 comp rx=0 tx=0).stop 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.063+0000 7f9753d36640 1 -- 172.21.15.86:0/1397237226 shutdown_connections 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.063+0000 7f9753d36640 1 --2- 172.21.15.86:0/1397237226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c102ef0 0x7f974c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.063+0000 7f9753d36640 1 --2- 172.21.15.86:0/1397237226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c1029b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.064+0000 7f9753d36640 1 --2- 172.21.15.86:0/1397237226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c100e20 0x7f974c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.064+0000 7f9753d36640 1 -- 172.21.15.86:0/1397237226 >> 172.21.15.86:0/1397237226 conn(0x7f974c0fc7d0 msgr2=0x7f974c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.064+0000 7f9753d36640 1 -- 172.21.15.86:0/1397237226 shutdown_connections 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.064+0000 7f9753d36640 1 -- 172.21.15.86:0/1397237226 wait complete. 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.064+0000 7f9753d36640 1 Processor -- start 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 -- start start 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 0x7f974c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 0x7f974c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c110e80 con 0x7f974c102570 2024-09-05T06:44:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97512d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97512d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58184/0 (socket says 172.21.15.86:58184) 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97512d1640 1 -- 172.21.15.86:0/1720118357 learned_addr learned my addr 172.21.15.86:0/1720118357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9751ad2640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 0x7f974c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f974c110d00 con 0x7f974c100e20 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 0x7f974c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f974c111000 con 0x7f974c102ef0 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 -- 172.21.15.86:0/1720118357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 msgr2=0x7f974c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 0x7f974c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 -- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 msgr2=0x7f974c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.065+0000 7f97522d3640 1 -- 172.21.15.86:0/1720118357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19d390 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9751ad2640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 0x7f974c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f97522d3640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 0x7f974c19ccb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9744009fd0 tx=0x7f974400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9742ffd640 1 -- 172.21.15.86:0/1720118357 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974400ce20 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9742ffd640 1 -- 172.21.15.86:0/1720118357 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9744005020 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19d680 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9742ffd640 1 -- 172.21.15.86:0/1720118357 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744010690 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.066+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a4f60 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.068+0000 7f9742ffd640 1 -- 172.21.15.86:0/1720118357 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f97440051d0 con 0x7f974c102ef0 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f975610f740 1 Processor -- start 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f975610f740 1 -- start start 2024-09-05T06:44:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f9742ffd640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 0x7f97380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f975610f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe7f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f975610f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.069+0000 7f975610f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe8ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f975610f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595fbde52e0 con 0x5595fbe8a680 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f975610f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595fbd8cce0 con 0x5595fbe7fdc0 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f975610f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595fbde12a0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f9742ffd640 1 -- 172.21.15.86:0/1720118357 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f97440cfef0 con 0x7f974c102ef0 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 0x7f97380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52120/0 (socket says 172.21.15.86:52120) 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 -- 172.21.15.86:0/3983061331 learned_addr learned my addr 172.21.15.86:0/3983061331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97522d3640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe8ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f9751ad2640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe7f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 -- 172.21.15.86:0/3983061331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 msgr2=0x5595fbe7f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe7f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 -- 172.21.15.86:0/3983061331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 msgr2=0x5595fbe8ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe8ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 -- 172.21.15.86:0/3983061331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fbd71440 con 0x5595fbe7fdc0 2024-09-05T06:44:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97522d3640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe8ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 0x7f97380a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f973c004650 tx=0x7f973c009680 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.070+0000 7f97512d1640 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f973c02aa70 tx=0x7f973c046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cbd999e346c9d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f9737fff640 1 -- 172.21.15.86:0/3983061331 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c04e070 con 0x5595fbe7fdc0 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f9737fff640 1 -- 172.21.15.86:0/3983061331 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f973c003e60 con 0x5595fbe7fdc0 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f9737fff640 1 -- 172.21.15.86:0/3983061331 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c005290 con 0x5595fbe7fdc0 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f975610f740 1 -- 172.21.15.86:0/3983061331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 msgr2=0x5595fbe8a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f975610f740 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f973c02aa70 tx=0x7f973c046000 comp rx=0 tx=0).stop 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f975610f740 1 -- 172.21.15.86:0/3983061331 shutdown_connections 2024-09-05T06:44:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.071+0000 7f975610f740 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe8ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7fdc0 0x5595fbe8a140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 --2- 172.21.15.86:0/3983061331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe7f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 -- 172.21.15.86:0/3983061331 >> 172.21.15.86:0/3983061331 conn(0x5595fbe7ba30 msgr2=0x5595fbe7be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 -- 172.21.15.86:0/3983061331 shutdown_connections 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 -- 172.21.15.86:0/3983061331 wait complete. 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 -- start start 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.072+0000 7f975610f740 1 Processor -- start 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe89f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7fdc0 0x5595fbe87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595fbe8d250 con 0x5595fbe8a680 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595fbda0050 con 0x5595fbe7f4a0 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595fbe8d3d0 con 0x5595fbe7fdc0 2024-09-05T06:44:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f97522d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe89f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f97522d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe87780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58204/0 (socket says 172.21.15.86:58204) 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f97522d3640 1 -- 172.21.15.86:0/4230512824 learned_addr learned my addr 172.21.15.86:0/4230512824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f97512d1640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7fdc0 0x5595fbe87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f975610f740 10 client.0 _ll_register_callbacks cb 0x5595fbde0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 -- 172.21.15.86:0/4230512824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7fdc0 msgr2=0x5595fbe87240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7fdc0 0x5595fbe87240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 -- 172.21.15.86:0/4230512824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 msgr2=0x5595fbe87780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe87780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 -- 172.21.15.86:0/4230512824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fbb563c0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f9751ad2640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe89f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9748011740 tx=0x7f9748011c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa80f2a265a97b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.073+0000 7f97357fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748010ba0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9748010d40 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973c003e60 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9748024050 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 10 client.25706 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595fbe98b90 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f973c003e60 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9748004c80 con 0x5595fbe7f4a0 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 10 client.25706 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 20 client.25706 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbe9a0a0 0x5595fbe9c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 4 client.25706 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.074+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595fbdf2850 con 0x5595fbe9a0a0 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.075+0000 7f97512d1640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbe9a0a0 0x5595fbe9c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.075+0000 7f97357fa640 10 client.25706 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.075+0000 7f97512d1640 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbe9a0a0 0x5595fbe9c530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8d903547b92902 server_cookie=6dfde9ca3a09ab75 in_seq=0 out_seq=0 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f97357fa640 1 -- 172.21.15.86:0/4230512824 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595fbdf2850 con 0x5595fbe9a0a0 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f97357fa640 10 client.25706 handle_command_reply: tid=0 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 client.25706 shutdown 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 msgr2=0x5595fbe89f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe89f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9748011740 tx=0x7f9748011c00 comp rx=0 tx=0).stop 2024-09-05T06:44:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 shutdown_connections 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbe9a0a0 0x5595fbe9c530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe8a680 0x5595fbe87780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7fdc0 0x5595fbe87240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f97357fa640 0 client.25706 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 --2- 172.21.15.86:0/4230512824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe89f90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 >> 172.21.15.86:0/4230512824 conn(0x5595fbe7ba30 msgr2=0x5595fbe81760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 shutdown_connections 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 1 -- 172.21.15.86:0/4230512824 wait complete. 2024-09-05T06:44:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.076+0000 7f975610f740 20 client.25706 trim_cache size 0 max 16384 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.096+0000 7f975610f740 1 Processor -- start 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 -- start start 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1ace00 con 0x5595fbe7fdc0 2024-09-05T06:44:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1ae9a0 con 0x5595fbe8a680 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f975610f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1ad4d0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97512d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f9751ad2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52146/0 (socket says 172.21.15.86:52146) 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 -- 172.21.15.86:0/2471768807 learned_addr learned my addr 172.21.15.86:0/2471768807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97512d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58210/0 (socket says 172.21.15.86:58210) 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f9751ad2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60844/0 (socket says 172.21.15.86:60844) 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 -- 172.21.15.86:0/2471768807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 msgr2=0x5595fbe964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 -- 172.21.15.86:0/2471768807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 msgr2=0x5595fbe7aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97522d3640 1 -- 172.21.15.86:0/2471768807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fbe7fbf0 con 0x5595fbe8a680 2024-09-05T06:44:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.097+0000 7f97512d1640 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.098+0000 7f97522d3640 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f974409bd30 tx=0x7f97440bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b07c4f5aa0ae99e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.098+0000 7f97377fe640 1 -- 172.21.15.86:0/2471768807 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744005540 con 0x5595fbe8a680 2024-09-05T06:44:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.098+0000 7f97377fe640 1 -- 172.21.15.86:0/2471768807 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974407ae50 con 0x5595fbe8a680 2024-09-05T06:44:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.098+0000 7f97377fe640 1 -- 172.21.15.86:0/2471768807 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97440a4a50 con 0x5595fbe8a680 2024-09-05T06:44:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 -- 172.21.15.86:0/2471768807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 msgr2=0x5595fbe92620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f974409bd30 tx=0x7f97440bc040 comp rx=0 tx=0).stop 2024-09-05T06:44:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 -- 172.21.15.86:0/2471768807 shutdown_connections 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe8a680 0x5595fbe92620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 --2- 172.21.15.86:0/2471768807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe7f4a0 0x5595fbe964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 -- 172.21.15.86:0/2471768807 >> 172.21.15.86:0/2471768807 conn(0x5595fbe7ba30 msgr2=0x5595fbe7a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 -- 172.21.15.86:0/2471768807 shutdown_connections 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.099+0000 7f975610f740 1 -- 172.21.15.86:0/2471768807 wait complete. 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 -- start start 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 Processor -- start 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe90b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe8a680 0x5595fbe91070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f975610f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1ae270 con 0x5595fbe7fdc0 2024-09-05T06:44:46.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.100+0000 7f9751ad2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f97512d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe90b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52152/0 (socket says 172.21.15.86:52152) 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 -- 172.21.15.86:0/1011009681 learned_addr learned my addr 172.21.15.86:0/1011009681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1adba0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1aade0 con 0x5595fbe8a680 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f97522d3640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe8a680 0x5595fbe91070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 -- 172.21.15.86:0/1011009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe8a680 msgr2=0x5595fbe91070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe8a680 0x5595fbe91070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 -- 172.21.15.86:0/1011009681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 msgr2=0x5595fbe90b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f975610f740 10 client.0 _ll_register_callbacks cb 0x7f974800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe90b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fbf04030 con 0x5595fbe7f4a0 2024-09-05T06:44:46.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f97512d1640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe90b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974407ae50 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f9751ad2640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f974801a530 tx=0x7f9748010dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102809882873ab89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.101+0000 7f97357fa640 10 client.25712 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f97357fa640 1 -- 172.21.15.86:0/1011009681 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974802e020 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f97357fa640 1 -- 172.21.15.86:0/1011009681 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9748005250 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f97357fa640 1 -- 172.21.15.86:0/1011009681 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97480054b0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 10 client.25712 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5595fbf051c0 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f97357fa640 1 -- 172.21.15.86:0/1011009681 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f974803b020 con 0x5595fbe7f4a0 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 10 client.25712 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 20 client.25712 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:46.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbf05770 0x5595fbf07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 4 client.25712 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595fbdecd60 con 0x5595fbf05770 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.102+0000 7f97512d1640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbf05770 0x5595fbf07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f97357fa640 10 client.25712 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f97512d1640 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbf05770 0x5595fbf07c00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948f01701f151e4d server_cookie=b99c5938a8dcc10f in_seq=0 out_seq=0 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f97357fa640 1 -- 172.21.15.86:0/1011009681 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5595fbdecd60 con 0x5595fbf05770 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f97357fa640 10 client.25712 handle_command_reply: tid=0 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f975610f740 1 client.25712 shutdown 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.103+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 msgr2=0x5595fbe905f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f974801a530 tx=0x7f9748010dd0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 shutdown_connections 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595fbf05770 0x5595fbf07c00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595fbe8a680 0x5595fbe91070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f97357fa640 0 client.25712 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595fbe7fdc0 0x5595fbe90b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 --2- 172.21.15.86:0/1011009681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595fbe7f4a0 0x5595fbe905f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 >> 172.21.15.86:0/1011009681 conn(0x5595fbe7ba30 msgr2=0x5595fbe7ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 shutdown_connections 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 1 -- 172.21.15.86:0/1011009681 wait complete. 2024-09-05T06:44:46.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.104+0000 7f975610f740 20 client.25712 trim_cache size 0 max 16384 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "2042aedd-4e3d-4623-8753-91cc7d95f239": { 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2042aedd-4e3d-4623-8753-91cc7d95f239", 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:46.110 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.105+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 msgr2=0x7f97380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.105+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 0x7f97380a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f973c004650 tx=0x7f973c009680 comp rx=0 tx=0).stop 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.105+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 msgr2=0x7f974c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.105+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 0x7f974c19ccb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9744009fd0 tx=0x7f974400ef10 comp rx=0 tx=0).stop 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 shutdown_connections 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f97380a2cf0 0x7f97380a51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c102ef0 0x7f974c19ccb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c102570 0x7f974c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 --2- 172.21.15.86:0/1720118357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f974c100e20 0x7f974c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 >> 172.21.15.86:0/1720118357 conn(0x7f974c0fc7d0 msgr2=0x7f974c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 shutdown_connections 2024-09-05T06:44:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:46.106+0000 7f9753d36640 1 -- 172.21.15.86:0/1720118357 wait complete. 2024-09-05T06:44:46.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:46 smithi086 ceph-mon[30943]: pgmap v108: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:46.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 58430 3.12 MB/sec execute 6 sec latency 17.651 ms 2024-09-05T06:44:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:46 smithi096 ceph-mon[34468]: pgmap v108: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:46.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:46 smithi115 ceph-mon[34791]: pgmap v108: 129 pgs: 129 active+clean; 59 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:44:47.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 59142 2.91 MB/sec execute 7 sec latency 17.258 ms 2024-09-05T06:44:47.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:47 smithi086 ceph-mon[30943]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-05T06:44:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:47 smithi096 ceph-mon[34468]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-05T06:44:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:47 smithi115 ceph-mon[34791]: pgmap v109: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.9 MiB/s wr, 199 op/s 2024-09-05T06:44:48.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 59765 3.25 MB/sec execute 8 sec latency 72.867 ms 2024-09-05T06:44:49.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60291 3.43 MB/sec execute 9 sec latency 73.205 ms 2024-09-05T06:44:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:49 smithi096 ceph-mon[34468]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-05T06:44:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:49 smithi115 ceph-mon[34791]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-05T06:44:49.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:49 smithi086 ceph-mon[30943]: pgmap v110: 129 pgs: 129 active+clean; 63 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-09-05T06:44:50.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60651 3.23 MB/sec execute 10 sec latency 41.402 ms 2024-09-05T06:44:50.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:50 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:50.890 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:50 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:44:50.928 INFO:tasks.cephfs.filesystem:scrub status for tag:2042aedd-4e3d-4623-8753-91cc7d95f239 - {'path': '/', 'tag': '2042aedd-4e3d-4623-8753-91cc7d95f239', 'options': 'recursive,force'} 2024-09-05T06:44:50.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:50 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 -- 172.21.15.86:0/2541398833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0102dd0 msgr2=0x7f24a01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/2541398833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0102dd0 0x7f24a01031b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2490009930 tx=0x7f249002a420 comp rx=0 tx=0).stop 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 -- 172.21.15.86:0/2541398833 shutdown_connections 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/2541398833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/2541398833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0103780 0x7f24a0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/2541398833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0102dd0 0x7f24a01031b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 -- 172.21.15.86:0/2541398833 >> 172.21.15.86:0/2541398833 conn(0x7f24a00fc7d0 msgr2=0x7f24a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 -- 172.21.15.86:0/2541398833 shutdown_connections 2024-09-05T06:44:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.390+0000 7f24a4dd7640 1 -- 172.21.15.86:0/2541398833 wait complete. 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 Processor -- start 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 -- start start 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 0x7f24a01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111f60 con 0x7f24a0103780 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111de0 con 0x7f24a0102dd0 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f24a4dd7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01120e0 con 0x7f24a0108300 2024-09-05T06:44:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f249dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f249e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 0x7f24a01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f249dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58222/0 (socket says 172.21.15.86:58222) 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f249dd9b640 1 -- 172.21.15.86:0/3176113764 learned_addr learned my addr 172.21.15.86:0/3176113764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.391+0000 7f249e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 0x7f24a01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52174/0 (socket says 172.21.15.86:52174) 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249ed9d640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 -- 172.21.15.86:0/3176113764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 msgr2=0x7f24a019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 -- 172.21.15.86:0/3176113764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 msgr2=0x7f24a01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 0x7f24a01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 -- 172.21.15.86:0/3176113764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a019d490 con 0x7f24a0103780 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249ed9d640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f249dd9b640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f249400e9f0 tx=0x7f249400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f248f7fe640 1 -- 172.21.15.86:0/3176113764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249400ccc0 con 0x7f24a0103780 2024-09-05T06:44:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f248f7fe640 1 -- 172.21.15.86:0/3176113764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24940108e0 con 0x7f24a0103780 2024-09-05T06:44:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a019d780 con 0x7f24a0103780 2024-09-05T06:44:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f248f7fe640 1 -- 172.21.15.86:0/3176113764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940187e0 con 0x7f24a0103780 2024-09-05T06:44:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.392+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01a5060 con 0x7f24a0103780 2024-09-05T06:44:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.394+0000 7f248f7fe640 1 -- 172.21.15.86:0/3176113764 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2494010a90 con 0x7f24a0103780 2024-09-05T06:44:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.395+0000 7f24a71b0740 1 Processor -- start 2024-09-05T06:44:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.395+0000 7f24a71b0740 1 -- start start 2024-09-05T06:44:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.395+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.395+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cd54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509cd7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f24a71b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556509c2f160 con 0x556509ccb170 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f24a71b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556509bd6720 con 0x556509cd5a30 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f24a71b0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556509c2b120 con 0x556509cca850 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.395+0000 7f248f7fe640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 0x7f24880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cd54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60872/0 (socket says 172.21.15.86:60872) 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cd54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58236/0 (socket says 172.21.15.86:58236) 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 -- 172.21.15.86:0/1594665703 learned_addr learned my addr 172.21.15.86:0/1594665703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249dd9b640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 0x7f24880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509cd7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f248f7fe640 1 -- 172.21.15.86:0/3176113764 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f24940cfc10 con 0x7f24a0103780 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 -- 172.21.15.86:0/1594665703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 msgr2=0x556509cd7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509cd7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 -- 172.21.15.86:0/1594665703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 msgr2=0x556509cd54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cd54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 -- 172.21.15.86:0/1594665703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565099a0ba0 con 0x556509cca850 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249dd9b640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 0x7f24880a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f249409b9f0 tx=0x7f2494009710 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.396+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2498009fd0 tx=0x7f2498009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4a0d11901b8325 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f248cff9640 1 -- 172.21.15.86:0/1594665703 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498010a60 con 0x556509cca850 2024-09-05T06:44:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f248cff9640 1 -- 172.21.15.86:0/1594665703 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2498023070 con 0x556509cca850 2024-09-05T06:44:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f248cff9640 1 -- 172.21.15.86:0/1594665703 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249800e610 con 0x556509cca850 2024-09-05T06:44:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 -- 172.21.15.86:0/1594665703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 msgr2=0x556509ccac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2498009fd0 tx=0x7f2498009a60 comp rx=0 tx=0).stop 2024-09-05T06:44:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 -- 172.21.15.86:0/1594665703 shutdown_connections 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509cd7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cd54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1594665703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509ccac30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.397+0000 7f24a71b0740 1 -- 172.21.15.86:0/1594665703 >> 172.21.15.86:0/1594665703 conn(0x556509cc6de0 msgr2=0x556509cc71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- 172.21.15.86:0/1594665703 shutdown_connections 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- 172.21.15.86:0/1594665703 wait complete. 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- start start 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 Processor -- start 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509cd33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556509cd8600 con 0x556509cd5a30 2024-09-05T06:44:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556509bb7900 con 0x556509ccb170 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556509cd8780 con 0x556509cca850 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.398+0000 7f24a71b0740 10 client.0 _ll_register_callbacks cb 0x556509bb5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52184/0 (socket says 172.21.15.86:52184) 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 -- 172.21.15.86:0/2237394924 learned_addr learned my addr 172.21.15.86:0/2237394924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249ed9d640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509cd33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 -- 172.21.15.86:0/2237394924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 msgr2=0x556509cd2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 -- 172.21.15.86:0/2237394924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 msgr2=0x556509cd33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509cd33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 -- 172.21.15.86:0/2237394924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556509bf7c50 con 0x556509ccb170 2024-09-05T06:44:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249ed9d640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f249e59c640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f249002a940 tx=0x7f2490031ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcefedb7e436399 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f2485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490035040 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2498023070 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2490031720 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.399+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2490004040 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 10 client.25724 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556509ce4350 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2498023070 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2490046480 con 0x556509ccb170 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 10 client.25724 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 10 client.25724 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:44:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 20 client.25724 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509ce7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 4 client.25724 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556509c3c890 con 0x556509ce5800 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.400+0000 7f249ed9d640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509ce7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f249ed9d640 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509ce7bf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422a05bea1176f08 server_cookie=d8573bc92cd2d326 in_seq=0 out_seq=0 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f2485ffb640 10 client.25724 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f2485ffb640 1 -- 172.21.15.86:0/2237394924 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556509c3c890 con 0x556509ce5800 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f2485ffb640 10 client.25724 handle_command_reply: tid=0 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 client.25724 shutdown 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 msgr2=0x556509cd2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f249002a940 tx=0x7f2490031ef0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 shutdown_connections 2024-09-05T06:44:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509ce7bf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509cd33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509cd2eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f2485ffb640 0 client.25724 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 --2- 172.21.15.86:0/2237394924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.401+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 >> 172.21.15.86:0/2237394924 conn(0x556509cc6de0 msgr2=0x556509cccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.402+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 shutdown_connections 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.402+0000 7f24a71b0740 1 -- 172.21.15.86:0/2237394924 wait complete. 2024-09-05T06:44:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.402+0000 7f24a71b0740 20 client.25724 trim_cache size 0 max 16384 2024-09-05T06:44:51.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 Processor -- start 2024-09-05T06:44:51.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 -- start start 2024-09-05T06:44:51.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509ced4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509ceda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01acf10 con 0x556509cd5a30 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01aeab0 con 0x556509ccb170 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.419+0000 7f24a71b0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01ad5e0 con 0x556509cca850 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509ceda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60888/0 (socket says 172.21.15.86:60888) 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 -- 172.21.15.86:0/1997416740 learned_addr learned my addr 172.21.15.86:0/1997416740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249e59c640 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509ced4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 -- 172.21.15.86:0/1997416740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 msgr2=0x556509ced4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509ced4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 -- 172.21.15.86:0/1997416740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 msgr2=0x556509ceda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509ceda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 -- 172.21.15.86:0/1997416740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556509cc23f0 con 0x556509cca850 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f249ed9d640 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2498009a30 tx=0x7f2498017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b948ef97e408da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f2487fff640 1 -- 172.21.15.86:0/1997416740 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498010a60 con 0x556509cca850 2024-09-05T06:44:51.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f2487fff640 1 -- 172.21.15.86:0/1997416740 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2498023040 con 0x556509cca850 2024-09-05T06:44:51.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.420+0000 7f2487fff640 1 -- 172.21.15.86:0/1997416740 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498034b50 con 0x556509cca850 2024-09-05T06:44:51.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- 172.21.15.86:0/1997416740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 msgr2=0x556509cd3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2498009a30 tx=0x7f2498017040 comp rx=0 tx=0).stop 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- 172.21.15.86:0/1997416740 shutdown_connections 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509cd5a30 0x556509ceda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509ccb170 0x556509ced4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 --2- 172.21.15.86:0/1997416740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd3240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- 172.21.15.86:0/1997416740 >> 172.21.15.86:0/1997416740 conn(0x556509cc6de0 msgr2=0x556509cd49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- 172.21.15.86:0/1997416740 shutdown_connections 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- 172.21.15.86:0/1997416740 wait complete. 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.421+0000 7f24a71b0740 1 -- start start 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 Processor -- start 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509d4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01ae380 con 0x556509ccb170 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01adcb0 con 0x556509cd5a30 2024-09-05T06:44:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01aaef0 con 0x556509cca850 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f24a71b0740 10 client.0 _ll_register_callbacks cb 0x7f2490009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 -- 172.21.15.86:0/717297457 learned_addr learned my addr 172.21.15.86:0/717297457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249ed9d640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249ed9d640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509d4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 -- 172.21.15.86:0/717297457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 msgr2=0x556509cd9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 -- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 msgr2=0x556509d4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509d4f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249e59c640 1 -- 172.21.15.86:0/717297457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556509d4fa50 con 0x556509ccb170 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.422+0000 7f249ed9d640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f249e59c640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2490004720 tx=0x7f2490000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5c31597fe7c12c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:44:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490035070 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f24900041e0 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2498023040 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24900043e0 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f24a71b0740 10 client.16353 fetch_fsmap learned FSMap version 30 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556509d50d60 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2498023040 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2490031840 con 0x556509ccb170 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f24a71b0740 10 client.16353 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.423+0000 7f24a71b0740 10 client.16353 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f24a71b0740 20 client.16353 populate_metadata read hostname 'smithi086' 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509d53670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:44:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f24a71b0740 4 client.16353 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T06:44:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x556509c36be0 con 0x556509ce5800 2024-09-05T06:44:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f249ed9d640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509d53670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:44:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f2485ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.424+0000 7f249ed9d640 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509d53670 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba40a6a06bf71822 server_cookie=d235cdbb4bd7d2d0 in_seq=0 out_seq=0 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f2485ffb640 1 -- 172.21.15.86:0/717297457 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556509c36be0 con 0x556509ce5800 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f2485ffb640 10 client.16353 handle_command_reply: tid=0 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 client.16353 shutdown 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 msgr2=0x556509cda3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2490004720 tx=0x7f2490000bc0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 shutdown_connections 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556509ce5800 0x556509d53670 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556509cd5a30 0x556509d4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556509ccb170 0x556509cda3e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 --2- 172.21.15.86:0/717297457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556509cca850 0x556509cd9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 >> 172.21.15.86:0/717297457 conn(0x556509cc6de0 msgr2=0x556509cc56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f2485ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 shutdown_connections 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.487+0000 7f24a71b0740 1 -- 172.21.15.86:0/717297457 wait complete. 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a71b0740 20 client.16353 trim_cache size 0 max 16384 2024-09-05T06:44:51.490 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 30, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 10, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "51df3dec": { 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 10, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 141.8, 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:44:51.491 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:44:51.492 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 msgr2=0x7f24880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 0x7f24880a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f249409b9f0 tx=0x7f2494009710 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 msgr2=0x7f24a0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f249400e9f0 tx=0x7f249400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 shutdown_connections 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24880a2d70 0x7f24880a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24a0108300 0x7f24a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a0103780 0x7f24a0198a20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 --2- 172.21.15.86:0/3176113764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a0102dd0 0x7f24a01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:44:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 >> 172.21.15.86:0/3176113764 conn(0x7f24a00fc7d0 msgr2=0x7f24a0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:44:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 shutdown_connections 2024-09-05T06:44:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:44:51.488+0000 7f24a4dd7640 1 -- 172.21.15.86:0/3176113764 wait complete. 2024-09-05T06:44:51.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.96 MB/sec execute 11 sec latency 49.635 ms 2024-09-05T06:44:51.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:44:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:51 smithi086 ceph-mon[30943]: pgmap v111: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T06:44:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:44:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:51 smithi096 ceph-mon[34468]: pgmap v111: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T06:44:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:44:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:51 smithi115 ceph-mon[34791]: pgmap v111: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T06:44:52.273 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '51df3dec', quiesce took 9.6 seconds. Will release after: 12.15 2024-09-05T06:44:52.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.72 MB/sec execute 12 sec latency 1049.734 ms 2024-09-05T06:44:53.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.51 MB/sec execute 13 sec latency 2049.783 ms 2024-09-05T06:44:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:53 smithi096 ceph-mon[34468]: pgmap v112: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:53 smithi115 ceph-mon[34791]: pgmap v112: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:53 smithi086 ceph-mon[30943]: pgmap v112: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:54.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.33 MB/sec execute 14 sec latency 3049.890 ms 2024-09-05T06:44:55.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.17 MB/sec execute 15 sec latency 4049.992 ms 2024-09-05T06:44:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:55 smithi096 ceph-mon[34468]: pgmap v113: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:55 smithi115 ceph-mon[34791]: pgmap v113: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:55.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:55 smithi086 ceph-mon[30943]: pgmap v113: 129 pgs: 129 active+clean; 65 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:44:56.509 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 2.04 MB/sec execute 16 sec latency 5050.083 ms 2024-09-05T06:44:57.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.92 MB/sec execute 17 sec latency 6050.217 ms 2024-09-05T06:44:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:57 smithi096 ceph-mon[34468]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:44:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:57 smithi115 ceph-mon[34791]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:44:57.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:57 smithi086 ceph-mon[30943]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-09-05T06:44:58.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.81 MB/sec execute 18 sec latency 7050.315 ms 2024-09-05T06:44:59.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.71 MB/sec execute 19 sec latency 8050.365 ms 2024-09-05T06:44:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:44:59 smithi096 ceph-mon[34468]: pgmap v115: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 673 KiB/s wr, 78 op/s 2024-09-05T06:44:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:44:59 smithi115 ceph-mon[34791]: pgmap v115: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 673 KiB/s wr, 78 op/s 2024-09-05T06:44:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:44:59 smithi086 ceph-mon[30943]: pgmap v115: 129 pgs: 129 active+clean; 62 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 673 KiB/s wr, 78 op/s 2024-09-05T06:45:00.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.63 MB/sec execute 20 sec latency 9050.450 ms 2024-09-05T06:45:01.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.55 MB/sec execute 21 sec latency 10050.496 ms 2024-09-05T06:45:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:01 smithi096 ceph-mon[34468]: pgmap v116: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 700 KiB/s wr, 98 op/s 2024-09-05T06:45:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:01 smithi115 ceph-mon[34791]: pgmap v116: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 700 KiB/s wr, 98 op/s 2024-09-05T06:45:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:01 smithi086 ceph-mon[30943]: pgmap v116: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 700 KiB/s wr, 98 op/s 2024-09-05T06:45:02.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.48 MB/sec execute 22 sec latency 11050.604 ms 2024-09-05T06:45:03.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.42 MB/sec execute 23 sec latency 12050.701 ms 2024-09-05T06:45:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:03 smithi096 ceph-mon[34468]: pgmap v117: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:03 smithi115 ceph-mon[34791]: pgmap v117: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:03 smithi086 ceph-mon[30943]: pgmap v117: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:04.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:45:04.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.36 MB/sec execute 24 sec latency 13050.754 ms 2024-09-05T06:45:04.682 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 -- 172.21.15.86:0/4285631617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 msgr2=0x7f5a54107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 --2- 172.21.15.86:0/4285631617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a54107010 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009900 tx=0x7f5a4402a410 comp rx=0 tx=0).stop 2024-09-05T06:45:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 -- 172.21.15.86:0/4285631617 shutdown_connections 2024-09-05T06:45:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 --2- 172.21.15.86:0/4285631617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a5410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 --2- 172.21.15.86:0/4285631617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 0x7f5a54101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 --2- 172.21.15.86:0/4285631617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a54107010 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 -- 172.21.15.86:0/4285631617 >> 172.21.15.86:0/4285631617 conn(0x7f5a54077210 msgr2=0x7f5a540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 -- 172.21.15.86:0/4285631617 shutdown_connections 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.142+0000 7f5a59920640 1 -- 172.21.15.86:0/4285631617 wait complete. 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 Processor -- start 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 -- start start 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 0x7f5a54197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a54197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54110ef0 con 0x7f5a54106c30 2024-09-05T06:45:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54110d70 con 0x7f5a541011a0 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.143+0000 7f5a59920640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54111070 con 0x7f5a54101b40 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36026/0 (socket says 172.21.15.86:36026) 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 -- 172.21.15.86:0/1562645816 learned_addr learned my addr 172.21.15.86:0/1562645816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a527fc640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a54197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a52ffd640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 0x7f5a54197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 -- 172.21.15.86:0/1562645816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 msgr2=0x7f5a54197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a54197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 -- 172.21.15.86:0/1562645816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 msgr2=0x7f5a54197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 0x7f5a54197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 -- 172.21.15.86:0/1562645816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5419c400 con 0x7f5a54106c30 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a527fc640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a54197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a537fe640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00db90 tx=0x7f5a4c00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00e980 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5419c690 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a4c009580 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c016650 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.144+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a54100090 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.145+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a541023c0 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.146+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a4c009730 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.147+0000 7f5a5891e640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 0x7f5a3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.147+0000 7f5a52ffd640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 0x7f5a3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.147+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5a4c0d0eb0 con 0x7f5a54106c30 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.147+0000 7f5a52ffd640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 0x7f5a3c0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a44006010 tx=0x7f5a44000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.150+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a4c1001b0 con 0x7f5a54106c30 2024-09-05T06:45:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.350+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a54100ab0 con 0x7f5a54106c30 2024-09-05T06:45:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.351+0000 7f5a5891e640 1 -- 172.21.15.86:0/1562645816 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7f5a4c09d800 con 0x7f5a54106c30 2024-09-05T06:45:05.351 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:45:05.352 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:45:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:45:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 msgr2=0x7f5a3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 0x7f5a3c0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a44006010 tx=0x7f5a44000bc0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 msgr2=0x7f5a5419bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00db90 tx=0x7f5a4c00dc80 comp rx=0 tx=0).stop 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 shutdown_connections 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5a3c0a2cf0 0x7f5a3c0a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54106c30 0x7f5a5419bd20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a54101b40 0x7f5a54197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 --2- 172.21.15.86:0/1562645816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a541011a0 0x7f5a54197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 >> 172.21.15.86:0/1562645816 conn(0x7f5a54077210 msgr2=0x7f5a54103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.354+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 shutdown_connections 2024-09-05T06:45:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:05.355+0000 7f5a59920640 1 -- 172.21.15.86:0/1562645816 wait complete. 2024-09-05T06:45:05.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.30 MB/sec execute 25 sec latency 14050.796 ms 2024-09-05T06:45:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:05 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1562645816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:05.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:05 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1562645816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:05 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1562645816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:06.114 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 51df3dec --release --await' 2024-09-05T06:45:06.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 51df3dec --release --await 2024-09-05T06:45:06.359 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:06.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 60917 1.25 MB/sec execute 26 sec latency 15050.862 ms 2024-09-05T06:45:06.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:06 smithi115 ceph-mon[34791]: pgmap v118: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:06.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:06 smithi086 ceph-mon[30943]: pgmap v118: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:06 smithi096 ceph-mon[34468]: pgmap v118: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-05T06:45:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- 172.21.15.86:0/57514201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 msgr2=0x7f3b9010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- 172.21.15.86:0/57514201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9010eaa0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009ca0 tx=0x7f3b8802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- 172.21.15.86:0/57514201 shutdown_connections 2024-09-05T06:45:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- 172.21.15.86:0/57514201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9010eaa0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- 172.21.15.86:0/57514201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- 172.21.15.86:0/57514201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 0x7f3b901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- 172.21.15.86:0/57514201 >> 172.21.15.86:0/57514201 conn(0x7f3b900fd8b0 msgr2=0x7f3b900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- 172.21.15.86:0/57514201 shutdown_connections 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- 172.21.15.86:0/57514201 wait complete. 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 Processor -- start 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 -- start start 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 0x7f3b901983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.842+0000 7f3b97059640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b90198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b97059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b97059640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90111ed0 con 0x7f3b90106f20 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b97059640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90111d50 con 0x7f3b901027a0 2024-09-05T06:45:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b97059640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90112050 con 0x7f3b90101df0 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36042/0 (socket says 172.21.15.86:36042) 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 -- 172.21.15.86:0/1811967189 learned_addr learned my addr 172.21.15.86:0/1811967189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b94df5640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 0x7f3b901983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b87fff640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b90198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 -- 172.21.15.86:0/1811967189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 msgr2=0x7f3b901983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 0x7f3b901983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 -- 172.21.15.86:0/1811967189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 msgr2=0x7f3b90198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b90198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 -- 172.21.15.86:0/1811967189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9019d370 con 0x7f3b90106f20 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b87fff640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b90198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b955f6640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009dc0 tx=0x7f3b88003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b85ffb640 1 -- 172.21.15.86:0/1811967189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004040 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b85ffb640 1 -- 172.21.15.86:0/1811967189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b88004690 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9019d600 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.843+0000 7f3b85ffb640 1 -- 172.21.15.86:0/1811967189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88030c90 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.844+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b901a4f40 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.846+0000 7f3b85ffb640 1 -- 172.21.15.86:0/1811967189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b880041e0 con 0x7f3b90106f20 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.846+0000 7f3b99432740 1 Processor -- start 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.846+0000 7f3b99432740 1 -- start start 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d81d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d84630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e21cdc620 con 0x560e21d77060 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b85ffb640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 0x7f3b780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e21c83e50 con 0x560e21d82240 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b99432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560e21cd85e0 con 0x560e21d77980 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36050/0 (socket says 172.21.15.86:36050) 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 -- 172.21.15.86:0/1284185854 learned_addr learned my addr 172.21.15.86:0/1284185854 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b955f6640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 0x7f3b780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b94df5640 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d81d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b85ffb640 1 -- 172.21.15.86:0/1811967189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b880f0ff0 con 0x7f3b90106f20 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 -- 172.21.15.86:0/1284185854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 msgr2=0x560e21d81d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d81d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 -- 172.21.15.86:0/1284185854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 msgr2=0x560e21d84630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d84630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 -- 172.21.15.86:0/1284185854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e21c74d40 con 0x560e21d77060 2024-09-05T06:45:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b94df5640 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d81d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b87fff640 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000aa90 tx=0x7f3b8000af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15e3ac17d50c38f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b955f6640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 0x7f3b780a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b880c5920 tx=0x7f3b8808d040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b76ffd640 1 -- 172.21.15.86:0/1284185854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000f040 con 0x560e21d77060 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.847+0000 7f3b76ffd640 1 -- 172.21.15.86:0/1284185854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80018070 con 0x560e21d77060 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b76ffd640 1 -- 172.21.15.86:0/1284185854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b800123c0 con 0x560e21d77060 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 -- 172.21.15.86:0/1284185854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 msgr2=0x560e21d77440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000aa90 tx=0x7f3b8000af50 comp rx=0 tx=0).stop 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 -- 172.21.15.86:0/1284185854 shutdown_connections 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d84630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d81d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 --2- 172.21.15.86:0/1284185854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d77440 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 -- 172.21.15.86:0/1284185854 >> 172.21.15.86:0/1284185854 conn(0x560e21d735f0 msgr2=0x560e21d739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 -- 172.21.15.86:0/1284185854 shutdown_connections 2024-09-05T06:45:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.848+0000 7f3b99432740 1 -- 172.21.15.86:0/1284185854 wait complete. 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 -- start start 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 Processor -- start 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d7fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e21d84e10 con 0x560e21d77060 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e21c64f10 con 0x560e21d82240 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560e21d84f90 con 0x560e21d77980 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b99432740 10 client.0 _ll_register_callbacks cb 0x560e21c88db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b94df5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b94df5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d7f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:34894/0 (socket says 172.21.15.86:34894) 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b94df5640 1 -- 172.21.15.86:0/3084520899 learned_addr learned my addr 172.21.15.86:0/3084520899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36060/0 (socket says 172.21.15.86:36060) 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.849+0000 7f3b87fff640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d7fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 -- 172.21.15.86:0/3084520899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 msgr2=0x560e21d7f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d7f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 -- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 msgr2=0x560e21d7fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d7fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 -- 172.21.15.86:0/3084520899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e219eb580 con 0x560e21d77060 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b87fff640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013c20 tx=0x7f3b80004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312135bab218ad1a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80028060 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b80018070 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b8000ed40 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b8002f050 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b99432740 10 client.16371 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e21ab5960 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.850+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80018070 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3b8001c410 con 0x560e21d77060 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 10 client.16371 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 10 client.16371 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 20 client.16371 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21d91d50 0x560e21d94140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 4 client.16371 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e21ce9b90 con 0x560e21d91d50 2024-09-05T06:45:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b94df5640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21d91d50 0x560e21d94140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b6ffff640 10 client.16371 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.851+0000 7f3b94df5640 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21d91d50 0x560e21d94140 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae4528dbc4101e2 server_cookie=872c87bd5c5749d4 in_seq=0 out_seq=0 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b6ffff640 1 -- 172.21.15.86:0/3084520899 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e21ce9b90 con 0x560e21d91d50 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b6ffff640 10 client.16371 handle_command_reply: tid=0 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 client.16371 shutdown 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 msgr2=0x560e21d7f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013c20 tx=0x7f3b80004680 comp rx=0 tx=0).stop 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 shutdown_connections 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21d91d50 0x560e21d94140 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d82240 0x560e21d7fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b6ffff640 0 client.16371 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 --2- 172.21.15.86:0/3084520899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77060 0x560e21d7f180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 >> 172.21.15.86:0/3084520899 conn(0x560e21d735f0 msgr2=0x560e21d792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 shutdown_connections 2024-09-05T06:45:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 1 -- 172.21.15.86:0/3084520899 wait complete. 2024-09-05T06:45:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.852+0000 7f3b99432740 20 client.16371 trim_cache size 0 max 16384 2024-09-05T06:45:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.869+0000 7f3b99432740 1 Processor -- start 2024-09-05T06:45:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.869+0000 7f3b99432740 1 -- start start 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d72f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d82240 0x560e21d7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901aefc0 con 0x560e21d77980 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901b0b60 con 0x560e21d77060 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b99432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901af690 con 0x560e21d82240 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36070/0 (socket says 172.21.15.86:36070) 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 -- 172.21.15.86:0/2946623978 learned_addr learned my addr 172.21.15.86:0/2946623978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d72f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d82240 0x560e21d7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 -- 172.21.15.86:0/2946623978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d82240 msgr2=0x560e21d7f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d82240 0x560e21d7f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 -- 172.21.15.86:0/2946623978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 msgr2=0x560e21d72f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d72f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 -- 172.21.15.86:0/2946623978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e21d810c0 con 0x560e21d77980 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009a80 tx=0x7f3b8c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8481848a95f86f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b767fc640 1 -- 172.21.15.86:0/2946623978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c021070 con 0x560e21d77980 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b767fc640 1 -- 172.21.15.86:0/2946623978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c016d50 con 0x560e21d77980 2024-09-05T06:45:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.870+0000 7f3b767fc640 1 -- 172.21.15.86:0/2946623978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c015620 con 0x560e21d77980 2024-09-05T06:45:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 -- 172.21.15.86:0/2946623978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 msgr2=0x560e21d73c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009a80 tx=0x7f3b8c009110 comp rx=0 tx=0).stop 2024-09-05T06:45:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 -- 172.21.15.86:0/2946623978 shutdown_connections 2024-09-05T06:45:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d82240 0x560e21d7f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d77980 0x560e21d73c00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 --2- 172.21.15.86:0/2946623978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d72f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 -- 172.21.15.86:0/2946623978 >> 172.21.15.86:0/2946623978 conn(0x560e21d735f0 msgr2=0x560e21d7f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 -- 172.21.15.86:0/2946623978 shutdown_connections 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.871+0000 7f3b99432740 1 -- 172.21.15.86:0/2946623978 wait complete. 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 -- start start 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 Processor -- start 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901b0430 con 0x560e21d82240 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901afd60 con 0x560e21d77060 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901acfa0 con 0x560e21d77980 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b99432740 10 client.0 _ll_register_callbacks cb 0x7f3b80013a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36086/0 (socket says 172.21.15.86:36086) 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b87fff640 1 -- 172.21.15.86:0/2088732696 learned_addr learned my addr 172.21.15.86:0/2088732696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.872+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 -- 172.21.15.86:0/2088732696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 msgr2=0x560e21d86810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d86810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 -- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 msgr2=0x560e21d862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 -- 172.21.15.86:0/2088732696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e21d87110 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b87fff640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013900 tx=0x7f3b80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2315a81eb94adb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80028040 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b8000eb40 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c016d50 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b80041050 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b99432740 10 client.16377 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e21e01230 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c016d50 con 0x560e21d82240 2024-09-05T06:45:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.873+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3b800209e0 con 0x560e21d82240 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 10 client.16377 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 10 client.16377 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 20 client.16377 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21e01860 0x560e21e03c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 4 client.16377 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "51df3dec", "release": true, "await": true}] 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "51df3dec", "release": true, "await": true}) -- 0x560e21ce40a0 con 0x560e21e01860 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21e01860 0x560e21e03c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b94df5640 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21e01860 0x560e21e03c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517dd5baf2bfd33e server_cookie=e0b34629b0bd3d1e in_seq=0 out_seq=0 2024-09-05T06:45:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.874+0000 7f3b6ffff640 10 client.16377 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b6ffff640 1 -- 172.21.15.86:0/2088732696 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e21ce40a0 con 0x560e21e01860 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b6ffff640 10 client.16377 handle_command_reply: tid=0 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 client.16377 shutdown 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 msgr2=0x560e21dffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013900 tx=0x7f3b80008040 comp rx=0 tx=0).stop 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 shutdown_connections 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560e21e01860 0x560e21e03c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e21d82240 0x560e21dffb80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560e21d77980 0x560e21d86810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b6ffff640 0 client.16377 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 --2- 172.21.15.86:0/2088732696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e21d77060 0x560e21d862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.880+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 >> 172.21.15.86:0/2088732696 conn(0x560e21d735f0 msgr2=0x560e21d792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 shutdown_connections 2024-09-05T06:45:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b99432740 1 -- 172.21.15.86:0/2088732696 wait complete. 2024-09-05T06:45:06.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b99432740 20 client.16377 trim_cache size 0 max 16384 2024-09-05T06:45:06.882 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:06.882 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 30, 2024-09-05T06:45:06.882 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:45:06.882 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 12, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "51df3dec": { 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 12, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 141.8, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:45:06.883 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:06.884 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 msgr2=0x7f3b780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 0x7f3b780a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b880c5920 tx=0x7f3b8808d040 comp rx=0 tx=0).stop 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 msgr2=0x7f3b9019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009dc0 tx=0x7f3b88003ba0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 shutdown_connections 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b780a2cf0 0x7f3b780a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b90106f20 0x7f3b9019cc90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b901027a0 0x7f3b90198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 --2- 172.21.15.86:0/1811967189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b90101df0 0x7f3b901983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:06.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 >> 172.21.15.86:0/1811967189 conn(0x7f3b900fd8b0 msgr2=0x7f3b900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:06.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 shutdown_connections 2024-09-05T06:45:06.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:06.881+0000 7f3b97059640 1 -- 172.21.15.86:0/1811967189 wait complete. 2024-09-05T06:45:07.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61267 1.24 MB/sec execute 27 sec latency 15423.095 ms 2024-09-05T06:45:07.727 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '51df3dec', total seconds elapsed: 25.1 2024-09-05T06:45:07.727 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T06:45:07.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:45:08.011 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 -- 172.21.15.86:0/4220289852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c100a90 msgr2=0x7f2a8c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 --2- 172.21.15.86:0/4220289852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c100a90 0x7f2a8c100ef0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009930 tx=0x7f2a7c02a420 comp rx=0 tx=0).stop 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 -- 172.21.15.86:0/4220289852 shutdown_connections 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 --2- 172.21.15.86:0/4220289852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c101430 0x7f2a8c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 --2- 172.21.15.86:0/4220289852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c100a90 0x7f2a8c100ef0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.495+0000 7f2a93283640 1 --2- 172.21.15.86:0/4220289852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c106a90 0x7f2a8c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.496+0000 7f2a93283640 1 -- 172.21.15.86:0/4220289852 >> 172.21.15.86:0/4220289852 conn(0x7f2a8c0fc7d0 msgr2=0x7f2a8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.496+0000 7f2a93283640 1 -- 172.21.15.86:0/4220289852 shutdown_connections 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.496+0000 7f2a93283640 1 -- 172.21.15.86:0/4220289852 wait complete. 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.496+0000 7f2a93283640 1 Processor -- start 2024-09-05T06:45:08.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.496+0000 7f2a93283640 1 -- start start 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c101430 0x7f2a8c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 0x7f2a8c1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c110d90 con 0x7f2a8c101430 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c110c10 con 0x7f2a8c100a90 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a93283640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c110f10 con 0x7f2a8c106a90 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58054/0 (socket says 172.21.15.86:58054) 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 -- 172.21.15.86:0/3582459915 learned_addr learned my addr 172.21.15.86:0/3582459915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9081e640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c101430 0x7f2a8c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a91820640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 0x7f2a8c1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 -- 172.21.15.86:0/3582459915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 msgr2=0x7f2a8c1a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 0x7f2a8c1a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 -- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c101430 msgr2=0x7f2a8c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c101430 0x7f2a8c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 -- 172.21.15.86:0/3582459915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1a4760 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a91820640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 0x7f2a8c1a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a9101f640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9f0 tx=0x7f2a8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800cd80 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.497+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a880108e0 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.498+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1a4930 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.498+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88018920 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.498+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a4e40 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.499+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c100ef0 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.500+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a88010a90 con 0x7f2a8c100a90 2024-09-05T06:45:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.500+0000 7f2a827fc640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 0x7f2a780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.501+0000 7f2a9081e640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 0x7f2a780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.501+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f2a880cfac0 con 0x7f2a8c100a90 2024-09-05T06:45:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.501+0000 7f2a9081e640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 0x7f2a780a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c031440 tx=0x7f2a7c005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.505+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a8809bcb0 con 0x7f2a8c100a90 2024-09-05T06:45:08.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61826 1.35 MB/sec execute 28 sec latency 21.041 ms 2024-09-05T06:45:08.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:08 smithi115 ceph-mon[34791]: pgmap v119: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 55 KiB/s wr, 42 op/s 2024-09-05T06:45:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:08 smithi086 ceph-mon[30943]: pgmap v119: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 55 KiB/s wr, 42 op/s 2024-09-05T06:45:08.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.756+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8c06adc0 con 0x7f2a8c100a90 2024-09-05T06:45:08.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.757+0000 7f2a827fc640 1 -- 172.21.15.86:0/3582459915 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7f2a8809ffb0 con 0x7f2a8c100a90 2024-09-05T06:45:08.757 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:45:08.757 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:45:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 msgr2=0x7f2a780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 0x7f2a780a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c031440 tx=0x7f2a7c005a20 comp rx=0 tx=0).stop 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 msgr2=0x7f2a8c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9f0 tx=0x7f2a8800eeb0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 shutdown_connections 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a780a2cf0 0x7f2a780a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c106a90 0x7f2a8c1a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a8c101430 0x7f2a8c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 --2- 172.21.15.86:0/3582459915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1991a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 >> 172.21.15.86:0/3582459915 conn(0x7f2a8c0fc7d0 msgr2=0x7f2a8c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 shutdown_connections 2024-09-05T06:45:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:08.759+0000 7f2a93283640 1 -- 172.21.15.86:0/3582459915 wait complete. 2024-09-05T06:45:08.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:08 smithi096 ceph-mon[34468]: pgmap v119: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 55 KiB/s wr, 42 op/s 2024-09-05T06:45:09.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62506 1.32 MB/sec execute 29 sec latency 15.074 ms 2024-09-05T06:45:09.521 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T06:45:09.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T06:45:09.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:09 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3582459915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:09.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:09 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3582459915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:09 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3582459915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 -- 172.21.15.86:0/2076791216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334103660 msgr2=0x7f433410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 --2- 172.21.15.86:0/2076791216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334103660 0x7f433410bb50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f432402a480 comp rx=0 tx=0).stop 2024-09-05T06:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 -- 172.21.15.86:0/2076791216 shutdown_connections 2024-09-05T06:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 --2- 172.21.15.86:0/2076791216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f433410c170 0x7f433410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 --2- 172.21.15.86:0/2076791216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334103660 0x7f433410bb50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 --2- 172.21.15.86:0/2076791216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4334102d40 0x7f4334103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.312+0000 7f433bffd640 1 -- 172.21.15.86:0/2076791216 >> 172.21.15.86:0/2076791216 conn(0x7f43340fc7a0 msgr2=0x7f43340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.313+0000 7f433bffd640 1 -- 172.21.15.86:0/2076791216 shutdown_connections 2024-09-05T06:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.313+0000 7f433bffd640 1 -- 172.21.15.86:0/2076791216 wait complete. 2024-09-05T06:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.313+0000 7f433bffd640 1 Processor -- start 2024-09-05T06:45:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.313+0000 7f433bffd640 1 -- start start 2024-09-05T06:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4334103660 0x7f43341988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f433410c170 0x7f433419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4334111e20 con 0x7f4334102d40 2024-09-05T06:45:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4334111ca0 con 0x7f433410c170 2024-09-05T06:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433bffd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4334111fa0 con 0x7f4334103660 2024-09-05T06:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36124/0 (socket says 172.21.15.86:36124) 2024-09-05T06:45:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 -- 172.21.15.86:0/2645437640 learned_addr learned my addr 172.21.15.86:0/2645437640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339598640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4334103660 0x7f43341988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f433a59a640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f433410c170 0x7f433419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 -- 172.21.15.86:0/2645437640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4334103660 msgr2=0x7f43341988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4334103660 0x7f43341988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 -- 172.21.15.86:0/2645437640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f433410c170 msgr2=0x7f433419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f433410c170 0x7f433419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 -- 172.21.15.86:0/2645437640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433419d330 con 0x7f4334102d40 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f4339d99640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.314+0000 7f432affd640 1 -- 172.21.15.86:0/2645437640 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd80 con 0x7f4334102d40 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.315+0000 7f432affd640 1 -- 172.21.15.86:0/2645437640 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43300108e0 con 0x7f4334102d40 2024-09-05T06:45:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.315+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433419d620 con 0x7f4334102d40 2024-09-05T06:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.315+0000 7f432affd640 1 -- 172.21.15.86:0/2645437640 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018920 con 0x7f4334102d40 2024-09-05T06:45:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.315+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a4f00 con 0x7f4334102d40 2024-09-05T06:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.317+0000 7f432affd640 1 -- 172.21.15.86:0/2645437640 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4330010a90 con 0x7f4334102d40 2024-09-05T06:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.317+0000 7f433e3d6740 1 Processor -- start 2024-09-05T06:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 -- start start 2024-09-05T06:45:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f432affd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 0x7f43200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce440160 con 0x55c3ce4dc170 2024-09-05T06:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce3e7720 con 0x55c3ce4db850 2024-09-05T06:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433e3d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce43c120 con 0x55c3ce4e6a30 2024-09-05T06:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433a59a640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 0x7f43200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f432affd640 1 -- 172.21.15.86:0/2645437640 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f43300cfb00 con 0x7f4334102d40 2024-09-05T06:45:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36130/0 (socket says 172.21.15.86:36130) 2024-09-05T06:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 -- 172.21.15.86:0/2326802477 learned_addr learned my addr 172.21.15.86:0/2326802477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339d99640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f433a59a640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 -- 172.21.15.86:0/2326802477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 msgr2=0x55c3ce4e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.318+0000 7f4339598640 1 -- 172.21.15.86:0/2326802477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 msgr2=0x55c3ce4dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f4339598640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f4339598640 1 -- 172.21.15.86:0/2326802477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ce1b1ba0 con 0x55c3ce4dc170 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f433a59a640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f433a59a640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 0x7f43200a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f432c005fd0 tx=0x7f432c010740 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f4339598640 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f4324032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369d2996b5b705f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f431ffff640 1 -- 172.21.15.86:0/2326802477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324040070 con 0x55c3ce4dc170 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f431ffff640 1 -- 172.21.15.86:0/2326802477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4324004030 con 0x55c3ce4dc170 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.319+0000 7f431ffff640 1 -- 172.21.15.86:0/2326802477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324035040 con 0x55c3ce4dc170 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- 172.21.15.86:0/2326802477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 msgr2=0x55c3ce4e64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4324009900 tx=0x7f4324032b00 comp rx=0 tx=0).stop 2024-09-05T06:45:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- 172.21.15.86:0/2326802477 shutdown_connections 2024-09-05T06:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e64f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2326802477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- 172.21.15.86:0/2326802477 >> 172.21.15.86:0/2326802477 conn(0x55c3ce4d7de0 msgr2=0x55c3ce4d81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- 172.21.15.86:0/2326802477 shutdown_connections 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- 172.21.15.86:0/2326802477 wait complete. 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 -- start start 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.320+0000 7f433e3d6740 1 Processor -- start 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4e3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce4eda10 con 0x55c3ce4e6a30 2024-09-05T06:45:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce3c8900 con 0x55c3ce4dc170 2024-09-05T06:45:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c3ce4edb90 con 0x55c3ce4db850 2024-09-05T06:45:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433e3d6740 10 client.0 _ll_register_callbacks cb 0x55c3ce3c6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58096/0 (socket says 172.21.15.86:58096) 2024-09-05T06:45:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 -- 172.21.15.86:0/464588955 learned_addr learned my addr 172.21.15.86:0/464588955 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339d99640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f433a59a640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 -- 172.21.15.86:0/464588955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 msgr2=0x55c3ce4e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 -- 172.21.15.86:0/464588955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 msgr2=0x55c3ce4e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.321+0000 7f4339598640 1 -- 172.21.15.86:0/464588955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3ce408c50 con 0x55c3ce4dc170 2024-09-05T06:45:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f4339598640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f432402ac70 tx=0x7f432402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b93c8f31895854 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f431d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:45:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432404d060 con 0x55c3ce4dc170 2024-09-05T06:45:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4324004030 con 0x55c3ce4dc170 2024-09-05T06:45:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f43240316a0 con 0x55c3ce4dc170 2024-09-05T06:45:10.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4324054050 con 0x55c3ce4dc170 2024-09-05T06:45:10.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f433e3d6740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:10.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.322+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3ce4f9610 con 0x55c3ce4dc170 2024-09-05T06:45:10.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4324004750 con 0x55c3ce4dc170 2024-09-05T06:45:10.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f43240035f0 con 0x55c3ce4dc170 2024-09-05T06:45:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 10 client.25757 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 20 client.25757 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce4fa960 0x55c3ce4fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 4 client.25757 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4324004030 con 0x55c3ce4fa960 2024-09-05T06:45:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f4339d99640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce4fa960 0x55c3ce4fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.323+0000 7f431d7fa640 10 client.25757 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f4339d99640 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce4fa960 0x55c3ce4fcd50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef093cd7d2d1e5c4 server_cookie=a07ff632dffbb3eb in_seq=0 out_seq=0 2024-09-05T06:45:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f431d7fa640 1 -- 172.21.15.86:0/464588955 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4324004030 con 0x55c3ce4fa960 2024-09-05T06:45:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f431d7fa640 10 client.25757 handle_command_reply: tid=0 2024-09-05T06:45:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 client.25757 shutdown 2024-09-05T06:45:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 msgr2=0x55c3ce4e3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f432402ac70 tx=0x7f432402a480 comp rx=0 tx=0).stop 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 shutdown_connections 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce4fa960 0x55c3ce4fcd50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f431d7fa640 0 client.25757 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4e3e40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 --2- 172.21.15.86:0/464588955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 >> 172.21.15.86:0/464588955 conn(0x55c3ce4d7de0 msgr2=0x55c3ce4ddb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.324+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 shutdown_connections 2024-09-05T06:45:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.325+0000 7f433e3d6740 1 -- 172.21.15.86:0/464588955 wait complete. 2024-09-05T06:45:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.325+0000 7f433e3d6740 20 client.25757 trim_cache size 0 max 16384 2024-09-05T06:45:10.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 Processor -- start 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 -- start start 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce502380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ace50 con 0x55c3ce4dc170 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ae9f0 con 0x55c3ce4db850 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f433e3d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ad520 con 0x55c3ce4e6a30 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.344+0000 7f4339598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36148/0 (socket says 172.21.15.86:36148) 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339598640 1 -- 172.21.15.86:0/2690801703 learned_addr learned my addr 172.21.15.86:0/2690801703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:10.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f433a59a640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce502380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 -- 172.21.15.86:0/2690801703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 msgr2=0x55c3ce502380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce502380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 -- 172.21.15.86:0/2690801703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 msgr2=0x55c3ce4de2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 -- 172.21.15.86:0/2690801703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ce4d7c10 con 0x55c3ce4db850 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339598640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f4339d99640 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4e2860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43300cce70 tx=0x7f433001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8da71251722a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f431f7fe640 1 -- 172.21.15.86:0/2690801703 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300cd0f0 con 0x55c3ce4db850 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f431f7fe640 1 -- 172.21.15.86:0/2690801703 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4330098360 con 0x55c3ce4db850 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.345+0000 7f431f7fe640 1 -- 172.21.15.86:0/2690801703 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330097a40 con 0x55c3ce4db850 2024-09-05T06:45:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- 172.21.15.86:0/2690801703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 msgr2=0x55c3ce4e2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4e2860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43300cce70 tx=0x7f433001a040 comp rx=0 tx=0).stop 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- 172.21.15.86:0/2690801703 shutdown_connections 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4e6a30 0x55c3ce502380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4dc170 0x55c3ce4de2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 --2- 172.21.15.86:0/2690801703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4db850 0x55c3ce4e2860 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- 172.21.15.86:0/2690801703 >> 172.21.15.86:0/2690801703 conn(0x55c3ce4d7de0 msgr2=0x55c3ce4d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- 172.21.15.86:0/2690801703 shutdown_connections 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- 172.21.15.86:0/2690801703 wait complete. 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.346+0000 7f433e3d6740 1 -- start start 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 Processor -- start 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ae2c0 con 0x55c3ce4e6a30 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43341adbf0 con 0x55c3ce4dc170 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43341aae30 con 0x55c3ce4db850 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433e3d6740 10 client.0 _ll_register_callbacks cb 0x7f4324009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433a59a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f4339598640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433a59a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36164/0 (socket says 172.21.15.86:36164) 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f4339598640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58114/0 (socket says 172.21.15.86:58114) 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f433a59a640 1 -- 172.21.15.86:0/1007216761 learned_addr learned my addr 172.21.15.86:0/1007216761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.347+0000 7f4339d99640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 -- 172.21.15.86:0/1007216761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 msgr2=0x55c3ce4f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 -- 172.21.15.86:0/1007216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 msgr2=0x55c3ce4efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 -- 172.21.15.86:0/1007216761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3ce4f02f0 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339d99640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433a59a640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f4339598640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f432404b040 tx=0x7f4324009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c9077920f1da9b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432405d060 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4324004110 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4330098360 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f432404d070 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433e3d6740 10 client.25763 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3ce565aa0 con 0x55c3ce4dc170 2024-09-05T06:45:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43240314a0 con 0x55c3ce4dc170 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4324005170 con 0x55c3ce4dc170 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433e3d6740 10 client.25763 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.348+0000 7f433e3d6740 10 client.25763 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f433e3d6740 20 client.25763 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce5661c0 0x55c3ce5685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f433e3d6740 4 client.25763 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x7f4330098360 con 0x55c3ce5661c0 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f4339d99640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce5661c0 0x55c3ce5685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f431d7fa640 10 client.25763 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.349+0000 7f4339d99640 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce5661c0 0x55c3ce5685e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2109cbc9930722 server_cookie=b0201a84d6dc7b4c in_seq=0 out_seq=0 2024-09-05T06:45:10.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f431d7fa640 1 -- 172.21.15.86:0/1007216761 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f4330098360 con 0x55c3ce5661c0 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f431d7fa640 10 client.25763 handle_command_reply: tid=0 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 client.25763 shutdown 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 msgr2=0x55c3ce4f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f432404b040 tx=0x7f4324009300 comp rx=0 tx=0).stop 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 shutdown_connections 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c3ce5661c0 0x55c3ce5685e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3ce4e6a30 0x55c3ce4efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3ce4dc170 0x55c3ce4f3320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 --2- 172.21.15.86:0/1007216761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3ce4db850 0x55c3ce4f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f431d7fa640 0 client.25763 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 >> 172.21.15.86:0/1007216761 conn(0x55c3ce4d7de0 msgr2=0x55c3ce4d66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 shutdown_connections 2024-09-05T06:45:10.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 1 -- 172.21.15.86:0/1007216761 wait complete. 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.419+0000 7f433e3d6740 20 client.25763 trim_cache size 0 max 16384 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 30, 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 14, 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "73dfa2a0": { 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 14, 2024-09-05T06:45:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 141.8, 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 msgr2=0x7f43200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 0x7f43200a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f432c005fd0 tx=0x7f432c010740 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 msgr2=0x7f4334198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 shutdown_connections 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f43200a2cf0 0x7f43200a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f433410c170 0x7f433419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4334103660 0x7f43341988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 --2- 172.21.15.86:0/2645437640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334102d40 0x7f4334198380 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 >> 172.21.15.86:0/2645437640 conn(0x7f43340fc7a0 msgr2=0x7f43340fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 shutdown_connections 2024-09-05T06:45:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:10.420+0000 7f433bffd640 1 -- 172.21.15.86:0/2645437640 wait complete. 2024-09-05T06:45:10.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:10 smithi086 ceph-mon[30943]: pgmap v120: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-05T06:45:10.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.35 MB/sec execute 30 sec latency 166.334 ms 2024-09-05T06:45:10.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:10 smithi115 ceph-mon[34791]: pgmap v120: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-05T06:45:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:10 smithi096 ceph-mon[34468]: pgmap v120: 129 pgs: 129 active+clean; 63 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-09-05T06:45:11.096 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '73dfa2a0', quiesce took 3.4 seconds. Will release after: 18.35 2024-09-05T06:45:11.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.31 MB/sec execute 31 sec latency 1166.403 ms 2024-09-05T06:45:12.510 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.27 MB/sec execute 32 sec latency 2166.465 ms 2024-09-05T06:45:12.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:12 smithi115 ceph-mon[34791]: pgmap v121: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 453 KiB/s wr, 77 op/s 2024-09-05T06:45:12.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:12 smithi086 ceph-mon[30943]: pgmap v121: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 453 KiB/s wr, 77 op/s 2024-09-05T06:45:12.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:12 smithi096 ceph-mon[34468]: pgmap v121: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 453 KiB/s wr, 77 op/s 2024-09-05T06:45:13.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.23 MB/sec execute 33 sec latency 3166.549 ms 2024-09-05T06:45:14.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.19 MB/sec execute 34 sec latency 4166.652 ms 2024-09-05T06:45:14.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:14 smithi115 ceph-mon[34791]: pgmap v122: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:14.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:14 smithi086 ceph-mon[30943]: pgmap v122: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:14.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:14 smithi096 ceph-mon[34468]: pgmap v122: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:15.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.16 MB/sec execute 35 sec latency 5166.769 ms 2024-09-05T06:45:16.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.13 MB/sec execute 36 sec latency 6166.889 ms 2024-09-05T06:45:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:16 smithi115 ceph-mon[34791]: pgmap v123: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:16 smithi086 ceph-mon[30943]: pgmap v123: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:16.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:16 smithi096 ceph-mon[34468]: pgmap v123: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 426 KiB/s wr, 56 op/s 2024-09-05T06:45:17.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.10 MB/sec execute 37 sec latency 7166.935 ms 2024-09-05T06:45:18.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.07 MB/sec execute 38 sec latency 8166.977 ms 2024-09-05T06:45:18.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:18 smithi115 ceph-mon[34791]: pgmap v124: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 434 KiB/s wr, 64 op/s 2024-09-05T06:45:18.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:18 smithi086 ceph-mon[30943]: pgmap v124: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 434 KiB/s wr, 64 op/s 2024-09-05T06:45:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:18 smithi096 ceph-mon[34468]: pgmap v124: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 434 KiB/s wr, 64 op/s 2024-09-05T06:45:19.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.04 MB/sec execute 39 sec latency 9167.062 ms 2024-09-05T06:45:20.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 1.02 MB/sec execute 40 sec latency 10167.178 ms 2024-09-05T06:45:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:20 smithi115 ceph-mon[34791]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 432 KiB/s wr, 58 op/s 2024-09-05T06:45:20.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:20 smithi086 ceph-mon[30943]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 432 KiB/s wr, 58 op/s 2024-09-05T06:45:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:20 smithi096 ceph-mon[34468]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 432 KiB/s wr, 58 op/s 2024-09-05T06:45:20.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:45:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:21.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.99 MB/sec execute 41 sec latency 11167.270 ms 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.645+0000 7f5789877640 1 -- 172.21.15.86:0/2312075093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578410c180 msgr2=0x7f578410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.645+0000 7f5789877640 1 --2- 172.21.15.86:0/2312075093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578410c180 0x7f578410e610 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c02a480 comp rx=0 tx=0).stop 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 -- 172.21.15.86:0/2312075093 shutdown_connections 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 --2- 172.21.15.86:0/2312075093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578410c180 0x7f578410e610 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 --2- 172.21.15.86:0/2312075093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5784103700 0x7f578410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 --2- 172.21.15.86:0/2312075093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784102de0 0x7f57841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 -- 172.21.15.86:0/2312075093 >> 172.21.15.86:0/2312075093 conn(0x7f57840fc7a0 msgr2=0x7f57840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 -- 172.21.15.86:0/2312075093 shutdown_connections 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 -- 172.21.15.86:0/2312075093 wait complete. 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 Processor -- start 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.646+0000 7f5789877640 1 -- start start 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784103700 0x7f5784198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 0x7f578419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5784111de0 con 0x7f5784102de0 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5784111c60 con 0x7f578410c180 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5784111f60 con 0x7f5784103700 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47644/0 (socket says 172.21.15.86:47644) 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 -- 172.21.15.86:0/62971087 learned_addr learned my addr 172.21.15.86:0/62971087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f57827fc640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784103700 0x7f5784198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f57837fe640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 0x7f578419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 -- 172.21.15.86:0/62971087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784103700 msgr2=0x7f5784198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784103700 0x7f5784198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 -- 172.21.15.86:0/62971087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 msgr2=0x7f578419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 0x7f578419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 -- 172.21.15.86:0/62971087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578419d2e0 con 0x7f5784102de0 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f57837fe640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 0x7f578419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5782ffd640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f577400db80 tx=0x7f577400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5788875640 1 -- 172.21.15.86:0/62971087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774017070 con 0x7f5784102de0 2024-09-05T06:45:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5788875640 1 -- 172.21.15.86:0/62971087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5774003b70 con 0x7f5784102de0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578419d5d0 con 0x7f5784102de0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5788875640 1 -- 172.21.15.86:0/62971087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740104c0 con 0x7f5784102de0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.647+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a4eb0 con 0x7f5784102de0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.649+0000 7f5788875640 1 -- 172.21.15.86:0/62971087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5774004020 con 0x7f5784102de0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 Processor -- start 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 -- start start 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f5788875640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 0x7f576c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff092142e0 con 0x55ff092bda90 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff091bbce0 con 0x55ff092ae360 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f57837fe640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 0x7f576c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f578bc50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff092102a0 con 0x55ff092aec80 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f5788875640 1 -- 172.21.15.86:0/62971087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f57740cfa20 con 0x7f5784102de0 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47652/0 (socket says 172.21.15.86:47652) 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.650+0000 7f57837fe640 1 -- 172.21.15.86:0/1975792797 learned_addr learned my addr 172.21.15.86:0/1975792797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57827fc640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f5782ffd640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 0x7f576c0a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f577c032fc0 tx=0x7f577c0314a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 -- 172.21.15.86:0/1975792797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 msgr2=0x55ff092bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 -- 172.21.15.86:0/1975792797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 msgr2=0x55ff092ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 -- 172.21.15.86:0/1975792797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff091a0440 con 0x55ff092bda90 2024-09-05T06:45:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f5782ffd640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f57837fe640 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f577c031510 tx=0x7f577c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873641ccd95d00d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f5769ffb640 1 -- 172.21.15.86:0/1975792797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c04e070 con 0x55ff092bda90 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f5769ffb640 1 -- 172.21.15.86:0/1975792797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f577c0318d0 con 0x55ff092bda90 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.651+0000 7f5769ffb640 1 -- 172.21.15.86:0/1975792797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c035050 con 0x55ff092bda90 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- 172.21.15.86:0/1975792797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 msgr2=0x55ff092bfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f577c031510 tx=0x7f577c046000 comp rx=0 tx=0).stop 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- 172.21.15.86:0/1975792797 shutdown_connections 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bfe80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 --2- 172.21.15.86:0/1975792797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- 172.21.15.86:0/1975792797 >> 172.21.15.86:0/1975792797 conn(0x55ff092aaa30 msgr2=0x55ff092aae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- 172.21.15.86:0/1975792797 shutdown_connections 2024-09-05T06:45:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- 172.21.15.86:0/1975792797 wait complete. 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.652+0000 7f578bc50740 1 -- start start 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 Processor -- start 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff092c0600 con 0x55ff092bda90 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff091cf050 con 0x55ff092ae360 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff092c0780 con 0x55ff092aec80 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 10 client.0 _ll_register_callbacks cb 0x55ff0920f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47664/0 (socket says 172.21.15.86:47664) 2024-09-05T06:45:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 -- 172.21.15.86:0/1360927064 learned_addr learned my addr 172.21.15.86:0/1360927064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f5782ffd640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57827fc640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 -- 172.21.15.86:0/1360927064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 msgr2=0x55ff092ba6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092ba6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 -- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 msgr2=0x55ff092bd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092bd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 -- 172.21.15.86:0/1360927064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff08f853c0 con 0x55ff092bda90 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57827fc640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f57837fe640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f577c02a9a0 tx=0x7f577c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee9a688bfc40907 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f5762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c04e070 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f577c003e30 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.653+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577c0318d0 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f577c061050 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 10 client.16413 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff092c0600 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f577c0318d0 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f577c05c030 con 0x55ff092bda90 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 10 client.16413 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 20 client.16413 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff092cd1b0 0x55ff092cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 4 client.16413 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff09221850 con 0x55ff092cd1b0 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.654+0000 7f5782ffd640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff092cd1b0 0x55ff092cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.655+0000 7f5782ffd640 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff092cd1b0 0x55ff092cf5a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c638f842738a893 server_cookie=fa0e59bca0157a44 in_seq=0 out_seq=0 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.655+0000 7f5762ffd640 10 client.16413 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.655+0000 7f5762ffd640 1 -- 172.21.15.86:0/1360927064 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff09221850 con 0x55ff092cd1b0 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.655+0000 7f5762ffd640 10 client.16413 handle_command_reply: tid=0 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.655+0000 7f578bc50740 1 client.16413 shutdown 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 msgr2=0x55ff092bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f577c02a9a0 tx=0x7f577c02a480 comp rx=0 tx=0).stop 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 shutdown_connections 2024-09-05T06:45:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff092cd1b0 0x55ff092cf5a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092bac40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 --2- 172.21.15.86:0/1360927064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092bd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 >> 172.21.15.86:0/1360927064 conn(0x55ff092aaa30 msgr2=0x55ff092b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f5762ffd640 0 client.16413 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 shutdown_connections 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 1 -- 172.21.15.86:0/1360927064 wait complete. 2024-09-05T06:45:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.656+0000 7f578bc50740 20 client.16413 trim_cache size 0 max 16384 2024-09-05T06:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 Processor -- start 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 -- start start 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57841acdf0 con 0x55ff092bda90 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57841ae990 con 0x55ff092ae360 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f578bc50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57841ad4c0 con 0x55ff092aec80 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.673+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47668/0 (socket says 172.21.15.86:47668) 2024-09-05T06:45:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 -- 172.21.15.86:0/3313348724 learned_addr learned my addr 172.21.15.86:0/3313348724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f5782ffd640 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57827fc640 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 -- 172.21.15.86:0/3313348724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 msgr2=0x55ff092bc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 -- 172.21.15.86:0/3313348724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 msgr2=0x55ff092c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 -- 172.21.15.86:0/3313348724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff092af060 con 0x55ff092bda90 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57837fe640 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f577c000c00 tx=0x7f577c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36bd043e8c0796c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57697fa640 1 -- 172.21.15.86:0/3313348724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c04e070 con 0x55ff092bda90 2024-09-05T06:45:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57697fa640 1 -- 172.21.15.86:0/3313348724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f577c049c20 con 0x55ff092bda90 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.674+0000 7f57697fa640 1 -- 172.21.15.86:0/3313348724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c054710 con 0x55ff092bda90 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- 172.21.15.86:0/3313348724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 msgr2=0x55ff092d4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f577c000c00 tx=0x7f577c009260 comp rx=0 tx=0).stop 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- 172.21.15.86:0/3313348724 shutdown_connections 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092d4e40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092aec80 0x55ff092bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 --2- 172.21.15.86:0/3313348724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092ae360 0x55ff092c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- 172.21.15.86:0/3313348724 >> 172.21.15.86:0/3313348724 conn(0x55ff092aaa30 msgr2=0x55ff092bbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- 172.21.15.86:0/3313348724 shutdown_connections 2024-09-05T06:45:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- 172.21.15.86:0/3313348724 wait complete. 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 -- start start 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.675+0000 7f578bc50740 1 Processor -- start 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092ae360 0x55ff092c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092aec80 0x55ff092c6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57841ae260 con 0x55ff092bda90 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57841adb90 con 0x55ff092aec80 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57841aadd0 con 0x55ff092ae360 2024-09-05T06:45:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f578bc50740 10 client.0 _ll_register_callbacks cb 0x7f577c04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47680/0 (socket says 172.21.15.86:47680) 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 -- 172.21.15.86:0/4168144590 learned_addr learned my addr 172.21.15.86:0/4168144590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57827fc640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092aec80 0x55ff092c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f5782ffd640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092ae360 0x55ff092c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 -- 172.21.15.86:0/4168144590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092ae360 msgr2=0x55ff092c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092ae360 0x55ff092c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 -- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092aec80 msgr2=0x55ff092c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092aec80 0x55ff092c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 -- 172.21.15.86:0/4168144590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff092c18b0 con 0x55ff092bda90 2024-09-05T06:45:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f57837fe640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f577c033000 tx=0x7f577c0318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aa65c58d5b948 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.676+0000 7f5762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c04e040 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f577c0496f0 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577c049c20 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f577c073050 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 10 client.16419 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff093379a0 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f577c049c20 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f577c05c030 con 0x55ff092bda90 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 10 client.16419 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 20 client.16419 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff093380c0 0x55ff0933a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 4 client.16419 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff0921bd60 con 0x55ff093380c0 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.677+0000 7f5782ffd640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff093380c0 0x55ff0933a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f5782ffd640 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff093380c0 0x55ff0933a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e554ae769fc4cf20 server_cookie=dae6ddeb6a4f199a in_seq=0 out_seq=0 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f5762ffd640 10 client.16419 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f5762ffd640 1 -- 172.21.15.86:0/4168144590 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff0921bd60 con 0x55ff093380c0 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f5762ffd640 10 client.16419 handle_command_reply: tid=0 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 client.16419 shutdown 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 msgr2=0x55ff092c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f577c033000 tx=0x7f577c0318d0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 shutdown_connections 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff093380c0 0x55ff0933a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff092bda90 0x55ff092c1080 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff092aec80 0x55ff092c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 --2- 172.21.15.86:0/4168144590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff092ae360 0x55ff092c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 >> 172.21.15.86:0/4168144590 conn(0x55ff092aaa30 msgr2=0x55ff092bc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f5762ffd640 0 client.16419 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 shutdown_connections 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 1 -- 172.21.15.86:0/4168144590 wait complete. 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.678+0000 7f578bc50740 20 client.16419 trim_cache size 0 max 16384 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:45:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:45:21.691 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 msgr2=0x7f576c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 0x7f576c0a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f577c032fc0 tx=0x7f577c0314a0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 msgr2=0x7f5784198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f577400db80 tx=0x7f577400b630 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 shutdown_connections 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f576c0a2cf0 0x7f576c0a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f578410c180 0x7f578419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5784103700 0x7f5784198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 --2- 172.21.15.86:0/62971087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5784102de0 0x7f5784198330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 >> 172.21.15.86:0/62971087 conn(0x7f57840fc7a0 msgr2=0x7f57840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 shutdown_connections 2024-09-05T06:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:21.679+0000 7f5789877640 1 -- 172.21.15.86:0/62971087 wait complete. 2024-09-05T06:45:22.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.97 MB/sec execute 42 sec latency 12167.364 ms 2024-09-05T06:45:22.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:22 smithi115 ceph-mon[34791]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 438 KiB/s wr, 63 op/s 2024-09-05T06:45:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:22 smithi086 ceph-mon[30943]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 438 KiB/s wr, 63 op/s 2024-09-05T06:45:22.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:22 smithi096 ceph-mon[34468]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 438 KiB/s wr, 63 op/s 2024-09-05T06:45:23.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:45:23.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:45:23.300 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:23.511 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.94 MB/sec execute 43 sec latency 13167.486 ms 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 -- 172.21.15.86:0/3458474268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b60101280 msgr2=0x7f1b6010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 --2- 172.21.15.86:0/3458474268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b60101280 0x7f1b6010df40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009900 tx=0x7f1b5002a410 comp rx=0 tx=0).stop 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 -- 172.21.15.86:0/3458474268 shutdown_connections 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 --2- 172.21.15.86:0/3458474268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6010e480 0x7f1b60110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 --2- 172.21.15.86:0/3458474268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b60101280 0x7f1b6010df40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 --2- 172.21.15.86:0/3458474268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 0x7f1b60100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.738+0000 7f1b67749640 1 -- 172.21.15.86:0/3458474268 >> 172.21.15.86:0/3458474268 conn(0x7f1b600fc7a0 msgr2=0x7f1b600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.739+0000 7f1b67749640 1 -- 172.21.15.86:0/3458474268 shutdown_connections 2024-09-05T06:45:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.739+0000 7f1b67749640 1 -- 172.21.15.86:0/3458474268 wait complete. 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.739+0000 7f1b67749640 1 Processor -- start 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.739+0000 7f1b67749640 1 -- start start 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 0x7f1b6019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 0x7f1b6019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60114060 con 0x7f1b6010e480 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60113ee0 con 0x7f1b60101280 2024-09-05T06:45:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b67749640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601141e0 con 0x7f1b601008d0 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36878/0 (socket says 172.21.15.86:36878) 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 -- 172.21.15.86:0/662618387 learned_addr learned my addr 172.21.15.86:0/662618387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b654e5640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 0x7f1b6019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b64ce4640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 0x7f1b6019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 -- 172.21.15.86:0/662618387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 msgr2=0x7f1b6019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 0x7f1b6019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 -- 172.21.15.86:0/662618387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 msgr2=0x7f1b6019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 0x7f1b6019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 -- 172.21.15.86:0/662618387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6019f5b0 con 0x7f1b6010e480 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b64ce4640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 0x7f1b6019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b65ce6640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800cce0 tx=0x7f1b5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.740+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800f450 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.741+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6019f8a0 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.741+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b5800faa0 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.741+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58018af0 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.741+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601a7180 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.741+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b60100cb0 con 0x7f1b6010e480 2024-09-05T06:45:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.743+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1b5800fc50 con 0x7f1b6010e480 2024-09-05T06:45:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.744+0000 7f1b567fc640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 0x7f1b4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.744+0000 7f1b654e5640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 0x7f1b4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.744+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f1b580d0050 con 0x7f1b6010e480 2024-09-05T06:45:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.745+0000 7f1b654e5640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 0x7f1b4c0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c005fd0 tx=0x7f1b5c005e20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:23.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.747+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b58014030 con 0x7f1b6010e480 2024-09-05T06:45:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.946+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b6002d7c0 con 0x7f1b6010e480 2024-09-05T06:45:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.946+0000 7f1b567fc640 1 -- 172.21.15.86:0/662618387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11977 (secure 0 0 0) 0x7f1b5809c920 con 0x7f1b6010e480 2024-09-05T06:45:23.947 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:45:23.947 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"btime":"2024-09-05T06:43:00:425244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:43:00.385198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:45:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 msgr2=0x7f1b4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 0x7f1b4c0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c005fd0 tx=0x7f1b5c005e20 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 msgr2=0x7f1b6019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800cce0 tx=0x7f1b5800a620 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 shutdown_connections 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1b4c0a2cf0 0x7f1b4c0a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6010e480 0x7f1b6019eed0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b60101280 0x7f1b6019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 --2- 172.21.15.86:0/662618387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b601008d0 0x7f1b6019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.948+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 >> 172.21.15.86:0/662618387 conn(0x7f1b600fc7a0 msgr2=0x7f1b600feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.949+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 shutdown_connections 2024-09-05T06:45:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:23.949+0000 7f1b67749640 1 -- 172.21.15.86:0/662618387 wait complete. 2024-09-05T06:45:24.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.92 MB/sec execute 44 sec latency 14167.600 ms 2024-09-05T06:45:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:24 smithi115 ceph-mon[34791]: pgmap v127: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/662618387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:24.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:24 smithi086 ceph-mon[30943]: pgmap v127: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:24.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/662618387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:24.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:45:24.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:24 smithi096 ceph-mon[34468]: pgmap v127: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/662618387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 -- 172.21.15.86:0/3458714135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 msgr2=0x7f3230100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 --2- 172.21.15.86:0/3458714135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f3230100ec0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3220009930 tx=0x7f322002a420 comp rx=0 tx=0).stop 2024-09-05T06:45:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 -- 172.21.15.86:0/3458714135 shutdown_connections 2024-09-05T06:45:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 --2- 172.21.15.86:0/3458714135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230101400 0x7f323010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 --2- 172.21.15.86:0/3458714135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f3230100ec0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 --2- 172.21.15.86:0/3458714135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230106a60 0x7f3230106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.420+0000 7f3237da5640 1 -- 172.21.15.86:0/3458714135 >> 172.21.15.86:0/3458714135 conn(0x7f32300fc780 msgr2=0x7f32300feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- 172.21.15.86:0/3458714135 shutdown_connections 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- 172.21.15.86:0/3458714135 wait complete. 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 Processor -- start 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- start start 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 0x7f3230197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230106a60 0x7f323019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3230110d30 con 0x7f3230100a60 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3230110bb0 con 0x7f3230101400 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3237da5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3230110eb0 con 0x7f3230106a60 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.421+0000 7f3235b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36888/0 (socket says 172.21.15.86:36888) 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 -- 172.21.15.86:0/2197113755 learned_addr learned my addr 172.21.15.86:0/2197113755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3236342640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230106a60 0x7f323019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235340640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 0x7f3230197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 -- 172.21.15.86:0/2197113755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230106a60 msgr2=0x7f323019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230106a60 0x7f323019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 -- 172.21.15.86:0/2197113755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 msgr2=0x7f3230197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 0x7f3230197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 -- 172.21.15.86:0/2197113755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019c280 con 0x7f3230100a60 2024-09-05T06:45:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235340640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 0x7f3230197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3235b41640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b4a0 tx=0x7f322c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3226ffd640 1 -- 172.21.15.86:0/2197113755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c014070 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3226ffd640 1 -- 172.21.15.86:0/2197113755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f322c010ac0 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3226ffd640 1 -- 172.21.15.86:0/2197113755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c017c30 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323019c570 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.422+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a3e50 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.424+0000 7f3226ffd640 1 -- 172.21.15.86:0/2197113755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f322c017070 con 0x7f3230100a60 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 Processor -- start 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 -- start start 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 0x557472a95c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574729f92f0 con 0x557472aa0580 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574729a0cf0 con 0x557472a96d50 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f323a17e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574729f52b0 con 0x557472a957e0 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f3235340640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f3235340640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36894/0 (socket says 172.21.15.86:36894) 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f3235340640 1 -- 172.21.15.86:0/2709615243 learned_addr learned my addr 172.21.15.86:0/2709615243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.425+0000 7f3226ffd640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 0x7f321c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3236342640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3236342640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 0x7f321c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235b41640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 0x557472a95c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 -- 172.21.15.86:0/2709615243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 msgr2=0x557472a95c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3226ffd640 1 -- 172.21.15.86:0/2197113755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f322c0cedf0 con 0x7f3230100a60 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 0x557472a95c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 -- 172.21.15.86:0/2709615243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 msgr2=0x557472a952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 -- 172.21.15.86:0/2709615243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574729853a0 con 0x557472aa0580 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3236342640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235b41640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 0x557472a95c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3235340640 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3220009930 tx=0x7f3220031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa4eb0f57f7ad9f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f3236342640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 0x7f321c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3228005c40 tx=0x7f3228007520 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f321bfff640 1 -- 172.21.15.86:0/2709615243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220035050 con 0x557472aa0580 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f321bfff640 1 -- 172.21.15.86:0/2709615243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3220004030 con 0x557472aa0580 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.426+0000 7f321bfff640 1 -- 172.21.15.86:0/2709615243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322003bae0 con 0x557472aa0580 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- 172.21.15.86:0/2709615243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 msgr2=0x557472aa2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3220009930 tx=0x7f3220031dc0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- 172.21.15.86:0/2709615243 shutdown_connections 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472aa2970 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a957e0 0x557472a95c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 --2- 172.21.15.86:0/2709615243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- 172.21.15.86:0/2709615243 >> 172.21.15.86:0/2709615243 conn(0x557472a5c310 msgr2=0x557472a91e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- 172.21.15.86:0/2709615243 shutdown_connections 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- 172.21.15.86:0/2709615243 wait complete. 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 -- start start 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.427+0000 7f323a17e740 1 Processor -- start 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472a9b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a96d50 0x557472a9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557472a8f7d0 con 0x557472aa0580 2024-09-05T06:45:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574729b4060 con 0x557472a957e0 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557472a8f950 con 0x557472a96d50 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f323a17e740 10 client.0 _ll_register_callbacks cb 0x5574727078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3235b41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472a9b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36896/0 (socket says 172.21.15.86:36896) 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3235b41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472a9b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57914/0 (socket says 172.21.15.86:57914) 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 -- 172.21.15.86:0/858842508 learned_addr learned my addr 172.21.15.86:0/858842508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3235340640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a96d50 0x557472a9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 -- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a96d50 msgr2=0x557472a9ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a96d50 0x557472a9ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 -- 172.21.15.86:0/858842508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 msgr2=0x557472a9b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472a9b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.428+0000 7f3236342640 1 -- 172.21.15.86:0/858842508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747276a3c0 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f3236342640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3228011e80 tx=0x7f3228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4218da51d6d59a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228024030 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3220004030 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3228020690 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3228031050 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557472aac970 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3220004030 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f322801d050 con 0x557472aa0580 2024-09-05T06:45:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 10 client.16437 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 20 client.16437 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472aade30 0x557472ab02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 4 client.16437 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.429+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557472a06860 con 0x557472aade30 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.430+0000 7f3235b41640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472aade30 0x557472ab02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.430+0000 7f3235b41640 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472aade30 0x557472ab02d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee0308705a36cef server_cookie=c8beaefe9aae2f78 in_seq=0 out_seq=0 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.430+0000 7f32197fa640 10 client.16437 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f32197fa640 1 -- 172.21.15.86:0/858842508 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557472a06860 con 0x557472aade30 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f32197fa640 10 client.16437 handle_command_reply: tid=0 2024-09-05T06:45:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 client.16437 shutdown 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 msgr2=0x557472a9bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3228011e80 tx=0x7f3228004a40 comp rx=0 tx=0).stop 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 shutdown_connections 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472aade30 0x557472ab02d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472aa0580 0x557472a9bfc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472a96d50 0x557472a9ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f32197fa640 0 client.16437 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 --2- 172.21.15.86:0/858842508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472a9b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 >> 172.21.15.86:0/858842508 conn(0x557472a5c310 msgr2=0x557472a97b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 shutdown_connections 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 1 -- 172.21.15.86:0/858842508 wait complete. 2024-09-05T06:45:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.431+0000 7f323a17e740 20 client.16437 trim_cache size 0 max 16384 2024-09-05T06:45:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 Processor -- start 2024-09-05T06:45:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 -- start start 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a90be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472a9edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ace30 con 0x557472a957e0 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ae9d0 con 0x557472a96d50 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f323a17e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ad500 con 0x557472aa0580 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f3235b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f3235b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36900/0 (socket says 172.21.15.86:36900) 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.445+0000 7f3235b41640 1 -- 172.21.15.86:0/989817652 learned_addr learned my addr 172.21.15.86:0/989817652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3236342640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472a9edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235340640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a90be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 -- 172.21.15.86:0/989817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 msgr2=0x557472a9edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472a9edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 -- 172.21.15.86:0/989817652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 msgr2=0x557472a90be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a90be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 -- 172.21.15.86:0/989817652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557472a96970 con 0x557472a957e0 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235340640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a90be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f3235b41640 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f322c00be60 tx=0x7f322c01c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e823648d38f852 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f321b7fe640 1 -- 172.21.15.86:0/989817652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c079e60 con 0x557472a957e0 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f321b7fe640 1 -- 172.21.15.86:0/989817652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f322c09f3f0 con 0x557472a957e0 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.446+0000 7f321b7fe640 1 -- 172.21.15.86:0/989817652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c09f0d0 con 0x557472a957e0 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- 172.21.15.86:0/989817652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 msgr2=0x557472a9bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f322c00be60 tx=0x7f322c01c690 comp rx=0 tx=0).stop 2024-09-05T06:45:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- 172.21.15.86:0/989817652 shutdown_connections 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472a9edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a96d50 0x557472a90be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 --2- 172.21.15.86:0/989817652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a957e0 0x557472a9bc10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- 172.21.15.86:0/989817652 >> 172.21.15.86:0/989817652 conn(0x557472a5c310 msgr2=0x557472a9b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- 172.21.15.86:0/989817652 shutdown_connections 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- 172.21.15.86:0/989817652 wait complete. 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 -- start start 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.447+0000 7f323a17e740 1 Processor -- start 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472ab5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472aa8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ae2a0 con 0x557472a96d50 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32301adbd0 con 0x557472a957e0 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32301aae10 con 0x557472aa0580 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 10 client.0 _ll_register_callbacks cb 0x7f32280145b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36914/0 (socket says 172.21.15.86:36914) 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 -- 172.21.15.86:0/2786871263 learned_addr learned my addr 172.21.15.86:0/2786871263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3236342640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472aa8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235b41640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472ab5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 -- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 msgr2=0x557472aa8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472aa8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 -- 172.21.15.86:0/2786871263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 msgr2=0x557472ab5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472ab5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 -- 172.21.15.86:0/2786871263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557472aa9510 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f3235340640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f322003b750 tx=0x7f322003b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74403c68797caf32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f32197fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220052030 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3220041040 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.448+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322c09f3f0 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32200318c0 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 10 client.16443 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557472b19450 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f322c09f3f0 con 0x557472a96d50 2024-09-05T06:45:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f322003bb20 con 0x557472a96d50 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 10 client.16443 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 10 client.16443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 20 client.16443 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472b198f0 0x557472b1bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 4 client.16443 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557472a00d70 con 0x557472b198f0 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.449+0000 7f3235b41640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472b198f0 0x557472b1bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f3235b41640 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472b198f0 0x557472b1bce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=742be8320c8473dc server_cookie=4a6846eb5e70557f in_seq=0 out_seq=0 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f32197fa640 10 client.16443 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f32197fa640 1 -- 172.21.15.86:0/2786871263 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557472a00d70 con 0x557472b198f0 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f32197fa640 10 client.16443 handle_command_reply: tid=0 2024-09-05T06:45:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f323a17e740 1 client.16443 shutdown 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 msgr2=0x557472ab5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f322003b750 tx=0x7f322003b6e0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 shutdown_connections 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.450+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x557472b198f0 0x557472b1bce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557472aa0580 0x557472aa8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557472a96d50 0x557472ab5680 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f32197fa640 0 client.16443 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 --2- 172.21.15.86:0/2786871263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557472a957e0 0x557472ab5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 >> 172.21.15.86:0/2786871263 conn(0x557472a5c310 msgr2=0x557472a9f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 shutdown_connections 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 1 -- 172.21.15.86:0/2786871263 wait complete. 2024-09-05T06:45:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f323a17e740 20 client.16443 trim_cache size 0 max 16384 2024-09-05T06:45:25.460 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 msgr2=0x7f321c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 0x7f321c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3228005c40 tx=0x7f3228007520 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 msgr2=0x7f32301972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b4a0 tx=0x7f322c00b960 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 shutdown_connections 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f321c0a2cf0 0x7f321c0a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3230106a60 0x7f323019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3230101400 0x7f3230197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 --2- 172.21.15.86:0/2197113755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3230100a60 0x7f32301972d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 >> 172.21.15.86:0/2197113755 conn(0x7f32300fc780 msgr2=0x7f32300fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.451+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 shutdown_connections 2024-09-05T06:45:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:25.452+0000 7f3237da5640 1 -- 172.21.15.86:0/2197113755 wait complete. 2024-09-05T06:45:25.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.90 MB/sec execute 45 sec latency 15167.643 ms 2024-09-05T06:45:26.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:45:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:26.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.88 MB/sec execute 46 sec latency 16167.685 ms 2024-09-05T06:45:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:26 smithi086 ceph-mon[30943]: pgmap v128: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:26.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:26 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:26.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:26 smithi096 ceph-mon[34468]: pgmap v128: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:26.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:26 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:26.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 -- 172.21.15.86:0/4262687973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c106f10 msgr2=0x7f487c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 --2- 172.21.15.86:0/4262687973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c106f10 0x7f487c10ea90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f487402a410 comp rx=0 tx=0).stop 2024-09-05T06:45:26.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 -- 172.21.15.86:0/4262687973 shutdown_connections 2024-09-05T06:45:26.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 --2- 172.21.15.86:0/4262687973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c106f10 0x7f487c10ea90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 --2- 172.21.15.86:0/4262687973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c102790 0x7f487c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 --2- 172.21.15.86:0/4262687973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c101de0 0x7f487c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 -- 172.21.15.86:0/4262687973 >> 172.21.15.86:0/4262687973 conn(0x7f487c0fd8c0 msgr2=0x7f487c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 -- 172.21.15.86:0/4262687973 shutdown_connections 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.868+0000 7f48842fa640 1 -- 172.21.15.86:0/4262687973 wait complete. 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 Processor -- start 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 -- start start 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c101de0 0x7f487c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c106f10 0x7f487c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f487c111e20 con 0x7f487c102790 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f487c111ca0 con 0x7f487c106f10 2024-09-05T06:45:26.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.869+0000 7f48842fa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f487c111fa0 con 0x7f487c101de0 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36938/0 (socket says 172.21.15.86:36938) 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 -- 172.21.15.86:0/2599991791 learned_addr learned my addr 172.21.15.86:0/2599991791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4882096640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c101de0 0x7f487c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4882897640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c106f10 0x7f487c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 -- 172.21.15.86:0/2599991791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c101de0 msgr2=0x7f487c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c101de0 0x7f487c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 -- 172.21.15.86:0/2599991791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c106f10 msgr2=0x7f487c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c106f10 0x7f487c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 -- 172.21.15.86:0/2599991791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c19d2f0 con 0x7f487c102790 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f4881895640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d770 tx=0x7f486c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f48737fe640 1 -- 172.21.15.86:0/2599991791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c017070 con 0x7f487c102790 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c19d5e0 con 0x7f487c102790 2024-09-05T06:45:26.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f48737fe640 1 -- 172.21.15.86:0/2599991791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c0054d0 con 0x7f487c102790 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f48737fe640 1 -- 172.21.15.86:0/2599991791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c004cf0 con 0x7f487c102790 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.870+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a4ec0 con 0x7f487c102790 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.873+0000 7f48866d3740 1 Processor -- start 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.873+0000 7f48737fe640 1 -- 172.21.15.86:0/2599991791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f486c004e90 con 0x7f487c102790 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.873+0000 7f48866d3740 1 -- start start 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 0x55934a9e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 0x55934a9f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55934a94f2f0 con 0x55934a9eab40 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55934a8f6cf0 con 0x55934a9e95d0 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48866d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55934a94b2b0 con 0x55934a9f4370 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36942/0 (socket says 172.21.15.86:36942) 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 -- 172.21.15.86:0/3350410778 learned_addr learned my addr 172.21.15.86:0/3350410778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4881895640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 0x55934a9f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882096640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 0x55934a9e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 -- 172.21.15.86:0/3350410778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 msgr2=0x55934a9f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 0x55934a9f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 -- 172.21.15.86:0/3350410778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 msgr2=0x55934a9e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 0x55934a9e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 -- 172.21.15.86:0/3350410778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934a8db3a0 con 0x55934a9eab40 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882096640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 0x55934a9e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4881895640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 0x55934a9f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48737fe640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 0x7f48680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882897640 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f487402a930 tx=0x7f4874032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55faa2319b739599 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4882096640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 0x7f48680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f48737fe640 1 -- 172.21.15.86:0/2599991791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f486c0cf150 con 0x7f487c102790 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.874+0000 7f4870dea640 1 -- 172.21.15.86:0/3350410778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874040070 con 0x55934a9eab40 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f4870dea640 1 -- 172.21.15.86:0/3350410778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48740316a0 con 0x55934a9eab40 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f4870dea640 1 -- 172.21.15.86:0/3350410778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874035040 con 0x55934a9eab40 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 -- 172.21.15.86:0/3350410778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 msgr2=0x55934a9e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f4882096640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 0x7f48680a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55934a9e6ac0 tx=0x7f487800a590 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f487402a930 tx=0x7f4874032e50 comp rx=0 tx=0).stop 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 -- 172.21.15.86:0/3350410778 shutdown_connections 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9f4370 0x55934a9f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9e95d0 0x55934a9e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 --2- 172.21.15.86:0/3350410778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934a9e9090 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 -- 172.21.15.86:0/3350410778 >> 172.21.15.86:0/3350410778 conn(0x55934a9b2310 msgr2=0x55934a9e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 -- 172.21.15.86:0/3350410778 shutdown_connections 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.875+0000 7f48866d3740 1 -- 172.21.15.86:0/3350410778 wait complete. 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 -- start start 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 Processor -- start 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 0x55934a9f1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934a9f1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55934a9f7010 con 0x55934a9e95d0 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55934a90a060 con 0x55934a9f4370 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55934a9f7190 con 0x55934a9eab40 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f48866d3740 10 client.0 _ll_register_callbacks cb 0x55934a65d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36950/0 (socket says 172.21.15.86:36950) 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 -- 172.21.15.86:0/136030662 learned_addr learned my addr 172.21.15.86:0/136030662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882897640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934a9f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4881895640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 0x55934a9f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 -- 172.21.15.86:0/136030662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 msgr2=0x55934a9f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 0x55934a9f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 -- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 msgr2=0x55934a9f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934a9f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4882096640 1 -- 172.21.15.86:0/136030662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934a6c03c0 con 0x55934a9e95d0 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.876+0000 7f4881895640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 0x55934a9f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4882096640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4878011df0 tx=0x7f4878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9be3159843ee2ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:26.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878004d20 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4878005020 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48740316a0 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f487802e050 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 10 client.16455 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55934aa02a80 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48740316a0 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f487801e640 con 0x55934a9e95d0 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 10 client.16455 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 10 client.16455 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:45:26.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 20 client.16455 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa03f40 0x55934aa06370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 4 client.16455 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.877+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55934a95c860 con 0x55934aa03f40 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.878+0000 7f4881895640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa03f40 0x55934aa06370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.878+0000 7f4881895640 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa03f40 0x55934aa06370 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=527191567234a1ab server_cookie=83ce215714a77ae4 in_seq=0 out_seq=0 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.878+0000 7f4865ffb640 10 client.16455 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f4865ffb640 1 -- 172.21.15.86:0/136030662 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55934a95c860 con 0x55934aa03f40 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f4865ffb640 10 client.16455 handle_command_reply: tid=0 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 client.16455 shutdown 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 msgr2=0x55934a9f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4878011df0 tx=0x7f4878004a40 comp rx=0 tx=0).stop 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 shutdown_connections 2024-09-05T06:45:26.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa03f40 0x55934aa06370 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934a9f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9eab40 0x55934a9f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f4865ffb640 0 client.16455 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 --2- 172.21.15.86:0/136030662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9e95d0 0x55934a9f3d70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 >> 172.21.15.86:0/136030662 conn(0x55934a9b2310 msgr2=0x55934a9eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 shutdown_connections 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 1 -- 172.21.15.86:0/136030662 wait complete. 2024-09-05T06:45:26.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.879+0000 7f48866d3740 20 client.16455 trim_cache size 0 max 16384 2024-09-05T06:45:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 Processor -- start 2024-09-05T06:45:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 -- start start 2024-09-05T06:45:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934aa002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934aa0c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1af020 con 0x55934a9eab40 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1b0b90 con 0x55934a9f4370 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f48866d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1af6f0 con 0x55934a9e95d0 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36966/0 (socket says 172.21.15.86:36966) 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 -- 172.21.15.86:0/111603448 learned_addr learned my addr 172.21.15.86:0/111603448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4882897640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934aa0c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4882096640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934aa002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 -- 172.21.15.86:0/111603448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 msgr2=0x55934aa002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934aa002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 -- 172.21.15.86:0/111603448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 msgr2=0x55934aa0c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934aa0c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 -- 172.21.15.86:0/111603448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934aa007f0 con 0x55934a9eab40 2024-09-05T06:45:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4882096640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934aa002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.894+0000 7f4881895640 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f486c0057a0 tx=0x7f486c0d1660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f2c2c5d9d9129 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f4867fff640 1 -- 172.21.15.86:0/111603448 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c020040 con 0x55934a9eab40 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f4867fff640 1 -- 172.21.15.86:0/111603448 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c009c30 con 0x55934a9eab40 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f4867fff640 1 -- 172.21.15.86:0/111603448 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c0cc740 con 0x55934a9eab40 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 -- 172.21.15.86:0/111603448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 msgr2=0x55934aa0bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f486c0057a0 tx=0x7f486c0d1660 comp rx=0 tx=0).stop 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 -- 172.21.15.86:0/111603448 shutdown_connections 2024-09-05T06:45:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9f4370 0x55934aa0c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9eab40 0x55934aa0bc90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 --2- 172.21.15.86:0/111603448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934aa002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 -- 172.21.15.86:0/111603448 >> 172.21.15.86:0/111603448 conn(0x55934a9b2310 msgr2=0x55934a9ec2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 -- 172.21.15.86:0/111603448 shutdown_connections 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.895+0000 7f48866d3740 1 -- 172.21.15.86:0/111603448 wait complete. 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 -- start start 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 Processor -- start 2024-09-05T06:45:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934a9fc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9eab40 0x55934a9fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1b0490 con 0x55934a9f4370 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1afdc0 con 0x55934a9eab40 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1ad000 con 0x55934a9e95d0 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f48866d3740 10 client.0 _ll_register_callbacks cb 0x7f487800ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f4882897640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f4882897640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36974/0 (socket says 172.21.15.86:36974) 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f4882897640 1 -- 172.21.15.86:0/749386565 learned_addr learned my addr 172.21.15.86:0/749386565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f4881895640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9eab40 0x55934a9fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.896+0000 7f4882096640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934a9fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 -- 172.21.15.86:0/749386565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 msgr2=0x55934a9fc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934a9fc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 -- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9eab40 msgr2=0x55934a9fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9eab40 0x55934a9fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 -- 172.21.15.86:0/749386565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934a9f9930 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4882897640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f4874005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9fe608eef0d03e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874052060 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f48740314a0 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55934a956d70 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4874059050 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f48866d3740 10 client.16461 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55934aa6f3c0 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.897+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55934a956d70 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f487404d410 con 0x55934a9f4370 2024-09-05T06:45:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 10 client.16461 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 10 client.16461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 20 client.16461 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa6f750 0x55934aa71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 4 client.16461 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55934a8d3170 con 0x55934aa6f750 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4882096640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa6f750 0x55934aa71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4865ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4882096640 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa6f750 0x55934aa71b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29751ff66b370e94 server_cookie=a36c4f6629f2da4e in_seq=0 out_seq=0 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4865ffb640 1 -- 172.21.15.86:0/749386565 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55934a8d3170 con 0x55934aa6f750 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f4865ffb640 10 client.16461 handle_command_reply: tid=0 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.898+0000 7f48866d3740 1 client.16461 shutdown 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 msgr2=0x55934a9f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f4874005780 comp rx=0 tx=0).stop 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 shutdown_connections 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55934aa6f750 0x55934aa71b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55934a9f4370 0x55934a9f93f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f4865ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55934a9eab40 0x55934a9fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 --2- 172.21.15.86:0/749386565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55934a9e95d0 0x55934a9fc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 >> 172.21.15.86:0/749386565 conn(0x55934a9b2310 msgr2=0x55934a9eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 shutdown_connections 2024-09-05T06:45:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 1 -- 172.21.15.86:0/749386565 wait complete. 2024-09-05T06:45:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48866d3740 20 client.16461 trim_cache size 0 max 16384 2024-09-05T06:45:26.911 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:45:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 msgr2=0x7f48680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 0x7f48680a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55934a9e6ac0 tx=0x7f487800a590 comp rx=0 tx=0).stop 2024-09-05T06:45:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 msgr2=0x7f487c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.899+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d770 tx=0x7f486c00dc30 comp rx=0 tx=0).stop 2024-09-05T06:45:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 shutdown_connections 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48680a2cf0 0x7f48680a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f487c106f10 0x7f487c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f487c102790 0x7f487c198880 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 --2- 172.21.15.86:0/2599991791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f487c101de0 0x7f487c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 >> 172.21.15.86:0/2599991791 conn(0x7f487c0fd8c0 msgr2=0x7f487c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 shutdown_connections 2024-09-05T06:45:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:26.900+0000 7f48842fa640 1 -- 172.21.15.86:0/2599991791 wait complete. 2024-09-05T06:45:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:26 smithi115 ceph-mon[34791]: pgmap v128: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-05T06:45:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:26 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:27.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.86 MB/sec execute 47 sec latency 17167.748 ms 2024-09-05T06:45:27.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:45:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:28.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.85 MB/sec execute 48 sec latency 18167.774 ms 2024-09-05T06:45:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 -- 172.21.15.86:0/4007261162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc101de0 msgr2=0x7fd3dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 --2- 172.21.15.86:0/4007261162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc101de0 0x7fd3dc1021c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009900 tx=0x7fd3cc02a410 comp rx=0 tx=0).stop 2024-09-05T06:45:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 -- 172.21.15.86:0/4007261162 shutdown_connections 2024-09-05T06:45:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 --2- 172.21.15.86:0/4007261162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc106f10 0x7fd3dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 --2- 172.21.15.86:0/4007261162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc102790 0x7fd3dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 --2- 172.21.15.86:0/4007261162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc101de0 0x7fd3dc1021c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 -- 172.21.15.86:0/4007261162 >> 172.21.15.86:0/4007261162 conn(0x7fd3dc0fd8c0 msgr2=0x7fd3dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 -- 172.21.15.86:0/4007261162 shutdown_connections 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.505+0000 7fd3e16db640 1 -- 172.21.15.86:0/4007261162 wait complete. 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 Processor -- start 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 -- start start 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 0x7fd3dc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc106f10 0x7fd3dc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111e40 con 0x7fd3dc102790 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111cc0 con 0x7fd3dc101de0 2024-09-05T06:45:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3e16db640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111fc0 con 0x7fd3dc106f10 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36994/0 (socket says 172.21.15.86:36994) 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 -- 172.21.15.86:0/2407317782 learned_addr learned my addr 172.21.15.86:0/2407317782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3daffd640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 0x7fd3dc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc106f10 0x7fd3dc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 -- 172.21.15.86:0/2407317782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc106f10 msgr2=0x7fd3dc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc106f10 0x7fd3dc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 -- 172.21.15.86:0/2407317782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 msgr2=0x7fd3dc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 0x7fd3dc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 -- 172.21.15.86:0/2407317782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19d3b0 con 0x7fd3dc102790 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3daffd640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 0x7fd3dc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.506+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400b4a0 tx=0x7fd3c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.507+0000 7fd3d8ff9640 1 -- 172.21.15.86:0/2407317782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4014070 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.507+0000 7fd3d8ff9640 1 -- 172.21.15.86:0/2407317782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c4010ac0 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.507+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19d6a0 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.507+0000 7fd3d8ff9640 1 -- 172.21.15.86:0/2407317782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4017c30 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.507+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a4f80 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.509+0000 7fd3d8ff9640 1 -- 172.21.15.86:0/2407317782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd3c4017070 con 0x7fd3dc102790 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.509+0000 7fd3e3ab4740 1 Processor -- start 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 -- start start 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adf1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ad572f0 con 0x55ab0adf15d0 2024-09-05T06:45:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab0acfecf0 con 0x55ab0adfc370 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ad532b0 con 0x55ab0adf2b40 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3d8ff9640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adf1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36996/0 (socket says 172.21.15.86:36996) 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adf1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35796/0 (socket says 172.21.15.86:35796) 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 -- 172.21.15.86:0/695638880 learned_addr learned my addr 172.21.15.86:0/695638880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3d8ff9640 1 -- 172.21.15.86:0/2407317782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd3c40cee10 con 0x7fd3dc102790 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 -- 172.21.15.86:0/695638880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 msgr2=0x55ab0adf1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adf1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 -- 172.21.15.86:0/695638880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 msgr2=0x55ab0adfe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.510+0000 7fd3daffd640 1 -- 172.21.15.86:0/695638880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0ace33a0 con 0x55ab0adf15d0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.511+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 0x7fd3c80a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005fd0 tx=0x7fd3d4004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.511+0000 7fd3daffd640 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02a930 tx=0x7fd3cc032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fda4cb1ca5ba08e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.511+0000 7fd3d17fa640 1 -- 172.21.15.86:0/695638880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc03f070 con 0x55ab0adf15d0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.511+0000 7fd3d17fa640 1 -- 172.21.15.86:0/695638880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3cc031760 con 0x55ab0adf15d0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.511+0000 7fd3d17fa640 1 -- 172.21.15.86:0/695638880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc035070 con 0x55ab0adf15d0 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/695638880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 msgr2=0x55ab0adf1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02a930 tx=0x7fd3cc032e50 comp rx=0 tx=0).stop 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/695638880 shutdown_connections 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adf1a10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/695638880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adf1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/695638880 >> 172.21.15.86:0/695638880 conn(0x55ab0adba310 msgr2=0x55ab0adedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/695638880 shutdown_connections 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/695638880 wait complete. 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- start start 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 Processor -- start 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf15d0 0x55ab0adfbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adf95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab0adff010 con 0x55ab0adf2b40 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ad12060 con 0x55ab0adfc370 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.512+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab0adff190 con 0x55ab0adf15d0 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3e3ab4740 10 client.0 _ll_register_callbacks cb 0x55ab0aa658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37012/0 (socket says 172.21.15.86:37012) 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 -- 172.21.15.86:0/3405079166 learned_addr learned my addr 172.21.15.86:0/3405079166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf15d0 0x55ab0adfbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adf95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 -- 172.21.15.86:0/3405079166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf15d0 msgr2=0x55ab0adfbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf15d0 0x55ab0adfbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 -- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 msgr2=0x55ab0adf95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adf95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 -- 172.21.15.86:0/3405079166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0aac83c0 con 0x55ab0adf2b40 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adf95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4017480 tx=0x7fd3c409b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2b11a7587e2d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c401e040 con 0x55ab0adf2b40 2024-09-05T06:45:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd3c409f0c0 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3cc031760 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3c40a3cd0 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3e3ab4740 10 client.16473 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.513+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ab0ae0ab20 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3cc031760 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd3c4010c70 con 0x55ab0adf2b40 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 10 client.16473 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 10 client.16473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 20 client.16473 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae0e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 4 client.16473 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0ad64860 con 0x55ab0ae0c030 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae0e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3c27fc640 10 client.16473 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.514+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae0e420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9ef1afc59efada server_cookie=94a8e0d6e3e1672e in_seq=0 out_seq=0 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3405079166 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab0ad64860 con 0x55ab0ae0c030 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3c27fc640 10 client.16473 handle_command_reply: tid=0 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3e3ab4740 1 client.16473 shutdown 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 msgr2=0x55ab0adf8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4017480 tx=0x7fd3c409b760 comp rx=0 tx=0).stop 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 shutdown_connections 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.515+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae0e420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3c27fc640 0 client.16473 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adf95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf2b40 0x55ab0adf8fe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3405079166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf15d0 0x55ab0adfbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 >> 172.21.15.86:0/3405079166 conn(0x55ab0adba310 msgr2=0x55ab0adf3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 shutdown_connections 2024-09-05T06:45:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3405079166 wait complete. 2024-09-05T06:45:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.516+0000 7fd3e3ab4740 20 client.16473 trim_cache size 0 max 16384 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 Processor -- start 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 -- start start 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 0x55ab0ae082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ace10 con 0x55ab0adfc370 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ae980 con 0x55ab0adf15d0 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ad4e0 con 0x55ab0adf2b40 2024-09-05T06:45:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37014/0 (socket says 172.21.15.86:37014) 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.529+0000 7fd3db7fe640 1 -- 172.21.15.86:0/601687247 learned_addr learned my addr 172.21.15.86:0/601687247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3daffd640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 0x55ab0ae082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 -- 172.21.15.86:0/601687247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 msgr2=0x55ab0adfd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 -- 172.21.15.86:0/601687247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 msgr2=0x55ab0ae082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 0x55ab0ae082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 -- 172.21.15.86:0/601687247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0adfeba0 con 0x55ab0adfc370 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3daffd640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 0x55ab0ae082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009ef0 tx=0x7fd3d4014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4045ac8c3d23b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3d0ff9640 1 -- 172.21.15.86:0/601687247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40108e0 con 0x55ab0adfc370 2024-09-05T06:45:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3d0ff9640 1 -- 172.21.15.86:0/601687247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d4027070 con 0x55ab0adfc370 2024-09-05T06:45:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.530+0000 7fd3d0ff9640 1 -- 172.21.15.86:0/601687247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d402cbd0 con 0x55ab0adfc370 2024-09-05T06:45:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/601687247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 msgr2=0x55ab0adfdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009ef0 tx=0x7fd3d4014070 comp rx=0 tx=0).stop 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/601687247 shutdown_connections 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adfc370 0x55ab0adfdca0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/601687247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adf15d0 0x55ab0ae082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/601687247 >> 172.21.15.86:0/601687247 conn(0x55ab0adba310 msgr2=0x55ab0adf78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/601687247 shutdown_connections 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/601687247 wait complete. 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 -- start start 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.531+0000 7fd3e3ab4740 1 Processor -- start 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ae280 con 0x55ab0adf15d0 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1adbb0 con 0x55ab0adfc370 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1aadf0 con 0x55ab0adf2b40 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 10 client.0 _ll_register_callbacks cb 0x7fd3c400b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37024/0 (socket says 172.21.15.86:37024) 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 -- 172.21.15.86:0/3905025064 learned_addr learned my addr 172.21.15.86:0/3905025064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 -- 172.21.15.86:0/3905025064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 msgr2=0x55ab0adfda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 -- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 msgr2=0x55ab0adfdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 -- 172.21.15.86:0/3905025064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0ae02a50 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3db7fe640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3daffd640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc031a70 tx=0x7fd3cc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b183116561e6a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc03f040 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd3cc031cf0 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.532+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d4027070 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc035040 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 10 client.16479 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ab0ae77040 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d4027070 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd3cc003710 con 0x55ab0adf15d0 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 10 client.16479 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 10 client.16479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:45:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 20 client.16479 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae79330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 4 client.16479 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab0ad5ed70 con 0x55ab0ae0c030 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.533+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae79330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3d3fff640 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae79330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=739c66c0f3cc359f server_cookie=d6bee204c0d95136 in_seq=0 out_seq=0 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3c27fc640 10 client.16479 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3c27fc640 1 -- 172.21.15.86:0/3905025064 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab0ad5ed70 con 0x55ab0ae0c030 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3c27fc640 10 client.16479 handle_command_reply: tid=0 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 client.16479 shutdown 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 msgr2=0x55ab0adfd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc031a70 tx=0x7fd3cc005d50 comp rx=0 tx=0).stop 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 shutdown_connections 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ab0ae0c030 0x55ab0ae79330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3c27fc640 0 client.16479 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0adfc370 0x55ab0adfdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab0adf2b40 0x55ab0adfda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 --2- 172.21.15.86:0/3905025064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab0adf15d0 0x55ab0adfd510 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 >> 172.21.15.86:0/3905025064 conn(0x55ab0adba310 msgr2=0x55ab0adfb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 shutdown_connections 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 1 -- 172.21.15.86:0/3905025064 wait complete. 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.534+0000 7fd3e3ab4740 20 client.16479 trim_cache size 0 max 16384 2024-09-05T06:45:28.540 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 msgr2=0x7fd3c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 0x7fd3c80a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005fd0 tx=0x7fd3d4004a40 comp rx=0 tx=0).stop 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 msgr2=0x7fd3dc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400b4a0 tx=0x7fd3c400b960 comp rx=0 tx=0).stop 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 shutdown_connections 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3dc106f10 0x7fd3dc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102790 0x7fd3dc198940 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 --2- 172.21.15.86:0/2407317782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3dc101de0 0x7fd3dc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:28.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 >> 172.21.15.86:0/2407317782 conn(0x7fd3dc0fd8c0 msgr2=0x7fd3dc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:28.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 shutdown_connections 2024-09-05T06:45:28.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:28.535+0000 7fd3e16db640 1 -- 172.21.15.86:0/2407317782 wait complete. 2024-09-05T06:45:28.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:28 smithi086 ceph-mon[30943]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-05T06:45:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:28 smithi096 ceph-mon[34468]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-05T06:45:28.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:28 smithi115 ceph-mon[34791]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-05T06:45:29.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:45:29.441 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:29 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:45:29.441 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:29 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:45:29.441 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:29 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:45:29.441 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:29 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:45:29.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:45:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:29.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.83 MB/sec execute 49 sec latency 19167.805 ms 2024-09-05T06:45:29.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:29 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:45:29.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:29 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:45:29.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:29 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:45:29.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:29 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:45:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:29 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:45:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:29 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:45:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:29 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:45:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:29 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:45:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 -- 172.21.15.86:0/3984958543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 msgr2=0x7fd4ec100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3984958543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec100ef0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009930 tx=0x7fd4dc02a420 comp rx=0 tx=0).stop 2024-09-05T06:45:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 -- 172.21.15.86:0/3984958543 shutdown_connections 2024-09-05T06:45:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3984958543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 0x7fd4ec10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3984958543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec100ef0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3984958543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 0x7fd4ec106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 -- 172.21.15.86:0/3984958543 >> 172.21.15.86:0/3984958543 conn(0x7fd4ec0fc7d0 msgr2=0x7fd4ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 -- 172.21.15.86:0/3984958543 shutdown_connections 2024-09-05T06:45:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.975+0000 7fd4f3392640 1 -- 172.21.15.86:0/3984958543 wait complete. 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 Processor -- start 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 -- start start 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 0x7fd4ec1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 0x7fd4ec1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec110d40 con 0x7fd4ec100a90 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec110bc0 con 0x7fd4ec101430 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f3392640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec110ec0 con 0x7fd4ec106a90 2024-09-05T06:45:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f112e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f112e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37044/0 (socket says 172.21.15.86:37044) 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f112e640 1 -- 172.21.15.86:0/3233480365 learned_addr learned my addr 172.21.15.86:0/3233480365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 0x7fd4ec1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.976+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 0x7fd4ec1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 -- 172.21.15.86:0/3233480365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 msgr2=0x7fd4ec1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 0x7fd4ec1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 -- 172.21.15.86:0/3233480365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 msgr2=0x7fd4ec1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 0x7fd4ec1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 -- 172.21.15.86:0/3233480365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec1a4750 con 0x7fd4ec100a90 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800d950 tx=0x7fd4e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4e27fc640 1 -- 172.21.15.86:0/3233480365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8017070 con 0x7fd4ec100a90 2024-09-05T06:45:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4e27fc640 1 -- 172.21.15.86:0/3233480365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4e8009c30 con 0x7fd4ec100a90 2024-09-05T06:45:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1a4920 con 0x7fd4ec100a90 2024-09-05T06:45:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4e27fc640 1 -- 172.21.15.86:0/3233480365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800f620 con 0x7fd4ec100a90 2024-09-05T06:45:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.977+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1a4de0 con 0x7fd4ec100a90 2024-09-05T06:45:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4e27fc640 1 -- 172.21.15.86:0/3233480365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd4e80109a0 con 0x7fd4ec100a90 2024-09-05T06:45:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 Processor -- start 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 -- start start 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b18ee9e2f0 con 0x55b18ef39b40 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b18ee45cf0 con 0x55b18ef385d0 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f576b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b18ee9a2b0 con 0x55b18ef43370 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f192f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f192f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37058/0 (socket says 172.21.15.86:37058) 2024-09-05T06:45:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.980+0000 7fd4f192f640 1 -- 172.21.15.86:0/3032083248 learned_addr learned my addr 172.21.15.86:0/3032083248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4e27fc640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 0x7fd4d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 -- 172.21.15.86:0/3032083248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 msgr2=0x55b18ef45760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef45760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 -- 172.21.15.86:0/3032083248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 msgr2=0x55b18ef38a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef38a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 -- 172.21.15.86:0/3032083248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18ee2a3a0 con 0x55b18ef39b40 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef45760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4e27fc640 1 -- 172.21.15.86:0/3233480365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd4e80cf410 con 0x7fd4ec100a90 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007f40 tx=0x7fd4e400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a5991dbd3ab2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 0x7fd4d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4d77fe640 1 -- 172.21.15.86:0/3032083248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010940 con 0x55b18ef39b40 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4d77fe640 1 -- 172.21.15.86:0/3032083248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4e40183f0 con 0x55b18ef39b40 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.981+0000 7fd4d77fe640 1 -- 172.21.15.86:0/3032083248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017600 con 0x55b18ef39b40 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 -- 172.21.15.86:0/3032083248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 msgr2=0x55b18ef38090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 0x7fd4d80a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd4e401a040 tx=0x7fd4e4017d80 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007f40 tx=0x7fd4e400f850 comp rx=0 tx=0).stop 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 -- 172.21.15.86:0/3032083248 shutdown_connections 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef45760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef38a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3032083248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef38090 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 -- 172.21.15.86:0/3032083248 >> 172.21.15.86:0/3032083248 conn(0x55b18ef01310 msgr2=0x55b18ef34c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 -- 172.21.15.86:0/3032083248 shutdown_connections 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.982+0000 7fd4f576b740 1 -- 172.21.15.86:0/3032083248 wait complete. 2024-09-05T06:45:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 -- start start 2024-09-05T06:45:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 Processor -- start 2024-09-05T06:45:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef42cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b18ef46070 con 0x55b18ef39b40 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b18ee59060 con 0x55b18ef385d0 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b18ef461f0 con 0x55b18ef43370 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f576b740 10 client.0 _ll_register_callbacks cb 0x55b18ebac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37060/0 (socket says 172.21.15.86:37060) 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.983+0000 7fd4f092d640 1 -- 172.21.15.86:0/511027205 learned_addr learned my addr 172.21.15.86:0/511027205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f192f640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f112e640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef42cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 -- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 msgr2=0x55b18ef404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 -- 172.21.15.86:0/511027205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 msgr2=0x55b18ef42cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef42cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 -- 172.21.15.86:0/511027205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18ec0f3c0 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f192f640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f112e640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef42cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f092d640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02a940 tx=0x7fd4dc0318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80693464743aa02a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc035040 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd4dc004380 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e40183f0 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc04e050 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f576b740 10 client.16491 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.984+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b18ef51ab0 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4e40183f0 con 0x55b18ef39b40 2024-09-05T06:45:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd4dc004c20 con 0x55b18ef39b40 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 10 client.16491 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 10 client.16491 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 20 client.16491 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18ef52f70 0x55b18ef553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 4 client.16491 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18eeab860 con 0x55b18ef52f70 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.985+0000 7fd4f112e640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18ef52f70 0x55b18ef553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4d4ff9640 10 client.16491 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f112e640 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18ef52f70 0x55b18ef553f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80fbcca622b0c8 server_cookie=74190a11df65de89 in_seq=0 out_seq=0 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/511027205 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b18eeab860 con 0x55b18ef52f70 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4d4ff9640 10 client.16491 handle_command_reply: tid=0 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f576b740 1 client.16491 shutdown 2024-09-05T06:45:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 msgr2=0x55b18ef3ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02a940 tx=0x7fd4dc0318e0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 shutdown_connections 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.986+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18ef52f70 0x55b18ef553f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef43370 0x55b18ef404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef3ff60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 --2- 172.21.15.86:0/511027205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef385d0 0x55b18ef42cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 >> 172.21.15.86:0/511027205 conn(0x55b18ef01310 msgr2=0x55b18ef3a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4d4ff9640 0 client.16491 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 shutdown_connections 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 1 -- 172.21.15.86:0/511027205 wait complete. 2024-09-05T06:45:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:29.987+0000 7fd4f576b740 20 client.16491 trim_cache size 0 max 16384 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.000+0000 7fd4f576b740 1 Processor -- start 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 -- start start 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1aef50 con 0x55b18ef39b40 2024-09-05T06:45:30.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1b0ac0 con 0x55b18ef43370 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f576b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1af620 con 0x55b18ef385d0 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37070/0 (socket says 172.21.15.86:37070) 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 -- 172.21.15.86:0/2852715288 learned_addr learned my addr 172.21.15.86:0/2852715288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f112e640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f192f640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 -- 172.21.15.86:0/2852715288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 msgr2=0x55b18ef4f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef4f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 -- 172.21.15.86:0/2852715288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 msgr2=0x55b18ef3ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef3ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.001+0000 7fd4f092d640 1 -- 172.21.15.86:0/2852715288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18ef3e4b0 con 0x55b18ef39b40 2024-09-05T06:45:30.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.002+0000 7fd4f192f640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef3ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:30.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.002+0000 7fd4f092d640 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc000c00 tx=0x7fd4dc003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fb979e2d535d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:30.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.002+0000 7fd4d6ffd640 1 -- 172.21.15.86:0/2852715288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc057070 con 0x55b18ef39b40 2024-09-05T06:45:30.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.002+0000 7fd4d6ffd640 1 -- 172.21.15.86:0/2852715288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4dc003dd0 con 0x55b18ef39b40 2024-09-05T06:45:30.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.002+0000 7fd4d6ffd640 1 -- 172.21.15.86:0/2852715288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc035070 con 0x55b18ef39b40 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- 172.21.15.86:0/2852715288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 msgr2=0x55b18ef54f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc000c00 tx=0x7fd4dc003a10 comp rx=0 tx=0).stop 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- 172.21.15.86:0/2852715288 shutdown_connections 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef3ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef54f80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 --2- 172.21.15.86:0/2852715288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- 172.21.15.86:0/2852715288 >> 172.21.15.86:0/2852715288 conn(0x55b18ef01310 msgr2=0x55b18ef3e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- 172.21.15.86:0/2852715288 shutdown_connections 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- 172.21.15.86:0/2852715288 wait complete. 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.003+0000 7fd4f576b740 1 -- start start 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 Processor -- start 2024-09-05T06:45:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef47b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1b03c0 con 0x55b18ef39b40 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1afcf0 con 0x55b18ef43370 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1acf30 con 0x55b18ef385d0 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f576b740 10 client.0 _ll_register_callbacks cb 0x7fd4dc032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f112e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f092d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37080/0 (socket says 172.21.15.86:37080) 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.004+0000 7fd4f092d640 1 -- 172.21.15.86:0/3884210288 learned_addr learned my addr 172.21.15.86:0/3884210288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f192f640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef47b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 -- 172.21.15.86:0/3884210288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 msgr2=0x55b18ef47b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef47b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 -- 172.21.15.86:0/3884210288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 msgr2=0x55b18ef47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 -- 172.21.15.86:0/3884210288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18efc10a0 con 0x55b18ef385d0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f112e640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef470f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cc780 tx=0x7fd4e80d2f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6b0e1633288aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8020040 con 0x55b18ef385d0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd4e80cca00 con 0x55b18ef385d0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4dc003dd0 con 0x55b18ef385d0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4e809b090 con 0x55b18ef385d0 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f576b740 10 client.25831 fetch_fsmap learned FSMap version 31 2024-09-05T06:45:30.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.005+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b18efc2210 con 0x55b18ef385d0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4dc003dd0 con 0x55b18ef385d0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd4e8097bc0 con 0x55b18ef385d0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 10 client.25831 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 10 client.25831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 20 client.25831 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18efc2930 0x55b18efc4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 4 client.25831 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b18eea5d70 con 0x55b18efc2930 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18efc2930 0x55b18efc4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4d4ff9640 10 client.25831 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.006+0000 7fd4f092d640 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18efc2930 0x55b18efc4d20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dacf4cd654416a70 server_cookie=8b94974a626135c1 in_seq=0 out_seq=0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4d4ff9640 1 -- 172.21.15.86:0/3884210288 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b18eea5d70 con 0x55b18efc2930 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4d4ff9640 10 client.25831 handle_command_reply: tid=0 2024-09-05T06:45:30.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 client.25831 shutdown 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 msgr2=0x55b18ef470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef470f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cc780 tx=0x7fd4e80d2f60 comp rx=0 tx=0).stop 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 shutdown_connections 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b18efc2930 0x55b18efc4d20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18ef43370 0x55b18ef47b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4d4ff9640 0 client.25831 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b18ef39b40 0x55b18ef47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 --2- 172.21.15.86:0/3884210288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b18ef385d0 0x55b18ef470f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 >> 172.21.15.86:0/3884210288 conn(0x55b18ef01310 msgr2=0x55b18ef3f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 shutdown_connections 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 1 -- 172.21.15.86:0/3884210288 wait complete. 2024-09-05T06:45:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.007+0000 7fd4f576b740 20 client.25831 trim_cache size 0 max 16384 2024-09-05T06:45:30.013 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:45:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 msgr2=0x7fd4d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 0x7fd4d80a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd4e401a040 tx=0x7fd4e4017d80 comp rx=0 tx=0).stop 2024-09-05T06:45:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 msgr2=0x7fd4ec199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800d950 tx=0x7fd4e800de10 comp rx=0 tx=0).stop 2024-09-05T06:45:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 shutdown_connections 2024-09-05T06:45:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4d80a2cf0 0x7fd4d80a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4ec106a90 0x7fd4ec1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4ec101430 0x7fd4ec1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 --2- 172.21.15.86:0/3233480365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ec100a90 0x7fd4ec199190 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 >> 172.21.15.86:0/3233480365 conn(0x7fd4ec0fc7d0 msgr2=0x7fd4ec0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 shutdown_connections 2024-09-05T06:45:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:30.008+0000 7fd4f3392640 1 -- 172.21.15.86:0/3233480365 wait complete. 2024-09-05T06:45:30.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.81 MB/sec execute 50 sec latency 20167.853 ms 2024-09-05T06:45:30.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:30 smithi086 ceph-mon[30943]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-05T06:45:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:30 smithi096 ceph-mon[34468]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-05T06:45:30.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:30 smithi115 ceph-mon[34791]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 9 op/s 2024-09-05T06:45:31.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 -- 172.21.15.86:0/2429782672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 msgr2=0x7f506c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 --2- 172.21.15.86:0/2429782672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c107ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5068009930 tx=0x7f506802a420 comp rx=0 tx=0).stop 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 -- 172.21.15.86:0/2429782672 shutdown_connections 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 --2- 172.21.15.86:0/2429782672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c1082e0 0x7f506c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 --2- 172.21.15.86:0/2429782672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c107ba0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 --2- 172.21.15.86:0/2429782672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c102db0 0x7f506c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 -- 172.21.15.86:0/2429782672 >> 172.21.15.86:0/2429782672 conn(0x7f506c0fc7d0 msgr2=0x7f506c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 -- 172.21.15.86:0/2429782672 shutdown_connections 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.274+0000 7f50734a6640 1 -- 172.21.15.86:0/2429782672 wait complete. 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 Processor -- start 2024-09-05T06:45:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 -- start start 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c102db0 0x7f506c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c1082e0 0x7f506c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f506c111fa0 con 0x7f506c103760 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f506c111e20 con 0x7f506c102db0 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f50734a6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f506c112120 con 0x7f506c1082e0 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f5070a41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f5070a41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37092/0 (socket says 172.21.15.86:37092) 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f5070a41640 1 -- 172.21.15.86:0/920862896 learned_addr learned my addr 172.21.15.86:0/920862896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f5071242640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c102db0 0x7f506c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.275+0000 7f5071a43640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c1082e0 0x7f506c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 -- 172.21.15.86:0/920862896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c1082e0 msgr2=0x7f506c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c1082e0 0x7f506c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 -- 172.21.15.86:0/920862896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c102db0 msgr2=0x7f506c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c102db0 0x7f506c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 -- 172.21.15.86:0/920862896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c19d560 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f5070a41640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5068005940 tx=0x7f50680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506803f030 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5068003cb0 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506c19d7f0 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068035070 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.276+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506c1a5130 con 0x7f506c103760 2024-09-05T06:45:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.277+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506c06a7d0 con 0x7f506c103760 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.278+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5068042030 con 0x7f506c103760 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.279+0000 7f50627fc640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 0x7f50580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.279+0000 7f5071242640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 0x7f50580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.280+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f50680f0190 con 0x7f506c103760 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.280+0000 7f5071242640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 0x7f50580a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f505c005fd0 tx=0x7f505c00b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.282+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50680bc970 con 0x7f506c103760 2024-09-05T06:45:31.368 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:31.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.502+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f506c10a2c0 con 0x7f506c103760 2024-09-05T06:45:31.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.503+0000 7f50627fc640 1 -- 172.21.15.86:0/920862896 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f50680bc270 con 0x7f506c103760 2024-09-05T06:45:31.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:45:31.503 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:45:31.504 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:45:31.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 msgr2=0x7f50580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 0x7f50580a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f505c005fd0 tx=0x7f505c00b390 comp rx=0 tx=0).stop 2024-09-05T06:45:31.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 msgr2=0x7f506c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5068005940 tx=0x7f50680042f0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 shutdown_connections 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50580a2cf0 0x7f50580a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f506c1082e0 0x7f506c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f506c103760 0x7f506c198af0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 --2- 172.21.15.86:0/920862896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c102db0 0x7f506c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 >> 172.21.15.86:0/920862896 conn(0x7f506c0fc7d0 msgr2=0x7f506c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 shutdown_connections 2024-09-05T06:45:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.505+0000 7f50734a6640 1 -- 172.21.15.86:0/920862896 wait complete. 2024-09-05T06:45:31.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.80 MB/sec execute 51 sec latency 21167.879 ms 2024-09-05T06:45:31.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:31 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:31.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:31 smithi086 ceph-mon[30943]: pgmap v131: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-05T06:45:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:31 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:31 smithi096 ceph-mon[34468]: pgmap v131: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-05T06:45:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 -- 172.21.15.86:0/3872328806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2410c170 msgr2=0x7fdc2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/3872328806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2410c170 0x7fdc2410e610 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009ca0 tx=0x7fdc1c02a950 comp rx=0 tx=0).stop 2024-09-05T06:45:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 -- 172.21.15.86:0/3872328806 shutdown_connections 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/3872328806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc2410c170 0x7fdc2410e610 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/3872328806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/3872328806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc24102d40 0x7fdc24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 -- 172.21.15.86:0/3872328806 >> 172.21.15.86:0/3872328806 conn(0x7fdc240fc7a0 msgr2=0x7fdc240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.870+0000 7fdc2a99b640 1 -- 172.21.15.86:0/3872328806 shutdown_connections 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 -- 172.21.15.86:0/3872328806 wait complete. 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 Processor -- start 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 -- start start 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc24198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2410c170 0x7fdc2419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24111e00 con 0x7fdc24102d40 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24111c80 con 0x7fdc2410c170 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.871+0000 7fdc2a99b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24111f80 con 0x7fdc24103660 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37110/0 (socket says 172.21.15.86:37110) 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 -- 172.21.15.86:0/4146741367 learned_addr learned my addr 172.21.15.86:0/4146741367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc28f38640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2410c170 0x7fdc2419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc237fe640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc24198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 -- 172.21.15.86:0/4146741367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 msgr2=0x7fdc24198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc24198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 -- 172.21.15.86:0/4146741367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2410c170 msgr2=0x7fdc2419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2410c170 0x7fdc2419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 -- 172.21.15.86:0/4146741367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2419d2f0 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc237fe640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc24198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc23fff640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d950 tx=0x7fdc1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc217fa640 1 -- 172.21.15.86:0/4146741367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14017070 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2419d5e0 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc217fa640 1 -- 172.21.15.86:0/4146741367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc14009c30 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc217fa640 1 -- 172.21.15.86:0/4146741367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400f620 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.872+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a4ec0 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.875+0000 7fdc217fa640 1 -- 172.21.15.86:0/4146741367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc140109a0 con 0x7fdc24102d40 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.875+0000 7fdc2cd74740 1 Processor -- start 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.875+0000 7fdc2cd74740 1 -- start start 2024-09-05T06:45:31.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c540a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 0x561a1c551b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a1c4a62f0 con 0x561a1c541b40 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a1c44dcf0 con 0x561a1c5405d0 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a1c4a22b0 con 0x561a1c54f780 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37122/0 (socket says 172.21.15.86:37122) 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 -- 172.21.15.86:0/901739569 learned_addr learned my addr 172.21.15.86:0/901739569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc23fff640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c540a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc237fe640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 0x561a1c551b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 -- 172.21.15.86:0/901739569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 msgr2=0x561a1c551b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc217fa640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 0x7fdc0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 0x561a1c551b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 -- 172.21.15.86:0/901739569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 msgr2=0x561a1c540a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c540a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 -- 172.21.15.86:0/901739569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1c4323a0 con 0x561a1c541b40 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc237fe640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 0x561a1c551b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc217fa640 1 -- 172.21.15.86:0/4146741367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc140cf480 con 0x7fdc24102d40 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 0x7fdc0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.876+0000 7fdc28f38640 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009ca0 tx=0x7fdc1c02fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b59d4c5bfd9e57a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc0a7fc640 1 -- 172.21.15.86:0/901739569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c034040 con 0x561a1c541b40 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc0a7fc640 1 -- 172.21.15.86:0/901739569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc1c004040 con 0x561a1c541b40 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc0a7fc640 1 -- 172.21.15.86:0/901739569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c030680 con 0x561a1c541b40 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 -- 172.21.15.86:0/901739569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 msgr2=0x561a1c540090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc28f38640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 0x7fdc0c0a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c04a000 tx=0x7fdc1c04d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009ca0 tx=0x7fdc1c02fd40 comp rx=0 tx=0).stop 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 -- 172.21.15.86:0/901739569 shutdown_connections 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c54f780 0x561a1c551b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c540a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/901739569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c541b40 0x561a1c540090 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 -- 172.21.15.86:0/901739569 >> 172.21.15.86:0/901739569 conn(0x561a1c509310 msgr2=0x561a1c53cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 -- 172.21.15.86:0/901739569 shutdown_connections 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.877+0000 7fdc2cd74740 1 -- 172.21.15.86:0/901739569 wait complete. 2024-09-05T06:45:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 -- start start 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 Processor -- start 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c54f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c54c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a1c552420 con 0x561a1c54f780 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a1c461060 con 0x561a1c5405d0 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a1c5525a0 con 0x561a1c541b40 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc2cd74740 10 client.0 _ll_register_callbacks cb 0x561a1c1b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37124/0 (socket says 172.21.15.86:37124) 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.878+0000 7fdc28f38640 1 -- 172.21.15.86:0/3752483310 learned_addr learned my addr 172.21.15.86:0/3752483310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc237fe640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c54c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc23fff640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c54f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 -- 172.21.15.86:0/3752483310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 msgr2=0x561a1c54c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c54c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 -- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 msgr2=0x561a1c54f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c54f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 -- 172.21.15.86:0/3752483310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1c2173c0 con 0x561a1c54f780 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005e30 tx=0x7fdc1c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8cce628e532037 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c034050 con 0x561a1c54f780 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc1c004b40 con 0x561a1c54f780 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1c004040 con 0x561a1c54f780 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdc1c064050 con 0x561a1c54f780 2024-09-05T06:45:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc2cd74740 10 client.16515 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a1c55dde0 con 0x561a1c54f780 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc1c004040 con 0x561a1c54f780 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.879+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdc1c05f030 con 0x561a1c54f780 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 20 client.16515 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c55f130 0x561a1c561520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 4 client.16515 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1c4b3860 con 0x561a1c55f130 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc23fff640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c55f130 0x561a1c561520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc037fe640 10 client.16515 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.880+0000 7fdc23fff640 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c55f130 0x561a1c561520 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2589f7f573c4f745 server_cookie=67227af974e6eaa8 in_seq=0 out_seq=0 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc037fe640 1 -- 172.21.15.86:0/3752483310 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a1c4b3860 con 0x561a1c55f130 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc037fe640 10 client.16515 handle_command_reply: tid=0 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 client.16515 shutdown 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 msgr2=0x561a1c54c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005e30 tx=0x7fdc1c0095e0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 shutdown_connections 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c55f130 0x561a1c561520 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c54c980 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc037fe640 0 client.16515 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c54c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3752483310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c5405d0 0x561a1c54f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 >> 172.21.15.86:0/3752483310 conn(0x561a1c509310 msgr2=0x561a1c542930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 shutdown_connections 2024-09-05T06:45:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3752483310 wait complete. 2024-09-05T06:45:31.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.881+0000 7fdc2cd74740 20 client.16515 trim_cache size 0 max 16384 2024-09-05T06:45:31.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:31 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:45:31.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:31 smithi115 ceph-mon[34791]: pgmap v131: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.894+0000 7fdc2cd74740 1 Processor -- start 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.894+0000 7fdc2cd74740 1 -- start start 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c5405d0 0x561a1c53ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c541b40 0x561a1c55b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241ace30 con 0x561a1c54f780 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241ae970 con 0x561a1c541b40 2024-09-05T06:45:31.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241ad500 con 0x561a1c5405d0 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37130/0 (socket says 172.21.15.86:37130) 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 -- 172.21.15.86:0/3138791008 learned_addr learned my addr 172.21.15.86:0/3138791008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc23fff640 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c5405d0 0x561a1c53ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 -- 172.21.15.86:0/3138791008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c5405d0 msgr2=0x561a1c53ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c5405d0 0x561a1c53ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 -- 172.21.15.86:0/3138791008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c541b40 msgr2=0x561a1c55b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc237fe640 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c541b40 0x561a1c55b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c541b40 0x561a1c55b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 -- 172.21.15.86:0/3138791008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1c53c4f0 con 0x561a1c54f780 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc28f38640 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c030040 tx=0x7fdc1c02fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c319667b64cbe92a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc09ffb640 1 -- 172.21.15.86:0/3138791008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c034040 con 0x561a1c54f780 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc09ffb640 1 -- 172.21.15.86:0/3138791008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc1c030c40 con 0x561a1c54f780 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.895+0000 7fdc09ffb640 1 -- 172.21.15.86:0/3138791008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c053040 con 0x561a1c54f780 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3138791008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 msgr2=0x561a1c566b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c030040 tx=0x7fdc1c02fa50 comp rx=0 tx=0).stop 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3138791008 shutdown_connections 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c54f780 0x561a1c566b50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c541b40 0x561a1c55b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/3138791008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c5405d0 0x561a1c53ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3138791008 >> 172.21.15.86:0/3138791008 conn(0x561a1c509310 msgr2=0x561a1c53c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3138791008 shutdown_connections 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- 172.21.15.86:0/3138791008 wait complete. 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.896+0000 7fdc2cd74740 1 -- start start 2024-09-05T06:45:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 Processor -- start 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c557d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c54f780 0x561a1c554800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241ae2a0 con 0x561a1c5405d0 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241adbd0 con 0x561a1c54f780 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241aae10 con 0x561a1c541b40 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc2cd74740 10 client.0 _ll_register_callbacks cb 0x7fdc1c005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37132/0 (socket says 172.21.15.86:37132) 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 -- 172.21.15.86:0/2320964830 learned_addr learned my addr 172.21.15.86:0/2320964830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc237fe640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c557d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc28f38640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c54f780 0x561a1c554800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 -- 172.21.15.86:0/2320964830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 msgr2=0x561a1c557d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c557d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 -- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c54f780 msgr2=0x561a1c554800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c54f780 0x561a1c554800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.897+0000 7fdc23fff640 1 -- 172.21.15.86:0/2320964830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1c5583e0 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc23fff640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdc1409b2c0 tx=0x7fdc1400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a07bf0cbd3d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14020040 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdc14084860 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1c030c40 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdc14009de0 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 10 client.16521 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a1c5ca0a0 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc1c030c40 con 0x561a1c5405d0 2024-09-05T06:45:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdc140cf120 con 0x561a1c5405d0 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 10 client.16521 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 20 client.16521 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c5ca770 0x561a1c5ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 4 client.16521 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a1c4add70 con 0x561a1c5ca770 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.898+0000 7fdc237fe640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c5ca770 0x561a1c5ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc237fe640 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c5ca770 0x561a1c5ccb60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e83bf3b9cf3176d server_cookie=eed6a69ce15519b in_seq=0 out_seq=0 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc037fe640 10 client.16521 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc037fe640 1 -- 172.21.15.86:0/2320964830 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a1c4add70 con 0x561a1c5ca770 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc037fe640 10 client.16521 handle_command_reply: tid=0 2024-09-05T06:45:31.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 client.16521 shutdown 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 msgr2=0x561a1c557830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdc1409b2c0 tx=0x7fdc1400bf60 comp rx=0 tx=0).stop 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 shutdown_connections 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561a1c5ca770 0x561a1c5ccb60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a1c54f780 0x561a1c554800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a1c541b40 0x561a1c557d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc037fe640 0 client.16521 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 --2- 172.21.15.86:0/2320964830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a1c5405d0 0x561a1c557830 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 >> 172.21.15.86:0/2320964830 conn(0x561a1c509310 msgr2=0x561a1c54dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 shutdown_connections 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.899+0000 7fdc2cd74740 1 -- 172.21.15.86:0/2320964830 wait complete. 2024-09-05T06:45:31.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2cd74740 20 client.16521 trim_cache size 0 max 16384 2024-09-05T06:45:31.947 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:45:31.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 msgr2=0x7fdc0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 0x7fdc0c0a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c04a000 tx=0x7fdc1c04d000 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 msgr2=0x7fdc24198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d950 tx=0x7fdc1400de10 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 shutdown_connections 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdc0c0a2cf0 0x7fdc0c0a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2410c170 0x7fdc2419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc24103660 0x7fdc24198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 --2- 172.21.15.86:0/4146741367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc24102d40 0x7fdc24198340 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 >> 172.21.15.86:0/4146741367 conn(0x7fdc240fc7a0 msgr2=0x7fdc240fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 shutdown_connections 2024-09-05T06:45:31.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:31.900+0000 7fdc2a99b640 1 -- 172.21.15.86:0/4146741367 wait complete. 2024-09-05T06:45:32.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:32 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/920862896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:32.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.78 MB/sec execute 52 sec latency 22167.951 ms 2024-09-05T06:45:32.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:32 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/920862896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:32.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:32 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/920862896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:45:33.389 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 73dfa2a0 --release --await' 2024-09-05T06:45:33.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 73dfa2a0 --release --await 2024-09-05T06:45:33.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 62925 0.77 MB/sec execute 53 sec latency 23167.995 ms 2024-09-05T06:45:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:33.691 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:45:33.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:45:33.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:33 smithi086 ceph-mon[30943]: pgmap v132: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:33 smithi096 ceph-mon[34468]: pgmap v132: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:33 smithi115 ceph-mon[34791]: pgmap v132: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 -- 172.21.15.86:0/1415865570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 msgr2=0x7f6a9c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/1415865570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1029b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009900 tx=0x7f6a8c02a480 comp rx=0 tx=0).stop 2024-09-05T06:45:34.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 -- 172.21.15.86:0/1415865570 shutdown_connections 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/1415865570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 0x7f6a9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/1415865570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1029b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/1415865570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 0x7f6a9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 -- 172.21.15.86:0/1415865570 >> 172.21.15.86:0/1415865570 conn(0x7f6a9c0fc7d0 msgr2=0x7f6a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 -- 172.21.15.86:0/1415865570 shutdown_connections 2024-09-05T06:45:34.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.141+0000 7f6aa41f4640 1 -- 172.21.15.86:0/1415865570 wait complete. 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 Processor -- start 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 -- start start 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 0x7f6a9c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 0x7f6a9c1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c110e30 con 0x7f6a9c102570 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c110cb0 con 0x7f6a9c100e20 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa41f4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c110fb0 con 0x7f6a9c102ef0 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa178f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa178f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34696/0 (socket says 172.21.15.86:34696) 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa178f640 1 -- 172.21.15.86:0/3540451398 learned_addr learned my addr 172.21.15.86:0/3540451398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 0x7f6a9c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa2791640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 0x7f6a9c1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.142+0000 7f6aa178f640 1 -- 172.21.15.86:0/3540451398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 msgr2=0x7f6a9c1a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa178f640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 0x7f6a9c1a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa178f640 1 -- 172.21.15.86:0/3540451398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 msgr2=0x7f6a9c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa178f640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 0x7f6a9c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa178f640 1 -- 172.21.15.86:0/3540451398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c1a4740 con 0x7f6a9c102570 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa178f640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006010 tx=0x7f6a8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6a92ffd640 1 -- 172.21.15.86:0/3540451398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c03f030 con 0x7f6a9c102570 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6a92ffd640 1 -- 172.21.15.86:0/3540451398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a8c003aa0 con 0x7f6a9c102570 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c196f70 con 0x7f6a9c102570 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6a92ffd640 1 -- 172.21.15.86:0/3540451398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c035070 con 0x7f6a9c102570 2024-09-05T06:45:34.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.143+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c197440 con 0x7f6a9c102570 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.145+0000 7f6a92ffd640 1 -- 172.21.15.86:0/3540451398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6a8c042030 con 0x7f6a9c102570 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 Processor -- start 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 -- start start 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca060 0x556d50fca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d50f2f620 con 0x556d50fca980 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d50ed6e50 con 0x556d50fd5240 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556d50f2b5e0 con 0x556d50fca060 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34708/0 (socket says 172.21.15.86:34708) 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa1f90640 1 -- 172.21.15.86:0/2296110689 learned_addr learned my addr 172.21.15.86:0/2296110689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6a92ffd640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 0x7f6a880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa2791640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca060 0x556d50fca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.146+0000 7f6aa178f640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 -- 172.21.15.86:0/2296110689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca060 msgr2=0x556d50fca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca060 0x556d50fca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 -- 172.21.15.86:0/2296110689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 msgr2=0x556d50fd7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa2791640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 0x7f6a880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 -- 172.21.15.86:0/2296110689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d50ec7d40 con 0x556d50fca980 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa178f640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6a92ffd640 1 -- 172.21.15.86:0/3540451398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6a8c0efbf0 con 0x7f6a9c102570 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ad00 tx=0x7f6a9800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34830f063ec4e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6a87fff640 1 -- 172.21.15.86:0/2296110689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801d070 con 0x556d50fca980 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6a87fff640 1 -- 172.21.15.86:0/2296110689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a98018af0 con 0x556d50fca980 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.147+0000 7f6a87fff640 1 -- 172.21.15.86:0/2296110689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98010b40 con 0x556d50fca980 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 -- 172.21.15.86:0/2296110689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 msgr2=0x556d50fd4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ad00 tx=0x7f6a9800f790 comp rx=0 tx=0).stop 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa2791640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 0x7f6a880a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005c40 tx=0x7f6a94009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 -- 172.21.15.86:0/2296110689 shutdown_connections 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd4d00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/2296110689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca060 0x556d50fca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 -- 172.21.15.86:0/2296110689 >> 172.21.15.86:0/2296110689 conn(0x556d50fc65f0 msgr2=0x556d50fc69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 -- 172.21.15.86:0/2296110689 shutdown_connections 2024-09-05T06:45:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.148+0000 7f6aa65cd740 1 -- 172.21.15.86:0/2296110689 wait complete. 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 -- start start 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 Processor -- start 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fd2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d50fd7e10 con 0x556d50fca060 2024-09-05T06:45:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d50eb7f10 con 0x556d50fd5240 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556d50fd7f90 con 0x556d50fca980 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa65cd740 10 client.0 _ll_register_callbacks cb 0x556d50edbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.149+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34718/0 (socket says 172.21.15.86:34718) 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 -- 172.21.15.86:0/909830418 learned_addr learned my addr 172.21.15.86:0/909830418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa178f640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fd2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa2791640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 -- 172.21.15.86:0/909830418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 msgr2=0x556d50fd2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fd2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 -- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 msgr2=0x556d50fd2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 -- 172.21.15.86:0/909830418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d50c3e580 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa2791640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6a9801b040 tx=0x7f6a98018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6972142fdb79e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801d070 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6a98010a80 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.150+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98018af0 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a98030050 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 10 client.16533 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d50d08960 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a98018af0 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6a980046c0 con 0x556d50fca060 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 10 client.16533 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 10 client.16533 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 20 client.16533 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d50fe4cd0 0x556d50fe70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 4 client.16533 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d50f3cb90 con 0x556d50fe4cd0 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.151+0000 7f6aa178f640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d50fe4cd0 0x556d50fe70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.152+0000 7f6a857fa640 10 client.16533 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.152+0000 7f6aa178f640 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d50fe4cd0 0x556d50fe70c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb1ae6dd5a1879e server_cookie=d2e7c71391e1eef6 in_seq=0 out_seq=0 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.152+0000 7f6a857fa640 1 -- 172.21.15.86:0/909830418 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d50f3cb90 con 0x556d50fe4cd0 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.152+0000 7f6a857fa640 10 client.16533 handle_command_reply: tid=0 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 client.16533 shutdown 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 msgr2=0x556d50fd21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6a9801b040 tx=0x7f6a98018670 comp rx=0 tx=0).stop 2024-09-05T06:45:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 shutdown_connections 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d50fe4cd0 0x556d50fe70c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fd2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fd2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/909830418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fd21f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 >> 172.21.15.86:0/909830418 conn(0x556d50fc65f0 msgr2=0x556d50fcc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6a857fa640 0 client.16533 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 shutdown_connections 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.153+0000 7f6aa65cd740 1 -- 172.21.15.86:0/909830418 wait complete. 2024-09-05T06:45:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.154+0000 7f6aa65cd740 20 client.16533 trim_cache size 0 max 16384 2024-09-05T06:45:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.178+0000 7f6aa65cd740 1 Processor -- start 2024-09-05T06:45:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.178+0000 7f6aa65cd740 1 -- start start 2024-09-05T06:45:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.178+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.178+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b1230 con 0x556d50fca060 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b2d70 con 0x556d50fd5240 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b1900 con 0x556d50fca980 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34732/0 (socket says 172.21.15.86:34732) 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 -- 172.21.15.86:0/3678891707 learned_addr learned my addr 172.21.15.86:0/3678891707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa178f640 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa2791640 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 -- 172.21.15.86:0/3678891707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 msgr2=0x556d50fecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 -- 172.21.15.86:0/3678891707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 msgr2=0x556d50fcf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fcf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 -- 172.21.15.86:0/3678891707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d50fd4b60 con 0x556d50fca060 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800a8b0 tx=0x7f6a9800a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee64ea608d3ba0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.179+0000 7f6a877fe640 1 -- 172.21.15.86:0/3678891707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98039070 con 0x556d50fca060 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6a877fe640 1 -- 172.21.15.86:0/3678891707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a98027070 con 0x556d50fca060 2024-09-05T06:45:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6a877fe640 1 -- 172.21.15.86:0/3678891707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9801d070 con 0x556d50fca060 2024-09-05T06:45:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 -- 172.21.15.86:0/3678891707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 msgr2=0x556d50fec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800a8b0 tx=0x7f6a9800a400 comp rx=0 tx=0).stop 2024-09-05T06:45:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 -- 172.21.15.86:0/3678891707 shutdown_connections 2024-09-05T06:45:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fd5240 0x556d50fcf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fca980 0x556d50fecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/3678891707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca060 0x556d50fec6c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 -- 172.21.15.86:0/3678891707 >> 172.21.15.86:0/3678891707 conn(0x556d50fc65f0 msgr2=0x556d50fd21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 -- 172.21.15.86:0/3678891707 shutdown_connections 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.180+0000 7f6aa65cd740 1 -- 172.21.15.86:0/3678891707 wait complete. 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 -- start start 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 Processor -- start 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fca060 0x556d50fd91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fd5240 0x556d50fd9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b26a0 con 0x556d50fca980 2024-09-05T06:45:34.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b1fd0 con 0x556d50fca060 2024-09-05T06:45:34.185 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.181+0000 7f6aa65cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1af210 con 0x556d50fd5240 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34734/0 (socket says 172.21.15.86:34734) 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 -- 172.21.15.86:0/883464063 learned_addr learned my addr 172.21.15.86:0/883464063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fca060 0x556d50fd91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa2791640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fd5240 0x556d50fd9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa65cd740 10 client.0 _ll_register_callbacks cb 0x7f6a9800a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 -- 172.21.15.86:0/883464063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fd5240 msgr2=0x556d50fd9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fd5240 0x556d50fd9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 -- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fca060 msgr2=0x556d50fd91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fca060 0x556d50fd91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 -- 172.21.15.86:0/883464063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5104e890 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa178f640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0ed560 tx=0x7f6a8c08ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef5292306eb8095 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6a857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c049040 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6a8c0b83a0 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98027070 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c086dc0 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa65cd740 10 client.16539 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.182+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d5104fbd0 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a8c0bc430 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6a8c0bbf90 con 0x556d50fca980 2024-09-05T06:45:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 10 client.16539 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 10 client.16539 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 20 client.16539 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d51050350 0x556d51052770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 4 client.16539 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "73dfa2a0", "release": true, "await": true}] 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "73dfa2a0", "release": true, "await": true}) -- 0x7f6a98027070 con 0x556d51050350 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.183+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d51050350 0x556d51052770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.184+0000 7f6aa1f90640 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d51050350 0x556d51052770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a88aede3d0aa0f server_cookie=a21c333eca627f0a in_seq=0 out_seq=0 2024-09-05T06:45:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.184+0000 7f6a857fa640 10 client.16539 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6a857fa640 1 -- 172.21.15.86:0/883464063 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6a98027070 con 0x556d51050350 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6a857fa640 10 client.16539 handle_command_reply: tid=0 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 client.16539 shutdown 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 msgr2=0x556d50fd9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0ed560 tx=0x7f6a8c08ac60 comp rx=0 tx=0).stop 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 shutdown_connections 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556d51050350 0x556d51052770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556d50fd5240 0x556d50fd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d50fca980 0x556d50fd9710 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6a857fa640 0 client.16539 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 --2- 172.21.15.86:0/883464063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d50fca060 0x556d50fd91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 >> 172.21.15.86:0/883464063 conn(0x556d50fc65f0 msgr2=0x556d50fc4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 shutdown_connections 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 1 -- 172.21.15.86:0/883464063 wait complete. 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.190+0000 7f6aa65cd740 20 client.16539 trim_cache size 0 max 16384 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 16, 2024-09-05T06:45:34.193 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "73dfa2a0": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 16, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 141.8, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:45:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 msgr2=0x7f6a880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 0x7f6a880a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005c40 tx=0x7f6a94009500 comp rx=0 tx=0).stop 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 msgr2=0x7f6a9c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006010 tx=0x7f6a8c004270 comp rx=0 tx=0).stop 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 shutdown_connections 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6a880a2d70 0x7f6a880a5230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c102ef0 0x7f6a9c1a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c102570 0x7f6a9c1996c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 --2- 172.21.15.86:0/3540451398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a9c100e20 0x7f6a9c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 >> 172.21.15.86:0/3540451398 conn(0x7f6a9c0fc7d0 msgr2=0x7f6a9c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 shutdown_connections 2024-09-05T06:45:34.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:34.191+0000 7f6aa41f4640 1 -- 172.21.15.86:0/3540451398 wait complete. 2024-09-05T06:45:34.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 63248 0.84 MB/sec execute 54 sec latency 23849.524 ms 2024-09-05T06:45:35.067 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '73dfa2a0', total seconds elapsed: 27.3 2024-09-05T06:45:35.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 63734 0.86 MB/sec execute 55 sec latency 13.706 ms 2024-09-05T06:45:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.617+0000 7f0dfb95b640 1 -- 172.21.15.86:0/276765466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df41009b0 msgr2=0x7f0df4100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.617+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/276765466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df41009b0 0x7f0df4100d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df002a410 comp rx=0 tx=0).stop 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 -- 172.21.15.86:0/276765466 shutdown_connections 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/276765466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df410e4d0 0x7f0df41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/276765466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/276765466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df41009b0 0x7f0df4100d90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 -- 172.21.15.86:0/276765466 >> 172.21.15.86:0/276765466 conn(0x7f0df40fc7d0 msgr2=0x7f0df40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 -- 172.21.15.86:0/276765466 shutdown_connections 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 -- 172.21.15.86:0/276765466 wait complete. 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 Processor -- start 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.618+0000 7f0dfb95b640 1 -- start start 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df41009b0 0x7f0df419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41140e0 con 0x7f0df410e4d0 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4113f60 con 0x7f0df41009b0 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dfb95b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4114260 con 0x7f0df4101360 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df8ef6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34750/0 (socket says 172.21.15.86:34750) 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 -- 172.21.15.86:0/3287612790 learned_addr learned my addr 172.21.15.86:0/3287612790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df96f7640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df41009b0 0x7f0df419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 -- 172.21.15.86:0/3287612790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 msgr2=0x7f0df419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 -- 172.21.15.86:0/3287612790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df41009b0 msgr2=0x7f0df419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df41009b0 0x7f0df419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 -- 172.21.15.86:0/3287612790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df419f6b0 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00eb10 tx=0x7f0dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dea7fc640 1 -- 172.21.15.86:0/3287612790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec004b20 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.619+0000 7f0dea7fc640 1 -- 172.21.15.86:0/3287612790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dec004cc0 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.620+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df419f9a0 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.620+0000 7f0dea7fc640 1 -- 172.21.15.86:0/3287612790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec010690 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.620+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a7280 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.622+0000 7f0dfdd34740 1 Processor -- start 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.622+0000 7f0dea7fc640 1 -- 172.21.15.86:0/3287612790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0dec005020 con 0x7f0df410e4d0 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.622+0000 7f0dfdd34740 1 -- start start 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560308cfd280 con 0x560308d98480 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560308ca4b10 con 0x560308d97b60 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560308cf9240 con 0x560308da2d40 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34766/0 (socket says 172.21.15.86:34766) 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 -- 172.21.15.86:0/4249161162 learned_addr learned my addr 172.21.15.86:0/4249161162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 -- 172.21.15.86:0/4249161162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 msgr2=0x560308da5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 -- 172.21.15.86:0/4249161162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 msgr2=0x560308d97f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d97f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df96f7640 1 -- 172.21.15.86:0/4249161162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560308cb11f0 con 0x560308d98480 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dea7fc640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 0x7f0de00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 0x7f0de00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.623+0000 7f0dea7fc640 1 -- 172.21.15.86:0/3287612790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0dec0cfcc0 con 0x7f0df410e4d0 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.624+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df0032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68f741bade4ac8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.624+0000 7f0ddf7fe640 1 -- 172.21.15.86:0/4249161162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0040070 con 0x560308d98480 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.624+0000 7f0ddf7fe640 1 -- 172.21.15.86:0/4249161162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df0004030 con 0x560308d98480 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.624+0000 7f0ddf7fe640 1 -- 172.21.15.86:0/4249161162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0035040 con 0x560308d98480 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4249161162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 msgr2=0x560308da2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df0032b00 comp rx=0 tx=0).stop 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 0x7f0de00a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x560308d95470 tx=0x7f0de4005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4249161162 shutdown_connections 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d98480 0x560308da2800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4249161162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4249161162 >> 172.21.15.86:0/4249161162 conn(0x560308d940f0 msgr2=0x560308d944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4249161162 shutdown_connections 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4249161162 wait complete. 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 -- start start 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.625+0000 7f0dfdd34740 1 Processor -- start 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d98480 0x560308da0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560308da5910 con 0x560308da2d40 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560308cb7f10 con 0x560308d97b60 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560308da5a90 con 0x560308d98480 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0dfdd34740 10 client.0 _ll_register_callbacks cb 0x560308c80a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df9ef8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df9ef8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34774/0 (socket says 172.21.15.86:34774) 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df9ef8640 1 -- 172.21.15.86:0/4012470505 learned_addr learned my addr 172.21.15.86:0/4012470505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d98480 0x560308da0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 -- 172.21.15.86:0/4012470505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d98480 msgr2=0x560308da0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d98480 0x560308da0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 -- 172.21.15.86:0/4012470505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 msgr2=0x560308da0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 -- 172.21.15.86:0/4012470505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560308a6e6a0 con 0x560308d97b60 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0df96f7640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d9fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df0009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9af21252a983ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.626+0000 7f0ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0035070 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0df00316a0 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df0004030 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df0055050 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 10 client.25871 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560308db14e0 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df0004030 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f0df0004830 con 0x560308d97b60 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 10 client.25871 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 20 client.25871 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308db2830 0x560308db4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 4 client.25871 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:35.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.627+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560308d0a7f0 con 0x560308db2830 2024-09-05T06:45:35.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.628+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308db2830 0x560308db4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.628+0000 7f0ddcff9640 10 client.25871 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:35.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.628+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308db2830 0x560308db4c20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2f34e0edb729d9 server_cookie=6b5f6707db4e96b2 in_seq=0 out_seq=0 2024-09-05T06:45:35.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:35 smithi086 ceph-mon[30943]: pgmap v133: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0ddcff9640 1 -- 172.21.15.86:0/4012470505 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560308d0a7f0 con 0x560308db2830 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0ddcff9640 10 client.25871 handle_command_reply: tid=0 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 client.25871 shutdown 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 msgr2=0x560308d9fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d9fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df0009590 comp rx=0 tx=0).stop 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 shutdown_connections 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308db2830 0x560308db4c20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308da0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d98480 0x560308da0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/4012470505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d97b60 0x560308d9fbe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 >> 172.21.15.86:0/4012470505 conn(0x560308d940f0 msgr2=0x560308d99e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0ddcff9640 0 client.25871 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 shutdown_connections 2024-09-05T06:45:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 1 -- 172.21.15.86:0/4012470505 wait complete. 2024-09-05T06:45:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.629+0000 7f0dfdd34740 20 client.25871 trim_cache size 0 max 16384 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 Processor -- start 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 -- start start 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 0x560308dba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308d92a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41b1220 con 0x560308da2d40 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41b2d60 con 0x560308d98480 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41b18f0 con 0x560308d97b60 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df9ef8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308d92a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55546/0 (socket says 172.21.15.86:55546) 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 -- 172.21.15.86:0/895359252 learned_addr learned my addr 172.21.15.86:0/895359252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df96f7640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 0x560308dba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 -- 172.21.15.86:0/895359252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 msgr2=0x560308dba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 0x560308dba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 -- 172.21.15.86:0/895359252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 msgr2=0x560308d92a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308d92a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df8ef6640 1 -- 172.21.15.86:0/895359252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560308d8f830 con 0x560308d98480 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df96f7640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 0x560308dba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.650+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308d92a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.651+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0de40058c0 tx=0x7f0de40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36f59a0d91010c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.651+0000 7f0ddeffd640 1 -- 172.21.15.86:0/895359252 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40044e0 con 0x560308d98480 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.651+0000 7f0ddeffd640 1 -- 172.21.15.86:0/895359252 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0de4020e20 con 0x560308d98480 2024-09-05T06:45:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.651+0000 7f0ddeffd640 1 -- 172.21.15.86:0/895359252 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4005510 con 0x560308d98480 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- 172.21.15.86:0/895359252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 msgr2=0x560308dba790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0de40058c0 tx=0x7f0de40046c0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- 172.21.15.86:0/895359252 shutdown_connections 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308da2d40 0x560308d92a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dba790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/895359252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308d97b60 0x560308dba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- 172.21.15.86:0/895359252 >> 172.21.15.86:0/895359252 conn(0x560308d940f0 msgr2=0x560308da1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- 172.21.15.86:0/895359252 shutdown_connections 2024-09-05T06:45:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- 172.21.15.86:0/895359252 wait complete. 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 -- start start 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.652+0000 7f0dfdd34740 1 Processor -- start 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41b2690 con 0x560308d97b60 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41b1fc0 con 0x560308d98480 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41af200 con 0x560308da2d40 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 10 client.0 _ll_register_callbacks cb 0x7f0de4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34792/0 (socket says 172.21.15.86:34792) 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 -- 172.21.15.86:0/3289323745 learned_addr learned my addr 172.21.15.86:0/3289323745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df9ef8640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 -- 172.21.15.86:0/3289323745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 msgr2=0x560308da63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 -- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 msgr2=0x560308dab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 -- 172.21.15.86:0/3289323745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560308da6bc0 con 0x560308d97b60 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0df96f7640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0df0004c00 tx=0x7f0df002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31b401d57b2b800 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df005e060 con 0x560308d97b60 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0df0004f70 con 0x560308d97b60 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.653+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de4020e20 con 0x560308d97b60 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df0065050 con 0x560308d97b60 2024-09-05T06:45:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560308e1ce00 con 0x560308d97b60 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0de4020e20 con 0x560308d97b60 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f0df0003b00 con 0x560308d97b60 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 20 client.16557 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308e1d4c0 0x560308e1f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 4 client.16557 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560308d04d00 con 0x560308e1d4c0 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.654+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308e1d4c0 0x560308e1f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.655+0000 7f0ddcff9640 10 client.16557 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.655+0000 7f0df8ef6640 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308e1d4c0 0x560308e1f8e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4ffca4716b0c0c server_cookie=6bc8691eb5be4c4 in_seq=0 out_seq=0 2024-09-05T06:45:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0ddcff9640 1 -- 172.21.15.86:0/3289323745 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560308d04d00 con 0x560308e1d4c0 2024-09-05T06:45:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0ddcff9640 10 client.16557 handle_command_reply: tid=0 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 client.16557 shutdown 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 msgr2=0x560308daafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0df0004c00 tx=0x7f0df002ab10 comp rx=0 tx=0).stop 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 shutdown_connections 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560308e1d4c0 0x560308e1f8e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560308da2d40 0x560308da63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0ddcff9640 0 client.16557 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560308d98480 0x560308dab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 --2- 172.21.15.86:0/3289323745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560308d97b60 0x560308daafe0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 >> 172.21.15.86:0/3289323745 conn(0x560308d940f0 msgr2=0x560308d99580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 shutdown_connections 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 1 -- 172.21.15.86:0/3289323745 wait complete. 2024-09-05T06:45:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.656+0000 7f0dfdd34740 20 client.16557 trim_cache size 0 max 16384 2024-09-05T06:45:35.662 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:45:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3b25f3d4-90a2-43e6-9f04-0fea2ac7be3d", 2024-09-05T06:45:35.662 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:45:35.662 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 msgr2=0x7f0de00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 0x7f0de00a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x560308d95470 tx=0x7f0de4005850 comp rx=0 tx=0).stop 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 msgr2=0x7f0df419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00eb10 tx=0x7f0dec00c2d0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 shutdown_connections 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0de00a2d70 0x7f0de00a5230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0df410e4d0 0x7f0df419efd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df4101360 0x7f0df419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 --2- 172.21.15.86:0/3287612790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df41009b0 0x7f0df419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 >> 172.21.15.86:0/3287612790 conn(0x7f0df40fc7d0 msgr2=0x7f0df40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 shutdown_connections 2024-09-05T06:45:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:35.657+0000 7f0dfb95b640 1 -- 172.21.15.86:0/3287612790 wait complete. 2024-09-05T06:45:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:35 smithi096 ceph-mon[34468]: pgmap v133: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:35 smithi115 ceph-mon[34791]: pgmap v133: 129 pgs: 129 active+clean; 66 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T06:45:36.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:45:36.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 64233 0.92 MB/sec execute 56 sec latency 17.678 ms 2024-09-05T06:45:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:45:36.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:36 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:36 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:45:36.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:36.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:36 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:45:37.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 -- 172.21.15.86:0/3781812049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 msgr2=0x7fd9a0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 --2- 172.21.15.86:0/3781812049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a0100f00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd99002a410 comp rx=0 tx=0).stop 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 -- 172.21.15.86:0/3781812049 shutdown_connections 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 --2- 172.21.15.86:0/3781812049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 0x7fd9a0110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 --2- 172.21.15.86:0/3781812049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 --2- 172.21.15.86:0/3781812049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a0100f00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 -- 172.21.15.86:0/3781812049 >> 172.21.15.86:0/3781812049 conn(0x7fd9a00fc7d0 msgr2=0x7fd9a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 -- 172.21.15.86:0/3781812049 shutdown_connections 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.232+0000 7fd9a6015640 1 -- 172.21.15.86:0/3781812049 wait complete. 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 Processor -- start 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 -- start start 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 0x7fd9a019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0114130 con 0x7fd9a0100b20 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0113fb0 con 0x7fd9a010e500 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd9a6015640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01142b0 con 0x7fd9a0101440 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34812/0 (socket says 172.21.15.86:34812) 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd99f7fe640 1 -- 172.21.15.86:0/1451738967 learned_addr learned my addr 172.21.15.86:0/1451738967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd99effd640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.233+0000 7fd99ffff640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 0x7fd9a019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 -- 172.21.15.86:0/1451738967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 msgr2=0x7fd9a019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 -- 172.21.15.86:0/1451738967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 msgr2=0x7fd9a019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 0x7fd9a019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 -- 172.21.15.86:0/1451738967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a019f660 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99effd640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd990005940 tx=0x7fd9900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99cff9640 1 -- 172.21.15.86:0/1451738967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99003f030 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99cff9640 1 -- 172.21.15.86:0/1451738967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9900044c0 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a019f8f0 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd99cff9640 1 -- 172.21.15.86:0/1451738967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990035070 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.234+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a01a7230 con 0x7fd9a0100b20 2024-09-05T06:45:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.236+0000 7fd99cff9640 1 -- 172.21.15.86:0/1451738967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd990042030 con 0x7fd9a0100b20 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 Processor -- start 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 -- start start 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd99cff9640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 0x7fd9880a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537330c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x565373317140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x565373319a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653732722e0 con 0x565373317680 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565373219ce0 con 0x56537330cdc0 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56537326e2a0 con 0x56537330c4a0 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd99ffff640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 0x7fd9880a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.237+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x565373319a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x565373319a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34818/0 (socket says 172.21.15.86:34818) 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99ffff640 1 -- 172.21.15.86:0/863227892 learned_addr learned my addr 172.21.15.86:0/863227892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x565373317140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99cff9640 1 -- 172.21.15.86:0/1451738967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd9900efb00 con 0x7fd9a0100b20 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537330c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 -- 172.21.15.86:0/863227892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 msgr2=0x56537330c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537330c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 -- 172.21.15.86:0/863227892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 msgr2=0x565373319a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x565373319a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 -- 172.21.15.86:0/863227892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653731fe440 con 0x56537330cdc0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99ffff640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 0x7fd9880a52b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998009f60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.238+0000 7fd99effd640 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x565373317140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9a006a140 tx=0x7fd9940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4e5626eebe3f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd985ffb640 1 -- 172.21.15.86:0/863227892 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004020 con 0x56537330cdc0 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd985ffb640 1 -- 172.21.15.86:0/863227892 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9940041c0 con 0x56537330cdc0 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd985ffb640 1 -- 172.21.15.86:0/863227892 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400f740 con 0x56537330cdc0 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 -- 172.21.15.86:0/863227892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 msgr2=0x565373317140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x565373317140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9a006a140 tx=0x7fd9940118d0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 -- 172.21.15.86:0/863227892 shutdown_connections 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x565373319a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x565373317140 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/863227892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537330c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.239+0000 7fd9a83ee740 1 -- 172.21.15.86:0/863227892 >> 172.21.15.86:0/863227892 conn(0x565373308a30 msgr2=0x565373308e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- 172.21.15.86:0/863227892 shutdown_connections 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- 172.21.15.86:0/863227892 wait complete. 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- start start 2024-09-05T06:45:37.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 Processor -- start 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330c4a0 0x565373316ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 0x565373314180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56537331a250 con 0x565373317680 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56537322d050 con 0x56537330c4a0 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.240+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56537331a3d0 con 0x56537330cdc0 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd9a83ee740 10 client.0 _ll_register_callbacks cb 0x56537326d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34828/0 (socket says 172.21.15.86:34828) 2024-09-05T06:45:37.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99ffff640 1 -- 172.21.15.86:0/1421529912 learned_addr learned my addr 172.21.15.86:0/1421529912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 0x565373314180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330c4a0 0x565373316ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 -- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330c4a0 msgr2=0x565373316ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330c4a0 0x565373316ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 -- 172.21.15.86:0/1421529912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 msgr2=0x5653733146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 -- 172.21.15.86:0/1421529912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565372fe33c0 con 0x56537330cdc0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd99effd640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 0x565373314180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd994005890 tx=0x7fd994003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c90ebcb6a8fb777 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd97effd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994028020 con 0x56537330cdc0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd994016040 con 0x56537330cdc0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.241+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9940041c0 con 0x56537330cdc0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd994004370 con 0x56537330cdc0 2024-09-05T06:45:37.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 10 client.25900 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565373325b20 con 0x56537330cdc0 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9940041c0 con 0x56537330cdc0 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd994004b60 con 0x56537330cdc0 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 10 client.25900 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 20 client.25900 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565373326e70 0x565373329260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 4 client.25900 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537327f850 con 0x565373326e70 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.242+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565373326e70 0x565373329260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.243+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565373326e70 0x565373329260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892c6082dc39471e server_cookie=1f905535fb53a5a3 in_seq=0 out_seq=0 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.243+0000 7fd97effd640 10 client.25900 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.243+0000 7fd97effd640 1 -- 172.21.15.86:0/1421529912 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56537327f850 con 0x565373326e70 2024-09-05T06:45:37.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.243+0000 7fd97effd640 10 client.25900 handle_command_reply: tid=0 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.243+0000 7fd9a83ee740 1 client.25900 shutdown 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 msgr2=0x565373314180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 0x565373314180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd994005890 tx=0x7fd994003ac0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 shutdown_connections 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565373326e70 0x565373329260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330cdc0 0x565373314180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/1421529912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330c4a0 0x565373316ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 >> 172.21.15.86:0/1421529912 conn(0x565373308a30 msgr2=0x56537330e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd97effd640 0 client.25900 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 shutdown_connections 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 1 -- 172.21.15.86:0/1421529912 wait complete. 2024-09-05T06:45:37.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.244+0000 7fd9a83ee740 20 client.25900 trim_cache size 0 max 16384 2024-09-05T06:45:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 Processor -- start 2024-09-05T06:45:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 -- start start 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537332ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.262+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01acdf0 con 0x565373317680 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01ae930 con 0x56537330cdc0 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01ad4c0 con 0x56537330c4a0 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537332ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55578/0 (socket says 172.21.15.86:55578) 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 -- 172.21.15.86:0/4031685756 learned_addr learned my addr 172.21.15.86:0/4031685756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 -- 172.21.15.86:0/4031685756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 msgr2=0x56537332ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537332ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 -- 172.21.15.86:0/4031685756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 msgr2=0x5653733113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 -- 172.21.15.86:0/4031685756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653733082e0 con 0x56537330cdc0 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd99effd640 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd994009710 tx=0x7fd9940059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72895b3d6566961 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd9857fa640 1 -- 172.21.15.86:0/4031685756 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994028040 con 0x56537330cdc0 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd9857fa640 1 -- 172.21.15.86:0/4031685756 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd994004510 con 0x56537330cdc0 2024-09-05T06:45:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.263+0000 7fd9857fa640 1 -- 172.21.15.86:0/4031685756 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400f8f0 con 0x56537330cdc0 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 -- 172.21.15.86:0/4031685756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 msgr2=0x56537332efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd994009710 tx=0x7fd9940059b0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 -- 172.21.15.86:0/4031685756 shutdown_connections 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x5653733113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537332efb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/4031685756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537332ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 -- 172.21.15.86:0/4031685756 >> 172.21.15.86:0/4031685756 conn(0x565373308a30 msgr2=0x565373307b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 -- 172.21.15.86:0/4031685756 shutdown_connections 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.264+0000 7fd9a83ee740 1 -- 172.21.15.86:0/4031685756 wait complete. 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 -- start start 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 Processor -- start 2024-09-05T06:45:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537331fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537331d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01ae260 con 0x565373317680 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01adb90 con 0x56537330cdc0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a01aadf0 con 0x56537330c4a0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537331fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd9a83ee740 10 client.0 _ll_register_callbacks cb 0x7fd998009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd99ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34846/0 (socket says 172.21.15.86:34846) 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd99ffff640 1 -- 172.21.15.86:0/729817105 learned_addr learned my addr 172.21.15.86:0/729817105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.265+0000 7fd99effd640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537331d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 -- 172.21.15.86:0/729817105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 msgr2=0x56537331fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537331fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 -- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 msgr2=0x56537331d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537331d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 -- 172.21.15.86:0/729817105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537330cbf0 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537331fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd99ffff640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd998011610 tx=0x7fd998009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c06a9e48300483 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd97effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998026040 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fd99802c440 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd994004510 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd99803a050 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd9a83ee740 10 client.16575 fetch_fsmap learned FSMap version 32 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.266+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565373391a20 con 0x565373317680 2024-09-05T06:45:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd994004510 con 0x565373317680 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd998029030 con 0x565373317680 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 10 client.16575 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 20 client.16575 populate_metadata read hostname 'smithi086' 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5653733921f0 0x565373394610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 4 client.16575 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565373279d60 con 0x5653733921f0 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.267+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5653733921f0 0x565373394610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd99f7fe640 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5653733921f0 0x565373394610 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782fe2703eb8c98c server_cookie=6c67e4117fbe9900 in_seq=0 out_seq=0 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd97effd640 10 client.16575 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd97effd640 1 -- 172.21.15.86:0/729817105 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565373279d60 con 0x5653733921f0 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd97effd640 10 client.16575 handle_command_reply: tid=0 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 client.16575 shutdown 2024-09-05T06:45:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 msgr2=0x56537331d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd998011610 tx=0x7fd998009500 comp rx=0 tx=0).stop 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 shutdown_connections 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5653733921f0 0x565373394610 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565373317680 0x56537331d9d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56537330cdc0 0x56537331d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 --2- 172.21.15.86:0/729817105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537330c4a0 0x56537331fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 >> 172.21.15.86:0/729817105 conn(0x565373308a30 msgr2=0x5653733126d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd97effd640 0 client.16575 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 shutdown_connections 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 1 -- 172.21.15.86:0/729817105 wait complete. 2024-09-05T06:45:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.268+0000 7fd9a83ee740 20 client.16575 trim_cache size 0 max 16384 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "3b25f3d4-90a2-43e6-9f04-0fea2ac7be3d": { 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3b25f3d4-90a2-43e6-9f04-0fea2ac7be3d", 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:45:37.275 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 msgr2=0x7fd9880a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 0x7fd9880a52b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998009f60 comp rx=0 tx=0).stop 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 msgr2=0x7fd9a019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd990005940 tx=0x7fd9900040c0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 shutdown_connections 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9880a2df0 0x7fd9880a52b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9a010e500 0x7fd9a019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9a0101440 0x7fd9a019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 --2- 172.21.15.86:0/1451738967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a0100b20 0x7fd9a019a6b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:45:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 >> 172.21.15.86:0/1451738967 conn(0x7fd9a00fc7d0 msgr2=0x7fd9a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:45:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 shutdown_connections 2024-09-05T06:45:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:45:37.269+0000 7fd9a6015640 1 -- 172.21.15.86:0/1451738967 wait complete. 2024-09-05T06:45:37.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 64535 0.91 MB/sec execute 57 sec latency 29.582 ms 2024-09-05T06:45:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:37 smithi096 ceph-mon[34468]: pgmap v134: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 389 KiB/s wr, 37 op/s 2024-09-05T06:45:37.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:37 smithi086 ceph-mon[30943]: pgmap v134: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 389 KiB/s wr, 37 op/s 2024-09-05T06:45:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:37 smithi115 ceph-mon[34791]: pgmap v134: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 389 KiB/s wr, 37 op/s 2024-09-05T06:45:38.111 INFO:tasks.cephfs.filesystem:scrub status for tag:3b25f3d4-90a2-43e6-9f04-0fea2ac7be3d - {'path': '/', 'tag': '3b25f3d4-90a2-43e6-9f04-0fea2ac7be3d', 'options': 'recursive,force'} 2024-09-05T06:45:38.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 65077 0.93 MB/sec execute 58 sec latency 16.573 ms 2024-09-05T06:45:39.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 65745 0.97 MB/sec execute 59 sec latency 17.748 ms 2024-09-05T06:45:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:39 smithi096 ceph-mon[34468]: pgmap v135: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 387 KiB/s wr, 32 op/s 2024-09-05T06:45:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:39 smithi115 ceph-mon[34791]: pgmap v135: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 387 KiB/s wr, 32 op/s 2024-09-05T06:45:39.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:39 smithi086 ceph-mon[30943]: pgmap v135: 129 pgs: 129 active+clean; 68 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 387 KiB/s wr, 32 op/s 2024-09-05T06:45:40.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 66419 0.99 MB/sec execute 60 sec latency 13.778 ms 2024-09-05T06:45:41.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 67181 1.08 MB/sec execute 61 sec latency 14.702 ms 2024-09-05T06:45:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:41 smithi096 ceph-mon[34468]: pgmap v136: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-05T06:45:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:41 smithi115 ceph-mon[34791]: pgmap v136: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-05T06:45:41.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:41 smithi086 ceph-mon[30943]: pgmap v136: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-05T06:45:42.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 67677 1.14 MB/sec execute 62 sec latency 11.537 ms 2024-09-05T06:45:43.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 67987 1.12 MB/sec execute 63 sec latency 12.021 ms 2024-09-05T06:45:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:43 smithi096 ceph-mon[34468]: pgmap v137: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:43 smithi115 ceph-mon[34791]: pgmap v137: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:43.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:43 smithi086 ceph-mon[30943]: pgmap v137: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:44.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68517 1.14 MB/sec execute 64 sec latency 13.605 ms 2024-09-05T06:45:45.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 69185 1.17 MB/sec execute 65 sec latency 13.895 ms 2024-09-05T06:45:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:45 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:45:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:45 smithi086 ceph-mon[30943]: pgmap v138: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:45 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:45:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:45 smithi096 ceph-mon[34468]: pgmap v138: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:45 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:45:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:45 smithi115 ceph-mon[34791]: pgmap v138: 129 pgs: 129 active+clean; 70 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-05T06:45:46.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 69850 1.18 MB/sec execute 66 sec latency 16.737 ms 2024-09-05T06:45:46.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:46 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:45:46.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:46 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:45:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:46 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:45:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:46 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:45:46.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:46 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:45:46.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:46 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:45:47.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 70697 1.27 MB/sec execute 67 sec latency 12.861 ms 2024-09-05T06:45:47.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:47 smithi086 ceph-mon[30943]: pgmap v139: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-09-05T06:45:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:47 smithi096 ceph-mon[34468]: pgmap v139: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-09-05T06:45:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:47 smithi115 ceph-mon[34791]: pgmap v139: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-09-05T06:45:48.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71163 1.32 MB/sec execute 68 sec latency 13.208 ms 2024-09-05T06:45:49.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71427 1.30 MB/sec execute 69 sec latency 38.244 ms 2024-09-05T06:45:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:49 smithi096 ceph-mon[34468]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T06:45:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:49 smithi115 ceph-mon[34791]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T06:45:49.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:49 smithi086 ceph-mon[30943]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T06:45:50.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71901 1.30 MB/sec execute 70 sec latency 14.393 ms 2024-09-05T06:45:51.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 72524 1.34 MB/sec execute 71 sec latency 13.724 ms 2024-09-05T06:45:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:51 smithi096 ceph-mon[34468]: pgmap v141: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:51 smithi115 ceph-mon[34791]: pgmap v141: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:51.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:51 smithi086 ceph-mon[30943]: pgmap v141: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:52.512 INFO:tasks.workunit.client.0.smithi086.stdout: 1 73291 1.34 MB/sec execute 72 sec latency 14.069 ms 2024-09-05T06:45:53.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 73959 1.40 MB/sec execute 73 sec latency 13.007 ms 2024-09-05T06:45:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:53 smithi096 ceph-mon[34468]: pgmap v142: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:53 smithi115 ceph-mon[34791]: pgmap v142: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:53.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:53 smithi086 ceph-mon[30943]: pgmap v142: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:54.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 74547 1.45 MB/sec execute 74 sec latency 15.110 ms 2024-09-05T06:45:55.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 74939 1.45 MB/sec execute 75 sec latency 13.664 ms 2024-09-05T06:45:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:55 smithi096 ceph-mon[34468]: pgmap v143: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:55 smithi115 ceph-mon[34791]: pgmap v143: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:55.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:55 smithi086 ceph-mon[30943]: pgmap v143: 129 pgs: 129 active+clean; 75 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T06:45:56.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75335 1.44 MB/sec execute 76 sec latency 12.189 ms 2024-09-05T06:45:57.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75976 1.48 MB/sec execute 77 sec latency 14.358 ms 2024-09-05T06:45:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:57 smithi096 ceph-mon[34468]: pgmap v144: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:57 smithi115 ceph-mon[34791]: pgmap v144: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:57.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:57 smithi086 ceph-mon[30943]: pgmap v144: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-05T06:45:58.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 76677 1.47 MB/sec execute 78 sec latency 16.205 ms 2024-09-05T06:45:59.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 77303 1.52 MB/sec execute 79 sec latency 13.660 ms 2024-09-05T06:45:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:45:59 smithi096 ceph-mon[34468]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-05T06:45:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:45:59 smithi115 ceph-mon[34791]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-05T06:45:59.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:45:59 smithi086 ceph-mon[30943]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-05T06:46:00.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 77917 1.55 MB/sec execute 80 sec latency 13.535 ms 2024-09-05T06:46:01.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78415 1.58 MB/sec execute 81 sec latency 18.474 ms 2024-09-05T06:46:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:01 smithi096 ceph-mon[34468]: pgmap v146: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-05T06:46:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:01 smithi115 ceph-mon[34791]: pgmap v146: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-05T06:46:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:01 smithi086 ceph-mon[30943]: pgmap v146: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-05T06:46:02.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78751 1.57 MB/sec execute 82 sec latency 13.352 ms 2024-09-05T06:46:03.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 79307 1.57 MB/sec execute 83 sec latency 15.265 ms 2024-09-05T06:46:03.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:03 smithi096 ceph-mon[34468]: pgmap v147: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:03.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:03 smithi115 ceph-mon[34791]: pgmap v147: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:03 smithi086 ceph-mon[30943]: pgmap v147: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:04.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 79991 1.59 MB/sec execute 84 sec latency 26.485 ms 2024-09-05T06:46:05.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 80640 1.60 MB/sec execute 85 sec latency 13.480 ms 2024-09-05T06:46:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:05 smithi096 ceph-mon[34468]: pgmap v148: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:05 smithi115 ceph-mon[34791]: pgmap v148: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:05.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:05 smithi086 ceph-mon[30943]: pgmap v148: 129 pgs: 129 active+clean; 80 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-09-05T06:46:06.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 81391 1.66 MB/sec execute 86 sec latency 13.574 ms 2024-09-05T06:46:07.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 81866 1.69 MB/sec execute 87 sec latency 14.087 ms 2024-09-05T06:46:07.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:07 smithi086 ceph-mon[30943]: pgmap v149: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T06:46:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:07 smithi096 ceph-mon[34468]: pgmap v149: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T06:46:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:07 smithi115 ceph-mon[34791]: pgmap v149: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T06:46:08.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:46:08.402 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:08.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 82142 1.67 MB/sec execute 88 sec latency 13.221 ms 2024-09-05T06:46:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 -- 172.21.15.86:0/1325487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 msgr2=0x7f3b9c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1325487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c10ec70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009cd0 tx=0x7f3b9402a8d0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 -- 172.21.15.86:0/1325487026 shutdown_connections 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1325487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c10ec70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1325487026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c103780 0x7f3b9c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1325487026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 -- 172.21.15.86:0/1325487026 >> 172.21.15.86:0/1325487026 conn(0x7f3b9c0fc7d0 msgr2=0x7f3b9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 -- 172.21.15.86:0/1325487026 shutdown_connections 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.915+0000 7f3ba1879640 1 -- 172.21.15.86:0/1325487026 wait complete. 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 Processor -- start 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 -- start start 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c103780 0x7f3b9c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c111f80 con 0x7f3b9c108300 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c111e00 con 0x7f3b9c102dd0 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3ba1879640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c112100 con 0x7f3b9c103780 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3b9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3b9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42388/0 (socket says 172.21.15.86:42388) 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3b9b7fe640 1 -- 172.21.15.86:0/1123557436 learned_addr learned my addr 172.21.15.86:0/1123557436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.916+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c103780 0x7f3b9c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 -- 172.21.15.86:0/1123557436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c103780 msgr2=0x7f3b9c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c103780 0x7f3b9c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 -- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 msgr2=0x7f3b9c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 -- 172.21.15.86:0/1123557436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c19f670 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c19a6c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00d720 tx=0x7f3b8c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3ba0877640 1 -- 172.21.15.86:0/1123557436 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c017070 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c19f960 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c1a7240 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3ba0877640 1 -- 172.21.15.86:0/1123557436 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c00de30 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.917+0000 7f3ba0877640 1 -- 172.21.15.86:0/1123557436 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0047a0 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba0877640 1 -- 172.21.15.86:0/1123557436 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b8c004940 con 0x7f3b9c102dd0 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 Processor -- start 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 -- start start 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba0877640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 0x7f3b840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba0877640 1 -- 172.21.15.86:0/1123557436 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b8c0cf280 con 0x7f3b9c102dd0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01328dc0 0x555c01333140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01335a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c0128e2e0 con 0x555c01333680 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c01235ce0 con 0x555c01328dc0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c0128a2a0 con 0x555c013284a0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.920+0000 7f3b9b7fe640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 0x7f3b840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36976/0 (socket says 172.21.15.86:36976) 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 -- 172.21.15.86:0/1431653471 learned_addr learned my addr 172.21.15.86:0/1431653471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01328dc0 0x555c01333140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01335a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 -- 172.21.15.86:0/1431653471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01328dc0 msgr2=0x555c01333140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01328dc0 0x555c01333140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 -- 172.21.15.86:0/1431653471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 msgr2=0x555c01335a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01335a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 -- 172.21.15.86:0/1431653471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0121a440 con 0x555c013284a0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9b7fe640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 0x7f3b840a5230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006010 tx=0x7f3b94031b80 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c098e60 tx=0x7f3b8c0044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc4475f7744eb6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.921+0000 7f3b81ffb640 1 -- 172.21.15.86:0/1431653471 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c009bd0 con 0x555c013284a0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3b81ffb640 1 -- 172.21.15.86:0/1431653471 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c0cf0d0 con 0x555c013284a0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3b81ffb640 1 -- 172.21.15.86:0/1431653471 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0bb310 con 0x555c013284a0 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1431653471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 msgr2=0x555c01328880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c098e60 tx=0x7f3b8c0044e0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1431653471 shutdown_connections 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01335a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01328dc0 0x555c01333140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1431653471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c013284a0 0x555c01328880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1431653471 >> 172.21.15.86:0/1431653471 conn(0x555c01324a30 msgr2=0x555c01324e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1431653471 shutdown_connections 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.922+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1431653471 wait complete. 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 -- start start 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 Processor -- start 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01330800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c01336250 con 0x555c01333680 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c01249050 con 0x555c013284a0 2024-09-05T06:46:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c013363d0 con 0x555c01328dc0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3ba3c52740 10 client.0 _ll_register_callbacks cb 0x555c01289160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3b9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01330800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55644/0 (socket says 172.21.15.86:55644) 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3b9a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.923+0000 7f3b9affd640 1 -- 172.21.15.86:0/1488121836 learned_addr learned my addr 172.21.15.86:0/1488121836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 -- 172.21.15.86:0/1488121836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 msgr2=0x555c013302a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013302a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 -- 172.21.15.86:0/1488121836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 msgr2=0x555c01330800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01330800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 -- 172.21.15.86:0/1488121836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c00fff3c0 con 0x555c013284a0 2024-09-05T06:46:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b9affd640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0988f0 tx=0x7f3b8c0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88f5a5f34b2cf34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c09f770 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c0cf0d0 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b8c0981e0 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c0983e0 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3ba3c52740 10 client.25907 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.924+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c01341c80 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b8c0cf0d0 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3b8c0b4240 con 0x555c013284a0 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 10 client.25907 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 20 client.25907 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c01343220 0x555c01345640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 4 client.25907 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0129b850 con 0x555c01343220 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c01343220 0x555c01345640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.925+0000 7f3b7affd640 10 client.25907 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c01343220 0x555c01345640 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e03889c5c260bd server_cookie=4d833eec91248bf8 in_seq=0 out_seq=0 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3b7affd640 1 -- 172.21.15.86:0/1488121836 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c0129b850 con 0x555c01343220 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3b7affd640 10 client.25907 handle_command_reply: tid=0 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3ba3c52740 1 client.25907 shutdown 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 msgr2=0x555c01332ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.926+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0988f0 tx=0x7f3b8c0057b0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 shutdown_connections 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c01343220 0x555c01345640 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c01333680 0x555c01330800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/1488121836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c013284a0 0x555c01332ff0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 >> 172.21.15.86:0/1488121836 conn(0x555c01324a30 msgr2=0x555c0132a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3b7affd640 0 client.25907 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 shutdown_connections 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 1 -- 172.21.15.86:0/1488121836 wait complete. 2024-09-05T06:46:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.927+0000 7f3ba3c52740 20 client.25907 trim_cache size 0 max 16384 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.948+0000 7f3ba3c52740 1 Processor -- start 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 -- start start 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c0133f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c0134af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8800ff20 con 0x555c013284a0 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88011a60 con 0x555c01333680 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880105f0 con 0x555c01328dc0 2024-09-05T06:46:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c0133f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42410/0 (socket says 172.21.15.86:42410) 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9affd640 1 -- 172.21.15.86:0/366731748 learned_addr learned my addr 172.21.15.86:0/366731748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9b7fe640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c0134af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/366731748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 msgr2=0x555c0134af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c0134af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/366731748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 msgr2=0x555c01330270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/366731748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c01330b70 con 0x555c01328dc0 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9b7fe640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c0134af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.949+0000 7f3b9affd640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.950+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c0133f510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555c0132fb30 tx=0x7f3b90005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54c686fe189f579 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.950+0000 7f3b817fa640 1 -- 172.21.15.86:0/366731748 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90004030 con 0x555c01328dc0 2024-09-05T06:46:08.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.950+0000 7f3b817fa640 1 -- 172.21.15.86:0/366731748 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b900035f0 con 0x555c01328dc0 2024-09-05T06:46:08.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.950+0000 7f3b817fa640 1 -- 172.21.15.86:0/366731748 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90020720 con 0x555c01328dc0 2024-09-05T06:46:08.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- 172.21.15.86:0/366731748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 msgr2=0x555c0133f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c0133f510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555c0132fb30 tx=0x7f3b90005f90 comp rx=0 tx=0).stop 2024-09-05T06:46:08.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- 172.21.15.86:0/366731748 shutdown_connections 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c0134af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c0133f510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/366731748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c01330270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- 172.21.15.86:0/366731748 >> 172.21.15.86:0/366731748 conn(0x555c01324a30 msgr2=0x555c0132fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- 172.21.15.86:0/366731748 shutdown_connections 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- 172.21.15.86:0/366731748 wait complete. 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 -- start start 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.951+0000 7f3ba3c52740 1 Processor -- start 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c01338a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88011390 con 0x555c013284a0 2024-09-05T06:46:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88010cc0 con 0x555c01333680 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8800df00 con 0x555c01328dc0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3ba3c52740 10 client.0 _ll_register_callbacks cb 0x7f3b900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42414/0 (socket says 172.21.15.86:42414) 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9affd640 1 -- 172.21.15.86:0/156886143 learned_addr learned my addr 172.21.15.86:0/156886143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9b7fe640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c01338a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 msgr2=0x555c01338a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c01338a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/156886143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 msgr2=0x555c0133bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9a7fc640 1 -- 172.21.15.86:0/156886143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c013abf20 con 0x555c01328dc0 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.952+0000 7f3b9affd640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b9a7fc640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013382c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555c01320610 tx=0x7f3b900121d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8a862f8cb6d0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90037060 con 0x555c01328dc0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3b90004430 con 0x555c01328dc0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b900035f0 con 0x555c01328dc0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b9003e050 con 0x555c01328dc0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3ba3c52740 10 client.25924 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c013ad000 con 0x555c01328dc0 2024-09-05T06:46:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.953+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b900035f0 con 0x555c01328dc0 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3b90004700 con 0x555c01328dc0 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 10 client.25924 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 10 client.25924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 20 client.25924 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c013ad580 0x555c013af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 4 client.25924 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c01295d60 con 0x555c013ad580 2024-09-05T06:46:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3b9affd640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c013ad580 0x555c013af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3b9affd640 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c013ad580 0x555c013af9a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5934aeabbe9de3d2 server_cookie=997bfacecf33a80 in_seq=0 out_seq=0 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.954+0000 7f3b7affd640 10 client.25924 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3b7affd640 1 -- 172.21.15.86:0/156886143 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555c01295d60 con 0x555c013ad580 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3b7affd640 10 client.25924 handle_command_reply: tid=0 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 client.25924 shutdown 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 msgr2=0x555c013382c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013382c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555c01320610 tx=0x7f3b900121d0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 shutdown_connections 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555c013ad580 0x555c013af9a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c01333680 0x555c01338a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c01328dc0 0x555c013382c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 --2- 172.21.15.86:0/156886143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c013284a0 0x555c0133bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3b7affd640 0 client.25924 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 >> 172.21.15.86:0/156886143 conn(0x555c01324a30 msgr2=0x555c01323350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 shutdown_connections 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 1 -- 172.21.15.86:0/156886143 wait complete. 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.955+0000 7f3ba3c52740 20 client.25924 trim_cache size 0 max 16384 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:46:08.961 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 msgr2=0x7f3b840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 0x7f3b840a5230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006010 tx=0x7f3b94031b80 comp rx=0 tx=0).stop 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 msgr2=0x7f3b9c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c19a6c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00d720 tx=0x7f3b8c00dbe0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 shutdown_connections 2024-09-05T06:46:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b840a2d70 0x7f3b840a5230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b9c108300 0x7f3b9c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b9c103780 0x7f3b9c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 --2- 172.21.15.86:0/1123557436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c102dd0 0x7f3b9c19a6c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 >> 172.21.15.86:0/1123557436 conn(0x7f3b9c0fc7d0 msgr2=0x7f3b9c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 shutdown_connections 2024-09-05T06:46:08.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:08.956+0000 7f3ba1879640 1 -- 172.21.15.86:0/1123557436 wait complete. 2024-09-05T06:46:09.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 82587 1.66 MB/sec execute 89 sec latency 19.334 ms 2024-09-05T06:46:09.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:09 smithi086 ceph-mon[30943]: pgmap v150: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T06:46:09.794 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:46:09.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:46:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:09 smithi096 ceph-mon[34468]: pgmap v150: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T06:46:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:09 smithi115 ceph-mon[34791]: pgmap v150: 129 pgs: 129 active+clean; 85 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-09-05T06:46:10.061 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:10.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 83020 1.69 MB/sec execute 90 sec latency 361.001 ms 2024-09-05T06:46:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- 172.21.15.86:0/780064983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 msgr2=0x7f3858107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 --2- 172.21.15.86:0/780064983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858107bc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3854009930 tx=0x7f385402a420 comp rx=0 tx=0).stop 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- 172.21.15.86:0/780064983 shutdown_connections 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 --2- 172.21.15.86:0/780064983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858108300 0x7f385810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 --2- 172.21.15.86:0/780064983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858107bc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 --2- 172.21.15.86:0/780064983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858102dd0 0x7f38581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- 172.21.15.86:0/780064983 >> 172.21.15.86:0/780064983 conn(0x7f38580fc7d0 msgr2=0x7f38580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- 172.21.15.86:0/780064983 shutdown_connections 2024-09-05T06:46:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- 172.21.15.86:0/780064983 wait complete. 2024-09-05T06:46:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 Processor -- start 2024-09-05T06:46:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.541+0000 7f3860337640 1 -- start start 2024-09-05T06:46:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858102dd0 0x7f3858198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858108300 0x7f385819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3858111f00 con 0x7f3858103780 2024-09-05T06:46:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3858111d80 con 0x7f3858108300 2024-09-05T06:46:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3858112080 con 0x7f3858102dd0 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42440/0 (socket says 172.21.15.86:42440) 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 -- 172.21.15.86:0/551312516 learned_addr learned my addr 172.21.15.86:0/551312516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385e0d3640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858102dd0 0x7f3858198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385e8d4640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858108300 0x7f385819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 -- 172.21.15.86:0/551312516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858102dd0 msgr2=0x7f3858198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858102dd0 0x7f3858198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 -- 172.21.15.86:0/551312516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858108300 msgr2=0x7f385819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858108300 0x7f385819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 -- 172.21.15.86:0/551312516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385819d3b0 con 0x7f3858103780 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f385d8d2640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3854009230 tx=0x7f3854004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385403f030 con 0x7f3858103780 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385819d640 con 0x7f3858103780 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38540035f0 con 0x7f3858103780 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854035070 con 0x7f3858103780 2024-09-05T06:46:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.542+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581a4f80 con 0x7f3858103780 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.543+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38581031b0 con 0x7f3858103780 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.545+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3854003aa0 con 0x7f3858103780 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.546+0000 7f384b7fe640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 0x7f38440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.546+0000 7f385e0d3640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 0x7f38440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.546+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f38540f07a0 con 0x7f3858103780 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.546+0000 7f385e0d3640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 0x7f38440a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f384c00abe0 tx=0x7f384c00a2d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.549+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38540bcfe0 con 0x7f3858103780 2024-09-05T06:46:10.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.751+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385810a290 con 0x7f3858103780 2024-09-05T06:46:10.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.752+0000 7f384b7fe640 1 -- 172.21.15.86:0/551312516 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f38540a5e20 con 0x7f3858103780 2024-09-05T06:46:10.753 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:46:10.753 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:46:10.753 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:46:10.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 msgr2=0x7f38440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 0x7f38440a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f384c00abe0 tx=0x7f384c00a2d0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 msgr2=0x7f3858198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3854009230 tx=0x7f3854004270 comp rx=0 tx=0).stop 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 shutdown_connections 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38440a2cf0 0x7f38440a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3858108300 0x7f385819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3858103780 0x7f3858198940 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 --2- 172.21.15.86:0/551312516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3858102dd0 0x7f3858198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 >> 172.21.15.86:0/551312516 conn(0x7f38580fc7d0 msgr2=0x7f3858105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 shutdown_connections 2024-09-05T06:46:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:10.754+0000 7f3860337640 1 -- 172.21.15.86:0/551312516 wait complete. 2024-09-05T06:46:11.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/551312516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:11.123 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/551312516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/551312516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:11.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 83020 1.67 MB/sec execute 91 sec latency 1361.047 ms 2024-09-05T06:46:11.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:46:11.943 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:11.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:11 smithi086 ceph-mon[30943]: pgmap v151: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-09-05T06:46:12.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:11 smithi115 ceph-mon[34791]: pgmap v151: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-09-05T06:46:12.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:11 smithi096 ceph-mon[34468]: pgmap v151: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-09-05T06:46:12.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 83348 1.66 MB/sec execute 92 sec latency 1613.009 ms 2024-09-05T06:46:12.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 -- 172.21.15.86:0/492183472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8102790 msgr2=0x7f83a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 --2- 172.21.15.86:0/492183472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8102790 0x7f83a81067e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f839c009900 tx=0x7f839c02a410 comp rx=0 tx=0).stop 2024-09-05T06:46:12.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 -- 172.21.15.86:0/492183472 shutdown_connections 2024-09-05T06:46:12.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 --2- 172.21.15.86:0/492183472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8106f10 0x7f83a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 --2- 172.21.15.86:0/492183472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8102790 0x7f83a81067e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 --2- 172.21.15.86:0/492183472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 0x7f83a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 -- 172.21.15.86:0/492183472 >> 172.21.15.86:0/492183472 conn(0x7f83a80fd8c0 msgr2=0x7f83a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 -- 172.21.15.86:0/492183472 shutdown_connections 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.457+0000 7f83adf6f640 1 -- 172.21.15.86:0/492183472 wait complete. 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 Processor -- start 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 -- start start 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 0x7f83a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8106f10 0x7f83a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111e30 con 0x7f83a8106f10 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111cb0 con 0x7f83a8102790 2024-09-05T06:46:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83adf6f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111fb0 con 0x7f83a8101de0 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55698/0 (socket says 172.21.15.86:55698) 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 -- 172.21.15.86:0/775575764 learned_addr learned my addr 172.21.15.86:0/775575764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a7fff640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8106f10 0x7f83a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a77fe640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 0x7f83a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 -- 172.21.15.86:0/775575764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 msgr2=0x7f83a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 0x7f83a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 -- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8106f10 msgr2=0x7f83a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8106f10 0x7f83a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.458+0000 7f83a6ffd640 1 -- 172.21.15.86:0/775575764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81a6810 con 0x7f83a8102790 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839c005ab0 tx=0x7f839c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83a4ff9640 1 -- 172.21.15.86:0/775575764 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c03f070 con 0x7f83a8102790 2024-09-05T06:46:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81a6aa0 con 0x7f83a8102790 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83a4ff9640 1 -- 172.21.15.86:0/775575764 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839c031d60 con 0x7f83a8102790 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83a4ff9640 1 -- 172.21.15.86:0/775575764 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c035030 con 0x7f83a8102790 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.459+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a6f80 con 0x7f83a8102790 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.461+0000 7f83a4ff9640 1 -- 172.21.15.86:0/775575764 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f839c042070 con 0x7f83a8102790 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 Processor -- start 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 -- start start 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5573d8b1e2f0 con 0x5573d8bb9b40 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573d8ac5cf0 con 0x5573d8bb85d0 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83b0348740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5573d8b1a2b0 con 0x5573d8bc3370 2024-09-05T06:46:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42464/0 (socket says 172.21.15.86:42464) 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a7fff640 1 -- 172.21.15.86:0/3833849224 learned_addr learned my addr 172.21.15.86:0/3833849224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a4ff9640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 0x7f83900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a7fff640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 0x7f83900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 -- 172.21.15.86:0/3833849224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 msgr2=0x5573d8bc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 -- 172.21.15.86:0/3833849224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 msgr2=0x5573d8bb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a77fe640 1 -- 172.21.15.86:0/3833849224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d8aaa3a0 con 0x5573d8bb85d0 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a7fff640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:12.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.462+0000 7f83a4ff9640 1 -- 172.21.15.86:0/775575764 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f839c0effc0 con 0x7f83a8102790 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83a77fe640 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bb8a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f83980107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8887931ad7b6f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83a7fff640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 0x7f83900a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f83a00049d0 tx=0x7f83a0004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f838dffb640 1 -- 172.21.15.86:0/3833849224 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839801c480 con 0x5573d8bb85d0 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f838dffb640 1 -- 172.21.15.86:0/3833849224 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839801cad0 con 0x5573d8bb85d0 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f838dffb640 1 -- 172.21.15.86:0/3833849224 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398011bd0 con 0x5573d8bb85d0 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 -- 172.21.15.86:0/3833849224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 msgr2=0x5573d8bb8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bb8a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f83980107a0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 -- 172.21.15.86:0/3833849224 shutdown_connections 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bb8a10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 --2- 172.21.15.86:0/3833849224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb9b40 0x5573d8bb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.463+0000 7f83b0348740 1 -- 172.21.15.86:0/3833849224 >> 172.21.15.86:0/3833849224 conn(0x5573d8b81310 msgr2=0x5573d8bb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- 172.21.15.86:0/3833849224 shutdown_connections 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- 172.21.15.86:0/3833849224 wait complete. 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- start start 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 Processor -- start 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 0x5573d8bc07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5573d8bc6070 con 0x5573d8bb85d0 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573d8ad9060 con 0x5573d8bc3370 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5573d8bc61f0 con 0x5573d8bb9b40 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83b0348740 10 client.0 _ll_register_callbacks cb 0x5573d882c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.464+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42468/0 (socket says 172.21.15.86:42468) 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 0x5573d8bc07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 -- 172.21.15.86:0/1747231957 learned_addr learned my addr 172.21.15.86:0/1747231957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 -- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 msgr2=0x5573d8bc0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 -- 172.21.15.86:0/1747231957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 msgr2=0x5573d8bc07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 0x5573d8bc07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 -- 172.21.15.86:0/1747231957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d888f3c0 con 0x5573d8bb85d0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a7fff640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 0x5573d8bc07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83a77fe640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f839800bf50 tx=0x7f839800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a009218e6ed20e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f8386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839802b060 con 0x5573d8bb85d0 2024-09-05T06:46:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f839801c690 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f839801cad0 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.465+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8398032050 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 10 client.16611 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573d8bd1ac0 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839801cad0 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8398026880 con 0x5573d8bb85d0 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 10 client.16611 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 10 client.16611 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 20 client.16611 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8bd32a0 0x5573d8bd56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 4 client.16611 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573d8b2b860 con 0x5573d8bd32a0 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.466+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8bd32a0 0x5573d8bd56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f8386ffd640 10 client.16611 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8bd32a0 0x5573d8bd56f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53b60d065b4b73d8 server_cookie=5e0b2c10e6c8d6fa in_seq=0 out_seq=0 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f8386ffd640 1 -- 172.21.15.86:0/1747231957 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573d8b2b860 con 0x5573d8bd32a0 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f8386ffd640 10 client.16611 handle_command_reply: tid=0 2024-09-05T06:46:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83b0348740 1 client.16611 shutdown 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 msgr2=0x5573d8bc2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f839800bf50 tx=0x7f839800bf80 comp rx=0 tx=0).stop 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 shutdown_connections 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.467+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8bd32a0 0x5573d8bd56f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bc3370 0x5573d8bc07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 --2- 172.21.15.86:0/1747231957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bc2da0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 >> 172.21.15.86:0/1747231957 conn(0x5573d8b81310 msgr2=0x5573d8bba9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f8386ffd640 0 client.16611 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:46:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 shutdown_connections 2024-09-05T06:46:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 1 -- 172.21.15.86:0/1747231957 wait complete. 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.468+0000 7f83b0348740 20 client.16611 trim_cache size 0 max 16384 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 Processor -- start 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 -- start start 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 0x5573d8bb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81b1290 con 0x5573d8bb85d0 2024-09-05T06:46:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81b2dd0 con 0x5573d8bb9b40 2024-09-05T06:46:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83b0348740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81b1960 con 0x5573d8bc3370 2024-09-05T06:46:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 0x5573d8bb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46108/0 (socket says 172.21.15.86:46108) 2024-09-05T06:46:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 -- 172.21.15.86:0/659777417 learned_addr learned my addr 172.21.15.86:0/659777417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a7fff640 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 -- 172.21.15.86:0/659777417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 msgr2=0x5573d8bb4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bb4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 -- 172.21.15.86:0/659777417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 msgr2=0x5573d8bb4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 0x5573d8bb4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a77fe640 1 -- 172.21.15.86:0/659777417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d8bbb120 con 0x5573d8bb85d0 2024-09-05T06:46:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.484+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 0x5573d8bb4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83a77fe640 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f839800b890 tx=0x7f839801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc62ef2e9500ff04 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f838d7fa640 1 -- 172.21.15.86:0/659777417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839802b040 con 0x5573d8bb85d0 2024-09-05T06:46:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f838d7fa640 1 -- 172.21.15.86:0/659777417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839801ca90 con 0x5573d8bb85d0 2024-09-05T06:46:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f838d7fa640 1 -- 172.21.15.86:0/659777417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980266d0 con 0x5573d8bb85d0 2024-09-05T06:46:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 -- 172.21.15.86:0/659777417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 msgr2=0x5573d8bbf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f839800b890 tx=0x7f839801c8a0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 -- 172.21.15.86:0/659777417 shutdown_connections 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bc3370 0x5573d8bb4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb9b40 0x5573d8bb4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 --2- 172.21.15.86:0/659777417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bb85d0 0x5573d8bbf910 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.485+0000 7f83b0348740 1 -- 172.21.15.86:0/659777417 >> 172.21.15.86:0/659777417 conn(0x5573d8b81310 msgr2=0x5573d8bbe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- 172.21.15.86:0/659777417 shutdown_connections 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- 172.21.15.86:0/659777417 wait complete. 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- start start 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 Processor -- start 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81b2700 con 0x5573d8bc3370 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81b2030 con 0x5573d8bb85d0 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81af270 con 0x5573d8bb9b40 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83b0348740 10 client.0 _ll_register_callbacks cb 0x7f83a0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.486+0000 7f83a6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46110/0 (socket says 172.21.15.86:46110) 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43566/0 (socket says 172.21.15.86:43566) 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 -- 172.21.15.86:0/414976681 learned_addr learned my addr 172.21.15.86:0/414976681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 -- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 msgr2=0x5573d8bc6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 -- 172.21.15.86:0/414976681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 msgr2=0x5573d8bcbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 -- 172.21.15.86:0/414976681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d8bc71f0 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a77fe640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a6ffd640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83a7fff640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fc0 tx=0x7f83a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da24470e9edca796 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f8386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0031060 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f83a00104f0 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f839801ca90 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83a0038050 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83b0348740 10 client.16617 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573d8c3d030 con 0x5573d8bc3370 2024-09-05T06:46:12.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.487+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839801ca90 con 0x5573d8bc3370 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83a001f040 con 0x5573d8bc3370 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 10 client.16617 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 10 client.16617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 20 client.16617 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8c3d750 0x5573d8c3fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 4 client.16617 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573d8b25d70 con 0x5573d8c3d750 2024-09-05T06:46:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83a77fe640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8c3d750 0x5573d8c3fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f8386ffd640 10 client.16617 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83a77fe640 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8c3d750 0x5573d8c3fb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=856e08c9604fb982 server_cookie=a1b9741c2608972c in_seq=0 out_seq=0 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f8386ffd640 1 -- 172.21.15.86:0/414976681 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573d8b25d70 con 0x5573d8c3d750 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f8386ffd640 10 client.16617 handle_command_reply: tid=0 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.488+0000 7f83b0348740 1 client.16617 shutdown 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 msgr2=0x5573d8bc69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fc0 tx=0x7f83a0004a40 comp rx=0 tx=0).stop 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 shutdown_connections 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5573d8c3d750 0x5573d8c3fb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573d8bc3370 0x5573d8bc69c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f8386ffd640 0 client.16617 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5573d8bb9b40 0x5573d8bc6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 --2- 172.21.15.86:0/414976681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573d8bb85d0 0x5573d8bcbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 >> 172.21.15.86:0/414976681 conn(0x5573d8b81310 msgr2=0x5573d8bbe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 shutdown_connections 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 1 -- 172.21.15.86:0/414976681 wait complete. 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83b0348740 20 client.16617 trim_cache size 0 max 16384 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 msgr2=0x7f83900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 0x7f83900a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f83a00049d0 tx=0x7f83a0004ab0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 msgr2=0x7f83a8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839c005ab0 tx=0x7f839c003c80 comp rx=0 tx=0).stop 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 shutdown_connections 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83900a2cf0 0x7f83900a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a8106f10 0x7f83a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83a8102790 0x7f83a8074a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 --2- 172.21.15.86:0/775575764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83a8101de0 0x7f83a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 >> 172.21.15.86:0/775575764 conn(0x7f83a80fd8c0 msgr2=0x7f83a80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 shutdown_connections 2024-09-05T06:46:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:12.489+0000 7f83adf6f640 1 -- 172.21.15.86:0/775575764 wait complete. 2024-09-05T06:46:13.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:46:13.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 84001 1.66 MB/sec execute 93 sec latency 64.154 ms 2024-09-05T06:46:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:13 smithi086 ceph-mon[30943]: pgmap v152: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:13.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:13 smithi096 ceph-mon[34468]: pgmap v152: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:13 smithi115 ceph-mon[34791]: pgmap v152: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 -- 172.21.15.86:0/2598373961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 msgr2=0x7fa05810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 --2- 172.21.15.86:0/2598373961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa05810da40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa050009ca0 tx=0x7fa05002a950 comp rx=0 tx=0).stop 2024-09-05T06:46:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 -- 172.21.15.86:0/2598373961 shutdown_connections 2024-09-05T06:46:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 --2- 172.21.15.86:0/2598373961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa05810da40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 --2- 172.21.15.86:0/2598373961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 0x7fa0581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 --2- 172.21.15.86:0/2598373961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa058101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 -- 172.21.15.86:0/2598373961 >> 172.21.15.86:0/2598373961 conn(0x7fa0580fc7d0 msgr2=0x7fa0580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 -- 172.21.15.86:0/2598373961 shutdown_connections 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.171+0000 7fa06031f640 1 -- 172.21.15.86:0/2598373961 wait complete. 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 Processor -- start 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 -- start start 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 0x7fa0580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa0580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110e30 con 0x7fa058102ef0 2024-09-05T06:46:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110cb0 con 0x7fa058100e20 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa06031f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110fb0 con 0x7fa058102570 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa05e8bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa0580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43588/0 (socket says 172.21.15.86:43588) 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.172+0000 7fa05e0bb640 1 -- 172.21.15.86:0/968424435 learned_addr learned my addr 172.21.15.86:0/968424435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 0x7fa0580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 -- 172.21.15.86:0/968424435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 msgr2=0x7fa0580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 0x7fa0580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 -- 172.21.15.86:0/968424435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 msgr2=0x7fa0580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 -- 172.21.15.86:0/968424435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0581a6850 con 0x7fa058102ef0 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa05e8bc640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa0580fd700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa050005cc0 tx=0x7fa050031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa04f7fe640 1 -- 172.21.15.86:0/968424435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050034040 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0581a6ae0 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa04f7fe640 1 -- 172.21.15.86:0/968424435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa05002f6b0 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa04f7fe640 1 -- 172.21.15.86:0/968424435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004430 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.173+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a7010 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.176+0000 7fa04f7fe640 1 -- 172.21.15.86:0/968424435 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa050037020 con 0x7fa058102ef0 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.176+0000 7fa0626f8740 1 Processor -- start 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 -- start start 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bfa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad06b532f0 con 0x55ad06bed5d0 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ad06afacf0 con 0x55ad06bf8370 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa0626f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ad06b4f2b0 con 0x55ad06beeb40 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46116/0 (socket says 172.21.15.86:46116) 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 -- 172.21.15.86:0/600202571 learned_addr learned my addr 172.21.15.86:0/600202571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bfa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa04f7fe640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 -- 172.21.15.86:0/600202571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 msgr2=0x55ad06bed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 -- 172.21.15.86:0/600202571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 msgr2=0x55ad06bfa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bfa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.177+0000 7fa05e0bb640 1 -- 172.21.15.86:0/600202571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad06adf3a0 con 0x55ad06bed5d0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa05e8bc640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa054011650 tx=0x7fa054009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6977cc16ca12366e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa04f7fe640 1 -- 172.21.15.86:0/968424435 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa0500f03f0 con 0x7fa058102ef0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa04cdea640 1 -- 172.21.15.86:0/600202571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05401e070 con 0x55ad06bed5d0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa04cdea640 1 -- 172.21.15.86:0/600202571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa05400fb70 con 0x55ad06bed5d0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.178+0000 7fa04cdea640 1 -- 172.21.15.86:0/600202571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054019c50 con 0x55ad06bed5d0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- 172.21.15.86:0/600202571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 msgr2=0x55ad06beda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa05e8bc640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 0x7fa0440a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0500ed920 tx=0x7fa0500a7040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa054011650 tx=0x7fa054009f90 comp rx=0 tx=0).stop 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- 172.21.15.86:0/600202571 shutdown_connections 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bfa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bed5d0 0x55ad06beda10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 --2- 172.21.15.86:0/600202571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- 172.21.15.86:0/600202571 >> 172.21.15.86:0/600202571 conn(0x55ad06bb6310 msgr2=0x55ad06be9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- 172.21.15.86:0/600202571 shutdown_connections 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- 172.21.15.86:0/600202571 wait complete. 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 -- start start 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.179+0000 7fa0626f8740 1 Processor -- start 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 0x55ad06bf7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bf5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad06bfb070 con 0x55ad06beeb40 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ad06b0e060 con 0x55ad06bf8370 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ad06bfb1f0 con 0x55ad06bed5d0 2024-09-05T06:46:14.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 10 client.0 _ll_register_callbacks cb 0x55ad068618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 0x55ad06bf7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46120/0 (socket says 172.21.15.86:46120) 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 -- 172.21.15.86:0/472173926 learned_addr learned my addr 172.21.15.86:0/472173926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bf5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 -- 172.21.15.86:0/472173926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 msgr2=0x55ad06bf7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 0x55ad06bf7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 -- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 msgr2=0x55ad06bf5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bf5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 -- 172.21.15.86:0/472173926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad068c43c0 con 0x55ad06beeb40 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 0x55ad06bf7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa054009ef0 tx=0x7fa054009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8512980b7b3029de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054026020 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa05400f6c0 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.180+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05400fb70 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa05402e050 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 10 client.16629 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad06c06b70 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa05400fb70 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa054021070 con 0x55ad06beeb40 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 10 client.16629 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 10 client.16629 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 20 client.16629 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c07f70 0x55ad06c0a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 4 client.16629 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad06b60860 con 0x55ad06c07f70 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.181+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c07f70 0x55ad06c0a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.182+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c07f70 0x55ad06c0a360 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a23449d9a5012bb server_cookie=5c201dc310850a1c in_seq=0 out_seq=0 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.182+0000 7fa041ffb640 10 client.16629 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa041ffb640 1 -- 172.21.15.86:0/472173926 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad06b60860 con 0x55ad06c07f70 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa041ffb640 10 client.16629 handle_command_reply: tid=0 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 client.16629 shutdown 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 msgr2=0x55ad06bf5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa054009ef0 tx=0x7fa054009f20 comp rx=0 tx=0).stop 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 shutdown_connections 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c07f70 0x55ad06c0a360 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa041ffb640 0 client.16629 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bf8370 0x55ad06bf5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06bf5030 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 --2- 172.21.15.86:0/472173926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bed5d0 0x55ad06bf7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 >> 172.21.15.86:0/472173926 conn(0x55ad06bb6310 msgr2=0x55ad06bef930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 shutdown_connections 2024-09-05T06:46:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 1 -- 172.21.15.86:0/472173926 wait complete. 2024-09-05T06:46:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.183+0000 7fa0626f8740 20 client.16629 trim_cache size 0 max 16384 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 Processor -- start 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 -- start start 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bf0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bf8370 0x55ad06bf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b11c0 con 0x55ad06bf8370 2024-09-05T06:46:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b2d00 con 0x55ad06bed5d0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa0626f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b1890 con 0x55ad06beeb40 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.197+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bf8370 0x55ad06bf0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43606/0 (socket says 172.21.15.86:43606) 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 -- 172.21.15.86:0/3718837069 learned_addr learned my addr 172.21.15.86:0/3718837069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bf0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 -- 172.21.15.86:0/3718837069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 msgr2=0x55ad06bf0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bf0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 -- 172.21.15.86:0/3718837069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bf8370 msgr2=0x55ad06bf0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bf8370 0x55ad06bf0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 -- 172.21.15.86:0/3718837069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad06bea960 con 0x55ad06bed5d0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ad06c0aae0 tx=0x7fa048005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ef4836e11fbb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa043fff640 1 -- 172.21.15.86:0/3718837069 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800ec70 con 0x55ad06bed5d0 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa043fff640 1 -- 172.21.15.86:0/3718837069 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04800ee10 con 0x55ad06bed5d0 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.198+0000 7fa043fff640 1 -- 172.21.15.86:0/3718837069 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048022d50 con 0x55ad06bed5d0 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- 172.21.15.86:0/3718837069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 msgr2=0x55ad06c04340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ad06c0aae0 tx=0x7fa048005f90 comp rx=0 tx=0).stop 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- 172.21.15.86:0/3718837069 shutdown_connections 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06bf8370 0x55ad06bf0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06beeb40 0x55ad06bf0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3718837069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c04340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- 172.21.15.86:0/3718837069 >> 172.21.15.86:0/3718837069 conn(0x55ad06bb6310 msgr2=0x55ad06bf70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- 172.21.15.86:0/3718837069 shutdown_connections 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- 172.21.15.86:0/3718837069 wait complete. 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.199+0000 7fa0626f8740 1 -- start start 2024-09-05T06:46:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 Processor -- start 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06c00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bf8370 0x55ad06bf1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b2630 con 0x55ad06beeb40 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b1f60 con 0x55ad06bed5d0 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581af1a0 con 0x55ad06bf8370 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa0626f8740 10 client.0 _ll_register_callbacks cb 0x7fa05401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06c00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05d8ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43616/0 (socket says 172.21.15.86:43616) 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05d8ba640 1 -- 172.21.15.86:0/3056601007 learned_addr learned my addr 172.21.15.86:0/3056601007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bf8370 0x55ad06bf1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 -- 172.21.15.86:0/3056601007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bf8370 msgr2=0x55ad06bf1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bf8370 0x55ad06bf1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 -- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 msgr2=0x55ad06c00690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05e0bb640 1 -- 172.21.15.86:0/3056601007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad06bf5fd0 con 0x55ad06beeb40 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.200+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa05e0bb640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06c00bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa0540056f0 tx=0x7fa05400f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a5ccece7ce4c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054026040 con 0x55ad06beeb40 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04800ee10 con 0x55ad06beeb40 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa054004040 con 0x55ad06beeb40 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa05403f050 con 0x55ad06beeb40 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 10 client.16635 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad06c72390 con 0x55ad06beeb40 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04800ee10 con 0x55ad06beeb40 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa05403a030 con 0x55ad06beeb40 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 10 client.16635 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 10 client.16635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 20 client.16635 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c729e0 0x55ad06c74dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 4 client.16635 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.201+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad06b5ad70 con 0x55ad06c729e0 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c729e0 0x55ad06c74dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa05d8ba640 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c729e0 0x55ad06c74dd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7c5e57de3a7a0b6 server_cookie=904b70f850a23e2 in_seq=0 out_seq=0 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa041ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa041ffb640 1 -- 172.21.15.86:0/3056601007 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad06b5ad70 con 0x55ad06c729e0 2024-09-05T06:46:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa041ffb640 10 client.16635 handle_command_reply: tid=0 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 client.16635 shutdown 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 msgr2=0x55ad06c00bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06c00bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa0540056f0 tx=0x7fa05400f4c0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 shutdown_connections 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ad06c729e0 0x55ad06c74dd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ad06bf8370 0x55ad06bf1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad06beeb40 0x55ad06c00bd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa041ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 --2- 172.21.15.86:0/3056601007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ad06bed5d0 0x55ad06c00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 >> 172.21.15.86:0/3056601007 conn(0x55ad06bb6310 msgr2=0x55ad06bf37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 shutdown_connections 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.202+0000 7fa0626f8740 1 -- 172.21.15.86:0/3056601007 wait complete. 2024-09-05T06:46:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa0626f8740 20 client.16635 trim_cache size 0 max 16384 2024-09-05T06:46:14.213 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 msgr2=0x7fa0440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 0x7fa0440a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0500ed920 tx=0x7fa0500a7040 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 msgr2=0x7fa0580fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa0580fd700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa050005cc0 tx=0x7fa050031f40 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 shutdown_connections 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa058102ef0 0x7fa0580fd700 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa058102570 0x7fa0580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 --2- 172.21.15.86:0/968424435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa058100e20 0x7fa0580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 >> 172.21.15.86:0/968424435 conn(0x7fa0580fc7d0 msgr2=0x7fa05806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 shutdown_connections 2024-09-05T06:46:14.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:14.203+0000 7fa06031f640 1 -- 172.21.15.86:0/968424435 wait complete. 2024-09-05T06:46:14.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 84637 1.70 MB/sec execute 94 sec latency 48.510 ms 2024-09-05T06:46:15.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:46:15.356 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:15.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85212 1.74 MB/sec execute 95 sec latency 13.588 ms 2024-09-05T06:46:15.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:15 smithi086 ceph-mon[30943]: pgmap v153: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:15 smithi096 ceph-mon[34468]: pgmap v153: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:15 smithi115 ceph-mon[34791]: pgmap v153: 129 pgs: 129 active+clean; 84 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:46:16.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85553 1.73 MB/sec execute 96 sec latency 77.678 ms 2024-09-05T06:46:17.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85762 1.72 MB/sec execute 97 sec latency 102.100 ms 2024-09-05T06:46:17.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:17 smithi086 ceph-mon[30943]: pgmap v154: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-05T06:46:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:17 smithi096 ceph-mon[34468]: pgmap v154: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 -- 172.21.15.86:0/1970852744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102570 msgr2=0x7efc1c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 --2- 172.21.15.86:0/1970852744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102570 0x7efc1c1029b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efc08009900 tx=0x7efc0802a480 comp rx=0 tx=0).stop 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 -- 172.21.15.86:0/1970852744 shutdown_connections 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 --2- 172.21.15.86:0/1970852744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102ef0 0x7efc1c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 --2- 172.21.15.86:0/1970852744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102570 0x7efc1c1029b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 --2- 172.21.15.86:0/1970852744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.833+0000 7efc23149640 1 -- 172.21.15.86:0/1970852744 >> 172.21.15.86:0/1970852744 conn(0x7efc1c0fc7d0 msgr2=0x7efc1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.834+0000 7efc23149640 1 -- 172.21.15.86:0/1970852744 shutdown_connections 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.834+0000 7efc23149640 1 -- 172.21.15.86:0/1970852744 wait complete. 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.834+0000 7efc23149640 1 Processor -- start 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.834+0000 7efc23149640 1 -- start start 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 0x7efc1c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 0x7efc1c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c111ef0 con 0x7efc1c102ef0 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c111d70 con 0x7efc1c102570 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc23149640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c112070 con 0x7efc1c100e20 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 0x7efc1c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49064/0 (socket says 172.21.15.86:49064) 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc20ee5640 1 -- 172.21.15.86:0/2781003149 learned_addr learned my addr 172.21.15.86:0/2781003149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc1bfff640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 0x7efc1c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 -- 172.21.15.86:0/2781003149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 msgr2=0x7efc1c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 -- 172.21.15.86:0/2781003149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 msgr2=0x7efc1c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 0x7efc1c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.835+0000 7efc216e6640 1 -- 172.21.15.86:0/2781003149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c19d3c0 con 0x7efc1c102ef0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc1bfff640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 0x7efc1c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc216e6640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 0x7efc1c19cce0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efc1400ec50 tx=0x7efc1400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc19ffb640 1 -- 172.21.15.86:0/2781003149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14003b70 con 0x7efc1c102ef0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc19ffb640 1 -- 172.21.15.86:0/2781003149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc14003d10 con 0x7efc1c102ef0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c19d6b0 con 0x7efc1c102ef0 2024-09-05T06:46:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc19ffb640 1 -- 172.21.15.86:0/2781003149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14010830 con 0x7efc1c102ef0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.836+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c1a4f90 con 0x7efc1c102ef0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.839+0000 7efc19ffb640 1 -- 172.21.15.86:0/2781003149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc14003ec0 con 0x7efc1c102ef0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.839+0000 7efc25522740 1 Processor -- start 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.839+0000 7efc25522740 1 -- start start 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f0034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f010b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56321ef652f0 con 0x56321f00e780 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56321ef0ccf0 con 0x56321f004f50 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc25522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56321ef612b0 con 0x56321f0039e0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f010b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49072/0 (socket says 172.21.15.86:49072) 2024-09-05T06:46:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 -- 172.21.15.86:0/227370673 learned_addr learned my addr 172.21.15.86:0/227370673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc1bfff640 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f0034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 -- 172.21.15.86:0/227370673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 msgr2=0x56321f0034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f0034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 -- 172.21.15.86:0/227370673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 msgr2=0x56321f010b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f010b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc20ee5640 1 -- 172.21.15.86:0/227370673 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321eef13a0 con 0x56321f0039e0 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc1bfff640 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f010b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.840+0000 7efc19ffb640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 0x7efc0c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc20ee5640 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc10009dc0 tx=0x7efc10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e4381224a9b700 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc19ffb640 1 -- 172.21.15.86:0/2781003149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efc140cfe00 con 0x7efc1c102ef0 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc06ffd640 1 -- 172.21.15.86:0/227370673 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10005220 con 0x56321f0039e0 2024-09-05T06:46:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc1bfff640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 0x7efc0c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc06ffd640 1 -- 172.21.15.86:0/227370673 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc10004c50 con 0x56321f0039e0 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.841+0000 7efc06ffd640 1 -- 172.21.15.86:0/227370673 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10019810 con 0x56321f0039e0 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 -- 172.21.15.86:0/227370673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 msgr2=0x56321f003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc10009dc0 tx=0x7efc10011810 comp rx=0 tx=0).stop 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc1bfff640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 0x7efc0c0a5300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ffe90 tx=0x7efc0802a480 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 -- 172.21.15.86:0/227370673 shutdown_connections 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f010b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f003e20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 --2- 172.21.15.86:0/227370673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f0034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 -- 172.21.15.86:0/227370673 >> 172.21.15.86:0/227370673 conn(0x56321efc8310 msgr2=0x56321f000070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 -- 172.21.15.86:0/227370673 shutdown_connections 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.842+0000 7efc25522740 1 -- 172.21.15.86:0/227370673 wait complete. 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 -- start start 2024-09-05T06:46:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 Processor -- start 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f00e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f00b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc25522740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56321f0114d0 con 0x56321f0039e0 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc216e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc216e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49078/0 (socket says 172.21.15.86:49078) 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.843+0000 7efc216e6640 1 -- 172.21.15.86:0/1877461680 learned_addr learned my addr 172.21.15.86:0/1877461680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.844+0000 7efc1bfff640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f00b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.844+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56321ef20060 con 0x56321f004f50 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.844+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56321f011650 con 0x56321f00e780 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.844+0000 7efc20ee5640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f00e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.844+0000 7efc25522740 10 client.0 _ll_register_callbacks cb 0x56321ec738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:17.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 -- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 msgr2=0x56321f00b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f00b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 -- 172.21.15.86:0/1877461680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 msgr2=0x56321f00e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f00e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321ecd63c0 con 0x56321f00e780 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc20ee5640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f00e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc216e6640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc140a4620 tx=0x7efc1409bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a07ccc004434a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efbfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:46:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc140b4070 con 0x56321f00e780 2024-09-05T06:46:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efc14004d40 con 0x56321f00e780 2024-09-05T06:46:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.845+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc10004c50 con 0x56321f00e780 2024-09-05T06:46:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efc1409bee0 con 0x56321f00e780 2024-09-05T06:46:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 10 client.25972 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56321f01cf10 con 0x56321f00e780 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc140a0290 con 0x56321f00e780 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efc14097cc0 con 0x56321f00e780 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 10 client.25972 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 10 client.25972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 20 client.25972 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f01e320 0x56321f020730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 4 client.25972 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efc10004c50 con 0x56321f01e320 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.846+0000 7efc20ee5640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f01e320 0x56321f020730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.847+0000 7efbfbfff640 10 client.25972 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.847+0000 7efc20ee5640 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f01e320 0x56321f020730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aa5034fc466591d server_cookie=7f15a2e2ce9d0938 in_seq=0 out_seq=0 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efbfbfff640 1 -- 172.21.15.86:0/1877461680 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efc10004c50 con 0x56321f01e320 2024-09-05T06:46:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efbfbfff640 10 client.25972 handle_command_reply: tid=0 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 client.25972 shutdown 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 msgr2=0x56321f00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc140a4620 tx=0x7efc1409bb10 comp rx=0 tx=0).stop 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 shutdown_connections 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f01e320 0x56321f020730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f00b8e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f00b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 --2- 172.21.15.86:0/1877461680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f00e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 >> 172.21.15.86:0/1877461680 conn(0x56321efc8310 msgr2=0x56321f005d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efbfbfff640 0 client.25972 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 shutdown_connections 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.848+0000 7efc25522740 1 -- 172.21.15.86:0/1877461680 wait complete. 2024-09-05T06:46:17.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.849+0000 7efc25522740 20 client.25972 trim_cache size 0 max 16384 2024-09-05T06:46:17.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 Processor -- start 2024-09-05T06:46:17.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 -- start start 2024-09-05T06:46:17.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f00fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1acdd0 con 0x56321f00e780 2024-09-05T06:46:17.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1ae910 con 0x56321f004f50 2024-09-05T06:46:17.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.864+0000 7efc25522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1ad4a0 con 0x56321f0039e0 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49084/0 (socket says 172.21.15.86:49084) 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43668/0 (socket says 172.21.15.86:43668) 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc20ee5640 1 -- 172.21.15.86:0/721535430 learned_addr learned my addr 172.21.15.86:0/721535430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc216e6640 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f00fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 -- 172.21.15.86:0/721535430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 msgr2=0x56321f01a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 -- 172.21.15.86:0/721535430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 msgr2=0x56321f00fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f00fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 -- 172.21.15.86:0/721535430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321f006530 con 0x56321f004f50 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc20ee5640 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc1bfff640 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc08009230 tx=0x7efc08040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ad5e29f34d1a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.865+0000 7efc067fc640 1 -- 172.21.15.86:0/721535430 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08035040 con 0x56321f004f50 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc067fc640 1 -- 172.21.15.86:0/721535430 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc08050070 con 0x56321f004f50 2024-09-05T06:46:17.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc067fc640 1 -- 172.21.15.86:0/721535430 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08004da0 con 0x56321f004f50 2024-09-05T06:46:17.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 -- 172.21.15.86:0/721535430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 msgr2=0x56321f020330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc08009230 tx=0x7efc08040000 comp rx=0 tx=0).stop 2024-09-05T06:46:17.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 -- 172.21.15.86:0/721535430 shutdown_connections 2024-09-05T06:46:17.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f00e780 0x56321f00fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f020330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 --2- 172.21.15.86:0/721535430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f0039e0 0x56321f01a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 -- 172.21.15.86:0/721535430 >> 172.21.15.86:0/721535430 conn(0x56321efc8310 msgr2=0x56321f009c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 -- 172.21.15.86:0/721535430 shutdown_connections 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.866+0000 7efc25522740 1 -- 172.21.15.86:0/721535430 wait complete. 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 -- start start 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 Processor -- start 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f0168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f0118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1ae240 con 0x56321f0039e0 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1adb70 con 0x56321f004f50 2024-09-05T06:46:17.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c1aadb0 con 0x56321f00e780 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.867+0000 7efc25522740 10 client.0 _ll_register_callbacks cb 0x7efc1400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc20ee5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f0168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43680/0 (socket says 172.21.15.86:43680) 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc1bfff640 1 -- 172.21.15.86:0/155188811 learned_addr learned my addr 172.21.15.86:0/155188811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f0118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 -- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 msgr2=0x56321f016e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 -- 172.21.15.86:0/155188811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 msgr2=0x56321f0168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f0168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 -- 172.21.15.86:0/155188811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321f011de0 con 0x56321f00e780 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc20ee5640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f0168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc1bfff640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:17.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc216e6640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f0118a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc140988b0 tx=0x7efc1409ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c3b8a3e611a236 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efbfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc140b4070 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efc14004d40 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc08050070 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efc14097e40 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc25522740 10 client.25978 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.868+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56321f0896d0 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc08050070 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efc140bab20 con 0x56321f00e780 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 10 client.25978 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 10 client.25978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 20 client.25978 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:17.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f089ce0 0x56321f08c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 4 client.25978 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56321ef72860 con 0x56321f089ce0 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc20ee5640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f089ce0 0x56321f08c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc20ee5640 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f089ce0 0x56321f08c100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fe4a77359c93650 server_cookie=a4de5d3c71ff9f4b in_seq=0 out_seq=0 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efbfbfff640 10 client.25978 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efbfbfff640 1 -- 172.21.15.86:0/155188811 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56321ef72860 con 0x56321f089ce0 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efbfbfff640 10 client.25978 handle_command_reply: tid=0 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.869+0000 7efc25522740 1 client.25978 shutdown 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 msgr2=0x56321f0118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f0118a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc140988b0 tx=0x7efc1409ba90 comp rx=0 tx=0).stop 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 shutdown_connections 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x56321f089ce0 0x56321f08c100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56321f00e780 0x56321f0118a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56321f004f50 0x56321f016e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efbfbfff640 0 client.25978 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 --2- 172.21.15.86:0/155188811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321f0039e0 0x56321f0168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 >> 172.21.15.86:0/155188811 conn(0x56321efc8310 msgr2=0x56321f005520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 shutdown_connections 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 1 -- 172.21.15.86:0/155188811 wait complete. 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc25522740 20 client.25978 trim_cache size 0 max 16384 2024-09-05T06:46:17.876 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 msgr2=0x7efc0c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.870+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 0x7efc0c0a5300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ffe90 tx=0x7efc0802a480 comp rx=0 tx=0).stop 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 msgr2=0x7efc1c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 0x7efc1c19cce0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efc1400ec50 tx=0x7efc1400c6a0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 shutdown_connections 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efc0c0a2e40 0x7efc0c0a5300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1c102ef0 0x7efc1c19cce0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc1c102570 0x7efc1c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 --2- 172.21.15.86:0/2781003149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc1c100e20 0x7efc1c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:17.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 >> 172.21.15.86:0/2781003149 conn(0x7efc1c0fc7d0 msgr2=0x7efc1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:17.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 shutdown_connections 2024-09-05T06:46:17.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:17.871+0000 7efc23149640 1 -- 172.21.15.86:0/2781003149 wait complete. 2024-09-05T06:46:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:17 smithi115 ceph-mon[34791]: pgmap v154: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-09-05T06:46:18.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 86314 1.72 MB/sec execute 98 sec latency 18.193 ms 2024-09-05T06:46:18.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:46:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:19.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 86825 1.73 MB/sec execute 99 sec latency 21.834 ms 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 -- 172.21.15.86:0/2853350745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102570 msgr2=0x7f64141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 --2- 172.21.15.86:0/2853350745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102570 0x7f64141029b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6400009900 tx=0x7f640002a480 comp rx=0 tx=0).stop 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 -- 172.21.15.86:0/2853350745 shutdown_connections 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 --2- 172.21.15.86:0/2853350745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102ef0 0x7f641410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 --2- 172.21.15.86:0/2853350745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102570 0x7f64141029b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 --2- 172.21.15.86:0/2853350745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 0x7f6414101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 -- 172.21.15.86:0/2853350745 >> 172.21.15.86:0/2853350745 conn(0x7f64140fc7d0 msgr2=0x7f64140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 -- 172.21.15.86:0/2853350745 shutdown_connections 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.570+0000 7f641ad98640 1 -- 172.21.15.86:0/2853350745 wait complete. 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 Processor -- start 2024-09-05T06:46:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 -- start start 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 0x7f6414197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102570 0x7f6414197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6414110e30 con 0x7f6414102ef0 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6414110cb0 con 0x7f6414102570 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.571+0000 7f641ad98640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6414110fb0 con 0x7f6414100e20 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46208/0 (socket says 172.21.15.86:46208) 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 -- 172.21.15.86:0/3124824661 learned_addr learned my addr 172.21.15.86:0/3124824661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6418b34640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 0x7f6414197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6413fff640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102570 0x7f6414197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 -- 172.21.15.86:0/3124824661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 msgr2=0x7f6414197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 0x7f6414197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 -- 172.21.15.86:0/3124824661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102570 msgr2=0x7f6414197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102570 0x7f6414197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 -- 172.21.15.86:0/3124824661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641419c2c0 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6419335640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f640c00eca0 tx=0x7f640c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6411ffb640 1 -- 172.21.15.86:0/3124824661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c004b20 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641419c5b0 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6411ffb640 1 -- 172.21.15.86:0/3124824661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f640c004cc0 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.572+0000 7f6411ffb640 1 -- 172.21.15.86:0/3124824661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c017670 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.573+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6414069f70 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f6411ffb640 1 -- 172.21.15.86:0/3124824661 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f640c005020 con 0x7f6414102ef0 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 Processor -- start 2024-09-05T06:46:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 -- start start 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32acea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32ad9370 0x55af32adb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af32a342f0 con 0x55af32ace5d0 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af329dbcf0 con 0x55af32acfb40 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f641d171740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af32a302b0 con 0x55af32ad9370 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.575+0000 7f6411ffb640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 0x7f64040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43698/0 (socket says 172.21.15.86:43698) 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 -- 172.21.15.86:0/3592711809 learned_addr learned my addr 172.21.15.86:0/3592711809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6419335640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 0x7f64040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32ad9370 0x55af32adb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6418b34640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32acea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6411ffb640 1 -- 172.21.15.86:0/3124824661 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f640c0d1280 con 0x7f6414102ef0 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 -- 172.21.15.86:0/3592711809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32ad9370 msgr2=0x55af32adb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32ad9370 0x55af32adb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 -- 172.21.15.86:0/3592711809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 msgr2=0x55af32acea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32acea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6413fff640 1 -- 172.21.15.86:0/3592711809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af329c03a0 con 0x55af32acfb40 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6418b34640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.576+0000 7f6419335640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 0x7f64040a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f640c09d170 tx=0x7f640c093040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.577+0000 7f6413fff640 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55af32acf070 tx=0x7f6400000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d29db3779527d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.577+0000 7f63feffd640 1 -- 172.21.15.86:0/3592711809 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400035070 con 0x55af32acfb40 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.577+0000 7f63feffd640 1 -- 172.21.15.86:0/3592711809 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6400041860 con 0x55af32acfb40 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.577+0000 7f63feffd640 1 -- 172.21.15.86:0/3592711809 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000044e0 con 0x55af32acfb40 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- 172.21.15.86:0/3592711809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 msgr2=0x55af32ace090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55af32acf070 tx=0x7f6400000bc0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- 172.21.15.86:0/3592711809 shutdown_connections 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32ad9370 0x55af32adb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 --2- 172.21.15.86:0/3592711809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32acfb40 0x55af32ace090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- 172.21.15.86:0/3592711809 >> 172.21.15.86:0/3592711809 conn(0x55af32a97310 msgr2=0x55af32acac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- 172.21.15.86:0/3592711809 shutdown_connections 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- 172.21.15.86:0/3592711809 wait complete. 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 -- start start 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.578+0000 7f641d171740 1 Processor -- start 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32ad8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ad9370 0x55af32ad6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af32adc010 con 0x55af32ace5d0 2024-09-05T06:46:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af329ef060 con 0x55af32ad9370 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af32adc190 con 0x55af32acfb40 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f641d171740 10 client.0 _ll_register_callbacks cb 0x55af327428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32ad8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32ad8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46234/0 (socket says 172.21.15.86:46234) 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6413fff640 1 -- 172.21.15.86:0/4232527895 learned_addr learned my addr 172.21.15.86:0/4232527895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6413fff640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ad9370 0x55af32ad6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 -- 172.21.15.86:0/4232527895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ad9370 msgr2=0x55af32ad6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ad9370 0x55af32ad6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 -- 172.21.15.86:0/4232527895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 msgr2=0x55af32ad8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32ad8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 -- 172.21.15.86:0/4232527895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af327a53c0 con 0x55af32acfb40 2024-09-05T06:46:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f6418b34640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad6080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64080058c0 tx=0x7f6408012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d023c4587b0227 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f63f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.579+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408010dc0 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6400041860 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6408004020 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6408025050 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 10 client.25987 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af32ae7be0 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6400041860 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f640801a430 con 0x55af32acfb40 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 10 client.25987 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 10 client.25987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 20 client.25987 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32ae90a0 0x55af32aeb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 4 client.25987 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af32a41860 con 0x55af32ae90a0 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.580+0000 7f6413fff640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32ae90a0 0x55af32aeb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.581+0000 7f63f3fff640 10 client.25987 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.581+0000 7f6413fff640 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32ae90a0 0x55af32aeb520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e831684d66ab8f36 server_cookie=11e8befb47023b39 in_seq=0 out_seq=0 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.581+0000 7f63f3fff640 1 -- 172.21.15.86:0/4232527895 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af32a41860 con 0x55af32ae90a0 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.581+0000 7f63f3fff640 10 client.25987 handle_command_reply: tid=0 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.581+0000 7f641d171740 1 client.25987 shutdown 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 msgr2=0x55af32ad6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad6080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64080058c0 tx=0x7f6408012be0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 shutdown_connections 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32ae90a0 0x55af32aeb520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ad9370 0x55af32ad6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad6080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 --2- 172.21.15.86:0/4232527895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ace5d0 0x55af32ad8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 >> 172.21.15.86:0/4232527895 conn(0x55af32a97310 msgr2=0x55af32ad0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f63f3fff640 0 client.25987 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:46:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 shutdown_connections 2024-09-05T06:46:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 1 -- 172.21.15.86:0/4232527895 wait complete. 2024-09-05T06:46:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.582+0000 7f641d171740 20 client.25987 trim_cache size 0 max 16384 2024-09-05T06:46:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 Processor -- start 2024-09-05T06:46:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 -- start start 2024-09-05T06:46:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32aca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64141accb0 con 0x55af32ad9370 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ae7f0 con 0x55af32ace5d0 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.596+0000 7f641d171740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ad380 con 0x55af32acfb40 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49118/0 (socket says 172.21.15.86:49118) 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46250/0 (socket says 172.21.15.86:46250) 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 -- 172.21.15.86:0/1147024283 learned_addr learned my addr 172.21.15.86:0/1147024283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6413fff640 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32aca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 -- 172.21.15.86:0/1147024283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 msgr2=0x55af32aca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32aca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 -- 172.21.15.86:0/1147024283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 msgr2=0x55af32af0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 -- 172.21.15.86:0/1147024283 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af32acb180 con 0x55af32acfb40 2024-09-05T06:46:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6413fff640 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f6418b34640 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f640801a9c0 tx=0x7f640800ae30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9388941f8cee40 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f63fe7fc640 1 -- 172.21.15.86:0/1147024283 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408010bc0 con 0x55af32acfb40 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f63fe7fc640 1 -- 172.21.15.86:0/1147024283 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6408010d60 con 0x55af32acfb40 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.597+0000 7f63fe7fc640 1 -- 172.21.15.86:0/1147024283 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408031a80 con 0x55af32acfb40 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- 172.21.15.86:0/1147024283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 msgr2=0x55af32acac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f640801a9c0 tx=0x7f640800ae30 comp rx=0 tx=0).stop 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- 172.21.15.86:0/1147024283 shutdown_connections 2024-09-05T06:46:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32af0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32acac40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 --2- 172.21.15.86:0/1147024283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32aca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- 172.21.15.86:0/1147024283 >> 172.21.15.86:0/1147024283 conn(0x55af32a97310 msgr2=0x55af32ac5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- 172.21.15.86:0/1147024283 shutdown_connections 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- 172.21.15.86:0/1147024283 wait complete. 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.598+0000 7f641d171740 1 -- start start 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 Processor -- start 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32ad3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ae120 con 0x55af32ad9370 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64141ada50 con 0x55af32ace5d0 2024-09-05T06:46:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64141aac90 con 0x55af32acfb40 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f641d171740 10 client.0 _ll_register_callbacks cb 0x7f640800a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f6413fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f6413fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43714/0 (socket says 172.21.15.86:43714) 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f6413fff640 1 -- 172.21.15.86:0/737161278 learned_addr learned my addr 172.21.15.86:0/737161278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f6418b34640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.599+0000 7f6413fff640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32ad3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 -- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 msgr2=0x55af32ad35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 -- 172.21.15.86:0/737161278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 msgr2=0x55af32ad3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32ad3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 -- 172.21.15.86:0/737161278 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af32b52dc0 con 0x55af32ace5d0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f6413fff640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400051040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fcee55311677c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f63f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640005f020 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6408010d60 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6400031a10 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6400035040 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f641d171740 10 client.25985 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af32b54080 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.600+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6408010d60 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6400043400 con 0x55af32ace5d0 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 10 client.25985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 20 client.25985 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32b54250 0x55af32b56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 4 client.25985 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:46:19.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af32a3bd70 con 0x55af32b54250 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f6418b34640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32b54250 0x55af32b56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f63f3fff640 10 client.25985 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f6418b34640 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32b54250 0x55af32b56640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc22add720fe54f7 server_cookie=224775c7f9ee0a46 in_seq=0 out_seq=0 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f63f3fff640 1 -- 172.21.15.86:0/737161278 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af32a3bd70 con 0x55af32b54250 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f63f3fff640 10 client.25985 handle_command_reply: tid=0 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.601+0000 7f641d171740 1 client.25985 shutdown 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 msgr2=0x55af32ad3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400051040 comp rx=0 tx=0).stop 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 shutdown_connections 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55af32b54250 0x55af32b56640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f63f3fff640 0 client.25985 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af32ad9370 0x55af32ad3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af32acfb40 0x55af32ad35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 --2- 172.21.15.86:0/737161278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af32ace5d0 0x55af32ad3090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 >> 172.21.15.86:0/737161278 conn(0x55af32a97310 msgr2=0x55af32aca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 shutdown_connections 2024-09-05T06:46:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 1 -- 172.21.15.86:0/737161278 wait complete. 2024-09-05T06:46:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.602+0000 7f641d171740 20 client.25985 trim_cache size 0 max 16384 2024-09-05T06:46:19.608 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 msgr2=0x7f64040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 0x7f64040a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f640c09d170 tx=0x7f640c093040 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 msgr2=0x7f641419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f640c00eca0 tx=0x7f640c00b690 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 shutdown_connections 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f64040a2cf0 0x7f64040a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6414102ef0 0x7f641419bbe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6414102570 0x7f6414197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 --2- 172.21.15.86:0/3124824661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6414100e20 0x7f6414197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 >> 172.21.15.86:0/3124824661 conn(0x7f64140fc7d0 msgr2=0x7f64140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 shutdown_connections 2024-09-05T06:46:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:19.603+0000 7f641ad98640 1 -- 172.21.15.86:0/3124824661 wait complete. 2024-09-05T06:46:19.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:19 smithi086 ceph-mon[30943]: pgmap v155: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-05T06:46:19.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:19 smithi096 ceph-mon[34468]: pgmap v155: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-05T06:46:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:19 smithi115 ceph-mon[34791]: pgmap v155: 129 pgs: 129 active+clean; 89 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-05T06:46:20.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 87481 1.73 MB/sec execute 100 sec latency 28.811 ms 2024-09-05T06:46:20.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:46:20.901 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 -- 172.21.15.86:0/3710042092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102570 msgr2=0x7fdec81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 --2- 172.21.15.86:0/3710042092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102570 0x7fdec81029b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb002a480 comp rx=0 tx=0).stop 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 -- 172.21.15.86:0/3710042092 shutdown_connections 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 --2- 172.21.15.86:0/3710042092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102ef0 0x7fdec810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 --2- 172.21.15.86:0/3710042092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102570 0x7fdec81029b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 --2- 172.21.15.86:0/3710042092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 0x7fdec8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 -- 172.21.15.86:0/3710042092 >> 172.21.15.86:0/3710042092 conn(0x7fdec80fc7d0 msgr2=0x7fdec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 -- 172.21.15.86:0/3710042092 shutdown_connections 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.399+0000 7fdecd42d640 1 -- 172.21.15.86:0/3710042092 wait complete. 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 Processor -- start 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 -- start start 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 0x7fdec8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102ef0 0x7fdec819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111f40 con 0x7fdec8102ef0 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111dc0 con 0x7fdec8100e20 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.400+0000 7fdecd42d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81120c0 con 0x7fdec8102570 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 0x7fdec8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102ef0 0x7fdec819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49146/0 (socket says 172.21.15.86:49146) 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 -- 172.21.15.86:0/4184834197 learned_addr learned my addr 172.21.15.86:0/4184834197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 -- 172.21.15.86:0/4184834197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 msgr2=0x7fdec8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 0x7fdec8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 -- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102ef0 msgr2=0x7fdec819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102ef0 0x7fdec819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 -- 172.21.15.86:0/4184834197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec819d3b0 con 0x7fdec8102570 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdebffff640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002aa70 tx=0x7fdeb0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdec4ff9640 1 -- 172.21.15.86:0/4184834197 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003f070 con 0x7fdec8102570 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdec4ff9640 1 -- 172.21.15.86:0/4184834197 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb003bb20 con 0x7fdec8102570 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdec4ff9640 1 -- 172.21.15.86:0/4184834197 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035030 con 0x7fdec8102570 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec819d640 con 0x7fdec8102570 2024-09-05T06:46:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.401+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a4f80 con 0x7fdec8102570 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.404+0000 7fdec4ff9640 1 -- 172.21.15.86:0/4184834197 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdeb0042070 con 0x7fdec8102570 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.404+0000 7fdecf806740 1 Processor -- start 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.404+0000 7fdecf806740 1 -- start start 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec4ff9640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 0x7fdeb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a935fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a936a370 0x55f9a936c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9a92c52f0 con 0x55f9a9360b40 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9a926ccf0 con 0x55f9a936a370 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdecf806740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9a92c12b0 con 0x55f9a935f5d0 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec4ff9640 1 -- 172.21.15.86:0/4184834197 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdeb00efef0 con 0x7fdec8102570 2024-09-05T06:46:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46292/0 (socket says 172.21.15.86:46292) 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 -- 172.21.15.86:0/579533012 learned_addr learned my addr 172.21.15.86:0/579533012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec77fe640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 0x7fdeb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdebffff640 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a935fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 -- 172.21.15.86:0/579533012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 msgr2=0x55f9a935fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a935fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 -- 172.21.15.86:0/579533012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a936a370 msgr2=0x55f9a936c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a936a370 0x55f9a936c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.405+0000 7fdec6ffd640 1 -- 172.21.15.86:0/579533012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a92513a0 con 0x55f9a9360b40 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.406+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009cf0 tx=0x7fdeb8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ff4c5cde0dec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.406+0000 7fdec77fe640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 0x7fdeb40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55f9a9360070 tx=0x7fdec0011810 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.406+0000 7fdebd7fa640 1 -- 172.21.15.86:0/579533012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8004230 con 0x55f9a9360b40 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.406+0000 7fdebd7fa640 1 -- 172.21.15.86:0/579533012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb80043d0 con 0x55f9a9360b40 2024-09-05T06:46:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.406+0000 7fdebd7fa640 1 -- 172.21.15.86:0/579533012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80197e0 con 0x55f9a9360b40 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- 172.21.15.86:0/579533012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 msgr2=0x55f9a935f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009cf0 tx=0x7fdeb8011810 comp rx=0 tx=0).stop 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- 172.21.15.86:0/579533012 shutdown_connections 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a936a370 0x55f9a936c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a935fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 --2- 172.21.15.86:0/579533012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a935f090 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- 172.21.15.86:0/579533012 >> 172.21.15.86:0/579533012 conn(0x55f9a9328310 msgr2=0x55f9a935bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- 172.21.15.86:0/579533012 shutdown_connections 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- 172.21.15.86:0/579533012 wait complete. 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 -- start start 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.407+0000 7fdecf806740 1 Processor -- start 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a9366f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 0x55f9a93674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9a936d070 con 0x55f9a9360b40 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9a9280060 con 0x55f9a935f5d0 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9a936d1f0 con 0x55f9a936a370 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdecf806740 10 client.0 _ll_register_callbacks cb 0x55f9a8fd38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 0x55f9a93674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43756/0 (socket says 172.21.15.86:43756) 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 -- 172.21.15.86:0/3219749587 learned_addr learned my addr 172.21.15.86:0/3219749587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdebffff640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a9366f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 -- 172.21.15.86:0/3219749587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 msgr2=0x55f9a93674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 0x55f9a93674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 -- 172.21.15.86:0/3219749587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 msgr2=0x55f9a9366f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a9366f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec6ffd640 1 -- 172.21.15.86:0/3219749587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a90363c0 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdebffff640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a9366f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.408+0000 7fdec77fe640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 0x55f9a93674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80058c0 tx=0x7fdeb80038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48af46beb40c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdeae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8027070 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb80043d0 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdeb8005450 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdeb8033050 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdecf806740 10 client.25994 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9a937ce90 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.409+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb80043d0 con 0x55f9a935f5d0 2024-09-05T06:46:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdeb8004c90 con 0x55f9a935f5d0 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 10 client.25994 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 20 client.25994 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a937e620 0x55f9a9380a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 4 client.25994 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a92d2860 con 0x55f9a937e620 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdebffff640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a937e620 0x55f9a9380a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdebffff640 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a937e620 0x55f9a9380a90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd708b1b59d0eab server_cookie=24663446524b96de in_seq=0 out_seq=0 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.410+0000 7fdeae7fc640 10 client.25994 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3219749587 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9a92d2860 con 0x55f9a937e620 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdeae7fc640 10 client.25994 handle_command_reply: tid=0 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 client.25994 shutdown 2024-09-05T06:46:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 msgr2=0x55f9a9369cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80058c0 tx=0x7fdeb80038a0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 shutdown_connections 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a937e620 0x55f9a9380a90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a936a370 0x55f9a93674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a9360b40 0x55f9a9366f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 --2- 172.21.15.86:0/3219749587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a9369cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdeae7fc640 0 client.25994 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.411+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 >> 172.21.15.86:0/3219749587 conn(0x55f9a9328310 msgr2=0x55f9a9361960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.412+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 shutdown_connections 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.412+0000 7fdecf806740 1 -- 172.21.15.86:0/3219749587 wait complete. 2024-09-05T06:46:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.412+0000 7fdecf806740 20 client.25994 trim_cache size 0 max 16384 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 Processor -- start 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 -- start start 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a9376310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 0x55f9a9360300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81acdf0 con 0x55f9a936a370 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ae930 con 0x55f9a9360b40 2024-09-05T06:46:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdecf806740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ad4c0 con 0x55f9a935f5d0 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46312/0 (socket says 172.21.15.86:46312) 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdec77fe640 1 -- 172.21.15.86:0/3304608477 learned_addr learned my addr 172.21.15.86:0/3304608477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 0x55f9a9360300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a9376310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 -- 172.21.15.86:0/3304608477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 msgr2=0x55f9a9376310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a9376310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 -- 172.21.15.86:0/3304608477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 msgr2=0x55f9a9356ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdebffff640 1 -- 172.21.15.86:0/3304608477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a935c960 con 0x55f9a9360b40 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.427+0000 7fdec77fe640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.428+0000 7fdebffff640 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 0x55f9a9360300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00ed200 tx=0x7fdeb00dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93ce4e7f4c05262 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.428+0000 7fdebcff9640 1 -- 172.21.15.86:0/3304608477 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0049040 con 0x55f9a9360b40 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.428+0000 7fdebcff9640 1 -- 172.21.15.86:0/3304608477 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb00dc0b0 con 0x55f9a9360b40 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.428+0000 7fdebcff9640 1 -- 172.21.15.86:0/3304608477 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00b86f0 con 0x55f9a9360b40 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- 172.21.15.86:0/3304608477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 msgr2=0x55f9a9360300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 0x55f9a9360300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00ed200 tx=0x7fdeb00dc040 comp rx=0 tx=0).stop 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- 172.21.15.86:0/3304608477 shutdown_connections 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a9356ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a9360b40 0x55f9a9360300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- 172.21.15.86:0/3304608477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a935f5d0 0x55f9a9376310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- 172.21.15.86:0/3304608477 >> 172.21.15.86:0/3304608477 conn(0x55f9a9328310 msgr2=0x55f9a9365840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- 172.21.15.86:0/3304608477 shutdown_connections 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- 172.21.15.86:0/3304608477 wait complete. 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 -- start start 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 Processor -- start 2024-09-05T06:46:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a936d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.429+0000 7fdecf806740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 0x55f9a936d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81ae260 con 0x55f9a936a370 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81adb90 con 0x55f9a935f5d0 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81aadd0 con 0x55f9a9360b40 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 10 client.0 _ll_register_callbacks cb 0x7fdeb0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a936d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46324/0 (socket says 172.21.15.86:46324) 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 -- 172.21.15.86:0/3083721739 learned_addr learned my addr 172.21.15.86:0/3083721739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdebffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 0x55f9a936d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 -- 172.21.15.86:0/3083721739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 msgr2=0x55f9a936d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 0x55f9a936d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 -- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 msgr2=0x55f9a936d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a936d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 -- 172.21.15.86:0/3083721739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a93e7530 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdebffff640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 0x55f9a936d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a936d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdec77fe640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009a80 tx=0x7fdec0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16af61664e5591 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdeae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00275f0 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdec0027790 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.430+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9a92ccd70 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdec0035050 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 10 client.16686 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9a93e8870 con 0x55f9a936a370 2024-09-05T06:46:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f9a92ccd70 con 0x55f9a936a370 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdec001ca50 con 0x55f9a936a370 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 10 client.16686 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 20 client.16686 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a93e8ff0 0x55f9a93eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 4 client.16686 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9a9249170 con 0x55f9a93e8ff0 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.431+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a93e8ff0 0x55f9a93eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdec6ffd640 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a93e8ff0 0x55f9a93eb410 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7112f88e64ae80ec server_cookie=6003e6a74029cc01 in_seq=0 out_seq=0 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdeae7fc640 10 client.16686 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdeae7fc640 1 -- 172.21.15.86:0/3083721739 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9a9249170 con 0x55f9a93e8ff0 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdeae7fc640 10 client.16686 handle_command_reply: tid=0 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 client.16686 shutdown 2024-09-05T06:46:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 msgr2=0x55f9a936dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009a80 tx=0x7fdec0004ab0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 shutdown_connections 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f9a93e8ff0 0x55f9a93eb410 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a936a370 0x55f9a936dec0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9a9360b40 0x55f9a936d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdeae7fc640 0 client.16686 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 --2- 172.21.15.86:0/3083721739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a935f5d0 0x55f9a936d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.432+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 >> 172.21.15.86:0/3083721739 conn(0x55f9a9328310 msgr2=0x55f9a9361110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 shutdown_connections 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecf806740 1 -- 172.21.15.86:0/3083721739 wait complete. 2024-09-05T06:46:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecf806740 20 client.16686 trim_cache size 0 max 16384 2024-09-05T06:46:21.442 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:46:21.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 msgr2=0x7fdeb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 0x7fdeb40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55f9a9360070 tx=0x7fdec0011810 comp rx=0 tx=0).stop 2024-09-05T06:46:21.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 msgr2=0x7fdec8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002aa70 tx=0x7fdeb0003c10 comp rx=0 tx=0).stop 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 shutdown_connections 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdeb40a2cf0 0x7fdeb40a51b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec8102ef0 0x7fdec819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdec8102570 0x7fdec8198940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 --2- 172.21.15.86:0/4184834197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdec8100e20 0x7fdec8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 >> 172.21.15.86:0/4184834197 conn(0x7fdec80fc7d0 msgr2=0x7fdec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.433+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 shutdown_connections 2024-09-05T06:46:21.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:21.434+0000 7fdecd42d640 1 -- 172.21.15.86:0/4184834197 wait complete. 2024-09-05T06:46:21.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 88092 1.77 MB/sec execute 101 sec latency 26.941 ms 2024-09-05T06:46:21.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:21 smithi086 ceph-mon[30943]: pgmap v156: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-05T06:46:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:21 smithi096 ceph-mon[34468]: pgmap v156: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-05T06:46:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:21 smithi115 ceph-mon[34791]: pgmap v156: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-09-05T06:46:22.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 88681 1.80 MB/sec execute 102 sec latency 13.896 ms 2024-09-05T06:46:23.230 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:46:23.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:46:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:23.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 89091 1.80 MB/sec execute 103 sec latency 11.982 ms 2024-09-05T06:46:23.627 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:23 smithi086 ceph-mon[30943]: pgmap v157: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:23 smithi096 ceph-mon[34468]: pgmap v157: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:23 smithi115 ceph-mon[34791]: pgmap v157: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- 172.21.15.86:0/186301243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 msgr2=0x7f6c3c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 --2- 172.21.15.86:0/186301243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c101200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009900 tx=0x7f6c2c02a480 comp rx=0 tx=0).stop 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- 172.21.15.86:0/186301243 shutdown_connections 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 --2- 172.21.15.86:0/186301243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 --2- 172.21.15.86:0/186301243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102570 0x7f6c3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 --2- 172.21.15.86:0/186301243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c101200 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- 172.21.15.86:0/186301243 >> 172.21.15.86:0/186301243 conn(0x7f6c3c0fc7d0 msgr2=0x7f6c3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- 172.21.15.86:0/186301243 shutdown_connections 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- 172.21.15.86:0/186301243 wait complete. 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 Processor -- start 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.035+0000 7f6c41646640 1 -- start start 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 0x7f6c3c103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c110e30 con 0x7f6c3c100e20 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c110cb0 con 0x7f6c3c102570 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c41646640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c110fb0 con 0x7f6c3c102ef0 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 0x7f6c3c103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54352/0 (socket says 172.21.15.86:54352) 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3affd640 1 -- 172.21.15.86:0/4218352743 learned_addr learned my addr 172.21.15.86:0/4218352743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 -- 172.21.15.86:0/4218352743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 msgr2=0x7f6c3c1041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c1041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 -- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 msgr2=0x7f6c3c103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 -- 172.21.15.86:0/4218352743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1a6990 con 0x7f6c3c102570 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c3affd640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.036+0000 7f6c327fc640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 0x7f6c3c103c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d720 tx=0x7f6c2400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.037+0000 7f6c38ff9640 1 -- 172.21.15.86:0/4218352743 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24017070 con 0x7f6c3c102570 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.037+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1a6c80 con 0x7f6c3c102570 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.037+0000 7f6c38ff9640 1 -- 172.21.15.86:0/4218352743 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c2400de30 con 0x7f6c3c102570 2024-09-05T06:46:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.037+0000 7f6c38ff9640 1 -- 172.21.15.86:0/4218352743 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c240047a0 con 0x7f6c3c102570 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.037+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1a71c0 con 0x7f6c3c102570 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.039+0000 7f6c38ff9640 1 -- 172.21.15.86:0/4218352743 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c240099d0 con 0x7f6c3c102570 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.039+0000 7f6c43a1f740 1 Processor -- start 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.039+0000 7f6c43a1f740 1 -- start start 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c38ff9640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 0x7f6c280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 0x7f6c280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c38ff9640 1 -- 172.21.15.86:0/4218352743 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6c240cf460 con 0x7f6c3c102570 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b10a480 0x56154b114800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 0x56154b117130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56154b06f280 con 0x56154b114d40 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56154b016b10 con 0x56154b109b60 2024-09-05T06:46:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56154b06b240 con 0x56154b10a480 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 0x56154b117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:60112/0 (socket says 172.21.15.86:60112) 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 0x56154b117130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54356/0 (socket says 172.21.15.86:54356) 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 -- 172.21.15.86:0/2668982827 learned_addr learned my addr 172.21.15.86:0/2668982827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 -- 172.21.15.86:0/2668982827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b10a480 msgr2=0x56154b114800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b10a480 0x56154b114800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 -- 172.21.15.86:0/2668982827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 msgr2=0x56154b117130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 0x7f6c280a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009370 tx=0x7f6c34013ba0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 0x56154b117130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.040+0000 7f6c3affd640 1 -- 172.21.15.86:0/2668982827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154b0231f0 con 0x56154b109b60 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.041+0000 7f6c3affd640 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009a20 tx=0x7f6c2c0314b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b94dd830664eef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.041+0000 7f6c317fa640 1 -- 172.21.15.86:0/2668982827 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c040070 con 0x56154b109b60 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.041+0000 7f6c317fa640 1 -- 172.21.15.86:0/2668982827 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c2c005040 con 0x56154b109b60 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.041+0000 7f6c317fa640 1 -- 172.21.15.86:0/2668982827 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c035040 con 0x56154b109b60 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- 172.21.15.86:0/2668982827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 msgr2=0x56154b109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009a20 tx=0x7f6c2c0314b0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- 172.21.15.86:0/2668982827 shutdown_connections 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b114d40 0x56154b117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b10a480 0x56154b114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/2668982827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b109f40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- 172.21.15.86:0/2668982827 >> 172.21.15.86:0/2668982827 conn(0x56154b1060f0 msgr2=0x56154b1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- 172.21.15.86:0/2668982827 shutdown_connections 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- 172.21.15.86:0/2668982827 wait complete. 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 -- start start 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.042+0000 7f6c43a1f740 1 Processor -- start 2024-09-05T06:46:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b111c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b114d40 0x56154b1126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56154b117910 con 0x56154b10a480 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56154b029f10 con 0x56154b109b60 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56154b117a90 con 0x56154b114d40 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 10 client.0 _ll_register_callbacks cb 0x56154aff2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54372/0 (socket says 172.21.15.86:54372) 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 -- 172.21.15.86:0/1372474793 learned_addr learned my addr 172.21.15.86:0/1372474793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b114d40 0x56154b1126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 -- 172.21.15.86:0/1372474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b114d40 msgr2=0x56154b1126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b114d40 0x56154b1126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 -- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 msgr2=0x56154b111c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b111c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 -- 172.21.15.86:0/1372474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154ade06a0 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c2409af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47994f2e2602d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c227fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2409b540 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6c240cf170 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.043+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2c005040 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c240b9050 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 10 client.16698 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56154b123600 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c2c005040 con 0x56154b10a480 2024-09-05T06:46:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6c24097bb0 con 0x56154b10a480 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 10 client.16698 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 20 client.16698 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b124a10 0x56154b126e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 4 client.16698 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56154b07c7f0 con 0x56154b124a10 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.044+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b124a10 0x56154b126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.045+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b124a10 0x56154b126e00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab5b43433bd51ba server_cookie=6b772a3f8b126b55 in_seq=0 out_seq=0 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.045+0000 7f6c227fc640 10 client.16698 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.045+0000 7f6c227fc640 1 -- 172.21.15.86:0/1372474793 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56154b07c7f0 con 0x56154b124a10 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.045+0000 7f6c227fc640 10 client.16698 handle_command_reply: tid=0 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.045+0000 7f6c43a1f740 1 client.16698 shutdown 2024-09-05T06:46:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 msgr2=0x56154b112170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c2409af90 comp rx=0 tx=0).stop 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 shutdown_connections 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b124a10 0x56154b126e00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b114d40 0x56154b1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b10a480 0x56154b112170 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1372474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b109b60 0x56154b111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 >> 172.21.15.86:0/1372474793 conn(0x56154b1060f0 msgr2=0x56154b10bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c227fc640 0 client.16698 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 shutdown_connections 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1372474793 wait complete. 2024-09-05T06:46:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.046+0000 7f6c43a1f740 20 client.16698 trim_cache size 0 max 16384 2024-09-05T06:46:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.067+0000 7f6c43a1f740 1 Processor -- start 2024-09-05T06:46:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.067+0000 7f6c43a1f740 1 -- start start 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b12c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b12cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b11b0 con 0x56154b110c10 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b2cf0 con 0x56154b114d40 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b1880 con 0x56154b109b60 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54384/0 (socket says 172.21.15.86:54384) 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b12c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 -- 172.21.15.86:0/519941584 learned_addr learned my addr 172.21.15.86:0/519941584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c327fc640 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b12cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 -- 172.21.15.86:0/519941584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 msgr2=0x56154b12c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b12c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 -- 172.21.15.86:0/519941584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 msgr2=0x56154b12cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b12cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 -- 172.21.15.86:0/519941584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154b114500 con 0x56154b110c10 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.068+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400fd30 tx=0x7f6c3401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d949089f2f7537 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.069+0000 7f6c30ff9640 1 -- 172.21.15.86:0/519941584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34011470 con 0x56154b110c10 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.069+0000 7f6c30ff9640 1 -- 172.21.15.86:0/519941584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c34011ac0 con 0x56154b110c10 2024-09-05T06:46:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.069+0000 7f6c30ff9640 1 -- 172.21.15.86:0/519941584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34028a70 con 0x56154b110c10 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.069+0000 7f6c43a1f740 1 -- 172.21.15.86:0/519941584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 msgr2=0x56154b10b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.069+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400fd30 tx=0x7f6c3401c000 comp rx=0 tx=0).stop 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- 172.21.15.86:0/519941584 shutdown_connections 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b10b5b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b12cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/519941584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b12c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- 172.21.15.86:0/519941584 >> 172.21.15.86:0/519941584 conn(0x56154b1060f0 msgr2=0x56154b110500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- 172.21.15.86:0/519941584 shutdown_connections 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- 172.21.15.86:0/519941584 wait complete. 2024-09-05T06:46:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- start start 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 Processor -- start 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b119700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b2620 con 0x56154b110c10 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b1f50 con 0x56154b114d40 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1af190 con 0x56154b109b60 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c43a1f740 10 client.0 _ll_register_callbacks cb 0x7f6c24005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.070+0000 7f6c327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:33562/0 (socket says 172.21.15.86:33562) 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 -- 172.21.15.86:0/1694917346 learned_addr learned my addr 172.21.15.86:0/1694917346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54388/0 (socket says 172.21.15.86:54388) 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3b7fe640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b119700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 -- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 msgr2=0x56154b119700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b119700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 -- 172.21.15.86:0/1694917346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 msgr2=0x56154b1191c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 -- 172.21.15.86:0/1694917346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154b18e8c0 con 0x56154b109b60 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c3affd640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c004650 tx=0x7f6c2c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aaebd83c17ec82f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c227fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c053020 con 0x56154b109b60 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c34011ac0 con 0x56154b109b60 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.071+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6c2c004760 con 0x56154b109b60 2024-09-05T06:46:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c05e050 con 0x56154b109b60 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 10 client.26026 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56154b18fbe0 con 0x56154b109b60 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c34011ac0 con 0x56154b109b60 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6c2c04f690 con 0x56154b109b60 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 20 client.26026 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b18fe60 0x56154b192250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 4 client.26026 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56154b076d00 con 0x56154b18fe60 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.072+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b18fe60 0x56154b192250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.073+0000 7f6c327fc640 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b18fe60 0x56154b192250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2938e72e88c04c86 server_cookie=eec8bf19bfd510f0 in_seq=0 out_seq=0 2024-09-05T06:46:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.073+0000 7f6c227fc640 10 client.26026 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.073+0000 7f6c227fc640 1 -- 172.21.15.86:0/1694917346 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56154b076d00 con 0x56154b18fe60 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.073+0000 7f6c227fc640 10 client.26026 handle_command_reply: tid=0 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.073+0000 7f6c43a1f740 1 client.26026 shutdown 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 msgr2=0x56154b118c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c004650 tx=0x7f6c2c005de0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 shutdown_connections 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56154b18fe60 0x56154b192250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c227fc640 0 client.26026 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56154b114d40 0x56154b119700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b110c10 0x56154b1191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 --2- 172.21.15.86:0/1694917346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56154b109b60 0x56154b118c80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 >> 172.21.15.86:0/1694917346 conn(0x56154b1060f0 msgr2=0x56154b104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 shutdown_connections 2024-09-05T06:46:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 1 -- 172.21.15.86:0/1694917346 wait complete. 2024-09-05T06:46:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.074+0000 7f6c43a1f740 20 client.26026 trim_cache size 0 max 16384 2024-09-05T06:46:24.080 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:46:24.080 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:46:24.081 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "a479d661-402a-4e41-9566-25d3ba41715a", 2024-09-05T06:46:24.081 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:46:24.081 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:46:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 msgr2=0x7f6c280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 0x7f6c280a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009370 tx=0x7f6c34013ba0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 msgr2=0x7f6c3c103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 0x7f6c3c103c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d720 tx=0x7f6c2400dbe0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 shutdown_connections 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6c280a2d70 0x7f6c280a5230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c102570 0x7f6c3c103c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 --2- 172.21.15.86:0/4218352743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3c100e20 0x7f6c3c103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 >> 172.21.15.86:0/4218352743 conn(0x7f6c3c0fc7d0 msgr2=0x7f6c3c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 shutdown_connections 2024-09-05T06:46:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:24.075+0000 7f6c41646640 1 -- 172.21.15.86:0/4218352743 wait complete. 2024-09-05T06:46:24.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 89407 1.79 MB/sec execute 104 sec latency 27.960 ms 2024-09-05T06:46:24.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:24.845 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.846 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:24.846 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.846 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:46:24.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:46:24.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.149 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:25.513 INFO:tasks.workunit.client.0.smithi086.stdout: 1 89964 1.79 MB/sec execute 105 sec latency 14.743 ms 2024-09-05T06:46:25.532 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:25.532 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:46:25.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:25 smithi086 ceph-mon[30943]: pgmap v158: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:25.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 -- 172.21.15.86:0/3235509400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41036e0 msgr2=0x7fead4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 --2- 172.21.15.86:0/3235509400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41036e0 0x7fead4107b90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feac4009930 tx=0x7feac402a420 comp rx=0 tx=0).stop 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 -- 172.21.15.86:0/3235509400 shutdown_connections 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 --2- 172.21.15.86:0/3235509400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead41082a0 0x7fead410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 --2- 172.21.15.86:0/3235509400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41036e0 0x7fead4107b90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 --2- 172.21.15.86:0/3235509400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead4102d30 0x7fead4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 -- 172.21.15.86:0/3235509400 >> 172.21.15.86:0/3235509400 conn(0x7fead40fc7d0 msgr2=0x7fead40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 -- 172.21.15.86:0/3235509400 shutdown_connections 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.620+0000 7feadc4e6640 1 -- 172.21.15.86:0/3235509400 wait complete. 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 Processor -- start 2024-09-05T06:46:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 -- start start 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 0x7fead4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 0x7fead419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fead4111f10 con 0x7fead41082a0 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fead4111d90 con 0x7fead4102d30 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.621+0000 7feadc4e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fead4112090 con 0x7fead41036e0 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feadaa83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 0x7fead419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:60160/0 (socket says 172.21.15.86:60160) 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 -- 172.21.15.86:0/1352135918 learned_addr learned my addr 172.21.15.86:0/1352135918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7fead9a81640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 0x7fead4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 -- 172.21.15.86:0/1352135918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 msgr2=0x7fead4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 0x7fead4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 -- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 msgr2=0x7fead419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 0x7fead419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 -- 172.21.15.86:0/1352135918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead419d400 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feadaa83640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 0x7fead419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7fead9a81640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 0x7fead4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feada282640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fead000c920 tx=0x7fead000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feacb7fe640 1 -- 172.21.15.86:0/1352135918 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0007c70 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feacb7fe640 1 -- 172.21.15.86:0/1352135918 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead0007e10 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead419d6f0 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.622+0000 7feacb7fe640 1 -- 172.21.15.86:0/1352135918 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0018820 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.623+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead41a4fd0 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.625+0000 7feacb7fe640 1 -- 172.21.15.86:0/1352135918 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fead00189c0 con 0x7fead4102d30 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.625+0000 7feade8bf740 1 Processor -- start 2024-09-05T06:46:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.625+0000 7feade8bf740 1 -- start start 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e1249c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e1249ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e123f32e0 con 0x563e1248d380 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e1239ace0 con 0x563e1248dca0 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feade8bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563e123ef2a0 con 0x563e1249ca90 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feada282640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e1249c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54414/0 (socket says 172.21.15.86:54414) 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 -- 172.21.15.86:0/949071312 learned_addr learned my addr 172.21.15.86:0/949071312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feacb7fe640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 0x7feac00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7fead9a81640 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e1249ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 0x7feac00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 -- 172.21.15.86:0/949071312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 msgr2=0x563e1249ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feacb7fe640 1 -- 172.21.15.86:0/1352135918 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fead00cf960 con 0x7fead4102d30 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e1249ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 -- 172.21.15.86:0/949071312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 msgr2=0x563e1249c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e1249c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feadaa83640 1 -- 172.21.15.86:0/949071312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1237f440 con 0x563e1248d380 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.626+0000 7feada282640 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e1249c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.627+0000 7feadaa83640 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feacc009fd0 tx=0x7feacc00f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3933bff0842e4afe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.627+0000 7feadaa83640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 0x7feac00a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feacc024000 tx=0x7feacc022000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.627+0000 7feac8ff9640 1 -- 172.21.15.86:0/949071312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc023020 con 0x563e1248d380 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.627+0000 7feac8ff9640 1 -- 172.21.15.86:0/949071312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feacc0138a0 con 0x563e1248d380 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.627+0000 7feac8ff9640 1 -- 172.21.15.86:0/949071312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0107a0 con 0x563e1248d380 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- 172.21.15.86:0/949071312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 msgr2=0x563e1248d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feacc009fd0 tx=0x7feacc00f7d0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- 172.21.15.86:0/949071312 shutdown_connections 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e1249ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e1249c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 --2- 172.21.15.86:0/949071312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1248d760 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- 172.21.15.86:0/949071312 >> 172.21.15.86:0/949071312 conn(0x563e12489a30 msgr2=0x563e12489e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- 172.21.15.86:0/949071312 shutdown_connections 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- 172.21.15.86:0/949071312 wait complete. 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 -- start start 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.628+0000 7feade8bf740 1 Processor -- start 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248dca0 0x563e124996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1249ca90 0x563e12499c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e1249f660 con 0x563e1248d380 2024-09-05T06:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e123ae050 con 0x563e1249ca90 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563e1249f7e0 con 0x563e1248dca0 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 10 client.0 _ll_register_callbacks cb 0x563e123ee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54426/0 (socket says 172.21.15.86:54426) 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 -- 172.21.15.86:0/1957105783 learned_addr learned my addr 172.21.15.86:0/1957105783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feadaa83640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1249ca90 0x563e12499c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7fead9a81640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248dca0 0x563e124996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 -- 172.21.15.86:0/1957105783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248dca0 msgr2=0x563e124996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248dca0 0x563e124996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 -- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1249ca90 msgr2=0x563e12499c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1249ca90 0x563e12499c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 -- 172.21.15.86:0/1957105783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e121643c0 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feada282640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fead0098470 tx=0x7fead000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec16b311dc621ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feabdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead009c1a0 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fead0098110 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.629+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e12400850 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fead00a4710 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 10 client.16716 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e124ab110 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563e12400850 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fead00a4280 con 0x563e1248d380 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 10 client.16716 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 20 client.16716 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e124ac660 0x563e124aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 4 client.16716 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e123fad60 con 0x563e124ac660 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.630+0000 7fead9a81640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e124ac660 0x563e124aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.631+0000 7fead9a81640 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e124ac660 0x563e124aea50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59b9f60618ddefe server_cookie=bc290a0de404e03e in_seq=0 out_seq=0 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.631+0000 7feabdffb640 10 client.16716 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.631+0000 7feabdffb640 1 -- 172.21.15.86:0/1957105783 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e123fad60 con 0x563e124ac660 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.631+0000 7feabdffb640 10 client.16716 handle_command_reply: tid=0 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.631+0000 7feade8bf740 1 client.16716 shutdown 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 msgr2=0x563e1249c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fead0098470 tx=0x7fead000a620 comp rx=0 tx=0).stop 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 shutdown_connections 2024-09-05T06:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e124ac660 0x563e124aea50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1249ca90 0x563e12499c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248dca0 0x563e124996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feabdffb640 0 client.16716 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 --2- 172.21.15.86:0/1957105783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248d380 0x563e1249c440 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 >> 172.21.15.86:0/1957105783 conn(0x563e12489a30 msgr2=0x563e1248f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 shutdown_connections 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 1 -- 172.21.15.86:0/1957105783 wait complete. 2024-09-05T06:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.633+0000 7feade8bf740 20 client.16716 trim_cache size 0 max 16384 2024-09-05T06:46:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.650+0000 7feade8bf740 1 Processor -- start 2024-09-05T06:46:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.650+0000 7feade8bf740 1 -- start start 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 0x563e124a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e124b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fead41acdd0 con 0x563e1248dca0 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ae910 con 0x563e1248d380 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feade8bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ad4a0 con 0x563e1249ca90 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54438/0 (socket says 172.21.15.86:54438) 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 -- 172.21.15.86:0/4256184191 learned_addr learned my addr 172.21.15.86:0/4256184191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feada282640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 0x563e124a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feadaa83640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e124b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 -- 172.21.15.86:0/4256184191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 msgr2=0x563e124b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e124b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 -- 172.21.15.86:0/4256184191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 msgr2=0x563e124a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 0x563e124a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 -- 172.21.15.86:0/4256184191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1249a000 con 0x563e1248dca0 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feadaa83640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e124b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7feada282640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 0x563e124a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.651+0000 7fead9a81640 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7feac4006010 tx=0x7feac4004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac77718b79a3f2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feabffff640 1 -- 172.21.15.86:0/4256184191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4049070 con 0x563e1248dca0 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feabffff640 1 -- 172.21.15.86:0/4256184191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feac4004440 con 0x563e1248dca0 2024-09-05T06:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feabffff640 1 -- 172.21.15.86:0/4256184191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035040 con 0x563e1248dca0 2024-09-05T06:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 -- 172.21.15.86:0/4256184191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 msgr2=0x563e124b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7feac4006010 tx=0x7feac4004110 comp rx=0 tx=0).stop 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 -- 172.21.15.86:0/4256184191 shutdown_connections 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1249ca90 0x563e124b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1248dca0 0x563e124b4170 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 --2- 172.21.15.86:0/4256184191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248d380 0x563e124a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.652+0000 7feade8bf740 1 -- 172.21.15.86:0/4256184191 >> 172.21.15.86:0/4256184191 conn(0x563e12489a30 msgr2=0x563e12494d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- 172.21.15.86:0/4256184191 shutdown_connections 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- 172.21.15.86:0/4256184191 wait complete. 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- start start 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 Processor -- start 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248d380 0x563e124a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e124a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fead41ae240 con 0x563e1249ca90 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fead41adb70 con 0x563e1248dca0 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fead41aadb0 con 0x563e1248d380 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feadaa83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feadaa83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54448/0 (socket says 172.21.15.86:54448) 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7fead9a81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e124a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feadaa83640 1 -- 172.21.15.86:0/3495850001 learned_addr learned my addr 172.21.15.86:0/3495850001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feade8bf740 10 client.0 _ll_register_callbacks cb 0x7fead000fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.653+0000 7feada282640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248d380 0x563e124a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 -- 172.21.15.86:0/3495850001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248d380 msgr2=0x563e124a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248d380 0x563e124a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 -- 172.21.15.86:0/3495850001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 msgr2=0x563e124a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 -- 172.21.15.86:0/3495850001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e12516140 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feadaa83640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7fead9a81640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e124a06b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x563e1248ab70 tx=0x7feac40042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f525484e70cadfc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feabdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035030 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7feac40037f0 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac4004440 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feac4049040 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feade8bf740 10 client.26036 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e12517310 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.654+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feac4004440 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7feac4055030 con 0x563e1248dca0 2024-09-05T06:46:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 10 client.26036 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 20 client.26036 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e12517a00 0x563e12519e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 4 client.26036 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e12377210 con 0x563e12517a00 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feada282640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e12517a00 0x563e12519e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feabdffb640 10 client.26036 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feada282640 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e12517a00 0x563e12519e20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c02370afaf78de server_cookie=529869905e14d097 in_seq=0 out_seq=0 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feabdffb640 1 -- 172.21.15.86:0/3495850001 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e12377210 con 0x563e12517a00 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feabdffb640 10 client.26036 handle_command_reply: tid=0 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.655+0000 7feade8bf740 1 client.26036 shutdown 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 msgr2=0x563e124a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e124a06b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x563e1248ab70 tx=0x7feac40042c0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 shutdown_connections 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563e12517a00 0x563e12519e20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1249ca90 0x563e124a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e1248dca0 0x563e124a06b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 --2- 172.21.15.86:0/3495850001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e1248d380 0x563e124a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feabdffb640 0 client.26036 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 >> 172.21.15.86:0/3495850001 conn(0x563e12489a30 msgr2=0x563e12488350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 shutdown_connections 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 1 -- 172.21.15.86:0/3495850001 wait complete. 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feade8bf740 20 client.26036 trim_cache size 0 max 16384 2024-09-05T06:46:25.662 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "a479d661-402a-4e41-9566-25d3ba41715a": { 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "a479d661-402a-4e41-9566-25d3ba41715a", 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:25.663 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 msgr2=0x7feac00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 0x7feac00a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feacc024000 tx=0x7feacc022000 comp rx=0 tx=0).stop 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 msgr2=0x7fead4198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fead000c920 tx=0x7fead000cde0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 shutdown_connections 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feac00a2cf0 0x7feac00a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead41082a0 0x7fead419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fead41036e0 0x7fead4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 --2- 172.21.15.86:0/1352135918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fead4102d30 0x7fead4198450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:25.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 >> 172.21.15.86:0/1352135918 conn(0x7fead40fc7d0 msgr2=0x7fead4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:25.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.656+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 shutdown_connections 2024-09-05T06:46:25.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:25.657+0000 7feadc4e6640 1 -- 172.21.15.86:0/1352135918 wait complete. 2024-09-05T06:46:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:46:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:25 smithi096 ceph-mon[34468]: pgmap v158: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:46:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:46:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:25 smithi115 ceph-mon[34791]: pgmap v158: 129 pgs: 129 active+clean; 89 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-05T06:46:26.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 90546 1.80 MB/sec execute 106 sec latency 19.850 ms 2024-09-05T06:46:26.671 INFO:tasks.cephfs.filesystem:scrub status for tag:a479d661-402a-4e41-9566-25d3ba41715a - {'path': '/', 'tag': 'a479d661-402a-4e41-9566-25d3ba41715a', 'options': 'recursive,force'} 2024-09-05T06:46:27.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 91159 1.80 MB/sec execute 107 sec latency 19.381 ms 2024-09-05T06:46:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:27 smithi096 ceph-mon[34468]: pgmap v159: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T06:46:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:27 smithi115 ceph-mon[34791]: pgmap v159: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T06:46:27.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:27 smithi086 ceph-mon[30943]: pgmap v159: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-09-05T06:46:28.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92002 1.85 MB/sec execute 108 sec latency 13.699 ms 2024-09-05T06:46:29.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92469 1.88 MB/sec execute 109 sec latency 13.921 ms 2024-09-05T06:46:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:29 smithi096 ceph-mon[34468]: pgmap v160: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:29 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:46:29.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:29 smithi115 ceph-mon[34791]: pgmap v160: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:29.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:29 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:46:29.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:29 smithi086 ceph-mon[30943]: pgmap v160: 129 pgs: 129 active+clean; 91 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:29 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:46:30.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92736 1.86 MB/sec execute 110 sec latency 13.080 ms 2024-09-05T06:46:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:30 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:46:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:30 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:46:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:30 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:46:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:30 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:46:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:30 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:46:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:30 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:46:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:30 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:46:30.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:30 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:46:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:30 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:46:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:30 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:46:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:30 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:46:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:30 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:46:31.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 93208 1.85 MB/sec execute 111 sec latency 20.960 ms 2024-09-05T06:46:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:46:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:46:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:31 smithi096 ceph-mon[34468]: pgmap v161: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:46:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:46:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:46:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:31 smithi115 ceph-mon[34791]: pgmap v161: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:46:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:46:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:46:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:31 smithi086 ceph-mon[30943]: pgmap v161: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-09-05T06:46:32.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 93788 1.87 MB/sec execute 112 sec latency 16.381 ms 2024-09-05T06:46:33.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 94408 1.86 MB/sec execute 113 sec latency 18.565 ms 2024-09-05T06:46:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:33 smithi096 ceph-mon[34468]: pgmap v162: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:33 smithi115 ceph-mon[34791]: pgmap v162: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:33.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:33 smithi086 ceph-mon[30943]: pgmap v162: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:34.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 94987 1.89 MB/sec execute 114 sec latency 58.929 ms 2024-09-05T06:46:35.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 95626 1.91 MB/sec execute 115 sec latency 14.975 ms 2024-09-05T06:46:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:35 smithi096 ceph-mon[34468]: pgmap v163: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:35 smithi115 ceph-mon[34791]: pgmap v163: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:35.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:35 smithi086 ceph-mon[30943]: pgmap v163: 129 pgs: 129 active+clean; 96 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:46:36.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96134 1.93 MB/sec execute 116 sec latency 11.688 ms 2024-09-05T06:46:37.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96437 1.91 MB/sec execute 117 sec latency 16.413 ms 2024-09-05T06:46:37.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:37 smithi086 ceph-mon[30943]: pgmap v164: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-05T06:46:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:37 smithi096 ceph-mon[34468]: pgmap v164: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-05T06:46:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:37 smithi115 ceph-mon[34791]: pgmap v164: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-05T06:46:38.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96996 1.91 MB/sec execute 118 sec latency 16.162 ms 2024-09-05T06:46:39.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 97529 1.93 MB/sec execute 119 sec latency 20.278 ms 2024-09-05T06:46:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:39 smithi096 ceph-mon[34468]: pgmap v165: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-05T06:46:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:39 smithi115 ceph-mon[34791]: pgmap v165: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-05T06:46:39.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:39 smithi086 ceph-mon[30943]: pgmap v165: 129 pgs: 129 active+clean; 98 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-05T06:46:40.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 98184 1.92 MB/sec execute 120 sec latency 22.240 ms 2024-09-05T06:46:41.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 98840 1.95 MB/sec execute 121 sec latency 13.609 ms 2024-09-05T06:46:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:41 smithi096 ceph-mon[34468]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-09-05T06:46:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:41 smithi115 ceph-mon[34791]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-09-05T06:46:41.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:41 smithi086 ceph-mon[30943]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2024-09-05T06:46:42.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 99457 1.98 MB/sec execute 122 sec latency 13.321 ms 2024-09-05T06:46:43.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 99795 1.98 MB/sec execute 123 sec latency 13.537 ms 2024-09-05T06:46:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:43 smithi096 ceph-mon[34468]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:43 smithi115 ceph-mon[34791]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:43.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:43 smithi086 ceph-mon[30943]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:44.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100225 1.96 MB/sec execute 124 sec latency 14.081 ms 2024-09-05T06:46:45.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100796 1.98 MB/sec execute 125 sec latency 19.150 ms 2024-09-05T06:46:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:45 smithi096 ceph-mon[34468]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:45 smithi115 ceph-mon[34791]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:45.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:45 smithi086 ceph-mon[30943]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-09-05T06:46:46.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 101338 1.97 MB/sec execute 126 sec latency 20.282 ms 2024-09-05T06:46:47.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 101919 1.98 MB/sec execute 127 sec latency 18.007 ms 2024-09-05T06:46:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:47 smithi096 ceph-mon[34468]: pgmap v169: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T06:46:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:47 smithi115 ceph-mon[34791]: pgmap v169: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T06:46:47.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:47 smithi086 ceph-mon[30943]: pgmap v169: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T06:46:48.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 102668 2.01 MB/sec execute 128 sec latency 13.646 ms 2024-09-05T06:46:49.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103149 2.03 MB/sec execute 129 sec latency 13.448 ms 2024-09-05T06:46:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:49 smithi096 ceph-mon[34468]: pgmap v170: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:49 smithi115 ceph-mon[34791]: pgmap v170: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:49.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:49 smithi086 ceph-mon[30943]: pgmap v170: 129 pgs: 129 active+clean; 101 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T06:46:50.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103443 2.02 MB/sec execute 130 sec latency 36.141 ms 2024-09-05T06:46:51.514 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103952 2.02 MB/sec execute 131 sec latency 16.590 ms 2024-09-05T06:46:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:51 smithi096 ceph-mon[34468]: pgmap v171: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:46:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:51 smithi115 ceph-mon[34791]: pgmap v171: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:46:51.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:51 smithi086 ceph-mon[30943]: pgmap v171: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-09-05T06:46:52.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 104533 2.03 MB/sec execute 132 sec latency 16.469 ms 2024-09-05T06:46:53.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 105218 2.02 MB/sec execute 133 sec latency 18.787 ms 2024-09-05T06:46:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:53 smithi096 ceph-mon[34468]: pgmap v172: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:53 smithi115 ceph-mon[34791]: pgmap v172: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:53 smithi086 ceph-mon[30943]: pgmap v172: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:54.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 105881 2.05 MB/sec execute 134 sec latency 13.648 ms 2024-09-05T06:46:55.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 106474 2.07 MB/sec execute 135 sec latency 13.406 ms 2024-09-05T06:46:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:55 smithi096 ceph-mon[34468]: pgmap v173: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:55.854 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.3 2024-09-05T06:46:55.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:46:55.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:55 smithi086 ceph-mon[30943]: pgmap v173: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:55 smithi115 ceph-mon[34791]: pgmap v173: 129 pgs: 129 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-05T06:46:56.126 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:56.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 106843 2.07 MB/sec execute 136 sec latency 65.928 ms 2024-09-05T06:46:56.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 -- 172.21.15.86:0/3387552488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 msgr2=0x7fa9701056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 --2- 172.21.15.86:0/3387552488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa9701056b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa958009900 tx=0x7fa95802a410 comp rx=0 tx=0).stop 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 -- 172.21.15.86:0/3387552488 shutdown_connections 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 --2- 172.21.15.86:0/3387552488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970105de0 0x7fa97010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 --2- 172.21.15.86:0/3387552488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa9701056b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 --2- 172.21.15.86:0/3387552488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970101470 0x7fa970101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 -- 172.21.15.86:0/3387552488 >> 172.21.15.86:0/3387552488 conn(0x7fa970077230 msgr2=0x7fa9700fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 -- 172.21.15.86:0/3387552488 shutdown_connections 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.653+0000 7fa975958640 1 -- 172.21.15.86:0/3387552488 wait complete. 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 Processor -- start 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 -- start start 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970101470 0x7fa970198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970105de0 0x7fa97019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa970110cf0 con 0x7fa970101e20 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa970110b70 con 0x7fa970101470 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa975958640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa970110e70 con 0x7fa970105de0 2024-09-05T06:46:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51302/0 (socket says 172.21.15.86:51302) 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 -- 172.21.15.86:0/3943140511 learned_addr learned my addr 172.21.15.86:0/3943140511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa96effd640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970101470 0x7fa970198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa96f7fe640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970105de0 0x7fa97019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 -- 172.21.15.86:0/3943140511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970105de0 msgr2=0x7fa97019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970105de0 0x7fa97019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 -- 172.21.15.86:0/3943140511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970101470 msgr2=0x7fa970198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970101470 0x7fa970198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.654+0000 7fa967fff640 1 -- 172.21.15.86:0/3943140511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97019d2f0 con 0x7fa970101e20 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa967fff640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa958005940 tx=0x7fa9580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95803f030 con 0x7fa970101e20 2024-09-05T06:46:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa958003cb0 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97019d580 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958035070 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.655+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9701a4ec0 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.656+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa970101850 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.658+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa958042030 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.659+0000 7fa974956640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.659+0000 7fa96effd640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.659+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fa9580f01f0 con 0x7fa970101e20 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.660+0000 7fa96effd640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa960009bd0 tx=0x7fa960005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:56.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.662+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9580bc3e0 con 0x7fa970101e20 2024-09-05T06:46:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.909+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa97010a030 con 0x7fa970101e20 2024-09-05T06:46:56.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.910+0000 7fa974956640 1 -- 172.21.15.86:0/3943140511 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7fa9580c06e0 con 0x7fa970101e20 2024-09-05T06:46:56.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:46:56.911 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:46:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.913+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 msgr2=0x7fa95c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.913+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa960009bd0 tx=0x7fa960005f90 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.913+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 msgr2=0x7fa970198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.913+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa958005940 tx=0x7fa9580042f0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 shutdown_connections 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa95c0a2cf0 0x7fa95c0a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa970105de0 0x7fa97019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa970101e20 0x7fa970198880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 --2- 172.21.15.86:0/3943140511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa970101470 0x7fa970198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 >> 172.21.15.86:0/3943140511 conn(0x7fa970077230 msgr2=0x7fa970100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 shutdown_connections 2024-09-05T06:46:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:56.914+0000 7fa975958640 1 -- 172.21.15.86:0/3943140511 wait complete. 2024-09-05T06:46:57.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:56 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3943140511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:57.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:56 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3943140511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:57.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:56 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3943140511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:46:57.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107177 2.05 MB/sec execute 137 sec latency 18.368 ms 2024-09-05T06:46:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:57.829 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0' 2024-09-05T06:46:57.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0 2024-09-05T06:46:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:57 smithi086 ceph-mon[30943]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-05T06:46:58.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:57 smithi096 ceph-mon[34468]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 -- 172.21.15.86:0/3602411345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100aa0 msgr2=0x7f606c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 --2- 172.21.15.86:0/3602411345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100aa0 0x7f606c10ea20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6064009ca0 tx=0x7f606402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 -- 172.21.15.86:0/3602411345 shutdown_connections 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 --2- 172.21.15.86:0/3602411345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100aa0 0x7f606c10ea20 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 --2- 172.21.15.86:0/3602411345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c100100 0x7f606c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 --2- 172.21.15.86:0/3602411345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c106b80 0x7f606c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 -- 172.21.15.86:0/3602411345 >> 172.21.15.86:0/3602411345 conn(0x7f606c077230 msgr2=0x7f606c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 -- 172.21.15.86:0/3602411345 shutdown_connections 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.316+0000 7f6073a21640 1 -- 172.21.15.86:0/3602411345 wait complete. 2024-09-05T06:46:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.317+0000 7f6073a21640 1 Processor -- start 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.317+0000 7f6073a21640 1 -- start start 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.317+0000 7f6073a21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.317+0000 7f6073a21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c100aa0 0x7f606c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c106b80 0x7f606c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f606c111f00 con 0x7f606c100100 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f606c111d80 con 0x7f606c106b80 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f606c112080 con 0x7f606c100aa0 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51312/0 (socket says 172.21.15.86:51312) 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 -- 172.21.15.86:0/3663842164 learned_addr learned my addr 172.21.15.86:0/3663842164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6070fbc640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c100aa0 0x7f606c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c106b80 0x7f606c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 -- 172.21.15.86:0/3663842164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c100aa0 msgr2=0x7f606c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c100aa0 0x7f606c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 -- 172.21.15.86:0/3663842164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c106b80 msgr2=0x7f606c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c106b80 0x7f606c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 -- 172.21.15.86:0/3663842164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c19d330 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60717bd640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f606800d770 tx=0x7f606800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60627fc640 1 -- 172.21.15.86:0/3663842164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068017070 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60627fc640 1 -- 172.21.15.86:0/3663842164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60680054d0 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c19d620 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f60627fc640 1 -- 172.21.15.86:0/3663842164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068004cf0 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.318+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c1a4f00 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.321+0000 7f60627fc640 1 -- 172.21.15.86:0/3663842164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6068004e90 con 0x7f606c100100 2024-09-05T06:46:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 Processor -- start 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 -- start start 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375f1680 0x55e6375f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e63754c2e0 con 0x55e6375f1680 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6374f3ce0 con 0x55e6375e64a0 2024-09-05T06:46:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6075dfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e6375482a0 con 0x55e6375e6dc0 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f60717bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38064/0 (socket says 172.21.15.86:38064) 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 -- 172.21.15.86:0/1077506765 learned_addr learned my addr 172.21.15.86:0/1077506765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6070fbc640 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375f1680 0x55e6375f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f60627fc640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 0x7f60580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 -- 172.21.15.86:0/1077506765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 msgr2=0x55e6375f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 -- 172.21.15.86:0/1077506765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375f1680 msgr2=0x55e6375f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375f1680 0x55e6375f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.322+0000 7f6071fbe640 1 -- 172.21.15.86:0/1077506765 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6374d8440 con 0x55e6375e64a0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f60717bd640 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 0x7f60580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f60627fc640 1 -- 172.21.15.86:0/3663842164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f60680cf1f0 con 0x7f606c100100 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f6071fbe640 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6064009ca0 tx=0x7f6064031a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f803c0f99d2ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 0x7f60580a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6064004af0 tx=0x7f606404e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f60577fe640 1 -- 172.21.15.86:0/1077506765 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064042040 con 0x55e6375e64a0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f60577fe640 1 -- 172.21.15.86:0/1077506765 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6064036070 con 0x55e6375e64a0 2024-09-05T06:46:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.323+0000 7f60577fe640 1 -- 172.21.15.86:0/1077506765 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064030820 con 0x55e6375e64a0 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 -- 172.21.15.86:0/1077506765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 msgr2=0x55e6375e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6064009ca0 tx=0x7f6064031a40 comp rx=0 tx=0).stop 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 -- 172.21.15.86:0/1077506765 shutdown_connections 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375f1680 0x55e6375f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 --2- 172.21.15.86:0/1077506765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375e6880 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.324+0000 7f6075dfa740 1 -- 172.21.15.86:0/1077506765 >> 172.21.15.86:0/1077506765 conn(0x55e6375e2a30 msgr2=0x55e6375e2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.325+0000 7f6075dfa740 1 -- 172.21.15.86:0/1077506765 shutdown_connections 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.325+0000 7f6075dfa740 1 -- 172.21.15.86:0/1077506765 wait complete. 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.325+0000 7f6075dfa740 1 -- start start 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.325+0000 7f6075dfa740 1 Processor -- start 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e6375ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6375f4250 con 0x55e6375e64a0 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e637507050 con 0x55e6375f1680 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e6375f43d0 con 0x55e6375e6dc0 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6075dfa740 10 client.0 _ll_register_callbacks cb 0x55e637547160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f60717bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f60717bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51336/0 (socket says 172.21.15.86:51336) 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f60717bd640 1 -- 172.21.15.86:0/3916886281 learned_addr learned my addr 172.21.15.86:0/3916886281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e6375ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6070fbc640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 -- 172.21.15.86:0/3916886281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 msgr2=0x55e6375ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 -- 172.21.15.86:0/3916886281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 msgr2=0x55e6375f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 -- 172.21.15.86:0/3916886281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6372bd3c0 con 0x55e6375f1680 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6070fbc640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f60717bd640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6071fbe640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e6375ee8e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6064005e30 tx=0x7f6064004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a161670d7e0b943 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.326+0000 7f6054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606405f060 con 0x55e6375f1680 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6064004250 con 0x55e6375f1680 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6064036070 con 0x55e6375f1680 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6064066050 con 0x55e6375f1680 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6075dfa740 10 client.26051 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6375f4250 con 0x55e6375f1680 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6064036070 con 0x55e6375f1680 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.327+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6064030cd0 con 0x55e6375f1680 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 10 client.26051 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 20 client.26051 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637600df0 0x55e6376031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 4 client.26051 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e637559850 con 0x55e637600df0 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f60717bd640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637600df0 0x55e6376031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f6054ff9640 10 client.26051 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.328+0000 7f60717bd640 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637600df0 0x55e6376031e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17cf8a3f15bc8d49 server_cookie=efe8e2b6055872d5 in_seq=0 out_seq=0 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6054ff9640 1 -- 172.21.15.86:0/3916886281 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e637559850 con 0x55e637600df0 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6054ff9640 10 client.26051 handle_command_reply: tid=0 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 client.26051 shutdown 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 msgr2=0x55e6375ee8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e6375ee8e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6064005e30 tx=0x7f6064004a60 comp rx=0 tx=0).stop 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 shutdown_connections 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637600df0 0x55e6376031e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e6375ee8e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e6dc0 0x55e6375ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 --2- 172.21.15.86:0/3916886281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e64a0 0x55e6375f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 >> 172.21.15.86:0/3916886281 conn(0x55e6375e2a30 msgr2=0x55e6375e8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6054ff9640 0 client.26051 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 shutdown_connections 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.329+0000 7f6075dfa740 1 -- 172.21.15.86:0/3916886281 wait complete. 2024-09-05T06:46:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.330+0000 7f6075dfa740 20 client.26051 trim_cache size 0 max 16384 2024-09-05T06:46:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 Processor -- start 2024-09-05T06:46:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 -- start start 2024-09-05T06:46:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e64a0 0x55e6375fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e637608a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e637608f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1aeed0 con 0x55e6375e6dc0 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1b0a10 con 0x55e6375f1680 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6070fbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e637608a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6075dfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1af5a0 con 0x55e6375e64a0 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f60717bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e64a0 0x55e6375fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6070fbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e637608a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51346/0 (socket says 172.21.15.86:51346) 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6070fbc640 1 -- 172.21.15.86:0/2339671602 learned_addr learned my addr 172.21.15.86:0/2339671602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.350+0000 7f6071fbe640 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e637608f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 -- 172.21.15.86:0/2339671602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e64a0 msgr2=0x55e6375fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e64a0 0x55e6375fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 -- 172.21.15.86:0/2339671602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 msgr2=0x55e637608a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e637608a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 -- 172.21.15.86:0/2339671602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6375e3ae0 con 0x55e6375f1680 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6071fbe640 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e637608f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f606402ade0 tx=0x7f606402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d8d5508fd8410b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6056ffd640 1 -- 172.21.15.86:0/2339671602 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606405f040 con 0x55e6375f1680 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6056ffd640 1 -- 172.21.15.86:0/2339671602 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6064036030 con 0x55e6375f1680 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.351+0000 7f6056ffd640 1 -- 172.21.15.86:0/2339671602 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064042050 con 0x55e6375f1680 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- 172.21.15.86:0/2339671602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 msgr2=0x55e637608f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e637608f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f606402ade0 tx=0x7f606402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- 172.21.15.86:0/2339671602 shutdown_connections 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375f1680 0x55e637608f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e637608a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2339671602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375e64a0 0x55e6375fd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- 172.21.15.86:0/2339671602 >> 172.21.15.86:0/2339671602 conn(0x55e6375e2a30 msgr2=0x55e6375ed3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- 172.21.15.86:0/2339671602 shutdown_connections 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- 172.21.15.86:0/2339671602 wait complete. 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 -- start start 2024-09-05T06:46:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.352+0000 7f6075dfa740 1 Processor -- start 2024-09-05T06:46:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e6375f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375f1680 0x55e6375f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1b0340 con 0x55e6375e6dc0 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1afc70 con 0x55e6375e64a0 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1aceb0 con 0x55e6375f1680 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38094/0 (socket says 172.21.15.86:38094) 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6070fbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e6375f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 -- 172.21.15.86:0/2943217175 learned_addr learned my addr 172.21.15.86:0/2943217175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f6075dfa740 10 client.0 _ll_register_callbacks cb 0x7f605c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 -- 172.21.15.86:0/2943217175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375f1680 msgr2=0x55e6375f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375f1680 0x55e6375f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.353+0000 7f60717bd640 1 -- 172.21.15.86:0/2943217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 msgr2=0x55e6375f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f60717bd640 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e6375f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f60717bd640 1 -- 172.21.15.86:0/2943217175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6375e35e0 con 0x55e6375e64a0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f60717bd640 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f60680ccb60 tx=0x7f606800de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6670ab9f460e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068020040 con 0x55e6375e64a0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f6068097a50 con 0x55e6375e64a0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6064036030 con 0x55e6375e64a0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6068009c30 con 0x55e6375e64a0 2024-09-05T06:46:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6075dfa740 10 client.26057 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.354+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e63766fa00 con 0x55e6375e64a0 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6064036030 con 0x55e6375e64a0 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f60680bb140 con 0x55e6375e64a0 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 10 client.26057 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 20 client.26057 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637670100 0x55e637672560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 4 client.26057 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e637553d60 con 0x55e637670100 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.355+0000 7f6070fbc640 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637670100 0x55e637672560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.356+0000 7f6054ff9640 10 client.26057 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.356+0000 7f6070fbc640 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637670100 0x55e637672560 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b63c6fdf9c22ca server_cookie=383624a774690be0 in_seq=0 out_seq=0 2024-09-05T06:46:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6054ff9640 1 -- 172.21.15.86:0/2943217175 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e637553d60 con 0x55e637670100 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6054ff9640 10 client.26057 handle_command_reply: tid=0 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 client.26057 shutdown 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 msgr2=0x55e6375f9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f60680ccb60 tx=0x7f606800de80 comp rx=0 tx=0).stop 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 shutdown_connections 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e637670100 0x55e637672560 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6375f1680 0x55e6375f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6375e6dc0 0x55e6375f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 --2- 172.21.15.86:0/2943217175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6375e64a0 0x55e6375f9aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6054ff9640 0 client.26057 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 >> 172.21.15.86:0/2943217175 conn(0x55e6375e2a30 msgr2=0x55e6375efb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 shutdown_connections 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 1 -- 172.21.15.86:0/2943217175 wait complete. 2024-09-05T06:46:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.357+0000 7f6075dfa740 20 client.26057 trim_cache size 0 max 16384 2024-09-05T06:46:58.368 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:46:58.368 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:46:58.368 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:46:58.368 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:46:58.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 msgr2=0x7f60580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 0x7f60580a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6064004af0 tx=0x7f606404e000 comp rx=0 tx=0).stop 2024-09-05T06:46:58.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 msgr2=0x7f606c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f606800d770 tx=0x7f606800dc30 comp rx=0 tx=0).stop 2024-09-05T06:46:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 shutdown_connections 2024-09-05T06:46:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60580a2cf0 0x7f60580a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f606c106b80 0x7f606c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c100aa0 0x7f606c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 --2- 172.21.15.86:0/3663842164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c100100 0x7f606c198380 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 >> 172.21.15.86:0/3663842164 conn(0x7f606c077230 msgr2=0x7f606c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 shutdown_connections 2024-09-05T06:46:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:58.358+0000 7f6073a21640 1 -- 172.21.15.86:0/3663842164 wait complete. 2024-09-05T06:46:58.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:57 smithi115 ceph-mon[34791]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-09-05T06:46:58.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107701 2.05 MB/sec execute 138 sec latency 19.596 ms 2024-09-05T06:46:58.607 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:59.223 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:46:59.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:46:59.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108276 2.06 MB/sec execute 139 sec latency 18.346 ms 2024-09-05T06:46:59.747 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:46:59 smithi086 ceph-mon[30943]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.737+0000 7ff1304b7640 1 -- 172.21.15.86:0/1945172005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 msgr2=0x7ff12810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.737+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1945172005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12810eaa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff120009ca0 tx=0x7ff12002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- 172.21.15.86:0/1945172005 shutdown_connections 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1945172005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12810eaa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1945172005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1281027a0 0x7ff1281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1945172005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128101df0 0x7ff1281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- 172.21.15.86:0/1945172005 >> 172.21.15.86:0/1945172005 conn(0x7ff1280fd8f0 msgr2=0x7ff1280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- 172.21.15.86:0/1945172005 shutdown_connections 2024-09-05T06:46:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- 172.21.15.86:0/1945172005 wait complete. 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 Processor -- start 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- start start 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 0x7ff128198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111e30 con 0x7ff128106f20 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111cb0 con 0x7ff1281027a0 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.738+0000 7ff1304b7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111fb0 con 0x7ff128101df0 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12ea54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12e253640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 0x7ff128198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12ea54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51368/0 (socket says 172.21.15.86:51368) 2024-09-05T06:46:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12e253640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12ea54640 1 -- 172.21.15.86:0/1588954503 learned_addr learned my addr 172.21.15.86:0/1588954503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 -- 172.21.15.86:0/1588954503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 msgr2=0x7ff128198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 -- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 msgr2=0x7ff12819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 -- 172.21.15.86:0/1588954503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12819d3e0 con 0x7ff1281027a0 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12ea54640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12e253640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff12da52640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 0x7ff128198970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff11800e9f0 tx=0x7ff11800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff11f7fe640 1 -- 172.21.15.86:0/1588954503 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11800ccc0 con 0x7ff1281027a0 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12819d6d0 con 0x7ff1281027a0 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff11f7fe640 1 -- 172.21.15.86:0/1588954503 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff118005020 con 0x7ff1281027a0 2024-09-05T06:46:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff11f7fe640 1 -- 172.21.15.86:0/1588954503 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118004a00 con 0x7ff1281027a0 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.739+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281a4fb0 con 0x7ff1281027a0 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.742+0000 7ff11f7fe640 1 -- 172.21.15.86:0/1588954503 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff118004ba0 con 0x7ff1281027a0 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.742+0000 7ff132890740 1 Processor -- start 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.742+0000 7ff132890740 1 -- start start 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464301a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c464313140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4642672a0 con 0x55c464301660 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c46420eb10 con 0x55c464310d50 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff132890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c464263260 con 0x55c464301f80 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff11f7fe640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 0x7ff1140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12ea54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464301a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12ea54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464301a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51374/0 (socket says 172.21.15.86:51374) 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12ea54640 1 -- 172.21.15.86:0/3649791680 learned_addr learned my addr 172.21.15.86:0/3649791680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52088/0 (socket says 172.21.15.86:52088) 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12da52640 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c464313140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12ea54640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 0x7ff1140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 -- 172.21.15.86:0/3649791680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 msgr2=0x55c464313140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c464313140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 -- 172.21.15.86:0/3649791680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 msgr2=0x55c464301a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464301a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff12e253640 1 -- 172.21.15.86:0/3649791680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c463fd8b90 con 0x55c464301f80 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.743+0000 7ff11f7fe640 1 -- 172.21.15.86:0/1588954503 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ff1180cf780 con 0x7ff1281027a0 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.744+0000 7ff12e253640 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff1240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7818557baac8ed47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.744+0000 7ff11cff9640 1 -- 172.21.15.86:0/3649791680 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124004030 con 0x55c464301f80 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.744+0000 7ff11cff9640 1 -- 172.21.15.86:0/3649791680 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1240054d0 con 0x55c464301f80 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.744+0000 7ff12ea54640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 0x7ff1140a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1200319e0 tx=0x7ff12002a8c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.744+0000 7ff11cff9640 1 -- 172.21.15.86:0/3649791680 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124004c40 con 0x55c464301f80 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- 172.21.15.86:0/3649791680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 msgr2=0x55c464310810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff1240118d0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- 172.21.15.86:0/3649791680 shutdown_connections 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c464313140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c464310810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 --2- 172.21.15.86:0/3649791680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464301a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- 172.21.15.86:0/3649791680 >> 172.21.15.86:0/3649791680 conn(0x55c4642fdcf0 msgr2=0x55c4642fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- 172.21.15.86:0/3649791680 shutdown_connections 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- 172.21.15.86:0/3649791680 wait complete. 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.745+0000 7ff132890740 1 -- start start 2024-09-05T06:46:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 Processor -- start 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 0x55c46430dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c46430e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c464313980 con 0x55c464301f80 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c4641ef6e0 con 0x55c464310d50 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c464313b00 con 0x55c464301660 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff132890740 10 client.0 _ll_register_callbacks cb 0x55c464213a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51376/0 (socket says 172.21.15.86:51376) 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12ea54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c46430e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 -- 172.21.15.86:0/3485538493 learned_addr learned my addr 172.21.15.86:0/3485538493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12e253640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 0x55c46430dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 -- 172.21.15.86:0/3485538493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 msgr2=0x55c46430dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 0x55c46430dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 -- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 msgr2=0x55c46430e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c46430e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12da52640 1 -- 172.21.15.86:0/3485538493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c46431cee0 con 0x55c464301f80 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.746+0000 7ff12e253640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 0x55c46430dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff12da52640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff11800e9c0 tx=0x7ff11809b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ecdb2dac9ff8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11801c040 con 0x55c464301f80 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ff1180860f0 con 0x55c464301f80 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1240054d0 con 0x55c464301f80 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1180b9050 con 0x55c464301f80 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff132890740 10 client.16758 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.747+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4643043a0 con 0x55c464301f80 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1240054d0 con 0x55c464301f80 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff1180ccd10 con 0x55c464301f80 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 10 client.16758 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 10 client.16758 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 20 client.16758 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464305860 0x55c464307d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 4 client.16758 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4642749d0 con 0x55c464305860 2024-09-05T06:46:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.748+0000 7ff12e253640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464305860 0x55c464307d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.749+0000 7ff12e253640 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464305860 0x55c464307d20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1344ed83d93db1 server_cookie=95730d9f133cb1cd in_seq=0 out_seq=0 2024-09-05T06:46:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff111ffb640 10 client.16758 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff111ffb640 1 -- 172.21.15.86:0/3485538493 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4642749d0 con 0x55c464305860 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff111ffb640 10 client.16758 handle_command_reply: tid=0 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff132890740 1 client.16758 shutdown 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 msgr2=0x55c46430e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.750+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff11800e9c0 tx=0x7ff11809b280 comp rx=0 tx=0).stop 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 shutdown_connections 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464305860 0x55c464307d20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464310d50 0x55c46430e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301f80 0x55c46430e1a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 --2- 172.21.15.86:0/3485538493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301660 0x55c46430dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 >> 172.21.15.86:0/3485538493 conn(0x55c4642fdcf0 msgr2=0x55c464303a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff111ffb640 0 client.16758 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 shutdown_connections 2024-09-05T06:46:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 1 -- 172.21.15.86:0/3485538493 wait complete. 2024-09-05T06:46:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.751+0000 7ff132890740 20 client.16758 trim_cache size 0 max 16384 2024-09-05T06:46:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.769+0000 7ff132890740 1 Processor -- start 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.769+0000 7ff132890740 1 -- start start 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.769+0000 7ff132890740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 0x55c46430c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.769+0000 7ff132890740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c46430aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff132890740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 0x55c46430b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff132890740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281acfe0 con 0x55c464310d50 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff132890740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281aeb20 con 0x55c464301660 2024-09-05T06:46:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff132890740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281ad6b0 con 0x55c464301f80 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12da52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c46430aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12da52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c46430aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52102/0 (socket says 172.21.15.86:52102) 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12da52640 1 -- 172.21.15.86:0/8663555 learned_addr learned my addr 172.21.15.86:0/8663555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 0x55c46430b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12e253640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 0x55c46430c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 -- 172.21.15.86:0/8663555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 msgr2=0x55c46430aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c46430aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 -- 172.21.15.86:0/8663555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 msgr2=0x55c46430c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 0x55c46430c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12ea54640 1 -- 172.21.15.86:0/8663555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c464307f50 con 0x55c464310d50 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.770+0000 7ff12e253640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 0x55c46430c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.771+0000 7ff12ea54640 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 0x55c46430b400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff120009dc0 tx=0x7ff120045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2504d0a02a51693 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.771+0000 7ff113fff640 1 -- 172.21.15.86:0/8663555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120060070 con 0x55c464310d50 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.771+0000 7ff113fff640 1 -- 172.21.15.86:0/8663555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff120036050 con 0x55c464310d50 2024-09-05T06:46:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.771+0000 7ff113fff640 1 -- 172.21.15.86:0/8663555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120030940 con 0x55c464310d50 2024-09-05T06:46:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- 172.21.15.86:0/8663555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 msgr2=0x55c46430b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 0x55c46430b400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff120009dc0 tx=0x7ff120045000 comp rx=0 tx=0).stop 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- 172.21.15.86:0/8663555 shutdown_connections 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464310d50 0x55c46430b400 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464301f80 0x55c46430aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 --2- 172.21.15.86:0/8663555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301660 0x55c46430c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- 172.21.15.86:0/8663555 >> 172.21.15.86:0/8663555 conn(0x55c4642fdcf0 msgr2=0x55c46430fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- 172.21.15.86:0/8663555 shutdown_connections 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- 172.21.15.86:0/8663555 wait complete. 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 -- start start 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 Processor -- start 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.772+0000 7ff132890740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51396/0 (socket says 172.21.15.86:51396) 2024-09-05T06:46:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 -- 172.21.15.86:0/1002190881 learned_addr learned my addr 172.21.15.86:0/1002190881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301f80 0x55c4643223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464310d50 0x55c464322940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281ae450 con 0x55c464301660 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281add80 con 0x55c464301f80 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281aafc0 con 0x55c464310d50 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff132890740 10 client.0 _ll_register_callbacks cb 0x7ff11809b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12da52640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301f80 0x55c4643223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12ea54640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464310d50 0x55c464322940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 -- 172.21.15.86:0/1002190881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464310d50 msgr2=0x55c464322940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464310d50 0x55c464322940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 -- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301f80 msgr2=0x55c4643223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.773+0000 7ff12e253640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301f80 0x55c4643223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff12e253640 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c46430cc10 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff12e253640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff124009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54817a742f186075 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12400fc00 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ff12400fda0 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff120036050 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1240385f0 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff132890740 10 client.16764 fetch_fsmap learned FSMap version 32 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.774+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c464387730 con 0x55c464301660 2024-09-05T06:46:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff120036050 con 0x55c464301660 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff124019450 con 0x55c464301660 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 10 client.16764 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 10 client.16764 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 20 client.16764 populate_metadata read hostname 'smithi086' 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464387c30 0x55c46438a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 4 client.16764 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}] 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}) -- 0x55c46426ed20 con 0x55c464387c30 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.775+0000 7ff12da52640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464387c30 0x55c46438a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.776+0000 7ff111ffb640 10 client.16764 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.776+0000 7ff12da52640 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464387c30 0x55c46438a020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbfe6ea0ca695b5 server_cookie=9bc5fb41d6cfd268 in_seq=0 out_seq=0 2024-09-05T06:46:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:46:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:46:59 smithi096 ceph-mon[34468]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff111ffb640 1 -- 172.21.15.86:0/1002190881 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c46426ed20 con 0x55c464387c30 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff111ffb640 10 client.16764 handle_command_reply: tid=0 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 client.16764 shutdown 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 msgr2=0x55c464325070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff124009b50 comp rx=0 tx=0).stop 2024-09-05T06:46:59.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 shutdown_connections 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c464387c30 0x55c46438a020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c464310d50 0x55c464322940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c464301f80 0x55c4643223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff111ffb640 0 client.16764 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 --2- 172.21.15.86:0/1002190881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c464301660 0x55c464325070 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.836+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 >> 172.21.15.86:0/1002190881 conn(0x55c4642fdcf0 msgr2=0x55c4643039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 shutdown_connections 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff132890740 1 -- 172.21.15.86:0/1002190881 wait complete. 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff132890740 20 client.16764 trim_cache size 0 max 16384 2024-09-05T06:46:59.838 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 18, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "5117007c": { 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 18, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:46:59.839 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 154.3, 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:46:59.840 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:46:59.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 msgr2=0x7ff1140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 0x7ff1140a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1200319e0 tx=0x7ff12002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 msgr2=0x7ff128198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 0x7ff128198970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff11800e9f0 tx=0x7ff11800eeb0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 shutdown_connections 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1140a2cf0 0x7ff1140a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff128106f20 0x7ff12819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1281027a0 0x7ff128198970 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 --2- 172.21.15.86:0/1588954503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff128101df0 0x7ff128198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 >> 172.21.15.86:0/1588954503 conn(0x7ff1280fd8f0 msgr2=0x7ff1280fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 shutdown_connections 2024-09-05T06:46:59.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:46:59.837+0000 7ff1304b7640 1 -- 172.21.15.86:0/1588954503 wait complete. 2024-09-05T06:46:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:46:59 smithi115 ceph-mon[34791]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-09-05T06:47:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- 172.21.15.86:0/297679381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388100a60 msgr2=0x7fc388100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- 172.21.15.86:0/297679381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388100a60 0x7fc388100ec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc384009930 tx=0x7fc38402a420 comp rx=0 tx=0).stop 2024-09-05T06:47:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- 172.21.15.86:0/297679381 shutdown_connections 2024-09-05T06:47:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- 172.21.15.86:0/297679381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388101400 0x7fc38810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- 172.21.15.86:0/297679381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388100a60 0x7fc388100ec0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- 172.21.15.86:0/297679381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 0x7fc388106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- 172.21.15.86:0/297679381 >> 172.21.15.86:0/297679381 conn(0x7fc3880fc780 msgr2=0x7fc3880feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- 172.21.15.86:0/297679381 shutdown_connections 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- 172.21.15.86:0/297679381 wait complete. 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 Processor -- start 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- start start 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388100a60 0x7fc3881983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 0x7fc38819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111e30 con 0x7fc388101400 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111cb0 con 0x7fc388106a60 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.466+0000 7fc3901b0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111fb0 con 0x7fc388100a60 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51410/0 (socket says 172.21.15.86:51410) 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 -- 172.21.15.86:0/1794690169 learned_addr learned my addr 172.21.15.86:0/1794690169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38df4c640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388100a60 0x7fc3881983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38e74d640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 0x7fc38819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 -- 172.21.15.86:0/1794690169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388100a60 msgr2=0x7fc3881983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388100a60 0x7fc3881983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 -- 172.21.15.86:0/1794690169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 msgr2=0x7fc38819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 0x7fc38819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 -- 172.21.15.86:0/1794690169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38819d360 con 0x7fc388101400 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc38d74b640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc384005940 tx=0x7fc3840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38403f030 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3840318a0 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38819d5f0 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384035070 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.467+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881a4f30 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.468+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388100ec0 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.470+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc384042030 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.471+0000 7fc37affd640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 0x7fc3740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.471+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fc3840f0130 con 0x7fc388101400 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.471+0000 7fc38df4c640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 0x7fc3740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.471+0000 7fc38df4c640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 0x7fc3740a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009b50 tx=0x7fc37c005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:00.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.474+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3840bc320 con 0x7fc388101400 2024-09-05T06:47:00.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 2.05 MB/sec execute 140 sec latency 651.683 ms 2024-09-05T06:47:00.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.679+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc388109030 con 0x7fc388101400 2024-09-05T06:47:00.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.680+0000 7fc37affd640 1 -- 172.21.15.86:0/1794690169 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7fc3840c0620 con 0x7fc388101400 2024-09-05T06:47:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:47:00.680 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:47:00.680 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 msgr2=0x7fc3740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 0x7fc3740a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009b50 tx=0x7fc37c005f90 comp rx=0 tx=0).stop 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 msgr2=0x7fc3881988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc384005940 tx=0x7fc3840042f0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 shutdown_connections 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3740a2d70 0x7fc3740a5230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc388106a60 0x7fc38819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388101400 0x7fc3881988f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 --2- 172.21.15.86:0/1794690169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc388100a60 0x7fc3881983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 >> 172.21.15.86:0/1794690169 conn(0x7fc3880fc780 msgr2=0x7fc3880fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 shutdown_connections 2024-09-05T06:47:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:00.682+0000 7fc3901b0640 1 -- 172.21.15.86:0/1794690169 wait complete. 2024-09-05T06:47:00.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:00 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1794690169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:00 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1794690169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:01.060 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5117007c', quiesce took 5.2 seconds. Will release after: 29.099999999999998 2024-09-05T06:47:01.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:00 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1794690169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:01.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 2.03 MB/sec execute 141 sec latency 1651.714 ms 2024-09-05T06:47:01.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:47:01.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:01 smithi086 ceph-mon[30943]: pgmap v176: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-05T06:47:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:02.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:01 smithi096 ceph-mon[34468]: pgmap v176: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-05T06:47:02.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:01 smithi115 ceph-mon[34791]: pgmap v176: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-05T06:47:02.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 2.02 MB/sec execute 142 sec latency 2651.768 ms 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.623+0000 7f4094c97640 1 -- 172.21.15.86:0/171802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901036e0 msgr2=0x7f4090107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.623+0000 7f4094c97640 1 --2- 172.21.15.86:0/171802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901036e0 0x7f4090107b90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4078009930 tx=0x7f407802a420 comp rx=0 tx=0).stop 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- 172.21.15.86:0/171802448 shutdown_connections 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- 172.21.15.86:0/171802448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901082a0 0x7f409010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- 172.21.15.86:0/171802448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901036e0 0x7f4090107b90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- 172.21.15.86:0/171802448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- 172.21.15.86:0/171802448 >> 172.21.15.86:0/171802448 conn(0x7f40900fc7d0 msgr2=0x7f40900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- 172.21.15.86:0/171802448 shutdown_connections 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- 172.21.15.86:0/171802448 wait complete. 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 Processor -- start 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- start start 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901036e0 0x7f4090198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.624+0000 7f4094c97640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4090111ff0 con 0x7f40901082a0 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f4094c97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4090111e70 con 0x7f4090102d30 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f4094c97640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4090112170 con 0x7f40901036e0 2024-09-05T06:47:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52228/0 (socket says 172.21.15.86:52228) 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 -- 172.21.15.86:0/3309924087 learned_addr learned my addr 172.21.15.86:0/3309924087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408dd9b640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901036e0 0x7f4090198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408e59c640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 -- 172.21.15.86:0/3309924087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901036e0 msgr2=0x7f4090198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901036e0 0x7f4090198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 -- 172.21.15.86:0/3309924087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 msgr2=0x7f4090198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 -- 172.21.15.86:0/3309924087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409019d610 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408e59c640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f408ed9d640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f40877fe640 1 -- 172.21.15.86:0/3309924087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080011830 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409019d900 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f40877fe640 1 -- 172.21.15.86:0/3309924087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4080016070 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.625+0000 7f40877fe640 1 -- 172.21.15.86:0/3309924087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080010de0 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.626+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a51e0 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f40877fe640 1 -- 172.21.15.86:0/3309924087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f40800119d0 con 0x7f40901082a0 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 Processor -- start 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 -- start start 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c714090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c714a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c9c67a2f0 con 0x559c9c71f370 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c9c621cf0 con 0x559c9c715b40 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.628+0000 7f4097070740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c9c6762b0 con 0x559c9c7145d0 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52242/0 (socket says 172.21.15.86:52242) 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 -- 172.21.15.86:0/3367060045 learned_addr learned my addr 172.21.15.86:0/3367060045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c714090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408e59c640 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c714a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f40877fe640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 0x7f407c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 -- 172.21.15.86:0/3367060045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 msgr2=0x559c9c714a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c714a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 -- 172.21.15.86:0/3367060045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 msgr2=0x559c9c714090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c714090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 -- 172.21.15.86:0/3367060045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9c6063a0 con 0x559c9c71f370 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f40877fe640 1 -- 172.21.15.86:0/3309924087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f40800cede0 con 0x7f40901082a0 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408ed9d640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 0x7f407c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f408dd9b640 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4078005ec0 tx=0x7f4078005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b58850233a81e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f4084ff9640 1 -- 172.21.15.86:0/3367060045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035070 con 0x559c9c71f370 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f4084ff9640 1 -- 172.21.15.86:0/3367060045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40780054d0 con 0x559c9c71f370 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.629+0000 7f4084ff9640 1 -- 172.21.15.86:0/3367060045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078004a90 con 0x559c9c71f370 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 -- 172.21.15.86:0/3367060045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 msgr2=0x559c9c721760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4078005ec0 tx=0x7f4078005fd0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f408ed9d640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 0x7f407c0a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40800cc310 tx=0x7f40800046c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 -- 172.21.15.86:0/3367060045 shutdown_connections 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c721760 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c714a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 --2- 172.21.15.86:0/3367060045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c714090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 -- 172.21.15.86:0/3367060045 >> 172.21.15.86:0/3367060045 conn(0x559c9c6dd310 msgr2=0x559c9c710c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 -- 172.21.15.86:0/3367060045 shutdown_connections 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.630+0000 7f4097070740 1 -- 172.21.15.86:0/3367060045 wait complete. 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 -- start start 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 Processor -- start 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c71ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c71c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c9c722010 con 0x559c9c71f370 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c9c635060 con 0x559c9c715b40 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c9c722190 con 0x559c9c7145d0 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4097070740 10 client.0 _ll_register_callbacks cb 0x559c9c3888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52252/0 (socket says 172.21.15.86:52252) 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 -- 172.21.15.86:0/278800911 learned_addr learned my addr 172.21.15.86:0/278800911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408e59c640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c71c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c71ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 -- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 msgr2=0x559c9c71ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c71ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 -- 172.21.15.86:0/278800911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 msgr2=0x559c9c71c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c71c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 -- 172.21.15.86:0/278800911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9c3eb3c0 con 0x559c9c71f370 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f408dd9b640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4078001eb0 tx=0x7f4078004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d4925fcc851d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.631+0000 7f4075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035040 con 0x559c9c71f370 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f4078004a10 con 0x559c9c71f370 2024-09-05T06:47:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40780054d0 con 0x559c9c71f370 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4078004c10 con 0x559c9c71f370 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 10 client.16782 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c9c72dab0 con 0x559c9c71f370 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40780054d0 con 0x559c9c71f370 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f407803c440 con 0x559c9c71f370 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 10 client.16782 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 10 client.16782 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 20 client.16782 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c72ee00 0x559c9c7311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 4 client.16782 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c9c687860 con 0x559c9c72ee00 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.632+0000 7f408e59c640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c72ee00 0x559c9c7311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.633+0000 7f408e59c640 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c72ee00 0x559c9c7311f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1d2b3ebd8fcdbe0 server_cookie=ed4c17b908264c05 in_seq=0 out_seq=0 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.633+0000 7f4075ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4075ffb640 1 -- 172.21.15.86:0/278800911 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c9c687860 con 0x559c9c72ee00 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4075ffb640 10 client.16782 handle_command_reply: tid=0 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 client.16782 shutdown 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 msgr2=0x559c9c71c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4078001eb0 tx=0x7f4078004610 comp rx=0 tx=0).stop 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 shutdown_connections 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c72ee00 0x559c9c7311f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c71c610 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4075ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c71c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 --2- 172.21.15.86:0/278800911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c71ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 >> 172.21.15.86:0/278800911 conn(0x559c9c6dd310 msgr2=0x559c9c716960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 shutdown_connections 2024-09-05T06:47:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 1 -- 172.21.15.86:0/278800911 wait complete. 2024-09-05T06:47:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.634+0000 7f4097070740 20 client.16782 trim_cache size 0 max 16384 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 Processor -- start 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 -- start start 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c717120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c717660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40901af1f0 con 0x559c9c71f370 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b0d30 con 0x559c9c715b40 2024-09-05T06:47:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f4097070740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40901af8c0 con 0x559c9c7145d0 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52262/0 (socket says 172.21.15.86:52262) 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 -- 172.21.15.86:0/4069882081 learned_addr learned my addr 172.21.15.86:0/4069882081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408e59c640 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c717660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 -- 172.21.15.86:0/4069882081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 msgr2=0x559c9c717120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c717120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 -- 172.21.15.86:0/4069882081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 msgr2=0x559c9c717660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c717660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.647+0000 7f408dd9b640 1 -- 172.21.15.86:0/4069882081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9c721f50 con 0x559c9c71f370 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f408dd9b640 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4078009850 tx=0x7f4078031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c1f049b4091f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f4077fff640 1 -- 172.21.15.86:0/4069882081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078035070 con 0x559c9c71f370 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f4077fff640 1 -- 172.21.15.86:0/4069882081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4078004db0 con 0x559c9c71f370 2024-09-05T06:47:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f4077fff640 1 -- 172.21.15.86:0/4069882081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407803c6a0 con 0x559c9c71f370 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f4097070740 1 -- 172.21.15.86:0/4069882081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 msgr2=0x559c9c7192c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.648+0000 7f4097070740 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4078009850 tx=0x7f4078031520 comp rx=0 tx=0).stop 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- 172.21.15.86:0/4069882081 shutdown_connections 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c71f370 0x559c9c7192c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c715b40 0x559c9c717660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- 172.21.15.86:0/4069882081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c7145d0 0x559c9c717120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- 172.21.15.86:0/4069882081 >> 172.21.15.86:0/4069882081 conn(0x559c9c6dd310 msgr2=0x559c9c71e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- 172.21.15.86:0/4069882081 shutdown_connections 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- 172.21.15.86:0/4069882081 wait complete. 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- start start 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 Processor -- start 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 0x559c9c732190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c71f370 0x559c9c732c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b0660 con 0x559c9c715b40 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40901aff90 con 0x559c9c7145d0 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40901ad1d0 con 0x559c9c71f370 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f4097070740 10 client.0 _ll_register_callbacks cb 0x7f4078032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52274/0 (socket says 172.21.15.86:52274) 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 -- 172.21.15.86:0/187638605 learned_addr learned my addr 172.21.15.86:0/187638605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408dd9b640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 0x559c9c732190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 -- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c71f370 msgr2=0x559c9c732c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c71f370 0x559c9c732c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 -- 172.21.15.86:0/187638605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 msgr2=0x559c9c732190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 0x559c9c732190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408e59c640 1 -- 172.21.15.86:0/187638605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9c798940 con 0x559c9c715b40 2024-09-05T06:47:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.649+0000 7f408dd9b640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 0x559c9c732190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:02.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f408e59c640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x559c9c71d660 tx=0x7f408800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ead96a8b24a41c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:02.672 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:02.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408801e070 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f408801b990 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4078004db0 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f408801bb90 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 10 client.16788 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c9c799c90 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4078004db0 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f408800a490 con 0x559c9c715b40 2024-09-05T06:47:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 10 client.16788 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 10 client.16788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 20 client.16788 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c79a440 0x559c9c79c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 4 client.16788 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.650+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c9c681d70 con 0x559c9c79a440 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f408dd9b640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c79a440 0x559c9c79c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4075ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f408dd9b640 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c79a440 0x559c9c79c860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bdddda0a68b7367b server_cookie=21ef338436821643 in_seq=0 out_seq=0 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4075ffb640 1 -- 172.21.15.86:0/187638605 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c9c681d70 con 0x559c9c79a440 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4075ffb640 10 client.16788 handle_command_reply: tid=0 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 client.16788 shutdown 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 msgr2=0x559c9c7326d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x559c9c71d660 tx=0x7f408800bb00 comp rx=0 tx=0).stop 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 shutdown_connections 2024-09-05T06:47:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559c9c79a440 0x559c9c79c860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c9c71f370 0x559c9c732c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9c715b40 0x559c9c7326d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4075ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 --2- 172.21.15.86:0/187638605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c9c7145d0 0x559c9c732190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 >> 172.21.15.86:0/187638605 conn(0x559c9c6dd310 msgr2=0x559c9c70f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 shutdown_connections 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.651+0000 7f4097070740 1 -- 172.21.15.86:0/187638605 wait complete. 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4097070740 20 client.16788 trim_cache size 0 max 16384 2024-09-05T06:47:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 msgr2=0x7f407c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 0x7f407c0a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f40800cc310 tx=0x7f40800046c0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 msgr2=0x7f409019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000b8e0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 shutdown_connections 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f407c0a2cf0 0x7f407c0a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40901082a0 0x7f409019cf30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40901036e0 0x7f4090198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 --2- 172.21.15.86:0/3309924087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4090102d30 0x7f4090198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 >> 172.21.15.86:0/3309924087 conn(0x7f40900fc7d0 msgr2=0x7f4090105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 shutdown_connections 2024-09-05T06:47:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:02.652+0000 7f4094c97640 1 -- 172.21.15.86:0/3309924087 wait complete. 2024-09-05T06:47:03.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:47:03.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 2.01 MB/sec execute 143 sec latency 3651.811 ms 2024-09-05T06:47:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:03.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:03 smithi086 ceph-mon[30943]: pgmap v177: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:03 smithi096 ceph-mon[34468]: pgmap v177: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:03.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:03 smithi115 ceph-mon[34791]: pgmap v177: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:04.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 -- 172.21.15.86:0/77400578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c101de0 msgr2=0x7ffb8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 --2- 172.21.15.86:0/77400578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1021c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009900 tx=0x7ffb8402a410 comp rx=0 tx=0).stop 2024-09-05T06:47:04.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 -- 172.21.15.86:0/77400578 shutdown_connections 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 --2- 172.21.15.86:0/77400578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c106f10 0x7ffb8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 --2- 172.21.15.86:0/77400578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c102790 0x7ffb8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 --2- 172.21.15.86:0/77400578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1021c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 -- 172.21.15.86:0/77400578 >> 172.21.15.86:0/77400578 conn(0x7ffb8c0fd8c0 msgr2=0x7ffb8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 -- 172.21.15.86:0/77400578 shutdown_connections 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.098+0000 7ffb944c3640 1 -- 172.21.15.86:0/77400578 wait complete. 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 Processor -- start 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 -- start start 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c101de0 0x7ffb8c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c102790 0x7ffb8c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111e20 con 0x7ffb8c106f10 2024-09-05T06:47:04.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111ca0 con 0x7ffb8c102790 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb944c3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111fa0 con 0x7ffb8c101de0 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52288/0 (socket says 172.21.15.86:52288) 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 -- 172.21.15.86:0/4127821091 learned_addr learned my addr 172.21.15.86:0/4127821091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb9225f640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c101de0 0x7ffb8c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c102790 0x7ffb8c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 -- 172.21.15.86:0/4127821091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c101de0 msgr2=0x7ffb8c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c101de0 0x7ffb8c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 -- 172.21.15.86:0/4127821091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c102790 msgr2=0x7ffb8c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c102790 0x7ffb8c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.099+0000 7ffb92a60640 1 -- 172.21.15.86:0/4127821091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c19d310 con 0x7ffb8c106f10 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb92a60640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00eaa0 tx=0x7ffb7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb8b7fe640 1 -- 172.21.15.86:0/4127821091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00cdb0 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c19d600 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb8b7fe640 1 -- 172.21.15.86:0/4127821091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb7c005020 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb8b7fe640 1 -- 172.21.15.86:0/4127821091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0116d0 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.100+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c1a4ee0 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.102+0000 7ffb8b7fe640 1 -- 172.21.15.86:0/4127821091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb7c0051d0 con 0x7ffb8c106f10 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.102+0000 7ffb9689c740 1 Processor -- start 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 -- start start 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 0x561cd8ae9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8ae9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 0x561cd8af6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cd8a4f2f0 con 0x561cd8aeab40 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561cd89f6cf0 con 0x561cd8af4370 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9689c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cd8a4b2b0 con 0x561cd8ae95d0 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb8b7fe640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 0x7ffb740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 0x561cd8ae9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 0x561cd8ae9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52298/0 (socket says 172.21.15.86:52298) 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 -- 172.21.15.86:0/4182691454 learned_addr learned my addr 172.21.15.86:0/4182691454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb9225f640 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8ae9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb92a60640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 0x7ffb740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 0x561cd8af6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb8b7fe640 1 -- 172.21.15.86:0/4127821091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ffb7c0cf960 con 0x7ffb8c106f10 2024-09-05T06:47:04.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 -- 172.21.15.86:0/4182691454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 msgr2=0x561cd8ae9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8ae9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 -- 172.21.15.86:0/4182691454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 msgr2=0x561cd8ae9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 0x561cd8ae9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.103+0000 7ffb91a5e640 1 -- 172.21.15.86:0/4182691454 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd89db3a0 con 0x561cd8af4370 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.104+0000 7ffb92a60640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 0x7ffb740a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c06eb30 tx=0x7ffb7c06dab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.104+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 0x561cd8af6760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000be20 tx=0x7ffb80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f623624ac18a994b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.104+0000 7ffb88ff9640 1 -- 172.21.15.86:0/4182691454 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80014070 con 0x561cd8af4370 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.104+0000 7ffb88ff9640 1 -- 172.21.15.86:0/4182691454 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.104+0000 7ffb88ff9640 1 -- 172.21.15.86:0/4182691454 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80004d60 con 0x561cd8af4370 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- 172.21.15.86:0/4182691454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 msgr2=0x561cd8af6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 0x561cd8af6760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000be20 tx=0x7ffb80004730 comp rx=0 tx=0).stop 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- 172.21.15.86:0/4182691454 shutdown_connections 2024-09-05T06:47:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8af4370 0x561cd8af6760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8ae9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- 172.21.15.86:0/4182691454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8aeab40 0x561cd8ae9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- 172.21.15.86:0/4182691454 >> 172.21.15.86:0/4182691454 conn(0x561cd8ab2310 msgr2=0x561cd8ae5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- 172.21.15.86:0/4182691454 shutdown_connections 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- 172.21.15.86:0/4182691454 wait complete. 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- start start 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 Processor -- start 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8af3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8af0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cd8af7070 con 0x561cd8af4370 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561cd8a0a060 con 0x561cd8ae95d0 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cd8af71f0 con 0x561cd8aeab40 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.105+0000 7ffb9689c740 10 client.0 _ll_register_callbacks cb 0x561cd875d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52304/0 (socket says 172.21.15.86:52304) 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 -- 172.21.15.86:0/3548495215 learned_addr learned my addr 172.21.15.86:0/3548495215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8af3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb9225f640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8af0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 -- 172.21.15.86:0/3548495215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 msgr2=0x561cd8af0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8af0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 -- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 msgr2=0x561cd8af3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8af3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 -- 172.21.15.86:0/3548495215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd87c03c0 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb9225f640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8af0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004260 tx=0x7ffb80004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1322aedc47aa3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80014030 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ffb8001a070 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb80031050 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb9689c740 10 client.16800 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cd8b02ac0 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.106+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ffb80010830 con 0x561cd8af4370 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 10 client.16800 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 10 client.16800 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 20 client.16800 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b03e10 0x561cd8b06200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 4 client.16800 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd8a5c860 con 0x561cd8b03e10 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9225f640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b03e10 0x561cd8b06200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb71ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.107+0000 7ffb9225f640 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b03e10 0x561cd8b06200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47354689d8800fe2 server_cookie=733e66c3bea1bf1d in_seq=0 out_seq=0 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb71ffb640 1 -- 172.21.15.86:0/3548495215 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cd8a5c860 con 0x561cd8b03e10 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb71ffb640 10 client.16800 handle_command_reply: tid=0 2024-09-05T06:47:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 client.16800 shutdown 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 msgr2=0x561cd8af1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004260 tx=0x7ffb80004940 comp rx=0 tx=0).stop 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 shutdown_connections 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b03e10 0x561cd8b06200 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8af1570 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8af0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 --2- 172.21.15.86:0/3548495215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8af3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 >> 172.21.15.86:0/3548495215 conn(0x561cd8ab2310 msgr2=0x561cd8aeb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb71ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 shutdown_connections 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 1 -- 172.21.15.86:0/3548495215 wait complete. 2024-09-05T06:47:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.108+0000 7ffb9689c740 20 client.16800 trim_cache size 0 max 16384 2024-09-05T06:47:04.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 Processor -- start 2024-09-05T06:47:04.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 -- start start 2024-09-05T06:47:04.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8b00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8aec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1aefc0 con 0x561cd8af4370 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1b0b00 con 0x561cd8ae95d0 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb9689c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1af690 con 0x561cd8aeab40 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52308/0 (socket says 172.21.15.86:52308) 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1988462840 learned_addr learned my addr 172.21.15.86:0/1988462840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.121+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1988462840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 msgr2=0x561cd8aec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8aec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8aec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1988462840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 msgr2=0x561cd8b00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8b00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1988462840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd8aea780 con 0x561cd8af4370 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8aec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004910 tx=0x7ffb8000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b7cee8e006d98f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb73fff640 1 -- 172.21.15.86:0/1988462840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80014030 con 0x561cd8af4370 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb73fff640 1 -- 172.21.15.86:0/1988462840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb73fff640 1 -- 172.21.15.86:0/1988462840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80003f20 con 0x561cd8af4370 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 -- 172.21.15.86:0/1988462840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 msgr2=0x561cd8aec660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004910 tx=0x7ffb8000bcf0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 -- 172.21.15.86:0/1988462840 shutdown_connections 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aec660 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8aeab40 0x561cd8aec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1988462840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8ae95d0 0x561cd8b00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.122+0000 7ffb9689c740 1 -- 172.21.15.86:0/1988462840 >> 172.21.15.86:0/1988462840 conn(0x561cd8ab2310 msgr2=0x561cd8af00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- 172.21.15.86:0/1988462840 shutdown_connections 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- 172.21.15.86:0/1988462840 wait complete. 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- start start 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 Processor -- start 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8afc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 0x561cd8afc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1b0430 con 0x561cd8af4370 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1afd60 con 0x561cd8aeab40 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1acfa0 con 0x561cd8ae95d0 2024-09-05T06:47:04.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9689c740 10 client.0 _ll_register_callbacks cb 0x7ffb80004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb91a5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52316/0 (socket says 172.21.15.86:52316) 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1196086065 learned_addr learned my addr 172.21.15.86:0/1196086065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 0x561cd8afc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1196086065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 msgr2=0x561cd8afc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.123+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8afc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 msgr2=0x561cd8afc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 0x561cd8afc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb91a5e640 1 -- 172.21.15.86:0/1196086065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd8afd080 con 0x561cd8af4370 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 0x561cd8afc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb91a5e640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffb80010cf0 tx=0x7ffb800041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6d2ee16c90b54f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:04.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80014040 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7ffb80010420 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb80010df0 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 10 client.16806 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cd8b6ddd0 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8000bab0 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ffb8001e4a0 con 0x561cd8af4370 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 10 client.16806 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 10 client.16806 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 20 client.16806 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:04.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b6e3e0 0x561cd8b70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 4 client.16806 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.124+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cd8a56d70 con 0x561cd8b6e3e0 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b6e3e0 0x561cd8b70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9225f640 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b6e3e0 0x561cd8b70800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b152978a8584c01 server_cookie=1bec282d6ad343f2 in_seq=0 out_seq=0 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb71ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb71ffb640 1 -- 172.21.15.86:0/1196086065 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cd8a56d70 con 0x561cd8b6e3e0 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb71ffb640 10 client.16806 handle_command_reply: tid=0 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 client.16806 shutdown 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 msgr2=0x561cd8aed260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffb80010cf0 tx=0x7ffb800041f0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 shutdown_connections 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x561cd8b6e3e0 0x561cd8b70800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cd8af4370 0x561cd8aed260 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561cd8aeab40 0x561cd8afc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb71ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 --2- 172.21.15.86:0/1196086065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd8ae95d0 0x561cd8afc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 >> 172.21.15.86:0/1196086065 conn(0x561cd8ab2310 msgr2=0x561cd8af2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.125+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 shutdown_connections 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb9689c740 1 -- 172.21.15.86:0/1196086065 wait complete. 2024-09-05T06:47:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb9689c740 20 client.16806 trim_cache size 0 max 16384 2024-09-05T06:47:04.142 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:04.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 msgr2=0x7ffb740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 0x7ffb740a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c06eb30 tx=0x7ffb7c06dab0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 msgr2=0x7ffb8c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00eaa0 tx=0x7ffb7c00ef60 comp rx=0 tx=0).stop 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 shutdown_connections 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffb740a2cf0 0x7ffb740a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb8c102790 0x7ffb8c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 --2- 172.21.15.86:0/4127821091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb8c101de0 0x7ffb8c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 >> 172.21.15.86:0/4127821091 conn(0x7ffb8c0fd8c0 msgr2=0x7ffb8c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 shutdown_connections 2024-09-05T06:47:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:04.126+0000 7ffb944c3640 1 -- 172.21.15.86:0/4127821091 wait complete. 2024-09-05T06:47:04.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.99 MB/sec execute 144 sec latency 4651.844 ms 2024-09-05T06:47:05.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:47:05.283 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:05.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.98 MB/sec execute 145 sec latency 5651.881 ms 2024-09-05T06:47:05.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:05 smithi086 ceph-mon[30943]: pgmap v178: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 -- 172.21.15.86:0/3984151109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee01027c0 msgr2=0x7f8ee0106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/3984151109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee01027c0 0x7f8ee0106810 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009900 tx=0x7f8ec802a410 comp rx=0 tx=0).stop 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 -- 172.21.15.86:0/3984151109 shutdown_connections 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/3984151109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/3984151109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee01027c0 0x7f8ee0106810 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/3984151109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee0101e10 0x7f8ee01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 -- 172.21.15.86:0/3984151109 >> 172.21.15.86:0/3984151109 conn(0x7f8ee00fd8d0 msgr2=0x7f8ee00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 -- 172.21.15.86:0/3984151109 shutdown_connections 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.742+0000 7f8ee51bd640 1 -- 172.21.15.86:0/3984151109 wait complete. 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 Processor -- start 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 -- start start 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 0x7f8ee0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111e70 con 0x7f8ee0101e10 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111cf0 con 0x7f8ee0106f40 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8ee51bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111ff0 con 0x7f8ee01027c0 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8eded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8eded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52324/0 (socket says 172.21.15.86:52324) 2024-09-05T06:47:05.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.743+0000 7f8eded9d640 1 -- 172.21.15.86:0/2791640249 learned_addr learned my addr 172.21.15.86:0/2791640249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8edf59e640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 0x7f8ee0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 -- 172.21.15.86:0/2791640249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 msgr2=0x7f8ee0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 0x7f8ee0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 -- 172.21.15.86:0/2791640249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 msgr2=0x7f8ee0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 -- 172.21.15.86:0/2791640249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee01a68d0 con 0x7f8ee0101e10 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8edf59e640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 0x7f8ee0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8eded9d640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000b4a0 tx=0x7f8ed000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8edcd99640 1 -- 172.21.15.86:0/2791640249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0014070 con 0x7f8ee0101e10 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee01a6bc0 con 0x7f8ee0101e10 2024-09-05T06:47:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8edcd99640 1 -- 172.21.15.86:0/2791640249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed0010ac0 con 0x7f8ee0101e10 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8edcd99640 1 -- 172.21.15.86:0/2791640249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0017c30 con 0x7f8ee0101e10 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.744+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01a70b0 con 0x7f8ee0101e10 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.746+0000 7f8edcd99640 1 -- 172.21.15.86:0/2791640249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ed0017070 con 0x7f8ee0101e10 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 Processor -- start 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 -- start start 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edcd99640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 0x7f8ecc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edf59e640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 0x7f8ecc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edcd99640 1 -- 172.21.15.86:0/2791640249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f8ed00cebe0 con 0x7f8ee0101e10 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55913ab6b2f0 con 0x55913ac10370 2024-09-05T06:47:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55913ab12cf0 con 0x55913ac06b40 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ee7596740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55913ab672b0 con 0x55913ac055d0 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52328/0 (socket says 172.21.15.86:52328) 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8edf59e640 1 -- 172.21.15.86:0/3350098141 learned_addr learned my addr 172.21.15.86:0/3350098141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8eded9d640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.747+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 0x7f8ecc0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8014820 tx=0x7f8ed8014c20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 -- 172.21.15.86:0/3350098141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 msgr2=0x55913ac05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 -- 172.21.15.86:0/3350098141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 msgr2=0x55913ac05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 -- 172.21.15.86:0/3350098141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913aaf73a0 con 0x55913ac10370 2024-09-05T06:47:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8eded9d640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8012760 tx=0x7f8ed801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f051737225a27d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8ed57fa640 1 -- 172.21.15.86:0/3350098141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800eb60 con 0x55913ac10370 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8ed57fa640 1 -- 172.21.15.86:0/3350098141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed800ed00 con 0x55913ac10370 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.748+0000 7f8ed57fa640 1 -- 172.21.15.86:0/3350098141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed801f6c0 con 0x55913ac10370 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- 172.21.15.86:0/3350098141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 msgr2=0x55913ac12760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8012760 tx=0x7f8ed801e000 comp rx=0 tx=0).stop 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- 172.21.15.86:0/3350098141 shutdown_connections 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac12760 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3350098141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- 172.21.15.86:0/3350098141 >> 172.21.15.86:0/3350098141 conn(0x55913abce310 msgr2=0x55913ac01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- 172.21.15.86:0/3350098141 shutdown_connections 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- 172.21.15.86:0/3350098141 wait complete. 2024-09-05T06:47:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- start start 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 Processor -- start 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac10370 0x55913ac0d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55913ac13010 con 0x55913ac06b40 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55913ab26060 con 0x55913ac10370 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.749+0000 7f8ee7596740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55913ac13190 con 0x55913ac055d0 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ee7596740 10 client.0 _ll_register_callbacks cb 0x55913a8798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52334/0 (socket says 172.21.15.86:52334) 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 -- 172.21.15.86:0/1463842798 learned_addr learned my addr 172.21.15.86:0/1463842798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:05.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8eded9d640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 -- 172.21.15.86:0/1463842798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 msgr2=0x55913ac0fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac0fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 -- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac10370 msgr2=0x55913ac0d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac10370 0x55913ac0d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 -- 172.21.15.86:0/1463842798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913a8dc3c0 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8031440 tx=0x7f8ec8031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d850c6e7dd48026 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec803f070 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f8ec8004450 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed800ed00 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8035070 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ee7596740 10 client.16818 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.750+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55913ac1ea80 con 0x55913ac06b40 2024-09-05T06:47:05.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed800ed00 con 0x55913ac06b40 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec8004b40 con 0x55913ac06b40 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 10 client.16818 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 10 client.16818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 20 client.16818 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac1ff40 0x55913ac22380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 4 client.16818 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55913ab78860 con 0x55913ac1ff40 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8eded9d640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac1ff40 0x55913ac22380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8ec67fc640 10 client.16818 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.751+0000 7f8eded9d640 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac1ff40 0x55913ac22380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f47e08ab2e32e2 server_cookie=fcd33c21acd89d4d in_seq=0 out_seq=0 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ec67fc640 1 -- 172.21.15.86:0/1463842798 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55913ab78860 con 0x55913ac1ff40 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ec67fc640 10 client.16818 handle_command_reply: tid=0 2024-09-05T06:47:05.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ee7596740 1 client.16818 shutdown 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 msgr2=0x55913ac0cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8031440 tx=0x7f8ec8031750 comp rx=0 tx=0).stop 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 shutdown_connections 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.752+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac1ff40 0x55913ac22380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac10370 0x55913ac0d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac0cf90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ec67fc640 0 client.16818 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 --2- 172.21.15.86:0/1463842798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac0fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 >> 172.21.15.86:0/1463842798 conn(0x55913abce310 msgr2=0x55913ac07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 shutdown_connections 2024-09-05T06:47:05.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 1 -- 172.21.15.86:0/1463842798 wait complete. 2024-09-05T06:47:05.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.753+0000 7f8ee7596740 20 client.16818 trim_cache size 0 max 16384 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.765+0000 7f8ee7596740 1 Processor -- start 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.765+0000 7f8ee7596740 1 -- start start 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac0c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b1260 con 0x55913ac10370 2024-09-05T06:47:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b2da0 con 0x55913ac06b40 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ee7596740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b1930 con 0x55913ac055d0 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52336/0 (socket says 172.21.15.86:52336) 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 -- 172.21.15.86:0/3777108011 learned_addr learned my addr 172.21.15.86:0/3777108011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8eded9d640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 -- 172.21.15.86:0/3777108011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 msgr2=0x55913ac1c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac1c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 -- 172.21.15.86:0/3777108011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 msgr2=0x55913ac0c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac0c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac0c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 -- 172.21.15.86:0/3777108011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913ac02960 con 0x55913ac10370 2024-09-05T06:47:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac0c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.766+0000 7f8edf59e640 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8012800 tx=0x7f8ed8004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9761a82d3b4c4c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ed4ff9640 1 -- 172.21.15.86:0/3777108011 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8009ea0 con 0x55913ac10370 2024-09-05T06:47:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ed4ff9640 1 -- 172.21.15.86:0/3777108011 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed8004e30 con 0x55913ac10370 2024-09-05T06:47:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ed4ff9640 1 -- 172.21.15.86:0/3777108011 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8005710 con 0x55913ac10370 2024-09-05T06:47:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 -- 172.21.15.86:0/3777108011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 msgr2=0x55913ac0cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8012800 tx=0x7f8ed8004c10 comp rx=0 tx=0).stop 2024-09-05T06:47:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 -- 172.21.15.86:0/3777108011 shutdown_connections 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac10370 0x55913ac0cd90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac06b40 0x55913ac0c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 --2- 172.21.15.86:0/3777108011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac055d0 0x55913ac1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 -- 172.21.15.86:0/3777108011 >> 172.21.15.86:0/3777108011 conn(0x55913abce310 msgr2=0x55913ac08310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 -- 172.21.15.86:0/3777108011 shutdown_connections 2024-09-05T06:47:05.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.767+0000 7f8ee7596740 1 -- 172.21.15.86:0/3777108011 wait complete. 2024-09-05T06:47:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 -- start start 2024-09-05T06:47:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 Processor -- start 2024-09-05T06:47:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac055d0 0x55913ac18c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac10370 0x55913ac0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b26d0 con 0x55913ac06b40 2024-09-05T06:47:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b2000 con 0x55913ac055d0 2024-09-05T06:47:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01af240 con 0x55913ac10370 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ee7596740 10 client.0 _ll_register_callbacks cb 0x7f8ec8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52342/0 (socket says 172.21.15.86:52342) 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 -- 172.21.15.86:0/655922174 learned_addr learned my addr 172.21.15.86:0/655922174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8eded9d640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac055d0 0x55913ac18c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8edf59e640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac10370 0x55913ac0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 -- 172.21.15.86:0/655922174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac10370 msgr2=0x55913ac0a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac10370 0x55913ac0a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 -- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac055d0 msgr2=0x55913ac18c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac055d0 0x55913ac18c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.768+0000 7f8ed7fff640 1 -- 172.21.15.86:0/655922174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913ac05400 con 0x55913ac06b40 2024-09-05T06:47:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ed7fff640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009850 tx=0x7f8ec8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57efa88d0eafbe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8055020 con 0x55913ac06b40 2024-09-05T06:47:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f8ec80046c0 con 0x55913ac06b40 2024-09-05T06:47:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed8004e30 con 0x55913ac06b40 2024-09-05T06:47:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ec805d050 con 0x55913ac06b40 2024-09-05T06:47:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 10 client.16824 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55913ac8aa80 con 0x55913ac06b40 2024-09-05T06:47:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed8004e30 con 0x55913ac06b40 2024-09-05T06:47:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec8031be0 con 0x55913ac06b40 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 10 client.16824 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 10 client.16824 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 20 client.16824 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac8b250 0x55913ac8d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 4 client.16824 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.769+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55913ab72d70 con 0x55913ac8b250 2024-09-05T06:47:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8eded9d640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac8b250 0x55913ac8d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8eded9d640 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac8b250 0x55913ac8d670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f4bdc8efdcca8be server_cookie=abe400275c014e5 in_seq=0 out_seq=0 2024-09-05T06:47:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ec67fc640 10 client.16824 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ec67fc640 1 -- 172.21.15.86:0/655922174 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55913ab72d70 con 0x55913ac8b250 2024-09-05T06:47:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ec67fc640 10 client.16824 handle_command_reply: tid=0 2024-09-05T06:47:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 client.16824 shutdown 2024-09-05T06:47:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 msgr2=0x55913ac09d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009850 tx=0x7f8ec8003e60 comp rx=0 tx=0).stop 2024-09-05T06:47:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 shutdown_connections 2024-09-05T06:47:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55913ac8b250 0x55913ac8d670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55913ac10370 0x55913ac0a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ec67fc640 0 client.16824 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55913ac06b40 0x55913ac09d80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 --2- 172.21.15.86:0/655922174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55913ac055d0 0x55913ac18c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 >> 172.21.15.86:0/655922174 conn(0x55913abce310 msgr2=0x55913ac0b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 shutdown_connections 2024-09-05T06:47:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.770+0000 7f8ee7596740 1 -- 172.21.15.86:0/655922174 wait complete. 2024-09-05T06:47:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee7596740 20 client.16824 trim_cache size 0 max 16384 2024-09-05T06:47:05.793 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 msgr2=0x7f8ecc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 0x7f8ecc0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8014820 tx=0x7f8ed8014c20 comp rx=0 tx=0).stop 2024-09-05T06:47:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 msgr2=0x7f8ee00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:05.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000b4a0 tx=0x7f8ed000b960 comp rx=0 tx=0).stop 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 shutdown_connections 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ecc0a2cf0 0x7f8ecc0a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee0106f40 0x7f8ee0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee01027c0 0x7f8ee0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 --2- 172.21.15.86:0/2791640249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0101e10 0x7f8ee00744f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 >> 172.21.15.86:0/2791640249 conn(0x7f8ee00fd8d0 msgr2=0x7f8ee00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 shutdown_connections 2024-09-05T06:47:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:05.771+0000 7f8ee51bd640 1 -- 172.21.15.86:0/2791640249 wait complete. 2024-09-05T06:47:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:05 smithi096 ceph-mon[34468]: pgmap v178: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:05 smithi115 ceph-mon[34791]: pgmap v178: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T06:47:06.502 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:47:06.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.96 MB/sec execute 146 sec latency 6651.934 ms 2024-09-05T06:47:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.210+0000 7f83ceed0640 1 -- 172.21.15.86:0/2206398108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 msgr2=0x7f83c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.210+0000 7f83ceed0640 1 --2- 172.21.15.86:0/2206398108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c81067e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009900 tx=0x7f83bc02a410 comp rx=0 tx=0).stop 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- 172.21.15.86:0/2206398108 shutdown_connections 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- 172.21.15.86:0/2206398108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8106f10 0x7f83c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- 172.21.15.86:0/2206398108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c81067e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- 172.21.15.86:0/2206398108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8101de0 0x7f83c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- 172.21.15.86:0/2206398108 >> 172.21.15.86:0/2206398108 conn(0x7f83c80fd8c0 msgr2=0x7f83c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- 172.21.15.86:0/2206398108 shutdown_connections 2024-09-05T06:47:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- 172.21.15.86:0/2206398108 wait complete. 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 Processor -- start 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- start start 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 0x7f83c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 0x7f83c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8111e20 con 0x7f83c8102790 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.211+0000 7f83ceed0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8111ca0 con 0x7f83c8106f10 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83ceed0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8111fa0 con 0x7f83c8101de0 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52352/0 (socket says 172.21.15.86:52352) 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 -- 172.21.15.86:0/1615379692 learned_addr learned my addr 172.21.15.86:0/1615379692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 0x7f83c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 0x7f83c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 -- 172.21.15.86:0/1615379692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 msgr2=0x7f83c80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 0x7f83c80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 -- 172.21.15.86:0/1615379692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 msgr2=0x7f83c8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 0x7f83c8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 -- 172.21.15.86:0/1615379692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c81a6820 con 0x7f83c8102790 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 0x7f83c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 0x7f83c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c7fff640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005940 tx=0x7f83bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.212+0000 7f83c5ffb640 1 -- 172.21.15.86:0/1615379692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc03f030 con 0x7f83c8102790 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.213+0000 7f83c5ffb640 1 -- 172.21.15.86:0/1615379692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83bc003aa0 con 0x7f83c8102790 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.213+0000 7f83c5ffb640 1 -- 172.21.15.86:0/1615379692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc035070 con 0x7f83c8102790 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.213+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81a6ab0 con 0x7f83c8102790 2024-09-05T06:47:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.213+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81a6f40 con 0x7f83c8102790 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.215+0000 7f83c5ffb640 1 -- 172.21.15.86:0/1615379692 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83bc042030 con 0x7f83c8102790 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.215+0000 7f83d12a9740 1 Processor -- start 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.215+0000 7f83d12a9740 1 -- start start 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444637aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564446385370 0x564446387760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644462e02f0 con 0x56444637bb40 2024-09-05T06:47:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564446287cf0 con 0x564446385370 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83d12a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644462dc2b0 con 0x56444637a5d0 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83c5ffb640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 0x7f83b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52360/0 (socket says 172.21.15.86:52360) 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 -- 172.21.15.86:0/1210679804 learned_addr learned my addr 172.21.15.86:0/1210679804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444637aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83c7fff640 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564446385370 0x564446387760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 0x7f83b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83c5ffb640 1 -- 172.21.15.86:0/1615379692 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f83bc0f00d0 con 0x7f83c8102790 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 -- 172.21.15.86:0/1210679804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 msgr2=0x56444637aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444637aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 -- 172.21.15.86:0/1210679804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564446385370 msgr2=0x564446387760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564446385370 0x564446387760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 -- 172.21.15.86:0/1210679804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444626c3a0 con 0x56444637bb40 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.216+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83c00149a0 tx=0x7f83c0014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ac4e15b1c646c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83aeffd640 1 -- 172.21.15.86:0/1210679804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0021070 con 0x56444637bb40 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83aeffd640 1 -- 172.21.15.86:0/1210679804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c0013460 con 0x56444637bb40 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83aeffd640 1 -- 172.21.15.86:0/1210679804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000f5f0 con 0x56444637bb40 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 0x7f83b00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83c001e000 tx=0x7f83c0019000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 -- 172.21.15.86:0/1210679804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 msgr2=0x56444637a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83c00149a0 tx=0x7f83c0014e60 comp rx=0 tx=0).stop 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 -- 172.21.15.86:0/1210679804 shutdown_connections 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564446385370 0x564446387760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444637aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1210679804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637bb40 0x56444637a090 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 -- 172.21.15.86:0/1210679804 >> 172.21.15.86:0/1210679804 conn(0x564446343310 msgr2=0x564446376c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 -- 172.21.15.86:0/1210679804 shutdown_connections 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.217+0000 7f83d12a9740 1 -- 172.21.15.86:0/1210679804 wait complete. 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 -- start start 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 Processor -- start 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637a5d0 0x564446384d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637bb40 0x564446382030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564446388070 con 0x564446385370 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56444629b060 con 0x56444637a5d0 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644463881f0 con 0x56444637bb40 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83d12a9740 10 client.0 _ll_register_callbacks cb 0x564445fee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52370/0 (socket says 172.21.15.86:52370) 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83cd46d640 1 -- 172.21.15.86:0/3909250155 learned_addr learned my addr 172.21.15.86:0/3909250155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.218+0000 7f83c7fff640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637bb40 0x564446382030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 -- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637bb40 msgr2=0x564446382030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637bb40 0x564446382030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 -- 172.21.15.86:0/3909250155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637a5d0 msgr2=0x564446384d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637a5d0 0x564446384d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637a5d0 0x564446384d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 -- 172.21.15.86:0/3909250155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644460513c0 con 0x564446385370 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83cd46d640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005cc0 tx=0x7f83c00138c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5356221eaa5cd465 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0021040 con 0x564446385370 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f83c0004a00 con 0x564446385370 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c0013460 con 0x564446385370 2024-09-05T06:47:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83c0014360 con 0x564446385370 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 10 client.16836 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564446393b40 con 0x564446385370 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c0013460 con 0x564446385370 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83c0004c70 con 0x564446385370 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 10 client.16836 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 10 client.16836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.219+0000 7f83d12a9740 20 client.16836 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564446394e90 0x564446397280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83d12a9740 4 client.16836 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644462ed860 con 0x564446394e90 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564446394e90 0x564446397280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83a7fff640 10 client.16836 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.220+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564446394e90 0x564446397280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd5fb033b4d76111 server_cookie=3e4bac2abafa88a1 in_seq=0 out_seq=0 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83a7fff640 1 -- 172.21.15.86:0/3909250155 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644462ed860 con 0x564446394e90 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83a7fff640 10 client.16836 handle_command_reply: tid=0 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 client.16836 shutdown 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 msgr2=0x564446382640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005cc0 tx=0x7f83c00138c0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 shutdown_connections 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564446394e90 0x564446397280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446382640 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637bb40 0x564446382030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83a7fff640 0 client.16836 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3909250155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637a5d0 0x564446384d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 >> 172.21.15.86:0/3909250155 conn(0x564446343310 msgr2=0x56444637c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 shutdown_connections 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 1 -- 172.21.15.86:0/3909250155 wait complete. 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.221+0000 7f83d12a9740 20 client.16836 trim_cache size 0 max 16384 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 Processor -- start 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 -- start start 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x56444637d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564446385370 0x56444639cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81b1150 con 0x56444637a5d0 2024-09-05T06:47:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81b2c90 con 0x56444637bb40 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83d12a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81b1820 con 0x564446385370 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52380/0 (socket says 172.21.15.86:52380) 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 -- 172.21.15.86:0/3431893707 learned_addr learned my addr 172.21.15.86:0/3431893707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 -- 172.21.15.86:0/3431893707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564446385370 msgr2=0x56444639cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564446385370 0x56444639cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 -- 172.21.15.86:0/3431893707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 msgr2=0x56444637d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x56444637d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 -- 172.21.15.86:0/3431893707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444637ad50 con 0x56444637a5d0 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.234+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x564446397ac0 tx=0x7f83b800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29409e6bfdf8aad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83ae7fc640 1 -- 172.21.15.86:0/3431893707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b801f020 con 0x56444637a5d0 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83ae7fc640 1 -- 172.21.15.86:0/3431893707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83b8012900 con 0x56444637a5d0 2024-09-05T06:47:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83ae7fc640 1 -- 172.21.15.86:0/3431893707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b801b6e0 con 0x56444637a5d0 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 -- 172.21.15.86:0/3431893707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 msgr2=0x56444637d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x564446397ac0 tx=0x7f83b800b5a0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 -- 172.21.15.86:0/3431893707 shutdown_connections 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564446385370 0x56444639cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x56444637d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 --2- 172.21.15.86:0/3431893707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444637a5d0 0x56444637d120 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 -- 172.21.15.86:0/3431893707 >> 172.21.15.86:0/3431893707 conn(0x564446343310 msgr2=0x564446375d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 -- 172.21.15.86:0/3431893707 shutdown_connections 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.235+0000 7f83d12a9740 1 -- 172.21.15.86:0/3431893707 wait complete. 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 -- start start 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 Processor -- start 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444638dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x564446389040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81b25c0 con 0x564446385370 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81b1ef0 con 0x56444637bb40 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81af130 con 0x56444637a5d0 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83d12a9740 10 client.0 _ll_register_callbacks cb 0x7f83c001f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52394/0 (socket says 172.21.15.86:52394) 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 -- 172.21.15.86:0/1467182368 learned_addr learned my addr 172.21.15.86:0/1467182368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83c7fff640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x564446389040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444638dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 -- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 msgr2=0x56444638dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444638dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 -- 172.21.15.86:0/1467182368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 msgr2=0x564446389040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x564446389040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.236+0000 7f83cd46d640 1 -- 172.21.15.86:0/1467182368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564446389e80 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83cd46d640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f83c001dfd0 tx=0x7f83c0005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214e7e46cb4f3957 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0018070 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f83c0008740 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b8012900 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83c0047050 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83d12a9740 10 client.16842 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644463ffc50 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83b8012900 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.237+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83c003d030 con 0x564446385370 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 10 client.16842 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 10 client.16842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 20 client.16842 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5644464002b0 0x5644464026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 4 client.16842 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.238+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644462e7d70 con 0x5644464002b0 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5644464002b0 0x5644464026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83ccc6c640 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5644464002b0 0x5644464026a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90f221adcd3f7c0 server_cookie=401d4814845a9bca in_seq=0 out_seq=0 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83a7fff640 10 client.16842 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83a7fff640 1 -- 172.21.15.86:0/1467182368 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644462e7d70 con 0x5644464002b0 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83a7fff640 10 client.16842 handle_command_reply: tid=0 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 client.16842 shutdown 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 msgr2=0x564446389650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f83c001dfd0 tx=0x7f83c0005210 comp rx=0 tx=0).stop 2024-09-05T06:47:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 shutdown_connections 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5644464002b0 0x5644464026a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564446385370 0x564446389650 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56444637bb40 0x564446389040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83a7fff640 0 client.16842 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 --2- 172.21.15.86:0/1467182368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56444637a5d0 0x56444638dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.239+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 >> 172.21.15.86:0/1467182368 conn(0x564446343310 msgr2=0x564446375500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 shutdown_connections 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83d12a9740 1 -- 172.21.15.86:0/1467182368 wait complete. 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83d12a9740 20 client.16842 trim_cache size 0 max 16384 2024-09-05T06:47:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 msgr2=0x7f83b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 0x7f83b00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83c001e000 tx=0x7f83c0019000 comp rx=0 tx=0).stop 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 msgr2=0x7f83c8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005940 tx=0x7f83bc004270 comp rx=0 tx=0).stop 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 shutdown_connections 2024-09-05T06:47:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f83b00a2cf0 0x7f83b00a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83c8106f10 0x7f83c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83c8102790 0x7f83c8074a30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 --2- 172.21.15.86:0/1615379692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c8101de0 0x7f83c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 >> 172.21.15.86:0/1615379692 conn(0x7f83c80fd8c0 msgr2=0x7f83c80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 shutdown_connections 2024-09-05T06:47:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:07.240+0000 7f83ceed0640 1 -- 172.21.15.86:0/1615379692 wait complete. 2024-09-05T06:47:07.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.95 MB/sec execute 147 sec latency 7651.957 ms 2024-09-05T06:47:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:07 smithi096 ceph-mon[34468]: pgmap v179: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-09-05T06:47:07.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:07 smithi086 ceph-mon[30943]: pgmap v179: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-09-05T06:47:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:07 smithi115 ceph-mon[34791]: pgmap v179: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-09-05T06:47:08.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:47:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:08.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.94 MB/sec execute 148 sec latency 8652.052 ms 2024-09-05T06:47:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 -- 172.21.15.86:0/3074718612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106f10 msgr2=0x7efe5c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 --2- 172.21.15.86:0/3074718612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106f10 0x7efe5c10ea90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7efe54009ca0 tx=0x7efe5402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 -- 172.21.15.86:0/3074718612 shutdown_connections 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 --2- 172.21.15.86:0/3074718612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106f10 0x7efe5c10ea90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 --2- 172.21.15.86:0/3074718612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c102790 0x7efe5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 --2- 172.21.15.86:0/3074718612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c101de0 0x7efe5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 -- 172.21.15.86:0/3074718612 >> 172.21.15.86:0/3074718612 conn(0x7efe5c0fd8c0 msgr2=0x7efe5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 -- 172.21.15.86:0/3074718612 shutdown_connections 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.795+0000 7efe60db7640 1 -- 172.21.15.86:0/3074718612 wait complete. 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 Processor -- start 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 -- start start 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 0x7efe5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 0x7efe5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c111e40 con 0x7efe5c102790 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c111cc0 con 0x7efe5c106f10 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe60db7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c111fc0 con 0x7efe5c101de0 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe59d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52420/0 (socket says 172.21.15.86:52420) 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe59d9b640 1 -- 172.21.15.86:0/311912394 learned_addr learned my addr 172.21.15.86:0/311912394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe5a59c640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 0x7efe5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.796+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 0x7efe5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 -- 172.21.15.86:0/311912394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 msgr2=0x7efe5c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 0x7efe5c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 -- 172.21.15.86:0/311912394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 msgr2=0x7efe5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 0x7efe5c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 -- 172.21.15.86:0/311912394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a67e0 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe5a59c640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 0x7efe5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 0x7efe5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe59d9b640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c920 tx=0x7efe4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe537fe640 1 -- 172.21.15.86:0/311912394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44007c70 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a6ad0 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe537fe640 1 -- 172.21.15.86:0/311912394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe44007e10 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe537fe640 1 -- 172.21.15.86:0/311912394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44018820 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.797+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a7060 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.799+0000 7efe537fe640 1 -- 172.21.15.86:0/311912394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efe440189c0 con 0x7efe5c102790 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.799+0000 7efe63190740 1 Processor -- start 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 -- start start 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b378370 0x55b53b37a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b53b2d32f0 con 0x55b53b36eb40 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b53b27acf0 con 0x55b53b36d5d0 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe63190740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b53b2cf2b0 con 0x55b53b378370 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe537fe640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 0x7efe480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52424/0 (socket says 172.21.15.86:52424) 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2884752122 learned_addr learned my addr 172.21.15.86:0/2884752122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50354/0 (socket says 172.21.15.86:50354) 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 0x7efe480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe537fe640 1 -- 172.21.15.86:0/311912394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efe440cfa30 con 0x7efe5c102790 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe59d9b640 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b378370 0x55b53b37a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2884752122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b378370 msgr2=0x55b53b37a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b378370 0x55b53b37a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2884752122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 msgr2=0x55b53b36da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2884752122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b53b25f3a0 con 0x55b53b36eb40 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.800+0000 7efe5a59c640 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe54032b00 tx=0x7efe54032b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4aa4b51dde99d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe50ff9640 1 -- 172.21.15.86:0/2884752122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54003970 con 0x55b53b36eb40 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe50ff9640 1 -- 172.21.15.86:0/2884752122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe54036050 con 0x55b53b36eb40 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 0x7efe480a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe5404c000 tx=0x7efe5404d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe50ff9640 1 -- 172.21.15.86:0/2884752122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540305c0 con 0x55b53b36eb40 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe63190740 1 -- 172.21.15.86:0/2884752122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 msgr2=0x55b53b36d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.801+0000 7efe63190740 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe54032b00 tx=0x7efe54032b30 comp rx=0 tx=0).stop 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- 172.21.15.86:0/2884752122 shutdown_connections 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b378370 0x55b53b37a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b36da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- 172.21.15.86:0/2884752122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b36eb40 0x55b53b36d090 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- 172.21.15.86:0/2884752122 >> 172.21.15.86:0/2884752122 conn(0x55b53b336310 msgr2=0x55b53b369c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- 172.21.15.86:0/2884752122 shutdown_connections 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- 172.21.15.86:0/2884752122 wait complete. 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- start start 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 Processor -- start 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36d5d0 0x55b53b377d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 0x55b53b375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b53b37aef0 con 0x55b53b378370 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b53b28e060 con 0x55b53b36eb40 2024-09-05T06:47:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b53b37b070 con 0x55b53b36d5d0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe63190740 10 client.0 _ll_register_callbacks cb 0x55b53afe18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52438/0 (socket says 172.21.15.86:52438) 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.802+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2009880809 learned_addr learned my addr 172.21.15.86:0/2009880809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5a59c640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36d5d0 0x55b53b377d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe59d9b640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 0x55b53b375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2009880809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36d5d0 msgr2=0x55b53b377d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36d5d0 0x55b53b377d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 msgr2=0x55b53b375010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 0x55b53b375010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 -- 172.21.15.86:0/2009880809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b53b0443c0 con 0x55b53b378370 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe59d9b640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 0x55b53b375010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efe540097b0 tx=0x7efe54003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812af8f1d3d1da58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54059070 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efe54030820 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe54036050 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe54064050 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe63190740 10 client.16854 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b53b37aef0 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.803+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe54036050 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efe5402f4c0 con 0x55b53b378370 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 10 client.16854 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 20 client.16854 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b389f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 4 client.16854 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b53b2e0860 con 0x55b53b387b10 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe5a59c640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b389f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe41ffb640 10 client.16854 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.804+0000 7efe5a59c640 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b389f00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cee8ef2d056306 server_cookie=2737665caf66e01 in_seq=0 out_seq=0 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe41ffb640 1 -- 172.21.15.86:0/2009880809 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b53b2e0860 con 0x55b53b387b10 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe41ffb640 10 client.16854 handle_command_reply: tid=0 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 client.16854 shutdown 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 msgr2=0x55b53b375600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efe540097b0 tx=0x7efe54003ba0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 shutdown_connections 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b389f00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b375600 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36eb40 0x55b53b375010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe41ffb640 0 client.16854 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 --2- 172.21.15.86:0/2009880809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36d5d0 0x55b53b377d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 >> 172.21.15.86:0/2009880809 conn(0x55b53b336310 msgr2=0x55b53b36f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 shutdown_connections 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 1 -- 172.21.15.86:0/2009880809 wait complete. 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.805+0000 7efe63190740 20 client.16854 trim_cache size 0 max 16384 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 Processor -- start 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 -- start start 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b384190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b370120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1b11e0 con 0x55b53b378370 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1b2d20 con 0x55b53b36d5d0 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe63190740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1b18b0 con 0x55b53b36eb40 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52452/0 (socket says 172.21.15.86:52452) 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe5ad9d640 1 -- 172.21.15.86:0/3655029670 learned_addr learned my addr 172.21.15.86:0/3655029670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe59d9b640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b370120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.818+0000 7efe5a59c640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b384190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 -- 172.21.15.86:0/3655029670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 msgr2=0x55b53b370120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b370120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 -- 172.21.15.86:0/3655029670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 msgr2=0x55b53b384190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b384190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 -- 172.21.15.86:0/3655029670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b53b369ed0 con 0x55b53b378370 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe59d9b640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b370120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5a59c640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b384190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe5402a900 tx=0x7efe54009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9972a4e8075dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe43fff640 1 -- 172.21.15.86:0/3655029670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54059030 con 0x55b53b378370 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe43fff640 1 -- 172.21.15.86:0/3655029670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe54036070 con 0x55b53b378370 2024-09-05T06:47:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe43fff640 1 -- 172.21.15.86:0/3655029670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5402f660 con 0x55b53b378370 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe63190740 1 -- 172.21.15.86:0/3655029670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 msgr2=0x55b53b370660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.819+0000 7efe63190740 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe5402a900 tx=0x7efe54009540 comp rx=0 tx=0).stop 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- 172.21.15.86:0/3655029670 shutdown_connections 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b370660 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b370120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- 172.21.15.86:0/3655029670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b384190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- 172.21.15.86:0/3655029670 >> 172.21.15.86:0/3655029670 conn(0x55b53b336310 msgr2=0x55b53b377090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- 172.21.15.86:0/3655029670 shutdown_connections 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- 172.21.15.86:0/3655029670 wait complete. 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- start start 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 Processor -- start 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b380990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b37bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1b2650 con 0x55b53b378370 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1b1f80 con 0x55b53b36d5d0 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1af1c0 con 0x55b53b36eb40 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe63190740 10 client.0 _ll_register_callbacks cb 0x7efe5404b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe5ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52458/0 (socket says 172.21.15.86:52458) 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe5ad9d640 1 -- 172.21.15.86:0/1185852660 learned_addr learned my addr 172.21.15.86:0/1185852660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe59d9b640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b37bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.820+0000 7efe5a59c640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b380990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 -- 172.21.15.86:0/1185852660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 msgr2=0x55b53b37bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b37bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 -- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 msgr2=0x55b53b380990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b380990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 -- 172.21.15.86:0/1185852660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b53b37c9e0 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5a59c640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b380990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe5ad9d640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe54003c30 tx=0x7efe54003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9619c3389295353 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54055070 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7efe54004250 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe54036070 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe54059070 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 10 client.16860 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b53b3fae30 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe54036070 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efe54030820 con 0x55b53b378370 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 10 client.16860 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 20 client.16860 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b3fd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.821+0000 7efe63190740 4 client.16860 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b53b2dad70 con 0x55b53b387b10 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe5a59c640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b3fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe41ffb640 10 client.16860 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe5a59c640 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b3fd310 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2987e24c8a47c25 server_cookie=4c3001a296b7f538 in_seq=0 out_seq=0 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe41ffb640 1 -- 172.21.15.86:0/1185852660 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b53b2dad70 con 0x55b53b387b10 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe41ffb640 10 client.16860 handle_command_reply: tid=0 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.822+0000 7efe63190740 1 client.16860 shutdown 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 msgr2=0x55b53b37c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe54003c30 tx=0x7efe54003c60 comp rx=0 tx=0).stop 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 shutdown_connections 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b53b387b10 0x55b53b3fd310 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe41ffb640 0 client.16860 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b53b378370 0x55b53b37c200 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b53b36eb40 0x55b53b37bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 --2- 172.21.15.86:0/1185852660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b53b36d5d0 0x55b53b380990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 >> 172.21.15.86:0/1185852660 conn(0x55b53b336310 msgr2=0x55b53b3737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 shutdown_connections 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 1 -- 172.21.15.86:0/1185852660 wait complete. 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe63190740 20 client.16860 trim_cache size 0 max 16384 2024-09-05T06:47:08.840 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 msgr2=0x7efe480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 0x7efe480a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe5404c000 tx=0x7efe5404d000 comp rx=0 tx=0).stop 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 msgr2=0x7efe5c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.823+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c920 tx=0x7efe4400cde0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 shutdown_connections 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efe480a2cf0 0x7efe480a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe5c106f10 0x7efe5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c102790 0x7efe5c074a30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 --2- 172.21.15.86:0/311912394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c101de0 0x7efe5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:08.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 >> 172.21.15.86:0/311912394 conn(0x7efe5c0fd8c0 msgr2=0x7efe5c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:08.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 shutdown_connections 2024-09-05T06:47:08.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:08.824+0000 7efe60db7640 1 -- 172.21.15.86:0/311912394 wait complete. 2024-09-05T06:47:09.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.92 MB/sec execute 149 sec latency 9652.087 ms 2024-09-05T06:47:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:09 smithi096 ceph-mon[34468]: pgmap v180: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 346 KiB/s wr, 77 op/s 2024-09-05T06:47:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:09 smithi115 ceph-mon[34791]: pgmap v180: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 346 KiB/s wr, 77 op/s 2024-09-05T06:47:09.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:09 smithi086 ceph-mon[30943]: pgmap v180: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 346 KiB/s wr, 77 op/s 2024-09-05T06:47:10.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.91 MB/sec execute 150 sec latency 10652.136 ms 2024-09-05T06:47:10.556 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:47:10.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:47:10.809 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 -- 172.21.15.86:0/2349282343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 msgr2=0x7f477810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 --2- 172.21.15.86:0/2349282343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477810ea90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f477002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 -- 172.21.15.86:0/2349282343 shutdown_connections 2024-09-05T06:47:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 --2- 172.21.15.86:0/2349282343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477810ea90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 --2- 172.21.15.86:0/2349282343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778102790 0x7f47781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 --2- 172.21.15.86:0/2349282343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778101de0 0x7f47781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 -- 172.21.15.86:0/2349282343 >> 172.21.15.86:0/2349282343 conn(0x7f47780fd8c0 msgr2=0x7f47780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 -- 172.21.15.86:0/2349282343 shutdown_connections 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.299+0000 7f477dd4e640 1 -- 172.21.15.86:0/2349282343 wait complete. 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 Processor -- start 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 -- start start 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778101de0 0x7f47781982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778102790 0x7f47781987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111e70 con 0x7f4778106f10 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111cf0 con 0x7f4778102790 2024-09-05T06:47:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f477dd4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4778111ff0 con 0x7f4778101de0 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f4777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f4777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52474/0 (socket says 172.21.15.86:52474) 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.300+0000 7f4777fff640 1 -- 172.21.15.86:0/3684574475 learned_addr learned my addr 172.21.15.86:0/3684574475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f476ffff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778102790 0x7f47781987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f47777fe640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778101de0 0x7f47781982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 -- 172.21.15.86:0/3684574475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778101de0 msgr2=0x7f47781982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778101de0 0x7f47781982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 -- 172.21.15.86:0/3684574475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778102790 msgr2=0x7f47781987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778102790 0x7f47781987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 -- 172.21.15.86:0/3684574475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477819d260 con 0x7f4778106f10 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f4777fff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4770031040 tx=0x7f4770003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f47757fa640 1 -- 172.21.15.86:0/3684574475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770003ce0 con 0x7f4778106f10 2024-09-05T06:47:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f47757fa640 1 -- 172.21.15.86:0/3684574475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47700044f0 con 0x7f4778106f10 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477819d4f0 con 0x7f4778106f10 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f47757fa640 1 -- 172.21.15.86:0/3684574475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770030bb0 con 0x7f4778106f10 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.301+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a4e30 con 0x7f4778106f10 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.303+0000 7f47757fa640 1 -- 172.21.15.86:0/3684574475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47700046a0 con 0x7f4778106f10 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 Processor -- start 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 -- start start 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f47757fa640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 0x7f47640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76da40 0x559dcb76de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76e360 0x559dcb77cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6d3280 con 0x559dcb76da40 2024-09-05T06:47:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb67ab10 con 0x559dcb77d150 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4780127740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6cf240 con 0x559dcb76e360 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f476ffff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 0x7f47640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f47757fa640 1 -- 172.21.15.86:0/3684574475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f47700f0c70 con 0x7f4778106f10 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f47777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76da40 0x559dcb76de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4777fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4777fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50384/0 (socket says 172.21.15.86:50384) 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f4777fff640 1 -- 172.21.15.86:0/1386954378 learned_addr learned my addr 172.21.15.86:0/1386954378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.304+0000 7f476ffff640 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76e360 0x559dcb77cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 -- 172.21.15.86:0/1386954378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76e360 msgr2=0x559dcb77cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76e360 0x559dcb77cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 -- 172.21.15.86:0/1386954378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76da40 msgr2=0x559dcb76de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76da40 0x559dcb76de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 -- 172.21.15.86:0/1386954378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcb6871f0 con 0x559dcb77d150 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f476ffff640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 0x7f47640a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f476000a5c0 tx=0x7f4760013940 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f4777fff640 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4770005e30 tx=0x7f47700bcb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461aac2a69d0d24b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f476dffb640 1 -- 172.21.15.86:0/1386954378 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700393b0 con 0x559dcb77d150 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f476dffb640 1 -- 172.21.15.86:0/1386954378 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47700d7020 con 0x559dcb77d150 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.305+0000 7f476dffb640 1 -- 172.21.15.86:0/1386954378 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700bcf80 con 0x559dcb77d150 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- 172.21.15.86:0/1386954378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 msgr2=0x559dcb77f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4770005e30 tx=0x7f47700bcb80 comp rx=0 tx=0).stop 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- 172.21.15.86:0/1386954378 shutdown_connections 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77f540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76e360 0x559dcb77cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- 172.21.15.86:0/1386954378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76da40 0x559dcb76de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- 172.21.15.86:0/1386954378 >> 172.21.15.86:0/1386954378 conn(0x559dcb76a0f0 msgr2=0x559dcb76a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- 172.21.15.86:0/1386954378 shutdown_connections 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- 172.21.15.86:0/1386954378 wait complete. 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- start start 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 Processor -- start 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb77a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559dcb67ab10 con 0x559dcb76e360 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb68df10 con 0x559dcb77d150 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.306+0000 7f4780127740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559dcb77fae0 con 0x559dcb76da40 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4780127740 10 client.0 _ll_register_callbacks cb 0x559dcb656a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52502/0 (socket says 172.21.15.86:52502) 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 -- 172.21.15.86:0/3990140593 learned_addr learned my addr 172.21.15.86:0/3990140593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4777fff640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f47777fe640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb77a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 -- 172.21.15.86:0/3990140593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 msgr2=0x559dcb77a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb77a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 -- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 msgr2=0x559dcb77ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 -- 172.21.15.86:0/3990140593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcb4446a0 con 0x559dcb76e360 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4777fff640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f47777fe640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb77a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f476ffff640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4760011de0 tx=0x7f4760004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda60e7f0b6ee334 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476002a680 con 0x559dcb76e360 2024-09-05T06:47:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f476001f450 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47700d7020 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f476002a820 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4780127740 10 client.16872 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.307+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dcb78b560 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47700d7020 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f476001fa60 con 0x559dcb76e360 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 10 client.16872 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 20 client.16872 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb78ca20 0x559dcb78eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 4 client.16872 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dcb6e07f0 con 0x559dcb78ca20 2024-09-05T06:47:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.308+0000 7f47777fe640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb78ca20 0x559dcb78eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f47777fe640 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb78ca20 0x559dcb78eec0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d669db1426525d server_cookie=2bf4bf2ef29500f9 in_seq=0 out_seq=0 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4756ffd640 10 client.16872 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4756ffd640 1 -- 172.21.15.86:0/3990140593 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559dcb6e07f0 con 0x559dcb78ca20 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4756ffd640 10 client.16872 handle_command_reply: tid=0 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4780127740 1 client.16872 shutdown 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 msgr2=0x559dcb77a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.309+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4760011de0 tx=0x7f4760004730 comp rx=0 tx=0).stop 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 shutdown_connections 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb78ca20 0x559dcb78eec0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb77d150 0x559dcb77ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb77a5e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4756ffd640 0 client.16872 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 --2- 172.21.15.86:0/3990140593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb77a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 >> 172.21.15.86:0/3990140593 conn(0x559dcb76a0f0 msgr2=0x559dcb76fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 shutdown_connections 2024-09-05T06:47:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 1 -- 172.21.15.86:0/3990140593 wait complete. 2024-09-05T06:47:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.310+0000 7f4780127740 20 client.16872 trim_cache size 0 max 16384 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.327+0000 7f4780127740 1 Processor -- start 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.327+0000 7f4780127740 1 -- start start 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.327+0000 7f4780127740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb7947d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4780127740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76e360 0x559dcb794d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4780127740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4780127740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47781acdd0 con 0x559dcb77d150 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4780127740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47781ae910 con 0x559dcb76e360 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4780127740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47781ad4a0 con 0x559dcb76da40 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52518/0 (socket says 172.21.15.86:52518) 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 -- 172.21.15.86:0/237146581 learned_addr learned my addr 172.21.15.86:0/237146581 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f47777fe640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb7947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f476ffff640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76e360 0x559dcb794d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 -- 172.21.15.86:0/237146581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 msgr2=0x559dcb7947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb7947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 -- 172.21.15.86:0/237146581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76e360 msgr2=0x559dcb794d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76e360 0x559dcb794d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 -- 172.21.15.86:0/237146581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcb78f540 con 0x559dcb77d150 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f47777fe640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb7947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f4777fff640 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f47700bb230 tx=0x7f477002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d8fe7d72f585cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f476d7fa640 1 -- 172.21.15.86:0/237146581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700098b0 con 0x559dcb77d150 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f476d7fa640 1 -- 172.21.15.86:0/237146581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47700d7070 con 0x559dcb77d150 2024-09-05T06:47:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.328+0000 7f476d7fa640 1 -- 172.21.15.86:0/237146581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770044d20 con 0x559dcb77d150 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 -- 172.21.15.86:0/237146581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 msgr2=0x559dcb777300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f47700bb230 tx=0x7f477002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 -- 172.21.15.86:0/237146581 shutdown_connections 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb77d150 0x559dcb777300 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76e360 0x559dcb794d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 --2- 172.21.15.86:0/237146581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb76da40 0x559dcb7947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 -- 172.21.15.86:0/237146581 >> 172.21.15.86:0/237146581 conn(0x559dcb76a0f0 msgr2=0x559dcb77c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 -- 172.21.15.86:0/237146581 shutdown_connections 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.329+0000 7f4780127740 1 -- 172.21.15.86:0/237146581 wait complete. 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 -- start start 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 Processor -- start 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 0x559dcb7851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 0x559dcb781a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47781ae240 con 0x559dcb76e360 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47781adb70 con 0x559dcb76da40 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47781aadb0 con 0x559dcb77d150 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4780127740 10 client.0 _ll_register_callbacks cb 0x7f476000aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f476ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f476ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52532/0 (socket says 172.21.15.86:52532) 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f476ffff640 1 -- 172.21.15.86:0/1978017898 learned_addr learned my addr 172.21.15.86:0/1978017898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f47777fe640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 0x559dcb7851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f476ffff640 1 -- 172.21.15.86:0/1978017898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 msgr2=0x559dcb781a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.330+0000 7f4777fff640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 0x559dcb781a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f476ffff640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 0x559dcb781a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f476ffff640 1 -- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 msgr2=0x559dcb7851d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f476ffff640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 0x559dcb7851d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f476ffff640 1 -- 172.21.15.86:0/1978017898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcb781f90 con 0x559dcb76e360 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4777fff640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 0x559dcb781a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f47777fe640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 0x559dcb7851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f476ffff640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f476001fe80 tx=0x7f47600058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bb5e4c3d6dd564 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476003b020 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47700d7070 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f476002add0 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4760046050 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4780127740 10 client.16878 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dcb7f8540 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47700d7070 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.331+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4760037580 con 0x559dcb76e360 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 10 client.16878 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 20 client.16878 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb7f8ce0 0x559dcb7fb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 4 client.16878 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:47:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559dcb6dad00 con 0x559dcb7f8ce0 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f47777fe640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb7f8ce0 0x559dcb7fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f47777fe640 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb7f8ce0 0x559dcb7fb100 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca95f3c74b1aacd0 server_cookie=ff70f25122a99837 in_seq=0 out_seq=0 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.332+0000 7f4756ffd640 10 client.16878 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.333+0000 7f4756ffd640 1 -- 172.21.15.86:0/1978017898 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559dcb6dad00 con 0x559dcb7f8ce0 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.333+0000 7f4756ffd640 10 client.16878 handle_command_reply: tid=0 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.333+0000 7f4780127740 1 client.16878 shutdown 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.333+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 msgr2=0x559dcb781510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.333+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f476001fe80 tx=0x7f47600058d0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 shutdown_connections 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559dcb7f8ce0 0x559dcb7fb100 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559dcb77d150 0x559dcb781a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcb76e360 0x559dcb781510 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 --2- 172.21.15.86:0/1978017898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb76da40 0x559dcb7851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 >> 172.21.15.86:0/1978017898 conn(0x559dcb76a0f0 msgr2=0x559dcb768a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4756ffd640 0 client.16878 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 shutdown_connections 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 1 -- 172.21.15.86:0/1978017898 wait complete. 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f4780127740 20 client.16878 trim_cache size 0 max 16384 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "4c663d94-7b86-4461-a02e-65c76a2303da", 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:47:11.339 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:47:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 msgr2=0x7f47640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 0x7f47640a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f476000a5c0 tx=0x7f4760013940 comp rx=0 tx=0).stop 2024-09-05T06:47:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 msgr2=0x7f477819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4770031040 tx=0x7f4770003970 comp rx=0 tx=0).stop 2024-09-05T06:47:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 shutdown_connections 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47640a2d70 0x7f47640a5230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4778106f10 0x7f477819cb80 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4778102790 0x7f47781987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 --2- 172.21.15.86:0/3684574475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4778101de0 0x7f47781982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 >> 172.21.15.86:0/3684574475 conn(0x7f47780fd8c0 msgr2=0x7f47780feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 shutdown_connections 2024-09-05T06:47:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:11.334+0000 7f477dd4e640 1 -- 172.21.15.86:0/3684574475 wait complete. 2024-09-05T06:47:11.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.90 MB/sec execute 151 sec latency 11652.214 ms 2024-09-05T06:47:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:11 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:47:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:11 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:11 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:11 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:11 smithi086 ceph-mon[30943]: pgmap v181: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 354 KiB/s wr, 82 op/s 2024-09-05T06:47:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:11 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:47:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:11 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:11 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:11 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:11 smithi096 ceph-mon[34468]: pgmap v181: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 354 KiB/s wr, 82 op/s 2024-09-05T06:47:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:11 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:47:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:11 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:11 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:11 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:11 smithi115 ceph-mon[34791]: pgmap v181: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 354 KiB/s wr, 82 op/s 2024-09-05T06:47:12.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:47:12.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.89 MB/sec execute 152 sec latency 12652.240 ms 2024-09-05T06:47:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:12 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:12 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:12.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:12 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 -- 172.21.15.86:0/4179126761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c102ef0 msgr2=0x7f146c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 --2- 172.21.15.86:0/4179126761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c102ef0 0x7f146c10da40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1464009900 tx=0x7f146402a480 comp rx=0 tx=0).stop 2024-09-05T06:47:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 -- 172.21.15.86:0/4179126761 shutdown_connections 2024-09-05T06:47:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 --2- 172.21.15.86:0/4179126761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c102ef0 0x7f146c10da40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 --2- 172.21.15.86:0/4179126761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 0x7f146c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 --2- 172.21.15.86:0/4179126761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c100e20 0x7f146c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 -- 172.21.15.86:0/4179126761 >> 172.21.15.86:0/4179126761 conn(0x7f146c0fc7d0 msgr2=0x7f146c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 -- 172.21.15.86:0/4179126761 shutdown_connections 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 -- 172.21.15.86:0/4179126761 wait complete. 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.016+0000 7f14745bf640 1 Processor -- start 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 -- start start 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 0x7f146c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 0x7f146c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111f50 con 0x7f146c100e20 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111dd0 con 0x7f146c102ef0 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f14745bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1120d0 con 0x7f146c102570 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43154/0 (socket says 172.21.15.86:43154) 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 -- 172.21.15.86:0/4275852283 learned_addr learned my addr 172.21.15.86:0/4275852283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f1471b5a640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 0x7f146c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f1472b5c640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 0x7f146c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 -- 172.21.15.86:0/4275852283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 msgr2=0x7f146c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 0x7f146c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 -- 172.21.15.86:0/4275852283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 msgr2=0x7f146c10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 0x7f146c10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 -- 172.21.15.86:0/4275852283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c1a6810 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f1472b5c640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 0x7f146c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f147235b640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f145c00cce0 tx=0x7f145c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.017+0000 7f146b7fe640 1 -- 172.21.15.86:0/4275852283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00f650 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.018+0000 7f146b7fe640 1 -- 172.21.15.86:0/4275852283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c00fca0 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.018+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c1a6b00 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.018+0000 7f146b7fe640 1 -- 172.21.15.86:0/4275852283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c018b90 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.018+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1a7040 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.020+0000 7f1476998740 1 Processor -- start 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.020+0000 7f146b7fe640 1 -- 172.21.15.86:0/4275852283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f145c00f7f0 con 0x7f146c100e20 2024-09-05T06:47:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 -- start start 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d457122e0 con 0x560d457b7680 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d456b9ce0 con 0x560d457acdc0 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1476998740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d4570e2a0 con 0x560d457ac4a0 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43162/0 (socket says 172.21.15.86:43162) 2024-09-05T06:47:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 -- 172.21.15.86:0/4000320114 learned_addr learned my addr 172.21.15.86:0/4000320114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f147235b640 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1472b5c640 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 -- 172.21.15.86:0/4000320114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 msgr2=0x560d457ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 -- 172.21.15.86:0/4000320114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 msgr2=0x560d457b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f1471b5a640 1 -- 172.21.15.86:0/4000320114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4569e440 con 0x560d457b7680 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.021+0000 7f146b7fe640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 0x7f14540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1472b5c640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 0x7f14540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1471b5a640 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f146000b690 tx=0x7f146000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dff9576f5945aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f146b7fe640 1 -- 172.21.15.86:0/4275852283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f145c0cfd00 con 0x7f146c100e20 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1468ff9640 1 -- 172.21.15.86:0/4000320114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460003bd0 con 0x560d457b7680 2024-09-05T06:47:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1468ff9640 1 -- 172.21.15.86:0/4000320114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1460015070 con 0x560d457b7680 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1468ff9640 1 -- 172.21.15.86:0/4000320114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460004ce0 con 0x560d457b7680 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1476998740 1 -- 172.21.15.86:0/4000320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 msgr2=0x560d457b9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1476998740 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f146000b690 tx=0x7f146000bb50 comp rx=0 tx=0).stop 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.022+0000 7f1472b5c640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 0x7f14540a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1464005ab0 tx=0x7f1464005b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- 172.21.15.86:0/4000320114 shutdown_connections 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457b7680 0x560d457b9a70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- 172.21.15.86:0/4000320114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- 172.21.15.86:0/4000320114 >> 172.21.15.86:0/4000320114 conn(0x560d457a8a30 msgr2=0x560d457a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- 172.21.15.86:0/4000320114 shutdown_connections 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- 172.21.15.86:0/4000320114 wait complete. 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- start start 2024-09-05T06:47:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 Processor -- start 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 0x560d457b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d457ba250 con 0x560d457ac4a0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d456cd050 con 0x560d457acdc0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d457ba3d0 con 0x560d457b7680 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f1476998740 10 client.0 _ll_register_callbacks cb 0x560d4570d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43172/0 (socket says 172.21.15.86:43172) 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.023+0000 7f147235b640 1 -- 172.21.15.86:0/666973013 learned_addr learned my addr 172.21.15.86:0/666973013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1472b5c640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 0x560d457b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1471b5a640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 -- 172.21.15.86:0/666973013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 msgr2=0x560d457b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 0x560d457b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 -- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 msgr2=0x560d457b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 -- 172.21.15.86:0/666973013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d454833c0 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1472b5c640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 0x560d457b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f147235b640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f145c00ccb0 tx=0x7f145c007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ec6215a28603e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c09b980 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f145c0a4620 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1460015070 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f145c09b7a0 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1476998740 10 client.16890 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d457c5bc0 con 0x560d457ac4a0 2024-09-05T06:47:13.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.024+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1460015070 con 0x560d457ac4a0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f145c097b90 con 0x560d457ac4a0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 10 client.16890 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 20 client.16890 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457c6fd0 0x560d457c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 4 client.16890 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4571f850 con 0x560d457c6fd0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1471b5a640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457c6fd0 0x560d457c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1471b5a640 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457c6fd0 0x560d457c9440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91ee76e935fe0a4 server_cookie=ff701e3635e27768 in_seq=0 out_seq=0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.025+0000 7f1451ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1451ffb640 1 -- 172.21.15.86:0/666973013 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d4571f850 con 0x560d457c6fd0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1451ffb640 10 client.16890 handle_command_reply: tid=0 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 client.16890 shutdown 2024-09-05T06:47:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 msgr2=0x560d457b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f145c00ccb0 tx=0x7f145c007850 comp rx=0 tx=0).stop 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 shutdown_connections 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457c6fd0 0x560d457c9440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457b7680 0x560d457b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d457b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 --2- 172.21.15.86:0/666973013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457ac4a0 0x560d457b6f00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 >> 172.21.15.86:0/666973013 conn(0x560d457a8a30 msgr2=0x560d457ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1451ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 shutdown_connections 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 1 -- 172.21.15.86:0/666973013 wait complete. 2024-09-05T06:47:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.026+0000 7f1476998740 20 client.16890 trim_cache size 0 max 16384 2024-09-05T06:47:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 Processor -- start 2024-09-05T06:47:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 -- start start 2024-09-05T06:47:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457ac4a0 0x560d457b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 0x560d457a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b10c0 con 0x560d457a8a30 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b2c00 con 0x560d457ac4a0 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f1476998740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b1790 con 0x560d457acdc0 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.043+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43176/0 (socket says 172.21.15.86:43176) 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 -- 172.21.15.86:0/1728257420 learned_addr learned my addr 172.21.15.86:0/1728257420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f1472b5c640 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 0x560d457a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 -- 172.21.15.86:0/1728257420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 msgr2=0x560d457a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 0x560d457a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 -- 172.21.15.86:0/1728257420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457ac4a0 msgr2=0x560d457b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457ac4a0 0x560d457b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 -- 172.21.15.86:0/1728257420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d457b3160 con 0x560d457a8a30 2024-09-05T06:47:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f1472b5c640 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 0x560d457a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:13.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f147235b640 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f145c09a0f0 tx=0x7f145c09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0eee0d36237ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f1453fff640 1 -- 172.21.15.86:0/1728257420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c09e030 con 0x560d457a8a30 2024-09-05T06:47:13.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f1453fff640 1 -- 172.21.15.86:0/1728257420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c0ba020 con 0x560d457a8a30 2024-09-05T06:47:13.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.044+0000 7f1453fff640 1 -- 172.21.15.86:0/1728257420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c09be40 con 0x560d457a8a30 2024-09-05T06:47:13.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- 172.21.15.86:0/1728257420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 msgr2=0x560d457c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f145c09a0f0 tx=0x7f145c09bcd0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- 172.21.15.86:0/1728257420 shutdown_connections 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457acdc0 0x560d457a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457ac4a0 0x560d457b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- 172.21.15.86:0/1728257420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457c34b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- 172.21.15.86:0/1728257420 >> 172.21.15.86:0/1728257420 conn(0x560d457b4850 msgr2=0x560d457a7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- 172.21.15.86:0/1728257420 shutdown_connections 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- 172.21.15.86:0/1728257420 wait complete. 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- start start 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 Processor -- start 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d458394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b2530 con 0x560d457a8a30 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b1e60 con 0x560d457acdc0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1af0a0 con 0x560d457ac4a0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.045+0000 7f1476998740 10 client.0 _ll_register_callbacks cb 0x7f145c00ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43180/0 (socket says 172.21.15.86:43180) 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 -- 172.21.15.86:0/351857079 learned_addr learned my addr 172.21.15.86:0/351857079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1471b5a640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1472b5c640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d458394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 -- 172.21.15.86:0/351857079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 msgr2=0x560d457bbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457bbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 -- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 msgr2=0x560d458394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d458394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 -- 172.21.15.86:0/351857079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d457bc4c0 con 0x560d457a8a30 2024-09-05T06:47:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1472b5c640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d458394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f147235b640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f145c000f80 tx=0x7f145c09bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c708fe1e22a62b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c09e070 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f145c0a4820 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145c0ba020 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f145c09be40 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1476998740 10 client.16896 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.046+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d4583aa50 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c0ba020 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f145c0a1380 con 0x560d457a8a30 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 20 client.16896 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457b7680 0x560d4583cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 4 client.16896 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d45719d60 con 0x560d457b7680 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1471b5a640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457b7680 0x560d4583cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1451ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1471b5a640 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457b7680 0x560d4583cf30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8379d36f5b30cf server_cookie=384092c3cdc575a8 in_seq=0 out_seq=0 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1451ffb640 1 -- 172.21.15.86:0/351857079 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d45719d60 con 0x560d457b7680 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1451ffb640 10 client.16896 handle_command_reply: tid=0 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.047+0000 7f1476998740 1 client.16896 shutdown 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 msgr2=0x560d457bb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f145c000f80 tx=0x7f145c09bd40 comp rx=0 tx=0).stop 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 shutdown_connections 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d457b7680 0x560d4583cf30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d457acdc0 0x560d458394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d457ac4a0 0x560d457bbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 --2- 172.21.15.86:0/351857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d457a8a30 0x560d457bb6f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1451ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 >> 172.21.15.86:0/351857079 conn(0x560d457b4850 msgr2=0x560d457a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 shutdown_connections 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 1 -- 172.21.15.86:0/351857079 wait complete. 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f1476998740 20 client.16896 trim_cache size 0 max 16384 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:47:13.054 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: "4c663d94-7b86-4461-a02e-65c76a2303da": { 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "4c663d94-7b86-4461-a02e-65c76a2303da", 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:13.055 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 msgr2=0x7f14540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 0x7f14540a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1464005ab0 tx=0x7f1464005b50 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 msgr2=0x7f146c10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f145c00cce0 tx=0x7f145c00a620 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 shutdown_connections 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14540a2cf0 0x7f14540a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c102ef0 0x7f146c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f146c102570 0x7f146c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 --2- 172.21.15.86:0/4275852283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146c100e20 0x7f146c10e4d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.048+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 >> 172.21.15.86:0/4275852283 conn(0x7f146c0fc7d0 msgr2=0x7f146c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.049+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 shutdown_connections 2024-09-05T06:47:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:13.049+0000 7f14745bf640 1 -- 172.21.15.86:0/4275852283 wait complete. 2024-09-05T06:47:13.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.87 MB/sec execute 153 sec latency 13652.311 ms 2024-09-05T06:47:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:13 smithi096 ceph-mon[34468]: pgmap v182: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:13.873 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:13 smithi086 ceph-mon[30943]: pgmap v182: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:13.904 INFO:tasks.cephfs.filesystem:scrub status for tag:4c663d94-7b86-4461-a02e-65c76a2303da - {'path': '/', 'tag': '4c663d94-7b86-4461-a02e-65c76a2303da', 'options': 'recursive,force'} 2024-09-05T06:47:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:13 smithi115 ceph-mon[34791]: pgmap v182: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:14.515 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.86 MB/sec execute 154 sec latency 14652.369 ms 2024-09-05T06:47:15.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.85 MB/sec execute 155 sec latency 15652.421 ms 2024-09-05T06:47:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:15 smithi096 ceph-mon[34468]: pgmap v183: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:15 smithi115 ceph-mon[34791]: pgmap v183: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:15.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:15 smithi086 ceph-mon[30943]: pgmap v183: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-05T06:47:16.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.84 MB/sec execute 156 sec latency 16652.521 ms 2024-09-05T06:47:17.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.83 MB/sec execute 157 sec latency 17652.570 ms 2024-09-05T06:47:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:17 smithi096 ceph-mon[34468]: pgmap v184: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-05T06:47:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:17 smithi115 ceph-mon[34791]: pgmap v184: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-05T06:47:17.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:17 smithi086 ceph-mon[30943]: pgmap v184: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-05T06:47:18.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.81 MB/sec execute 158 sec latency 18652.643 ms 2024-09-05T06:47:19.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.80 MB/sec execute 159 sec latency 19652.681 ms 2024-09-05T06:47:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:19 smithi096 ceph-mon[34468]: pgmap v185: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-05T06:47:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:19 smithi115 ceph-mon[34791]: pgmap v185: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-05T06:47:19.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:19 smithi086 ceph-mon[30943]: pgmap v185: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-09-05T06:47:20.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.79 MB/sec execute 160 sec latency 20652.716 ms 2024-09-05T06:47:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:20 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:47:20.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:20 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:47:20.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:20 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:47:21.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.78 MB/sec execute 161 sec latency 21652.757 ms 2024-09-05T06:47:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:47:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:47:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:21 smithi096 ceph-mon[34468]: pgmap v186: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:47:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:47:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:47:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:21 smithi115 ceph-mon[34791]: pgmap v186: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:47:21.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:47:21.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:47:21.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:21 smithi086 ceph-mon[30943]: pgmap v186: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-05T06:47:22.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.77 MB/sec execute 162 sec latency 22652.834 ms 2024-09-05T06:47:23.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.76 MB/sec execute 163 sec latency 23652.928 ms 2024-09-05T06:47:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:23 smithi096 ceph-mon[34468]: pgmap v187: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:23 smithi115 ceph-mon[34791]: pgmap v187: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:23.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:23 smithi086 ceph-mon[30943]: pgmap v187: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:24.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.75 MB/sec execute 164 sec latency 24653.060 ms 2024-09-05T06:47:25.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.74 MB/sec execute 165 sec latency 25653.119 ms 2024-09-05T06:47:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:25 smithi096 ceph-mon[34468]: pgmap v188: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:25 smithi115 ceph-mon[34791]: pgmap v188: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:25.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:25 smithi086 ceph-mon[30943]: pgmap v188: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-05T06:47:26.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.73 MB/sec execute 166 sec latency 26653.237 ms 2024-09-05T06:47:27.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.72 MB/sec execute 167 sec latency 27653.336 ms 2024-09-05T06:47:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:27 smithi086 ceph-mon[30943]: pgmap v189: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-05T06:47:28.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:27 smithi096 ceph-mon[34468]: pgmap v189: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-05T06:47:28.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:27 smithi115 ceph-mon[34791]: pgmap v189: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-05T06:47:28.516 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.71 MB/sec execute 168 sec latency 28653.484 ms 2024-09-05T06:47:29.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.70 MB/sec execute 169 sec latency 29653.544 ms 2024-09-05T06:47:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:29 smithi096 ceph-mon[34468]: pgmap v190: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-05T06:47:29.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:29 smithi115 ceph-mon[34791]: pgmap v190: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-05T06:47:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:29 smithi086 ceph-mon[30943]: pgmap v190: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-05T06:47:30.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:47:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:30.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.69 MB/sec execute 170 sec latency 30653.575 ms 2024-09-05T06:47:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:30 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:47:30.811 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:30 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:47:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 -- 172.21.15.86:0/2652295574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58106a80 msgr2=0x7f8b58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/2652295574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58106a80 0x7f8b58106e60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009930 tx=0x7f8b4802a420 comp rx=0 tx=0).stop 2024-09-05T06:47:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 -- 172.21.15.86:0/2652295574 shutdown_connections 2024-09-05T06:47:30.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/2652295574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58101420 0x7f8b5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/2652295574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 0x7f8b58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/2652295574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58106a80 0x7f8b58106e60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 -- 172.21.15.86:0/2652295574 >> 172.21.15.86:0/2652295574 conn(0x7f8b580fc7a0 msgr2=0x7f8b580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 -- 172.21.15.86:0/2652295574 shutdown_connections 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.893+0000 7f8b5d88e640 1 -- 172.21.15.86:0/2652295574 wait complete. 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 Processor -- start 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 -- start start 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 0x7f8b580fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 0x7f8b580fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58110cf0 con 0x7f8b58101420 2024-09-05T06:47:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58110b70 con 0x7f8b58100a80 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b5d88e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58110e70 con 0x7f8b58106a80 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43182/0 (socket says 172.21.15.86:43182) 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 -- 172.21.15.86:0/3707221008 learned_addr learned my addr 172.21.15.86:0/3707221008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b56ffd640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 0x7f8b580fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b577fe640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 0x7f8b580fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 -- 172.21.15.86:0/3707221008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 msgr2=0x7f8b580fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 0x7f8b580fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 -- 172.21.15.86:0/3707221008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 msgr2=0x7f8b580fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 0x7f8b580fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b4ffff640 1 -- 172.21.15.86:0/3707221008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b581a6850 con 0x7f8b58101420 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.894+0000 7f8b577fe640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 0x7f8b580fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b4ffff640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000d950 tx=0x7f8b4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40017030 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b581a6b40 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b40004020 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b400047a0 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.895+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b581a7080 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.896+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b58100ee0 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.897+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8b400041d0 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.898+0000 7f8b5c88c640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 0x7f8b440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.898+0000 7f8b56ffd640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 0x7f8b440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.898+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f8b400cf2d0 con 0x7f8b58101420 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.898+0000 7f8b56ffd640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 0x7f8b440a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b48000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:30.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:30.902+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b4009bb10 con 0x7f8b58101420 2024-09-05T06:47:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:30 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:47:31.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.103+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b58109a70 con 0x7f8b58101420 2024-09-05T06:47:31.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.104+0000 7f8b5c88c640 1 -- 172.21.15.86:0/3707221008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f8b4009b410 con 0x7f8b58101420 2024-09-05T06:47:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:47:31.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:47:31.105 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:47:31.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 msgr2=0x7f8b440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:31.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 0x7f8b440a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b48000bc0 comp rx=0 tx=0).stop 2024-09-05T06:47:31.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 msgr2=0x7f8b580fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:31.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000d950 tx=0x7f8b4000de10 comp rx=0 tx=0).stop 2024-09-05T06:47:31.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 shutdown_connections 2024-09-05T06:47:31.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8b440a2cf0 0x7f8b440a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b58106a80 0x7f8b580fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b58101420 0x7f8b580fd440 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 --2- 172.21.15.86:0/3707221008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b58100a80 0x7f8b580fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 >> 172.21.15.86:0/3707221008 conn(0x7f8b580fc7a0 msgr2=0x7f8b5806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 shutdown_connections 2024-09-05T06:47:31.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:31.106+0000 7f8b5d88e640 1 -- 172.21.15.86:0/3707221008 wait complete. 2024-09-05T06:47:31.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.68 MB/sec execute 171 sec latency 31653.602 ms 2024-09-05T06:47:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:47:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:47:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:47:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:31 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3707221008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:31 smithi086 ceph-mon[30943]: pgmap v191: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-05T06:47:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:47:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:47:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:47:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:31 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3707221008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:31 smithi096 ceph-mon[34468]: pgmap v191: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-05T06:47:31.878 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 5117007c --release --await' 2024-09-05T06:47:31.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 5117007c --release --await 2024-09-05T06:47:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:47:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:47:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:47:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:31 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3707221008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:31 smithi115 ceph-mon[34791]: pgmap v191: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-05T06:47:32.135 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:32.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108447 1.67 MB/sec execute 172 sec latency 32653.633 ms 2024-09-05T06:47:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 -- 172.21.15.86:0/1673310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 msgr2=0x7f71f4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 --2- 172.21.15.86:0/1673310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4103110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009cd0 tx=0x7f71e402a910 comp rx=0 tx=0).stop 2024-09-05T06:47:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 -- 172.21.15.86:0/1673310875 shutdown_connections 2024-09-05T06:47:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 --2- 172.21.15.86:0/1673310875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41082a0 0x7f71f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 --2- 172.21.15.86:0/1673310875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41036e0 0x7f71f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 --2- 172.21.15.86:0/1673310875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4103110 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 -- 172.21.15.86:0/1673310875 >> 172.21.15.86:0/1673310875 conn(0x7f71f40fc7d0 msgr2=0x7f71f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 -- 172.21.15.86:0/1673310875 shutdown_connections 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 -- 172.21.15.86:0/1673310875 wait complete. 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.652+0000 7f71f8e35640 1 Processor -- start 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 -- start start 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 0x7f71f4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 0x7f71f419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4112010 con 0x7f71f4102d30 2024-09-05T06:47:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4111e90 con 0x7f71f41036e0 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f8e35640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4112190 con 0x7f71f41082a0 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37202/0 (socket says 172.21.15.86:37202) 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 -- 172.21.15.86:0/2846741544 learned_addr learned my addr 172.21.15.86:0/2846741544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 0x7f71f419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 0x7f71f4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 -- 172.21.15.86:0/2846741544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 msgr2=0x7f71f419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 0x7f71f419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 -- 172.21.15.86:0/2846741544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 msgr2=0x7f71f4198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 0x7f71f4198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 -- 172.21.15.86:0/2846741544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f419d5f0 con 0x7f71f4102d30 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 0x7f71f419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 0x7f71f4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71f259c640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009df0 tx=0x7f71e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.653+0000 7f71e37fe640 1 -- 172.21.15.86:0/2846741544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004040 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.654+0000 7f71e37fe640 1 -- 172.21.15.86:0/2846741544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4004690 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.654+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f419d880 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.654+0000 7f71e37fe640 1 -- 172.21.15.86:0/2846741544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4030e90 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.654+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f41a51c0 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.656+0000 7f71e37fe640 1 -- 172.21.15.86:0/2846741544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f71e4030660 con 0x7f71f4102d30 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.656+0000 7f71fb20e740 1 Processor -- start 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.656+0000 7f71fb20e740 1 -- start start 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.656+0000 7f71e37fe640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 0x7f71dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 0x7f71dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55851d705620 con 0x55851d7a0060 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55851d6ace50 con 0x55851d7a0980 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71fb20e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55851d7015e0 con 0x55851d7ab240 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71e37fe640 1 -- 172.21.15.86:0/2846741544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f71e40f0e00 con 0x7f71f4102d30 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37206/0 (socket says 172.21.15.86:37206) 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 -- 172.21.15.86:0/2171375283 learned_addr learned my addr 172.21.15.86:0/2171375283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 -- 172.21.15.86:0/2171375283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 msgr2=0x55851d7ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 -- 172.21.15.86:0/2171375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 msgr2=0x55851d7aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 -- 172.21.15.86:0/2171375283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d69dd40 con 0x55851d7a0060 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 0x7f71dc0a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00a9e0 tx=0x7f71ec00a340 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71f259c640 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f71e40c5790 tx=0x7f71e4032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f115fd57f26f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71e0ff9640 1 -- 172.21.15.86:0/2171375283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40c1400 con 0x55851d7a0060 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.657+0000 7f71e0ff9640 1 -- 172.21.15.86:0/2171375283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4036040 con 0x55851d7a0060 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71e0ff9640 1 -- 172.21.15.86:0/2171375283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ee3f0 con 0x55851d7a0060 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 -- 172.21.15.86:0/2171375283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 msgr2=0x55851d7a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f71e40c5790 tx=0x7f71e4032fd0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 -- 172.21.15.86:0/2171375283 shutdown_connections 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2171375283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0060 0x55851d7a0440 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 -- 172.21.15.86:0/2171375283 >> 172.21.15.86:0/2171375283 conn(0x55851d79c5f0 msgr2=0x55851d79c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 -- 172.21.15.86:0/2171375283 shutdown_connections 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.658+0000 7f71fb20e740 1 -- 172.21.15.86:0/2171375283 wait complete. 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 -- start start 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 Processor -- start 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 0x55851d7a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55851d7adbd0 con 0x55851d7a0980 2024-09-05T06:47:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55851d68df10 con 0x55851d7a0060 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55851d7add50 con 0x55851d7ab240 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71fb20e740 10 client.0 _ll_register_callbacks cb 0x55851d6b1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37212/0 (socket says 172.21.15.86:37212) 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 -- 172.21.15.86:0/2251717176 learned_addr learned my addr 172.21.15.86:0/2251717176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f259c640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 0x55851d7a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 -- 172.21.15.86:0/2251717176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 msgr2=0x55851d7a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 -- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 msgr2=0x55851d7a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 0x55851d7a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f1d9b640 1 -- 172.21.15.86:0/2251717176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d414580 con 0x55851d7a0980 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.659+0000 7f71f259c640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 0x55851d7a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e2b0 tx=0x7f71e800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f22a5cd7adde08e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8018020 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f71e8013660 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4036040 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71e8023050 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71fb20e740 10 client.16914 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55851d4de960 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4036040 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.660+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f71e80148c0 con 0x55851d7a0980 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 10 client.16914 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 10 client.16914 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 20 client.16914 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d7babc0 0x55851d7bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 4 client.16914 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55851d712b90 con 0x55851d7babc0 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71f259c640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d7babc0 0x55851d7bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71f259c640 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d7babc0 0x55851d7bd060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9173a4bd38bcb65f server_cookie=86aceb546a6a999d in_seq=0 out_seq=0 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.661+0000 7f71d9ffb640 10 client.16914 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71d9ffb640 1 -- 172.21.15.86:0/2251717176 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55851d712b90 con 0x55851d7babc0 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71d9ffb640 10 client.16914 handle_command_reply: tid=0 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 client.16914 shutdown 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 msgr2=0x55851d7a86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e2b0 tx=0x7f71e800e770 comp rx=0 tx=0).stop 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 shutdown_connections 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d7babc0 0x55851d7bd060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7ab240 0x55851d7a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7a0980 0x55851d7a86e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 --2- 172.21.15.86:0/2251717176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0060 0x55851d7a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 >> 172.21.15.86:0/2251717176 conn(0x55851d79c5f0 msgr2=0x55851d79b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71d9ffb640 0 client.16914 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 shutdown_connections 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 1 -- 172.21.15.86:0/2251717176 wait complete. 2024-09-05T06:47:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.662+0000 7f71fb20e740 20 client.16914 trim_cache size 0 max 16384 2024-09-05T06:47:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 Processor -- start 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 -- start start 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41af210 con 0x55851d7ab240 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b0d50 con 0x55851d7a0980 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.679+0000 7f71fb20e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41af8e0 con 0x55851d7a0060 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37226/0 (socket says 172.21.15.86:37226) 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 -- 172.21.15.86:0/343559327 learned_addr learned my addr 172.21.15.86:0/343559327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f259c640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 -- 172.21.15.86:0/343559327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 msgr2=0x55851d7c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 -- 172.21.15.86:0/343559327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 msgr2=0x55851d7c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 -- 172.21.15.86:0/343559327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d7a7170 con 0x55851d7ab240 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f259c640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00eb20 tx=0x7f71ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671d1d6234755657 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71dbfff640 1 -- 172.21.15.86:0/343559327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec01b040 con 0x55851d7ab240 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71dbfff640 1 -- 172.21.15.86:0/343559327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ec024070 con 0x55851d7ab240 2024-09-05T06:47:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.680+0000 7f71dbfff640 1 -- 172.21.15.86:0/343559327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00f040 con 0x55851d7ab240 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- 172.21.15.86:0/343559327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 msgr2=0x55851d7a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00eb20 tx=0x7f71ec011000 comp rx=0 tx=0).stop 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- 172.21.15.86:0/343559327 shutdown_connections 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a1b00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 --2- 172.21.15.86:0/343559327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- 172.21.15.86:0/343559327 >> 172.21.15.86:0/343559327 conn(0x55851d79c5f0 msgr2=0x55851d79c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- 172.21.15.86:0/343559327 shutdown_connections 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- 172.21.15.86:0/343559327 wait complete. 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 -- start start 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.681+0000 7f71fb20e740 1 Processor -- start 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7a5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b0680 con 0x55851d7ab240 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41affb0 con 0x55851d7a0980 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41ad1f0 con 0x55851d7a0060 2024-09-05T06:47:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71fb20e740 10 client.0 _ll_register_callbacks cb 0x7f71e800cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37234/0 (socket says 172.21.15.86:37234) 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 -- 172.21.15.86:0/939120042 learned_addr learned my addr 172.21.15.86:0/939120042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f1d9b640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f259c640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 -- 172.21.15.86:0/939120042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 msgr2=0x55851d7a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 -- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 msgr2=0x55851d7a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 -- 172.21.15.86:0/939120042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d824a80 con 0x55851d7ab240 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f259c640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71f2d9d640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f71ec01fc70 tx=0x7f71ec006dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d24414e5a1276e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.682+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec02e070 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec024070 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f71ec00e790 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71ec03a050 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 10 client.16920 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55851d825dc0 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ec024070 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f71ec01e030 con 0x55851d7ab240 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 10 client.16920 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 10 client.16920 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 20 client.16920 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d826480 0x55851d8288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 4 client.16920 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5117007c", "release": true, "await": true}] 2024-09-05T06:47:32.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5117007c", "release": true, "await": true}) -- 0x55851d70d0a0 con 0x55851d826480 2024-09-05T06:47:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.683+0000 7f71f259c640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d826480 0x55851d8288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.684+0000 7f71f259c640 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d826480 0x55851d8288a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd5b38e4eaaa4a8 server_cookie=385d8f2e2b50ec0f in_seq=0 out_seq=0 2024-09-05T06:47:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.684+0000 7f71d9ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71d9ffb640 1 -- 172.21.15.86:0/939120042 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55851d70d0a0 con 0x55851d826480 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71d9ffb640 10 client.16920 handle_command_reply: tid=0 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 client.16920 shutdown 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 msgr2=0x55851d7a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f71ec01fc70 tx=0x7f71ec006dc0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 shutdown_connections 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55851d826480 0x55851d8288a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55851d7ab240 0x55851d7a5de0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55851d7a0980 0x55851d7a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 --2- 172.21.15.86:0/939120042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55851d7a0060 0x55851d7a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 >> 172.21.15.86:0/939120042 conn(0x55851d79c5f0 msgr2=0x55851d7a1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71d9ffb640 0 client.16920 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 shutdown_connections 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.690+0000 7f71fb20e740 1 -- 172.21.15.86:0/939120042 wait complete. 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71fb20e740 20 client.16920 trim_cache size 0 max 16384 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:47:32.692 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 20, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "5117007c": { 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 20, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 154.3, 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:47:32.693 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:47:32.694 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 msgr2=0x7f71dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 0x7f71dc0a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00a9e0 tx=0x7f71ec00a340 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 msgr2=0x7f71f4198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009df0 tx=0x7f71e4003ba0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 shutdown_connections 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f71dc0a2cf0 0x7f71dc0a51b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f41082a0 0x7f71f419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71f41036e0 0x7f71f4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 --2- 172.21.15.86:0/2846741544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f4102d30 0x7f71f4198640 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 >> 172.21.15.86:0/2846741544 conn(0x7f71f40fc7d0 msgr2=0x7f71f4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 shutdown_connections 2024-09-05T06:47:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:32.691+0000 7f71f8e35640 1 -- 172.21.15.86:0/2846741544 wait complete. 2024-09-05T06:47:33.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 108904 1.67 MB/sec execute 173 sec latency 32823.654 ms 2024-09-05T06:47:33.597 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5117007c', total seconds elapsed: 37.7 2024-09-05T06:47:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:33 smithi096 ceph-mon[34468]: pgmap v192: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:33 smithi115 ceph-mon[34791]: pgmap v192: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:33.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:33 smithi086 ceph-mon[30943]: pgmap v192: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:34.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 109742 1.70 MB/sec execute 174 sec latency 13.356 ms 2024-09-05T06:47:35.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110229 1.71 MB/sec execute 175 sec latency 10.982 ms 2024-09-05T06:47:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:35 smithi096 ceph-mon[34468]: pgmap v193: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:35 smithi115 ceph-mon[34791]: pgmap v193: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:35.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:35 smithi086 ceph-mon[30943]: pgmap v193: 129 pgs: 129 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-05T06:47:36.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110523 1.71 MB/sec execute 176 sec latency 13.080 ms 2024-09-05T06:47:37.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110990 1.70 MB/sec execute 177 sec latency 16.146 ms 2024-09-05T06:47:37.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:37 smithi086 ceph-mon[30943]: pgmap v194: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 664 KiB/s wr, 55 op/s 2024-09-05T06:47:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:37 smithi096 ceph-mon[34468]: pgmap v194: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 664 KiB/s wr, 55 op/s 2024-09-05T06:47:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:37 smithi115 ceph-mon[34791]: pgmap v194: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 664 KiB/s wr, 55 op/s 2024-09-05T06:47:38.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 111559 1.72 MB/sec execute 178 sec latency 22.064 ms 2024-09-05T06:47:39.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 112180 1.71 MB/sec execute 179 sec latency 22.307 ms 2024-09-05T06:47:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:39 smithi096 ceph-mon[34468]: pgmap v195: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 664 KiB/s wr, 50 op/s 2024-09-05T06:47:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:39 smithi115 ceph-mon[34791]: pgmap v195: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 664 KiB/s wr, 50 op/s 2024-09-05T06:47:39.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:39 smithi086 ceph-mon[30943]: pgmap v195: 129 pgs: 129 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 664 KiB/s wr, 50 op/s 2024-09-05T06:47:40.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 112892 1.73 MB/sec execute 180 sec latency 12.323 ms 2024-09-05T06:47:41.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 113486 1.76 MB/sec execute 181 sec latency 13.445 ms 2024-09-05T06:47:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:41 smithi096 ceph-mon[34468]: pgmap v196: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-09-05T06:47:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:41 smithi115 ceph-mon[34791]: pgmap v196: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-09-05T06:47:41.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:41 smithi086 ceph-mon[30943]: pgmap v196: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-09-05T06:47:42.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 113894 1.76 MB/sec execute 182 sec latency 13.398 ms 2024-09-05T06:47:43.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114208 1.75 MB/sec execute 183 sec latency 16.001 ms 2024-09-05T06:47:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:43 smithi096 ceph-mon[34468]: pgmap v197: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:43.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:47:43.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:43 smithi086 ceph-mon[30943]: pgmap v197: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:43 smithi115 ceph-mon[34791]: pgmap v197: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:44.244 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:44.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114755 1.75 MB/sec execute 184 sec latency 13.371 ms 2024-09-05T06:47:44.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 -- 172.21.15.86:0/642473425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 msgr2=0x7f9df410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 --2- 172.21.15.86:0/642473425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df410d940 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009cd0 tx=0x7f9dec02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 -- 172.21.15.86:0/642473425 shutdown_connections 2024-09-05T06:47:44.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 --2- 172.21.15.86:0/642473425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df410d940 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 --2- 172.21.15.86:0/642473425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4100a70 0x7f9df4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 --2- 172.21.15.86:0/642473425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4106a70 0x7f9df4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 -- 172.21.15.86:0/642473425 >> 172.21.15.86:0/642473425 conn(0x7f9df40fc790 msgr2=0x7f9df40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 -- 172.21.15.86:0/642473425 shutdown_connections 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.783+0000 7f9dfb379640 1 -- 172.21.15.86:0/642473425 wait complete. 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 Processor -- start 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 -- start start 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 0x7f9df4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 0x7f9df419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4110d70 con 0x7f9df4101410 2024-09-05T06:47:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4110bf0 con 0x7f9df4106a70 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9dfb379640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4110ef0 con 0x7f9df4100a70 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9df8914640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9df8914640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60722/0 (socket says 172.21.15.86:60722) 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9df8914640 1 -- 172.21.15.86:0/1465550262 learned_addr learned my addr 172.21.15.86:0/1465550262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9df9916640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 0x7f9df419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.784+0000 7f9df9115640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 0x7f9df4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 -- 172.21.15.86:0/1465550262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 msgr2=0x7f9df4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 0x7f9df4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 -- 172.21.15.86:0/1465550262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 msgr2=0x7f9df419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 0x7f9df419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 -- 172.21.15.86:0/1465550262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df419d300 con 0x7f9df4101410 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df9115640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 0x7f9df4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:44.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df9916640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 0x7f9df419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9df8914640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e9f0 tx=0x7f9de400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9dea7fc640 1 -- 172.21.15.86:0/1465550262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de400cd80 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df419d5f0 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9dea7fc640 1 -- 172.21.15.86:0/1465550262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de40108e0 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9dea7fc640 1 -- 172.21.15.86:0/1465550262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4018920 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.785+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df41a4ed0 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.787+0000 7f9dea7fc640 1 -- 172.21.15.86:0/1465550262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9de4010a90 con 0x7f9df4101410 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 Processor -- start 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 -- start start 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dea7fc640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 0x561862040550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x561862042e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561861f972e0 con 0x561862040a90 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561861f3ece0 con 0x561862031ca0 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9dfd752740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561861f932a0 con 0x561862031380 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9df9916640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.788+0000 7f9df9115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56676/0 (socket says 172.21.15.86:56676) 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 -- 172.21.15.86:0/172392589 learned_addr learned my addr 172.21.15.86:0/172392589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df8914640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 0x561862040550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9dea7fc640 1 -- 172.21.15.86:0/1465550262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9de40cfb60 con 0x7f9df4101410 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9916640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x561862042e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 -- 172.21.15.86:0/172392589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 msgr2=0x561862040550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 0x561862040550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 -- 172.21.15.86:0/172392589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 msgr2=0x561862042e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x561862042e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 -- 172.21.15.86:0/172392589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561861f23440 con 0x561862031380 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9916640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x561862042e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df8914640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 0x561862040550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9115640 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9df00114c0 tx=0x7f9df0011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5eee320754aa46a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9df9916640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 0x7f9de00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9dec004af0 tx=0x7f9dec005ef0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9ddf7fe640 1 -- 172.21.15.86:0/172392589 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0004020 con 0x561862031380 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9ddf7fe640 1 -- 172.21.15.86:0/172392589 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df00041c0 con 0x561862031380 2024-09-05T06:47:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.789+0000 7f9ddf7fe640 1 -- 172.21.15.86:0/172392589 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df000f8d0 con 0x561862031380 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- 172.21.15.86:0/172392589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 msgr2=0x561862031760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9df00114c0 tx=0x7f9df0011980 comp rx=0 tx=0).stop 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- 172.21.15.86:0/172392589 shutdown_connections 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x561862042e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031ca0 0x561862040550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 --2- 172.21.15.86:0/172392589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031380 0x561862031760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- 172.21.15.86:0/172392589 >> 172.21.15.86:0/172392589 conn(0x56186202da30 msgr2=0x56186202de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- 172.21.15.86:0/172392589 shutdown_connections 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- 172.21.15.86:0/172392589 wait complete. 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.790+0000 7f9dfd752740 1 -- start start 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 Processor -- start 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 0x56186203d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862040a90 0x56186203dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561862043660 con 0x561862031ca0 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561861f52050 con 0x561862031380 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618620437e0 con 0x561862040a90 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9dfd752740 10 client.0 _ll_register_callbacks cb 0x561861f92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37020/0 (socket says 172.21.15.86:37020) 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 -- 172.21.15.86:0/2050703101 learned_addr learned my addr 172.21.15.86:0/2050703101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9916640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862040a90 0x56186203dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df8914640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 0x56186203d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 -- 172.21.15.86:0/2050703101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862040a90 msgr2=0x56186203dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862040a90 0x56186203dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 -- 172.21.15.86:0/2050703101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 msgr2=0x56186203d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 0x56186203d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df9115640 1 -- 172.21.15.86:0/2050703101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561861d083c0 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.791+0000 7f9df8914640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 0x56186203d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9df9115640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011e80 tx=0x7f9df0009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8473f7a1a168111 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df001d040 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df00041c0 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9df0016050 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9df0030050 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 10 client.26195 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56186204f070 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df00041c0 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9df0020030 con 0x561862031380 2024-09-05T06:47:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 10 client.26195 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 20 client.26195 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561862050530 0x561862052920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 4 client.26195 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.792+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561861fa4850 con 0x561862050530 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.793+0000 7f9df8914640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561862050530 0x561862052920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.793+0000 7f9df8914640 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561862050530 0x561862052920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59753f88db9ca80 server_cookie=5ff17e55cf75efdf in_seq=0 out_seq=0 2024-09-05T06:47:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.793+0000 7f9ddcff9640 10 client.26195 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9ddcff9640 1 -- 172.21.15.86:0/2050703101 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561861fa4850 con 0x561862050530 2024-09-05T06:47:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9ddcff9640 10 client.26195 handle_command_reply: tid=0 2024-09-05T06:47:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 client.26195 shutdown 2024-09-05T06:47:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 msgr2=0x5618620403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011e80 tx=0x7f9df0009d00 comp rx=0 tx=0).stop 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 shutdown_connections 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561862050530 0x561862052920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862040a90 0x56186203dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862031ca0 0x56186203d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 --2- 172.21.15.86:0/2050703101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x5618620403e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9ddcff9640 0 client.26195 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 >> 172.21.15.86:0/2050703101 conn(0x56186202da30 msgr2=0x561862033790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 shutdown_connections 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 1 -- 172.21.15.86:0/2050703101 wait complete. 2024-09-05T06:47:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.794+0000 7f9dfd752740 20 client.26195 trim_cache size 0 max 16384 2024-09-05T06:47:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 Processor -- start 2024-09-05T06:47:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 -- start start 2024-09-05T06:47:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x56186203cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x56186203c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41acde0 con 0x561862040a90 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41ae920 con 0x561862031380 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9dfd752740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41ad4b0 con 0x561862031ca0 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9df9115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9df9916640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x56186203c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.815+0000 7f9df9115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37034/0 (socket says 172.21.15.86:37034) 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 -- 172.21.15.86:0/98355236 learned_addr learned my addr 172.21.15.86:0/98355236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df8914640 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x56186203cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 -- 172.21.15.86:0/98355236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 msgr2=0x56186203cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x56186203cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 -- 172.21.15.86:0/98355236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 msgr2=0x56186203c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x56186203c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 -- 172.21.15.86:0/98355236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561862031ad0 con 0x561862031380 2024-09-05T06:47:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9df9115640 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009cd0 tx=0x7f9df0004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d877f46bf4e3a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9ddeffd640 1 -- 172.21.15.86:0/98355236 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0015020 con 0x561862031380 2024-09-05T06:47:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9ddeffd640 1 -- 172.21.15.86:0/98355236 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df0004b20 con 0x561862031380 2024-09-05T06:47:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.816+0000 7f9ddeffd640 1 -- 172.21.15.86:0/98355236 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df001d040 con 0x561862031380 2024-09-05T06:47:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- 172.21.15.86:0/98355236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 msgr2=0x56186204c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009cd0 tx=0x7f9df0004fd0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- 172.21.15.86:0/98355236 shutdown_connections 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x56186203c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x56186203cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 --2- 172.21.15.86:0/98355236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x56186204c900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- 172.21.15.86:0/98355236 >> 172.21.15.86:0/98355236 conn(0x56186202da30 msgr2=0x56186202de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- 172.21.15.86:0/98355236 shutdown_connections 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- 172.21.15.86:0/98355236 wait complete. 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.817+0000 7f9dfd752740 1 -- start start 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 Processor -- start 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x561862048f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x561862044290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41ae250 con 0x561862040a90 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41adb80 con 0x561862031380 2024-09-05T06:47:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41aadc0 con 0x561862031ca0 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9dfd752740 10 client.0 _ll_register_callbacks cb 0x7f9de4097f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60770/0 (socket says 172.21.15.86:60770) 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df8914640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x561862044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 -- 172.21.15.86:0/4266836629 learned_addr learned my addr 172.21.15.86:0/4266836629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9115640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x561862048f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 -- 172.21.15.86:0/4266836629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 msgr2=0x561862044290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x561862044290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 -- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 msgr2=0x561862048f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.818+0000 7f9df9916640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x561862048f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9df9916640 1 -- 172.21.15.86:0/4266836629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186203df60 con 0x561862040a90 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9df8914640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x561862044290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9df9916640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009cd0 tx=0x7f9dec02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f1317b0c28edc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec031970 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f9dec030a70 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df0004b20 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9dec05e050 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9dfd752740 10 client.16938 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618620bac80 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.819+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df0004b20 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9dec048040 con 0x561862040a90 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 10 client.16938 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 20 client.16938 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618620bb3d0 0x5618620bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 4 client.16938 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561861f9ed60 con 0x5618620bb3d0 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9df9115640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618620bb3d0 0x5618620bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9df9115640 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618620bb3d0 0x5618620bd7f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f9ade9082b2ba1 server_cookie=edfe0425478fe7f3 in_seq=0 out_seq=0 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9ddcff9640 10 client.16938 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9ddcff9640 1 -- 172.21.15.86:0/4266836629 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561861f9ed60 con 0x5618620bb3d0 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.820+0000 7f9ddcff9640 10 client.16938 handle_command_reply: tid=0 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 client.16938 shutdown 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 msgr2=0x5618620447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009cd0 tx=0x7f9dec02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 shutdown_connections 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5618620bb3d0 0x5618620bd7f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561862040a90 0x5618620447d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561862031ca0 0x561862044290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 --2- 172.21.15.86:0/4266836629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561862031380 0x561862048f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9ddcff9640 0 client.16938 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 >> 172.21.15.86:0/4266836629 conn(0x56186202da30 msgr2=0x56186202cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 shutdown_connections 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 1 -- 172.21.15.86:0/4266836629 wait complete. 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.821+0000 7f9dfd752740 20 client.16938 trim_cache size 0 max 16384 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:47:44.827 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:47:44.828 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:47:44.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 msgr2=0x7f9de00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 0x7f9de00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9dec004af0 tx=0x7f9dec005ef0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 msgr2=0x7f9df4198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e9f0 tx=0x7f9de400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 shutdown_connections 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9de00a2cf0 0x7f9de00a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9df4106a70 0x7f9df419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df4101410 0x7f9df4198890 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 --2- 172.21.15.86:0/1465550262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9df4100a70 0x7f9df4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 >> 172.21.15.86:0/1465550262 conn(0x7f9df40fc790 msgr2=0x7f9df406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 shutdown_connections 2024-09-05T06:47:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:44.822+0000 7f9dfb379640 1 -- 172.21.15.86:0/1465550262 wait complete. 2024-09-05T06:47:45.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 115355 1.76 MB/sec execute 185 sec latency 17.439 ms 2024-09-05T06:47:45.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:45 smithi086 ceph-mon[30943]: pgmap v198: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:45.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:47:45.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:47:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:45 smithi096 ceph-mon[34468]: pgmap v198: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:45 smithi115 ceph-mon[34791]: pgmap v198: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-09-05T06:47:45.995 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:46.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 115934 1.76 MB/sec execute 186 sec latency 31.789 ms 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- 172.21.15.86:0/3467219467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1082a0 msgr2=0x7fac3c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 --2- 172.21.15.86:0/3467219467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1082a0 0x7fac3c10ec40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fac34009cd0 tx=0x7fac3402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- 172.21.15.86:0/3467219467 shutdown_connections 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 --2- 172.21.15.86:0/3467219467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1082a0 0x7fac3c10ec40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 --2- 172.21.15.86:0/3467219467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c1036e0 0x7fac3c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 --2- 172.21.15.86:0/3467219467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c102d30 0x7fac3c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- 172.21.15.86:0/3467219467 >> 172.21.15.86:0/3467219467 conn(0x7fac3c0fc7d0 msgr2=0x7fac3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- 172.21.15.86:0/3467219467 shutdown_connections 2024-09-05T06:47:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- 172.21.15.86:0/3467219467 wait complete. 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 Processor -- start 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.534+0000 7fac43128640 1 -- start start 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c102d30 0x7fac3c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 0x7fac3c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111f50 con 0x7fac3c1036e0 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111dd0 con 0x7fac3c1082a0 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1120d0 con 0x7fac3c102d30 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60786/0 (socket says 172.21.15.86:60786) 2024-09-05T06:47:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 -- 172.21.15.86:0/1727038294 learned_addr learned my addr 172.21.15.86:0/1727038294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac40ec4640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c102d30 0x7fac3c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac416c5640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 0x7fac3c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 -- 172.21.15.86:0/1727038294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c102d30 msgr2=0x7fac3c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c102d30 0x7fac3c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 -- 172.21.15.86:0/1727038294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 msgr2=0x7fac3c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 0x7fac3c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 -- 172.21.15.86:0/1727038294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c19d450 con 0x7fac3c1036e0 2024-09-05T06:47:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac416c5640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 0x7fac3c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac3bfff640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fac2800d750 tx=0x7fac2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28017070 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c19d740 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac280054d0 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.535+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004cf0 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.536+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a5020 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.537+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c103110 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.538+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fac28004e90 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.539+0000 7fac39ffb640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 0x7fac2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.539+0000 7fac40ec4640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 0x7fac2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.539+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fac280cf0e0 con 0x7fac3c1036e0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.539+0000 7fac40ec4640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 0x7fac2c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac30009bd0 tx=0x7fac30005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.543+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac2809b2d0 con 0x7fac3c1036e0 2024-09-05T06:47:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.781+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac3c10a2b0 con 0x7fac3c1036e0 2024-09-05T06:47:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.782+0000 7fac39ffb640 1 -- 172.21.15.86:0/1727038294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7fac2809f5d0 con 0x7fac3c1036e0 2024-09-05T06:47:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:47:46.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:47:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:47:46.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 msgr2=0x7fac2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 0x7fac2c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac30009bd0 tx=0x7fac30005f90 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 msgr2=0x7fac3c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fac2800d750 tx=0x7fac2800dc10 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 shutdown_connections 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac2c0a2cf0 0x7fac2c0a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c1082a0 0x7fac3c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c1036e0 0x7fac3c1989e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 --2- 172.21.15.86:0/1727038294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c102d30 0x7fac3c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 >> 172.21.15.86:0/1727038294 conn(0x7fac3c0fc7d0 msgr2=0x7fac3c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 shutdown_connections 2024-09-05T06:47:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:46.784+0000 7fac43128640 1 -- 172.21.15.86:0/1727038294 wait complete. 2024-09-05T06:47:47.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:46 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1727038294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:47.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:46 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1727038294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:47.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:46 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1727038294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:47:47.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 116559 1.78 MB/sec execute 187 sec latency 58.081 ms 2024-09-05T06:47:47.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:47:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:47.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:47 smithi086 ceph-mon[30943]: pgmap v199: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-05T06:47:48.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:47 smithi115 ceph-mon[34791]: pgmap v199: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-05T06:47:48.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:47 smithi096 ceph-mon[34468]: pgmap v199: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-05T06:47:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 -- 172.21.15.86:0/1431084358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 msgr2=0x7f373410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1431084358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373410ec40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f372c009cd0 tx=0x7f372c02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 -- 172.21.15.86:0/1431084358 shutdown_connections 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1431084358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373410ec40 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1431084358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37341036e0 0x7f3734107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1431084358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3734102d30 0x7f3734103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 -- 172.21.15.86:0/1431084358 >> 172.21.15.86:0/1431084358 conn(0x7f37340fc7d0 msgr2=0x7f37340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 -- 172.21.15.86:0/1431084358 shutdown_connections 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.492+0000 7f3738ac0640 1 -- 172.21.15.86:0/1431084358 wait complete. 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 Processor -- start 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 -- start start 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37341036e0 0x7f373419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3734111f50 con 0x7f37341082a0 2024-09-05T06:47:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3734111dd0 con 0x7f3734102d30 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f3738ac0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37341120d0 con 0x7f37341036e0 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.493+0000 7f373259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f3732d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37078/0 (socket says 172.21.15.86:37078) 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 -- 172.21.15.86:0/1796129060 learned_addr learned my addr 172.21.15.86:0/1796129060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f3731d9b640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37341036e0 0x7f373419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 -- 172.21.15.86:0/1796129060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37341036e0 msgr2=0x7f373419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37341036e0 0x7f373419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 -- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 msgr2=0x7f373419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 -- 172.21.15.86:0/1796129060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373419f610 con 0x7f3734102d30 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f3732d9d640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f373259c640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f372400d950 tx=0x7f372400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f37237fe640 1 -- 172.21.15.86:0/1796129060 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724017070 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f37237fe640 1 -- 172.21.15.86:0/1796129060 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3724009c30 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373419f900 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f37237fe640 1 -- 172.21.15.86:0/1796129060 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400f620 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.494+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a71e0 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f37237fe640 1 -- 172.21.15.86:0/1796129060 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37240109a0 con 0x7f3734102d30 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 Processor -- start 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 -- start start 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66562cb40 0x55a66562b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66562ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 0x55a665640f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a6655912f0 con 0x55a66563eb80 2024-09-05T06:47:48.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a665538cf0 con 0x55a66562b5d0 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f373ae99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a66558d2b0 con 0x55a66562cb40 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.497+0000 7f37237fe640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 0x7f371c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f373259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66562ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 0x55a665640f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f373259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66562ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37092/0 (socket says 172.21.15.86:37092) 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f373259c640 1 -- 172.21.15.86:0/4211056104 learned_addr learned my addr 172.21.15.86:0/4211056104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3732d9d640 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66562cb40 0x55a66562b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3732d9d640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 0x7f371c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 -- 172.21.15.86:0/4211056104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66562cb40 msgr2=0x55a66562b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66562cb40 0x55a66562b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 -- 172.21.15.86:0/4211056104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 msgr2=0x55a66562ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66562ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 -- 172.21.15.86:0/4211056104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66551d3a0 con 0x55a66563eb80 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f37237fe640 1 -- 172.21.15.86:0/1796129060 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f37240cf3d0 con 0x7f3734102d30 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3731d9b640 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 0x55a665640f70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f373406a370 tx=0x7f3728011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532c32bb28bdb6f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3732d9d640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 0x7f371c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f372c009430 tx=0x7f372c004a60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3720ff9640 1 -- 172.21.15.86:0/4211056104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728004050 con 0x55a66563eb80 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3720ff9640 1 -- 172.21.15.86:0/4211056104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37280054d0 con 0x55a66563eb80 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.498+0000 7f3720ff9640 1 -- 172.21.15.86:0/4211056104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728004bc0 con 0x55a66563eb80 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 -- 172.21.15.86:0/4211056104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 msgr2=0x55a665640f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 0x55a665640f70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f373406a370 tx=0x7f3728011810 comp rx=0 tx=0).stop 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 -- 172.21.15.86:0/4211056104 shutdown_connections 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66563eb80 0x55a665640f70 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66562ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 --2- 172.21.15.86:0/4211056104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66562cb40 0x55a66562b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 -- 172.21.15.86:0/4211056104 >> 172.21.15.86:0/4211056104 conn(0x55a6655f4310 msgr2=0x55a665627c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.499+0000 7f373ae99740 1 -- 172.21.15.86:0/4211056104 shutdown_connections 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 -- 172.21.15.86:0/4211056104 wait complete. 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 -- start start 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 Processor -- start 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66563e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a66563bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a665641880 con 0x55a66562cb40 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a66554c060 con 0x55a66562b5d0 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a665641a00 con 0x55a66563eb80 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373ae99740 10 client.0 _ll_register_callbacks cb 0x55a66529f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f3731d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f373259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66563e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f3731d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60816/0 (socket says 172.21.15.86:60816) 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.500+0000 7f3731d9b640 1 -- 172.21.15.86:0/2575775160 learned_addr learned my addr 172.21.15.86:0/2575775160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3732d9d640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a66563bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 -- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 msgr2=0x55a66563bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a66563bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 -- 172.21.15.86:0/2575775160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 msgr2=0x55a66563e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66563e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 -- 172.21.15.86:0/2575775160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6653023c0 con 0x55a66562cb40 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3732d9d640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a66563bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373259c640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66563e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f37280048a0 tx=0x7f37280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc60b4f69091d4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728028070 con 0x55a66562cb40 2024-09-05T06:47:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37280054d0 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f37280037f0 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3728034050 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373ae99740 10 client.16956 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a66564d2e0 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37280054d0 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3728004e80 con 0x55a66562cb40 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373ae99740 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.501+0000 7f373ae99740 10 client.16956 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373ae99740 20 client.16956 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a66564e690 0x55a665650a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373ae99740 4 client.16956 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a66559e860 con 0x55a66564e690 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373259c640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a66564e690 0x55a665650a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f373259c640 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a66564e690 0x55a665650a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55c43c7235a96f2c server_cookie=817202f2c27a489 in_seq=0 out_seq=0 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.502+0000 7f3719ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f3719ffb640 1 -- 172.21.15.86:0/2575775160 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a66559e860 con 0x55a66564e690 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f3719ffb640 10 client.16956 handle_command_reply: tid=0 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 client.16956 shutdown 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 msgr2=0x55a66563b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f37280048a0 tx=0x7f37280046c0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 shutdown_connections 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a66564e690 0x55a665650a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a66563bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a66563b7f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 --2- 172.21.15.86:0/2575775160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a66563e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f3719ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 >> 172.21.15.86:0/2575775160 conn(0x55a6655f4310 msgr2=0x55a66562d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 shutdown_connections 2024-09-05T06:47:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 1 -- 172.21.15.86:0/2575775160 wait complete. 2024-09-05T06:47:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.503+0000 7f373ae99740 20 client.16956 trim_cache size 0 max 16384 2024-09-05T06:47:48.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117100 1.79 MB/sec execute 188 sec latency 22.203 ms 2024-09-05T06:47:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 Processor -- start 2024-09-05T06:47:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 -- start start 2024-09-05T06:47:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562cb40 0x55a66563a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a665656270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ace40 con 0x55a66562b5d0 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ae980 con 0x55a66562cb40 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373ae99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ad510 con 0x55a66563eb80 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60832/0 (socket says 172.21.15.86:60832) 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.517+0000 7f373259c640 1 -- 172.21.15.86:0/703619910 learned_addr learned my addr 172.21.15.86:0/703619910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f3731d9b640 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562cb40 0x55a66563a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f3732d9d640 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a665656270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 -- 172.21.15.86:0/703619910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 msgr2=0x55a665656270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a665656270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 -- 172.21.15.86:0/703619910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562cb40 msgr2=0x55a66563a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562cb40 0x55a66563a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 -- 172.21.15.86:0/703619910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a665628730 con 0x55a66562b5d0 2024-09-05T06:47:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f373259c640 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3724086130 tx=0x7f372400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184cf507e7a246e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f371bfff640 1 -- 172.21.15.86:0/703619910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724020040 con 0x55a66562b5d0 2024-09-05T06:47:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f371bfff640 1 -- 172.21.15.86:0/703619910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372409f8c0 con 0x55a66562b5d0 2024-09-05T06:47:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.518+0000 7f371bfff640 1 -- 172.21.15.86:0/703619910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724097c30 con 0x55a66562b5d0 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- 172.21.15.86:0/703619910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 msgr2=0x55a66564ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3724086130 tx=0x7f372400bf60 comp rx=0 tx=0).stop 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- 172.21.15.86:0/703619910 shutdown_connections 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a665656270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562cb40 0x55a66563a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 --2- 172.21.15.86:0/703619910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562b5d0 0x55a66564ab20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- 172.21.15.86:0/703619910 >> 172.21.15.86:0/703619910 conn(0x55a6655f4310 msgr2=0x55a66563a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- 172.21.15.86:0/703619910 shutdown_connections 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- 172.21.15.86:0/703619910 wait complete. 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.519+0000 7f373ae99740 1 -- start start 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 Processor -- start 2024-09-05T06:47:48.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a665646d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a6656421f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ae2b0 con 0x55a66562cb40 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37341adbe0 con 0x55a66562b5d0 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37341aae20 con 0x55a66563eb80 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373ae99740 10 client.0 _ll_register_callbacks cb 0x7f372409b390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60844/0 (socket says 172.21.15.86:60844) 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 -- 172.21.15.86:0/2667898530 learned_addr learned my addr 172.21.15.86:0/2667898530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f373259c640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a665646d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3732d9d640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a6656421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 -- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 msgr2=0x55a6656421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a6656421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 -- 172.21.15.86:0/2667898530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 msgr2=0x55a665646d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a665646d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 -- 172.21.15.86:0/2667898530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a665642a20 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3731d9b640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3728005440 tx=0x7f3728011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91194ef5e4bb157d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.520+0000 7f3719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372803c020 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f3728004360 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372409f8c0 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3728047050 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 10 client.16962 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6656b8d20 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372409f8c0 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f37280048d0 con 0x55a66562cb40 2024-09-05T06:47:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 10 client.16962 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 10 client.16962 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 20 client.16962 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a6656b93d0 0x55a6656bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 4 client.16962 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.521+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a665598d70 con 0x55a6656b93d0 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.522+0000 7f373259c640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a6656b93d0 0x55a6656bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.522+0000 7f3719ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.522+0000 7f373259c640 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a6656b93d0 0x55a6656bb7f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=764dce5f031a060e server_cookie=ef273a24b41b437f in_seq=0 out_seq=0 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.522+0000 7f3719ffb640 1 -- 172.21.15.86:0/2667898530 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a665598d70 con 0x55a6656b93d0 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f3719ffb640 10 client.16962 handle_command_reply: tid=0 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 client.16962 shutdown 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 msgr2=0x55a6656472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3728005440 tx=0x7f3728011e10 comp rx=0 tx=0).stop 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 shutdown_connections 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a6656b93d0 0x55a6656bb7f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a66563eb80 0x55a6656421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a66562cb40 0x55a6656472a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f3719ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 --2- 172.21.15.86:0/2667898530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a66562b5d0 0x55a665646d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 >> 172.21.15.86:0/2667898530 conn(0x55a6655f4310 msgr2=0x55a66563a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 shutdown_connections 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 1 -- 172.21.15.86:0/2667898530 wait complete. 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.523+0000 7f373ae99740 20 client.16962 trim_cache size 0 max 16384 2024-09-05T06:47:48.529 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 msgr2=0x7f371c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 0x7f371c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f372c009430 tx=0x7f372c004a60 comp rx=0 tx=0).stop 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 msgr2=0x7f373419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f372400d950 tx=0x7f372400de10 comp rx=0 tx=0).stop 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 shutdown_connections 2024-09-05T06:47:48.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f371c0a2cf0 0x7f371c0a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37341082a0 0x7f373419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37341036e0 0x7f373419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 --2- 172.21.15.86:0/1796129060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102d30 0x7f373419a660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 >> 172.21.15.86:0/1796129060 conn(0x7f37340fc7d0 msgr2=0x7f3734100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 shutdown_connections 2024-09-05T06:47:48.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:48.524+0000 7f3738ac0640 1 -- 172.21.15.86:0/1796129060 wait complete. 2024-09-05T06:47:49.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:47:49.459 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:49.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117475 1.79 MB/sec execute 189 sec latency 13.493 ms 2024-09-05T06:47:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:49 smithi096 ceph-mon[34468]: pgmap v200: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:47:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:49 smithi115 ceph-mon[34791]: pgmap v200: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:47:49.971 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:49 smithi086 ceph-mon[30943]: pgmap v200: 129 pgs: 129 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-05T06:47:50.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117778 1.78 MB/sec execute 190 sec latency 35.802 ms 2024-09-05T06:47:51.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 118325 1.78 MB/sec execute 191 sec latency 13.498 ms 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- 172.21.15.86:0/909255130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 msgr2=0x7f5c341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/909255130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c341029b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009900 tx=0x7f5c2402a480 comp rx=0 tx=0).stop 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- 172.21.15.86:0/909255130 shutdown_connections 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/909255130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34102ef0 0x7f5c3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/909255130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102570 0x7f5c341029b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/909255130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34100e20 0x7f5c34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- 172.21.15.86:0/909255130 >> 172.21.15.86:0/909255130 conn(0x7f5c340fc7d0 msgr2=0x7f5c340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- 172.21.15.86:0/909255130 shutdown_connections 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- 172.21.15.86:0/909255130 wait complete. 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 Processor -- start 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.955+0000 7f5c3abc2640 1 -- start start 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 0x7f5c34199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110e30 con 0x7f5c34102ef0 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110cb0 con 0x7f5c34102570 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34110fb0 con 0x7f5c34100e20 2024-09-05T06:47:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60866/0 (socket says 172.21.15.86:60866) 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 -- 172.21.15.86:0/246189846 learned_addr learned my addr 172.21.15.86:0/246189846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3895e640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 0x7f5c34199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 -- 172.21.15.86:0/246189846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 msgr2=0x7f5c34199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 -- 172.21.15.86:0/246189846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 msgr2=0x7f5c34199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 0x7f5c34199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 -- 172.21.15.86:0/246189846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c341a47f0 con 0x7f5c34102ef0 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3915f640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00d750 tx=0x7f5c2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c29ffb640 1 -- 172.21.15.86:0/246189846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c017070 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.956+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341a49c0 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.957+0000 7f5c29ffb640 1 -- 172.21.15.86:0/246189846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c2c0054d0 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.957+0000 7f5c29ffb640 1 -- 172.21.15.86:0/246189846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c004c60 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.957+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a4f20 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.959+0000 7f5c29ffb640 1 -- 172.21.15.86:0/246189846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5c2c004e00 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.959+0000 7f5c29ffb640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 0x7f5c1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.959+0000 7f5c3cf9b740 1 Processor -- start 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 0x7f5c1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c29ffb640 1 -- 172.21.15.86:0/246189846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5c2c0cf0c0 con 0x7f5c34102ef0 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 -- start start 2024-09-05T06:47:51.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 0x55990f9b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9c3370 0x55990f9c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55990f91e2f0 con 0x55990f9b85d0 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55990f8c5cf0 con 0x55990f9b9b40 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55990f91a2b0 con 0x55990f9c3370 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 0x7f5c1c0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005fd0 tx=0x7f5c30009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 0x55990f9b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60880/0 (socket says 172.21.15.86:60880) 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c2bfff640 1 -- 172.21.15.86:0/1992807142 learned_addr learned my addr 172.21.15.86:0/1992807142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:51.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3915f640 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9c3370 0x55990f9c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 -- 172.21.15.86:0/1992807142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9c3370 msgr2=0x55990f9c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9c3370 0x55990f9c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 -- 172.21.15.86:0/1992807142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 msgr2=0x55990f9b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c3895e640 1 -- 172.21.15.86:0/1992807142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990f8aa3a0 con 0x55990f9b9b40 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.960+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.961+0000 7f5c3895e640 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 0x55990f9b8090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003c50 tx=0x7f5c30014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba8afffd0d651ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.961+0000 7f5c1affd640 1 -- 172.21.15.86:0/1992807142 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30027670 con 0x55990f9b9b40 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.961+0000 7f5c1affd640 1 -- 172.21.15.86:0/1992807142 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c30022070 con 0x55990f9b9b40 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.961+0000 7f5c1affd640 1 -- 172.21.15.86:0/1992807142 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001e990 con 0x55990f9b9b40 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/1992807142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 msgr2=0x55990f9b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 0x55990f9b8090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003c50 tx=0x7f5c30014000 comp rx=0 tx=0).stop 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/1992807142 shutdown_connections 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9c3370 0x55990f9c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/1992807142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b9b40 0x55990f9b8090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/1992807142 >> 172.21.15.86:0/1992807142 conn(0x55990f981310 msgr2=0x55990f9b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/1992807142 shutdown_connections 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/1992807142 wait complete. 2024-09-05T06:47:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 -- start start 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.962+0000 7f5c3cf9b740 1 Processor -- start 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 0x55990f9c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55990f9c6070 con 0x55990f9b9b40 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55990f8d9060 con 0x55990f9c3370 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55990f9c61f0 con 0x55990f9b85d0 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3cf9b740 10 client.0 _ll_register_callbacks cb 0x55990f62c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60884/0 (socket says 172.21.15.86:60884) 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c2bfff640 1 -- 172.21.15.86:0/722966196 learned_addr learned my addr 172.21.15.86:0/722966196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3895e640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 0x55990f9c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 -- 172.21.15.86:0/722966196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 msgr2=0x55990f9c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 0x55990f9c2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 -- 172.21.15.86:0/722966196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 msgr2=0x55990f9bffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 -- 172.21.15.86:0/722966196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990f68f3c0 con 0x55990f9c3370 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3895e640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 0x55990f9c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c3915f640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9c0570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0cc430 tx=0x7f5c2c005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8515f9f13742e6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.963+0000 7f5c13fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c020050 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5c2c097920 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c30022070 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c0b9050 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 10 client.26237 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55990f9d1af0 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c30022070 con 0x55990f9c3370 2024-09-05T06:47:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f5c2c09f1e0 con 0x55990f9c3370 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 10 client.26237 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 10 client.26237 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 20 client.26237 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990f9d2f50 0x55990f9d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 4 client.26237 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55990f92b860 con 0x55990f9d2f50 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.964+0000 7f5c3895e640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990f9d2f50 0x55990f9d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.965+0000 7f5c3895e640 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990f9d2f50 0x55990f9d5360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61cd4809d1355bcf server_cookie=242826757f49e6da in_seq=0 out_seq=0 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.965+0000 7f5c13fff640 10 client.26237 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c13fff640 1 -- 172.21.15.86:0/722966196 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55990f92b860 con 0x55990f9d2f50 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c13fff640 10 client.26237 handle_command_reply: tid=0 2024-09-05T06:47:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 client.26237 shutdown 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 msgr2=0x55990f9c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9c0570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0cc430 tx=0x7f5c2c005680 comp rx=0 tx=0).stop 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 shutdown_connections 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990f9d2f50 0x55990f9d5360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9c0570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b9b40 0x55990f9bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/722966196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b85d0 0x55990f9c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c13fff640 0 client.26237 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 >> 172.21.15.86:0/722966196 conn(0x55990f981310 msgr2=0x55990f9ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 shutdown_connections 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/722966196 wait complete. 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.966+0000 7f5c3cf9b740 20 client.26237 trim_cache size 0 max 16384 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 Processor -- start 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 -- start start 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341af030 con 0x55990f9b85d0 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341b0b70 con 0x55990f9c3370 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341af700 con 0x55990f9b9b40 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60892/0 (socket says 172.21.15.86:60892) 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 -- 172.21.15.86:0/2556363965 learned_addr learned my addr 172.21.15.86:0/2556363965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3915f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 -- 172.21.15.86:0/2556363965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 msgr2=0x55990f9c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 -- 172.21.15.86:0/2556363965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 msgr2=0x55990f9bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 -- 172.21.15.86:0/2556363965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990f9bef80 con 0x55990f9b85d0 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3915f640 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.980+0000 7f5c3895e640 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005c40 tx=0x7f5c30011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6ca5e9fa5009a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c1a7fc640 1 -- 172.21.15.86:0/2556363965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004c00 con 0x55990f9b85d0 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c1a7fc640 1 -- 172.21.15.86:0/2556363965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c30022070 con 0x55990f9b85d0 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c1a7fc640 1 -- 172.21.15.86:0/2556363965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005430 con 0x55990f9b85d0 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/2556363965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 msgr2=0x55990f9bf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005c40 tx=0x7f5c30011d70 comp rx=0 tx=0).stop 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/2556363965 shutdown_connections 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9c3370 0x55990f9bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/2556363965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9b85d0 0x55990f9bf890 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/2556363965 >> 172.21.15.86:0/2556363965 conn(0x55990f981310 msgr2=0x55990f9c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.981+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/2556363965 shutdown_connections 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/2556363965 wait complete. 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 -- start start 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 Processor -- start 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 0x55990f9c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341b04a0 con 0x55990f9c3370 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341afdd0 con 0x55990f9b85d0 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341ad010 con 0x55990f9b9b40 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3cf9b740 10 client.0 _ll_register_callbacks cb 0x7f5c2c00de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3895e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.982+0000 7f5c3895e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37164/0 (socket says 172.21.15.86:37164) 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 -- 172.21.15.86:0/4282786607 learned_addr learned my addr 172.21.15.86:0/4282786607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3915f640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 0x55990f9c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 -- 172.21.15.86:0/4282786607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 msgr2=0x55990f9cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 -- 172.21.15.86:0/4282786607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 msgr2=0x55990f9c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 0x55990f9c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 -- 172.21.15.86:0/4282786607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990f9c7230 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3915f640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 0x55990f9c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3895e640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003c20 tx=0x7f5c30005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d3d005feb08d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c13fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004bd0 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f5c30004d70 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c30022070 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c30049050 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3cf9b740 10 client.26243 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.983+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55990fa3d710 con 0x55990f9b85d0 2024-09-05T06:47:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c30022070 con 0x55990f9b85d0 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f5c30005560 con 0x55990f9b85d0 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 10 client.26243 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 10 client.26243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 20 client.26243 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990fa3da90 0x55990fa3fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 4 client.26243 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55990f925d70 con 0x55990fa3da90 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990fa3da90 0x55990fa3fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c13fff640 10 client.26243 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.984+0000 7f5c2bfff640 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990fa3da90 0x55990fa3fe80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec3951889d4e2f66 server_cookie=f9d41d494250e625 in_seq=0 out_seq=0 2024-09-05T06:47:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c13fff640 1 -- 172.21.15.86:0/4282786607 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55990f925d70 con 0x55990fa3da90 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c13fff640 10 client.26243 handle_command_reply: tid=0 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 client.26243 shutdown 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 msgr2=0x55990f9cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003c20 tx=0x7f5c30005000 comp rx=0 tx=0).stop 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 shutdown_connections 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55990fa3da90 0x55990fa3fe80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990f9c3370 0x55990f9c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55990f9b9b40 0x55990f9cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c13fff640 0 client.26243 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:47:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 --2- 172.21.15.86:0/4282786607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55990f9b85d0 0x55990f9cb650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 >> 172.21.15.86:0/4282786607 conn(0x55990f981310 msgr2=0x55990f9c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 shutdown_connections 2024-09-05T06:47:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 1 -- 172.21.15.86:0/4282786607 wait complete. 2024-09-05T06:47:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.985+0000 7f5c3cf9b740 20 client.26243 trim_cache size 0 max 16384 2024-09-05T06:47:51.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:51 smithi086 ceph-mon[30943]: pgmap v201: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 msgr2=0x7f5c1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 0x7f5c1c0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005fd0 tx=0x7f5c30009500 comp rx=0 tx=0).stop 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 msgr2=0x7f5c341a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00d750 tx=0x7f5c2c00dc10 comp rx=0 tx=0).stop 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 shutdown_connections 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5c1c0a2cf0 0x7f5c1c0a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c34102ef0 0x7f5c341a42b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c34102570 0x7f5c34199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 --2- 172.21.15.86:0/246189846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c34100e20 0x7f5c34199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:51.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 >> 172.21.15.86:0/246189846 conn(0x7f5c340fc7d0 msgr2=0x7f5c340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:51.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 shutdown_connections 2024-09-05T06:47:51.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:50.986+0000 7f5c3abc2640 1 -- 172.21.15.86:0/246189846 wait complete. 2024-09-05T06:47:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:51 smithi096 ceph-mon[34468]: pgmap v201: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-05T06:47:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:51 smithi115 ceph-mon[34791]: pgmap v201: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-05T06:47:52.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 118904 1.79 MB/sec execute 192 sec latency 18.787 ms 2024-09-05T06:47:52.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:47:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.509+0000 7f015d550640 1 -- 172.21.15.86:0/2479869744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158102ef0 msgr2=0x7f015810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.509+0000 7f015d550640 1 --2- 172.21.15.86:0/2479869744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158102ef0 0x7f015810da40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0150009ca0 tx=0x7f015002a950 comp rx=0 tx=0).stop 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 -- 172.21.15.86:0/2479869744 shutdown_connections 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 --2- 172.21.15.86:0/2479869744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158102ef0 0x7f015810da40 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 --2- 172.21.15.86:0/2479869744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 0x7f01581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 --2- 172.21.15.86:0/2479869744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158100e20 0x7f0158101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 -- 172.21.15.86:0/2479869744 >> 172.21.15.86:0/2479869744 conn(0x7f01580fc7d0 msgr2=0x7f01580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 -- 172.21.15.86:0/2479869744 shutdown_connections 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 -- 172.21.15.86:0/2479869744 wait complete. 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.510+0000 7f015d550640 1 Processor -- start 2024-09-05T06:47:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 -- start start 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 0x7f01580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 0x7f01580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110e30 con 0x7f0158100e20 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110cb0 con 0x7f0158102570 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f015d550640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0158110fb0 con 0x7f0158102ef0 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f0156ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f0156ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58784/0 (socket says 172.21.15.86:58784) 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f0156ffd640 1 -- 172.21.15.86:0/726622702 learned_addr learned my addr 172.21.15.86:0/726622702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 0x7f01580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01567fc640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 0x7f01580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 -- 172.21.15.86:0/726622702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 msgr2=0x7f01580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 0x7f01580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 -- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 msgr2=0x7f01580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f01577fe640 1 -- 172.21.15.86:0/726622702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01581a68d0 con 0x7f0158102ef0 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.511+0000 7f0156ffd640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.512+0000 7f01577fe640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 0x7f01580fd700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0150005cc0 tx=0x7f0150031f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.514+0000 7f013ffff640 1 -- 172.21.15.86:0/726622702 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150034040 con 0x7f0158102ef0 2024-09-05T06:47:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.514+0000 7f013ffff640 1 -- 172.21.15.86:0/726622702 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f015002f6b0 con 0x7f0158102ef0 2024-09-05T06:47:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.514+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581a6b60 con 0x7f0158102ef0 2024-09-05T06:47:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.515+0000 7f013ffff640 1 -- 172.21.15.86:0/726622702 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150004430 con 0x7f0158102ef0 2024-09-05T06:47:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.515+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581a7090 con 0x7f0158102ef0 2024-09-05T06:47:53.517 INFO:tasks.workunit.client.0.smithi086.stdout: 1 119488 1.79 MB/sec execute 193 sec latency 21.416 ms 2024-09-05T06:47:53.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.517+0000 7f013ffff640 1 -- 172.21.15.86:0/726622702 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0150037020 con 0x7f0158102ef0 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 Processor -- start 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 -- start start 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a19a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f013ffff640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 0x7f01400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff5397f2f0 con 0x55ff53a24370 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff53926cf0 con 0x55ff53a1ab40 2024-09-05T06:47:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f015f929740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff5397b2b0 con 0x55ff53a195d0 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52750/0 (socket says 172.21.15.86:52750) 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 -- 172.21.15.86:0/592582250 learned_addr learned my addr 172.21.15.86:0/592582250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58800/0 (socket says 172.21.15.86:58800) 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f0156ffd640 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a19a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 0x7f01400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f013ffff640 1 -- 172.21.15.86:0/726622702 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f01500f03c0 con 0x7f0158102ef0 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 -- 172.21.15.86:0/592582250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 msgr2=0x55ff53a19a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a19a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 -- 172.21.15.86:0/592582250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 msgr2=0x55ff53a26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a26760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.518+0000 7f01567fc640 1 -- 172.21.15.86:0/592582250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5390b3a0 con 0x55ff53a1ab40 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.519+0000 7f01567fc640 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f014c011a30 tx=0x7f014c011ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bb198b1a29ea5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.519+0000 7f01567fc640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 0x7f01400a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f014c00f850 tx=0x7f014c00e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.519+0000 7f013d7fa640 1 -- 172.21.15.86:0/592582250 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0035f0 con 0x55ff53a1ab40 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.519+0000 7f013d7fa640 1 -- 172.21.15.86:0/592582250 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0099a0 con 0x55ff53a1ab40 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.519+0000 7f013d7fa640 1 -- 172.21.15.86:0/592582250 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c01e800 con 0x55ff53a1ab40 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- 172.21.15.86:0/592582250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 msgr2=0x55ff53a19090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f014c011a30 tx=0x7f014c011ef0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- 172.21.15.86:0/592582250 shutdown_connections 2024-09-05T06:47:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 --2- 172.21.15.86:0/592582250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a1ab40 0x55ff53a19090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- 172.21.15.86:0/592582250 >> 172.21.15.86:0/592582250 conn(0x55ff539e2310 msgr2=0x55ff53a15c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- 172.21.15.86:0/592582250 shutdown_connections 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- 172.21.15.86:0/592582250 wait complete. 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 -- start start 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.520+0000 7f015f929740 1 Processor -- start 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a21650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff53a27010 con 0x55ff53a1ab40 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff5393a060 con 0x55ff53a24370 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff53a27190 con 0x55ff53a195d0 2024-09-05T06:47:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f015f929740 10 client.0 _ll_register_callbacks cb 0x55ff5368d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58814/0 (socket says 172.21.15.86:58814) 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a21650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01567fc640 1 -- 172.21.15.86:0/772735956 learned_addr learned my addr 172.21.15.86:0/772735956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f0156ffd640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a23d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 -- 172.21.15.86:0/772735956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 msgr2=0x55ff53a23d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a23d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 -- 172.21.15.86:0/772735956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 msgr2=0x55ff53a21040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 -- 172.21.15.86:0/772735956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff536f03c0 con 0x55ff53a24370 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01567fc640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f0156ffd640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a23d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f01577fe640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a21650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01500ed840 tx=0x7f01500bc3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a64acd3e58c3712 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f013aff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.521+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01500bbf70 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f0150003b70 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014c0099a0 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01500d5810 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 10 client.26255 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff53a32ae0 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c0099a0 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f015004d030 con 0x55ff53a24370 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 10 client.26255 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 10 client.26255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 20 client.26255 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a33f40 0x55ff53a36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 4 client.26255 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff5398c860 con 0x55ff53a33f40 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.522+0000 7f0156ffd640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a33f40 0x55ff53a36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f0156ffd640 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a33f40 0x55ff53a36370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e53ca4c84bddb716 server_cookie=d8896107e46092ea in_seq=0 out_seq=0 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f013aff5640 10 client.26255 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f013aff5640 1 -- 172.21.15.86:0/772735956 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff5398c860 con 0x55ff53a33f40 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f013aff5640 10 client.26255 handle_command_reply: tid=0 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f015f929740 1 client.26255 shutdown 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 msgr2=0x55ff53a21650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.523+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a21650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01500ed840 tx=0x7f01500bc3e0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 shutdown_connections 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a33f40 0x55ff53a36370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a21650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a21040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f013aff5640 0 client.26255 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 --2- 172.21.15.86:0/772735956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a23d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 >> 172.21.15.86:0/772735956 conn(0x55ff539e2310 msgr2=0x55ff53a1b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 shutdown_connections 2024-09-05T06:47:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 1 -- 172.21.15.86:0/772735956 wait complete. 2024-09-05T06:47:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.524+0000 7f015f929740 20 client.26255 trim_cache size 0 max 16384 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 Processor -- start 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 -- start start 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a1c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b0fc0 con 0x55ff53a1ab40 2024-09-05T06:47:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b2b00 con 0x55ff53a24370 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f015f929740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b1690 con 0x55ff53a195d0 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58818/0 (socket says 172.21.15.86:58818) 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 -- 172.21.15.86:0/1853998573 learned_addr learned my addr 172.21.15.86:0/1853998573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f0156ffd640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01577fe640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a1c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 -- 172.21.15.86:0/1853998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 msgr2=0x55ff53a22fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a22fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 -- 172.21.15.86:0/1853998573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 msgr2=0x55ff53a1c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a1c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.538+0000 7f01567fc640 1 -- 172.21.15.86:0/1853998573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff53a369c0 con 0x55ff53a1ab40 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.539+0000 7f01577fe640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a1c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.539+0000 7f01567fc640 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe543014994023 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.539+0000 7f013cff9640 1 -- 172.21.15.86:0/1853998573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0099a0 con 0x55ff53a1ab40 2024-09-05T06:47:53.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.539+0000 7f013cff9640 1 -- 172.21.15.86:0/1853998573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c034070 con 0x55ff53a1ab40 2024-09-05T06:47:53.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.539+0000 7f013cff9640 1 -- 172.21.15.86:0/1853998573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c004510 con 0x55ff53a1ab40 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- 172.21.15.86:0/1853998573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 msgr2=0x55ff53a1c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c013040 comp rx=0 tx=0).stop 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- 172.21.15.86:0/1853998573 shutdown_connections 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a24370 0x55ff53a1c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a1ab40 0x55ff53a1c120 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 --2- 172.21.15.86:0/1853998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a195d0 0x55ff53a22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- 172.21.15.86:0/1853998573 >> 172.21.15.86:0/1853998573 conn(0x55ff539e2310 msgr2=0x55ff53a228a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- 172.21.15.86:0/1853998573 shutdown_connections 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- 172.21.15.86:0/1853998573 wait complete. 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 -- start start 2024-09-05T06:47:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.540+0000 7f015f929740 1 Processor -- start 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a195d0 0x55ff53a2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a1ab40 0x55ff53a1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b2430 con 0x55ff53a24370 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01581b1d60 con 0x55ff53a195d0 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01581aefa0 con 0x55ff53a1ab40 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 10 client.0 _ll_register_callbacks cb 0x7f01500a70b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f0156ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a195d0 0x55ff53a2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58822/0 (socket says 172.21.15.86:58822) 2024-09-05T06:47:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 -- 172.21.15.86:0/3413610352 learned_addr learned my addr 172.21.15.86:0/3413610352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01567fc640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a1ab40 0x55ff53a1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 -- 172.21.15.86:0/3413610352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a1ab40 msgr2=0x55ff53a1de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a1ab40 0x55ff53a1de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 -- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a195d0 msgr2=0x55ff53a2cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a195d0 0x55ff53a2cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 -- 172.21.15.86:0/3413610352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff53a9dc20 con 0x55ff53a24370 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f01577fe640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f01500edd30 tx=0x7f01500bc510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4520bc57bc382d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f013aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01500bbf70 con 0x55ff53a24370 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f01500a5b50 con 0x55ff53a24370 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.541+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014c034070 con 0x55ff53a24370 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01500bc150 con 0x55ff53a24370 2024-09-05T06:47:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 10 client.16998 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff53a9dc20 con 0x55ff53a24370 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f014c034070 con 0x55ff53a24370 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f015004d070 con 0x55ff53a24370 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 10 client.16998 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 10 client.16998 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 20 client.16998 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a9f120 0x55ff53aa1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 4 client.16998 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff53986d70 con 0x55ff53a9f120 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.542+0000 7f0156ffd640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a9f120 0x55ff53aa1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f013aff5640 10 client.16998 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f0156ffd640 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a9f120 0x55ff53aa1510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4e9a0c358cf49c6 server_cookie=3d90c19edeed1769 in_seq=0 out_seq=0 2024-09-05T06:47:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f013aff5640 1 -- 172.21.15.86:0/3413610352 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff53986d70 con 0x55ff53a9f120 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f013aff5640 10 client.16998 handle_command_reply: tid=0 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 client.16998 shutdown 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 msgr2=0x55ff53a1e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f01500edd30 tx=0x7f01500bc510 comp rx=0 tx=0).stop 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 shutdown_connections 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ff53a9f120 0x55ff53aa1510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff53a24370 0x55ff53a1e5c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff53a1ab40 0x55ff53a1de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 --2- 172.21.15.86:0/3413610352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff53a195d0 0x55ff53a2cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 >> 172.21.15.86:0/3413610352 conn(0x55ff539e2310 msgr2=0x55ff53a230c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f013aff5640 0 client.16998 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 shutdown_connections 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 1 -- 172.21.15.86:0/3413610352 wait complete. 2024-09-05T06:47:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.543+0000 7f015f929740 20 client.16998 trim_cache size 0 max 16384 2024-09-05T06:47:53.551 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:53.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 msgr2=0x7f01400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 0x7f01400a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f014c00f850 tx=0x7f014c00e000 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 msgr2=0x7f01580fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 0x7f01580fd700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0150005cc0 tx=0x7f0150031f40 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 shutdown_connections 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01400a2d70 0x7f01400a5230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0158102ef0 0x7f01580fd700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0158102570 0x7f01580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 --2- 172.21.15.86:0/726622702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0158100e20 0x7f01580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 >> 172.21.15.86:0/726622702 conn(0x7f01580fc7d0 msgr2=0x7f015806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 shutdown_connections 2024-09-05T06:47:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:53.544+0000 7f015d550640 1 -- 172.21.15.86:0/726622702 wait complete. 2024-09-05T06:47:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:53 smithi086 ceph-mon[30943]: pgmap v202: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:53 smithi096 ceph-mon[34468]: pgmap v202: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:53 smithi115 ceph-mon[34791]: pgmap v202: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:54.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:47:54.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 120262 1.81 MB/sec execute 194 sec latency 13.715 ms 2024-09-05T06:47:54.628 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.120+0000 7fdb2572e640 1 -- 172.21.15.86:0/1859427428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20101e20 msgr2=0x7fdb201056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.120+0000 7fdb2572e640 1 --2- 172.21.15.86:0/1859427428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20101e20 0x7fdb201056b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009900 tx=0x7fdb0802a410 comp rx=0 tx=0).stop 2024-09-05T06:47:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 -- 172.21.15.86:0/1859427428 shutdown_connections 2024-09-05T06:47:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 --2- 172.21.15.86:0/1859427428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20105de0 0x7fdb2010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 --2- 172.21.15.86:0/1859427428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20101e20 0x7fdb201056b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 --2- 172.21.15.86:0/1859427428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101470 0x7fdb20101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 -- 172.21.15.86:0/1859427428 >> 172.21.15.86:0/1859427428 conn(0x7fdb20077230 msgr2=0x7fdb200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 -- 172.21.15.86:0/1859427428 shutdown_connections 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 -- 172.21.15.86:0/1859427428 wait complete. 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 Processor -- start 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.121+0000 7fdb2572e640 1 -- start start 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20101470 0x7fdb200fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 0x7fdb200fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20110cf0 con 0x7fdb20105de0 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20110b70 con 0x7fdb20101470 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20110e70 con 0x7fdb20101e20 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58838/0 (socket says 172.21.15.86:58838) 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 -- 172.21.15.86:0/4119820071 learned_addr learned my addr 172.21.15.86:0/4119820071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb17fff640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 0x7fdb200fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1effd640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20101470 0x7fdb200fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 -- 172.21.15.86:0/4119820071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 msgr2=0x7fdb200fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 0x7fdb200fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 -- 172.21.15.86:0/4119820071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20101470 msgr2=0x7fdb200fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20101470 0x7fdb200fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 -- 172.21.15.86:0/4119820071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb201a6810 con 0x7fdb20105de0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb17fff640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 0x7fdb200fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1f7fe640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800eb10 tx=0x7fdb1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1cff9640 1 -- 172.21.15.86:0/4119820071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18004b20 con 0x7fdb20105de0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1cff9640 1 -- 172.21.15.86:0/4119820071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb18004cc0 con 0x7fdb20105de0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201a6b00 con 0x7fdb20105de0 2024-09-05T06:47:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.122+0000 7fdb1cff9640 1 -- 172.21.15.86:0/4119820071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18010690 con 0x7fdb20105de0 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.123+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201a7010 con 0x7fdb20105de0 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb1cff9640 1 -- 172.21.15.86:0/4119820071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb18005020 con 0x7fdb20105de0 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 Processor -- start 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 -- start start 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 0x555b4f0e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0ee370 0x555b4f0f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b4f0492f0 con 0x555b4f0e4b40 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b4eff0cf0 con 0x555b4f0ee370 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.125+0000 7fdb27b07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555b4f0452b0 con 0x555b4f0e35d0 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58844/0 (socket says 172.21.15.86:58844) 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb17fff640 1 -- 172.21.15.86:0/3316201838 learned_addr learned my addr 172.21.15.86:0/3316201838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 0x555b4f0e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0ee370 0x555b4f0f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1cff9640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 0x7fdb0c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 -- 172.21.15.86:0/3316201838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0ee370 msgr2=0x555b4f0f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0ee370 0x555b4f0f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 -- 172.21.15.86:0/3316201838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 msgr2=0x555b4f0e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 -- 172.21.15.86:0/3316201838 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4efd53a0 con 0x555b4f0e35d0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1f7fe640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 0x7fdb0c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1cff9640 1 -- 172.21.15.86:0/4119820071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdb180cfcc0 con 0x7fdb20105de0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 0x555b4f0e3a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009c00 tx=0x7fdb10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989855f490075e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.126+0000 7fdb157fa640 1 -- 172.21.15.86:0/3316201838 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004230 con 0x555b4f0e35d0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb157fa640 1 -- 172.21.15.86:0/3316201838 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb100043d0 con 0x555b4f0e35d0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb157fa640 1 -- 172.21.15.86:0/3316201838 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004c60 con 0x555b4f0e35d0 2024-09-05T06:47:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 -- 172.21.15.86:0/3316201838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 msgr2=0x555b4f0e3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb1f7fe640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 0x7fdb0c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb1809bb00 tx=0x7fdb180b0040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 0x555b4f0e3a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009c00 tx=0x7fdb10011810 comp rx=0 tx=0).stop 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 -- 172.21.15.86:0/3316201838 shutdown_connections 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0ee370 0x555b4f0f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0e35d0 0x555b4f0e3a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3316201838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 -- 172.21.15.86:0/3316201838 >> 172.21.15.86:0/3316201838 conn(0x555b4f0ac310 msgr2=0x555b4f0dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.127+0000 7fdb27b07740 1 -- 172.21.15.86:0/3316201838 shutdown_connections 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 -- 172.21.15.86:0/3316201838 wait complete. 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 -- start start 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 Processor -- start 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f0eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b4f0f1070 con 0x555b4f0e35d0 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b4f004060 con 0x555b4f0e4b40 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555b4f0f11f0 con 0x555b4f0ee370 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb27b07740 10 client.0 _ll_register_callbacks cb 0x555b4ed578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58852/0 (socket says 172.21.15.86:58852) 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 -- 172.21.15.86:0/378099359 learned_addr learned my addr 172.21.15.86:0/378099359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb1effd640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f0eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 -- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 msgr2=0x555b4f0eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 -- 172.21.15.86:0/378099359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 msgr2=0x555b4f0eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.128+0000 7fdb17fff640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f0eaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb17fff640 1 -- 172.21.15.86:0/378099359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4edba3c0 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb17fff640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006010 tx=0x7fdb08005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141ea92d016f444c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08035040 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb100043d0 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdb08004770 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb08053050 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 10 client.17010 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b4f0fcae0 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb100043d0 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdb0803c9b0 con 0x555b4f0e35d0 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 10 client.17010 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 10 client.17010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 20 client.17010 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f0fdef0 0x555b4f1002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 4 client.17010 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.129+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b4f056860 con 0x555b4f0fdef0 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.130+0000 7fdb1effd640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f0fdef0 0x555b4f1002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.130+0000 7fdb1effd640 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f0fdef0 0x555b4f1002e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3571d882b4b458fd server_cookie=edc8af8d6ddbed72 in_seq=0 out_seq=0 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.130+0000 7fdb067fc640 10 client.17010 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb067fc640 1 -- 172.21.15.86:0/378099359 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b4f056860 con 0x555b4f0fdef0 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb067fc640 10 client.17010 handle_command_reply: tid=0 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 client.17010 shutdown 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 msgr2=0x555b4f0edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006010 tx=0x7fdb08005b50 comp rx=0 tx=0).stop 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 shutdown_connections 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f0fdef0 0x555b4f1002e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f0eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb067fc640 0 client.17010 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 --2- 172.21.15.86:0/378099359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f0edcb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 >> 172.21.15.86:0/378099359 conn(0x555b4f0ac310 msgr2=0x555b4f0e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 shutdown_connections 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 1 -- 172.21.15.86:0/378099359 wait complete. 2024-09-05T06:47:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.131+0000 7fdb27b07740 20 client.17010 trim_cache size 0 max 16384 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.145+0000 7fdb27b07740 1 Processor -- start 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.145+0000 7fdb27b07740 1 -- start start 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 0x555b4f0fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0efca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201b11a0 con 0x555b4f0e4b40 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201b2ce0 con 0x555b4f0e35d0 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb27b07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201b1870 con 0x555b4f0ee370 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58854/0 (socket says 172.21.15.86:58854) 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 0x555b4f0fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb1effd640 1 -- 172.21.15.86:0/3532289993 learned_addr learned my addr 172.21.15.86:0/3532289993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 -- 172.21.15.86:0/3532289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 msgr2=0x555b4f0efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 -- 172.21.15.86:0/3532289993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 msgr2=0x555b4f0ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.146+0000 7fdb17fff640 1 -- 172.21.15.86:0/3532289993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4f0ee020 con 0x555b4f0e35d0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.147+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 0x555b4f0fa340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb08004720 tx=0x7fdb0802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc7c103738554f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.147+0000 7fdb14ff9640 1 -- 172.21.15.86:0/3532289993 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08035040 con 0x555b4f0e35d0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.147+0000 7fdb14ff9640 1 -- 172.21.15.86:0/3532289993 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb08004c20 con 0x555b4f0e35d0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.147+0000 7fdb14ff9640 1 -- 172.21.15.86:0/3532289993 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0803cae0 con 0x555b4f0e35d0 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 -- 172.21.15.86:0/3532289993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 msgr2=0x555b4f0fa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 0x555b4f0fa340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb08004720 tx=0x7fdb0802a410 comp rx=0 tx=0).stop 2024-09-05T06:47:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 -- 172.21.15.86:0/3532289993 shutdown_connections 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f0efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e4b40 0x555b4f0ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3532289993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e35d0 0x555b4f0fa340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 -- 172.21.15.86:0/3532289993 >> 172.21.15.86:0/3532289993 conn(0x555b4f0ac310 msgr2=0x555b4f0eaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 -- 172.21.15.86:0/3532289993 shutdown_connections 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.148+0000 7fdb27b07740 1 -- 172.21.15.86:0/3532289993 wait complete. 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.149+0000 7fdb27b07740 1 -- start start 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.149+0000 7fdb27b07740 1 Processor -- start 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.151+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.151+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.151+0000 7fdb27b07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f1062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.151+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.151+0000 7fdb1effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52840/0 (socket says 172.21.15.86:52840) 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 -- 172.21.15.86:0/3679551597 learned_addr learned my addr 172.21.15.86:0/3679551597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201b2610 con 0x555b4f0e35d0 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201b1f40 con 0x555b4f0e4b40 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201af180 con 0x555b4f0ee370 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb27b07740 10 client.0 _ll_register_callbacks cb 0x7fdb080095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 -- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 msgr2=0x555b4f1062d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:47:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f1062d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 -- 172.21.15.86:0/3679551597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 msgr2=0x555b4f105850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f105850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.152+0000 7fdb1effd640 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4f0f4b00 con 0x555b4f0e4b40 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb1effd640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x555b4f0e0ac0 tx=0x7fdb10009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e525eddaa5b14dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb067fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1001e040 con 0x555b4f0e4b40 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7fdb1002c6d0 con 0x555b4f0e4b40 2024-09-05T06:47:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.153+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb08004c20 con 0x555b4f0e4b40 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb1002c910 con 0x555b4f0e4b40 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 10 client.26279 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b4f168ec0 con 0x555b4f0e4b40 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb08004c20 con 0x555b4f0e4b40 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fdb10024040 con 0x555b4f0e4b40 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 10 client.26279 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 10 client.26279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 20 client.26279 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f169530 0x555b4f16b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 4 client.26279 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b4f050d70 con 0x555b4f169530 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.154+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f169530 0x555b4f16b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb067fc640 10 client.26279 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb17fff640 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f169530 0x555b4f16b970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ca93177dd3a1a55 server_cookie=bad45148a6faa6ef in_seq=0 out_seq=0 2024-09-05T06:47:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb067fc640 1 -- 172.21.15.86:0/3679551597 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b4f050d70 con 0x555b4f169530 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb067fc640 10 client.26279 handle_command_reply: tid=0 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 client.26279 shutdown 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 msgr2=0x555b4f105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x555b4f0e0ac0 tx=0x7fdb10009680 comp rx=0 tx=0).stop 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 shutdown_connections 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555b4f169530 0x555b4f16b970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb067fc640 0 client.26279 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555b4f0ee370 0x555b4f1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b4f0e4b40 0x555b4f105d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 --2- 172.21.15.86:0/3679551597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b4f0e35d0 0x555b4f105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 >> 172.21.15.86:0/3679551597 conn(0x555b4f0ac310 msgr2=0x555b4f0de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 shutdown_connections 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.155+0000 7fdb27b07740 1 -- 172.21.15.86:0/3679551597 wait complete. 2024-09-05T06:47:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb27b07740 20 client.26279 trim_cache size 0 max 16384 2024-09-05T06:47:55.163 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 msgr2=0x7fdb0c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 0x7fdb0c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb1809bb00 tx=0x7fdb180b0040 comp rx=0 tx=0).stop 2024-09-05T06:47:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 msgr2=0x7fdb200fd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800eb10 tx=0x7fdb1800c2d0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 shutdown_connections 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdb0c0a2d70 0x7fdb0c0a5230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb20105de0 0x7fdb200fd180 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb20101e20 0x7fdb200fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 --2- 172.21.15.86:0/4119820071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb20101470 0x7fdb200fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 >> 172.21.15.86:0/4119820071 conn(0x7fdb20077230 msgr2=0x7fdb201002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 shutdown_connections 2024-09-05T06:47:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:55.156+0000 7fdb2572e640 1 -- 172.21.15.86:0/4119820071 wait complete. 2024-09-05T06:47:55.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 120730 1.83 MB/sec execute 195 sec latency 13.980 ms 2024-09-05T06:47:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:55 smithi096 ceph-mon[34468]: pgmap v203: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:55 smithi115 ceph-mon[34791]: pgmap v203: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:55.982 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:55 smithi086 ceph-mon[30943]: pgmap v203: 129 pgs: 129 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-05T06:47:56.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:47:56.273 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:56.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 121068 1.82 MB/sec execute 196 sec latency 16.656 ms 2024-09-05T06:47:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 -- 172.21.15.86:0/3096010092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980102d90 msgr2=0x7f3980103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 --2- 172.21.15.86:0/3096010092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980102d90 0x7f3980103170 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3970009930 tx=0x7f397002a420 comp rx=0 tx=0).stop 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 -- 172.21.15.86:0/3096010092 shutdown_connections 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 --2- 172.21.15.86:0/3096010092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980108290 0x7f398010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 --2- 172.21.15.86:0/3096010092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f3980107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 --2- 172.21.15.86:0/3096010092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980102d90 0x7f3980103170 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 -- 172.21.15.86:0/3096010092 >> 172.21.15.86:0/3096010092 conn(0x7f39800fc7d0 msgr2=0x7f39800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 -- 172.21.15.86:0/3096010092 shutdown_connections 2024-09-05T06:47:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.778+0000 7f3985167640 1 -- 172.21.15.86:0/3096010092 wait complete. 2024-09-05T06:47:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 Processor -- start 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 -- start start 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 0x7f3980198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f39801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3980111f40 con 0x7f3980108290 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3980111dc0 con 0x7f3980102d90 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f3985167640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39801120c0 con 0x7f3980103740 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f397f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f397f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58886/0 (socket says 172.21.15.86:58886) 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f397f59e640 1 -- 172.21.15.86:0/2269514644 learned_addr learned my addr 172.21.15.86:0/2269514644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.779+0000 7f397ed9d640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 0x7f3980198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397e59c640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f39801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 -- 172.21.15.86:0/2269514644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 msgr2=0x7f39801989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f39801989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 -- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 msgr2=0x7f398019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 -- 172.21.15.86:0/2269514644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398019d430 con 0x7f3980102d90 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397e59c640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f39801989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397f59e640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f397ed9d640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 0x7f3980198480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3970031440 tx=0x7f3970004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f3967fff640 1 -- 172.21.15.86:0/2269514644 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397003f070 con 0x7f3980102d90 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f3967fff640 1 -- 172.21.15.86:0/2269514644 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3970003ca0 con 0x7f3980102d90 2024-09-05T06:47:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398019d6c0 con 0x7f3980102d90 2024-09-05T06:47:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f3967fff640 1 -- 172.21.15.86:0/2269514644 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970035070 con 0x7f3980102d90 2024-09-05T06:47:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.780+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39801a5000 con 0x7f3980102d90 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3967fff640 1 -- 172.21.15.86:0/2269514644 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f397003b4f0 con 0x7f3980102d90 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3987540740 1 Processor -- start 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3987540740 1 -- start start 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3967fff640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 0x7f39680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3987540740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3987540740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d75d0 0x55dad78d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.783+0000 7f3987540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f3987540740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dad783d2f0 con 0x55dad78d8b40 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f3987540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dad77e4cf0 con 0x55dad78e2370 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f3987540740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dad78392b0 con 0x55dad78d75d0 2024-09-05T06:47:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 0x7f39680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f3967fff640 1 -- 172.21.15.86:0/2269514644 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f39700efbe0 con 0x7f3980102d90 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58888/0 (socket says 172.21.15.86:58888) 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397ed9d640 1 -- 172.21.15.86:0/287071542 learned_addr learned my addr 172.21.15.86:0/287071542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397e59c640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d75d0 0x55dad78d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 -- 172.21.15.86:0/287071542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d75d0 msgr2=0x55dad78d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d75d0 0x55dad78d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 -- 172.21.15.86:0/287071542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 msgr2=0x55dad78d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 -- 172.21.15.86:0/287071542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad77c93a0 con 0x55dad78e2370 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397ed9d640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 0x7f39680a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3978009fd0 tx=0x7f3978013460 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.784+0000 7f397f59e640 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78e4760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3978013fc0 tx=0x7f397801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6e18804e5d97ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f39657fa640 1 -- 172.21.15.86:0/287071542 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397800edc0 con 0x55dad78e2370 2024-09-05T06:47:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f39657fa640 1 -- 172.21.15.86:0/287071542 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978026070 con 0x55dad78e2370 2024-09-05T06:47:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f39657fa640 1 -- 172.21.15.86:0/287071542 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978010ce0 con 0x55dad78e2370 2024-09-05T06:47:56.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 -- 172.21.15.86:0/287071542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 msgr2=0x55dad78e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78e4760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3978013fc0 tx=0x7f397801d000 comp rx=0 tx=0).stop 2024-09-05T06:47:56.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 -- 172.21.15.86:0/287071542 shutdown_connections 2024-09-05T06:47:56.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78e4760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d75d0 0x55dad78d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 --2- 172.21.15.86:0/287071542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d8b40 0x55dad78d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 -- 172.21.15.86:0/287071542 >> 172.21.15.86:0/287071542 conn(0x55dad78a0310 msgr2=0x55dad78d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 -- 172.21.15.86:0/287071542 shutdown_connections 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.785+0000 7f3987540740 1 -- 172.21.15.86:0/287071542 wait complete. 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 -- start start 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 Processor -- start 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78df640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dad78e9420 con 0x55dad78d75d0 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dad77f8060 con 0x55dad78d8b40 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dad78e95a0 con 0x55dad78e2370 2024-09-05T06:47:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f3987540740 10 client.0 _ll_register_callbacks cb 0x55dad754b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58902/0 (socket says 172.21.15.86:58902) 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f397ed9d640 1 -- 172.21.15.86:0/923164119 learned_addr learned my addr 172.21.15.86:0/923164119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f397f59e640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.786+0000 7f397e59c640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 -- 172.21.15.86:0/923164119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 msgr2=0x55dad78df640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78df640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 -- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 msgr2=0x55dad78df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78df030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 -- 172.21.15.86:0/923164119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad75ae3c0 con 0x55dad78d75d0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f397ed9d640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f39700b8180 tx=0x7f39700bbb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14751642543ccd5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700a3060 con 0x55dad78d75d0 2024-09-05T06:47:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f39700b8390 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978026070 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3970048040 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3987540740 10 client.17028 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dad78f4f40 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.787+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978026070 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f39700d3070 con 0x55dad78d75d0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 10 client.17028 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 20 client.17028 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad78f63f0 0x55dad78f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 4 client.17028 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dad784a860 con 0x55dad78f63f0 2024-09-05T06:47:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f397e59c640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad78f63f0 0x55dad78f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f397e59c640 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad78f63f0 0x55dad78f8870 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ee045c410a5c87 server_cookie=85539bae52d75e99 in_seq=0 out_seq=0 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.788+0000 7f3962ff5640 10 client.17028 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.789+0000 7f3962ff5640 1 -- 172.21.15.86:0/923164119 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dad784a860 con 0x55dad78f63f0 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.789+0000 7f3962ff5640 10 client.17028 handle_command_reply: tid=0 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.789+0000 7f3987540740 1 client.17028 shutdown 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 msgr2=0x55dad78e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f39700b8180 tx=0x7f39700bbb70 comp rx=0 tx=0).stop 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 shutdown_connections 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad78f63f0 0x55dad78f8870 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 --2- 172.21.15.86:0/923164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d75d0 0x55dad78e1d80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 >> 172.21.15.86:0/923164119 conn(0x55dad78a0310 msgr2=0x55dad78d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3962ff5640 0 client.17028 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 shutdown_connections 2024-09-05T06:47:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 1 -- 172.21.15.86:0/923164119 wait complete. 2024-09-05T06:47:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.790+0000 7f3987540740 20 client.17028 trim_cache size 0 max 16384 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 Processor -- start 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 -- start start 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78d9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39801acdf0 con 0x55dad78d3f20 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39801ae930 con 0x55dad78d8b40 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.804+0000 7f3987540740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39801ad4c0 con 0x55dad78e2370 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58914/0 (socket says 172.21.15.86:58914) 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 -- 172.21.15.86:0/2286379359 learned_addr learned my addr 172.21.15.86:0/2286379359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397e59c640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78d9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397ed9d640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 -- 172.21.15.86:0/2286379359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 msgr2=0x55dad78d9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78d9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 -- 172.21.15.86:0/2286379359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 msgr2=0x55dad78f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 -- 172.21.15.86:0/2286379359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad78d83a0 con 0x55dad78d3f20 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397e59c640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f397f59e640 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3978009aa0 tx=0x7f3978010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4497826b5bc7cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f3964ff9640 1 -- 172.21.15.86:0/2286379359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397802e020 con 0x55dad78d3f20 2024-09-05T06:47:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.805+0000 7f3964ff9640 1 -- 172.21.15.86:0/2286379359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978026040 con 0x55dad78d3f20 2024-09-05T06:47:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3964ff9640 1 -- 172.21.15.86:0/2286379359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978010430 con 0x55dad78d3f20 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 -- 172.21.15.86:0/2286379359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 msgr2=0x55dad78fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3978009aa0 tx=0x7f3978010a80 comp rx=0 tx=0).stop 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 -- 172.21.15.86:0/2286379359 shutdown_connections 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78fe140 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78e2370 0x55dad78d9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 --2- 172.21.15.86:0/2286379359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78d8b40 0x55dad78f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 -- 172.21.15.86:0/2286379359 >> 172.21.15.86:0/2286379359 conn(0x55dad78a0310 msgr2=0x55dad78d9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 -- 172.21.15.86:0/2286379359 shutdown_connections 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.806+0000 7f3987540740 1 -- 172.21.15.86:0/2286379359 wait complete. 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 -- start start 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 Processor -- start 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 0x55dad78dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39801ae260 con 0x55dad78d3f20 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39801adb90 con 0x55dad78e2370 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39801aadd0 con 0x55dad78d8b40 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f3987540740 10 client.0 _ll_register_callbacks cb 0x7f39700ecef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58918/0 (socket says 172.21.15.86:58918) 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 -- 172.21.15.86:0/4092266321 learned_addr learned my addr 172.21.15.86:0/4092266321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397f59e640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397e59c640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 0x55dad78dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 -- 172.21.15.86:0/4092266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 msgr2=0x55dad78dc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 0x55dad78dc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 -- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 msgr2=0x55dad78dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397ed9d640 1 -- 172.21.15.86:0/4092266321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad79601e0 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.807+0000 7f397e59c640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 0x55dad78dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f397ed9d640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f39700ed000 tx=0x7f39700b85c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b39a94fc050f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700b7fc0 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f39700037f0 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978026040 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f39700bc420 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 10 client.17034 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dad79601e0 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978026040 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f39700d3070 con 0x55dad78d3f20 2024-09-05T06:47:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 10 client.17034 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 20 client.17034 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.808+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad7961830 0x55dad7963c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 4 client.17034 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dad7844d70 con 0x55dad7961830 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f397e59c640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad7961830 0x55dad7963c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f397e59c640 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad7961830 0x55dad7963c20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7fa935f87dbd5b9 server_cookie=311ac1cce2e8a1ed in_seq=0 out_seq=0 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3962ff5640 10 client.17034 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3962ff5640 1 -- 172.21.15.86:0/4092266321 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dad7844d70 con 0x55dad7961830 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3962ff5640 10 client.17034 handle_command_reply: tid=0 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 client.17034 shutdown 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 msgr2=0x55dad78dc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f39700ed000 tx=0x7f39700b85c0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 shutdown_connections 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55dad7961830 0x55dad7963c20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad78e2370 0x55dad78dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad78d8b40 0x55dad78dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3962ff5640 0 client.17034 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 --2- 172.21.15.86:0/4092266321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad78d3f20 0x55dad78dc110 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 >> 172.21.15.86:0/4092266321 conn(0x55dad78a0310 msgr2=0x55dad78e0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.809+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 shutdown_connections 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3987540740 1 -- 172.21.15.86:0/4092266321 wait complete. 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3987540740 20 client.17034 trim_cache size 0 max 16384 2024-09-05T06:47:56.815 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 msgr2=0x7f39680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 0x7f39680a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3978009fd0 tx=0x7f3978013460 comp rx=0 tx=0).stop 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 msgr2=0x7f3980198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 0x7f3980198480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3970031440 tx=0x7f3970004270 comp rx=0 tx=0).stop 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 shutdown_connections 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39680a2d70 0x7f39680a5230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3980108290 0x7f398019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3980103740 0x7f39801989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 --2- 172.21.15.86:0/2269514644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102d90 0x7f3980198480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 >> 172.21.15.86:0/2269514644 conn(0x7f39800fc7d0 msgr2=0x7f3980106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 shutdown_connections 2024-09-05T06:47:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:56.810+0000 7f3985167640 1 -- 172.21.15.86:0/2269514644 wait complete. 2024-09-05T06:47:57.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 121431 1.82 MB/sec execute 197 sec latency 45.426 ms 2024-09-05T06:47:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:57 smithi096 ceph-mon[34468]: pgmap v204: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:47:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:57 smithi115 ceph-mon[34791]: pgmap v204: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:47:57.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:57 smithi086 ceph-mon[30943]: pgmap v204: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:47:58.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 122036 1.83 MB/sec execute 198 sec latency 18.491 ms 2024-09-05T06:47:58.580 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:47:58.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:47:58.855 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 -- 172.21.15.86:0/1802551518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 msgr2=0x7f29a01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 --2- 172.21.15.86:0/1802551518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a01043d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2994009900 tx=0x7f299402a410 comp rx=0 tx=0).stop 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 -- 172.21.15.86:0/1802551518 shutdown_connections 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 --2- 172.21.15.86:0/1802551518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 0x7f29a0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 --2- 172.21.15.86:0/1802551518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 0x7f29a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 --2- 172.21.15.86:0/1802551518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a01043d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 -- 172.21.15.86:0/1802551518 >> 172.21.15.86:0/1802551518 conn(0x7f29a00ffab0 msgr2=0x7f29a0101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 -- 172.21.15.86:0/1802551518 shutdown_connections 2024-09-05T06:47:59.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.383+0000 7f29a7243640 1 -- 172.21.15.86:0/1802551518 wait complete. 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 Processor -- start 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 -- start start 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 0x7f29a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 0x7f29a019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0114050 con 0x7f29a0103ff0 2024-09-05T06:47:59.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113ed0 con 0x7f29a01049a0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.384+0000 7f29a7243640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01141d0 con 0x7f29a0109120 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58950/0 (socket says 172.21.15.86:58950) 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 -- 172.21.15.86:0/4290466822 learned_addr learned my addr 172.21.15.86:0/4290466822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f299ffff640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 0x7f29a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a57e0640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 0x7f29a019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 -- 172.21.15.86:0/4290466822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 msgr2=0x7f29a019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 0x7f29a019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 -- 172.21.15.86:0/4290466822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 msgr2=0x7f29a019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 0x7f29a019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 -- 172.21.15.86:0/4290466822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a019f540 con 0x7f29a0103ff0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2994005ab0 tx=0x7f2994003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.385+0000 7f299dffb640 1 -- 172.21.15.86:0/4290466822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299403f070 con 0x7f29a0103ff0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.386+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a019f7d0 con 0x7f29a0103ff0 2024-09-05T06:47:59.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.386+0000 7f299dffb640 1 -- 172.21.15.86:0/4290466822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2994031d60 con 0x7f29a0103ff0 2024-09-05T06:47:59.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.386+0000 7f299dffb640 1 -- 172.21.15.86:0/4290466822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994035030 con 0x7f29a0103ff0 2024-09-05T06:47:59.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.386+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a7110 con 0x7f29a0103ff0 2024-09-05T06:47:59.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.389+0000 7f299dffb640 1 -- 172.21.15.86:0/4290466822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2994042070 con 0x7f29a0103ff0 2024-09-05T06:47:59.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.389+0000 7f29a961c740 1 Processor -- start 2024-09-05T06:47:59.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 -- start start 2024-09-05T06:47:59.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 0x558379f6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558379ec6280 con 0x558379f60b60 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558379e6db10 con 0x558379f6bd40 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a961c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558379ec2240 con 0x558379f61480 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a57e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a57e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58966/0 (socket says 172.21.15.86:58966) 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a57e0640 1 -- 172.21.15.86:0/743442608 learned_addr learned my addr 172.21.15.86:0/743442608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f299ffff640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 0x558379f6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.390+0000 7f299dffb640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 0x7f29900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 -- 172.21.15.86:0/743442608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 msgr2=0x558379f6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f6b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 -- 172.21.15.86:0/743442608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 msgr2=0x558379f60f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 -- 172.21.15.86:0/743442608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558379e7a1f0 con 0x558379f6bd40 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f29a57e0640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299dffb640 1 -- 172.21.15.86:0/4290466822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f29940efc90 con 0x7f29a0103ff0 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f29a57e0640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 0x7f29900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f299ffff640 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 0x558379f6e130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f298c009bf0 tx=0x7f298c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a377f77da45577a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f298affd640 1 -- 172.21.15.86:0/743442608 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c004050 con 0x558379f6bd40 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f298affd640 1 -- 172.21.15.86:0/743442608 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c0054d0 con 0x558379f6bd40 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f298affd640 1 -- 172.21.15.86:0/743442608 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c004d60 con 0x558379f6bd40 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.391+0000 7f29a57e0640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 0x7f29900a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2998012610 tx=0x7f2998004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 -- 172.21.15.86:0/743442608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 msgr2=0x558379f6e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 0x558379f6e130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f298c009bf0 tx=0x7f298c011810 comp rx=0 tx=0).stop 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 -- 172.21.15.86:0/743442608 shutdown_connections 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f6bd40 0x558379f6e130 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 --2- 172.21.15.86:0/743442608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f60b60 0x558379f60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 -- 172.21.15.86:0/743442608 >> 172.21.15.86:0/743442608 conn(0x558379f5d0f0 msgr2=0x558379f5d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 -- 172.21.15.86:0/743442608 shutdown_connections 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.392+0000 7f29a961c740 1 -- 172.21.15.86:0/743442608 wait complete. 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 -- start start 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 Processor -- start 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f6bd40 0x558379f69620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558379f6e910 con 0x558379f6bd40 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558379e80f10 con 0x558379f60b60 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558379f6ea90 con 0x558379f61480 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a961c740 10 client.0 _ll_register_callbacks cb 0x558379e49a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f299ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a57e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f6bd40 0x558379f69620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.393+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52924/0 (socket says 172.21.15.86:52924) 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3330549328 learned_addr learned my addr 172.21.15.86:0/3330549328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f299ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49206/0 (socket says 172.21.15.86:49206) 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3330549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 msgr2=0x558379f690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3330549328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f6bd40 msgr2=0x558379f69620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f6bd40 0x558379f69620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3330549328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558379c376a0 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29940bbdb0 tx=0x7f2994009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31342539e84ab83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f297ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994049030 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f29940ed220 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c0054d0 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f29940bb780 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a961c740 10 client.26309 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.394+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558379f7a4a0 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c0054d0 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f299404c060 con 0x558379f60b60 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 10 client.26309 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 20 client.26309 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379f7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 4 client.26309 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558379ed37f0 con 0x558379f7b900 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f299ffff640 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379f7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f297ffff640 10 client.26309 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.395+0000 7f299ffff640 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379f7dd30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b951a352582f38a1 server_cookie=c94137828c453226 in_seq=0 out_seq=0 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f297ffff640 1 -- 172.21.15.86:0/3330549328 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558379ed37f0 con 0x558379f7b900 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f297ffff640 10 client.26309 handle_command_reply: tid=0 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f29a961c740 1 client.26309 shutdown 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 msgr2=0x558379f68ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29940bbdb0 tx=0x7f2994009260 comp rx=0 tx=0).stop 2024-09-05T06:47:59.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.396+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 shutdown_connections 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379f7dd30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f6bd40 0x558379f69620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f61480 0x558379f690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f297ffff640 0 client.26309 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 --2- 172.21.15.86:0/3330549328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f68ba0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 >> 172.21.15.86:0/3330549328 conn(0x558379f5d0f0 msgr2=0x558379f62e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 shutdown_connections 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 1 -- 172.21.15.86:0/3330549328 wait complete. 2024-09-05T06:47:59.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.397+0000 7f29a961c740 20 client.26309 trim_cache size 0 max 16384 2024-09-05T06:47:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 Processor -- start 2024-09-05T06:47:59.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 -- start start 2024-09-05T06:47:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f83af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01b12e0 con 0x558379f61480 2024-09-05T06:47:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01b2e20 con 0x558379f60b60 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a961c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01b19b0 con 0x558379f6bd40 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f299ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f83af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52926/0 (socket says 172.21.15.86:52926) 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3347241448 learned_addr learned my addr 172.21.15.86:0/3347241448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.418+0000 7f29a57e0640 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3347241448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 msgr2=0x558379f65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f65e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3347241448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 msgr2=0x558379f83af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f83af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 -- 172.21.15.86:0/3347241448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558379f5cf20 con 0x558379f60b60 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f299402a9d0 tx=0x7f299403b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d147c54db8b9180f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f298a7fc640 1 -- 172.21.15.86:0/3347241448 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994049050 con 0x558379f60b60 2024-09-05T06:47:59.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f298a7fc640 1 -- 172.21.15.86:0/3347241448 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29940f3430 con 0x558379f60b60 2024-09-05T06:47:59.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.419+0000 7f298a7fc640 1 -- 172.21.15.86:0/3347241448 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29940c47b0 con 0x558379f60b60 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- 172.21.15.86:0/3347241448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 msgr2=0x558379f835b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f299402a9d0 tx=0x7f299403b460 comp rx=0 tx=0).stop 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- 172.21.15.86:0/3347241448 shutdown_connections 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f83af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 --2- 172.21.15.86:0/3347241448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f835b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- 172.21.15.86:0/3347241448 >> 172.21.15.86:0/3347241448 conn(0x558379f5d0f0 msgr2=0x558379f62dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- 172.21.15.86:0/3347241448 shutdown_connections 2024-09-05T06:47:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- 172.21.15.86:0/3347241448 wait complete. 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 -- start start 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.420+0000 7f29a961c740 1 Processor -- start 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01b2750 con 0x558379f61480 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01b2080 con 0x558379f60b60 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01af2c0 con 0x558379f6bd40 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a961c740 10 client.0 _ll_register_callbacks cb 0x7f2994005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f299ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52932/0 (socket says 172.21.15.86:52932) 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f299ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58996/0 (socket says 172.21.15.86:58996) 2024-09-05T06:47:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 -- 172.21.15.86:0/2600517914 learned_addr learned my addr 172.21.15.86:0/2600517914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a57e0640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 -- 172.21.15.86:0/2600517914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 msgr2=0x558379f70cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f70cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 -- 172.21.15.86:0/2600517914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 msgr2=0x558379f74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a4fdf640 1 -- 172.21.15.86:0/2600517914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558379f711f0 con 0x558379f60b60 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f299ffff640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.421+0000 7f29a57e0640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a4fdf640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29940f35e0 tx=0x7f299403b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a2884100b4ffaf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994049070 con 0x558379f60b60 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9517+0+0 (secure 0 0 0) 0x7f29940c4760 con 0x558379f60b60 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29940f3430 con 0x558379f60b60 2024-09-05T06:47:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f29940c0120 con 0x558379f60b60 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 10 client.26315 fetch_fsmap learned FSMap version 32 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558379f711f0 con 0x558379f60b60 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29940f3430 con 0x558379f60b60 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f29940b7b70 con 0x558379f60b60 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 10 client.26315 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 20 client.26315 populate_metadata read hostname 'smithi086' 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379fe9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 4 client.26315 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558379ecdd00 con 0x558379f7b900 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.422+0000 7f299ffff640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379fe9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.423+0000 7f297ffff640 10 client.26315 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:59.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.423+0000 7f299ffff640 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379fe9060 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20754ed7e941b562 server_cookie=9bf86049af141eec in_seq=0 out_seq=0 2024-09-05T06:47:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f297ffff640 1 -- 172.21.15.86:0/2600517914 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558379ecdd00 con 0x558379f7b900 2024-09-05T06:47:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f297ffff640 10 client.26315 handle_command_reply: tid=0 2024-09-05T06:47:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f29a961c740 1 client.26315 shutdown 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 msgr2=0x558379f73ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29940f35e0 tx=0x7f299403b4d0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 shutdown_connections 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.424+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558379f7b900 0x558379fe9060 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558379f6bd40 0x558379f70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f297ffff640 0 client.26315 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558379f61480 0x558379f74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 --2- 172.21.15.86:0/2600517914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558379f60b60 0x558379f73ce0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 >> 172.21.15.86:0/2600517914 conn(0x558379f5d0f0 msgr2=0x558379f625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 shutdown_connections 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 1 -- 172.21.15.86:0/2600517914 wait complete. 2024-09-05T06:47:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a961c740 20 client.26315 trim_cache size 0 max 16384 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5ceda0f7-2897-4b21-b59d-325a33d8396a", 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:47:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 msgr2=0x7f29900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 0x7f29900a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2998012610 tx=0x7f2998004a40 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 msgr2=0x7f29a019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2994005ab0 tx=0x7f2994003c80 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 shutdown_connections 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29900a2cf0 0x7f29900a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29a0109120 0x7f29a019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29a01049a0 0x7f29a019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 --2- 172.21.15.86:0/4290466822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103ff0 0x7f29a019a590 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.425+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 >> 172.21.15.86:0/4290466822 conn(0x7f29a00ffab0 msgr2=0x7f29a0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.426+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 shutdown_connections 2024-09-05T06:47:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:47:59.426+0000 7f29a7243640 1 -- 172.21.15.86:0/4290466822 wait complete. 2024-09-05T06:47:59.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 122516 1.82 MB/sec execute 199 sec latency 46.708 ms 2024-09-05T06:47:59.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:59 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:47:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:59 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:59 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:47:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:47:59 smithi086 ceph-mon[30943]: pgmap v205: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-09-05T06:47:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:59 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:47:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:59 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:59 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:47:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:47:59 smithi096 ceph-mon[34468]: pgmap v205: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-09-05T06:47:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:59 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:47:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:59 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:47:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:59 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:47:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:47:59 smithi115 ceph-mon[34791]: pgmap v205: 129 pgs: 129 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-09-05T06:48:00.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:48:00.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 123100 1.83 MB/sec execute 200 sec latency 19.942 ms 2024-09-05T06:48:00.567 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.690 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:00.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- 172.21.15.86:0/3150265327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24101dd0 msgr2=0x7f8f241021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3150265327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24101dd0 0x7f8f241021b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f2002a410 comp rx=0 tx=0).stop 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- 172.21.15.86:0/3150265327 shutdown_connections 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3150265327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24106f00 0x7f8f2410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3150265327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24102780 0x7f8f241067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3150265327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24101dd0 0x7f8f241021b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- 172.21.15.86:0/3150265327 >> 172.21.15.86:0/3150265327 conn(0x7f8f240fd8b0 msgr2=0x7f8f240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- 172.21.15.86:0/3150265327 shutdown_connections 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- 172.21.15.86:0/3150265327 wait complete. 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 Processor -- start 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.078+0000 7f8f2c096640 1 -- start start 2024-09-05T06:48:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 0x7f8f2419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 0x7f8f2419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24111dd0 con 0x7f8f24106f00 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24111c50 con 0x7f8f24102780 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2c096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24111f50 con 0x7f8f24101dd0 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2a633640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 0x7f8f2419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2a633640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59000/0 (socket says 172.21.15.86:59000) 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 0x7f8f2419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2a633640 1 -- 172.21.15.86:0/3828602987 learned_addr learned my addr 172.21.15.86:0/3828602987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 -- 172.21.15.86:0/3828602987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 msgr2=0x7f8f2419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 0x7f8f2419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 -- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 msgr2=0x7f8f2419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 -- 172.21.15.86:0/3828602987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2419f580 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29e32640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 0x7f8f2419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f2a633640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f29631640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 0x7f8f2419ab10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e9f0 tx=0x7f8f1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.079+0000 7f8f1affd640 1 -- 172.21.15.86:0/3828602987 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400ccc0 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.080+0000 7f8f1affd640 1 -- 172.21.15.86:0/3828602987 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f140108e0 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.080+0000 7f8f1affd640 1 -- 172.21.15.86:0/3828602987 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14018880 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.080+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2419f870 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.080+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a7150 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.082+0000 7f8f1affd640 1 -- 172.21.15.86:0/3828602987 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8f14010a90 con 0x7f8f24102780 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 Processor -- start 2024-09-05T06:48:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 -- start start 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307540880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 0x55b30754b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 0x55b30754da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b3074a62e0 con 0x55b3075404a0 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b30744dce0 con 0x55b30754b680 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b3074a22a0 con 0x55b307540dc0 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f29631640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 0x55b30754da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.083+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 0x55b30754b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 0x55b30754b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49240/0 (socket says 172.21.15.86:49240) 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29e32640 1 -- 172.21.15.86:0/2238633107 learned_addr learned my addr 172.21.15.86:0/2238633107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f2a633640 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307540880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 -- 172.21.15.86:0/2238633107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 msgr2=0x55b30754b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 0x55b30754b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 -- 172.21.15.86:0/2238633107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 msgr2=0x55b307540880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f1affd640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 0x7f8f100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307540880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 -- 172.21.15.86:0/2238633107 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b307432440 con 0x55b30754b680 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f2a633640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 0x7f8f100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f29631640 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 0x55b30754da70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f140cce50 tx=0x7f8f14098030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8c836273dcc23a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f1affd640 1 -- 172.21.15.86:0/3828602987 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9693+0+0 (secure 0 0 0) 0x7f8f140cfb90 con 0x7f8f24102780 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.084+0000 7f8f0ffff640 1 -- 172.21.15.86:0/2238633107 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14026030 con 0x55b30754b680 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.085+0000 7f8f0ffff640 1 -- 172.21.15.86:0/2238633107 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f14010e00 con 0x55b30754b680 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.085+0000 7f8f0ffff640 1 -- 172.21.15.86:0/2238633107 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1409bcd0 con 0x55b30754b680 2024-09-05T06:48:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.085+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2238633107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 msgr2=0x55b30754da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.085+0000 7f8f2a633640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 0x7f8f100a5230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0049d0 tx=0x7f8f1c004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.085+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 0x55b30754da70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f140cce50 tx=0x7f8f14098030 comp rx=0 tx=0).stop 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2238633107 shutdown_connections 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30754b680 0x55b30754da70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b307540dc0 0x55b30754b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2238633107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307540880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2238633107 >> 172.21.15.86:0/2238633107 conn(0x55b30753ca30 msgr2=0x55b30753ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2238633107 shutdown_connections 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2238633107 wait complete. 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 -- start start 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.086+0000 7f8f2e46f740 1 Processor -- start 2024-09-05T06:48:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307548200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30754b680 0x55b307548740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b30754e1f0 con 0x55b307540dc0 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b307461050 con 0x55b3075404a0 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b30754e370 con 0x55b30754b680 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2e46f740 10 client.0 _ll_register_callbacks cb 0x55b3074a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29631640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307548200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29631640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307548200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59024/0 (socket says 172.21.15.86:59024) 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29631640 1 -- 172.21.15.86:0/1245524735 learned_addr learned my addr 172.21.15.86:0/1245524735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52984/0 (socket says 172.21.15.86:52984) 2024-09-05T06:48:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f2a633640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30754b680 0x55b307548740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 -- 172.21.15.86:0/1245524735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30754b680 msgr2=0x55b307548740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30754b680 0x55b307548740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 -- 172.21.15.86:0/1245524735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 msgr2=0x55b307548200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307548200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 -- 172.21.15.86:0/1245524735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3072173c0 con 0x55b3075404a0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f29e32640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f20005c40 tx=0x7f8f20031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ee1aa94c792f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.087+0000 7f8f0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20035040 con 0x55b3075404a0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3074b3850 con 0x55b3075404a0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9693+0+0 (secure 0 0 0) 0x7f8f20004240 con 0x55b3075404a0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f20004440 con 0x55b3075404a0 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 10 client.26327 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b307559b40 con 0x55b3075404a0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b3074b3850 con 0x55b3075404a0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8f20004c00 con 0x55b3075404a0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 10 client.26327 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 20 client.26327 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30755b000 0x55b30755d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 4 client.26327 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3074add60 con 0x55b30755b000 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.088+0000 7f8f29631640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30755b000 0x55b30755d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f0d7fa640 10 client.26327 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f29631640 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30755b000 0x55b30755d3f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b873615f5c30ce7f server_cookie=337e4ba30336fe3c in_seq=0 out_seq=0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/1245524735 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3074add60 con 0x55b30755b000 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f0d7fa640 10 client.26327 handle_command_reply: tid=0 2024-09-05T06:48:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f2e46f740 1 client.26327 shutdown 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 msgr2=0x55b30754af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.089+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f20005c40 tx=0x7f8f20031740 comp rx=0 tx=0).stop 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 shutdown_connections 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30755b000 0x55b30755d3f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30754b680 0x55b307548740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307548200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/1245524735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3075404a0 0x55b30754af50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f0d7fa640 0 client.26327 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 >> 172.21.15.86:0/1245524735 conn(0x55b30753ca30 msgr2=0x55b307542760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 shutdown_connections 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 1 -- 172.21.15.86:0/1245524735 wait complete. 2024-09-05T06:48:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.090+0000 7f8f2e46f740 20 client.26327 trim_cache size 0 max 16384 2024-09-05T06:48:01.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 Processor -- start 2024-09-05T06:48:01.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 -- start start 2024-09-05T06:48:01.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30753ca30 0x55b307557450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b307540dc0 0x55b3075631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b14d0 con 0x55b3075404a0 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b3010 con 0x55b307540dc0 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b1ba0 con 0x55b30753ca30 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59034/0 (socket says 172.21.15.86:59034) 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 -- 172.21.15.86:0/2486628117 learned_addr learned my addr 172.21.15.86:0/2486628117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f2a633640 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b307540dc0 0x55b3075631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29e32640 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30753ca30 0x55b307557450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 -- 172.21.15.86:0/2486628117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30753ca30 msgr2=0x55b307557450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30753ca30 0x55b307557450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 -- 172.21.15.86:0/2486628117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b307540dc0 msgr2=0x55b3075631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b307540dc0 0x55b3075631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.107+0000 7f8f29631640 1 -- 172.21.15.86:0/2486628117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30755d760 con 0x55b3075404a0 2024-09-05T06:48:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f29631640 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e9c0 tx=0x7f8f140d36e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcaeeb414d0d4e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f0f7fe640 1 -- 172.21.15.86:0/2486628117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1409ffd0 con 0x55b3075404a0 2024-09-05T06:48:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f0f7fe640 1 -- 172.21.15.86:0/2486628117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f140a4890 con 0x55b3075404a0 2024-09-05T06:48:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f0f7fe640 1 -- 172.21.15.86:0/2486628117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f140cd0d0 con 0x55b3075404a0 2024-09-05T06:48:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2486628117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 msgr2=0x55b307562c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.108+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e9c0 tx=0x7f8f140d36e0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2486628117 shutdown_connections 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b307540dc0 0x55b3075631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3075404a0 0x55b307562c90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/2486628117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b30753ca30 0x55b307557450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2486628117 >> 172.21.15.86:0/2486628117 conn(0x55b3075488a0 msgr2=0x55b307546ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2486628117 shutdown_connections 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 -- 172.21.15.86:0/2486628117 wait complete. 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 -- start start 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 Processor -- start 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.109+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3075404a0 0x55b307545070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307545800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b2940 con 0x55b307540dc0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b2270 con 0x55b30753ca30 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241af4d0 con 0x55b3075404a0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 10 client.0 _ll_register_callbacks cb 0x7f8f20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:53002/0 (socket says 172.21.15.86:53002) 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2a633640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307545800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 -- 172.21.15.86:0/458176882 learned_addr learned my addr 172.21.15.86:0/458176882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29631640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3075404a0 0x55b307545070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 -- 172.21.15.86:0/458176882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3075404a0 msgr2=0x55b307545070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3075404a0 0x55b307545070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 -- 172.21.15.86:0/458176882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 msgr2=0x55b307545800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307545800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 -- 172.21.15.86:0/458176882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3075c4c60 con 0x55b30753ca30 2024-09-05T06:48:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f29e32640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a450 tx=0x7f8f20004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f30287c88a5c7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20057020 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9693+0+0 (secure 0 0 0) 0x7f8f200053d0 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.110+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f140a4890 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f20062050 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 10 client.26333 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b3075c5dc0 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f140a4890 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8f20004010 con 0x55b30753ca30 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 10 client.26333 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 20 client.26333 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30754b680 0x55b3075c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 4 client.26333 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b30742a210 con 0x55b30754b680 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.111+0000 7f8f29631640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30754b680 0x55b3075c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f0d7fa640 10 client.26333 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f29631640 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30754b680 0x55b3075c84e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d2361d6e761687 server_cookie=7a04ead4c998a5fa in_seq=0 out_seq=0 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f0d7fa640 1 -- 172.21.15.86:0/458176882 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b30742a210 con 0x55b30754b680 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f0d7fa640 10 client.26333 handle_command_reply: tid=0 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 client.26333 shutdown 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 msgr2=0x55b307553e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002a450 tx=0x7f8f20004e50 comp rx=0 tx=0).stop 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 shutdown_connections 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b30754b680 0x55b3075c84e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b307540dc0 0x55b307545800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3075404a0 0x55b307545070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f0d7fa640 0 client.26333 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 --2- 172.21.15.86:0/458176882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b30753ca30 0x55b307553e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 >> 172.21.15.86:0/458176882 conn(0x55b3075488a0 msgr2=0x55b30753b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 shutdown_connections 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 1 -- 172.21.15.86:0/458176882 wait complete. 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.112+0000 7f8f2e46f740 20 client.26333 trim_cache size 0 max 16384 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:48:01.117 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: "5ceda0f7-2897-4b21-b59d-325a33d8396a": { 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5ceda0f7-2897-4b21-b59d-325a33d8396a", 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:01.118 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 msgr2=0x7f8f100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 0x7f8f100a5230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0049d0 tx=0x7f8f1c004a40 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 msgr2=0x7f8f2419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 0x7f8f2419ab10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e9f0 tx=0x7f8f1400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 shutdown_connections 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8f100a2d70 0x7f8f100a5230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f24106f00 0x7f8f2419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f24102780 0x7f8f2419ab10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 --2- 172.21.15.86:0/3828602987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f24101dd0 0x7f8f2419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 >> 172.21.15.86:0/3828602987 conn(0x7f8f240fd8b0 msgr2=0x7f8f240fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 shutdown_connections 2024-09-05T06:48:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:01.113+0000 7f8f2c096640 1 -- 172.21.15.86:0/3828602987 wait complete. 2024-09-05T06:48:01.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 123913 1.85 MB/sec execute 201 sec latency 21.587 ms 2024-09-05T06:48:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:01 smithi086 ceph-mon[30943]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T06:48:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:01 smithi086 ceph-mon[30943]: pgmap v207: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-05T06:48:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:01 smithi096 ceph-mon[34468]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T06:48:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:01 smithi096 ceph-mon[34468]: pgmap v207: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-05T06:48:01.881 INFO:tasks.cephfs.filesystem:scrub status for tag:5ceda0f7-2897-4b21-b59d-325a33d8396a - {'path': '/', 'tag': '5ceda0f7-2897-4b21-b59d-325a33d8396a', 'options': 'recursive,force'} 2024-09-05T06:48:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:01 smithi115 ceph-mon[34791]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T06:48:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:01 smithi115 ceph-mon[34791]: pgmap v207: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-05T06:48:02.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 124342 1.86 MB/sec execute 202 sec latency 18.262 ms 2024-09-05T06:48:03.236 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:02 smithi086 ceph-mon[30943]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T06:48:03.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:02 smithi096 ceph-mon[34468]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T06:48:03.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:02 smithi115 ceph-mon[34791]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T06:48:03.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 124642 1.86 MB/sec execute 203 sec latency 12.315 ms 2024-09-05T06:48:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:03 smithi086 ceph-mon[30943]: pgmap v209: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T06:48:04.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:03 smithi096 ceph-mon[34468]: pgmap v209: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T06:48:04.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:03 smithi115 ceph-mon[34791]: pgmap v209: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T06:48:04.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 125095 1.85 MB/sec execute 204 sec latency 18.801 ms 2024-09-05T06:48:05.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 125671 1.86 MB/sec execute 205 sec latency 19.959 ms 2024-09-05T06:48:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:05 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:05 smithi096 ceph-mon[34468]: pgmap v210: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-05T06:48:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:05 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:05 smithi115 ceph-mon[34791]: pgmap v210: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-05T06:48:05.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:05 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:05 smithi086 ceph-mon[30943]: pgmap v210: 129 pgs: 129 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-05T06:48:06.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 126233 1.86 MB/sec execute 206 sec latency 22.033 ms 2024-09-05T06:48:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:06.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:06.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:06.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:07.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 126777 1.86 MB/sec execute 207 sec latency 20.031 ms 2024-09-05T06:48:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:07 smithi096 ceph-mon[34468]: pgmap v211: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T06:48:07.887 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:07 smithi086 ceph-mon[30943]: pgmap v211: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T06:48:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:07 smithi115 ceph-mon[34791]: pgmap v211: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T06:48:08.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 127508 1.88 MB/sec execute 208 sec latency 11.913 ms 2024-09-05T06:48:09.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 128005 1.89 MB/sec execute 209 sec latency 11.523 ms 2024-09-05T06:48:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:09 smithi096 ceph-mon[34468]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-05T06:48:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:09 smithi115 ceph-mon[34791]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-05T06:48:09.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:09 smithi086 ceph-mon[30943]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-09-05T06:48:10.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 128295 1.89 MB/sec execute 210 sec latency 12.118 ms 2024-09-05T06:48:11.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 128810 1.89 MB/sec execute 211 sec latency 17.143 ms 2024-09-05T06:48:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:11 smithi096 ceph-mon[34468]: pgmap v213: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-05T06:48:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:11 smithi115 ceph-mon[34791]: pgmap v213: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-05T06:48:11.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:11 smithi086 ceph-mon[30943]: pgmap v213: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-09-05T06:48:12.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 129379 1.89 MB/sec execute 212 sec latency 21.355 ms 2024-09-05T06:48:13.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 130055 1.89 MB/sec execute 213 sec latency 22.137 ms 2024-09-05T06:48:13.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:13 smithi096 ceph-mon[34468]: pgmap v214: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-05T06:48:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:13 smithi115 ceph-mon[34791]: pgmap v214: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-05T06:48:13.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:13 smithi086 ceph-mon[30943]: pgmap v214: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-09-05T06:48:14.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 130704 1.91 MB/sec execute 214 sec latency 13.337 ms 2024-09-05T06:48:15.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 131280 1.92 MB/sec execute 215 sec latency 13.337 ms 2024-09-05T06:48:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:15 smithi096 ceph-mon[34468]: pgmap v215: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-05T06:48:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:15 smithi115 ceph-mon[34791]: pgmap v215: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-05T06:48:15.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:15 smithi086 ceph-mon[30943]: pgmap v215: 129 pgs: 129 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-09-05T06:48:16.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 131649 1.92 MB/sec execute 216 sec latency 13.166 ms 2024-09-05T06:48:17.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 131971 1.91 MB/sec execute 217 sec latency 16.255 ms 2024-09-05T06:48:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:17 smithi096 ceph-mon[34468]: pgmap v216: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-05T06:48:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:17 smithi115 ceph-mon[34791]: pgmap v216: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-05T06:48:17.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:17 smithi086 ceph-mon[30943]: pgmap v216: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-05T06:48:18.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 132528 1.91 MB/sec execute 218 sec latency 13.311 ms 2024-09-05T06:48:19.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 133102 1.92 MB/sec execute 219 sec latency 20.312 ms 2024-09-05T06:48:19.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:19 smithi096 ceph-mon[34468]: pgmap v217: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:48:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:19 smithi115 ceph-mon[34791]: pgmap v217: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:48:19.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:19 smithi086 ceph-mon[30943]: pgmap v217: 129 pgs: 129 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:48:20.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 133702 1.92 MB/sec execute 220 sec latency 17.019 ms 2024-09-05T06:48:21.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 134520 1.94 MB/sec execute 221 sec latency 13.652 ms 2024-09-05T06:48:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:21 smithi096 ceph-mon[34468]: pgmap v218: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-05T06:48:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:21 smithi115 ceph-mon[34791]: pgmap v218: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-05T06:48:21.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:21 smithi086 ceph-mon[30943]: pgmap v218: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.3 MiB/s wr, 197 op/s 2024-09-05T06:48:22.518 INFO:tasks.workunit.client.0.smithi086.stdout: 1 134965 1.95 MB/sec execute 222 sec latency 13.431 ms 2024-09-05T06:48:23.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 135282 1.95 MB/sec execute 223 sec latency 13.182 ms 2024-09-05T06:48:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:23 smithi096 ceph-mon[34468]: pgmap v219: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:23 smithi115 ceph-mon[34791]: pgmap v219: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:23.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:23 smithi086 ceph-mon[30943]: pgmap v219: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:24.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 135734 1.94 MB/sec execute 224 sec latency 16.402 ms 2024-09-05T06:48:25.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 136324 1.95 MB/sec execute 225 sec latency 18.960 ms 2024-09-05T06:48:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:25 smithi096 ceph-mon[34468]: pgmap v220: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:25 smithi115 ceph-mon[34791]: pgmap v220: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:25.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:25 smithi086 ceph-mon[30943]: pgmap v220: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-09-05T06:48:26.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 136905 1.95 MB/sec execute 226 sec latency 20.787 ms 2024-09-05T06:48:27.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 137441 1.95 MB/sec execute 227 sec latency 17.224 ms 2024-09-05T06:48:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:27 smithi096 ceph-mon[34468]: pgmap v221: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:48:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:27 smithi115 ceph-mon[34791]: pgmap v221: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:48:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:27 smithi086 ceph-mon[30943]: pgmap v221: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:48:28.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 138155 1.97 MB/sec execute 228 sec latency 13.236 ms 2024-09-05T06:48:29.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 138671 1.98 MB/sec execute 229 sec latency 10.598 ms 2024-09-05T06:48:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:29 smithi096 ceph-mon[34468]: pgmap v222: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:48:29.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:29 smithi115 ceph-mon[34791]: pgmap v222: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:48:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:29 smithi086 ceph-mon[30943]: pgmap v222: 129 pgs: 129 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-09-05T06:48:30.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 138969 1.97 MB/sec execute 230 sec latency 16.924 ms 2024-09-05T06:48:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:30 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:48:30.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:30 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:48:31.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:30 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:48:31.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 139498 1.97 MB/sec execute 231 sec latency 13.698 ms 2024-09-05T06:48:31.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:48:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:48:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:48:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:48:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:31 smithi115 ceph-mon[34791]: pgmap v223: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-05T06:48:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:48:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:48:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:48:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:31 smithi086 ceph-mon[30943]: pgmap v223: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-05T06:48:32.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:48:32.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:48:32.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:48:32.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:31 smithi096 ceph-mon[34468]: pgmap v223: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-05T06:48:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:32.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 140024 1.98 MB/sec execute 232 sec latency 32.639 ms 2024-09-05T06:48:32.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 -- 172.21.15.86:0/111142633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c102570 msgr2=0x7f402c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 --2- 172.21.15.86:0/111142633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c102570 0x7f402c1029b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f401402a480 comp rx=0 tx=0).stop 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 -- 172.21.15.86:0/111142633 shutdown_connections 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 --2- 172.21.15.86:0/111142633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102ef0 0x7f402c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 --2- 172.21.15.86:0/111142633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c102570 0x7f402c1029b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 --2- 172.21.15.86:0/111142633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c100e20 0x7f402c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 -- 172.21.15.86:0/111142633 >> 172.21.15.86:0/111142633 conn(0x7f402c0fc7d0 msgr2=0x7f402c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 -- 172.21.15.86:0/111142633 shutdown_connections 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.646+0000 7f4030c3c640 1 -- 172.21.15.86:0/111142633 wait complete. 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 Processor -- start 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 -- start start 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102570 0x7f402c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 0x7f402c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f402c110e30 con 0x7f402c100e20 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f402c110cb0 con 0x7f402c102570 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.647+0000 7f4030c3c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f402c110fb0 con 0x7f402c102ef0 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55776/0 (socket says 172.21.15.86:55776) 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 -- 172.21.15.86:0/2889484722 learned_addr learned my addr 172.21.15.86:0/2889484722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402ad9d640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 0x7f402c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f4023fff640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102570 0x7f402c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 -- 172.21.15.86:0/2889484722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 msgr2=0x7f402c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 0x7f402c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 -- 172.21.15.86:0/2889484722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102570 msgr2=0x7f402c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102570 0x7f402c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 -- 172.21.15.86:0/2889484722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c19d3f0 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402ad9d640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 0x7f402c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f402a59c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d950 tx=0x7f401c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f40237fe640 1 -- 172.21.15.86:0/2889484722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c017070 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f40237fe640 1 -- 172.21.15.86:0/2889484722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c009c30 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f40237fe640 1 -- 172.21.15.86:0/2889484722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00f620 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c19d6e0 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.648+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a4fc0 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.651+0000 7f40237fe640 1 -- 172.21.15.86:0/2889484722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f401c0109a0 con 0x7f402c100e20 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.651+0000 7f4033015740 1 Processor -- start 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.651+0000 7f4033015740 1 -- start start 2024-09-05T06:48:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45380 0x56505db45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 0x56505db56e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f40237fe640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 0x7f40180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56505daab2e0 con 0x56505db54a90 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56505da52ce0 con 0x56505db45380 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f402a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4023fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 0x56505db56e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f402a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52430/0 (socket says 172.21.15.86:52430) 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f402ad9d640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 0x7f40180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f402a59c640 1 -- 172.21.15.86:0/596215402 learned_addr learned my addr 172.21.15.86:0/596215402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:32.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56505daa72a0 con 0x56505db45ca0 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f402ad9d640 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45380 0x56505db45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f40237fe640 1 -- 172.21.15.86:0/2889484722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f401c0cf410 con 0x7f402c100e20 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4023fff640 1 -- 172.21.15.86:0/596215402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 msgr2=0x56505db54550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4023fff640 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.652+0000 7f4023fff640 1 -- 172.21.15.86:0/596215402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45380 msgr2=0x56505db45760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4023fff640 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45380 0x56505db45760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4023fff640 1 -- 172.21.15.86:0/596215402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56505da37440 con 0x56505db54a90 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f402a59c640 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4023fff640 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 0x56505db56e80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4014005940 tx=0x7f4014005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2c0672a009d891 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f402ad9d640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 0x7f40180a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4024005fd0 tx=0x7f4024011690 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4020ff9640 1 -- 172.21.15.86:0/596215402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014040070 con 0x56505db54a90 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4020ff9640 1 -- 172.21.15.86:0/596215402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4014004030 con 0x56505db54a90 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.653+0000 7f4020ff9640 1 -- 172.21.15.86:0/596215402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014035040 con 0x56505db54a90 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 msgr2=0x56505db56e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 0x56505db56e80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4014005940 tx=0x7f4014005c00 comp rx=0 tx=0).stop 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 shutdown_connections 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db54a90 0x56505db56e80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db45ca0 0x56505db54550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 --2- 172.21.15.86:0/596215402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45380 0x56505db45760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 >> 172.21.15.86:0/596215402 conn(0x56505db41a30 msgr2=0x56505db41e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 shutdown_connections 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- 172.21.15.86:0/596215402 wait complete. 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 -- start start 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.654+0000 7f4033015740 1 Processor -- start 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56505db57660 con 0x56505db45380 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56505da66050 con 0x56505db45ca0 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56505db577e0 con 0x56505db54a90 2024-09-05T06:48:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4033015740 10 client.0 _ll_register_callbacks cb 0x56505daa6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55794/0 (socket says 172.21.15.86:55794) 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 -- 172.21.15.86:0/3466452463 learned_addr learned my addr 172.21.15.86:0/3466452463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4023fff640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402ad9d640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 -- 172.21.15.86:0/3466452463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 msgr2=0x56505db51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 -- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 msgr2=0x56505db51680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db51680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 -- 172.21.15.86:0/3466452463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56505d81c3c0 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f402a59c640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f401c097f70 tx=0x7f401c06a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b9aceaff28e8fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.655+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c020030 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f401c082bd0 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4014004030 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f401c0ccba0 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 10 client.17082 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56505db48220 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4014004030 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f401c06d0d0 con 0x56505db45380 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 20 client.17082 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505db49630 0x56505db4bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 4 client.17082 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56505dab8850 con 0x56505db49630 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.656+0000 7f4023fff640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505db49630 0x56505db4bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.657+0000 7f4023fff640 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505db49630 0x56505db4bac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d87c38582692f5 server_cookie=48332298897d5a3d in_seq=0 out_seq=0 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.657+0000 7f4011ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4011ffb640 1 -- 172.21.15.86:0/3466452463 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56505dab8850 con 0x56505db49630 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4011ffb640 10 client.17082 handle_command_reply: tid=0 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 client.17082 shutdown 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 msgr2=0x56505db543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f401c097f70 tx=0x7f401c06a4d0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 shutdown_connections 2024-09-05T06:48:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505db49630 0x56505db4bac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 --2- 172.21.15.86:0/3466452463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45380 0x56505db543d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 >> 172.21.15.86:0/3466452463 conn(0x56505db41a30 msgr2=0x56505db47790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4011ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 shutdown_connections 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 1 -- 172.21.15.86:0/3466452463 wait complete. 2024-09-05T06:48:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.658+0000 7f4033015740 20 client.17082 trim_cache size 0 max 16384 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.678+0000 7f4033015740 1 Processor -- start 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.678+0000 7f4033015740 1 -- start start 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 0x56505db683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1aced0 con 0x56505db51cb0 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1aea10 con 0x56505db45ca0 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4033015740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ad5a0 con 0x56505db54a90 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f402ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 0x56505db683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52448/0 (socket says 172.21.15.86:52448) 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 -- 172.21.15.86:0/1910496471 learned_addr learned my addr 172.21.15.86:0/1910496471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f402a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 -- 172.21.15.86:0/1910496471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 msgr2=0x56505db60900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db60900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 -- 172.21.15.86:0/1910496471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 msgr2=0x56505db683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 0x56505db683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f4023fff640 1 -- 172.21.15.86:0/1910496471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56505db45ad0 con 0x56505db54a90 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.679+0000 7f402ad9d640 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 0x56505db683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.680+0000 7f4023fff640 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221af6d6a81bc3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.680+0000 7f4013fff640 1 -- 172.21.15.86:0/1910496471 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014040040 con 0x56505db54a90 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.680+0000 7f4013fff640 1 -- 172.21.15.86:0/1910496471 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40140318e0 con 0x56505db54a90 2024-09-05T06:48:32.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.680+0000 7f4013fff640 1 -- 172.21.15.86:0/1910496471 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014035040 con 0x56505db54a90 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- 172.21.15.86:0/1910496471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 msgr2=0x56505db67eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014005d50 comp rx=0 tx=0).stop 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- 172.21.15.86:0/1910496471 shutdown_connections 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db51cb0 0x56505db683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db54a90 0x56505db67eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- 172.21.15.86:0/1910496471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db45ca0 0x56505db60900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- 172.21.15.86:0/1910496471 >> 172.21.15.86:0/1910496471 conn(0x56505db41a30 msgr2=0x56505db4cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- 172.21.15.86:0/1910496471 shutdown_connections 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- 172.21.15.86:0/1910496471 wait complete. 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 -- start start 2024-09-05T06:48:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 Processor -- start 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 0x56505db59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.681+0000 7f4033015740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db54a90 0x56505dbc3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4033015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ae340 con 0x56505db45ca0 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4033015740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1adc70 con 0x56505db54a90 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4033015740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1aaed0 con 0x56505db51cb0 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4033015740 10 client.0 _ll_register_callbacks cb 0x7f401c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f402a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 0x56505db59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52462/0 (socket says 172.21.15.86:52462) 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 -- 172.21.15.86:0/452626024 learned_addr learned my addr 172.21.15.86:0/452626024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:32.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f402ad9d640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db54a90 0x56505dbc3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 -- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db54a90 msgr2=0x56505dbc3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db54a90 0x56505dbc3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 -- 172.21.15.86:0/452626024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 msgr2=0x56505db59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 0x56505db59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 -- 172.21.15.86:0/452626024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56505db50630 con 0x56505db51cb0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f402a59c640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 0x56505db59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4023fff640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4014009430 tx=0x7f4014031c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc914e241d3c76e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401405c060 con 0x56505db51cb0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56505dab2d60 con 0x56505db51cb0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.682+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f4014031430 con 0x56505db51cb0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4014063050 con 0x56505db51cb0 2024-09-05T06:48:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 10 client.26332 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56505dbcb6d0 con 0x56505db51cb0 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56505dab2d60 con 0x56505db51cb0 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f40140047e0 con 0x56505db51cb0 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 10 client.26332 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 20 client.26332 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505dbcb950 0x56505dbcdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 4 client.26332 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56505da2f210 con 0x56505dbcb950 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.683+0000 7f402a59c640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505dbcb950 0x56505dbcdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4011ffb640 10 client.26332 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f402a59c640 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505dbcb950 0x56505dbcdd40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357cf1be65d3587 server_cookie=f197977de85b0e5f in_seq=0 out_seq=0 2024-09-05T06:48:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4011ffb640 1 -- 172.21.15.86:0/452626024 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56505da2f210 con 0x56505dbcb950 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4011ffb640 10 client.26332 handle_command_reply: tid=0 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 client.26332 shutdown 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 msgr2=0x56505db59690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4014009430 tx=0x7f4014031c30 comp rx=0 tx=0).stop 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 shutdown_connections 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56505dbcb950 0x56505dbcdd40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56505db54a90 0x56505dbc3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56505db51cb0 0x56505db59690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4011ffb640 0 client.26332 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 --2- 172.21.15.86:0/452626024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56505db45ca0 0x56505db59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 >> 172.21.15.86:0/452626024 conn(0x56505db41a30 msgr2=0x56505db40350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 shutdown_connections 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 1 -- 172.21.15.86:0/452626024 wait complete. 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.684+0000 7f4033015740 20 client.26332 trim_cache size 0 max 16384 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:48:32.697 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 msgr2=0x7f40180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 0x7f40180a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4024005fd0 tx=0x7f4024011690 comp rx=0 tx=0).stop 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 msgr2=0x7f402c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d950 tx=0x7f401c00de10 comp rx=0 tx=0).stop 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 shutdown_connections 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40180a2cf0 0x7f40180a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c102ef0 0x7f402c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f402c102570 0x7f402c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 --2- 172.21.15.86:0/2889484722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c100e20 0x7f402c198440 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 >> 172.21.15.86:0/2889484722 conn(0x7f402c0fc7d0 msgr2=0x7f402c06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 shutdown_connections 2024-09-05T06:48:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:32.685+0000 7f4030c3c640 1 -- 172.21.15.86:0/2889484722 wait complete. 2024-09-05T06:48:33.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:48:33.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:48:33.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 140669 1.97 MB/sec execute 233 sec latency 18.876 ms 2024-09-05T06:48:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:33 smithi086 ceph-mon[30943]: pgmap v224: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:33 smithi096 ceph-mon[34468]: pgmap v224: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:33 smithi115 ceph-mon[34791]: pgmap v224: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 -- 172.21.15.86:0/1974364523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0102570 msgr2=0x7f51a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/1974364523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0102570 0x7f51a01029b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5194009890 tx=0x7f519402a410 comp rx=0 tx=0).stop 2024-09-05T06:48:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 -- 172.21.15.86:0/1974364523 shutdown_connections 2024-09-05T06:48:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/1974364523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102ef0 0x7f51a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/1974364523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0102570 0x7f51a01029b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/1974364523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0100e20 0x7f51a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 -- 172.21.15.86:0/1974364523 >> 172.21.15.86:0/1974364523 conn(0x7f51a00fc7d0 msgr2=0x7f51a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 -- 172.21.15.86:0/1974364523 shutdown_connections 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.298+0000 7f51a5bb0640 1 -- 172.21.15.86:0/1974364523 wait complete. 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 Processor -- start 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 -- start start 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0102ef0 0x7f51a019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0110e30 con 0x7f51a0102570 2024-09-05T06:48:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0110cb0 con 0x7f51a0100e20 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f51a5bb0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0110fb0 con 0x7f51a0102ef0 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50310/0 (socket says 172.21.15.86:50310) 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55832/0 (socket says 172.21.15.86:55832) 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519f7fe640 1 -- 172.21.15.86:0/2095525004 learned_addr learned my addr 172.21.15.86:0/2095525004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.299+0000 7f519ffff640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0102ef0 0x7f51a019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 -- 172.21.15.86:0/2095525004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0102ef0 msgr2=0x7f51a019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0102ef0 0x7f51a019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 -- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 msgr2=0x7f51a01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 -- 172.21.15.86:0/2095525004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a019d440 con 0x7f51a0100e20 2024-09-05T06:48:34.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519effd640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519f7fe640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f519000d770 tx=0x7f519000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017070 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a019d730 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51900054d0 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190004cf0 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.300+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01a5010 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.301+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a0101200 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.303+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5190004e90 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.303+0000 7f519cff9640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 0x7f51880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.303+0000 7f519effd640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 0x7f51880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.304+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f51900cf6a0 con 0x7f51a0100e20 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.304+0000 7f519effd640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 0x7f51880a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194005e00 tx=0x7f5194009490 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.309+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f519009b830 con 0x7f51a0100e20 2024-09-05T06:48:34.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 141283 1.99 MB/sec execute 234 sec latency 13.477 ms 2024-09-05T06:48:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.538+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a010a0a0 con 0x7f51a0100e20 2024-09-05T06:48:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.539+0000 7f519cff9640 1 -- 172.21.15.86:0/2095525004 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f519009fb30 con 0x7f51a0100e20 2024-09-05T06:48:34.540 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:48:34.541 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:48:34.541 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 msgr2=0x7f51880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 0x7f51880a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194005e00 tx=0x7f5194009490 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 msgr2=0x7f51a0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f519000d770 tx=0x7f519000dc30 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 shutdown_connections 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51880a2cf0 0x7f51880a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a0102ef0 0x7f51a019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a0102570 0x7f51a01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 --2- 172.21.15.86:0/2095525004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a0100e20 0x7f51a0198490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 >> 172.21.15.86:0/2095525004 conn(0x7f51a00fc7d0 msgr2=0x7f51a006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 shutdown_connections 2024-09-05T06:48:34.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:34.541+0000 7f51a5bb0640 1 -- 172.21.15.86:0/2095525004 wait complete. 2024-09-05T06:48:34.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:34 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2095525004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:34.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:34 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2095525004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:34 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2095525004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:35.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:48:35.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 141859 2.00 MB/sec execute 235 sec latency 11.945 ms 2024-09-05T06:48:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:35.850 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:35 smithi086 ceph-mon[30943]: pgmap v225: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:35 smithi115 ceph-mon[34791]: pgmap v225: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:36.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:35 smithi096 ceph-mon[34468]: pgmap v225: 129 pgs: 129 active+clean; 145 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.081+0000 7f0901585640 1 -- 172.21.15.86:0/2777776785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc1036e0 msgr2=0x7f08fc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.081+0000 7f0901585640 1 --2- 172.21.15.86:0/2777776785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc1036e0 0x7f08fc107b90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009930 tx=0x7f08f002a420 comp rx=0 tx=0).stop 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 -- 172.21.15.86:0/2777776785 shutdown_connections 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 --2- 172.21.15.86:0/2777776785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1082a0 0x7f08fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 --2- 172.21.15.86:0/2777776785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc1036e0 0x7f08fc107b90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 --2- 172.21.15.86:0/2777776785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc102d30 0x7f08fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 -- 172.21.15.86:0/2777776785 >> 172.21.15.86:0/2777776785 conn(0x7f08fc0fc7d0 msgr2=0x7f08fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 -- 172.21.15.86:0/2777776785 shutdown_connections 2024-09-05T06:48:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 -- 172.21.15.86:0/2777776785 wait complete. 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 Processor -- start 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.082+0000 7f0901585640 1 -- start start 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1036e0 0x7f08fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 0x7f08fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc111f50 con 0x7f08fc102d30 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc111dd0 con 0x7f08fc1082a0 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1120d0 con 0x7f08fc1036e0 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55854/0 (socket says 172.21.15.86:55854) 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 -- 172.21.15.86:0/3418054552 learned_addr learned my addr 172.21.15.86:0/3418054552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 0x7f08fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1036e0 0x7f08fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 -- 172.21.15.86:0/3418054552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1036e0 msgr2=0x7f08fc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1036e0 0x7f08fc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 -- 172.21.15.86:0/3418054552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 msgr2=0x7f08fc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 0x7f08fc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 -- 172.21.15.86:0/3418054552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc1a69d0 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 0x7f08fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08faffd640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08e3fff640 1 -- 172.21.15.86:0/3418054552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017070 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f08e3fff640 1 -- 172.21.15.86:0/3418054552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ec009c30 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.083+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc1a6cc0 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.084+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1a7180 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.084+0000 7f08e3fff640 1 -- 172.21.15.86:0/3418054552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00f620 con 0x7f08fc102d30 2024-09-05T06:48:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f08e3fff640 1 -- 172.21.15.86:0/3418054552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f08ec0109a0 con 0x7f08fc102d30 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f090395e740 1 Processor -- start 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f090395e740 1 -- start start 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f08e3fff640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 0x7f08e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f090395e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 0x55577d0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.086+0000 7f090395e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f090395e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0d2370 0x55577d0d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f090395e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55577d02d2f0 con 0x55577d0d2370 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f090395e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55577cfd4cf0 con 0x55577d0c75d0 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f090395e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55577d0292b0 con 0x55577d0c8b40 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08e3fff640 1 -- 172.21.15.86:0/3418054552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08ec0cf420 con 0x7f08fc102d30 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 0x7f08e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50328/0 (socket says 172.21.15.86:50328) 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fa7fc640 1 -- 172.21.15.86:0/3703900453 learned_addr learned my addr 172.21.15.86:0/3703900453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0d2370 0x55577d0d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 0x55577d0c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 -- 172.21.15.86:0/3703900453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 msgr2=0x55577d0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 -- 172.21.15.86:0/3703900453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0d2370 msgr2=0x55577d0d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0d2370 0x55577d0d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 -- 172.21.15.86:0/3703900453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577cfb93a0 con 0x55577d0c8b40 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 0x7f08e40a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009fd0 tx=0x7f08f4012570 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08faffd640 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 0x55577d0c7090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0cc730 tx=0x7f08ec09af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89eb57f84c0daeb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08e17fa640 1 -- 172.21.15.86:0/3703900453 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec020040 con 0x55577d0c8b40 2024-09-05T06:48:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08e17fa640 1 -- 172.21.15.86:0/3703900453 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ec022c00 con 0x55577d0c8b40 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.087+0000 7f08e17fa640 1 -- 172.21.15.86:0/3703900453 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0cf170 con 0x55577d0c8b40 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 -- 172.21.15.86:0/3703900453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 msgr2=0x55577d0c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 0x55577d0c7090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0cc730 tx=0x7f08ec09af10 comp rx=0 tx=0).stop 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 -- 172.21.15.86:0/3703900453 shutdown_connections 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0d2370 0x55577d0d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 --2- 172.21.15.86:0/3703900453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c8b40 0x55577d0c7090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 -- 172.21.15.86:0/3703900453 >> 172.21.15.86:0/3703900453 conn(0x55577d090310 msgr2=0x55577d0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 -- 172.21.15.86:0/3703900453 shutdown_connections 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.088+0000 7f090395e740 1 -- 172.21.15.86:0/3703900453 wait complete. 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 -- start start 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 Processor -- start 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55577d0d5010 con 0x55577d0c8b40 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55577cfe8060 con 0x55577d0c75d0 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55577d0d5190 con 0x55577d0d2370 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f090395e740 10 client.0 _ll_register_callbacks cb 0x55577cd3b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55870/0 (socket says 172.21.15.86:55870) 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50342/0 (socket says 172.21.15.86:50342) 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fa7fc640 1 -- 172.21.15.86:0/1885828377 learned_addr learned my addr 172.21.15.86:0/1885828377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fb7fe640 1 -- 172.21.15.86:0/1885828377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 msgr2=0x55577d0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fb7fe640 1 -- 172.21.15.86:0/1885828377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 msgr2=0x55577d0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.089+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08fb7fe640 1 -- 172.21.15.86:0/1885828377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577cd9e3c0 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08faffd640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0cf7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55577d0c8070 tx=0x7f08f401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c89fb191b616345 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08deff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4020790 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec022c00 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08f4020930 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08f4020b70 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f090395e740 10 client.26350 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.090+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55577d0e0b40 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ec022c00 con 0x55577d0d2370 2024-09-05T06:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f08f401f070 con 0x55577d0d2370 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 10 client.26350 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 10 client.26350 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 20 client.26350 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d0e1f50 0x55577d0e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 4 client.26350 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55577d03a860 con 0x55577d0e1f50 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.091+0000 7f08faffd640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d0e1f50 0x55577d0e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.092+0000 7f08faffd640 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d0e1f50 0x55577d0e43d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c338bf527858475 server_cookie=9c0f1e3d58daa1e6 in_seq=0 out_seq=0 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.092+0000 7f08deff5640 10 client.26350 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.092+0000 7f08deff5640 1 -- 172.21.15.86:0/1885828377 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55577d03a860 con 0x55577d0e1f50 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.092+0000 7f08deff5640 10 client.26350 handle_command_reply: tid=0 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.092+0000 7f090395e740 1 client.26350 shutdown 2024-09-05T06:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 msgr2=0x55577d0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0cf7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55577d0c8070 tx=0x7f08f401c000 comp rx=0 tx=0).stop 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 shutdown_connections 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d0e1f50 0x55577d0e43d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0cf7e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 --2- 172.21.15.86:0/1885828377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 >> 172.21.15.86:0/1885828377 conn(0x55577d090310 msgr2=0x55577d0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f08deff5640 0 client.26350 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 shutdown_connections 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 1 -- 172.21.15.86:0/1885828377 wait complete. 2024-09-05T06:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.093+0000 7f090395e740 20 client.26350 trim_cache size 0 max 16384 2024-09-05T06:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.107+0000 7f090395e740 1 Processor -- start 2024-09-05T06:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.107+0000 7f090395e740 1 -- start start 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.107+0000 7f090395e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.107+0000 7f090395e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.107+0000 7f090395e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f090395e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b1350 con 0x55577d0c8b40 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f090395e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b2e90 con 0x55577d0c75d0 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f090395e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b1a20 con 0x55577d0d2370 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55882/0 (socket says 172.21.15.86:55882) 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 -- 172.21.15.86:0/4072511494 learned_addr learned my addr 172.21.15.86:0/4072511494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 -- 172.21.15.86:0/4072511494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 msgr2=0x55577d0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 -- 172.21.15.86:0/4072511494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 msgr2=0x55577d0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 -- 172.21.15.86:0/4072511494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577d0de790 con 0x55577d0c8b40 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08faffd640 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f08f002a940 tx=0x7f08f0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76d140d16cde038 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08e0ff9640 1 -- 172.21.15.86:0/4072511494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0035040 con 0x55577d0c8b40 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08e0ff9640 1 -- 172.21.15.86:0/4072511494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f0031cf0 con 0x55577d0c8b40 2024-09-05T06:48:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.108+0000 7f08e0ff9640 1 -- 172.21.15.86:0/4072511494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00052a0 con 0x55577d0c8b40 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 -- 172.21.15.86:0/4072511494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 msgr2=0x55577d0ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f08f002a940 tx=0x7f08f0005c00 comp rx=0 tx=0).stop 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 -- 172.21.15.86:0/4072511494 shutdown_connections 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0d2370 0x55577d0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0ce910 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 --2- 172.21.15.86:0/4072511494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0c75d0 0x55577d0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 -- 172.21.15.86:0/4072511494 >> 172.21.15.86:0/4072511494 conn(0x55577d090310 msgr2=0x55577d0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 -- 172.21.15.86:0/4072511494 shutdown_connections 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.109+0000 7f090395e740 1 -- 172.21.15.86:0/4072511494 wait complete. 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 -- start start 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 Processor -- start 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c75d0 0x55577d0da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 0x55577d0d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b27c0 con 0x55577d0c8b40 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b20f0 con 0x55577d0d2370 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1af350 con 0x55577d0c75d0 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f090395e740 10 client.0 _ll_register_callbacks cb 0x7f08f40127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f08fa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55884/0 (socket says 172.21.15.86:55884) 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f08fa7fc640 1 -- 172.21.15.86:0/3316627091 learned_addr learned my addr 172.21.15.86:0/3316627091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f08faffd640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c75d0 0x55577d0da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.110+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 0x55577d0d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 -- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c75d0 msgr2=0x55577d0da820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c75d0 0x55577d0da820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 -- 172.21.15.86:0/3316627091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 msgr2=0x55577d0d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 0x55577d0d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 -- 172.21.15.86:0/3316627091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577d0d65d0 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fb7fe640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 0x55577d0d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08fa7fc640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009850 tx=0x7f08f0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe667c0ffb52688 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08deff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f004f070 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08f0005250 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f0031cf0 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08f005b050 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f090395e740 10 client.17112 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.111+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55577d15d930 con 0x55577d0c8b40 2024-09-05T06:48:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f0031cf0 con 0x55577d0c8b40 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f08f003d450 con 0x55577d0c8b40 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 10 client.17112 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 10 client.17112 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 20 client.17112 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d15e080 0x55577d1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 4 client.17112 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55577d034d70 con 0x55577d15e080 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f08faffd640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d15e080 0x55577d1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f08faffd640 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d15e080 0x55577d1604a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4b86793fa5162ef server_cookie=27869823ccc2d09a in_seq=0 out_seq=0 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.112+0000 7f08deff5640 10 client.17112 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f08deff5640 1 -- 172.21.15.86:0/3316627091 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55577d034d70 con 0x55577d15e080 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f08deff5640 10 client.17112 handle_command_reply: tid=0 2024-09-05T06:48:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 client.17112 shutdown 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 msgr2=0x55577d0d5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009850 tx=0x7f08f0005900 comp rx=0 tx=0).stop 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 shutdown_connections 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55577d15e080 0x55577d1604a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55577d0d2370 0x55577d0d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55577d0c8b40 0x55577d0d5b50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 --2- 172.21.15.86:0/3316627091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55577d0c75d0 0x55577d0da820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 >> 172.21.15.86:0/3316627091 conn(0x55577d090310 msgr2=0x55577d0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f08deff5640 0 client.17112 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 shutdown_connections 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 1 -- 172.21.15.86:0/3316627091 wait complete. 2024-09-05T06:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.113+0000 7f090395e740 20 client.17112 trim_cache size 0 max 16384 2024-09-05T06:48:36.120 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 msgr2=0x7f08e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 0x7f08e40a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009fd0 tx=0x7f08f4012570 comp rx=0 tx=0).stop 2024-09-05T06:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 msgr2=0x7f08fc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00de10 comp rx=0 tx=0).stop 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 shutdown_connections 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08e40a2cf0 0x7f08e40a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08fc1082a0 0x7f08fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08fc1036e0 0x7f08fc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 --2- 172.21.15.86:0/3418054552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc102d30 0x7f08fc100990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 >> 172.21.15.86:0/3418054552 conn(0x7f08fc0fc7d0 msgr2=0x7f08fc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 shutdown_connections 2024-09-05T06:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:36.114+0000 7f0901585640 1 -- 172.21.15.86:0/3418054552 wait complete. 2024-09-05T06:48:36.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 142239 2.00 MB/sec execute 236 sec latency 21.135 ms 2024-09-05T06:48:36.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:48:37.066 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:37.297 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T06:48:37.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:48:37.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 142516 1.99 MB/sec execute 237 sec latency 39.049 ms 2024-09-05T06:48:37.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- 172.21.15.86:0/641413201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 msgr2=0x7f33a010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- 172.21.15.86:0/641413201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a010ec30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3398009cd0 tx=0x7f339802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- 172.21.15.86:0/641413201 shutdown_connections 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- 172.21.15.86:0/641413201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a010ec30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- 172.21.15.86:0/641413201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- 172.21.15.86:0/641413201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- 172.21.15.86:0/641413201 >> 172.21.15.86:0/641413201 conn(0x7f33a00fc7d0 msgr2=0x7f33a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- 172.21.15.86:0/641413201 shutdown_connections 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- 172.21.15.86:0/641413201 wait complete. 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 Processor -- start 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- start start 2024-09-05T06:48:37.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0111f40 con 0x7f33a0108290 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0111dc0 con 0x7f33a0103740 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.572+0000 7f33a71d1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01120c0 con 0x7f33a0102d90 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a576e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a576e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55898/0 (socket says 172.21.15.86:55898) 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50366/0 (socket says 172.21.15.86:50366) 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a01009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52562/0 (socket says 172.21.15.86:52562) 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a576e640 1 -- 172.21.15.86:0/3162449339 learned_addr learned my addr 172.21.15.86:0/3162449339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:37.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 -- 172.21.15.86:0/3162449339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 msgr2=0x7f33a01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 -- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 msgr2=0x7f33a0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 -- 172.21.15.86:0/3162449339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a01a69a0 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a576e640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339ffff640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f338c00e9c0 tx=0x7f338c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339dffb640 1 -- 172.21.15.86:0/3162449339 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c00cd80 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a01a6c90 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339dffb640 1 -- 172.21.15.86:0/3162449339 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f338c0035f0 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.573+0000 7f339dffb640 1 -- 172.21.15.86:0/3162449339 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c010690 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.574+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01a7170 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.576+0000 7f339dffb640 1 -- 172.21.15.86:0/3162449339 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f338c0037a0 con 0x7f33a0103740 2024-09-05T06:48:37.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.576+0000 7f33a95aa740 1 Processor -- start 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.576+0000 7f33a95aa740 1 -- start start 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339dffb640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 0x7f33900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0dfb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e08760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ff0d612f0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ff0d08cf0 con 0x557ff0e06370 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a95aa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557ff0d5d2b0 con 0x557ff0dfcb40 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a576e640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 0x7f33900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339dffb640 1 -- 172.21.15.86:0/3162449339 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f338c0cfab0 con 0x7f33a0103740 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55900/0 (socket says 172.21.15.86:55900) 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 -- 172.21.15.86:0/3615544998 learned_addr learned my addr 172.21.15.86:0/3615544998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a576e640 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e08760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0dfb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 -- 172.21.15.86:0/3615544998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 msgr2=0x557ff0dfb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0dfb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 -- 172.21.15.86:0/3615544998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 msgr2=0x557ff0e08760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e08760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 -- 172.21.15.86:0/3615544998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff0ced3a0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f339ffff640 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f338c000c00 tx=0x7f338c09ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daa9fd7e32029b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f33a576e640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 0x7f33900a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3398009430 tx=0x7f3398009880 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.577+0000 7f338affd640 1 -- 172.21.15.86:0/3615544998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c09c290 con 0x557ff0dfb5d0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f338affd640 1 -- 172.21.15.86:0/3615544998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f338c09bb90 con 0x557ff0dfb5d0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f338affd640 1 -- 172.21.15.86:0/3615544998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0cd040 con 0x557ff0dfb5d0 2024-09-05T06:48:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 -- 172.21.15.86:0/3615544998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 msgr2=0x557ff0dfba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f338c000c00 tx=0x7f338c09ba70 comp rx=0 tx=0).stop 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 -- 172.21.15.86:0/3615544998 shutdown_connections 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e08760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0dfba10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3615544998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0dfb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 -- 172.21.15.86:0/3615544998 >> 172.21.15.86:0/3615544998 conn(0x557ff0dc4310 msgr2=0x557ff0df7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 -- 172.21.15.86:0/3615544998 shutdown_connections 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.578+0000 7f33a95aa740 1 -- 172.21.15.86:0/3615544998 wait complete. 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 -- start start 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 Processor -- start 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0e03070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ff0e09070 con 0x557ff0e06370 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ff0d1c060 con 0x557ff0dfb5d0 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557ff0e091f0 con 0x557ff0dfcb40 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a95aa740 10 client.0 _ll_register_callbacks cb 0x557ff0a6f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a576e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50398/0 (socket says 172.21.15.86:50398) 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f339ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0e03070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.579+0000 7f33a576e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55908/0 (socket says 172.21.15.86:55908) 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a4f6d640 1 -- 172.21.15.86:0/3844749352 learned_addr learned my addr 172.21.15.86:0/3844749352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 -- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 msgr2=0x557ff0e05dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 -- 172.21.15.86:0/3844749352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 msgr2=0x557ff0e03800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 -- 172.21.15.86:0/3844749352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff0ad23c0 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a576e640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f339ffff640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0e03070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f338c0a0100 tx=0x7f338c0ccde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1892effc4ba8df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f337ffff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0a4800 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f338c09bb90 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f338c0cd040 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f338c09c290 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a95aa740 10 client.26368 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.580+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ff0e09070 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f338c09bb90 con 0x557ff0dfcb40 2024-09-05T06:48:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f338c086050 con 0x557ff0dfcb40 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 10 client.26368 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 10 client.26368 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 20 client.26368 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e15d00 0x557ff0e180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 4 client.26368 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ff0d6e860 con 0x557ff0e15d00 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.581+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e15d00 0x557ff0e180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f337ffff640 10 client.26368 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e15d00 0x557ff0e180f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24253384fc1e9bd2 server_cookie=58bc808f9c47f999 in_seq=0 out_seq=0 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f337ffff640 1 -- 172.21.15.86:0/3844749352 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ff0d6e860 con 0x557ff0e15d00 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f337ffff640 10 client.26368 handle_command_reply: tid=0 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f33a95aa740 1 client.26368 shutdown 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 msgr2=0x557ff0e03070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.582+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0e03070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f338c0a0100 tx=0x7f338c0ccde0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 shutdown_connections 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e15d00 0x557ff0e180f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0e06370 0x557ff0e03800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f337ffff640 0 client.26368 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0e03070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 --2- 172.21.15.86:0/3844749352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0dfb5d0 0x557ff0e05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 >> 172.21.15.86:0/3844749352 conn(0x557ff0dc4310 msgr2=0x557ff0dfd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 shutdown_connections 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 1 -- 172.21.15.86:0/3844749352 wait complete. 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.583+0000 7f33a95aa740 20 client.26368 trim_cache size 0 max 16384 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.596+0000 7f33a95aa740 1 Processor -- start 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.596+0000 7f33a95aa740 1 -- start start 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0df6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b13d0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b2f10 con 0x557ff0e06370 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a95aa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b1aa0 con 0x557ff0dfcb40 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55916/0 (socket says 172.21.15.86:55916) 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 -- 172.21.15.86:0/2596234836 learned_addr learned my addr 172.21.15.86:0/2596234836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a576e640 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f339ffff640 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0df6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 -- 172.21.15.86:0/2596234836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 msgr2=0x557ff0df6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0df6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 -- 172.21.15.86:0/2596234836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 msgr2=0x557ff0e1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 -- 172.21.15.86:0/2596234836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff0e058e0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x557ff0e18690 tx=0x7f339400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2080f1a1617d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.597+0000 7f338a7fc640 1 -- 172.21.15.86:0/2596234836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394003c20 con 0x557ff0dfb5d0 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f338a7fc640 1 -- 172.21.15.86:0/2596234836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3394003dc0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f338a7fc640 1 -- 172.21.15.86:0/2596234836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33940116a0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 -- 172.21.15.86:0/2596234836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 msgr2=0x557ff0df7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x557ff0e18690 tx=0x7f339400b390 comp rx=0 tx=0).stop 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 -- 172.21.15.86:0/2596234836 shutdown_connections 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0e1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfcb40 0x557ff0df6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 --2- 172.21.15.86:0/2596234836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfb5d0 0x557ff0df7d70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 -- 172.21.15.86:0/2596234836 >> 172.21.15.86:0/2596234836 conn(0x557ff0dc4310 msgr2=0x557ff0df64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 -- 172.21.15.86:0/2596234836 shutdown_connections 2024-09-05T06:48:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.598+0000 7f33a95aa740 1 -- 172.21.15.86:0/2596234836 wait complete. 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 -- start start 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 Processor -- start 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfb5d0 0x557ff0e0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0dff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b2840 con 0x557ff0dfcb40 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b2170 con 0x557ff0e06370 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01af3d0 con 0x557ff0dfb5d0 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a95aa740 10 client.0 _ll_register_callbacks cb 0x7f3398009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f339ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f339ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55932/0 (socket says 172.21.15.86:55932) 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f339ffff640 1 -- 172.21.15.86:0/393895214 learned_addr learned my addr 172.21.15.86:0/393895214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfb5d0 0x557ff0e0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.599+0000 7f33a576e640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0dff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 -- 172.21.15.86:0/393895214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfb5d0 msgr2=0x557ff0e0e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfb5d0 0x557ff0e0e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 -- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 msgr2=0x557ff0dff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0dff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 -- 172.21.15.86:0/393895214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff0e00170 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f339ffff640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f338c09bb20 tx=0x7f338c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5587e3a94678de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f337ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c09b6d0 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f338c0ba2a0 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3394003dc0 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f338c0a4800 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f33a95aa740 10 client.17130 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ff0e00170 con 0x557ff0dfcb40 2024-09-05T06:48:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.600+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3394003dc0 con 0x557ff0dfcb40 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f338c086040 con 0x557ff0dfcb40 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 10 client.17130 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 10 client.17130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 20 client.17130 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e80590 0x557ff0e82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 4 client.17130 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ff0d68d70 con 0x557ff0e80590 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e80590 0x557ff0e82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a4f6d640 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e80590 0x557ff0e82980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2bd680516fab7c53 server_cookie=9939f311f224e1eb in_seq=0 out_seq=0 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f337ffff640 10 client.17130 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f337ffff640 1 -- 172.21.15.86:0/393895214 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ff0d68d70 con 0x557ff0e80590 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f337ffff640 10 client.17130 handle_command_reply: tid=0 2024-09-05T06:48:37.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.601+0000 7f33a95aa740 1 client.17130 shutdown 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 msgr2=0x557ff0e0ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f338c09bb20 tx=0x7f338c009310 comp rx=0 tx=0).stop 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 shutdown_connections 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557ff0e80590 0x557ff0e82980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ff0e06370 0x557ff0dff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ff0dfcb40 0x557ff0e0ec50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 --2- 172.21.15.86:0/393895214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ff0dfb5d0 0x557ff0e0e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 >> 172.21.15.86:0/393895214 conn(0x557ff0dc4310 msgr2=0x557ff0e050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f337ffff640 0 client.17130 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 shutdown_connections 2024-09-05T06:48:37.625 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 1 -- 172.21.15.86:0/393895214 wait complete. 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a95aa740 20 client.17130 trim_cache size 0 max 16384 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 msgr2=0x7f33900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 0x7f33900a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3398009430 tx=0x7f3398009880 comp rx=0 tx=0).stop 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 msgr2=0x7f33a0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.602+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f338c00e9c0 tx=0x7f338c00ee80 comp rx=0 tx=0).stop 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 shutdown_connections 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33900a2cf0 0x7f33900a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33a0108290 0x7f33a0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33a0103740 0x7f33a0100f30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 --2- 172.21.15.86:0/3162449339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a0102d90 0x7f33a01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 >> 172.21.15.86:0/3162449339 conn(0x7f33a00fc7d0 msgr2=0x7f33a0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 shutdown_connections 2024-09-05T06:48:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:37.603+0000 7f33a71d1640 1 -- 172.21.15.86:0/3162449339 wait complete. 2024-09-05T06:48:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:37 smithi086 ceph-mon[30943]: pgmap v226: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-09-05T06:48:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:37 smithi096 ceph-mon[34468]: pgmap v226: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-09-05T06:48:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:37 smithi115 ceph-mon[34791]: pgmap v226: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-09-05T06:48:38.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:48:38.430 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:38.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 143066 1.99 MB/sec execute 238 sec latency 13.429 ms 2024-09-05T06:48:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.928+0000 7f9ed6b41640 1 -- 172.21.15.86:0/3707321264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 msgr2=0x7f9ed010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.928+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/3707321264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed010ec40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009cd0 tx=0x7f9ec802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- 172.21.15.86:0/3707321264 shutdown_connections 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/3707321264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed010ec40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/3707321264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/3707321264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- 172.21.15.86:0/3707321264 >> 172.21.15.86:0/3707321264 conn(0x7f9ed00fc7d0 msgr2=0x7f9ed00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- 172.21.15.86:0/3707321264 shutdown_connections 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- 172.21.15.86:0/3707321264 wait complete. 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 Processor -- start 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- start start 2024-09-05T06:48:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0111f50 con 0x7f9ed01082a0 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0111dd0 con 0x7f9ed0102d30 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.929+0000 7f9ed6b41640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed01120d0 con 0x7f9ed01036e0 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed48dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed48dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50426/0 (socket says 172.21.15.86:50426) 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed48dd640 1 -- 172.21.15.86:0/4028637627 learned_addr learned my addr 172.21.15.86:0/4028637627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 -- 172.21.15.86:0/4028637627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 msgr2=0x7f9ed0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 -- 172.21.15.86:0/4028637627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 msgr2=0x7f9ed0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 -- 172.21.15.86:0/4028637627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed01a6960 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ecffff640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed48dd640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed50de640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed0101410 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9ec802ae00 tx=0x7f9ec8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8004040 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec8004690 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed01a6bf0 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.930+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8030e20 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.931+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed01a7130 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.932+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed0103110 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.933+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ec80305a0 con 0x7f9ed01082a0 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.933+0000 7f9ecdffb640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 0x7f9ec00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.934+0000 7f9ed48dd640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 0x7f9ec00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:38.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.934+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f9ec802e030 con 0x7f9ed01082a0 2024-09-05T06:48:38.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.934+0000 7f9ed48dd640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 0x7f9ec00a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4011550 tx=0x7f9ec4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:38.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:38.939+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ec804b060 con 0x7f9ed01082a0 2024-09-05T06:48:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:39.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.203+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed010a2b0 con 0x7f9ed01082a0 2024-09-05T06:48:39.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.204+0000 7f9ecdffb640 1 -- 172.21.15.86:0/4028637627 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f9ec80bd600 con 0x7f9ed01082a0 2024-09-05T06:48:39.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:48:39.205 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:48:39.206 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:48:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 msgr2=0x7f9ec00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 0x7f9ec00a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4011550 tx=0x7f9ec4005f90 comp rx=0 tx=0).stop 2024-09-05T06:48:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 msgr2=0x7f9ed0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed0101410 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9ec802ae00 tx=0x7f9ec8003970 comp rx=0 tx=0).stop 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 shutdown_connections 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9ec00a2d70 0x7f9ec00a5230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed01082a0 0x7f9ed0101410 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed01036e0 0x7f9ed0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 --2- 172.21.15.86:0/4028637627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ed0102d30 0x7f9ed0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.208+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 >> 172.21.15.86:0/4028637627 conn(0x7f9ed00fc7d0 msgr2=0x7f9ed0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.209+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 shutdown_connections 2024-09-05T06:48:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.209+0000 7f9ed6b41640 1 -- 172.21.15.86:0/4028637627 wait complete. 2024-09-05T06:48:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:39 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4028637627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:39.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 143563 2.00 MB/sec execute 239 sec latency 28.386 ms 2024-09-05T06:48:39.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:39 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4028637627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:39.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:39 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4028637627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:39.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 -- 172.21.15.86:0/1165335251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 msgr2=0x7feb1c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 --2- 172.21.15.86:0/1165335251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c1021c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002a410 comp rx=0 tx=0).stop 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 -- 172.21.15.86:0/1165335251 shutdown_connections 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 --2- 172.21.15.86:0/1165335251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 0x7feb1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 --2- 172.21.15.86:0/1165335251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 0x7feb1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 --2- 172.21.15.86:0/1165335251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c1021c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 -- 172.21.15.86:0/1165335251 >> 172.21.15.86:0/1165335251 conn(0x7feb1c0fd8c0 msgr2=0x7feb1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 -- 172.21.15.86:0/1165335251 shutdown_connections 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.678+0000 7feb2309a640 1 -- 172.21.15.86:0/1165335251 wait complete. 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 Processor -- start 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 -- start start 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 0x7feb1c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 0x7feb1c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c111e20 con 0x7feb1c101de0 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c111ca0 con 0x7feb1c106f10 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb2309a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c111fa0 con 0x7feb1c102790 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55974/0 (socket says 172.21.15.86:55974) 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb20e36640 1 -- 172.21.15.86:0/2616945930 learned_addr learned my addr 172.21.15.86:0/2616945930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb21637640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 0x7feb1c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.679+0000 7feb1bfff640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 0x7feb1c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 -- 172.21.15.86:0/2616945930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 msgr2=0x7feb1c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 0x7feb1c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 -- 172.21.15.86:0/2616945930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 msgr2=0x7feb1c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 0x7feb1c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 -- 172.21.15.86:0/2616945930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19d300 con 0x7feb1c101de0 2024-09-05T06:48:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb20e36640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7feb10005940 tx=0x7feb100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb19ffb640 1 -- 172.21.15.86:0/2616945930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10035050 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb19ffb640 1 -- 172.21.15.86:0/2616945930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb100035f0 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c19d590 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb19ffb640 1 -- 172.21.15.86:0/2616945930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100044e0 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.680+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c074420 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.683+0000 7feb25473740 1 Processor -- start 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.683+0000 7feb19ffb640 1 -- 172.21.15.86:0/2616945930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feb100037a0 con 0x7feb1c101de0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.683+0000 7feb25473740 1 -- start start 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123db40 0x55870123c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x558701249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587011a22f0 con 0x55870123c5d0 2024-09-05T06:48:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558701149cf0 con 0x55870123db40 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb25473740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55870119e2b0 con 0x558701247370 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55986/0 (socket says 172.21.15.86:55986) 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 -- 172.21.15.86:0/3350004124 learned_addr learned my addr 172.21.15.86:0/3350004124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb1bfff640 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x558701249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb21637640 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123db40 0x55870123c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 -- 172.21.15.86:0/3350004124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 msgr2=0x558701249760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x558701249760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 -- 172.21.15.86:0/3350004124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123db40 msgr2=0x55870123c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123db40 0x55870123c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 -- 172.21.15.86:0/3350004124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870112e3a0 con 0x55870123c5d0 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb19ffb640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 0x7feb0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb21637640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 0x7feb0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb19ffb640 1 -- 172.21.15.86:0/2616945930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7feb100f4090 con 0x7feb1c101de0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.684+0000 7feb20e36640 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7feb100f1310 tx=0x7feb100fc220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16397a91d680496 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb06ffd640 1 -- 172.21.15.86:0/3350004124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100bc410 con 0x55870123c5d0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb06ffd640 1 -- 172.21.15.86:0/3350004124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb100c0870 con 0x55870123c5d0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb06ffd640 1 -- 172.21.15.86:0/3350004124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10005d50 con 0x55870123c5d0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb25473740 1 -- 172.21.15.86:0/3350004124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 msgr2=0x55870123ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb21637640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 0x7feb0c0a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb14005fd0 tx=0x7feb14009600 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.685+0000 7feb25473740 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7feb100f1310 tx=0x7feb100fc220 comp rx=0 tx=0).stop 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- 172.21.15.86:0/3350004124 shutdown_connections 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x558701249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x55870123ca10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- 172.21.15.86:0/3350004124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123db40 0x55870123c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- 172.21.15.86:0/3350004124 >> 172.21.15.86:0/3350004124 conn(0x558701205310 msgr2=0x558701238c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- 172.21.15.86:0/3350004124 shutdown_connections 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- 172.21.15.86:0/3350004124 wait complete. 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- start start 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 Processor -- start 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x558701244030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558701247370 0x558701244640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587011a22f0 con 0x55870123c5d0 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.686+0000 7feb25473740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55870115d060 con 0x558701247370 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb25473740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558701249e30 con 0x55870123db40 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x558701244030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb25473740 10 client.0 _ll_register_callbacks cb 0x558700eb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55992/0 (socket says 172.21.15.86:55992) 2024-09-05T06:48:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 -- 172.21.15.86:0/339413019 learned_addr learned my addr 172.21.15.86:0/339413019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb21637640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558701247370 0x558701244640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 -- 172.21.15.86:0/339413019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 msgr2=0x558701244030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x558701244030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 -- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558701247370 msgr2=0x558701244640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558701247370 0x558701244640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 -- 172.21.15.86:0/339413019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558700f133c0 con 0x55870123c5d0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb20e36640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feb100f1450 tx=0x7feb100f18d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b71319e3ec4d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feafbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100bfad0 con 0x55870123c5d0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7feb100bc7e0 con 0x55870123c5d0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb100c0870 con 0x55870123c5d0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb100bfcb0 con 0x55870123c5d0 2024-09-05T06:48:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb25473740 10 client.17148 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.687+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587012556c0 con 0x55870123c5d0 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb100d9080 con 0x55870123c5d0 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7feb100bace0 con 0x55870123c5d0 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 10 client.17148 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 10 client.17148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 20 client.17148 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x558701256a10 0x558701258e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 4 client.17148 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587011af860 con 0x558701256a10 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb1bfff640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x558701256a10 0x558701258e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feafbfff640 10 client.17148 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.688+0000 7feb1bfff640 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x558701256a10 0x558701258e00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ccd0bf9c066c376 server_cookie=89b089eba39db95a in_seq=0 out_seq=0 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feafbfff640 1 -- 172.21.15.86:0/339413019 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587011af860 con 0x558701256a10 2024-09-05T06:48:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feafbfff640 10 client.17148 handle_command_reply: tid=0 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 client.17148 shutdown 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 msgr2=0x558701246d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feb100f1450 tx=0x7feb100f18d0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 shutdown_connections 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x558701256a10 0x558701258e00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558701247370 0x558701244640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x558701244030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feafbfff640 0 client.17148 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 --2- 172.21.15.86:0/339413019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123c5d0 0x558701246d80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 >> 172.21.15.86:0/339413019 conn(0x558701205310 msgr2=0x55870123e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 shutdown_connections 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 1 -- 172.21.15.86:0/339413019 wait complete. 2024-09-05T06:48:39.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.689+0000 7feb25473740 20 client.17148 trim_cache size 0 max 16384 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.704+0000 7feb25473740 1 Processor -- start 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.704+0000 7feb25473740 1 -- start start 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x558701252f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x55870125e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1acf60 con 0x558701247370 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1aeaa0 con 0x55870123c5d0 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb25473740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ad630 con 0x55870123db40 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb21637640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb21637640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55998/0 (socket says 172.21.15.86:55998) 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb21637640 1 -- 172.21.15.86:0/3818732809 learned_addr learned my addr 172.21.15.86:0/3818732809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x558701252f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb1bfff640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x55870125e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 -- 172.21.15.86:0/3818732809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 msgr2=0x55870125e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x55870125e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 -- 172.21.15.86:0/3818732809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 msgr2=0x55870125eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 -- 172.21.15.86:0/3818732809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558701244fd0 con 0x55870123c5d0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb21637640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb20e36640 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x558701252f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb10006010 tx=0x7feb100c0170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3b1c9214b5b73b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.705+0000 7feb067fc640 1 -- 172.21.15.86:0/3818732809 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100bff40 con 0x55870123c5d0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb067fc640 1 -- 172.21.15.86:0/3818732809 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb100c0870 con 0x55870123c5d0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb067fc640 1 -- 172.21.15.86:0/3818732809 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100fc350 con 0x55870123c5d0 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 -- 172.21.15.86:0/3818732809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 msgr2=0x558701252f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x558701252f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb10006010 tx=0x7feb100c0170 comp rx=0 tx=0).stop 2024-09-05T06:48:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 -- 172.21.15.86:0/3818732809 shutdown_connections 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558701247370 0x55870125eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55870123db40 0x55870125e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 --2- 172.21.15.86:0/3818732809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x558701252f10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 -- 172.21.15.86:0/3818732809 >> 172.21.15.86:0/3818732809 conn(0x558701205310 msgr2=0x558701243120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 -- 172.21.15.86:0/3818732809 shutdown_connections 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.706+0000 7feb25473740 1 -- 172.21.15.86:0/3818732809 wait complete. 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 -- start start 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 Processor -- start 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x55870124f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x55870124b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ae3d0 con 0x55870123db40 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1add00 con 0x55870123c5d0 2024-09-05T06:48:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1aaf60 con 0x558701247370 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb25473740 10 client.0 _ll_register_callbacks cb 0x7feb14009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56010/0 (socket says 172.21.15.86:56010) 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 -- 172.21.15.86:0/1071096111 learned_addr learned my addr 172.21.15.86:0/1071096111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb20e36640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x55870124f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb21637640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x55870124b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 -- 172.21.15.86:0/1071096111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 msgr2=0x55870124b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x55870124b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 -- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 msgr2=0x55870124f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.707+0000 7feb1bfff640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x55870124f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feb1bfff640 1 -- 172.21.15.86:0/1071096111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870124b9e0 con 0x55870123db40 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feb1bfff640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7feb08004700 tx=0x7feb080041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dee6ce2a8a50991 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feafbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08003dd0 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7feb08005020 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.708+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb100c0870 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb080044b0 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 10 client.17154 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587012c17a0 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb080047a0 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7feb08020410 con 0x55870123db40 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 10 client.17154 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 20 client.17154 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5587012c1f50 0x5587012c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 4 client.17154 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:48:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7feb100c0870 con 0x5587012c1f50 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.709+0000 7feb20e36640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5587012c1f50 0x5587012c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb20e36640 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5587012c1f50 0x5587012c4370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3ae6e5232e3e771 server_cookie=f5347b573c4d3b73 in_seq=0 out_seq=0 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feafbfff640 10 client.17154 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feafbfff640 1 -- 172.21.15.86:0/1071096111 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7feb100c0870 con 0x5587012c1f50 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feafbfff640 10 client.17154 handle_command_reply: tid=0 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 client.17154 shutdown 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 msgr2=0x55870124abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7feb08004700 tx=0x7feb080041a0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 shutdown_connections 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5587012c1f50 0x5587012c4370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558701247370 0x55870124b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55870123db40 0x55870124abf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feafbfff640 0 client.17154 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:48:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 --2- 172.21.15.86:0/1071096111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870123c5d0 0x55870124f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 >> 172.21.15.86:0/1071096111 conn(0x558701205310 msgr2=0x5587012374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 shutdown_connections 2024-09-05T06:48:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 1 -- 172.21.15.86:0/1071096111 wait complete. 2024-09-05T06:48:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.710+0000 7feb25473740 20 client.17154 trim_cache size 0 max 16384 2024-09-05T06:48:39.727 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 msgr2=0x7feb0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 0x7feb0c0a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb14005fd0 tx=0x7feb14009600 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 msgr2=0x7feb1c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7feb10005940 tx=0x7feb100042f0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 shutdown_connections 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feb0c0a2cf0 0x7feb0c0a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c106f10 0x7feb1c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb1c102790 0x7feb1c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 --2- 172.21.15.86:0/2616945930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c101de0 0x7feb1c198350 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 >> 172.21.15.86:0/2616945930 conn(0x7feb1c0fd8c0 msgr2=0x7feb1c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 shutdown_connections 2024-09-05T06:48:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:39.711+0000 7feb2309a640 1 -- 172.21.15.86:0/2616945930 wait complete. 2024-09-05T06:48:40.457 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:40 smithi086 ceph-mon[30943]: pgmap v227: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:48:40.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:48:40.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 144236 2.00 MB/sec execute 240 sec latency 19.504 ms 2024-09-05T06:48:40.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:40 smithi096 ceph-mon[34468]: pgmap v227: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:48:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:40 smithi115 ceph-mon[34791]: pgmap v227: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:48:40.981 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T06:48:40.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T06:48:41.095 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:41.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 144838 2.01 MB/sec execute 241 sec latency 32.138 ms 2024-09-05T06:48:41.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 -- 172.21.15.86:0/3704471132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100e20 msgr2=0x7f08dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 --2- 172.21.15.86:0/3704471132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100e20 0x7f08dc101200 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009900 tx=0x7f08cc02a480 comp rx=0 tx=0).stop 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 -- 172.21.15.86:0/3704471132 shutdown_connections 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 --2- 172.21.15.86:0/3704471132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc102ef0 0x7f08dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 --2- 172.21.15.86:0/3704471132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102570 0x7f08dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 --2- 172.21.15.86:0/3704471132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc100e20 0x7f08dc101200 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 -- 172.21.15.86:0/3704471132 >> 172.21.15.86:0/3704471132 conn(0x7f08dc0fc7d0 msgr2=0x7f08dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 -- 172.21.15.86:0/3704471132 shutdown_connections 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.588+0000 7f08e1e86640 1 -- 172.21.15.86:0/3704471132 wait complete. 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 Processor -- start 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 -- start start 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc100e20 0x7f08dc19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102ef0 0x7f08dc19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10fd40 con 0x7f08dc102570 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10fbc0 con 0x7f08dc102ef0 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08e1e86640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10fec0 con 0x7f08dc100e20 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56030/0 (socket says 172.21.15.86:56030) 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 -- 172.21.15.86:0/1982083666 learned_addr learned my addr 172.21.15.86:0/1982083666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08db7fe640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc100e20 0x7f08dc19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08dbfff640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102ef0 0x7f08dc19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 -- 172.21.15.86:0/1982083666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc100e20 msgr2=0x7f08dc19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc100e20 0x7f08dc19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 -- 172.21.15.86:0/1982083666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102ef0 msgr2=0x7f08dc19ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102ef0 0x7f08dc19ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.589+0000 7f08daffd640 1 -- 172.21.15.86:0/1982083666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1a07a0 con 0x7f08dc102570 2024-09-05T06:48:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08daffd640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9f0 tx=0x7f08d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08d8ff9640 1 -- 172.21.15.86:0/1982083666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d000cd80 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08d8ff9640 1 -- 172.21.15.86:0/1982083666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d00108e0 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1a0a90 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08d8ff9640 1 -- 172.21.15.86:0/1982083666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0018920 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.590+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1a7150 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.592+0000 7f08d8ff9640 1 -- 172.21.15.86:0/1982083666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f08d0010a90 con 0x7f08dc102570 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.592+0000 7f08e425f740 1 Processor -- start 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.592+0000 7f08e425f740 1 -- start start 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559838cc02f0 con 0x559838d65370 2024-09-05T06:48:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559838c67cf0 con 0x559838d5a5d0 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08e425f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559838cbc2b0 con 0x559838d5bb40 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52702/0 (socket says 172.21.15.86:52702) 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08dbfff640 1 -- 172.21.15.86:0/44660863 learned_addr learned my addr 172.21.15.86:0/44660863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08d8ff9640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 0x7f08c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 -- 172.21.15.86:0/44660863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 msgr2=0x559838d5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 -- 172.21.15.86:0/44660863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 msgr2=0x559838d67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08db7fe640 1 -- 172.21.15.86:0/44660863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559838c4c3a0 con 0x559838d5a5d0 2024-09-05T06:48:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08dbfff640 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08daffd640 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.593+0000 7f08dbfff640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 0x7f08c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08db7fe640 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d5aa10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005940 tx=0x7f08cc031bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f318e1b400b360a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08d8ff9640 1 -- 172.21.15.86:0/1982083666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08d00cfc10 con 0x7f08dc102570 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08c1ffb640 1 -- 172.21.15.86:0/44660863 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc040070 con 0x559838d5a5d0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08c1ffb640 1 -- 172.21.15.86:0/44660863 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc031630 con 0x559838d5a5d0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08c1ffb640 1 -- 172.21.15.86:0/44660863 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc035040 con 0x559838d5a5d0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08e425f740 1 -- 172.21.15.86:0/44660863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 msgr2=0x559838d5aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d5aa10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005940 tx=0x7f08cc031bd0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.594+0000 7f08dbfff640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 0x7f08c40a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011580 tx=0x7f08d4004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 -- 172.21.15.86:0/44660863 shutdown_connections 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d5aa10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 --2- 172.21.15.86:0/44660863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 -- 172.21.15.86:0/44660863 >> 172.21.15.86:0/44660863 conn(0x559838d23310 msgr2=0x559838d56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 -- 172.21.15.86:0/44660863 shutdown_connections 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 -- 172.21.15.86:0/44660863 wait complete. 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 -- start start 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.595+0000 7f08e425f740 1 Processor -- start 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d64d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559838d68010 con 0x559838d5bb40 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559838c7b060 con 0x559838d5a5d0 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559838d68190 con 0x559838d65370 2024-09-05T06:48:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 10 client.0 _ll_register_callbacks cb 0x5598389ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56048/0 (socket says 172.21.15.86:56048) 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 -- 172.21.15.86:0/4243804234 learned_addr learned my addr 172.21.15.86:0/4243804234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08dbfff640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08db7fe640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d64d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 -- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 msgr2=0x559838d625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 -- 172.21.15.86:0/4243804234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 msgr2=0x559838d64d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d64d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 -- 172.21.15.86:0/4243804234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559838a313c0 con 0x559838d5bb40 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08daffd640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9c0 tx=0x7f08d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b096b6462ae3868 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08baffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0010630 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc031630 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.596+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08d009b140 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08d0010de0 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 10 client.17166 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559838d73a80 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc031630 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f08d0018ac0 con 0x559838d5bb40 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 10 client.17166 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 10 client.17166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 20 client.17166 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838d77390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 4 client.17166 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559838ccd860 con 0x559838d74f40 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08db7fe640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838d77390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08db7fe640 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838d77390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebb317d5c6fae81e server_cookie=8fcefb117c50e69d in_seq=0 out_seq=0 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.598+0000 7f08baffd640 10 client.17166 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08baffd640 1 -- 172.21.15.86:0/4243804234 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559838ccd860 con 0x559838d74f40 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08baffd640 10 client.17166 handle_command_reply: tid=0 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 client.17166 shutdown 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 msgr2=0x559838d61ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9c0 tx=0x7f08d0009310 comp rx=0 tx=0).stop 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 shutdown_connections 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838d77390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d61ff0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08baffd640 0 client.17166 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:48:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 --2- 172.21.15.86:0/4243804234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d64d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 >> 172.21.15.86:0/4243804234 conn(0x559838d23310 msgr2=0x559838d5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 shutdown_connections 2024-09-05T06:48:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 1 -- 172.21.15.86:0/4243804234 wait complete. 2024-09-05T06:48:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.599+0000 7f08e425f740 20 client.17166 trim_cache size 0 max 16384 2024-09-05T06:48:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 Processor -- start 2024-09-05T06:48:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 -- start start 2024-09-05T06:48:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d5b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b1320 con 0x559838d5bb40 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b2e60 con 0x559838d5a5d0 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08e425f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b19f0 con 0x559838d65370 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50546/0 (socket says 172.21.15.86:50546) 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08db7fe640 1 -- 172.21.15.86:0/2159954804 learned_addr learned my addr 172.21.15.86:0/2159954804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.615+0000 7f08dbfff640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08daffd640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d5b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 -- 172.21.15.86:0/2159954804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 msgr2=0x559838d51ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d51ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 -- 172.21.15.86:0/2159954804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 msgr2=0x559838d5b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d5b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 -- 172.21.15.86:0/2159954804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559838d60540 con 0x559838d5a5d0 2024-09-05T06:48:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08daffd640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d5b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:48:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08db7fe640 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009900 tx=0x7f08cc0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d81e692a0c402c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08c17fa640 1 -- 172.21.15.86:0/2159954804 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc040070 con 0x559838d5a5d0 2024-09-05T06:48:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08c17fa640 1 -- 172.21.15.86:0/2159954804 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc005050 con 0x559838d5a5d0 2024-09-05T06:48:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.616+0000 7f08c17fa640 1 -- 172.21.15.86:0/2159954804 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc035040 con 0x559838d5a5d0 2024-09-05T06:48:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 -- 172.21.15.86:0/2159954804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 msgr2=0x559838d712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009900 tx=0x7f08cc0058d0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 -- 172.21.15.86:0/2159954804 shutdown_connections 2024-09-05T06:48:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d65370 0x559838d51ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d5bb40 0x559838d5b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 --2- 172.21.15.86:0/2159954804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d712b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 -- 172.21.15.86:0/2159954804 >> 172.21.15.86:0/2159954804 conn(0x559838d23310 msgr2=0x559838d610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 -- 172.21.15.86:0/2159954804 shutdown_connections 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.617+0000 7f08e425f740 1 -- 172.21.15.86:0/2159954804 wait complete. 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.618+0000 7f08e425f740 1 -- start start 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.618+0000 7f08e425f740 1 Processor -- start 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.618+0000 7f08e425f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d6d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.618+0000 7f08e425f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08e425f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08e425f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b2790 con 0x559838d65370 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08e425f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b20c0 con 0x559838d5a5d0 2024-09-05T06:48:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08e425f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1af320 con 0x559838d5bb40 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08e425f740 10 client.0 _ll_register_callbacks cb 0x7f08d0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56068/0 (socket says 172.21.15.86:56068) 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 -- 172.21.15.86:0/2773413504 learned_addr learned my addr 172.21.15.86:0/2773413504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08daffd640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 -- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 msgr2=0x559838d6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 -- 172.21.15.86:0/2773413504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 msgr2=0x559838d6d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d6d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.619+0000 7f08dbfff640 1 -- 172.21.15.86:0/2773413504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559838d5f030 con 0x559838d65370 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08dbfff640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f08d400ef00 tx=0x7f08d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcffa60873df2703 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d402d060 con 0x559838d65370 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f08d4012860 con 0x559838d65370 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc005050 con 0x559838d65370 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08d401ee20 con 0x559838d65370 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08e425f740 10 client.17169 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559838de7f10 con 0x559838d65370 2024-09-05T06:48:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc005050 con 0x559838d65370 2024-09-05T06:48:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.620+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f08d4027030 con 0x559838d65370 2024-09-05T06:48:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 10 client.17169 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:48:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 20 client.17169 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838dea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 4 client.17169 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559838cc7d70 con 0x559838d74f40 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08db7fe640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838dea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08db7fe640 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838dea700 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d57b0f15ae51a0f server_cookie=7fd18936abca9a41 in_seq=0 out_seq=0 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.621+0000 7f08baffd640 10 client.17169 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08baffd640 1 -- 172.21.15.86:0/2773413504 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559838cc7d70 con 0x559838d74f40 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08baffd640 10 client.17169 handle_command_reply: tid=0 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 client.17169 shutdown 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 msgr2=0x559838d5e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f08d400ef00 tx=0x7f08d4004ab0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 shutdown_connections 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x559838d74f40 0x559838dea700 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559838d65370 0x559838d5e820 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559838d5bb40 0x559838d6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 --2- 172.21.15.86:0/2773413504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559838d5a5d0 0x559838d6d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08baffd640 0 client.17169 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 >> 172.21.15.86:0/2773413504 conn(0x559838d23310 msgr2=0x559838d608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 shutdown_connections 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 1 -- 172.21.15.86:0/2773413504 wait complete. 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.622+0000 7f08e425f740 20 client.17169 trim_cache size 0 max 16384 2024-09-05T06:48:41.627 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:48:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 msgr2=0x7f08c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 0x7f08c40a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011580 tx=0x7f08d4004a40 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 msgr2=0x7f08dc19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9f0 tx=0x7f08d000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 shutdown_connections 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08c40a2cf0 0x7f08c40a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08dc102ef0 0x7f08dc19ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08dc102570 0x7f08dc19f740 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 --2- 172.21.15.86:0/1982083666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08dc100e20 0x7f08dc19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.623+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 >> 172.21.15.86:0/1982083666 conn(0x7f08dc0fc7d0 msgr2=0x7f08dc1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.624+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 shutdown_connections 2024-09-05T06:48:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:41.624+0000 7f08e1e86640 1 -- 172.21.15.86:0/1982083666 wait complete. 2024-09-05T06:48:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:41 smithi096 ceph-mon[34468]: pgmap v228: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-05T06:48:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:41 smithi115 ceph-mon[34791]: pgmap v228: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-05T06:48:41.976 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:41 smithi086 ceph-mon[30943]: pgmap v228: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 -- 172.21.15.86:0/2698436981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102ef0 msgr2=0x7f701410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 --2- 172.21.15.86:0/2698436981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102ef0 0x7f701410da40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f700c009ca0 tx=0x7f700c02a950 comp rx=0 tx=0).stop 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 -- 172.21.15.86:0/2698436981 shutdown_connections 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 --2- 172.21.15.86:0/2698436981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102ef0 0x7f701410da40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 --2- 172.21.15.86:0/2698436981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014102570 0x7f70141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 --2- 172.21.15.86:0/2698436981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014100e20 0x7f7014101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 -- 172.21.15.86:0/2698436981 >> 172.21.15.86:0/2698436981 conn(0x7f70140fc7d0 msgr2=0x7f70140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 -- 172.21.15.86:0/2698436981 shutdown_connections 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.460+0000 7f7018fc3640 1 -- 172.21.15.86:0/2698436981 wait complete. 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 Processor -- start 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 -- start start 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 0x7f70141988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 0x7f701419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7014110e30 con 0x7f7014102570 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7014110cb0 con 0x7f7014102ef0 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.461+0000 7f7018fc3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7014110fb0 con 0x7f7014100e20 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52740/0 (socket says 172.21.15.86:52740) 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 0x7f70141988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 -- 172.21.15.86:0/4118585504 learned_addr learned my addr 172.21.15.86:0/4118585504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f7012d9d640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 0x7f701419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 -- 172.21.15.86:0/4118585504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 msgr2=0x7f701419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 0x7f701419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 -- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 msgr2=0x7f70141988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 0x7f70141988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 -- 172.21.15.86:0/4118585504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701419d330 con 0x7f7014100e20 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f7012d9d640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 0x7f701419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f7011d9b640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 0x7f70141988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f701259c640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9f0 tx=0x7f700400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f70037fe640 1 -- 172.21.15.86:0/4118585504 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400cd80 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701419d620 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f70037fe640 1 -- 172.21.15.86:0/4118585504 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70040108e0 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.462+0000 7f70037fe640 1 -- 172.21.15.86:0/4118585504 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004018920 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.463+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70140fcbb0 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.465+0000 7f70037fe640 1 -- 172.21.15.86:0/4118585504 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7004010a90 con 0x7f7014100e20 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.465+0000 7f701b39c740 1 Processor -- start 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 -- start start 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f7080 0x55e7e9501400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e9503d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7e945c2a0 con 0x55e7e9501940 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7e9403b10 con 0x55e7e94f7080 2024-09-05T06:48:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701b39c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7e9458260 con 0x55e7e94f6760 2024-09-05T06:48:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e9503d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52752/0 (socket says 172.21.15.86:52752) 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f7012d9d640 1 -- 172.21.15.86:0/3945528579 learned_addr learned my addr 172.21.15.86:0/3945528579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.466+0000 7f701259c640 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f7080 0x55e7e9501400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 -- 172.21.15.86:0/3945528579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 msgr2=0x55e7e94f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 -- 172.21.15.86:0/3945528579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f7080 msgr2=0x55e7e9501400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f7080 0x55e7e9501400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 -- 172.21.15.86:0/3945528579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e91cdb90 con 0x55e7e9501940 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f70037fe640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 0x7f6ffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7011d9b640 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e9503d30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7008005d00 tx=0x7f700800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c9fb7aa61621e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7012d9d640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 0x7f6ffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f70037fe640 1 -- 172.21.15.86:0/4118585504 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f70040cfb60 con 0x7f7014100e20 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7000ff9640 1 -- 172.21.15.86:0/3945528579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008011700 con 0x55e7e9501940 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7000ff9640 1 -- 172.21.15.86:0/3945528579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008011d50 con 0x55e7e9501940 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.467+0000 7f7000ff9640 1 -- 172.21.15.86:0/3945528579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700801a380 con 0x55e7e9501940 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 -- 172.21.15.86:0/3945528579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 msgr2=0x55e7e9503d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e9503d30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7008005d00 tx=0x7f700800bf90 comp rx=0 tx=0).stop 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f7012d9d640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 0x7f6ffc0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f700c02fd10 tx=0x7f700c005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 -- 172.21.15.86:0/3945528579 shutdown_connections 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e9503d30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f7080 0x55e7e9501400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 --2- 172.21.15.86:0/3945528579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 -- 172.21.15.86:0/3945528579 >> 172.21.15.86:0/3945528579 conn(0x55e7e94f2cf0 msgr2=0x55e7e94f30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 -- 172.21.15.86:0/3945528579 shutdown_connections 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.468+0000 7f701b39c740 1 -- 172.21.15.86:0/3945528579 wait complete. 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 -- start start 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 Processor -- start 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e9501940 0x55e7e94ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7e9504510 con 0x55e7e94f7080 2024-09-05T06:48:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7e93e46e0 con 0x55e7e9501940 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7e9504690 con 0x55e7e94f6760 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701b39c740 10 client.0 _ll_register_callbacks cb 0x55e7e9408a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56094/0 (socket says 172.21.15.86:56094) 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f7011d9b640 1 -- 172.21.15.86:0/1880700377 learned_addr learned my addr 172.21.15.86:0/1880700377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e9501940 0x55e7e94ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f701259c640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.469+0000 7f7011d9b640 1 -- 172.21.15.86:0/1880700377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 msgr2=0x55e7e94fe820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94fe820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f7011d9b640 1 -- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e9501940 msgr2=0x55e7e94ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e9501940 0x55e7e94ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f7011d9b640 1 -- 172.21.15.86:0/1880700377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e950da00 con 0x55e7e94f7080 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f700801c7b0 tx=0x7f7008005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5f02206f901062 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f6ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700800bc50 con 0x55e7e94f7080 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f7008004020 con 0x55e7e94f7080 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008011d50 con 0x55e7e94f7080 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7008004220 con 0x55e7e94f7080 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f701b39c740 10 client.17181 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7e9510390 con 0x55e7e94f7080 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.470+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008011d50 con 0x55e7e94f7080 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f70080039a0 con 0x55e7e94f7080 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 10 client.17181 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 10 client.17181 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 20 client.17181 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e95117f0 0x55e7e9513be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 4 client.17181 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e94699d0 con 0x55e7e95117f0 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701259c640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e95117f0 0x55e7e9513be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f701259c640 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e95117f0 0x55e7e9513be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38158bb0bcb6b0fe server_cookie=38d8a711cc1f3ed0 in_seq=0 out_seq=0 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.471+0000 7f6ff9ffb640 10 client.17181 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/1880700377 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7e94699d0 con 0x55e7e95117f0 2024-09-05T06:48:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f6ff9ffb640 10 client.17181 handle_command_reply: tid=0 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 client.17181 shutdown 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 msgr2=0x55e7e94fed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f700801c7b0 tx=0x7f7008005740 comp rx=0 tx=0).stop 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 shutdown_connections 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e95117f0 0x55e7e9513be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e9501940 0x55e7e94ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fed60 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 --2- 172.21.15.86:0/1880700377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f6760 0x55e7e94fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 >> 172.21.15.86:0/1880700377 conn(0x55e7e94f2cf0 msgr2=0x55e7e94f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f6ff9ffb640 0 client.17181 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 shutdown_connections 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.472+0000 7f701b39c740 1 -- 172.21.15.86:0/1880700377 wait complete. 2024-09-05T06:48:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.473+0000 7f701b39c740 20 client.17181 trim_cache size 0 max 16384 2024-09-05T06:48:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.491+0000 7f701b39c740 1 Processor -- start 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.491+0000 7f701b39c740 1 -- start start 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 0x55e7e9513800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b57b0 con 0x55e7e9501940 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b72f0 con 0x55e7e94f6760 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701b39c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b5e80 con 0x55e7e94f7080 2024-09-05T06:48:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 0x55e7e9513800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58268/0 (socket says 172.21.15.86:58268) 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7012d9d640 1 -- 172.21.15.86:0/1630696519 learned_addr learned my addr 172.21.15.86:0/1630696519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f701259c640 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 -- 172.21.15.86:0/1630696519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 msgr2=0x55e7e94f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 -- 172.21.15.86:0/1630696519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 msgr2=0x55e7e95193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 -- 172.21.15.86:0/1630696519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e94f3980 con 0x55e7e94f7080 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7012d9d640 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f7011d9b640 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 0x55e7e9513800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7008011010 tx=0x7f700800b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ecec98438e77c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.492+0000 7f6ffbfff640 1 -- 172.21.15.86:0/1630696519 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008003fb0 con 0x55e7e94f7080 2024-09-05T06:48:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f6ffbfff640 1 -- 172.21.15.86:0/1630696519 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008003aa0 con 0x55e7e94f7080 2024-09-05T06:48:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f6ffbfff640 1 -- 172.21.15.86:0/1630696519 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008026810 con 0x55e7e94f7080 2024-09-05T06:48:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 -- 172.21.15.86:0/1630696519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 msgr2=0x55e7e9513800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 0x55e7e9513800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7008011010 tx=0x7f700800b410 comp rx=0 tx=0).stop 2024-09-05T06:48:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 -- 172.21.15.86:0/1630696519 shutdown_connections 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e9501940 0x55e7e95193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e94f7080 0x55e7e9513800 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 --2- 172.21.15.86:0/1630696519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 -- 172.21.15.86:0/1630696519 >> 172.21.15.86:0/1630696519 conn(0x55e7e94f2cf0 msgr2=0x55e7e95008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 -- 172.21.15.86:0/1630696519 shutdown_connections 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.493+0000 7f701b39c740 1 -- 172.21.15.86:0/1630696519 wait complete. 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 -- start start 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 Processor -- start 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e9501940 0x55e7e94fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b6c20 con 0x55e7e94f7080 2024-09-05T06:48:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b6550 con 0x55e7e94f6760 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70141b37b0 con 0x55e7e9501940 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701b39c740 10 client.0 _ll_register_callbacks cb 0x7f700800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55574/0 (socket says 172.21.15.86:55574) 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701259c640 1 -- 172.21.15.86:0/183096391 learned_addr learned my addr 172.21.15.86:0/183096391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7012d9d640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e9501940 0x55e7e94fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 -- 172.21.15.86:0/183096391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e9501940 msgr2=0x55e7e94fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e9501940 0x55e7e94fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 -- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 msgr2=0x55e7e94fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f7011d9b640 1 -- 172.21.15.86:0/183096391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e957b4a0 con 0x55e7e94f7080 2024-09-05T06:48:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.494+0000 7f701259c640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f7011d9b640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fbea0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55e7e94f9250 tx=0x7f700800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b8d6aefb4b5fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f6ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008041070 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f70080035f0 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008003aa0 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7008049050 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f701b39c740 10 client.17187 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7e957c700 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.495+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008003aa0 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f7008044030 con 0x55e7e94f7080 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 10 client.17187 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 10 client.17187 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:48:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 20 client.17187 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e957ce20 0x55e7e957f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 4 client.17187 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55e7e9463d20 con 0x55e7e957ce20 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701259c640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e957ce20 0x55e7e957f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f701259c640 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e957ce20 0x55e7e957f210 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e58450b10da4f9c server_cookie=225f99d6db5be20f in_seq=0 out_seq=0 2024-09-05T06:48:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.496+0000 7f6ff9ffb640 10 client.17187 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:42.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 2.02 MB/sec execute 242 sec latency 22.098 ms 2024-09-05T06:48:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f6ff9ffb640 1 -- 172.21.15.86:0/183096391 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e7e9463d20 con 0x55e7e957ce20 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f6ff9ffb640 10 client.17187 handle_command_reply: tid=0 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 client.17187 shutdown 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 msgr2=0x55e7e94fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fbea0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55e7e94f9250 tx=0x7f700800bfc0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 shutdown_connections 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e7e957ce20 0x55e7e957f210 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7e9501940 0x55e7e94fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f6ff9ffb640 0 client.17187 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7e94f7080 0x55e7e94fbea0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 --2- 172.21.15.86:0/183096391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7e94f6760 0x55e7e94fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 >> 172.21.15.86:0/183096391 conn(0x55e7e94f2cf0 msgr2=0x55e7e94fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 shutdown_connections 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 1 -- 172.21.15.86:0/183096391 wait complete. 2024-09-05T06:48:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.564+0000 7f701b39c740 20 client.17187 trim_cache size 0 max 16384 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 22, 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "9a3f8548": { 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 22, 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:48:42.567 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:48:42.568 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:42.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:42.569 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 msgr2=0x7f6ffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 0x7f6ffc0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f700c02fd10 tx=0x7f700c005c50 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 msgr2=0x7f7014198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f700400e9f0 tx=0x7f700400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 shutdown_connections 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6ffc0a2cf0 0x7f6ffc0a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7014102ef0 0x7f701419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7014102570 0x7f70141988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 --2- 172.21.15.86:0/4118585504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7014100e20 0x7f7014198360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 >> 172.21.15.86:0/4118585504 conn(0x7f70140fc7d0 msgr2=0x7f701406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 shutdown_connections 2024-09-05T06:48:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:42.565+0000 7f7018fc3640 1 -- 172.21.15.86:0/4118585504 wait complete. 2024-09-05T06:48:43.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:48:43.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 2.02 MB/sec execute 243 sec latency 1022.124 ms 2024-09-05T06:48:43.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:43 smithi086 ceph-mon[30943]: pgmap v229: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:43.777 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9a3f8548', quiesce took 6.5 seconds. Will release after: 3.5 2024-09-05T06:48:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:43 smithi096 ceph-mon[34468]: pgmap v229: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:43 smithi115 ceph-mon[34791]: pgmap v229: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 -- 172.21.15.86:0/948645865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0103660 msgr2=0x7f82f010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 --2- 172.21.15.86:0/948645865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0103660 0x7f82f010bb50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009900 tx=0x7f82dc02a480 comp rx=0 tx=0).stop 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 -- 172.21.15.86:0/948645865 shutdown_connections 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 --2- 172.21.15.86:0/948645865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f010c170 0x7f82f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 --2- 172.21.15.86:0/948645865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0103660 0x7f82f010bb50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 --2- 172.21.15.86:0/948645865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0102d40 0x7f82f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 -- 172.21.15.86:0/948645865 >> 172.21.15.86:0/948645865 conn(0x7f82f00fc7a0 msgr2=0x7f82f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 -- 172.21.15.86:0/948645865 shutdown_connections 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.294+0000 7f82f70b8640 1 -- 172.21.15.86:0/948645865 wait complete. 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 Processor -- start 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 -- start start 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 0x7f82f0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f010c170 0x7f82f019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0111e80 con 0x7f82f0102d40 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0111d00 con 0x7f82f010c170 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f70b8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0112000 con 0x7f82f0103660 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58300/0 (socket says 172.21.15.86:58300) 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 -- 172.21.15.86:0/3638448022 learned_addr learned my addr 172.21.15.86:0/3638448022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82effff640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 0x7f82f0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 -- 172.21.15.86:0/3638448022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 msgr2=0x7f82f0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 0x7f82f0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f5655640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f010c170 0x7f82f019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 -- 172.21.15.86:0/3638448022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f010c170 msgr2=0x7f82f019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f010c170 0x7f82f019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82f4e54640 1 -- 172.21.15.86:0/3638448022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f019d3b0 con 0x7f82f0102d40 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.295+0000 7f82effff640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 0x7f82f0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82f4e54640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f82e400b4a0 tx=0x7f82e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82edffb640 1 -- 172.21.15.86:0/3638448022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4014070 con 0x7f82f0102d40 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82edffb640 1 -- 172.21.15.86:0/3638448022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82e4010ac0 con 0x7f82f0102d40 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f019d6a0 con 0x7f82f0102d40 2024-09-05T06:48:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82edffb640 1 -- 172.21.15.86:0/3638448022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4017c30 con 0x7f82f0102d40 2024-09-05T06:48:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.296+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01a4f80 con 0x7f82f0102d40 2024-09-05T06:48:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.298+0000 7f82edffb640 1 -- 172.21.15.86:0/3638448022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f82e4017070 con 0x7f82f0102d40 2024-09-05T06:48:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 Processor -- start 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 -- start start 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82edffb640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 0x7f82e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56263c7492f0 con 0x56263c7e35d0 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56263c6f0cf0 con 0x56263c7e4b40 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f9491740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56263c7452b0 con 0x56263c7ee370 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f5655640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 0x7f82e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58304/0 (socket says 172.21.15.86:58304) 2024-09-05T06:48:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82effff640 1 -- 172.21.15.86:0/2139441584 learned_addr learned my addr 172.21.15.86:0/2139441584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82edffb640 1 -- 172.21.15.86:0/3638448022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f82e40cef60 con 0x7f82f0102d40 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.299+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 -- 172.21.15.86:0/2139441584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 msgr2=0x56263c7f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 -- 172.21.15.86:0/2139441584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 msgr2=0x56263c7e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 -- 172.21.15.86:0/2139441584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56263c6d53a0 con 0x56263c7e35d0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82f5655640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 0x7f82e00a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005fd0 tx=0x7f82e80097a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82effff640 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f82dc02a9a0 tx=0x7f82dc032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def4b7a7acbd0db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82daffd640 1 -- 172.21.15.86:0/2139441584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc03f070 con 0x56263c7e35d0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82daffd640 1 -- 172.21.15.86:0/2139441584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82dc0316a0 con 0x56263c7e35d0 2024-09-05T06:48:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.300+0000 7f82daffd640 1 -- 172.21.15.86:0/2139441584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc035070 con 0x56263c7e35d0 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- 172.21.15.86:0/2139441584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 msgr2=0x56263c7e3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f82dc02a9a0 tx=0x7f82dc032da0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- 172.21.15.86:0/2139441584 shutdown_connections 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e3a10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 --2- 172.21.15.86:0/2139441584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- 172.21.15.86:0/2139441584 >> 172.21.15.86:0/2139441584 conn(0x56263c7ac310 msgr2=0x56263c7dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- 172.21.15.86:0/2139441584 shutdown_connections 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- 172.21.15.86:0/2139441584 wait complete. 2024-09-05T06:48:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 -- start start 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.301+0000 7f82f9491740 1 Processor -- start 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 0x56263c7eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56263c7f1010 con 0x56263c7e4b40 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56263c704060 con 0x56263c7e35d0 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56263c7f1190 con 0x56263c7ee370 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 10 client.0 _ll_register_callbacks cb 0x56263c4578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58318/0 (socket says 172.21.15.86:58318) 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 -- 172.21.15.86:0/132483446 learned_addr learned my addr 172.21.15.86:0/132483446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f5655640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f4e54640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 0x56263c7eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 -- 172.21.15.86:0/132483446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 msgr2=0x56263c7eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 -- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 msgr2=0x56263c7eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 0x56263c7eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 -- 172.21.15.86:0/132483446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263c4ba3c0 con 0x56263c7e4b40 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f5655640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f4e54640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 0x56263c7eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82effff640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005940 tx=0x7f82dc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2528cda29f0641e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82cffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc04a030 con 0x56263c7e4b40 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f82dc004020 con 0x56263c7e4b40 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.302+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc0316a0 con 0x56263c7e4b40 2024-09-05T06:48:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82dc054050 con 0x56263c7e4b40 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 10 client.17199 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56263c7fcb30 con 0x56263c7e4b40 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82dc0316a0 con 0x56263c7e4b40 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f82dc003680 con 0x56263c7e4b40 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 10 client.17199 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 20 client.17199 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c7fde80 0x56263c800270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 4 client.17199 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56263c756860 con 0x56263c7fde80 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.303+0000 7f82f4e54640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c7fde80 0x56263c800270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.304+0000 7f82f4e54640 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c7fde80 0x56263c800270 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa0f98a043b6a3a server_cookie=816bb821885eae7b in_seq=0 out_seq=0 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.304+0000 7f82cffff640 10 client.17199 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.304+0000 7f82cffff640 1 -- 172.21.15.86:0/132483446 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56263c756860 con 0x56263c7fde80 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.304+0000 7f82cffff640 10 client.17199 handle_command_reply: tid=0 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.304+0000 7f82f9491740 1 client.17199 shutdown 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 msgr2=0x56263c7eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005940 tx=0x7f82dc005d50 comp rx=0 tx=0).stop 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 shutdown_connections 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c7fde80 0x56263c800270 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7ee370 0x56263c7eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e4b40 0x56263c7eb060 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 --2- 172.21.15.86:0/132483446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e35d0 0x56263c7eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82cffff640 0 client.17199 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 >> 172.21.15.86:0/132483446 conn(0x56263c7ac310 msgr2=0x56263c7e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 shutdown_connections 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 1 -- 172.21.15.86:0/132483446 wait complete. 2024-09-05T06:48:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.305+0000 7f82f9491740 20 client.17199 trim_cache size 0 max 16384 2024-09-05T06:48:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 Processor -- start 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 -- start start 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e35d0 0x56263c7dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7ea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01af170 con 0x56263c7ee370 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b0cb0 con 0x56263c7e4b40 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f9491740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01af840 con 0x56263c7e35d0 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58326/0 (socket says 172.21.15.86:58326) 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 -- 172.21.15.86:0/2458295328 learned_addr learned my addr 172.21.15.86:0/2458295328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e35d0 0x56263c7dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82effff640 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 -- 172.21.15.86:0/2458295328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e35d0 msgr2=0x56263c7dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e35d0 0x56263c7dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 -- 172.21.15.86:0/2458295328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 msgr2=0x56263c7ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7ea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.318+0000 7f82f5655640 1 -- 172.21.15.86:0/2458295328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56263c7e0960 con 0x56263c7ee370 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.319+0000 7f82f5655640 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f82e8012520 tx=0x7f82e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7ffae5706254b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.319+0000 7f82da7fc640 1 -- 172.21.15.86:0/2458295328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004d20 con 0x56263c7ee370 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.319+0000 7f82da7fc640 1 -- 172.21.15.86:0/2458295328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82e8016070 con 0x56263c7ee370 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.319+0000 7f82da7fc640 1 -- 172.21.15.86:0/2458295328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8020730 con 0x56263c7ee370 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- 172.21.15.86:0/2458295328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 msgr2=0x56263c8058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f82e8012520 tx=0x7f82e8014000 comp rx=0 tx=0).stop 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- 172.21.15.86:0/2458295328 shutdown_connections 2024-09-05T06:48:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7ee370 0x56263c8058a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7e4b40 0x56263c7ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- 172.21.15.86:0/2458295328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e35d0 0x56263c7dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- 172.21.15.86:0/2458295328 >> 172.21.15.86:0/2458295328 conn(0x56263c7ac310 msgr2=0x56263c7df700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- 172.21.15.86:0/2458295328 shutdown_connections 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- 172.21.15.86:0/2458295328 wait complete. 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 -- start start 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 Processor -- start 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e4b40 0x56263c7e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.320+0000 7f82f9491740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7ee370 0x56263c7e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f9491740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b05e0 con 0x56263c7e35d0 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f9491740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01aff10 con 0x56263c7ee370 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f9491740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01ad170 con 0x56263c7e4b40 2024-09-05T06:48:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f9491740 10 client.0 _ll_register_callbacks cb 0x7f82dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58330/0 (socket says 172.21.15.86:58330) 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 -- 172.21.15.86:0/2832836552 learned_addr learned my addr 172.21.15.86:0/2832836552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82effff640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e4b40 0x56263c7e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f5655640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7ee370 0x56263c7e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 -- 172.21.15.86:0/2832836552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e4b40 msgr2=0x56263c7e8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e4b40 0x56263c7e8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 -- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7ee370 msgr2=0x56263c7e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7ee370 0x56263c7e8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 -- 172.21.15.86:0/2832836552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263c7f4aa0 con 0x56263c7e35d0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f4e54640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f82e4097ac0 tx=0x7f82e40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ff189caf56d438 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82cffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40b4070 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f82e409f1b0 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.321+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e8016070 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82e401e040 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 10 client.17205 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56263c871430 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82e8016070 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f82e40b5020 con 0x56263c7e35d0 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 10 client.17205 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 20 client.17205 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c871b50 0x56263c873f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 4 client.17205 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:48:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56263c750d70 con 0x56263c871b50 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.322+0000 7f82effff640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c871b50 0x56263c873f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82effff640 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c871b50 0x56263c873f40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdf59a2216e833b server_cookie=7845743b4e8fcf50 in_seq=0 out_seq=0 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82cffff640 10 client.17205 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82cffff640 1 -- 172.21.15.86:0/2832836552 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56263c750d70 con 0x56263c871b50 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82cffff640 10 client.17205 handle_command_reply: tid=0 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 client.17205 shutdown 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 msgr2=0x56263c7e8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f82e4097ac0 tx=0x7f82e40047d0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 shutdown_connections 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56263c871b50 0x56263c873f40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263c7ee370 0x56263c7e8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56263c7e4b40 0x56263c7e8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 --2- 172.21.15.86:0/2832836552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56263c7e35d0 0x56263c7e8140 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82cffff640 0 client.17205 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 >> 172.21.15.86:0/2832836552 conn(0x56263c7ac310 msgr2=0x56263c7ec8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 shutdown_connections 2024-09-05T06:48:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 1 -- 172.21.15.86:0/2832836552 wait complete. 2024-09-05T06:48:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.323+0000 7f82f9491740 20 client.17205 trim_cache size 0 max 16384 2024-09-05T06:48:44.336 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 msgr2=0x7f82e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 0x7f82e00a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005fd0 tx=0x7f82e80097a0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 msgr2=0x7f82f0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f82e400b4a0 tx=0x7f82e400b960 comp rx=0 tx=0).stop 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 shutdown_connections 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82e00a2cf0 0x7f82e00a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f010c170 0x7f82f019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f0103660 0x7f82f0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 --2- 172.21.15.86:0/3638448022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f0102d40 0x7f82f0198400 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 >> 172.21.15.86:0/3638448022 conn(0x7f82f00fc7a0 msgr2=0x7f82f00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 shutdown_connections 2024-09-05T06:48:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:44.324+0000 7f82f70b8640 1 -- 172.21.15.86:0/3638448022 wait complete. 2024-09-05T06:48:44.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 2.01 MB/sec execute 244 sec latency 2022.205 ms 2024-09-05T06:48:45.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 2.00 MB/sec execute 245 sec latency 3022.278 ms 2024-09-05T06:48:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:45 smithi096 ceph-mon[34468]: pgmap v230: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:45 smithi115 ceph-mon[34791]: pgmap v230: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:45.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:45 smithi086 ceph-mon[30943]: pgmap v230: 129 pgs: 129 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-09-05T06:48:46.130 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:48:46.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:48:46.393 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:46.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 1.99 MB/sec execute 246 sec latency 4022.383 ms 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 -- 172.21.15.86:0/953148460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 msgr2=0x7f50841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 --2- 172.21.15.86:0/953148460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50841029b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f506c009900 tx=0x7f506c02a480 comp rx=0 tx=0).stop 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 -- 172.21.15.86:0/953148460 shutdown_connections 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 --2- 172.21.15.86:0/953148460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 0x7f508410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 --2- 172.21.15.86:0/953148460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50841029b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 --2- 172.21.15.86:0/953148460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 0x7f5084101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 -- 172.21.15.86:0/953148460 >> 172.21.15.86:0/953148460 conn(0x7f50840fc7d0 msgr2=0x7f50840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 -- 172.21.15.86:0/953148460 shutdown_connections 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.864+0000 7f5088e65640 1 -- 172.21.15.86:0/953148460 wait complete. 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 Processor -- start 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 -- start start 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 0x7f50840fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 0x7f50840fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110e30 con 0x7f5084102570 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110cb0 con 0x7f5084102ef0 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5088e65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110fb0 con 0x7f5084100e20 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58348/0 (socket says 172.21.15.86:58348) 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 -- 172.21.15.86:0/4190135249 learned_addr learned my addr 172.21.15.86:0/4190135249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f508259c640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 0x7f50840fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5082d9d640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 0x7f50840fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 -- 172.21.15.86:0/4190135249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 msgr2=0x7f50840fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 0x7f50840fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 -- 172.21.15.86:0/4190135249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 msgr2=0x7f50840fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 0x7f50840fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.865+0000 7f5081d9b640 1 -- 172.21.15.86:0/4190135249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a68b0 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f5081d9b640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f507b7fe640 1 -- 172.21.15.86:0/4190135249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c03f030 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a6b40 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f507b7fe640 1 -- 172.21.15.86:0/4190135249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f506c003cb0 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f507b7fe640 1 -- 172.21.15.86:0/4190135249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c035070 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.866+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a7070 con 0x7f5084102570 2024-09-05T06:48:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.868+0000 7f507b7fe640 1 -- 172.21.15.86:0/4190135249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f506c042030 con 0x7f5084102570 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 Processor -- start 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 -- start start 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108823800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x562108826130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56210877e280 con 0x562108818b60 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562108725b10 con 0x562108819480 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508b23e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56210877a240 con 0x562108823d40 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f5082d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f5082d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58362/0 (socket says 172.21.15.86:58362) 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f5082d9d640 1 -- 172.21.15.86:0/721339635 learned_addr learned my addr 172.21.15.86:0/721339635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f507b7fe640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 0x7f50700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f508259c640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108823800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.869+0000 7f5081d9b640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x562108826130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f507b7fe640 1 -- 172.21.15.86:0/4190135249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f506c0efcc0 con 0x7f5084102570 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 -- 172.21.15.86:0/721339635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 msgr2=0x562108826130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x562108826130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 -- 172.21.15.86:0/721339635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 msgr2=0x562108823800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108823800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 -- 172.21.15.86:0/721339635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621087321f0 con 0x562108818b60 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f508259c640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108823800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5081d9b640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x562108826130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 0x7f50700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b760 tx=0x7f507c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d550eeac173061 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5078ff9640 1 -- 172.21.15.86:0/721339635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c023070 con 0x562108818b60 2024-09-05T06:48:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5078ff9640 1 -- 172.21.15.86:0/721339635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f507c01db30 con 0x562108818b60 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5078ff9640 1 -- 172.21.15.86:0/721339635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c01b940 con 0x562108818b60 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.870+0000 7f5082d9d640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 0x7f50700a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f507c020000 tx=0x7f507c01d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- 172.21.15.86:0/721339635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 msgr2=0x562108818f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b760 tx=0x7f507c00bc20 comp rx=0 tx=0).stop 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- 172.21.15.86:0/721339635 shutdown_connections 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x562108826130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108823800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 --2- 172.21.15.86:0/721339635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x562108818f40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- 172.21.15.86:0/721339635 >> 172.21.15.86:0/721339635 conn(0x5621088150f0 msgr2=0x5621088154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- 172.21.15.86:0/721339635 shutdown_connections 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- 172.21.15.86:0/721339635 wait complete. 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 -- start start 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.871+0000 7f508b23e740 1 Processor -- start 2024-09-05T06:48:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x562108820bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108821110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562108826910 con 0x562108823d40 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562108738f10 con 0x562108819480 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562108826a90 con 0x562108818b60 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 10 client.0 _ll_register_callbacks cb 0x562108701a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58378/0 (socket says 172.21.15.86:58378) 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 -- 172.21.15.86:0/3828084458 learned_addr learned my addr 172.21.15.86:0/3828084458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5081d9b640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108821110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508259c640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x562108820bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 -- 172.21.15.86:0/3828084458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 msgr2=0x562108820bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x562108820bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 -- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 msgr2=0x562108821110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108821110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 -- 172.21.15.86:0/3828084458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621084ef6a0 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f507c015040 tx=0x7f507c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e121a9654d70317 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c023040 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f507c01b8d0 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.872+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c01db30 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f507c037050 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 10 client.17217 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5621088324f0 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f507c01db30 con 0x562108823d40 2024-09-05T06:48:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f507c01dce0 con 0x562108823d40 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 10 client.17217 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 20 client.17217 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562108833950 0x562108835d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 4 client.17217 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210878b7f0 con 0x562108833950 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.873+0000 7f508259c640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562108833950 0x562108835d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.874+0000 7f5069ffb640 10 client.17217 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.874+0000 7f508259c640 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562108833950 0x562108835d40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c03c8f160ebcf409 server_cookie=502bdd701effab35 in_seq=0 out_seq=0 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.874+0000 7f5069ffb640 1 -- 172.21.15.86:0/3828084458 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56210878b7f0 con 0x562108833950 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.874+0000 7f5069ffb640 10 client.17217 handle_command_reply: tid=0 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.874+0000 7f508b23e740 1 client.17217 shutdown 2024-09-05T06:48:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 msgr2=0x562108821650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f507c015040 tx=0x7f507c00bee0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 shutdown_connections 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562108833950 0x562108835d40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108823d40 0x562108821650 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108821110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f5069ffb640 0 client.17217 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 --2- 172.21.15.86:0/3828084458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x562108820bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 >> 172.21.15.86:0/3828084458 conn(0x5621088150f0 msgr2=0x56210881ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 shutdown_connections 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 1 -- 172.21.15.86:0/3828084458 wait complete. 2024-09-05T06:48:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.875+0000 7f508b23e740 20 client.17217 trim_cache size 0 max 16384 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 Processor -- start 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 -- start start 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108822df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x56210883b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b12c0 con 0x562108818b60 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b2e00 con 0x562108819480 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508b23e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b1990 con 0x562108823d40 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58394/0 (socket says 172.21.15.86:58394) 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508259c640 1 -- 172.21.15.86:0/1689838154 learned_addr learned my addr 172.21.15.86:0/1689838154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f5081d9b640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108822df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f5082d9d640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x56210883b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.893+0000 7f508259c640 1 -- 172.21.15.86:0/1689838154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 msgr2=0x56210883b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f508259c640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x56210883b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f508259c640 1 -- 172.21.15.86:0/1689838154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 msgr2=0x562108822df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f508259c640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108822df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f508259c640 1 -- 172.21.15.86:0/1689838154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621088202c0 con 0x562108818b60 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f5081d9b640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108822df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f508259c640 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x5621088362e0 tx=0x7f5074014bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e655ca9a7e798f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f506bfff640 1 -- 172.21.15.86:0/1689838154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074024030 con 0x562108818b60 2024-09-05T06:48:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f506bfff640 1 -- 172.21.15.86:0/1689838154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5074003e20 con 0x562108818b60 2024-09-05T06:48:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.894+0000 7f506bfff640 1 -- 172.21.15.86:0/1689838154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740042b0 con 0x562108818b60 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- 172.21.15.86:0/1689838154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 msgr2=0x56210881b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x5621088362e0 tx=0x7f5074014bd0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- 172.21.15.86:0/1689838154 shutdown_connections 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108823d40 0x56210883b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108819480 0x562108822df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- 172.21.15.86:0/1689838154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108818b60 0x56210881b610 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- 172.21.15.86:0/1689838154 >> 172.21.15.86:0/1689838154 conn(0x5621088150f0 msgr2=0x56210881fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- 172.21.15.86:0/1689838154 shutdown_connections 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- 172.21.15.86:0/1689838154 wait complete. 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- start start 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 Processor -- start 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x56210882bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b2730 con 0x562108819480 2024-09-05T06:48:46.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b2060 con 0x562108823d40 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.895+0000 7f508b23e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f50841af2c0 con 0x562108818b60 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f508b23e740 10 client.0 _ll_register_callbacks cb 0x7f507c021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5081d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f508259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x56210882bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5081d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58398/0 (socket says 172.21.15.86:58398) 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f508259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x56210882bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56634/0 (socket says 172.21.15.86:56634) 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f508259c640 1 -- 172.21.15.86:0/3170867719 learned_addr learned my addr 172.21.15.86:0/3170867719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55700/0 (socket says 172.21.15.86:55700) 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 -- 172.21.15.86:0/3170867719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 msgr2=0x56210882bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x56210882bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 -- 172.21.15.86:0/3170867719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 msgr2=0x56210882c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 -- 172.21.15.86:0/3170867719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562108829250 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5081d9b640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5082d9d640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f507c0056b0 tx=0x7f507c01bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828293ee8b741fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.896+0000 7f5069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c023040 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5074003e20 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f507c004340 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f507c044050 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 10 client.26480 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56210889ebf0 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5074003e20 con 0x562108823d40 2024-09-05T06:48:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f507c03a070 con 0x562108823d40 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 10 client.26480 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 10 client.26480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 20 client.26480 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56210889f1f0 0x5621088a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 4 client.26480 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562108785d00 con 0x56210889f1f0 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.897+0000 7f508259c640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56210889f1f0 0x5621088a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.898+0000 7f508259c640 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56210889f1f0 0x5621088a1610 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ebd09c01ed8d03 server_cookie=f7093eaa23c0eaac in_seq=0 out_seq=0 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.898+0000 7f5069ffb640 10 client.26480 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f5069ffb640 1 -- 172.21.15.86:0/3170867719 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562108785d00 con 0x56210889f1f0 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f5069ffb640 10 client.26480 handle_command_reply: tid=0 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 client.26480 shutdown 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 msgr2=0x562108828ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f507c0056b0 tx=0x7f507c01bb00 comp rx=0 tx=0).stop 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 shutdown_connections 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56210889f1f0 0x5621088a1610 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562108823d40 0x562108828ce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562108819480 0x56210882c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f5069ffb640 0 client.26480 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 --2- 172.21.15.86:0/3170867719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562108818b60 0x56210882bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 >> 172.21.15.86:0/3170867719 conn(0x5621088150f0 msgr2=0x562108813a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 shutdown_connections 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 1 -- 172.21.15.86:0/3170867719 wait complete. 2024-09-05T06:48:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.899+0000 7f508b23e740 20 client.26480 trim_cache size 0 max 16384 2024-09-05T06:48:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:48:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "ffddb7d7-a416-4602-89b1-225a1b37e208", 2024-09-05T06:48:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:48:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 msgr2=0x7f50700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 0x7f50700a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f507c020000 tx=0x7f507c01d000 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 msgr2=0x7f50840fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c0042f0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 shutdown_connections 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50700a2d70 0x7f50700a5230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5084102ef0 0x7f50840fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5084102570 0x7f50840fd1c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 --2- 172.21.15.86:0/4190135249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5084100e20 0x7f50840fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 >> 172.21.15.86:0/4190135249 conn(0x7f50840fc7d0 msgr2=0x7f508406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 shutdown_connections 2024-09-05T06:48:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:46.900+0000 7f5088e65640 1 -- 172.21.15.86:0/4190135249 wait complete. 2024-09-05T06:48:47.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:48:47.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 1.98 MB/sec execute 247 sec latency 5022.436 ms 2024-09-05T06:48:47.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:47 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:47.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:47 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:47 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:47.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:47 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:47 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:47 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:47 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:47 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:47 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:47 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:47 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:47 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:47.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:48:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:48.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:48:48.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:48.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:48.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:48:48.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:48 smithi086 ceph-mon[30943]: pgmap v231: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-05T06:48:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 -- 172.21.15.86:0/4114055537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd18102d30 msgr2=0x7efd18103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 --2- 172.21.15.86:0/4114055537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd18102d30 0x7efd18103110 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efd08009930 tx=0x7efd0802a440 comp rx=0 tx=0).stop 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 -- 172.21.15.86:0/4114055537 shutdown_connections 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 --2- 172.21.15.86:0/4114055537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 0x7efd1810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 --2- 172.21.15.86:0/4114055537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd181036e0 0x7efd18107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 --2- 172.21.15.86:0/4114055537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd18102d30 0x7efd18103110 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 -- 172.21.15.86:0/4114055537 >> 172.21.15.86:0/4114055537 conn(0x7efd180fc7d0 msgr2=0x7efd180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 -- 172.21.15.86:0/4114055537 shutdown_connections 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 -- 172.21.15.86:0/4114055537 wait complete. 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.469+0000 7efd1dd59640 1 Processor -- start 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 -- start start 2024-09-05T06:48:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd18102d30 0x7efd18198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 0x7efd1819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111fb0 con 0x7efd181036e0 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111e30 con 0x7efd18102d30 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd1dd59640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd18112130 con 0x7efd181082a0 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58422/0 (socket says 172.21.15.86:58422) 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 -- 172.21.15.86:0/2687547663 learned_addr learned my addr 172.21.15.86:0/2687547663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd177fe640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd18102d30 0x7efd18198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd17fff640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 0x7efd1819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 -- 172.21.15.86:0/2687547663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 msgr2=0x7efd1819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 0x7efd1819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 -- 172.21.15.86:0/2687547663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd18102d30 msgr2=0x7efd18198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd18102d30 0x7efd18198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.470+0000 7efd16ffd640 1 -- 172.21.15.86:0/2687547663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1819d500 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd16ffd640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011830 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd0c016070 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1819d7f0 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c010df0 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.471+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a50d0 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.472+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1806a850 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.473+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd0c0119d0 con 0x7efd181036e0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.473+0000 7efd14ff9640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 0x7efd000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.474+0000 7efd177fe640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 0x7efd000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.474+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7efd0c0ceca0 con 0x7efd181036e0 2024-09-05T06:48:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.474+0000 7efd177fe640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 0x7efd000a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd08009900 tx=0x7efd08000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.478+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd0c09b590 con 0x7efd181036e0 2024-09-05T06:48:48.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 1.98 MB/sec execute 248 sec latency 6022.501 ms 2024-09-05T06:48:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:48 smithi115 ceph-mon[34791]: pgmap v231: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-05T06:48:48.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.697+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd1810a2b0 con 0x7efd181036e0 2024-09-05T06:48:48.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.698+0000 7efd14ff9640 1 -- 172.21.15.86:0/2687547663 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7efd0c09ae90 con 0x7efd181036e0 2024-09-05T06:48:48.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:48:48.698 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:48:48.699 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:48:48.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 msgr2=0x7efd000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:48.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 0x7efd000a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd08009900 tx=0x7efd08000bc0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 msgr2=0x7efd18198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c00b8e0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 shutdown_connections 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd000a2d70 0x7efd000a5230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd181082a0 0x7efd1819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd181036e0 0x7efd18198a90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 --2- 172.21.15.86:0/2687547663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd18102d30 0x7efd18198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 >> 172.21.15.86:0/2687547663 conn(0x7efd180fc7d0 msgr2=0x7efd18105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 shutdown_connections 2024-09-05T06:48:48.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:48.700+0000 7efd1dd59640 1 -- 172.21.15.86:0/2687547663 wait complete. 2024-09-05T06:48:48.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:48:48.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:48.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:48.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:48:48.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:48:48.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:48 smithi096 ceph-mon[34468]: pgmap v231: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-05T06:48:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 -- 172.21.15.86:0/1834325069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 msgr2=0x7fcb7810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 --2- 172.21.15.86:0/1834325069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7810da40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009900 tx=0x7fcb6802a480 comp rx=0 tx=0).stop 2024-09-05T06:48:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 -- 172.21.15.86:0/1834325069 shutdown_connections 2024-09-05T06:48:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 --2- 172.21.15.86:0/1834325069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7810da40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 --2- 172.21.15.86:0/1834325069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78102570 0x7fcb781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 --2- 172.21.15.86:0/1834325069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78100e20 0x7fcb78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 -- 172.21.15.86:0/1834325069 >> 172.21.15.86:0/1834325069 conn(0x7fcb780fc7d0 msgr2=0x7fcb780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 -- 172.21.15.86:0/1834325069 shutdown_connections 2024-09-05T06:48:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.162+0000 7fcb7c915640 1 -- 172.21.15.86:0/1834325069 wait complete. 2024-09-05T06:48:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 Processor -- start 2024-09-05T06:48:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 -- start start 2024-09-05T06:48:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 0x7fcb78197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78102570 0x7fcb78197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78110e30 con 0x7fcb78102ef0 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78110cb0 con 0x7fcb78102570 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7c915640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78110fb0 con 0x7fcb78100e20 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58438/0 (socket says 172.21.15.86:58438) 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 -- 172.21.15.86:0/464110601 learned_addr learned my addr 172.21.15.86:0/464110601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78102570 0x7fcb78197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7659c640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 0x7fcb78197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 -- 172.21.15.86:0/464110601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 msgr2=0x7fcb78197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 0x7fcb78197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 -- 172.21.15.86:0/464110601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78102570 msgr2=0x7fcb78197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78102570 0x7fcb78197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb76d9d640 1 -- 172.21.15.86:0/464110601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7819c1c0 con 0x7fcb78102ef0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.163+0000 7fcb7659c640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 0x7fcb78197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb76d9d640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb68005ab0 tx=0x7fcb680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb677fe640 1 -- 172.21.15.86:0/464110601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6803f030 con 0x7fcb78102ef0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb677fe640 1 -- 172.21.15.86:0/464110601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb68003cb0 con 0x7fcb78102ef0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7819c3f0 con 0x7fcb78102ef0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb677fe640 1 -- 172.21.15.86:0/464110601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68035070 con 0x7fcb78102ef0 2024-09-05T06:48:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.164+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7819c900 con 0x7fcb78102ef0 2024-09-05T06:48:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.166+0000 7fcb677fe640 1 -- 172.21.15.86:0/464110601 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb68042030 con 0x7fcb78102ef0 2024-09-05T06:48:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.166+0000 7fcb7ecee740 1 Processor -- start 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 -- start start 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b089dc0 0x55752b094140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b096a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55752afef2e0 con 0x55752b0894a0 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55752af96ce0 con 0x55752b094680 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55752afeb2a0 con 0x55752b089dc0 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb677fe640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 0x7fcb600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58448/0 (socket says 172.21.15.86:58448) 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 -- 172.21.15.86:0/83298772 learned_addr learned my addr 172.21.15.86:0/83298772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb7659c640 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b089dc0 0x55752b094140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb76d9d640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 0x7fcb600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb677fe640 1 -- 172.21.15.86:0/464110601 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7fcb680f00c0 con 0x7fcb78102ef0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 -- 172.21.15.86:0/83298772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b089dc0 msgr2=0x55752b094140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b089dc0 0x55752b094140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 -- 172.21.15.86:0/83298772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 msgr2=0x55752b096a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b096a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.167+0000 7fcb75d9b640 1 -- 172.21.15.86:0/83298772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55752af7b440 con 0x55752b0894a0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.168+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0138a0 tx=0x7fcb6c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0acb1d811230d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.168+0000 7fcb76d9d640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 0x7fcb600a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb680c4880 tx=0x7fcb680b7cc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.168+0000 7fcb64ff9640 1 -- 172.21.15.86:0/83298772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c019030 con 0x55752b0894a0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.168+0000 7fcb64ff9640 1 -- 172.21.15.86:0/83298772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c00f440 con 0x55752b0894a0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.168+0000 7fcb64ff9640 1 -- 172.21.15.86:0/83298772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c01d620 con 0x55752b0894a0 2024-09-05T06:48:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- 172.21.15.86:0/83298772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 msgr2=0x55752b089880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0138a0 tx=0x7fcb6c007b00 comp rx=0 tx=0).stop 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- 172.21.15.86:0/83298772 shutdown_connections 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b096a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b089dc0 0x55752b094140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/83298772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b089880 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- 172.21.15.86:0/83298772 >> 172.21.15.86:0/83298772 conn(0x55752b085a30 msgr2=0x55752b085e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- 172.21.15.86:0/83298772 shutdown_connections 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- 172.21.15.86:0/83298772 wait complete. 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 -- start start 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 Processor -- start 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b093f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.169+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b091780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55752b097250 con 0x55752b089dc0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55752afaa050 con 0x55752b094680 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55752b0973d0 con 0x55752b0894a0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 10 client.0 _ll_register_callbacks cb 0x55752afea160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58454/0 (socket says 172.21.15.86:58454) 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 -- 172.21.15.86:0/1931570048 learned_addr learned my addr 172.21.15.86:0/1931570048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b093f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b091780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 -- 172.21.15.86:0/1931570048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 msgr2=0x55752b093f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b093f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 -- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 msgr2=0x55752b091780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b091780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 -- 172.21.15.86:0/1931570048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55752ad603c0 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb70012480 tx=0x7fcb70012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d68498cddc8c993 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70004440 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7fcb700035f0 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c00f440 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb700037f0 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 10 client.17241 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.170+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55752b0a2c80 con 0x55752b089dc0 2024-09-05T06:48:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c00f440 con 0x55752b089dc0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcb70019070 con 0x55752b089dc0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 10 client.17241 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 20 client.17241 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b0a41e0 0x55752b0a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 4 client.17241 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55752affc850 con 0x55752b0a41e0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b0a41e0 0x55752b0a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb5dffb640 10 client.17241 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.171+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b0a41e0 0x55752b0a6600 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee65da4308354ef server_cookie=f711f8901d675931 in_seq=0 out_seq=0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1931570048 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55752affc850 con 0x55752b0a41e0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb5dffb640 10 client.17241 handle_command_reply: tid=0 2024-09-05T06:48:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 client.17241 shutdown 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 msgr2=0x55752b091240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb70012480 tx=0x7fcb70012940 comp rx=0 tx=0).stop 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 shutdown_connections 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b0a41e0 0x55752b0a6600 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b094680 0x55752b091780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b089dc0 0x55752b091240 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb5dffb640 0 client.17241 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1931570048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b093f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 >> 172.21.15.86:0/1931570048 conn(0x55752b085a30 msgr2=0x55752b08b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.172+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 shutdown_connections 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.173+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1931570048 wait complete. 2024-09-05T06:48:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.173+0000 7fcb7ecee740 20 client.17241 trim_cache size 0 max 16384 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 Processor -- start 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 -- start start 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b0abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b094680 0x55752b0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781b1190 con 0x55752b0894a0 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781b2cd0 con 0x55752b089dc0 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781b1860 con 0x55752b094680 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58458/0 (socket says 172.21.15.86:58458) 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 -- 172.21.15.86:0/321386515 learned_addr learned my addr 172.21.15.86:0/321386515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 -- 172.21.15.86:0/321386515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b094680 msgr2=0x55752b0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b094680 0x55752b0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 -- 172.21.15.86:0/321386515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 msgr2=0x55752b0abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b0abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 -- 172.21.15.86:0/321386515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55752b0a6970 con 0x55752b0894a0 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.190+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c016040 tx=0x7fcb6c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae323ccc0fe13be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb5ffff640 1 -- 172.21.15.86:0/321386515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c019040 con 0x55752b0894a0 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb5ffff640 1 -- 172.21.15.86:0/321386515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c0049e0 con 0x55752b0894a0 2024-09-05T06:48:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb5ffff640 1 -- 172.21.15.86:0/321386515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00f730 con 0x55752b0894a0 2024-09-05T06:48:49.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 -- 172.21.15.86:0/321386515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 msgr2=0x55752b0a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c016040 tx=0x7fcb6c007b00 comp rx=0 tx=0).stop 2024-09-05T06:48:49.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 -- 172.21.15.86:0/321386515 shutdown_connections 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b094680 0x55752b0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b0abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/321386515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b0894a0 0x55752b0a04b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 -- 172.21.15.86:0/321386515 >> 172.21.15.86:0/321386515 conn(0x55752b085a30 msgr2=0x55752b090330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 -- 172.21.15.86:0/321386515 shutdown_connections 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.191+0000 7fcb7ecee740 1 -- 172.21.15.86:0/321386515 wait complete. 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 -- start start 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 Processor -- start 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b09c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b09cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781b2600 con 0x55752b094680 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781b1f30 con 0x55752b089dc0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781af190 con 0x55752b0894a0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7ecee740 10 client.0 _ll_register_callbacks cb 0x7fcb7000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58468/0 (socket says 172.21.15.86:58468) 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 -- 172.21.15.86:0/1368461138 learned_addr learned my addr 172.21.15.86:0/1368461138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b09cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b09c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 -- 172.21.15.86:0/1368461138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 msgr2=0x55752b09c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b09c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 -- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 msgr2=0x55752b09cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b09cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.192+0000 7fcb75d9b640 1 -- 172.21.15.86:0/1368461138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55752b08e2c0 con 0x55752b094680 2024-09-05T06:48:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb75d9b640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007950 tx=0x7fcb6c0137b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d35cc0af909b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c03c060 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7fcb6c013050 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c0049e0 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb6c043050 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 10 client.17247 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55752b10ea10 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c0049e0 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcb6c005030 con 0x55752b094680 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 10 client.17247 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:48:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 20 client.17247 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b10eee0 0x55752b1112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 4 client.17247 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.193+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55752aff6d60 con 0x55752b10eee0 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b10eee0 0x55752b1112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb5dffb640 10 client.17247 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7659c640 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b10eee0 0x55752b1112d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c099242909c8c4e6 server_cookie=feceac77ea031a31 in_seq=0 out_seq=0 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb5dffb640 1 -- 172.21.15.86:0/1368461138 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55752aff6d60 con 0x55752b10eee0 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb5dffb640 10 client.17247 handle_command_reply: tid=0 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 client.17247 shutdown 2024-09-05T06:48:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 msgr2=0x55752b08da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007950 tx=0x7fcb6c0137b0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 shutdown_connections 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55752b10eee0 0x55752b1112d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55752b094680 0x55752b08da90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55752b089dc0 0x55752b09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 --2- 172.21.15.86:0/1368461138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55752b0894a0 0x55752b09c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb5dffb640 0 client.17247 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.194+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 >> 172.21.15.86:0/1368461138 conn(0x55752b085a30 msgr2=0x55752b084350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 shutdown_connections 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7ecee740 1 -- 172.21.15.86:0/1368461138 wait complete. 2024-09-05T06:48:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7ecee740 20 client.17247 trim_cache size 0 max 16384 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "ffddb7d7-a416-4602-89b1-225a1b37e208": { 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "ffddb7d7-a416-4602-89b1-225a1b37e208", 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:49.202 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 msgr2=0x7fcb600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 0x7fcb600a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb680c4880 tx=0x7fcb680b7cc0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 msgr2=0x7fcb7819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb68005ab0 tx=0x7fcb680042f0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 shutdown_connections 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb600a2cf0 0x7fcb600a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb78102ef0 0x7fcb7819bae0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78102570 0x7fcb78197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 --2- 172.21.15.86:0/464110601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb78100e20 0x7fcb78197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 >> 172.21.15.86:0/464110601 conn(0x7fcb780fc7d0 msgr2=0x7fcb78106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 shutdown_connections 2024-09-05T06:48:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:49.195+0000 7fcb7c915640 1 -- 172.21.15.86:0/464110601 wait complete. 2024-09-05T06:48:49.519 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 1.97 MB/sec execute 249 sec latency 7022.552 ms 2024-09-05T06:48:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:49 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2687547663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:49 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2687547663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:49.790 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 9a3f8548 --release --await' 2024-09-05T06:48:49.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 9a3f8548 --release --await 2024-09-05T06:48:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:49 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2687547663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:48:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:48:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:50 smithi086 ceph-mon[30943]: pgmap v232: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 864 KiB/s wr, 88 op/s 2024-09-05T06:48:50.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145407 1.96 MB/sec execute 250 sec latency 8022.614 ms 2024-09-05T06:48:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 -- 172.21.15.86:0/2121658752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 msgr2=0x7f8dec10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 --2- 172.21.15.86:0/2121658752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec10e600 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009ca0 tx=0x7f8de402a950 comp rx=0 tx=0).stop 2024-09-05T06:48:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 -- 172.21.15.86:0/2121658752 shutdown_connections 2024-09-05T06:48:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 --2- 172.21.15.86:0/2121658752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec10e600 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 --2- 172.21.15.86:0/2121658752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 0x7f8dec10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 --2- 172.21.15.86:0/2121658752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 -- 172.21.15.86:0/2121658752 >> 172.21.15.86:0/2121658752 conn(0x7f8dec0fc7a0 msgr2=0x7f8dec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 -- 172.21.15.86:0/2121658752 shutdown_connections 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.544+0000 7f8df194e640 1 -- 172.21.15.86:0/2121658752 wait complete. 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 Processor -- start 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 -- start start 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 0x7f8dec198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec111e10 con 0x7f8dec10c180 2024-09-05T06:48:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec111c90 con 0x7f8dec102da0 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8df194e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec111f90 con 0x7f8dec1036c0 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58492/0 (socket says 172.21.15.86:58492) 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8deb7fe640 1 -- 172.21.15.86:0/3443779615 learned_addr learned my addr 172.21.15.86:0/3443779615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.545+0000 7f8de3fff640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 0x7f8dec198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deaffd640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 -- 172.21.15.86:0/3443779615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 msgr2=0x7f8dec198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 0x7f8dec198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 -- 172.21.15.86:0/3443779615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 msgr2=0x7f8dec1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 -- 172.21.15.86:0/3443779615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec19d2a0 con 0x7f8dec10c180 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deaffd640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:50.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8deb7fe640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8de4004af0 tx=0x7f8de402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8df094c640 1 -- 172.21.15.86:0/3443779615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4034040 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8df094c640 1 -- 172.21.15.86:0/3443779615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de402fe80 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec19d530 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8df094c640 1 -- 172.21.15.86:0/3443779615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40046a0 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.546+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1a4e70 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.548+0000 7f8df094c640 1 -- 172.21.15.86:0/3443779615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8de4037020 con 0x7f8dec10c180 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 Processor -- start 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 -- start start 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f70115440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f7007a620 con 0x563f70120240 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f70021e50 con 0x563f70115060 2024-09-05T06:48:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df3d27740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f700765e0 con 0x563f70115980 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58500/0 (socket says 172.21.15.86:58500) 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 -- 172.21.15.86:0/1134005824 learned_addr learned my addr 172.21.15.86:0/1134005824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f70115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8deaffd640 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8df094c640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 0x7f8dd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 -- 172.21.15.86:0/1134005824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 msgr2=0x563f7011fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 -- 172.21.15.86:0/1134005824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 msgr2=0x563f70115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f70115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.549+0000 7f8de3fff640 1 -- 172.21.15.86:0/1134005824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f70012d40 con 0x563f70120240 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8deb7fe640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 0x7f8dd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400bf90 tx=0x7f8dd400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaac63560cad7bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8df094c640 1 -- 172.21.15.86:0/3443779615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f8de40f0630 con 0x7f8dec10c180 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8de1ffb640 1 -- 172.21.15.86:0/1134005824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd401e640 con 0x563f70120240 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8de1ffb640 1 -- 172.21.15.86:0/1134005824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd40108e0 con 0x563f70120240 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.550+0000 7f8de1ffb640 1 -- 172.21.15.86:0/1134005824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd401d810 con 0x563f70120240 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- 172.21.15.86:0/1134005824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 msgr2=0x563f70122630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8deb7fe640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 0x7f8dd80a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8de40edb00 tx=0x7f8de408f890 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400bf90 tx=0x7f8dd400b650 comp rx=0 tx=0).stop 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- 172.21.15.86:0/1134005824 shutdown_connections 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70120240 0x563f70122630 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1134005824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f70115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- 172.21.15.86:0/1134005824 >> 172.21.15.86:0/1134005824 conn(0x563f701115f0 msgr2=0x563f701119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- 172.21.15.86:0/1134005824 shutdown_connections 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- 172.21.15.86:0/1134005824 wait complete. 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- start start 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 Processor -- start 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f7011dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f70122e10 con 0x563f70115060 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f70002f10 con 0x563f70120240 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.551+0000 7f8df3d27740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f70122f90 con 0x563f70115980 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8df3d27740 10 client.0 _ll_register_callbacks cb 0x563f70026db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58514/0 (socket says 172.21.15.86:58514) 2024-09-05T06:48:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 -- 172.21.15.86:0/1080943418 learned_addr learned my addr 172.21.15.86:0/1080943418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8deaffd640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 -- 172.21.15.86:0/1080943418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 msgr2=0x563f7011d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 -- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 msgr2=0x563f7011dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f7011dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 -- 172.21.15.86:0/1080943418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6fd89580 con 0x563f70115060 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8de3fff640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400b6c0 tx=0x7f8dd4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b209c66c77cd25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8dcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4010a90 con 0x563f70115060 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f8dd4010c30 con 0x563f70115060 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd40108e0 con 0x563f70115060 2024-09-05T06:48:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8dd4036050 con 0x563f70115060 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8df3d27740 10 client.17259 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.552+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f6fe53960 con 0x563f70115060 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd40108e0 con 0x563f70115060 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8dd4031030 con 0x563f70115060 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 10 client.17259 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 10 client.17259 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 20 client.17259 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7012fe20 0x563f70132210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 4 client.17259 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f70087b90 con 0x563f7012fe20 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8deaffd640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7012fe20 0x563f70132210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8dcaffd640 10 client.17259 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.553+0000 7f8deaffd640 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7012fe20 0x563f70132210 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e339ceda466a99 server_cookie=b753ad4bcfe0c535 in_seq=0 out_seq=0 2024-09-05T06:48:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8dcaffd640 1 -- 172.21.15.86:0/1080943418 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f70087b90 con 0x563f7012fe20 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8dcaffd640 10 client.17259 handle_command_reply: tid=0 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 client.17259 shutdown 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 msgr2=0x563f7011d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400b6c0 tx=0x7f8dd4005760 comp rx=0 tx=0).stop 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 shutdown_connections 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7012fe20 0x563f70132210 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f7011dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115980 0x563f7011d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 --2- 172.21.15.86:0/1080943418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115060 0x563f7011d140 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 >> 172.21.15.86:0/1080943418 conn(0x563f701115f0 msgr2=0x563f701172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.554+0000 7f8dcaffd640 0 client.17259 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.555+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 shutdown_connections 2024-09-05T06:48:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.555+0000 7f8df3d27740 1 -- 172.21.15.86:0/1080943418 wait complete. 2024-09-05T06:48:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.555+0000 7f8df3d27740 20 client.17259 trim_cache size 0 max 16384 2024-09-05T06:48:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.571+0000 7f8df3d27740 1 Processor -- start 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.571+0000 7f8df3d27740 1 -- start start 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f701379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70120240 0x563f7011a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1af0e0 con 0x563f70115980 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1b0c20 con 0x563f70115060 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8df3d27740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1af7b0 con 0x563f70120240 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58528/0 (socket says 172.21.15.86:58528) 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 -- 172.21.15.86:0/2311631418 learned_addr learned my addr 172.21.15.86:0/2311631418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 -- 172.21.15.86:0/2311631418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70120240 msgr2=0x563f7011a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70120240 0x563f7011a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 -- 172.21.15.86:0/2311631418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 msgr2=0x563f701379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f701379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f701379a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 -- 172.21.15.86:0/2311631418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f701157b0 con 0x563f70115980 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc000c00 tx=0x7f8ddc009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f884d06fcd18d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8de17fa640 1 -- 172.21.15.86:0/2311631418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc004220 con 0x563f70115980 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.572+0000 7f8de17fa640 1 -- 172.21.15.86:0/2311631418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ddc0043c0 con 0x563f70115980 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8de17fa640 1 -- 172.21.15.86:0/2311631418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc01eaa0 con 0x563f70115980 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 -- 172.21.15.86:0/2311631418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 msgr2=0x563f70137ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc000c00 tx=0x7f8ddc009cf0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 -- 172.21.15.86:0/2311631418 shutdown_connections 2024-09-05T06:48:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70120240 0x563f7011a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70137ee0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2311631418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70115060 0x563f701379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 -- 172.21.15.86:0/2311631418 >> 172.21.15.86:0/2311631418 conn(0x563f701115f0 msgr2=0x563f7011ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 -- 172.21.15.86:0/2311631418 shutdown_connections 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.573+0000 7f8df3d27740 1 -- 172.21.15.86:0/2311631418 wait complete. 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 -- start start 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 Processor -- start 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115060 0x563f701281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f701251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1b0550 con 0x563f70115980 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1afe80 con 0x563f70120240 2024-09-05T06:48:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ad0e0 con 0x563f70115060 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8df3d27740 10 client.0 _ll_register_callbacks cb 0x7f8dd400bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58532/0 (socket says 172.21.15.86:58532) 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 -- 172.21.15.86:0/2324421247 learned_addr learned my addr 172.21.15.86:0/2324421247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f701251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115060 0x563f701281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 -- 172.21.15.86:0/2324421247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115060 msgr2=0x563f701281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115060 0x563f701281e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 -- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 msgr2=0x563f701251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.574+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f701251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8deaffd640 1 -- 172.21.15.86:0/2324421247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f701256f0 con 0x563f70115980 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f701251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:48:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8deaffd640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009a90 tx=0x7f8ddc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cad5aba9f8a5a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc01e920 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ddc0043c0 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9605+0+0 (secure 0 0 0) 0x7f8ddc01eac0 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ddc01ecc0 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 10 client.17265 fetch_fsmap learned FSMap version 32 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f7019b170 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ddc0043c0 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8ddc030020 con 0x563f70115980 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 10 client.17265 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 10 client.17265 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.575+0000 7f8df3d27740 20 client.17265 populate_metadata read hostname 'smithi086' 2024-09-05T06:48:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7019b7f0 0x563f7019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:48:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8df3d27740 4 client.17265 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9a3f8548", "release": true, "await": true}] 2024-09-05T06:48:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9a3f8548", "release": true, "await": true}) -- 0x563f700820a0 con 0x563f7019b7f0 2024-09-05T06:48:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7019b7f0 0x563f7019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:48:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8de3fff640 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7019b7f0 0x563f7019dc10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b5bdbf24a33973 server_cookie=d157bfdc6a8ded9 in_seq=0 out_seq=0 2024-09-05T06:48:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.576+0000 7f8dcaffd640 10 client.17265 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8dcaffd640 1 -- 172.21.15.86:0/2324421247 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f700820a0 con 0x563f7019b7f0 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8dcaffd640 10 client.17265 handle_command_reply: tid=0 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 client.17265 shutdown 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 msgr2=0x563f70128720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009a90 tx=0x7f8ddc009ac0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 shutdown_connections 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563f7019b7f0 0x563f7019dc10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f70120240 0x563f701251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f70115980 0x563f70128720 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 --2- 172.21.15.86:0/2324421247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f70115060 0x563f701281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 >> 172.21.15.86:0/2324421247 conn(0x563f701115f0 msgr2=0x563f70116ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8dcaffd640 0 client.17265 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 shutdown_connections 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 1 -- 172.21.15.86:0/2324421247 wait complete. 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.582+0000 7f8df3d27740 20 client.17265 trim_cache size 0 max 16384 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:48:50.588 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 24, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "9a3f8548": { 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 24, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:48:50.589 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:48:50.590 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 msgr2=0x7f8dd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 0x7f8dd80a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8de40edb00 tx=0x7f8de408f890 comp rx=0 tx=0).stop 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 msgr2=0x7f8dec19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8de4004af0 tx=0x7f8de402f5d0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 shutdown_connections 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8dd80a2cf0 0x7f8dd80a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec10c180 0x7f8dec19cbc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec1036c0 0x7f8dec198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 --2- 172.21.15.86:0/3443779615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dec102da0 0x7f8dec1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:48:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 >> 172.21.15.86:0/3443779615 conn(0x7f8dec0fc7a0 msgr2=0x7f8dec0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:48:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 shutdown_connections 2024-09-05T06:48:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:48:50.583+0000 7f8df194e640 1 -- 172.21.15.86:0/3443779615 wait complete. 2024-09-05T06:48:50.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:50 smithi115 ceph-mon[34791]: pgmap v232: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 864 KiB/s wr, 88 op/s 2024-09-05T06:48:50.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:50 smithi096 ceph-mon[34468]: pgmap v232: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 864 KiB/s wr, 88 op/s 2024-09-05T06:48:51.038 INFO:tasks.cephfs.filesystem:scrub status for tag:ffddb7d7-a416-4602-89b1-225a1b37e208 - {'path': '/', 'tag': 'ffddb7d7-a416-4602-89b1-225a1b37e208', 'options': 'recursive,force'} 2024-09-05T06:48:51.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145772 1.96 MB/sec execute 251 sec latency 8088.419 ms 2024-09-05T06:48:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:51 smithi096 ceph-mon[34468]: pgmap v233: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1014 KiB/s wr, 105 op/s 2024-09-05T06:48:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:51 smithi115 ceph-mon[34791]: pgmap v233: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1014 KiB/s wr, 105 op/s 2024-09-05T06:48:51.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:51 smithi086 ceph-mon[30943]: pgmap v233: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1014 KiB/s wr, 105 op/s 2024-09-05T06:48:52.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 146076 1.95 MB/sec execute 252 sec latency 13.444 ms 2024-09-05T06:48:53.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 146526 1.95 MB/sec execute 253 sec latency 116.953 ms 2024-09-05T06:48:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:53 smithi096 ceph-mon[34468]: pgmap v234: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:53 smithi115 ceph-mon[34791]: pgmap v234: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:53 smithi086 ceph-mon[30943]: pgmap v234: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:54.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 147020 1.96 MB/sec execute 254 sec latency 45.501 ms 2024-09-05T06:48:54.609 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9a3f8548', total seconds elapsed: 17.3 2024-09-05T06:48:55.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 147614 1.95 MB/sec execute 255 sec latency 20.423 ms 2024-09-05T06:48:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:55 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:55 smithi096 ceph-mon[34468]: pgmap v235: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:55 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:55 smithi115 ceph-mon[34791]: pgmap v235: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:55.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:55 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:55.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:55 smithi086 ceph-mon[30943]: pgmap v235: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 423 KiB/s wr, 41 op/s 2024-09-05T06:48:56.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 148219 1.96 MB/sec execute 256 sec latency 13.051 ms 2024-09-05T06:48:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:48:56.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:56.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:48:56.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:56.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:48:57.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 148798 1.97 MB/sec execute 257 sec latency 14.471 ms 2024-09-05T06:48:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:57 smithi096 ceph-mon[34468]: pgmap v236: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 984 KiB/s wr, 101 op/s 2024-09-05T06:48:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:57 smithi115 ceph-mon[34791]: pgmap v236: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 984 KiB/s wr, 101 op/s 2024-09-05T06:48:57.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:57 smithi086 ceph-mon[30943]: pgmap v236: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 984 KiB/s wr, 101 op/s 2024-09-05T06:48:58.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 149295 1.98 MB/sec execute 258 sec latency 14.049 ms 2024-09-05T06:48:59.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 149583 1.97 MB/sec execute 259 sec latency 15.592 ms 2024-09-05T06:48:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:48:59 smithi096 ceph-mon[34468]: pgmap v237: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-09-05T06:48:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:48:59 smithi115 ceph-mon[34791]: pgmap v237: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-09-05T06:48:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:48:59 smithi086 ceph-mon[30943]: pgmap v237: 129 pgs: 129 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-09-05T06:49:00.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 150093 1.97 MB/sec execute 260 sec latency 17.274 ms 2024-09-05T06:49:01.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 150613 1.98 MB/sec execute 261 sec latency 20.840 ms 2024-09-05T06:49:01.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:01 smithi096 ceph-mon[34468]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T06:49:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:01 smithi096 ceph-mon[34468]: pgmap v239: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-05T06:49:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:01 smithi115 ceph-mon[34791]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T06:49:01.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:01 smithi115 ceph-mon[34791]: pgmap v239: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-05T06:49:01.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:01 smithi086 ceph-mon[30943]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T06:49:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:01 smithi086 ceph-mon[30943]: pgmap v239: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-05T06:49:02.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 151228 1.97 MB/sec execute 262 sec latency 23.800 ms 2024-09-05T06:49:03.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:02 smithi086 ceph-mon[30943]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T06:49:03.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:02 smithi096 ceph-mon[34468]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T06:49:03.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:02 smithi115 ceph-mon[34791]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T06:49:03.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 151879 1.98 MB/sec execute 263 sec latency 13.381 ms 2024-09-05T06:49:04.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:03 smithi086 ceph-mon[30943]: pgmap v241: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-05T06:49:04.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:03 smithi096 ceph-mon[34468]: pgmap v241: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-05T06:49:04.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:03 smithi115 ceph-mon[34791]: pgmap v241: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-05T06:49:04.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 152391 1.99 MB/sec execute 264 sec latency 13.360 ms 2024-09-05T06:49:05.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 152868 2.00 MB/sec execute 265 sec latency 13.408 ms 2024-09-05T06:49:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:05 smithi096 ceph-mon[34468]: pgmap v242: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:49:05.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:05 smithi115 ceph-mon[34791]: pgmap v242: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:49:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:05 smithi086 ceph-mon[30943]: pgmap v242: 129 pgs: 129 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-05T06:49:06.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 153151 1.99 MB/sec execute 266 sec latency 15.279 ms 2024-09-05T06:49:07.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 153693 1.99 MB/sec execute 267 sec latency 15.127 ms 2024-09-05T06:49:07.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:07 smithi086 ceph-mon[30943]: pgmap v243: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-09-05T06:49:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:07 smithi096 ceph-mon[34468]: pgmap v243: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-09-05T06:49:07.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:07 smithi115 ceph-mon[34791]: pgmap v243: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-09-05T06:49:08.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 154236 2.00 MB/sec execute 268 sec latency 20.515 ms 2024-09-05T06:49:09.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 154872 2.00 MB/sec execute 269 sec latency 22.231 ms 2024-09-05T06:49:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:09 smithi096 ceph-mon[34468]: pgmap v244: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:09 smithi115 ceph-mon[34791]: pgmap v244: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:09.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:09 smithi086 ceph-mon[30943]: pgmap v244: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:10.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 155471 2.01 MB/sec execute 270 sec latency 14.741 ms 2024-09-05T06:49:11.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 156036 2.02 MB/sec execute 271 sec latency 50.958 ms 2024-09-05T06:49:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:11 smithi096 ceph-mon[34468]: pgmap v245: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-05T06:49:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:11 smithi115 ceph-mon[34791]: pgmap v245: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-05T06:49:11.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:11 smithi086 ceph-mon[30943]: pgmap v245: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-09-05T06:49:12.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 156451 2.02 MB/sec execute 272 sec latency 13.447 ms 2024-09-05T06:49:13.182 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.2 2024-09-05T06:49:13.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:49:13.456 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:13.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 156771 2.01 MB/sec execute 273 sec latency 17.386 ms 2024-09-05T06:49:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:13 smithi096 ceph-mon[34468]: pgmap v246: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:49:13.884 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:13 smithi086 ceph-mon[30943]: pgmap v246: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:49:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:13 smithi115 ceph-mon[34791]: pgmap v246: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.977+0000 7fc38274b640 1 -- 172.21.15.86:0/2317547597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 msgr2=0x7fc37c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.977+0000 7fc38274b640 1 --2- 172.21.15.86:0/2317547597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c107b90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc370009930 tx=0x7fc37002a420 comp rx=0 tx=0).stop 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 -- 172.21.15.86:0/2317547597 shutdown_connections 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 --2- 172.21.15.86:0/2317547597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 0x7fc37c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 --2- 172.21.15.86:0/2317547597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c107b90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 --2- 172.21.15.86:0/2317547597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 0x7fc37c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 -- 172.21.15.86:0/2317547597 >> 172.21.15.86:0/2317547597 conn(0x7fc37c0fc7d0 msgr2=0x7fc37c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 -- 172.21.15.86:0/2317547597 shutdown_connections 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 -- 172.21.15.86:0/2317547597 wait complete. 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 Processor -- start 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.978+0000 7fc38274b640 1 -- start start 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 0x7fc37c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 0x7fc37c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c111f50 con 0x7fc37c1036e0 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c111dd0 con 0x7fc37c102d30 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc38274b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1120d0 con 0x7fc37c1082a0 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34324/0 (socket says 172.21.15.86:34324) 2024-09-05T06:49:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 -- 172.21.15.86:0/477384286 learned_addr learned my addr 172.21.15.86:0/477384286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37bfff640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 0x7fc37c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc380ce8640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 0x7fc37c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 -- 172.21.15.86:0/477384286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 msgr2=0x7fc37c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 0x7fc37c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 -- 172.21.15.86:0/477384286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 msgr2=0x7fc37c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 0x7fc37c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 -- 172.21.15.86:0/477384286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c19d4c0 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.979+0000 7fc37b7fe640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc370031440 tx=0x7fc370004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.980+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37003f030 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.980+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c19d750 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.980+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc370003aa0 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.980+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370035070 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.980+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a5090 con 0x7fc37c1036e0 2024-09-05T06:49:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.981+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c06a850 con 0x7fc37c1036e0 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.982+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc370042030 con 0x7fc37c1036e0 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.983+0000 7fc3797fa640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 0x7fc3640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.983+0000 7fc37bfff640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 0x7fc3640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.983+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fc3700f02a0 con 0x7fc37c1036e0 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.983+0000 7fc37bfff640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 0x7fc3640a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc36c005d00 tx=0x7fc36c00b410 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:13.987+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3700bc430 con 0x7fc37c1036e0 2024-09-05T06:49:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.225+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc37c10a2b0 con 0x7fc37c1036e0 2024-09-05T06:49:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.226+0000 7fc3797fa640 1 -- 172.21.15.86:0/477384286 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7fc3700c0730 con 0x7fc37c1036e0 2024-09-05T06:49:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:49:14.226 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:49:14.227 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:49:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 msgr2=0x7fc3640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 0x7fc3640a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc36c005d00 tx=0x7fc36c00b410 comp rx=0 tx=0).stop 2024-09-05T06:49:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 msgr2=0x7fc37c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc370031440 tx=0x7fc370004270 comp rx=0 tx=0).stop 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 shutdown_connections 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc3640a2cf0 0x7fc3640a51b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c1082a0 0x7fc37c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c1036e0 0x7fc37c198a50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 --2- 172.21.15.86:0/477384286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c102d30 0x7fc37c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 >> 172.21.15.86:0/477384286 conn(0x7fc37c0fc7d0 msgr2=0x7fc37c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 shutdown_connections 2024-09-05T06:49:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:14.228+0000 7fc38274b640 1 -- 172.21.15.86:0/477384286 wait complete. 2024-09-05T06:49:14.520 INFO:tasks.workunit.client.0.smithi086.stdout: 1 157275 2.01 MB/sec execute 274 sec latency 29.030 ms 2024-09-05T06:49:14.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:14 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/477384286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:14.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:14 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/477384286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:14.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:14 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/477384286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:15.027 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 133.2 --await-for 45.0' 2024-09-05T06:49:15.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 133.2 --await-for 45.0 2024-09-05T06:49:15.321 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:15.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 157806 2.02 MB/sec execute 275 sec latency 18.838 ms 2024-09-05T06:49:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:15 smithi086 ceph-mon[30943]: pgmap v247: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-05T06:49:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:15 smithi096 ceph-mon[34468]: pgmap v247: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 -- 172.21.15.86:0/1256925873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 msgr2=0x7fe44c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 --2- 172.21.15.86:0/1256925873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c100ee0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009930 tx=0x7fe43c02a420 comp rx=0 tx=0).stop 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 -- 172.21.15.86:0/1256925873 shutdown_connections 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 --2- 172.21.15.86:0/1256925873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c101420 0x7fe44c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 --2- 172.21.15.86:0/1256925873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c100ee0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 --2- 172.21.15.86:0/1256925873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c106a80 0x7fe44c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 -- 172.21.15.86:0/1256925873 >> 172.21.15.86:0/1256925873 conn(0x7fe44c0fc7a0 msgr2=0x7fe44c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 -- 172.21.15.86:0/1256925873 shutdown_connections 2024-09-05T06:49:15.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.847+0000 7fe453da4640 1 -- 172.21.15.86:0/1256925873 wait complete. 2024-09-05T06:49:15.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 Processor -- start 2024-09-05T06:49:15.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 -- start start 2024-09-05T06:49:15.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c101420 0x7fe44c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 0x7fe44c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c110d30 con 0x7fe44c100a80 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c110bb0 con 0x7fe44c101420 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe453da4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c110eb0 con 0x7fe44c106a80 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe451b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe451b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34350/0 (socket says 172.21.15.86:34350) 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe451b40640 1 -- 172.21.15.86:0/4196615042 learned_addr learned my addr 172.21.15.86:0/4196615042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe452341640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 0x7fe44c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.848+0000 7fe45133f640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c101420 0x7fe44c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 -- 172.21.15.86:0/4196615042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 msgr2=0x7fe44c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 0x7fe44c1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 -- 172.21.15.86:0/4196615042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c101420 msgr2=0x7fe44c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c101420 0x7fe44c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 -- 172.21.15.86:0/4196615042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a47c0 con 0x7fe44c100a80 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe452341640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 0x7fe44c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe451b40640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe448009f50 tx=0x7fe44800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe442ffd640 1 -- 172.21.15.86:0/4196615042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448024070 con 0x7fe44c100a80 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe442ffd640 1 -- 172.21.15.86:0/4196615042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe44801f440 con 0x7fe44c100a80 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c196ff0 con 0x7fe44c100a80 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe442ffd640 1 -- 172.21.15.86:0/4196615042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800ce60 con 0x7fe44c100a80 2024-09-05T06:49:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.849+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1974b0 con 0x7fe44c100a80 2024-09-05T06:49:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.851+0000 7fe442ffd640 1 -- 172.21.15.86:0/4196615042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe448027070 con 0x7fe44c100a80 2024-09-05T06:49:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 Processor -- start 2024-09-05T06:49:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 -- start start 2024-09-05T06:49:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 0x55584d08bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d096400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe442ffd640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 0x7fe4380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d098d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55584cff12a0 con 0x55584d096940 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55584cf98b10 con 0x55584d08b760 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45617d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55584cfed260 con 0x55584d08c080 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe45133f640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 0x7fe4380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe452341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d098d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe451b40640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d096400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe452341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d098d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34358/0 (socket says 172.21.15.86:34358) 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.852+0000 7fe452341640 1 -- 172.21.15.86:0/1855546420 learned_addr learned my addr 172.21.15.86:0/1855546420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 0x55584d08bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe442ffd640 1 -- 172.21.15.86:0/4196615042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fe4480d7c30 con 0x7fe44c100a80 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 -- 172.21.15.86:0/1855546420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 msgr2=0x55584d096400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d096400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 -- 172.21.15.86:0/1855546420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 msgr2=0x55584d098d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d098d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 -- 172.21.15.86:0/1855546420 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584cd62b90 con 0x55584d08b760 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe451b40640 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d096400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe452341640 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 0x55584d08bb40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098b0 tx=0x7fe444009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a6789c93d84e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe45133f640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 0x7fe4380a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009900 tx=0x7fe43c000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe437fff640 1 -- 172.21.15.86:0/1855546420 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444004b20 con 0x55584d08b760 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe437fff640 1 -- 172.21.15.86:0/1855546420 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe444004cc0 con 0x55584d08b760 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.853+0000 7fe437fff640 1 -- 172.21.15.86:0/1855546420 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444005840 con 0x55584d08b760 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 -- 172.21.15.86:0/1855546420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 msgr2=0x55584d08bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 0x55584d08bb40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098b0 tx=0x7fe444009600 comp rx=0 tx=0).stop 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 -- 172.21.15.86:0/1855546420 shutdown_connections 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d098d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d096400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 --2- 172.21.15.86:0/1855546420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08b760 0x55584d08bb40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 -- 172.21.15.86:0/1855546420 >> 172.21.15.86:0/1855546420 conn(0x55584d087cf0 msgr2=0x55584d0880d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 -- 172.21.15.86:0/1855546420 shutdown_connections 2024-09-05T06:49:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.854+0000 7fe45617d740 1 -- 172.21.15.86:0/1855546420 wait complete. 2024-09-05T06:49:15.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 -- start start 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 Processor -- start 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d094250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55584d099510 con 0x55584d08c080 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55584cf796e0 con 0x55584d096940 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55584d099690 con 0x55584d08b760 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45617d740 10 client.0 _ll_register_callbacks cb 0x55584cf9da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45133f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45133f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34364/0 (socket says 172.21.15.86:34364) 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.855+0000 7fe45133f640 1 -- 172.21.15.86:0/2620535433 learned_addr learned my addr 172.21.15.86:0/2620535433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe451b40640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe452341640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d094250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 -- 172.21.15.86:0/2620535433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 msgr2=0x55584d0937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 -- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 msgr2=0x55584d094250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d094250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 -- 172.21.15.86:0/2620535433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584d0a29e0 con 0x55584d08c080 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe452341640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d094250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe451b40640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45133f640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009930 tx=0x7fe43c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ee00d8393cc531 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c04b070 con 0x55584d08c080 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fe43c003800 con 0x55584d08c080 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe444004cc0 con 0x55584d08c080 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe43c057050 con 0x55584d08c080 2024-09-05T06:49:15.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45617d740 10 client.17283 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.856+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55584d0a5350 con 0x55584d08c080 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe444004cc0 con 0x55584d08c080 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe43c047a00 con 0x55584d08c080 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 10 client.17283 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 10 client.17283 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 20 client.17283 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d0a6820 0x55584d0a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 4 client.17283 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55584cffe9d0 con 0x55584d0a6820 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe451b40640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d0a6820 0x55584d0a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe4357fa640 10 client.17283 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.857+0000 7fe451b40640 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d0a6820 0x55584d0a8c60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1ea59d6f9c2895 server_cookie=bb2207d101d9028e in_seq=0 out_seq=0 2024-09-05T06:49:15.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe4357fa640 1 -- 172.21.15.86:0/2620535433 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55584cffe9d0 con 0x55584d0a6820 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe4357fa640 10 client.17283 handle_command_reply: tid=0 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 client.17283 shutdown 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 msgr2=0x55584d093d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009930 tx=0x7fe43c009260 comp rx=0 tx=0).stop 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 shutdown_connections 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d0a6820 0x55584d0a8c60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d094250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08c080 0x55584d093d10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe4357fa640 0 client.17283 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 --2- 172.21.15.86:0/2620535433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.858+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 >> 172.21.15.86:0/2620535433 conn(0x55584d087cf0 msgr2=0x55584d08d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.859+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 shutdown_connections 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.859+0000 7fe45617d740 1 -- 172.21.15.86:0/2620535433 wait complete. 2024-09-05T06:49:15.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.859+0000 7fe45617d740 20 client.17283 trim_cache size 0 max 16384 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.877+0000 7fe45617d740 1 Processor -- start 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.877+0000 7fe45617d740 1 -- start start 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 0x55584d0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1acf60 con 0x55584d096940 2024-09-05T06:49:15.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1aeaa0 con 0x55584d08c080 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45617d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1ad630 con 0x55584d08b760 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe452341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe451b40640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe452341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34368/0 (socket says 172.21.15.86:34368) 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe452341640 1 -- 172.21.15.86:0/1951310893 learned_addr learned my addr 172.21.15.86:0/1951310893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 0x55584d0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 -- 172.21.15.86:0/1951310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 msgr2=0x55584d0940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 -- 172.21.15.86:0/1951310893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 msgr2=0x55584d0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 -- 172.21.15.86:0/1951310893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584d08beb0 con 0x55584d08c080 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe452341640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe451b40640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.878+0000 7fe45133f640 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 0x55584d0ae530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe43c004770 tx=0x7fe43c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40bf53291edd4ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.879+0000 7fe4377fe640 1 -- 172.21.15.86:0/1951310893 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c060070 con 0x55584d08c080 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.879+0000 7fe4377fe640 1 -- 172.21.15.86:0/1951310893 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43c03bd30 con 0x55584d08c080 2024-09-05T06:49:15.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.879+0000 7fe4377fe640 1 -- 172.21.15.86:0/1951310893 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c04b070 con 0x55584d08c080 2024-09-05T06:49:15.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- 172.21.15.86:0/1951310893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 msgr2=0x55584d0ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 0x55584d0ae530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe43c004770 tx=0x7fe43c005fd0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- 172.21.15.86:0/1951310893 shutdown_connections 2024-09-05T06:49:15.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d096940 0x55584d0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d08c080 0x55584d0ae530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 --2- 172.21.15.86:0/1951310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08b760 0x55584d0940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- 172.21.15.86:0/1951310893 >> 172.21.15.86:0/1951310893 conn(0x55584d087cf0 msgr2=0x55584d092790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- 172.21.15.86:0/1951310893 shutdown_connections 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- 172.21.15.86:0/1951310893 wait complete. 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 -- start start 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.880+0000 7fe45617d740 1 Processor -- start 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d09acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1ae3d0 con 0x55584d08b760 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1add00 con 0x55584d096940 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c1aaf60 con 0x55584d08c080 2024-09-05T06:49:15.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45617d740 10 client.0 _ll_register_callbacks cb 0x7fe43c031860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34384/0 (socket says 172.21.15.86:34384) 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 -- 172.21.15.86:0/1568411369 learned_addr learned my addr 172.21.15.86:0/1568411369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45133f640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d09acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe452341640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 -- 172.21.15.86:0/1568411369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 msgr2=0x55584d09acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d09acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 -- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 msgr2=0x55584d09b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d09b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 -- 172.21.15.86:0/1568411369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584d110660 con 0x55584d08b760 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe452341640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d09b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe45133f640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d09acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:15.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.881+0000 7fe451b40640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe4480d5540 tx=0x7fe4480b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c792f7596511a993 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44802e040 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fe4480a0540 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43c03bd30 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe4480a81d0 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 10 client.17289 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55584d111940 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43c03bd30 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe4480acce0 con 0x55584d08b760 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 10 client.17289 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 10 client.17289 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 20 client.17289 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:15.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d1120d0 0x55584d1144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:15.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 4 client.17289 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.2, "await_for": 45.0}] 2024-09-05T06:49:15.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.882+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.2, "await_for": 45.0}) -- 0x55584cff8d20 con 0x55584d1120d0 2024-09-05T06:49:15.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.883+0000 7fe45133f640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d1120d0 0x55584d1144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:15.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.883+0000 7fe4357fa640 10 client.17289 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:15.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.883+0000 7fe45133f640 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d1120d0 0x55584d1144c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77ae966eef9f62b server_cookie=43c4ccc5cbad8db3 in_seq=0 out_seq=0 2024-09-05T06:49:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:15 smithi115 ceph-mon[34791]: pgmap v247: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe4357fa640 1 -- 172.21.15.86:0/1568411369 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55584cff8d20 con 0x55584d1120d0 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe4357fa640 10 client.17289 handle_command_reply: tid=0 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 client.17289 shutdown 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 msgr2=0x55584d09a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe4480d5540 tx=0x7fe4480b3040 comp rx=0 tx=0).stop 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 shutdown_connections 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55584d1120d0 0x55584d1144c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe4357fa640 0 client.17289 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55584d096940 0x55584d09b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55584d08c080 0x55584d09acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 --2- 172.21.15.86:0/1568411369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584d08b760 0x55584d09a790 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.943+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 >> 172.21.15.86:0/1568411369 conn(0x55584d087cf0 msgr2=0x55584d086650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 shutdown_connections 2024-09-05T06:49:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe45617d740 1 -- 172.21.15.86:0/1568411369 wait complete. 2024-09-05T06:49:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe45617d740 20 client.17289 trim_cache size 0 max 16384 2024-09-05T06:49:15.945 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 26, 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "6e6ae99a": { 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 26, 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:49:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 133.2, 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:15.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 msgr2=0x7fe4380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 0x7fe4380a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009900 tx=0x7fe43c000bc0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 msgr2=0x7fe44c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe448009f50 tx=0x7fe44800ee50 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 shutdown_connections 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4380a2cf0 0x7fe4380a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe44c106a80 0x7fe44c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe44c101420 0x7fe44c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 --2- 172.21.15.86:0/4196615042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe44c100a80 0x7fe44c199200 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.944+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 >> 172.21.15.86:0/4196615042 conn(0x7fe44c0fc7a0 msgr2=0x7fe44c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.945+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 shutdown_connections 2024-09-05T06:49:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:15.945+0000 7fe453da4640 1 -- 172.21.15.86:0/4196615042 wait complete. 2024-09-05T06:49:16.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 2.01 MB/sec execute 276 sec latency 578.053 ms 2024-09-05T06:49:16.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6e6ae99a', quiesce took 3.6 seconds. Will release after: 9.6 2024-09-05T06:49:17.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 2.00 MB/sec execute 277 sec latency 1578.083 ms 2024-09-05T06:49:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:17 smithi096 ceph-mon[34468]: pgmap v248: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-05T06:49:17.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:17 smithi115 ceph-mon[34791]: pgmap v248: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-05T06:49:17.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:17 smithi086 ceph-mon[30943]: pgmap v248: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-05T06:49:18.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.99 MB/sec execute 278 sec latency 2578.203 ms 2024-09-05T06:49:19.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.99 MB/sec execute 279 sec latency 3578.280 ms 2024-09-05T06:49:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:19 smithi096 ceph-mon[34468]: pgmap v249: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:49:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:19 smithi115 ceph-mon[34791]: pgmap v249: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:49:19.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:19 smithi086 ceph-mon[30943]: pgmap v249: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-09-05T06:49:20.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.98 MB/sec execute 280 sec latency 4578.408 ms 2024-09-05T06:49:21.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:49:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:21.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.97 MB/sec execute 281 sec latency 5578.468 ms 2024-09-05T06:49:21.689 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:21 smithi086 ceph-mon[30943]: pgmap v250: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:49:21.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 -- 172.21.15.86:0/2825947018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 msgr2=0x7fc860100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 --2- 172.21.15.86:0/2825947018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860100ee0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc854009930 tx=0x7fc85402a420 comp rx=0 tx=0).stop 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 -- 172.21.15.86:0/2825947018 shutdown_connections 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 --2- 172.21.15.86:0/2825947018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860101420 0x7fc86010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 --2- 172.21.15.86:0/2825947018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860100ee0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 --2- 172.21.15.86:0/2825947018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860106a80 0x7fc860106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 -- 172.21.15.86:0/2825947018 >> 172.21.15.86:0/2825947018 conn(0x7fc8600fc7a0 msgr2=0x7fc8600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 -- 172.21.15.86:0/2825947018 shutdown_connections 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 -- 172.21.15.86:0/2825947018 wait complete. 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.770+0000 7fc8664a9640 1 Processor -- start 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 -- start start 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860101420 0x7fc860197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860106a80 0x7fc860197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc860110d30 con 0x7fc860100a80 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc860110bb0 con 0x7fc860106a80 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc8664a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc860110eb0 con 0x7fc860101420 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34390/0 (socket says 172.21.15.86:34390) 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 -- 172.21.15.86:0/4240677830 learned_addr learned my addr 172.21.15.86:0/4240677830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc864a46640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860106a80 0x7fc860197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860101420 0x7fc860197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 -- 172.21.15.86:0/4240677830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860101420 msgr2=0x7fc860197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860101420 0x7fc860197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 -- 172.21.15.86:0/4240677830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860106a80 msgr2=0x7fc860197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860106a80 0x7fc860197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 -- 172.21.15.86:0/4240677830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8601a68c0 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.771+0000 7fc85ffff640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d950 tx=0x7fc85000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.772+0000 7fc85d7fa640 1 -- 172.21.15.86:0/4240677830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850017030 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.772+0000 7fc85d7fa640 1 -- 172.21.15.86:0/4240677830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc850004020 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.772+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8601a6bb0 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.772+0000 7fc85d7fa640 1 -- 172.21.15.86:0/4240677830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850005570 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.772+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a70c0 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.774+0000 7fc85d7fa640 1 -- 172.21.15.86:0/4240677830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8500041d0 con 0x7fc860100a80 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.774+0000 7fc868882740 1 Processor -- start 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.774+0000 7fc868882740 1 -- start start 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f047494a0 0x560f04749880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04749dc0 0x560f04754140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f046af2e0 con 0x560f04754680 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f04656ce0 con 0x560f04749dc0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc868882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f046ab2a0 con 0x560f047494a0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85d7fa640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 0x7fc8480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34394/0 (socket says 172.21.15.86:34394) 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 -- 172.21.15.86:0/4156746708 learned_addr learned my addr 172.21.15.86:0/4156746708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc864a46640 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f047494a0 0x560f04749880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85ffff640 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04749dc0 0x560f04754140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc864a46640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 0x7fc8480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85d7fa640 1 -- 172.21.15.86:0/4240677830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fc8500cf130 con 0x7fc860100a80 2024-09-05T06:49:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 -- 172.21.15.86:0/4156746708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f047494a0 msgr2=0x560f04749880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f047494a0 0x560f04749880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 -- 172.21.15.86:0/4156746708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04749dc0 msgr2=0x560f04754140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04749dc0 0x560f04754140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 -- 172.21.15.86:0/4156746708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0463b440 con 0x560f04754680 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc864a46640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 0x7fc8480a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc858009b80 tx=0x7fc858009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.775+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc85402a940 tx=0x7fc854031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c77b9337e95929 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc8467fc640 1 -- 172.21.15.86:0/4156746708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854040070 con 0x560f04754680 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc8467fc640 1 -- 172.21.15.86:0/4156746708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc854031e50 con 0x560f04754680 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc8467fc640 1 -- 172.21.15.86:0/4156746708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854035040 con 0x560f04754680 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 -- 172.21.15.86:0/4156746708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 msgr2=0x560f04756a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc85402a940 tx=0x7fc854031ba0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 -- 172.21.15.86:0/4156746708 shutdown_connections 2024-09-05T06:49:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f04756a70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04749dc0 0x560f04754140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 --2- 172.21.15.86:0/4156746708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f047494a0 0x560f04749880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 -- 172.21.15.86:0/4156746708 >> 172.21.15.86:0/4156746708 conn(0x560f04745a30 msgr2=0x560f04745e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 -- 172.21.15.86:0/4156746708 shutdown_connections 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.776+0000 7fc868882740 1 -- 172.21.15.86:0/4156746708 wait complete. 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 -- start start 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 Processor -- start 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f04753f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04754680 0x560f04751750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f04757250 con 0x560f04749dc0 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f0466a050 con 0x560f047494a0 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f047573d0 con 0x560f04754680 2024-09-05T06:49:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc868882740 10 client.0 _ll_register_callbacks cb 0x560f046aa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34398/0 (socket says 172.21.15.86:34398) 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc85f7fe640 1 -- 172.21.15.86:0/822162483 learned_addr learned my addr 172.21.15.86:0/822162483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc85ffff640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f04753f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.777+0000 7fc864a46640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04754680 0x560f04751750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 -- 172.21.15.86:0/822162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04754680 msgr2=0x560f04751750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04754680 0x560f04751750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 -- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 msgr2=0x560f04753f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f04753f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 -- 172.21.15.86:0/822162483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f044203c0 con 0x560f04749dc0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc854031440 tx=0x7fc854005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7696ddaf7b0ea51e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85404c020 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fc854004770 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc854031e50 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc854057050 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc868882740 10 client.17301 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f04762bf0 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.778+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc854031e50 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc85403c5d0 con 0x560f04749dc0 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 10 client.17301 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 20 client.17301 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f04763f40 0x560f04766330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 4 client.17301 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f046bc850 con 0x560f04763f40 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc85ffff640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f04763f40 0x560f04766330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc83f7fe640 10 client.17301 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.779+0000 7fc85ffff640 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f04763f40 0x560f04766330 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf14b34653bbb712 server_cookie=5138f78f2127163f in_seq=0 out_seq=0 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc83f7fe640 1 -- 172.21.15.86:0/822162483 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f046bc850 con 0x560f04763f40 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc83f7fe640 10 client.17301 handle_command_reply: tid=0 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 client.17301 shutdown 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 msgr2=0x560f04751210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc854031440 tx=0x7fc854005510 comp rx=0 tx=0).stop 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 shutdown_connections 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f04763f40 0x560f04766330 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04754680 0x560f04751750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04749dc0 0x560f04751210 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc83f7fe640 0 client.17301 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 --2- 172.21.15.86:0/822162483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f04753f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 >> 172.21.15.86:0/822162483 conn(0x560f04745a30 msgr2=0x560f0474b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 shutdown_connections 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 1 -- 172.21.15.86:0/822162483 wait complete. 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.780+0000 7fc868882740 20 client.17301 trim_cache size 0 max 16384 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 Processor -- start 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 -- start start 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f047604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04744a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b13a0 con 0x560f04754680 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b2ee0 con 0x560f047494a0 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc868882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b1a70 con 0x560f04749dc0 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.797+0000 7fc864a46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34408/0 (socket says 172.21.15.86:34408) 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 -- 172.21.15.86:0/3333377773 learned_addr learned my addr 172.21.15.86:0/3333377773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc85ffff640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f047604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04744a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 -- 172.21.15.86:0/3333377773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 msgr2=0x560f04744a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04744a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 -- 172.21.15.86:0/3333377773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 msgr2=0x560f047604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f047604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 -- 172.21.15.86:0/3333377773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f047666a0 con 0x560f04754680 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04744a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc864a46640 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc858011610 tx=0x7fc858008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5074c00e0008072c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc845ffb640 1 -- 172.21.15.86:0/3333377773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858004b20 con 0x560f04754680 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc845ffb640 1 -- 172.21.15.86:0/3333377773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc858004cc0 con 0x560f04754680 2024-09-05T06:49:21.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.798+0000 7fc845ffb640 1 -- 172.21.15.86:0/3333377773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858005280 con 0x560f04754680 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- 172.21.15.86:0/3333377773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 msgr2=0x560f0476b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc858011610 tx=0x7fc858008000 comp rx=0 tx=0).stop 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- 172.21.15.86:0/3333377773 shutdown_connections 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f04754680 0x560f0476b8d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04744a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 --2- 172.21.15.86:0/3333377773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f047494a0 0x560f047604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- 172.21.15.86:0/3333377773 >> 172.21.15.86:0/3333377773 conn(0x560f04745a30 msgr2=0x560f04744350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- 172.21.15.86:0/3333377773 shutdown_connections 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- 172.21.15.86:0/3333377773 wait complete. 2024-09-05T06:49:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 -- start start 2024-09-05T06:49:21.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.799+0000 7fc868882740 1 Processor -- start 2024-09-05T06:49:21.813 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:21 smithi096 ceph-mon[34468]: pgmap v250: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04758ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04754680 0x560f047cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b2810 con 0x560f047494a0 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b2140 con 0x560f04754680 2024-09-05T06:49:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601af3a0 con 0x560f04749dc0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc868882740 10 client.0 _ll_register_callbacks cb 0x7fc850004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34416/0 (socket says 172.21.15.86:34416) 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 -- 172.21.15.86:0/1662232254 learned_addr learned my addr 172.21.15.86:0/1662232254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc864a46640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04754680 0x560f047cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04758ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 -- 172.21.15.86:0/1662232254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 msgr2=0x560f04758ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04758ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 -- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04754680 msgr2=0x560f047cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04754680 0x560f047cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 -- 172.21.15.86:0/1662232254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f047cdde0 con 0x560f047494a0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc85ffff640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc850097670 tx=0x7fc850018f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdb2f08f66df44c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.800+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85009b210 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fc850097880 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc858004cc0 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc85009f5c0 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 10 client.17307 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f047ceff0 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc858004cc0 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc850017c10 con 0x560f047494a0 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 10 client.17307 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 20 client.17307 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f047cf6c0 0x560f047d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 4 client.17307 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f046b6d60 con 0x560f047cf6c0 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.801+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f047cf6c0 0x560f047d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc83f7fe640 10 client.17307 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc85f7fe640 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f047cf6c0 0x560f047d1ab0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76aed5f9460a489b server_cookie=f5a58caeb8dc05b2 in_seq=0 out_seq=0 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc83f7fe640 1 -- 172.21.15.86:0/1662232254 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f046b6d60 con 0x560f047cf6c0 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc83f7fe640 10 client.17307 handle_command_reply: tid=0 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 client.17307 shutdown 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 msgr2=0x560f047587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc850097670 tx=0x7fc850018f10 comp rx=0 tx=0).stop 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 shutdown_connections 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f047cf6c0 0x560f047d1ab0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f04754680 0x560f047cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f04749dc0 0x560f04758ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 --2- 172.21.15.86:0/1662232254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f047494a0 0x560f047587a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 >> 172.21.15.86:0/1662232254 conn(0x560f04745a30 msgr2=0x560f04753290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc83f7fe640 0 client.17307 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 shutdown_connections 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 1 -- 172.21.15.86:0/1662232254 wait complete. 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.802+0000 7fc868882740 20 client.17307 trim_cache size 0 max 16384 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:49:21.820 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 msgr2=0x7fc8480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 0x7fc8480a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc858009b80 tx=0x7fc858009500 comp rx=0 tx=0).stop 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 msgr2=0x7fc860197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d950 tx=0x7fc85000dd50 comp rx=0 tx=0).stop 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 shutdown_connections 2024-09-05T06:49:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc8480a2cf0 0x7fc8480a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc860106a80 0x7fc860197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc860101420 0x7fc860197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 --2- 172.21.15.86:0/4240677830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc860100a80 0x7fc860197050 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 >> 172.21.15.86:0/4240677830 conn(0x7fc8600fc7a0 msgr2=0x7fc8600fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 shutdown_connections 2024-09-05T06:49:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:21.803+0000 7fc8664a9640 1 -- 172.21.15.86:0/4240677830 wait complete. 2024-09-05T06:49:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:21 smithi115 ceph-mon[34791]: pgmap v250: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-09-05T06:49:22.506 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:49:22.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:49:22.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.97 MB/sec execute 282 sec latency 6578.496 ms 2024-09-05T06:49:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 -- 172.21.15.86:0/3165950727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 msgr2=0x7f09481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 --2- 172.21.15.86:0/3165950727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f09481067f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0938009900 tx=0x7f093802a3f0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 -- 172.21.15.86:0/3165950727 shutdown_connections 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 --2- 172.21.15.86:0/3165950727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 0x7f094810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 --2- 172.21.15.86:0/3165950727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f09481067f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 --2- 172.21.15.86:0/3165950727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 -- 172.21.15.86:0/3165950727 >> 172.21.15.86:0/3165950727 conn(0x7f09480fd8f0 msgr2=0x7f09480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:23.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 -- 172.21.15.86:0/3165950727 shutdown_connections 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.343+0000 7f094f57a640 1 -- 172.21.15.86:0/3165950727 wait complete. 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 Processor -- start 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 -- start start 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 0x7f0948074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0948111e30 con 0x7f09481027a0 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0948111cb0 con 0x7f0948106f20 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094f57a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0948111fb0 con 0x7f0948101df0 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41866/0 (socket says 172.21.15.86:41866) 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 -- 172.21.15.86:0/2230697628 learned_addr learned my addr 172.21.15.86:0/2230697628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094d316640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094db17640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 0x7f0948074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 -- 172.21.15.86:0/2230697628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 msgr2=0x7f09480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 -- 172.21.15.86:0/2230697628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 msgr2=0x7f0948074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 0x7f0948074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094cb15640 1 -- 172.21.15.86:0/2230697628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09481a6870 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.344+0000 7f094d316640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f094cb15640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0938006010 tx=0x7f0938004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093803f030 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09481a6b00 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09380035f0 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938035070 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.345+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481a7030 con 0x7f09481027a0 2024-09-05T06:49:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.346+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09481021d0 con 0x7f09481027a0 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.347+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0938003b10 con 0x7f09481027a0 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.347+0000 7f093e7fc640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 0x7f09340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.348+0000 7f094d316640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 0x7f09340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.348+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f09380f04c0 con 0x7f09481027a0 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.348+0000 7f094d316640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 0x7f09340a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0944005fd0 tx=0x7f0944005ea0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:23.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.352+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09380bcca0 con 0x7f09481027a0 2024-09-05T06:49:23.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.96 MB/sec execute 283 sec latency 7578.538 ms 2024-09-05T06:49:23.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.555+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f094810a160 con 0x7f09481027a0 2024-09-05T06:49:23.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.556+0000 7f093e7fc640 1 -- 172.21.15.86:0/2230697628 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f09380bc5a0 con 0x7f09481027a0 2024-09-05T06:49:23.558 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:49:23.558 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:49:23.558 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 msgr2=0x7f09340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 0x7f09340a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0944005fd0 tx=0x7f0944005ea0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 msgr2=0x7f0948074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0938006010 tx=0x7f0938004270 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 shutdown_connections 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09340a2cf0 0x7f09340a51b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948106f20 0x7f0948074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09481027a0 0x7f0948074a30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 --2- 172.21.15.86:0/2230697628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0948101df0 0x7f09480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.558+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 >> 172.21.15.86:0/2230697628 conn(0x7f09480fd8f0 msgr2=0x7f09480fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:23.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.559+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 shutdown_connections 2024-09-05T06:49:23.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:23.559+0000 7f094f57a640 1 -- 172.21.15.86:0/2230697628 wait complete. 2024-09-05T06:49:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:23 smithi096 ceph-mon[34468]: pgmap v251: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:23.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:23 smithi086 ceph-mon[30943]: pgmap v251: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:23 smithi115 ceph-mon[34791]: pgmap v251: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:24.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:49:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:24.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.95 MB/sec execute 284 sec latency 8578.594 ms 2024-09-05T06:49:24.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2230697628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2230697628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:24.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2230697628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:25.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.95 MB/sec execute 285 sec latency 9578.638 ms 2024-09-05T06:49:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:25 smithi096 ceph-mon[34468]: pgmap v252: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:25.874 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:25 smithi086 ceph-mon[30943]: pgmap v252: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- 172.21.15.86:0/3519624670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc101420 msgr2=0x7f13bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 --2- 172.21.15.86:0/3519624670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc101420 0x7f13bc10d950 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009cd0 tx=0x7f13b402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- 172.21.15.86:0/3519624670 shutdown_connections 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 --2- 172.21.15.86:0/3519624670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc101420 0x7f13bc10d950 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 --2- 172.21.15.86:0/3519624670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc100a80 0x7f13bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 --2- 172.21.15.86:0/3519624670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc106a80 0x7f13bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- 172.21.15.86:0/3519624670 >> 172.21.15.86:0/3519624670 conn(0x7f13bc0fc7a0 msgr2=0x7f13bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- 172.21.15.86:0/3519624670 shutdown_connections 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- 172.21.15.86:0/3519624670 wait complete. 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 Processor -- start 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.982+0000 7f13c30b1640 1 -- start start 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 0x7f13bc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc101420 0x7f13bc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc110d30 con 0x7f13bc106a80 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc110bb0 con 0x7f13bc100a80 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c30b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc110eb0 con 0x7f13bc101420 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41886/0 (socket says 172.21.15.86:41886) 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 -- 172.21.15.86:0/1355081338 learned_addr learned my addr 172.21.15.86:0/1355081338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 0x7f13bc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc101420 0x7f13bc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 -- 172.21.15.86:0/1355081338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc101420 msgr2=0x7f13bc1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc101420 0x7f13bc1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 -- 172.21.15.86:0/1355081338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 msgr2=0x7f13bc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 0x7f13bc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 -- 172.21.15.86:0/1355081338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc1a4760 con 0x7f13bc106a80 2024-09-05T06:49:25.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 0x7f13bc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13c164e640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005cc0 tx=0x7f13b4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.983+0000 7f13b9ffb640 1 -- 172.21.15.86:0/1355081338 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4004450 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.984+0000 7f13b9ffb640 1 -- 172.21.15.86:0/1355081338 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13b4031c90 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.984+0000 7f13b9ffb640 1 -- 172.21.15.86:0/1355081338 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b403b400 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.984+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1a4930 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.984+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1a4e10 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13b9ffb640 1 -- 172.21.15.86:0/1355081338 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13b4004680 con 0x7f13bc106a80 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13c548a740 1 Processor -- start 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13c548a740 1 -- start start 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13b9ffb640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 0x7f13ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13c548a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac574f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13c548a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.986+0000 7f13c548a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac575c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c548a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cac56b52f0 con 0x55cac574f5d0 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c548a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cac565ccf0 con 0x55cac575a370 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c548a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cac56b12b0 con 0x55cac5750b40 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13b9ffb640 1 -- 172.21.15.86:0/1355081338 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f13b402e030 con 0x7f13bc106a80 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 0x7f13ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac574f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41900/0 (socket says 172.21.15.86:41900) 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac574f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:50782/0 (socket says 172.21.15.86:50782) 2024-09-05T06:49:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 -- 172.21.15.86:0/1364464824 learned_addr learned my addr 172.21.15.86:0/1364464824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13c164e640 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac575c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 -- 172.21.15.86:0/1364464824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 msgr2=0x55cac574f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac574f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 -- 172.21.15.86:0/1364464824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 msgr2=0x55cac575c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac575c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 -- 172.21.15.86:0/1364464824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac56413a0 con 0x55cac574f5d0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 0x7f13ac0a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13a8013800 tx=0x7f13a800ba50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f13a8010740 tx=0x7f13a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c81306ef94878a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13a6ffd640 1 -- 172.21.15.86:0/1364464824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8010e00 con 0x55cac574f5d0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13a6ffd640 1 -- 172.21.15.86:0/1364464824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13a8005020 con 0x55cac574f5d0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.987+0000 7f13a6ffd640 1 -- 172.21.15.86:0/1364464824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8004a00 con 0x55cac574f5d0 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- 172.21.15.86:0/1364464824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 msgr2=0x55cac574fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f13a8010740 tx=0x7f13a801a000 comp rx=0 tx=0).stop 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- 172.21.15.86:0/1364464824 shutdown_connections 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac575c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac574fa10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 --2- 172.21.15.86:0/1364464824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac574f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- 172.21.15.86:0/1364464824 >> 172.21.15.86:0/1364464824 conn(0x55cac5718310 msgr2=0x55cac574bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- 172.21.15.86:0/1364464824 shutdown_connections 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- 172.21.15.86:0/1364464824 wait complete. 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 -- start start 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.988+0000 7f13c548a740 1 Processor -- start 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac5756fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac57575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cac575d010 con 0x55cac574f5d0 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cac5670060 con 0x55cac575a370 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cac575d190 con 0x55cac5750b40 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c548a740 10 client.0 _ll_register_callbacks cb 0x55cac53c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41908/0 (socket says 172.21.15.86:41908) 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1397261873 learned_addr learned my addr 172.21.15.86:0/1397261873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac5756fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c164e640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac57575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 msgr2=0x55cac5756fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac5756fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1397261873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 msgr2=0x55cac57575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac57575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1397261873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac54263c0 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac5756fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c164e640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac57575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f13b0010750 tx=0x7f13b0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcfdf26294b0910 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.989+0000 7f139bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b001d060 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f13b000eda0 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a8005020 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13b0024050 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 10 client.17325 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cac5768b70 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13a8005020 con 0x55cac574f5d0 2024-09-05T06:49:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f13b0016430 con 0x55cac574f5d0 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 10 client.17325 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 10 client.17325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 20 client.17325 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac576a030 0x55cac576c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 4 client.17325 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.990+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac56c2860 con 0x55cac576a030 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.991+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac576a030 0x55cac576c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.991+0000 7f139bfff640 10 client.17325 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.991+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac576a030 0x55cac576c460 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=393cd6382d3593d8 server_cookie=ff44a6d1eb3986ae in_seq=0 out_seq=0 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f139bfff640 1 -- 172.21.15.86:0/1397261873 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cac56c2860 con 0x55cac576a030 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f139bfff640 10 client.17325 handle_command_reply: tid=0 2024-09-05T06:49:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 client.17325 shutdown 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 msgr2=0x55cac5759d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f13b0010750 tx=0x7f13b0010c10 comp rx=0 tx=0).stop 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 shutdown_connections 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac576a030 0x55cac576c460 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac575a370 0x55cac57575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac5750b40 0x55cac5756fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 --2- 172.21.15.86:0/1397261873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac5759d30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 >> 172.21.15.86:0/1397261873 conn(0x55cac5718310 msgr2=0x55cac5751960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f139bfff640 0 client.17325 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 shutdown_connections 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 1 -- 172.21.15.86:0/1397261873 wait complete. 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:24.992+0000 7f13c548a740 20 client.17325 trim_cache size 0 max 16384 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 Processor -- start 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 -- start start 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac5750b40 0x55cac574aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac574af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1af040 con 0x55cac574f5d0 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b0b80 con 0x55cac5750b40 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c548a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1af710 con 0x55cac575a370 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41920/0 (socket says 172.21.15.86:41920) 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1014320583 learned_addr learned my addr 172.21.15.86:0/1014320583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c164e640 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac574af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13bbfff640 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac5750b40 0x55cac574aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1014320583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 msgr2=0x55cac574af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac574af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1014320583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac5750b40 msgr2=0x55cac574aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac5750b40 0x55cac574aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 -- 172.21.15.86:0/1014320583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac5755f90 con 0x55cac574f5d0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.005+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005940 tx=0x7f13b000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809cc753e753a81a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13a67fc640 1 -- 172.21.15.86:0/1014320583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b002c030 con 0x55cac574f5d0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13a67fc640 1 -- 172.21.15.86:0/1014320583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13b0016e40 con 0x55cac574f5d0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13a67fc640 1 -- 172.21.15.86:0/1014320583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0010e60 con 0x55cac574f5d0 2024-09-05T06:49:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 -- 172.21.15.86:0/1014320583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 msgr2=0x55cac57568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005940 tx=0x7f13b000bf90 comp rx=0 tx=0).stop 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 -- 172.21.15.86:0/1014320583 shutdown_connections 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac574af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac5750b40 0x55cac574aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 --2- 172.21.15.86:0/1014320583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac574f5d0 0x55cac57568a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 -- 172.21.15.86:0/1014320583 >> 172.21.15.86:0/1014320583 conn(0x55cac5718310 msgr2=0x55cac574a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 -- 172.21.15.86:0/1014320583 shutdown_connections 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.006+0000 7f13c548a740 1 -- 172.21.15.86:0/1014320583 wait complete. 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 -- start start 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 Processor -- start 2024-09-05T06:49:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac574f5d0 0x55cac5762a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac575e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b04b0 con 0x55cac5750b40 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1afde0 con 0x55cac574f5d0 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1ad040 con 0x55cac575a370 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c548a740 10 client.0 _ll_register_callbacks cb 0x7f13b000b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41930/0 (socket says 172.21.15.86:41930) 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 -- 172.21.15.86:0/4002270296 learned_addr learned my addr 172.21.15.86:0/4002270296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c164e640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac575e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac574f5d0 0x55cac5762a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 -- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 msgr2=0x55cac575e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac575e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 -- 172.21.15.86:0/4002270296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac574f5d0 msgr2=0x55cac5762a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac574f5d0 0x55cac5762a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.007+0000 7f13bbfff640 1 -- 172.21.15.86:0/4002270296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac575e9e0 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13bbfff640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x55cac574cac0 tx=0x7f13a800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb5bad481242398 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80052d0 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f13a8005470 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b0016e40 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13a8042050 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 10 client.17331 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cac57d4c00 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13b0016e40 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f13a800f040 con 0x55cac5750b40 2024-09-05T06:49:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 10 client.17331 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 10 client.17331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 20 client.17331 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac57d5070 0x55cac57d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 4 client.17331 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.008+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cac56bcd70 con 0x55cac57d5070 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac57d5070 0x55cac57d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f139bfff640 10 client.17331 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c0e4d640 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac57d5070 0x55cac57d7490 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0b24fc1f25e6894 server_cookie=d63880a3562c45fd in_seq=0 out_seq=0 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f139bfff640 1 -- 172.21.15.86:0/4002270296 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cac56bcd70 con 0x55cac57d5070 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f139bfff640 10 client.17331 handle_command_reply: tid=0 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 client.17331 shutdown 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 msgr2=0x55cac575ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x55cac574cac0 tx=0x7f13a800b2c0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 shutdown_connections 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55cac57d5070 0x55cac57d7490 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cac575a370 0x55cac575e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cac5750b40 0x55cac575ddb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 --2- 172.21.15.86:0/4002270296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cac574f5d0 0x55cac5762a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f139bfff640 0 client.17331 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.009+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 >> 172.21.15.86:0/4002270296 conn(0x55cac5718310 msgr2=0x55cac5759060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.893 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 shutdown_connections 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c548a740 1 -- 172.21.15.86:0/4002270296 wait complete. 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c548a740 20 client.17331 trim_cache size 0 max 16384 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 msgr2=0x7f13ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 0x7f13ac0a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13a8013800 tx=0x7f13a800ba50 comp rx=0 tx=0).stop 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 msgr2=0x7f13bc1a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005cc0 tx=0x7f13b4003c20 comp rx=0 tx=0).stop 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 shutdown_connections 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f13ac0a2d70 0x7f13ac0a5230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc106a80 0x7f13bc1a4220 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13bc101420 0x7f13bc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 --2- 172.21.15.86:0/1355081338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13bc100a80 0x7f13bc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 >> 172.21.15.86:0/1355081338 conn(0x7f13bc0fc7a0 msgr2=0x7f13bc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 shutdown_connections 2024-09-05T06:49:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:25.010+0000 7f13c30b1640 1 -- 172.21.15.86:0/1355081338 wait complete. 2024-09-05T06:49:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:25 smithi115 ceph-mon[34791]: pgmap v252: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 590 KiB/s wr, 75 op/s 2024-09-05T06:49:26.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:49:26.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.94 MB/sec execute 286 sec latency 10578.704 ms 2024-09-05T06:49:27.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.93 MB/sec execute 287 sec latency 11578.733 ms 2024-09-05T06:49:27.735 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:27 smithi086 ceph-mon[30943]: pgmap v253: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 596 KiB/s wr, 80 op/s 2024-09-05T06:49:27.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:49:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:27 smithi096 ceph-mon[34468]: pgmap v253: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 596 KiB/s wr, 80 op/s 2024-09-05T06:49:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:27 smithi115 ceph-mon[34791]: pgmap v253: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 596 KiB/s wr, 80 op/s 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- 172.21.15.86:0/3387318993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 msgr2=0x7ff9ac10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/3387318993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac10ec30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009cd0 tx=0x7ff99c02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- 172.21.15.86:0/3387318993 shutdown_connections 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/3387318993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac10ec30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/3387318993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac103740 0x7ff9ac107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/3387318993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac102d90 0x7ff9ac103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- 172.21.15.86:0/3387318993 >> 172.21.15.86:0/3387318993 conn(0x7ff9ac0fc7d0 msgr2=0x7ff9ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- 172.21.15.86:0/3387318993 shutdown_connections 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- 172.21.15.86:0/3387318993 wait complete. 2024-09-05T06:49:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 Processor -- start 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.268+0000 7ff9b36b0640 1 -- start start 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac102d90 0x7ff9ac198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac103740 0x7ff9ac198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111fa0 con 0x7ff9ac108290 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111e20 con 0x7ff9ac103740 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac112120 con 0x7ff9ac102d90 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41948/0 (socket says 172.21.15.86:41948) 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 -- 172.21.15.86:0/1591659315 learned_addr learned my addr 172.21.15.86:0/1591659315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b0c4b640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac103740 0x7ff9ac198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b144c640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac102d90 0x7ff9ac198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 -- 172.21.15.86:0/1591659315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac102d90 msgr2=0x7ff9ac198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac102d90 0x7ff9ac198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 -- 172.21.15.86:0/1591659315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac103740 msgr2=0x7ff9ac198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac103740 0x7ff9ac198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 -- 172.21.15.86:0/1591659315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac19d4d0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b1c4d640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005cc0 tx=0x7ff99c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c004240 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac19d760 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff99c0305a0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.269+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0466c0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.270+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a50a0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.271+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac06a7f0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.272+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff99c030ac0 con 0x7ff9ac108290 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.272+0000 7ff9aa7fc640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 0x7ff9940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.272+0000 7ff9b144c640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 0x7ff9940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:28.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.272+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7ff99c02e030 con 0x7ff9ac108290 2024-09-05T06:49:28.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.273+0000 7ff9b144c640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 0x7ff9940a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009890 tx=0x7ff9a4008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:28.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.275+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff99c04b060 con 0x7ff9ac108290 2024-09-05T06:49:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:28.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.496+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9ac10a2a0 con 0x7ff9ac108290 2024-09-05T06:49:28.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.496+0000 7ff9aa7fc640 1 -- 172.21.15.86:0/1591659315 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7ff99c039380 con 0x7ff9ac108290 2024-09-05T06:49:28.497 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:49:28.497 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:49:28.497 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.498+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 msgr2=0x7ff9940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.498+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 0x7ff9940a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009890 tx=0x7ff9a4008040 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.498+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 msgr2=0x7ff9ac19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.498+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005cc0 tx=0x7ff99c003ba0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.498+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 shutdown_connections 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff9940a2cf0 0x7ff9940a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ac108290 0x7ff9ac19cdf0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9ac103740 0x7ff9ac198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 --2- 172.21.15.86:0/1591659315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9ac102d90 0x7ff9ac198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 >> 172.21.15.86:0/1591659315 conn(0x7ff9ac0fc7d0 msgr2=0x7ff9ac105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 shutdown_connections 2024-09-05T06:49:28.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:28.499+0000 7ff9b36b0640 1 -- 172.21.15.86:0/1591659315 wait complete. 2024-09-05T06:49:28.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.93 MB/sec execute 288 sec latency 12578.785 ms 2024-09-05T06:49:28.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:28 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1591659315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:28 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1591659315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:28.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:28 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1591659315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:29.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 6e6ae99a --release --await' 2024-09-05T06:49:29.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 6e6ae99a --release --await 2024-09-05T06:49:29.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.92 MB/sec execute 289 sec latency 13578.847 ms 2024-09-05T06:49:29.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:29 smithi086 ceph-mon[30943]: pgmap v254: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- 172.21.15.86:0/2174467628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102ef0 msgr2=0x7f74f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 --2- 172.21.15.86:0/2174467628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102ef0 0x7f74f010da40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009ca0 tx=0x7f74e802a950 comp rx=0 tx=0).stop 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- 172.21.15.86:0/2174467628 shutdown_connections 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 --2- 172.21.15.86:0/2174467628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102ef0 0x7f74f010da40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 --2- 172.21.15.86:0/2174467628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102570 0x7f74f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 --2- 172.21.15.86:0/2174467628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 0x7f74f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- 172.21.15.86:0/2174467628 >> 172.21.15.86:0/2174467628 conn(0x7f74f00fc7d0 msgr2=0x7f74f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- 172.21.15.86:0/2174467628 shutdown_connections 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- 172.21.15.86:0/2174467628 wait complete. 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 Processor -- start 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.559+0000 7f74f620f640 1 -- start start 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 0x7f74f0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102ef0 0x7f74f019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0110e30 con 0x7f74f0102570 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0110cb0 con 0x7f74f0100e20 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74f620f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0110fb0 con 0x7f74f0102ef0 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41964/0 (socket says 172.21.15.86:41964) 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 -- 172.21.15.86:0/2365574433 learned_addr learned my addr 172.21.15.86:0/2365574433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74effff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102ef0 0x7f74f019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 0x7f74f0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 -- 172.21.15.86:0/2365574433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102ef0 msgr2=0x7f74f019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102ef0 0x7f74f019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 -- 172.21.15.86:0/2365574433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 msgr2=0x7f74f0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 0x7f74f0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 -- 172.21.15.86:0/2365574433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f019d2b0 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.560+0000 7f74e7fff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f74d800c890 tx=0x7f74d800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.561+0000 7f74ed7fa640 1 -- 172.21.15.86:0/2365574433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004220 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.561+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f019d5a0 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.561+0000 7f74ed7fa640 1 -- 172.21.15.86:0/2365574433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d80043c0 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.561+0000 7f74ed7fa640 1 -- 172.21.15.86:0/2365574433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004a00 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.561+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a4e80 con 0x7f74f0102570 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 Processor -- start 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 -- start start 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef742b40 0x55ccef741090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef752b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccef6a72f0 con 0x55ccef750780 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccef64ecf0 con 0x55ccef742b40 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74f85e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ccef6a32b0 con 0x55ccef7415d0 2024-09-05T06:49:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74ed7fa640 1 -- 172.21.15.86:0/2365574433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74d80079b0 con 0x7f74f0102570 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74ef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74ef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:50840/0 (socket says 172.21.15.86:50840) 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1276831415 learned_addr learned my addr 172.21.15.86:0/1276831415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74effff640 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef742b40 0x55ccef741090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.564+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef752b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1276831415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef742b40 msgr2=0x55ccef741090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef742b40 0x55ccef741090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1276831415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 msgr2=0x55ccef752b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef752b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1276831415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccef6333a0 con 0x55ccef7415d0 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74e000ad00 tx=0x7f74e000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5ec64c00de6274c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ed7fa640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 0x7f74dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74e5ffb640 1 -- 172.21.15.86:0/1276831415 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e001d070 con 0x55ccef7415d0 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74e5ffb640 1 -- 172.21.15.86:0/1276831415 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74e0018af0 con 0x55ccef7415d0 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74e5ffb640 1 -- 172.21.15.86:0/1276831415 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0010b40 con 0x55ccef7415d0 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74effff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 0x7f74dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.565+0000 7f74ed7fa640 1 -- 172.21.15.86:0/2365574433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f74d80cf4b0 con 0x7f74f0102570 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- 172.21.15.86:0/1276831415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 msgr2=0x55ccef741a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74effff640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 0x7f74dc0a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a60 tx=0x7f74e8005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74e000ad00 tx=0x7f74e000f790 comp rx=0 tx=0).stop 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- 172.21.15.86:0/1276831415 shutdown_connections 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef752b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef7415d0 0x55ccef741a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1276831415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef742b40 0x55ccef741090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- 172.21.15.86:0/1276831415 >> 172.21.15.86:0/1276831415 conn(0x55ccef70a310 msgr2=0x55ccef73dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- 172.21.15.86:0/1276831415 shutdown_connections 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- 172.21.15.86:0/1276831415 wait complete. 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.566+0000 7f74f85e8740 1 -- start start 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 Processor -- start 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef7501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef750780 0x55ccef74da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccef753480 con 0x55ccef742b40 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccef662060 con 0x55ccef7415d0 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ccef753600 con 0x55ccef750780 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74f85e8740 10 client.0 _ll_register_callbacks cb 0x55ccef3b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:29.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41968/0 (socket says 172.21.15.86:41968) 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 -- 172.21.15.86:0/1329533770 learned_addr learned my addr 172.21.15.86:0/1329533770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74effff640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef750780 0x55ccef74da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef7501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 -- 172.21.15.86:0/1329533770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef750780 msgr2=0x55ccef74da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef750780 0x55ccef74da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 -- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 msgr2=0x55ccef7501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.567+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef7501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74e7fff640 1 -- 172.21.15.86:0/1329533770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccef4183c0 con 0x55ccef742b40 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005fd0 tx=0x7f74d8097170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec109b9ebc8c807 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d801d040 con 0x55ccef742b40 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f74d8007ca0 con 0x55ccef742b40 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e0018af0 con 0x55ccef742b40 2024-09-05T06:49:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74d809b0d0 con 0x55ccef742b40 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 10 client.17349 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccef75efb0 con 0x55ccef742b40 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74e0018af0 con 0x55ccef742b40 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f74d8097c00 con 0x55ccef742b40 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 10 client.17349 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 10 client.17349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 20 client.17349 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef760470 0x55ccef7628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 4 client.17349 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.568+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccef6b4860 con 0x55ccef760470 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.569+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef760470 0x55ccef7628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.569+0000 7f74ceffd640 10 client.17349 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.569+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef760470 0x55ccef7628c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62eb152fe3cd1bf0 server_cookie=1997554cd6656b29 in_seq=0 out_seq=0 2024-09-05T06:49:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74ceffd640 1 -- 172.21.15.86:0/1329533770 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccef6b4860 con 0x55ccef760470 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74ceffd640 10 client.17349 handle_command_reply: tid=0 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 client.17349 shutdown 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 msgr2=0x55ccef74d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005fd0 tx=0x7f74d8097170 comp rx=0 tx=0).stop 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 shutdown_connections 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef760470 0x55ccef7628c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef750780 0x55ccef74da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74ceffd640 0 client.17349 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef742b40 0x55ccef74d450 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1329533770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef7501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 >> 172.21.15.86:0/1329533770 conn(0x55ccef70a310 msgr2=0x55ccef743960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 shutdown_connections 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 1 -- 172.21.15.86:0/1329533770 wait complete. 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.570+0000 7f74f85e8740 20 client.17349 trim_cache size 0 max 16384 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 Processor -- start 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 -- start start 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef73de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef75c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01acf40 con 0x55ccef750780 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01aea80 con 0x55ccef7415d0 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74f85e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01ad610 con 0x55ccef742b40 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41982/0 (socket says 172.21.15.86:41982) 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74effff640 1 -- 172.21.15.86:0/3971941308 learned_addr learned my addr 172.21.15.86:0/3971941308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.583+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef73de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74e7fff640 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef75c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 -- 172.21.15.86:0/3971941308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 msgr2=0x55ccef75c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef75c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 -- 172.21.15.86:0/3971941308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 msgr2=0x55ccef73de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef73de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 -- 172.21.15.86:0/3971941308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccef75cc60 con 0x55ccef750780 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74effff640 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009ca0 tx=0x7f74e8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38d6169ffbad021 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74e57fa640 1 -- 172.21.15.86:0/3971941308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8034040 con 0x55ccef750780 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74e57fa640 1 -- 172.21.15.86:0/3971941308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74e8003b70 con 0x55ccef750780 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.584+0000 7f74e57fa640 1 -- 172.21.15.86:0/3971941308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80307c0 con 0x55ccef750780 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- 172.21.15.86:0/3971941308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 msgr2=0x55ccef768140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009ca0 tx=0x7f74e8041000 comp rx=0 tx=0).stop 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- 172.21.15.86:0/3971941308 shutdown_connections 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef750780 0x55ccef768140 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef75c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 --2- 172.21.15.86:0/3971941308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef7415d0 0x55ccef73de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- 172.21.15.86:0/3971941308 >> 172.21.15.86:0/3971941308 conn(0x55ccef70a310 msgr2=0x55ccef73d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- 172.21.15.86:0/3971941308 shutdown_connections 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- 172.21.15.86:0/3971941308 wait complete. 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 -- start start 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.585+0000 7f74f85e8740 1 Processor -- start 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef758dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef750780 0x55ccef755860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01ae3b0 con 0x55ccef7415d0 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01adce0 con 0x55ccef750780 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01aaf40 con 0x55ccef742b40 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74f85e8740 10 client.0 _ll_register_callbacks cb 0x7f74d80d2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41998/0 (socket says 172.21.15.86:41998) 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1082945393 learned_addr learned my addr 172.21.15.86:0/1082945393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74effff640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef750780 0x55ccef755860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef758dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1082945393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 msgr2=0x55ccef758dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef758dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef750780 msgr2=0x55ccef755860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef750780 0x55ccef755860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 -- 172.21.15.86:0/1082945393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccef755da0 con 0x55ccef7415d0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ef7fe640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f74e000fc90 tx=0x7f74e000a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa129b1ccec48008 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.586+0000 7f74ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e001d040 con 0x55ccef7415d0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f74e0029050 con 0x55ccef7415d0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e8003b70 con 0x55ccef7415d0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74e0038050 con 0x55ccef7415d0 2024-09-05T06:49:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 10 client.17355 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccef7cb750 con 0x55ccef7415d0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74e8003b70 con 0x55ccef7415d0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f74e0004660 con 0x55ccef7415d0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 10 client.17355 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 10 client.17355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 20 client.17355 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef7cbd30 0x55ccef7ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 4 client.17355 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccef6aed70 con 0x55ccef7cbd30 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.587+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef7cbd30 0x55ccef7ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74ceffd640 10 client.17355 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74e7fff640 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef7cbd30 0x55ccef7ce150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d462415dabf16a2 server_cookie=3c541994f9676540 in_seq=0 out_seq=0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74ceffd640 1 -- 172.21.15.86:0/1082945393 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccef6aed70 con 0x55ccef7cbd30 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74ceffd640 10 client.17355 handle_command_reply: tid=0 2024-09-05T06:49:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 client.17355 shutdown 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 msgr2=0x55ccef758890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f74e000fc90 tx=0x7f74e000a960 comp rx=0 tx=0).stop 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 shutdown_connections 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ccef7cbd30 0x55ccef7ce150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccef750780 0x55ccef755860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ccef742b40 0x55ccef758dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74ceffd640 0 client.17355 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 --2- 172.21.15.86:0/1082945393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccef7415d0 0x55ccef758890 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 >> 172.21.15.86:0/1082945393 conn(0x55ccef70a310 msgr2=0x55ccef73c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 shutdown_connections 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 1 -- 172.21.15.86:0/1082945393 wait complete. 2024-09-05T06:49:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.588+0000 7f74f85e8740 20 client.17355 trim_cache size 0 max 16384 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 msgr2=0x7f74dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 0x7f74dc0a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a60 tx=0x7f74e8005c50 comp rx=0 tx=0).stop 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 msgr2=0x7f74f0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f74d800c890 tx=0x7f74d800cc90 comp rx=0 tx=0).stop 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 shutdown_connections 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74dc0a2cf0 0x7f74dc0a51b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74f0102ef0 0x7f74f019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f0102570 0x7f74f0198840 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 --2- 172.21.15.86:0/2365574433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f0100e20 0x7f74f0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 >> 172.21.15.86:0/2365574433 conn(0x7f74f00fc7d0 msgr2=0x7f74f006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 shutdown_connections 2024-09-05T06:49:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:29.589+0000 7f74f620f640 1 -- 172.21.15.86:0/2365574433 wait complete. 2024-09-05T06:49:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:29 smithi096 ceph-mon[34468]: pgmap v254: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-05T06:49:29.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:29 smithi115 ceph-mon[34791]: pgmap v254: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-05T06:49:30.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:49:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:30.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158050 1.91 MB/sec execute 290 sec latency 14578.899 ms 2024-09-05T06:49:30.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 -- 172.21.15.86:0/1262535994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 msgr2=0x7f171c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 --2- 172.21.15.86:0/1262535994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c100ec0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1704009930 tx=0x7f170402a420 comp rx=0 tx=0).stop 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 -- 172.21.15.86:0/1262535994 shutdown_connections 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 --2- 172.21.15.86:0/1262535994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 --2- 172.21.15.86:0/1262535994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c100ec0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 --2- 172.21.15.86:0/1262535994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 -- 172.21.15.86:0/1262535994 >> 172.21.15.86:0/1262535994 conn(0x7f171c0fc780 msgr2=0x7f171c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 -- 172.21.15.86:0/1262535994 shutdown_connections 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.944+0000 7f1721579640 1 -- 172.21.15.86:0/1262535994 wait complete. 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 Processor -- start 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 -- start start 2024-09-05T06:49:30.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110d60 con 0x7f171c100a60 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110be0 con 0x7f171c101400 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f1721579640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110ee0 con 0x7f171c106a60 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42008/0 (socket says 172.21.15.86:42008) 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 -- 172.21.15.86:0/4237293006 learned_addr learned my addr 172.21.15.86:0/4237293006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171b7fe640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f17127fc640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 -- 172.21.15.86:0/4237293006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 msgr2=0x7f171c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 -- 172.21.15.86:0/4237293006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 msgr2=0x7f171c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.945+0000 7f171affd640 1 -- 172.21.15.86:0/4237293006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c19c250 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f17127fc640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f171b7fe640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f171affd640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f170c00c890 tx=0x7f170c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f1718ff9640 1 -- 172.21.15.86:0/4237293006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c004220 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f1718ff9640 1 -- 172.21.15.86:0/4237293006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f170c0043c0 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c19c540 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f1718ff9640 1 -- 172.21.15.86:0/4237293006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c004a00 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.946+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c1a3e20 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1718ff9640 1 -- 172.21.15.86:0/4237293006 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f170c004ba0 con 0x7f171c100a60 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 Processor -- start 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 -- start start 2024-09-05T06:49:30.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x55958609ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559585ff9620 con 0x559586094060 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559585fa0e50 con 0x55958609f240 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f1723952740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559585ff55e0 con 0x559586094980 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42010/0 (socket says 172.21.15.86:42010) 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.949+0000 7f171b7fe640 1 -- 172.21.15.86:0/629282312 learned_addr learned my addr 172.21.15.86:0/629282312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171affd640 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x55958609ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f17127fc640 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f1718ff9640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 0x7f17080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 -- 172.21.15.86:0/629282312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 msgr2=0x55958609ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x55958609ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 -- 172.21.15.86:0/629282312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 msgr2=0x5595860a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 -- 172.21.15.86:0/629282312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559585f91d40 con 0x559586094060 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 0x7f17080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f1718ff9640 1 -- 172.21.15.86:0/4237293006 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f170c0cf530 con 0x7f171c100a60 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f171b7fe640 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1714010540 tx=0x7f1714010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811fc35e942a24cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f17117fa640 1 -- 172.21.15.86:0/629282312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171401d400 con 0x559586094060 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f17117fa640 1 -- 172.21.15.86:0/629282312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f171401da50 con 0x559586094060 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.950+0000 7f17117fa640 1 -- 172.21.15.86:0/629282312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171401c690 con 0x559586094060 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- 172.21.15.86:0/629282312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 msgr2=0x559586094440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f171b7fe640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 0x7f17080a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1714021000 tx=0x7f171401e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1714010540 tx=0x7f1714010a00 comp rx=0 tx=0).stop 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- 172.21.15.86:0/629282312 shutdown_connections 2024-09-05T06:49:30.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x55958609ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 --2- 172.21.15.86:0/629282312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x559586094440 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- 172.21.15.86:0/629282312 >> 172.21.15.86:0/629282312 conn(0x5595860905f0 msgr2=0x5595860909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- 172.21.15.86:0/629282312 shutdown_connections 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- 172.21.15.86:0/629282312 wait complete. 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.951+0000 7f1723952740 1 -- start start 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 Processor -- start 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595860a1e10 con 0x559586094060 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559585f81f10 con 0x55958609f240 2024-09-05T06:49:30.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595860a1f90 con 0x559586094980 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f1723952740 10 client.0 _ll_register_callbacks cb 0x559585fa5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42018/0 (socket says 172.21.15.86:42018) 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 -- 172.21.15.86:0/2408784326 learned_addr learned my addr 172.21.15.86:0/2408784326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f17127fc640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171b7fe640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 -- 172.21.15.86:0/2408784326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 msgr2=0x5595860a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 -- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 msgr2=0x5595860a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 -- 172.21.15.86:0/2408784326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559585d08580 con 0x559586094060 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f171affd640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f170c097b10 tx=0x7f170c09b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808565466964679b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f17027fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:30.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.952+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c01d040 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f170c097d20 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f171401da50 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f170c0b3050 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 10 client.17367 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559585dd2960 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f171401da50 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f170c0bb070 con 0x559586094060 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 10 client.17367 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 10 client.17367 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 20 client.17367 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595860b0fb0 0x5595860b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 4 client.17367 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559586006b90 con 0x5595860b0fb0 2024-09-05T06:49:30.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.953+0000 7f17127fc640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595860b0fb0 0x5595860b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.954+0000 7f17127fc640 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595860b0fb0 0x5595860b3400 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d0326119f8e73c server_cookie=3281b2edf80e39e7 in_seq=0 out_seq=0 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.954+0000 7f17027fc640 10 client.17367 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f17027fc640 1 -- 172.21.15.86:0/2408784326 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559586006b90 con 0x5595860b0fb0 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f17027fc640 10 client.17367 handle_command_reply: tid=0 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 client.17367 shutdown 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 msgr2=0x5595860a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f170c097b10 tx=0x7f170c09b1f0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 shutdown_connections 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595860b0fb0 0x5595860b3400 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55958609f240 0x5595860a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f17027fc640 0 client.17367 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:30.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 --2- 172.21.15.86:0/2408784326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559586094060 0x5595860a2220 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 >> 172.21.15.86:0/2408784326 conn(0x5595860905f0 msgr2=0x559586096320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 shutdown_connections 2024-09-05T06:49:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 1 -- 172.21.15.86:0/2408784326 wait complete. 2024-09-05T06:49:30.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.955+0000 7f1723952740 20 client.17367 trim_cache size 0 max 16384 2024-09-05T06:49:30.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.971+0000 7f1723952740 1 Processor -- start 2024-09-05T06:49:30.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.971+0000 7f1723952740 1 -- start start 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094060 0x559586096ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 0x55958608f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ad000 con 0x55958609f240 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1aeb40 con 0x559586094980 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f1723952740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ad6d0 con 0x559586094060 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42020/0 (socket says 172.21.15.86:42020) 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 -- 172.21.15.86:0/979979905 learned_addr learned my addr 172.21.15.86:0/979979905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171affd640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094060 0x559586096ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f17127fc640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 0x55958608f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 -- 172.21.15.86:0/979979905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094060 msgr2=0x559586096ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094060 0x559586096ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 -- 172.21.15.86:0/979979905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 msgr2=0x55958608f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 0x55958608f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f171b7fe640 1 -- 172.21.15.86:0/979979905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595860ad360 con 0x55958609f240 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.972+0000 7f17127fc640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 0x55958608f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f171b7fe640 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1714010fd0 tx=0x7f1714028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871449246d4d2827 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1710ff9640 1 -- 172.21.15.86:0/979979905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171401c3a0 con 0x55958609f240 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1710ff9640 1 -- 172.21.15.86:0/979979905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1714036070 con 0x55958609f240 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1710ff9640 1 -- 172.21.15.86:0/979979905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171401cd90 con 0x55958609f240 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 -- 172.21.15.86:0/979979905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 msgr2=0x5595860a7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1714010fd0 tx=0x7f1714028000 comp rx=0 tx=0).stop 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 -- 172.21.15.86:0/979979905 shutdown_connections 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x5595860a7d20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094980 0x55958608f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 --2- 172.21.15.86:0/979979905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094060 0x559586096ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.973+0000 7f1723952740 1 -- 172.21.15.86:0/979979905 >> 172.21.15.86:0/979979905 conn(0x5595860905f0 msgr2=0x55958608ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- 172.21.15.86:0/979979905 shutdown_connections 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- 172.21.15.86:0/979979905 wait complete. 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- start start 2024-09-05T06:49:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 Processor -- start 2024-09-05T06:49:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094060 0x5595860a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x559586114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ae470 con 0x55958609f240 2024-09-05T06:49:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1adda0 con 0x559586094060 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ab000 con 0x559586094980 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f1723952740 10 client.0 _ll_register_callbacks cb 0x7f170c00a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f17127fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f17127fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:50924/0 (socket says 172.21.15.86:50924) 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f17127fc640 1 -- 172.21.15.86:0/3682465196 learned_addr learned my addr 172.21.15.86:0/3682465196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f171b7fe640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x559586114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.974+0000 7f171affd640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094060 0x5595860a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 -- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094060 msgr2=0x5595860a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094060 0x5595860a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 -- 172.21.15.86:0/3682465196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 msgr2=0x559586114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x559586114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 -- 172.21.15.86:0/3682465196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559586115280 con 0x559586094980 2024-09-05T06:49:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f171b7fe640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x559586114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17127fc640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f171c0fee90 tx=0x7f1704005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4aeb74ae24fc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17027fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704035040 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f1704005340 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1714036070 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1704053050 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f1723952740 10 client.26587 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.975+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55958611c700 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1714036070 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1704003880 con 0x559586094980 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 10 client.26587 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 10 client.26587 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 20 client.26587 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55958611cf00 0x55958611f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:30.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 4 client.26587 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6e6ae99a", "release": true, "await": true}] 2024-09-05T06:49:30.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6e6ae99a", "release": true, "await": true}) -- 0x5595860010a0 con 0x55958611cf00 2024-09-05T06:49:30.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f171affd640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55958611cf00 0x55958611f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:30.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f171affd640 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55958611cf00 0x55958611f320 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d944d7a50856b16 server_cookie=b36d3b76c303daa3 in_seq=0 out_seq=0 2024-09-05T06:49:30.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.976+0000 7f17027fc640 10 client.26587 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.982+0000 7f17027fc640 1 -- 172.21.15.86:0/3682465196 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5595860010a0 con 0x55958611cf00 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.982+0000 7f17027fc640 10 client.26587 handle_command_reply: tid=0 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.982+0000 7f1723952740 1 client.26587 shutdown 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 msgr2=0x5595860a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f171c0fee90 tx=0x7f1704005040 comp rx=0 tx=0).stop 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 shutdown_connections 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55958611cf00 0x55958611f320 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55958609f240 0x559586114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f17027fc640 0 client.26587 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559586094980 0x5595860a6890 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 --2- 172.21.15.86:0/3682465196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559586094060 0x5595860a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 >> 172.21.15.86:0/3682465196 conn(0x5595860905f0 msgr2=0x55958608f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 shutdown_connections 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 1 -- 172.21.15.86:0/3682465196 wait complete. 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1723952740 20 client.26587 trim_cache size 0 max 16384 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:49:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 28, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "6e6ae99a": { 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 28, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 133.2, 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:49:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:30.986 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 msgr2=0x7f17080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 0x7f17080a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1714021000 tx=0x7f171401e000 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 msgr2=0x7f171c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f170c00c890 tx=0x7f170c00cd50 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 shutdown_connections 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f17080a2cf0 0x7f17080a51b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f171c106a60 0x7f171c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f171c101400 0x7f171c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 --2- 172.21.15.86:0/4237293006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f171c100a60 0x7f171c1972a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.983+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 >> 172.21.15.86:0/4237293006 conn(0x7f171c0fc780 msgr2=0x7f171c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:30.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.984+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 shutdown_connections 2024-09-05T06:49:30.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:30.984+0000 7f1721579640 1 -- 172.21.15.86:0/4237293006 wait complete. 2024-09-05T06:49:31.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158401 1.91 MB/sec execute 291 sec latency 15042.718 ms 2024-09-05T06:49:31.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:31 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:49:31.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:31 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:49:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:31 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:49:31.852 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6e6ae99a', total seconds elapsed: 18.7 2024-09-05T06:49:31.930 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:32.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:32 smithi086 ceph-mon[30943]: pgmap v255: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-05T06:49:32.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:32 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:49:32.438 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:32 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:49:32.438 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:32 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:49:32.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 158992 1.92 MB/sec execute 292 sec latency 23.485 ms 2024-09-05T06:49:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 -- 172.21.15.86:0/731861115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364106f10 msgr2=0x7f836410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 --2- 172.21.15.86:0/731861115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364106f10 0x7f836410ea90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f835c009be0 tx=0x7f835c02a6c0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 -- 172.21.15.86:0/731861115 shutdown_connections 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 --2- 172.21.15.86:0/731861115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364106f10 0x7f836410ea90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 --2- 172.21.15.86:0/731861115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364102790 0x7f83641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 --2- 172.21.15.86:0/731861115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364101de0 0x7f83641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 -- 172.21.15.86:0/731861115 >> 172.21.15.86:0/731861115 conn(0x7f83640fd8e0 msgr2=0x7f83640ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 -- 172.21.15.86:0/731861115 shutdown_connections 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.528+0000 7f8368df1640 1 -- 172.21.15.86:0/731861115 wait complete. 2024-09-05T06:49:32.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 Processor -- start 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 -- start start 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364106f10 0x7f836419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111e20 con 0x7f8364102790 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111ca0 con 0x7f8364101de0 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8368df1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111fa0 con 0x7f8364106f10 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8361d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55302/0 (socket says 172.21.15.86:55302) 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8361d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37290/0 (socket says 172.21.15.86:37290) 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 -- 172.21.15.86:0/2341574562 learned_addr learned my addr 172.21.15.86:0/2341574562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f8362d9d640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364106f10 0x7f836419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 -- 172.21.15.86:0/2341574562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364106f10 msgr2=0x7f836419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364106f10 0x7f836419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 -- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 msgr2=0x7f8364198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.529+0000 7f836259c640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f836259c640 1 -- 172.21.15.86:0/2341574562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836419d2d0 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f8361d9b640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f836259c640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f835400e9c0 tx=0x7f835400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f83537fe640 1 -- 172.21.15.86:0/2341574562 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835400cde0 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f83537fe640 1 -- 172.21.15.86:0/2341574562 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83540035f0 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f83537fe640 1 -- 172.21.15.86:0/2341574562 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354010690 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836419d5c0 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.530+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641a4ea0 con 0x7f8364101de0 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.532+0000 7f836b1ca740 1 Processor -- start 2024-09-05T06:49:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 -- start start 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f83537fe640 1 -- 172.21.15.86:0/2341574562 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83540037a0 con 0x7f8364101de0 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de280a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de28d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561de1e62f0 con 0x5561de281b40 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5561de18dcf0 con 0x5561de2805d0 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836b1ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5561de1e22b0 con 0x5561de28b370 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f8362d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f836259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de280a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f8362d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37296/0 (socket says 172.21.15.86:37296) 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f8362d9d640 1 -- 172.21.15.86:0/3613752761 learned_addr learned my addr 172.21.15.86:0/3613752761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.533+0000 7f8361d9b640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de28d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 -- 172.21.15.86:0/3613752761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 msgr2=0x5561de28d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de28d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 -- 172.21.15.86:0/3613752761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 msgr2=0x5561de280090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 -- 172.21.15.86:0/3613752761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561de1723a0 con 0x5561de2805d0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8362d9d640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8361d9b640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de28d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f836259c640 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de280a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f83540ccea0 tx=0x7f83540d86d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6f14dfadcb6d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f83537fe640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8350ff9640 1 -- 172.21.15.86:0/3613752761 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540df070 con 0x5561de2805d0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8350ff9640 1 -- 172.21.15.86:0/3613752761 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835409c370 con 0x5561de2805d0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8350ff9640 1 -- 172.21.15.86:0/3613752761 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835409b5d0 con 0x5561de2805d0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f8362d9d640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.534+0000 7f83537fe640 1 -- 172.21.15.86:0/2341574562 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f83540cfb90 con 0x7f8364101de0 2024-09-05T06:49:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f8362d9d640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 0x7f834c0a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f835c005f00 tx=0x7f835c005bf0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- 172.21.15.86:0/3613752761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 msgr2=0x5561de280a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de280a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f83540ccea0 tx=0x7f83540d86d0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- 172.21.15.86:0/3613752761 shutdown_connections 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de28d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de280a10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 --2- 172.21.15.86:0/3613752761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de280090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- 172.21.15.86:0/3613752761 >> 172.21.15.86:0/3613752761 conn(0x5561de249310 msgr2=0x5561de27cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- 172.21.15.86:0/3613752761 shutdown_connections 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- 172.21.15.86:0/3613752761 wait complete. 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.535+0000 7f836b1ca740 1 -- start start 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 Processor -- start 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de28acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de287f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 0x5561de2884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561de292480 con 0x5561de281b40 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5561de1a1060 con 0x5561de28b370 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5561de292600 con 0x5561de2805d0 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836b1ca740 10 client.0 _ll_register_callbacks cb 0x5561ddef48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f8361d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de287f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f8361d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de287f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37310/0 (socket says 172.21.15.86:37310) 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f8361d9b640 1 -- 172.21.15.86:0/1664222467 learned_addr learned my addr 172.21.15.86:0/1664222467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 0x5561de2884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.536+0000 7f836259c640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de28acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 -- 172.21.15.86:0/1664222467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 msgr2=0x5561de28acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de28acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 -- 172.21.15.86:0/1664222467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 msgr2=0x5561de287f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de287f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 -- 172.21.15.86:0/1664222467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ddf573c0 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 0x5561de2884d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f835c02abe0 tx=0x7f835c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6748814b7520b483 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c04f020 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835409c370 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f835c030d30 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f835c05a050 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f836b1ca740 10 client.26609 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561de29de00 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.537+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835409c370 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f835c04b5a0 con 0x5561de28b370 2024-09-05T06:49:32.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 10 client.26609 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 10 client.26609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 20 client.26609 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de2a1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 4 client.26609 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561de1f3860 con 0x5561de29f310 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836259c640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de2a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f8349ffb640 10 client.26609 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.538+0000 7f836259c640 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de2a1790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db99bf9f27853ff server_cookie=25214681c2b34f51 in_seq=0 out_seq=0 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f8349ffb640 1 -- 172.21.15.86:0/1664222467 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561de1f3860 con 0x5561de29f310 2024-09-05T06:49:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f8349ffb640 10 client.26609 handle_command_reply: tid=0 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f836b1ca740 1 client.26609 shutdown 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 msgr2=0x5561de2884d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 0x5561de2884d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f835c02abe0 tx=0x7f835c02a6c0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 shutdown_connections 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.539+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de2a1790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f8349ffb640 0 client.26609 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de28b370 0x5561de2884d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de287f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1664222467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de28acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 >> 172.21.15.86:0/1664222467 conn(0x5561de249310 msgr2=0x5561de282930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 shutdown_connections 2024-09-05T06:49:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 1 -- 172.21.15.86:0/1664222467 wait complete. 2024-09-05T06:49:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.540+0000 7f836b1ca740 20 client.26609 trim_cache size 0 max 16384 2024-09-05T06:49:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:32 smithi096 ceph-mon[34468]: pgmap v255: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-05T06:49:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:32 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:49:32.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:32 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:49:32.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:32 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.554+0000 7f836b1ca740 1 Processor -- start 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.554+0000 7f836b1ca740 1 -- start start 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de27ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de281b40 0x5561de29b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 0x5561de287830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ad030 con 0x5561de28b370 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83641aeb70 con 0x5561de281b40 2024-09-05T06:49:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836b1ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ad700 con 0x5561de2805d0 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de27ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 0x5561de287830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de27ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:47320/0 (socket says 172.21.15.86:47320) 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f836259c640 1 -- 172.21.15.86:0/1836490620 learned_addr learned my addr 172.21.15.86:0/1836490620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8361d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de281b40 0x5561de29b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 -- 172.21.15.86:0/1836490620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 msgr2=0x5561de27ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de27ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 -- 172.21.15.86:0/1836490620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de281b40 msgr2=0x5561de29b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de281b40 0x5561de29b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 -- 172.21.15.86:0/1836490620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561de2864e0 con 0x5561de28b370 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.555+0000 7f8362d9d640 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 0x5561de287830 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f835c0099a0 tx=0x7f835c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b55dd07f6eeb2a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f834bfff640 1 -- 172.21.15.86:0/1836490620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c062020 con 0x5561de28b370 2024-09-05T06:49:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f834bfff640 1 -- 172.21.15.86:0/1836490620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835c037040 con 0x5561de28b370 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f834bfff640 1 -- 172.21.15.86:0/1836490620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c04f040 con 0x5561de28b370 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 -- 172.21.15.86:0/1836490620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 msgr2=0x5561de287830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 0x5561de287830 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f835c0099a0 tx=0x7f835c009480 comp rx=0 tx=0).stop 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 -- 172.21.15.86:0/1836490620 shutdown_connections 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de28b370 0x5561de287830 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de281b40 0x5561de29b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 --2- 172.21.15.86:0/1836490620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de2805d0 0x5561de27ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 -- 172.21.15.86:0/1836490620 >> 172.21.15.86:0/1836490620 conn(0x5561de249310 msgr2=0x5561de27c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 -- 172.21.15.86:0/1836490620 shutdown_connections 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.556+0000 7f836b1ca740 1 -- 172.21.15.86:0/1836490620 wait complete. 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 -- start start 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 Processor -- start 2024-09-05T06:49:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de285660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de285ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ae4a0 con 0x5561de281b40 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83641addd0 con 0x5561de2805d0 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83641ab030 con 0x5561de28b370 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836b1ca740 10 client.0 _ll_register_callbacks cb 0x7f835c042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55326/0 (socket says 172.21.15.86:55326) 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f836259c640 1 -- 172.21.15.86:0/165792422 learned_addr learned my addr 172.21.15.86:0/165792422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f8361d9b640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de285660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.557+0000 7f8362d9d640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de285ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 -- 172.21.15.86:0/165792422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 msgr2=0x5561de285ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de285ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 -- 172.21.15.86:0/165792422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 msgr2=0x5561de285660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de285660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 -- 172.21.15.86:0/165792422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561de309100 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836259c640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f835409bc00 tx=0x7f8354003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5118463adf9220 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f8349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540a7540 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c037040 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.558+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f8354099f70 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83540a7730 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 10 client.26615 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561de30a400 con 0x5561de2805d0 2024-09-05T06:49:32.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835c037040 con 0x5561de2805d0 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f83540cd120 con 0x5561de2805d0 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 10 client.26615 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 10 client.26615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 20 client.26615 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de30c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 4 client.26615 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561de1edd70 con 0x5561de29f310 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.559+0000 7f8361d9b640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de30c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f8361d9b640 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de30c690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=434db8b339fc4860 server_cookie=87ec2f32a66ed310 in_seq=0 out_seq=0 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f8349ffb640 10 client.26615 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f8349ffb640 1 -- 172.21.15.86:0/165792422 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561de1edd70 con 0x5561de29f310 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f8349ffb640 10 client.26615 handle_command_reply: tid=0 2024-09-05T06:49:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 client.26615 shutdown 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 msgr2=0x5561de285120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f835409bc00 tx=0x7f8354003e10 comp rx=0 tx=0).stop 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 shutdown_connections 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5561de29f310 0x5561de30c690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5561de28b370 0x5561de285ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f8349ffb640 0 client.26615 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561de281b40 0x5561de285660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 --2- 172.21.15.86:0/165792422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5561de2805d0 0x5561de285120 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 >> 172.21.15.86:0/165792422 conn(0x5561de249310 msgr2=0x5561de282110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 shutdown_connections 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.560+0000 7f836b1ca740 1 -- 172.21.15.86:0/165792422 wait complete. 2024-09-05T06:49:32.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f836b1ca740 20 client.26615 trim_cache size 0 max 16384 2024-09-05T06:49:32.570 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:49:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 msgr2=0x7f834c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 0x7f834c0a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f835c005f00 tx=0x7f835c005bf0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 msgr2=0x7f8364198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f835400e9c0 tx=0x7f835400ee80 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 shutdown_connections 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f834c0a2d70 0x7f834c0a5230 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8364106f10 0x7f836419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364102790 0x7f8364198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 --2- 172.21.15.86:0/2341574562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8364101de0 0x7f8364198320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 >> 172.21.15.86:0/2341574562 conn(0x7f83640fd8e0 msgr2=0x7f83640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:32.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.561+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 shutdown_connections 2024-09-05T06:49:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:32.562+0000 7f8368df1640 1 -- 172.21.15.86:0/2341574562 wait complete. 2024-09-05T06:49:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:32 smithi115 ceph-mon[34791]: pgmap v255: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-05T06:49:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:32 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:49:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:32 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:49:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:32 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:49:33.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:49:33.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 159487 1.92 MB/sec execute 293 sec latency 13.023 ms 2024-09-05T06:49:33.672 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:33.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:33 smithi086 ceph-mon[30943]: pgmap v256: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:33 smithi096 ceph-mon[34468]: pgmap v256: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:33 smithi115 ceph-mon[34791]: pgmap v256: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 -- 172.21.15.86:0/2490004793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad001027a0 msgr2=0x7fad001067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 --2- 172.21.15.86:0/2490004793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad001027a0 0x7fad001067f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7facf4009900 tx=0x7facf402a410 comp rx=0 tx=0).stop 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 -- 172.21.15.86:0/2490004793 shutdown_connections 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 --2- 172.21.15.86:0/2490004793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad00106f20 0x7fad0010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 --2- 172.21.15.86:0/2490004793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad001027a0 0x7fad001067f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 --2- 172.21.15.86:0/2490004793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 -- 172.21.15.86:0/2490004793 >> 172.21.15.86:0/2490004793 conn(0x7fad000fd8b0 msgr2=0x7fad000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 -- 172.21.15.86:0/2490004793 shutdown_connections 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.189+0000 7fad04837640 1 -- 172.21.15.86:0/2490004793 wait complete. 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 Processor -- start 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 -- start start 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad001027a0 0x7fad00074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 0x7fad00074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad00111e30 con 0x7fad00106f20 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad00111cb0 con 0x7fad001027a0 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7fad04837640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad00111fb0 con 0x7fad00101df0 2024-09-05T06:49:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7facfe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7facfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 0x7fad00074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7facfe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:47328/0 (socket says 172.21.15.86:47328) 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.190+0000 7facfe59c640 1 -- 172.21.15.86:0/2847324515 learned_addr learned my addr 172.21.15.86:0/2847324515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfdd9b640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad001027a0 0x7fad00074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 -- 172.21.15.86:0/2847324515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad001027a0 msgr2=0x7fad00074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad001027a0 0x7fad00074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 -- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 msgr2=0x7fad00074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 0x7fad00074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 -- 172.21.15.86:0/2847324515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad001a6800 con 0x7fad00101df0 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfed9d640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 0x7fad00074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facfe59c640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7facf000e9f0 tx=0x7facf000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facef7fe640 1 -- 172.21.15.86:0/2847324515 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf000cd80 con 0x7fad00101df0 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad001a6af0 con 0x7fad00101df0 2024-09-05T06:49:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facef7fe640 1 -- 172.21.15.86:0/2847324515 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf00054d0 con 0x7fad00101df0 2024-09-05T06:49:34.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7facef7fe640 1 -- 172.21.15.86:0/2847324515 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0010490 con 0x7fad00101df0 2024-09-05T06:49:34.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.191+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad001a7000 con 0x7fad00101df0 2024-09-05T06:49:34.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.194+0000 7facef7fe640 1 -- 172.21.15.86:0/2847324515 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7facf0005020 con 0x7fad00101df0 2024-09-05T06:49:34.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facef7fe640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 0x7face80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 Processor -- start 2024-09-05T06:49:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfdd9b640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 0x7face80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facef7fe640 1 -- 172.21.15.86:0/2847324515 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7facf00cfc50 con 0x7fad00101df0 2024-09-05T06:49:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 -- start start 2024-09-05T06:49:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c25d0 0x564c6c9c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 0x564c6c9cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c6c9282f0 con 0x564c6c9c3b40 2024-09-05T06:49:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c6c8cfcf0 con 0x564c6c9cd370 2024-09-05T06:49:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7fad06c10740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c6c9242b0 con 0x564c6c9c25d0 2024-09-05T06:49:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfdd9b640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 0x7face80a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facf4031440 tx=0x7facf4005b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfdd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c25d0 0x564c6c9c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37350/0 (socket says 172.21.15.86:37350) 2024-09-05T06:49:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 0x564c6c9cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.195+0000 7facfe59c640 1 -- 172.21.15.86:0/3633039371 learned_addr learned my addr 172.21.15.86:0/3633039371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 -- 172.21.15.86:0/3633039371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c25d0 msgr2=0x564c6c9c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c25d0 0x564c6c9c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 -- 172.21.15.86:0/3633039371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 msgr2=0x564c6c9cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 0x564c6c9cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 -- 172.21.15.86:0/3633039371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6c8b43a0 con 0x564c6c9c3b40 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfed9d640 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 0x564c6c9cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facfe59c640 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7facf00986a0 tx=0x7facf0005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53a7ecdd1cfdf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facecff9640 1 -- 172.21.15.86:0/3633039371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf009bcd0 con 0x564c6c9c3b40 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facecff9640 1 -- 172.21.15.86:0/3633039371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf00a0080 con 0x564c6c9c3b40 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.196+0000 7facecff9640 1 -- 172.21.15.86:0/3633039371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00a44b0 con 0x564c6c9c3b40 2024-09-05T06:49:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- 172.21.15.86:0/3633039371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 msgr2=0x564c6c9c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7facf00986a0 tx=0x7facf0005310 comp rx=0 tx=0).stop 2024-09-05T06:49:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- 172.21.15.86:0/3633039371 shutdown_connections 2024-09-05T06:49:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9cd370 0x564c6c9cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c25d0 0x564c6c9c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 --2- 172.21.15.86:0/3633039371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c3b40 0x564c6c9c2090 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- 172.21.15.86:0/3633039371 >> 172.21.15.86:0/3633039371 conn(0x564c6c98b310 msgr2=0x564c6c9bec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- 172.21.15.86:0/3633039371 shutdown_connections 2024-09-05T06:49:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- 172.21.15.86:0/3633039371 wait complete. 2024-09-05T06:49:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 -- start start 2024-09-05T06:49:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.197+0000 7fad06c10740 1 Processor -- start 2024-09-05T06:49:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c6c9d0070 con 0x564c6c9cd370 2024-09-05T06:49:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c6c8e3060 con 0x564c6c9c25d0 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c6c9d01f0 con 0x564c6c9c3b40 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 10 client.0 _ll_register_callbacks cb 0x564c6c6368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55368/0 (socket says 172.21.15.86:55368) 2024-09-05T06:49:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 -- 172.21.15.86:0/4081746893 learned_addr learned my addr 172.21.15.86:0/4081746893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 -- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 msgr2=0x564c6c9ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 -- 172.21.15.86:0/4081746893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 msgr2=0x564c6c9ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 -- 172.21.15.86:0/4081746893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c6c6993c0 con 0x564c6c9c25d0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7facfe59c640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7facf00a4410 tx=0x7facf00048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452ae0596f33f3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7face5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00a4940 con 0x564c6c9c25d0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7facf0099fd0 con 0x564c6c9c25d0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.198+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf00a0080 con 0x564c6c9c25d0 2024-09-05T06:49:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facf009bcd0 con 0x564c6c9c25d0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 10 client.26627 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c6c9dbb10 con 0x564c6c9c25d0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf00a0080 con 0x564c6c9c25d0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7facf00b3020 con 0x564c6c9c25d0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 10 client.26627 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 10 client.26627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 20 client.26627 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6c9dce60 0x564c6c9df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 4 client.26627 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c6c935860 con 0x564c6c9dce60 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.199+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6c9dce60 0x564c6c9df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7face5ffb640 10 client.26627 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6c9dce60 0x564c6c9df250 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef30bc7a948cf265 server_cookie=e97f04ce7e30d0d6 in_seq=0 out_seq=0 2024-09-05T06:49:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7face5ffb640 1 -- 172.21.15.86:0/4081746893 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c6c935860 con 0x564c6c9dce60 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7face5ffb640 10 client.26627 handle_command_reply: tid=0 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7fad06c10740 1 client.26627 shutdown 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 msgr2=0x564c6c9ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.200+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7facf00a4410 tx=0x7facf00048f0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 shutdown_connections 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6c9dce60 0x564c6c9df250 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 --2- 172.21.15.86:0/4081746893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9ccd60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 >> 172.21.15.86:0/4081746893 conn(0x564c6c98b310 msgr2=0x564c6c9c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7face5ffb640 0 client.26627 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 shutdown_connections 2024-09-05T06:49:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 1 -- 172.21.15.86:0/4081746893 wait complete. 2024-09-05T06:49:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.201+0000 7fad06c10740 20 client.26627 trim_cache size 0 max 16384 2024-09-05T06:49:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.214+0000 7fad06c10740 1 Processor -- start 2024-09-05T06:49:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.214+0000 7fad06c10740 1 -- start start 2024-09-05T06:49:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c25d0 0x564c6c9bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9cd370 0x564c6c9c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad001a8ac0 con 0x564c6c9c25d0 2024-09-05T06:49:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad001aa600 con 0x564c6c9c3b40 2024-09-05T06:49:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7fad06c10740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad001a9190 con 0x564c6c9cd370 2024-09-05T06:49:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9cd370 0x564c6c9c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55374/0 (socket says 172.21.15.86:55374) 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 -- 172.21.15.86:0/4063430491 learned_addr learned my addr 172.21.15.86:0/4063430491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfe59c640 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c25d0 0x564c6c9bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 -- 172.21.15.86:0/4063430491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9cd370 msgr2=0x564c6c9c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9cd370 0x564c6c9c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 -- 172.21.15.86:0/4063430491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c25d0 msgr2=0x564c6c9bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c25d0 0x564c6c9bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 -- 172.21.15.86:0/4063430491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6c9cff50 con 0x564c6c9c3b40 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.215+0000 7facfdd9b640 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facf4005c40 tx=0x7facf4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf7bc542efe7dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7face7fff640 1 -- 172.21.15.86:0/4063430491 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4035040 con 0x564c6c9c3b40 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7face7fff640 1 -- 172.21.15.86:0/4063430491 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf4058070 con 0x564c6c9c3b40 2024-09-05T06:49:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7face7fff640 1 -- 172.21.15.86:0/4063430491 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4004ad0 con 0x564c6c9c3b40 2024-09-05T06:49:34.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7fad06c10740 1 -- 172.21.15.86:0/4063430491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 msgr2=0x564c6c9c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7fad06c10740 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facf4005c40 tx=0x7facf4040000 comp rx=0 tx=0).stop 2024-09-05T06:49:34.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7fad06c10740 1 -- 172.21.15.86:0/4063430491 shutdown_connections 2024-09-05T06:49:34.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7fad06c10740 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9cd370 0x564c6c9c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.216+0000 7fad06c10740 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c3b40 0x564c6c9c5120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 --2- 172.21.15.86:0/4063430491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9c25d0 0x564c6c9bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- 172.21.15.86:0/4063430491 >> 172.21.15.86:0/4063430491 conn(0x564c6c98b310 msgr2=0x564c6c9cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- 172.21.15.86:0/4063430491 shutdown_connections 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- 172.21.15.86:0/4063430491 wait complete. 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- start start 2024-09-05T06:49:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 Processor -- start 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad001a9f30 con 0x564c6c9cd370 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad001a9860 con 0x564c6c9c25d0 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad00073ca0 con 0x564c6c9c3b40 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7fad06c10740 10 client.0 _ll_register_callbacks cb 0x7facf8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7facfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7facfe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7facfed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37374/0 (socket says 172.21.15.86:37374) 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7facfed9d640 1 -- 172.21.15.86:0/171669441 learned_addr learned my addr 172.21.15.86:0/171669441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.217+0000 7facfdd9b640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 -- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 msgr2=0x564c6c9e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 -- 172.21.15.86:0/171669441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 msgr2=0x564c6c9dfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9dfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 -- 172.21.15.86:0/171669441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c6ca46870 con 0x564c6c9cd370 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfdd9b640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfe59c640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9dfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7facfed9d640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7facf8004980 tx=0x7facf80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b51e72bcf11945 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7face5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8022020 con 0x564c6c9cd370 2024-09-05T06:49:34.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf4058070 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7facf8004d20 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facf802a050 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7fad06c10740 10 client.17409 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.218+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c6ca47bb0 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf4058070 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7facf80104c0 con 0x564c6c9cd370 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 10 client.17409 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 10 client.17409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 20 client.17409 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6ca483b0 0x564c6ca4a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 4 client.17409 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c6c92fd70 con 0x564c6ca483b0 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7facfe59c640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6ca483b0 0x564c6ca4a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7facfe59c640 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6ca483b0 0x564c6ca4a7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c403c1f8955de3c server_cookie=97ee2b954b28811a in_seq=0 out_seq=0 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.219+0000 7face5ffb640 10 client.17409 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7face5ffb640 1 -- 172.21.15.86:0/171669441 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c6c92fd70 con 0x564c6ca483b0 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7face5ffb640 10 client.17409 handle_command_reply: tid=0 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 client.17409 shutdown 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 msgr2=0x564c6c9e0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7facf8004980 tx=0x7facf80128e0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 shutdown_connections 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x564c6ca483b0 0x564c6ca4a7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c6c9cd370 0x564c6c9e0760 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c6c9c3b40 0x564c6c9e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7face5ffb640 0 client.17409 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 --2- 172.21.15.86:0/171669441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c6c9c25d0 0x564c6c9dfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 >> 172.21.15.86:0/171669441 conn(0x564c6c98b310 msgr2=0x564c6c9d8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 shutdown_connections 2024-09-05T06:49:34.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 1 -- 172.21.15.86:0/171669441 wait complete. 2024-09-05T06:49:34.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.220+0000 7fad06c10740 20 client.17409 trim_cache size 0 max 16384 2024-09-05T06:49:34.236 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 msgr2=0x7face80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 0x7face80a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facf4031440 tx=0x7facf4005b50 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 msgr2=0x7fad000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7facf000e9f0 tx=0x7facf000edf0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 shutdown_connections 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7face80a2cf0 0x7face80a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00106f20 0x7fad00074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad001027a0 0x7fad00074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 --2- 172.21.15.86:0/2847324515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad00101df0 0x7fad000744f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 >> 172.21.15.86:0/2847324515 conn(0x7fad000fd8b0 msgr2=0x7fad000fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 shutdown_connections 2024-09-05T06:49:34.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:34.221+0000 7fad04837640 1 -- 172.21.15.86:0/2847324515 wait complete. 2024-09-05T06:49:34.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 159959 1.93 MB/sec execute 294 sec latency 39.743 ms 2024-09-05T06:49:35.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:49:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:35.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 160260 1.93 MB/sec execute 295 sec latency 16.096 ms 2024-09-05T06:49:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:35 smithi086 ceph-mon[30943]: pgmap v257: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:35 smithi096 ceph-mon[34468]: pgmap v257: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:35.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 -- 172.21.15.86:0/3734594148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 msgr2=0x7f73d010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/3734594148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d010ec40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009cd0 tx=0x7f73c802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 -- 172.21.15.86:0/3734594148 shutdown_connections 2024-09-05T06:49:35.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/3734594148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d010ec40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/3734594148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 0x7f73d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/3734594148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 -- 172.21.15.86:0/3734594148 >> 172.21.15.86:0/3734594148 conn(0x7f73d00fc7d0 msgr2=0x7f73d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 -- 172.21.15.86:0/3734594148 shutdown_connections 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.834+0000 7f73d5bdf640 1 -- 172.21.15.86:0/3734594148 wait complete. 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.835+0000 7f73d5bdf640 1 Processor -- start 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.835+0000 7f73d5bdf640 1 -- start start 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.835+0000 7f73d5bdf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.835+0000 7f73d5bdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 0x7f73d01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111f80 con 0x7f73d01082a0 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111e00 con 0x7f73d0102d30 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0112100 con 0x7f73d01036e0 2024-09-05T06:49:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55410/0 (socket says 172.21.15.86:55410) 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37394/0 (socket says 172.21.15.86:37394) 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 -- 172.21.15.86:0/2053026497 learned_addr learned my addr 172.21.15.86:0/2053026497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73ceffd640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 0x7f73d01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 -- 172.21.15.86:0/2053026497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 msgr2=0x7f73d01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 0x7f73d01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 -- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 msgr2=0x7f73d019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 -- 172.21.15.86:0/2053026497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d019d430 con 0x7f73d0102d30 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73c000e9f0 tx=0x7f73c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73ccff9640 1 -- 172.21.15.86:0/2053026497 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c000cd80 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d019d720 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73ccff9640 1 -- 172.21.15.86:0/2053026497 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c00108e0 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73ccff9640 1 -- 172.21.15.86:0/2053026497 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0018880 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.836+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a5000 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73ccff9640 1 -- 172.21.15.86:0/2053026497 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f73c0010a90 con 0x7f73d0102d30 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 Processor -- start 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 -- start start 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e04999090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049995d0 0x556e04999a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e048ff2f0 con 0x556e049a4370 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e048a6cf0 con 0x556e0499ab40 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.839+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e048fb2b0 con 0x556e049995d0 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37404/0 (socket says 172.21.15.86:37404) 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 -- 172.21.15.86:0/1941771654 learned_addr learned my addr 172.21.15.86:0/1941771654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ccff9640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 0x7f73b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 -- 172.21.15.86:0/1941771654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049995d0 msgr2=0x556e04999a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73cffff640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e04999090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ccff9640 1 -- 172.21.15.86:0/2053026497 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f73c00cfd80 con 0x7f73d0102d30 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049995d0 0x556e04999a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049995d0 0x556e04999a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 -- 172.21.15.86:0/1941771654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 msgr2=0x556e04999090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e04999090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 -- 172.21.15.86:0/1941771654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0488b3a0 con 0x556e049a4370 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73cffff640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e04999090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73cffff640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 0x7f73b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73ceffd640 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009ba0 tx=0x7f73c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33ac1bb20221a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.840+0000 7f73b5ffb640 1 -- 172.21.15.86:0/1941771654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004230 con 0x556e049a4370 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73b5ffb640 1 -- 172.21.15.86:0/1941771654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c4009e10 con 0x556e049a4370 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73b5ffb640 1 -- 172.21.15.86:0/1941771654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004ca0 con 0x556e049a4370 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 -- 172.21.15.86:0/1941771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 msgr2=0x556e049a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73cffff640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 0x7f73b80a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73c8004af0 tx=0x7f73c80095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009ba0 tx=0x7f73c4011810 comp rx=0 tx=0).stop 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 -- 172.21.15.86:0/1941771654 shutdown_connections 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049a4370 0x556e049a6760 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049995d0 0x556e04999a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/1941771654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e04999090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 -- 172.21.15.86:0/1941771654 >> 172.21.15.86:0/1941771654 conn(0x556e04962310 msgr2=0x556e04995c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 -- 172.21.15.86:0/1941771654 shutdown_connections 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.841+0000 7f73d7fb8740 1 -- 172.21.15.86:0/1941771654 wait complete. 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 -- start start 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 Processor -- start 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e049a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e049a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e049a7070 con 0x556e049995d0 2024-09-05T06:49:35.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e048ba060 con 0x556e049a4370 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e049a71f0 con 0x556e0499ab40 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73d7fb8740 10 client.0 _ll_register_callbacks cb 0x556e0460d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37406/0 (socket says 172.21.15.86:37406) 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 -- 172.21.15.86:0/3685555786 learned_addr learned my addr 172.21.15.86:0/3685555786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cffff640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e049a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73ceffd640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e049a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 -- 172.21.15.86:0/3685555786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 msgr2=0x556e049a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e049a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 -- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 msgr2=0x556e049a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e049a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.842+0000 7f73cf7fe640 1 -- 172.21.15.86:0/3685555786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e046703c0 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f73c00cd090 tx=0x7f73c009b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c07111197c93ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0010630 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f73c009b250 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c4009e10 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73c00a01b0 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 10 client.17421 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e049b2a50 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c4009e10 con 0x556e049995d0 2024-09-05T06:49:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f73c009ad50 con 0x556e049995d0 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 10 client.17421 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 20 client.17421 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e049b3e60 0x556e049b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 4 client.17421 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.843+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e0490c860 con 0x556e049b3e60 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.844+0000 7f73ceffd640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e049b3e60 0x556e049b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.844+0000 7f73aeffd640 10 client.17421 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.844+0000 7f73ceffd640 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e049b3e60 0x556e049b62f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85416e458059c4d2 server_cookie=e80e744ef9b5cc01 in_seq=0 out_seq=0 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73aeffd640 1 -- 172.21.15.86:0/3685555786 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e0490c860 con 0x556e049b3e60 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73aeffd640 10 client.17421 handle_command_reply: tid=0 2024-09-05T06:49:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 client.17421 shutdown 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 msgr2=0x556e049a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f73c00cd090 tx=0x7f73c009b850 comp rx=0 tx=0).stop 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 shutdown_connections 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e049b3e60 0x556e049b62f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e049a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e049a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/3685555786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049a3cd0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 >> 172.21.15.86:0/3685555786 conn(0x556e04962310 msgr2=0x556e0499b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73aeffd640 0 client.17421 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 shutdown_connections 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 1 -- 172.21.15.86:0/3685555786 wait complete. 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.845+0000 7f73d7fb8740 20 client.17421 trim_cache size 0 max 16384 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 Processor -- start 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 -- start start 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e04995700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e04995c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01af210 con 0x556e049995d0 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01b0d50 con 0x556e049a4370 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.859+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01af8e0 con 0x556e0499ab40 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37420/0 (socket says 172.21.15.86:37420) 2024-09-05T06:49:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 -- 172.21.15.86:0/174021481 learned_addr learned my addr 172.21.15.86:0/174021481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73ceffd640 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e04995700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cffff640 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e04995c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 -- 172.21.15.86:0/174021481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 msgr2=0x556e04995700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e04995700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 -- 172.21.15.86:0/174021481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 msgr2=0x556e04995c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e04995c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 -- 172.21.15.86:0/174021481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e049954f0 con 0x556e049995d0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.860+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a4540 tx=0x7f73c009bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3845cee9fddae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.861+0000 7f73b57fa640 1 -- 172.21.15.86:0/174021481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00a1700 con 0x556e049995d0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.861+0000 7f73b57fa640 1 -- 172.21.15.86:0/174021481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c009aec0 con 0x556e049995d0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.861+0000 7f73b57fa640 1 -- 172.21.15.86:0/174021481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c009b540 con 0x556e049995d0 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- 172.21.15.86:0/174021481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 msgr2=0x556e049b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a4540 tx=0x7f73c009bf80 comp rx=0 tx=0).stop 2024-09-05T06:49:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- 172.21.15.86:0/174021481 shutdown_connections 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e049a4370 0x556e04995c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e0499ab40 0x556e04995700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/174021481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049b0310 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- 172.21.15.86:0/174021481 >> 172.21.15.86:0/174021481 conn(0x556e04962310 msgr2=0x556e049a04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- 172.21.15.86:0/174021481 shutdown_connections 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- 172.21.15.86:0/174021481 wait complete. 2024-09-05T06:49:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.862+0000 7f73d7fb8740 1 -- start start 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 Processor -- start 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e049a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 0x556e049a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01b0680 con 0x556e049995d0 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01affb0 con 0x556e0499ab40 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01ad210 con 0x556e049a4370 2024-09-05T06:49:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73d7fb8740 10 client.0 _ll_register_callbacks cb 0x7f73c0097980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37422/0 (socket says 172.21.15.86:37422) 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 0x556e049a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73cf7fe640 1 -- 172.21.15.86:0/948833963 learned_addr learned my addr 172.21.15.86:0/948833963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.863+0000 7f73ceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e049a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 -- 172.21.15.86:0/948833963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 msgr2=0x556e049a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 0x556e049a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 -- 172.21.15.86:0/948833963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 msgr2=0x556e049ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 -- 172.21.15.86:0/948833963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e049a8720 con 0x556e0499ab40 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73cffff640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 0x556e049a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73ceffd640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e049a7c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009b70 tx=0x7f73c4009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8f64f9b8213751 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c401e040 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7f73c4004180 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c009aec0 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73c4004380 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73d7fb8740 10 client.26648 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e04a1e8d0 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.864+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c009aec0 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f73c4018070 con 0x556e0499ab40 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 10 client.26648 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 20 client.26648 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e04a1ee30 0x556e04a21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 4 client.26648 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e04906d70 con 0x556e04a1ee30 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e04a1ee30 0x556e04a21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73aeffd640 10 client.26648 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.865+0000 7f73cf7fe640 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e04a1ee30 0x556e04a21250 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4125a05f16639b server_cookie=b228b6e2d55dca8e in_seq=0 out_seq=0 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73aeffd640 1 -- 172.21.15.86:0/948833963 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e04906d70 con 0x556e04a1ee30 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73aeffd640 10 client.26648 handle_command_reply: tid=0 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 client.26648 shutdown 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 msgr2=0x556e049a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e049a7c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009b70 tx=0x7f73c4009ea0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 shutdown_connections 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556e04a1ee30 0x556e04a21250 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e049a4370 0x556e049a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e0499ab40 0x556e049a7c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 --2- 172.21.15.86:0/948833963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e049995d0 0x556e049ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73aeffd640 0 client.26648 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 >> 172.21.15.86:0/948833963 conn(0x556e04962310 msgr2=0x556e0499f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 shutdown_connections 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 1 -- 172.21.15.86:0/948833963 wait complete. 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.866+0000 7f73d7fb8740 20 client.26648 trim_cache size 0 max 16384 2024-09-05T06:49:35.882 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 msgr2=0x7f73b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 0x7f73b80a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f73c8004af0 tx=0x7f73c80095c0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 msgr2=0x7f73d0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73c000e9f0 tx=0x7f73c000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 shutdown_connections 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f73b80a2cf0 0x7f73b80a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73d01082a0 0x7f73d019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d01036e0 0x7f73d01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 --2- 172.21.15.86:0/2053026497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102d30 0x7f73d0198480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 >> 172.21.15.86:0/2053026497 conn(0x7f73d00fc7d0 msgr2=0x7f73d0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 shutdown_connections 2024-09-05T06:49:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:35.867+0000 7f73d5bdf640 1 -- 172.21.15.86:0/2053026497 wait complete. 2024-09-05T06:49:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:35 smithi115 ceph-mon[34791]: pgmap v257: 129 pgs: 129 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-05T06:49:36.521 INFO:tasks.workunit.client.0.smithi086.stdout: 1 160795 1.93 MB/sec execute 296 sec latency 20.594 ms 2024-09-05T06:49:37.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 161307 1.93 MB/sec execute 297 sec latency 19.218 ms 2024-09-05T06:49:37.805 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:49:37.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:49:38.093 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:38.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:37 smithi086 ceph-mon[30943]: pgmap v258: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 937 KiB/s wr, 77 op/s 2024-09-05T06:49:38.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:37 smithi096 ceph-mon[34468]: pgmap v258: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 937 KiB/s wr, 77 op/s 2024-09-05T06:49:38.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:37 smithi115 ceph-mon[34791]: pgmap v258: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 937 KiB/s wr, 77 op/s 2024-09-05T06:49:38.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 161944 1.93 MB/sec execute 298 sec latency 19.416 ms 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 -- 172.21.15.86:0/549814285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14100a80 msgr2=0x7fba14100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/549814285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14100a80 0x7fba14100ee0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009930 tx=0x7fb9fc02a420 comp rx=0 tx=0).stop 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 -- 172.21.15.86:0/549814285 shutdown_connections 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/549814285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14101420 0x7fba1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/549814285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14100a80 0x7fba14100ee0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/549814285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 0x7fba14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 -- 172.21.15.86:0/549814285 >> 172.21.15.86:0/549814285 conn(0x7fba140fc7a0 msgr2=0x7fba140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 -- 172.21.15.86:0/549814285 shutdown_connections 2024-09-05T06:49:38.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.587+0000 7fba1aa2e640 1 -- 172.21.15.86:0/549814285 wait complete. 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 Processor -- start 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 -- start start 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 0x7fba140fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba14110d30 con 0x7fba14101420 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba14110bb0 con 0x7fba14106a80 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba1aa2e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba14110eb0 con 0x7fba14100a80 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:47436/0 (socket says 172.21.15.86:47436) 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37438/0 (socket says 172.21.15.86:37438) 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba18fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 0x7fba140fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 -- 172.21.15.86:0/974197766 learned_addr learned my addr 172.21.15.86:0/974197766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 -- 172.21.15.86:0/974197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 msgr2=0x7fba140fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 0x7fba140fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 -- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 msgr2=0x7fba140fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba13fff640 1 -- 172.21.15.86:0/974197766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba141a6840 con 0x7fba14100a80 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.588+0000 7fba0bfff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba13fff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba11ffb640 1 -- 172.21.15.86:0/974197766 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400cd80 con 0x7fba14100a80 2024-09-05T06:49:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba11ffb640 1 -- 172.21.15.86:0/974197766 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba040108e0 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba141a6b30 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba11ffb640 1 -- 172.21.15.86:0/974197766 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04018920 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.589+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141a7040 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.591+0000 7fba11ffb640 1 -- 172.21.15.86:0/974197766 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba04010a90 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.592+0000 7fba11ffb640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 0x7fba000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.592+0000 7fba0bfff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 0x7fba000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.592+0000 7fba11ffb640 1 -- 172.21.15.86:0/974197766 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fba040cfb20 con 0x7fba14100a80 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 Processor -- start 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 -- start start 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba0bfff640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 0x7fba000a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009900 tx=0x7fb9fc000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175aecf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a175a52280 con 0x55a175af7d40 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1759f9b10 con 0x55a175aed480 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba1ce07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a175a4e240 con 0x55a175aecb60 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55466/0 (socket says 172.21.15.86:55466) 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba0bfff640 1 -- 172.21.15.86:0/2217464501 learned_addr learned my addr 172.21.15.86:0/2217464501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba13fff640 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175aecf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 -- 172.21.15.86:0/2217464501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 msgr2=0x55a175aecf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175aecf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 -- 172.21.15.86:0/2217464501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 msgr2=0x55a175af7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.593+0000 7fba18fcb640 1 -- 172.21.15.86:0/2217464501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a175a061f0 con 0x55a175af7d40 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.594+0000 7fba18fcb640 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afa130 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009f00 tx=0x7fba0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d62206515030d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.594+0000 7fba0a7fc640 1 -- 172.21.15.86:0/2217464501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c004d20 con 0x55a175af7d40 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.594+0000 7fba0a7fc640 1 -- 172.21.15.86:0/2217464501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c004030 con 0x55a175af7d40 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.594+0000 7fba0a7fc640 1 -- 172.21.15.86:0/2217464501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c005260 con 0x55a175af7d40 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- 172.21.15.86:0/2217464501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 msgr2=0x55a175afa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afa130 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009f00 tx=0x7fba0c011810 comp rx=0 tx=0).stop 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- 172.21.15.86:0/2217464501 shutdown_connections 2024-09-05T06:49:38.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afa130 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 --2- 172.21.15.86:0/2217464501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175aecf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- 172.21.15.86:0/2217464501 >> 172.21.15.86:0/2217464501 conn(0x55a175ae90f0 msgr2=0x55a175ae94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- 172.21.15.86:0/2217464501 shutdown_connections 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- 172.21.15.86:0/2217464501 wait complete. 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.595+0000 7fba1ce07740 1 -- start start 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 Processor -- start 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175af4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175af5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a175afa910 con 0x55a175af7d40 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a175a0cf10 con 0x55a175aed480 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a175afaa90 con 0x55a175aecb60 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55470/0 (socket says 172.21.15.86:55470) 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba18fcb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175af5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba1ce07740 10 client.0 _ll_register_callbacks cb 0x55a1759d5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 -- 172.21.15.86:0/1133771927 learned_addr learned my addr 172.21.15.86:0/1133771927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba13fff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175af4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 -- 172.21.15.86:0/1133771927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 msgr2=0x55a175af4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175af4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 -- 172.21.15.86:0/1133771927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 msgr2=0x55a175af5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.596+0000 7fba0bfff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175af5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba0bfff640 1 -- 172.21.15.86:0/1133771927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1757c36a0 con 0x55a175aed480 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba13fff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175af4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba18fcb640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175af5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba0bfff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc031440 tx=0x7fb9fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7eda4924602eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fb9f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc035040 con 0x55a175aed480 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0c004030 con 0x55a175aed480 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fb9fc0055d0 con 0x55a175aed480 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc05b050 con 0x55a175aed480 2024-09-05T06:49:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba1ce07740 10 client.26660 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.597+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a175b06520 con 0x55a175aed480 2024-09-05T06:49:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.598+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c004030 con 0x55a175aed480 2024-09-05T06:49:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.598+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb9fc003a10 con 0x55a175aed480 2024-09-05T06:49:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 10 client.26660 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 20 client.26660 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b09e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 4 client.26660 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a175a5f7f0 con 0x55a175b079e0 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba13fff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b09e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fb9f37fe640 10 client.26660 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.599+0000 7fba13fff640 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b09e10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ea1b17f789c61a server_cookie=85256dc323fed77c in_seq=0 out_seq=0 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.600+0000 7fb9f37fe640 1 -- 172.21.15.86:0/1133771927 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a175a5f7f0 con 0x55a175b079e0 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.600+0000 7fb9f37fe640 10 client.26660 handle_command_reply: tid=0 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.600+0000 7fba1ce07740 1 client.26660 shutdown 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 msgr2=0x55a175af50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc031440 tx=0x7fb9fc009260 comp rx=0 tx=0).stop 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 shutdown_connections 2024-09-05T06:49:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b09e10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175af5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fb9f37fe640 0 client.26660 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175af50e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 --2- 172.21.15.86:0/1133771927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175af4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 >> 172.21.15.86:0/1133771927 conn(0x55a175ae90f0 msgr2=0x55a175ae8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 shutdown_connections 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 1 -- 172.21.15.86:0/1133771927 wait complete. 2024-09-05T06:49:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.601+0000 7fba1ce07740 20 client.26660 trim_cache size 0 max 16384 2024-09-05T06:49:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 Processor -- start 2024-09-05T06:49:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 -- start start 2024-09-05T06:49:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 0x55a175af6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aed480 0x55a175b0f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 0x55a175b0fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b1300 con 0x55a175aecb60 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b2e40 con 0x55a175af7d40 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba1ce07740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b19d0 con 0x55a175aed480 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba13fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 0x55a175af6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 0x55a175b0fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba13fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 0x55a175af6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37456/0 (socket says 172.21.15.86:37456) 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba13fff640 1 -- 172.21.15.86:0/4047706352 learned_addr learned my addr 172.21.15.86:0/4047706352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba0bfff640 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aed480 0x55a175b0f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 -- 172.21.15.86:0/4047706352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aed480 msgr2=0x55a175b0f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aed480 0x55a175b0f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 -- 172.21.15.86:0/4047706352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 msgr2=0x55a175af6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 0x55a175af6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.620+0000 7fba18fcb640 1 -- 172.21.15.86:0/4047706352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a175ae4830 con 0x55a175af7d40 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.621+0000 7fba18fcb640 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 0x55a175b0fc20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009270 tx=0x7fba0c0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ed6d636d9f538e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.621+0000 7fba09ffb640 1 -- 172.21.15.86:0/4047706352 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c027070 con 0x55a175af7d40 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.621+0000 7fba09ffb640 1 -- 172.21.15.86:0/4047706352 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c003ee0 con 0x55a175af7d40 2024-09-05T06:49:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.621+0000 7fba09ffb640 1 -- 172.21.15.86:0/4047706352 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c005040 con 0x55a175af7d40 2024-09-05T06:49:38.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- 172.21.15.86:0/4047706352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 msgr2=0x55a175b0fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 0x55a175b0fc20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009270 tx=0x7fba0c0044f0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- 172.21.15.86:0/4047706352 shutdown_connections 2024-09-05T06:49:38.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175af7d40 0x55a175b0fc20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aed480 0x55a175b0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 --2- 172.21.15.86:0/4047706352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175aecb60 0x55a175af6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- 172.21.15.86:0/4047706352 >> 172.21.15.86:0/4047706352 conn(0x55a175ae90f0 msgr2=0x55a175af62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- 172.21.15.86:0/4047706352 shutdown_connections 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- 172.21.15.86:0/4047706352 wait complete. 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 -- start start 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.622+0000 7fba1ce07740 1 Processor -- start 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175afb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba13fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175afb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b2770 con 0x55a175af7d40 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b20a0 con 0x55a175aed480 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba13fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:47464/0 (socket says 172.21.15.86:47464) 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba13fff640 1 -- 172.21.15.86:0/448824090 learned_addr learned my addr 172.21.15.86:0/448824090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fba141af300 con 0x55a175aecb60 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba18fcb640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba1ce07740 10 client.0 _ll_register_callbacks cb 0x7fba0c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 -- 172.21.15.86:0/448824090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 msgr2=0x55a175b003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 -- 172.21.15.86:0/448824090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 msgr2=0x55a175afbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:38.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.623+0000 7fba0bfff640 1 -- 172.21.15.86:0/448824090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a175aed2b0 con 0x55a175aed480 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba0bfff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175afb770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc003db0 tx=0x7fb9fc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da5a3ec54c4ce85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba13fff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fb9f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba18fcb640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc035050 con 0x55a175aed480 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fb9fc0055e0 con 0x55a175aed480 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0c003ee0 con 0x55a175aed480 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc06d050 con 0x55a175aed480 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba1ce07740 10 client.26666 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a175b724a0 con 0x55a175aed480 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.624+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c003ee0 con 0x55a175aed480 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb9fc0035f0 con 0x55a175aed480 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 10 client.26666 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 10 client.26666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 20 client.26666 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b74cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 4 client.26666 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a175a59d00 con 0x55a175b079e0 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.625+0000 7fba13fff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b74cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.626+0000 7fba13fff640 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b74cb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a71bb4597dc6c7 server_cookie=4ffc0ce499760031 in_seq=0 out_seq=0 2024-09-05T06:49:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.626+0000 7fb9f37fe640 10 client.26666 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fb9f37fe640 1 -- 172.21.15.86:0/448824090 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a175a59d00 con 0x55a175b079e0 2024-09-05T06:49:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fb9f37fe640 10 client.26666 handle_command_reply: tid=0 2024-09-05T06:49:38.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 client.26666 shutdown 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 msgr2=0x55a175afb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175afb770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc003db0 tx=0x7fb9fc005d50 comp rx=0 tx=0).stop 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 shutdown_connections 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a175b079e0 0x55a175b74cb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a175af7d40 0x55a175afbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a175aed480 0x55a175afb770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 --2- 172.21.15.86:0/448824090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a175aecb60 0x55a175b003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 >> 172.21.15.86:0/448824090 conn(0x55a175ae90f0 msgr2=0x55a175ae8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fb9f37fe640 0 client.26666 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 shutdown_connections 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 1 -- 172.21.15.86:0/448824090 wait complete. 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.627+0000 7fba1ce07740 20 client.26666 trim_cache size 0 max 16384 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e94ea73b-87da-4da1-8817-5584fbaadc5f", 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:49:38.633 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 msgr2=0x7fba000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 0x7fba000a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009900 tx=0x7fb9fc000bc0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 msgr2=0x7fba140fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 shutdown_connections 2024-09-05T06:49:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fba000a2cf0 0x7fba000a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba14106a80 0x7fba140fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14101420 0x7fba140fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 --2- 172.21.15.86:0/974197766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba14100a80 0x7fba140fce00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.628+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 >> 172.21.15.86:0/974197766 conn(0x7fba140fc7a0 msgr2=0x7fba1406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.629+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 shutdown_connections 2024-09-05T06:49:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:38.629+0000 7fba1aa2e640 1 -- 172.21.15.86:0/974197766 wait complete. 2024-09-05T06:49:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:38 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:49:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:38 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:38 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:49:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:38 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:38 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:49:39.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:38 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:38 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:49:39.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:38 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:38 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:49:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:38 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:38 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:49:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:38 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:39.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 162565 1.94 MB/sec execute 299 sec latency 14.014 ms 2024-09-05T06:49:39.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:49:39.901 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:39 smithi086 ceph-mon[30943]: pgmap v259: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 932 KiB/s wr, 71 op/s 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:39 smithi096 ceph-mon[34468]: pgmap v259: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 932 KiB/s wr, 71 op/s 2024-09-05T06:49:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.398+0000 7fb17455b640 1 -- 172.21.15.86:0/3112513736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 msgr2=0x7fb16c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.398+0000 7fb17455b640 1 --2- 172.21.15.86:0/3112513736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c10d950 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb164009cd0 tx=0x7fb16402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- 172.21.15.86:0/3112513736 shutdown_connections 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- 172.21.15.86:0/3112513736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c10d950 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- 172.21.15.86:0/3112513736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- 172.21.15.86:0/3112513736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 0x7fb16c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- 172.21.15.86:0/3112513736 >> 172.21.15.86:0/3112513736 conn(0x7fb16c0fc7a0 msgr2=0x7fb16c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- 172.21.15.86:0/3112513736 shutdown_connections 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- 172.21.15.86:0/3112513736 wait complete. 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 Processor -- start 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- start start 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 0x7fb16c1a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110d30 con 0x7fb16c101420 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110bb0 con 0x7fb16c100a80 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.399+0000 7fb17455b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110eb0 con 0x7fb16c106a80 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb1722f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37492/0 (socket says 172.21.15.86:37492) 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 -- 172.21.15.86:0/2704718565 learned_addr learned my addr 172.21.15.86:0/2704718565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb172af8640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 0x7fb16c1a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 -- 172.21.15.86:0/2704718565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 msgr2=0x7fb16c1a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 0x7fb16c1a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 -- 172.21.15.86:0/2704718565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 msgr2=0x7fb16c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 -- 172.21.15.86:0/2704718565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1a4780 con 0x7fb16c101420 2024-09-05T06:49:40.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb1722f7640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb171af6640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d950 tx=0x7fb15c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb1637fe640 1 -- 172.21.15.86:0/2704718565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c017070 con 0x7fb16c101420 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c1a4950 con 0x7fb16c101420 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb1637fe640 1 -- 172.21.15.86:0/2704718565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb15c009c30 con 0x7fb16c101420 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb1637fe640 1 -- 172.21.15.86:0/2704718565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00f620 con 0x7fb16c101420 2024-09-05T06:49:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.400+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1a4e60 con 0x7fb16c101420 2024-09-05T06:49:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.403+0000 7fb1637fe640 1 -- 172.21.15.86:0/2704718565 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb15c0109a0 con 0x7fb16c101420 2024-09-05T06:49:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.403+0000 7fb176934740 1 Processor -- start 2024-09-05T06:49:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.403+0000 7fb176934740 1 -- start start 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 0x564f6a840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 0x564f6a84b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a84b680 0x564f6a84da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f6a7a62e0 con 0x564f6a8404a0 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f6a74dce0 con 0x564f6a840dc0 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb176934740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f6a7a22a0 con 0x564f6a84b680 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb172af8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 0x564f6a840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1637fe640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 0x7fb1580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 0x564f6a84b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb172af8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 0x564f6a840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37496/0 (socket says 172.21.15.86:37496) 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb171af6640 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a84b680 0x564f6a84da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb172af8640 1 -- 172.21.15.86:0/4136704923 learned_addr learned my addr 172.21.15.86:0/4136704923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb172af8640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 0x7fb1580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1637fe640 1 -- 172.21.15.86:0/2704718565 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fb15c0cf430 con 0x7fb16c101420 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 -- 172.21.15.86:0/4136704923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a84b680 msgr2=0x564f6a84da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a84b680 0x564f6a84da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 -- 172.21.15.86:0/4136704923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 msgr2=0x564f6a840880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 0x564f6a840880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.404+0000 7fb1722f7640 1 -- 172.21.15.86:0/4136704923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6a732440 con 0x564f6a840dc0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.405+0000 7fb1722f7640 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 0x564f6a84b140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb16800bf70 tx=0x7fb1680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1977226ed30f1cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.405+0000 7fb172af8640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 0x7fb1580a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb164009a60 tx=0x7fb16402a8e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.405+0000 7fb160ff9640 1 -- 172.21.15.86:0/4136704923 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16801c680 con 0x564f6a840dc0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.405+0000 7fb160ff9640 1 -- 172.21.15.86:0/4136704923 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb16801ccd0 con 0x564f6a840dc0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.405+0000 7fb160ff9640 1 -- 172.21.15.86:0/4136704923 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168011dd0 con 0x564f6a840dc0 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- 172.21.15.86:0/4136704923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 msgr2=0x564f6a84b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 0x564f6a84b140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb16800bf70 tx=0x7fb1680107a0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- 172.21.15.86:0/4136704923 shutdown_connections 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a84b680 0x564f6a84da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a840dc0 0x564f6a84b140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 --2- 172.21.15.86:0/4136704923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a8404a0 0x564f6a840880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- 172.21.15.86:0/4136704923 >> 172.21.15.86:0/4136704923 conn(0x564f6a83ca30 msgr2=0x564f6a83ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- 172.21.15.86:0/4136704923 shutdown_connections 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- 172.21.15.86:0/4136704923 wait complete. 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 -- start start 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.406+0000 7fb176934740 1 Processor -- start 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a84af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 0x564f6a848780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f6a852660 con 0x564f6a840dc0 2024-09-05T06:49:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f6a761050 con 0x564f6a84b680 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f6a8527e0 con 0x564f6a8404a0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37512/0 (socket says 172.21.15.86:37512) 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 -- 172.21.15.86:0/3940880075 learned_addr learned my addr 172.21.15.86:0/3940880075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb1722f7640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a84af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb172af8640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 0x564f6a848780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb176934740 10 client.0 _ll_register_callbacks cb 0x564f6a7a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 -- 172.21.15.86:0/3940880075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 msgr2=0x564f6a84af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a84af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 -- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 msgr2=0x564f6a848780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 0x564f6a848780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 -- 172.21.15.86:0/3940880075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6a5173c0 con 0x564f6a840dc0 2024-09-05T06:49:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb1722f7640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a84af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb172af8640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 0x564f6a848780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb171af6640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0cc7a0 tx=0x7fb15c0860f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f815656ca9ca16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.407+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c020070 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fb15c0cc9c0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16801ccd0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb15c09b5c0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 10 client.17457 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f6a85e0d0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb16801ccd0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb15c0a3cf0 con 0x564f6a840dc0 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 10 client.17457 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 20 client.17457 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a85f590 0x564f6a861980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 4 client.17457 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.408+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f6a7b3850 con 0x564f6a85f590 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.409+0000 7fb1722f7640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a85f590 0x564f6a861980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.409+0000 7fb1722f7640 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a85f590 0x564f6a861980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887132605d82073f server_cookie=8a871d682f1d11b3 in_seq=0 out_seq=0 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.409+0000 7fb155ffb640 10 client.17457 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb155ffb640 1 -- 172.21.15.86:0/3940880075 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f6a7b3850 con 0x564f6a85f590 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb155ffb640 10 client.17457 handle_command_reply: tid=0 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 client.17457 shutdown 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 msgr2=0x564f6a848240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0cc7a0 tx=0x7fb15c0860f0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 shutdown_connections 2024-09-05T06:49:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a85f590 0x564f6a861980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a84b680 0x564f6a848780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a848240 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 --2- 172.21.15.86:0/3940880075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a84af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 >> 172.21.15.86:0/3940880075 conn(0x564f6a83ca30 msgr2=0x564f6a842760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb155ffb640 0 client.17457 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 shutdown_connections 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 1 -- 172.21.15.86:0/3940880075 wait complete. 2024-09-05T06:49:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.410+0000 7fb176934740 20 client.17457 trim_cache size 0 max 16384 2024-09-05T06:49:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.427+0000 7fb176934740 1 Processor -- start 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.427+0000 7fb176934740 1 -- start start 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a85b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a84a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c073ab0 con 0x564f6a840dc0 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a6260 con 0x564f6a83ca30 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb176934740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c074180 con 0x564f6a8404a0 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb172af8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a84a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb1722f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a85b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:47538/0 (socket says 172.21.15.86:47538) 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 -- 172.21.15.86:0/1578610602 learned_addr learned my addr 172.21.15.86:0/1578610602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 -- 172.21.15.86:0/1578610602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 msgr2=0x564f6a85b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a85b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 -- 172.21.15.86:0/1578610602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 msgr2=0x564f6a84a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a84a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 -- 172.21.15.86:0/1578610602 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6a8471f0 con 0x564f6a8404a0 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb172af8640 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a84a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb171af6640 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b780 tx=0x7fb15c097860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac14e4846f027f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb157fff640 1 -- 172.21.15.86:0/1578610602 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c014020 con 0x564f6a8404a0 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb157fff640 1 -- 172.21.15.86:0/1578610602 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb15c09f8c0 con 0x564f6a8404a0 2024-09-05T06:49:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.428+0000 7fb157fff640 1 -- 172.21.15.86:0/1578610602 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c020070 con 0x564f6a8404a0 2024-09-05T06:49:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- 172.21.15.86:0/1578610602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 msgr2=0x564f6a846b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b780 tx=0x7fb15c097860 comp rx=0 tx=0).stop 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- 172.21.15.86:0/1578610602 shutdown_connections 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a84a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a846b10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 --2- 172.21.15.86:0/1578610602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a85b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- 172.21.15.86:0/1578610602 >> 172.21.15.86:0/1578610602 conn(0x564f6a8488e0 msgr2=0x564f6a849aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- 172.21.15.86:0/1578610602 shutdown_connections 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- 172.21.15.86:0/1578610602 wait complete. 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.429+0000 7fb176934740 1 -- start start 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 Processor -- start 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a8540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a8c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a5b90 con 0x564f6a840dc0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a54c0 con 0x564f6a83ca30 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c071ab0 con 0x564f6a8404a0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb176934740 10 client.0 _ll_register_callbacks cb 0x7fb15c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55550/0 (socket says 172.21.15.86:55550) 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 -- 172.21.15.86:0/23780018 learned_addr learned my addr 172.21.15.86:0/23780018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb172af8640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a8c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb171af6640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a8540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 -- 172.21.15.86:0/23780018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 msgr2=0x564f6a8540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a8540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 -- 172.21.15.86:0/23780018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 msgr2=0x564f6a8c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a8c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 -- 172.21.15.86:0/23780018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6a8c96c0 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb172af8640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a8c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb1722f7640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x564f6a83d890 tx=0x7fb168005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d816a7920e60813 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.430+0000 7fb155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16802c030 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c09f8c0 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fb168011a80 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb168039050 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 10 client.26684 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f6a8ca880 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb15c09f8c0 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb16801c4a0 con 0x564f6a83ca30 2024-09-05T06:49:40.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 10 client.26684 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 20 client.26684 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a84b680 0x564f6a8cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 4 client.26684 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.431+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f6a7add60 con 0x564f6a84b680 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb171af6640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a84b680 0x564f6a8cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb155ffb640 10 client.26684 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb171af6640 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a84b680 0x564f6a8cce80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec1e6469ce553ea server_cookie=f4d0c8661ae9ecbd in_seq=0 out_seq=0 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb155ffb640 1 -- 172.21.15.86:0/23780018 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f6a7add60 con 0x564f6a84b680 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb155ffb640 10 client.26684 handle_command_reply: tid=0 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 client.26684 shutdown 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 msgr2=0x564f6a853ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x564f6a83d890 tx=0x7fb168005bb0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 shutdown_connections 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f6a84b680 0x564f6a8cce80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f6a840dc0 0x564f6a8c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f6a8404a0 0x564f6a8540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb155ffb640 0 client.26684 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 --2- 172.21.15.86:0/23780018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f6a83ca30 0x564f6a853ba0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 >> 172.21.15.86:0/23780018 conn(0x564f6a8488e0 msgr2=0x564f6a84a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 shutdown_connections 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.432+0000 7fb176934740 1 -- 172.21.15.86:0/23780018 wait complete. 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb176934740 20 client.26684 trim_cache size 0 max 16384 2024-09-05T06:49:40.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "e94ea73b-87da-4da1-8817-5584fbaadc5f": { 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e94ea73b-87da-4da1-8817-5584fbaadc5f", 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:40.439 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 msgr2=0x7fb1580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 0x7fb1580a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb164009a60 tx=0x7fb16402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 msgr2=0x7fb16c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d950 tx=0x7fb15c00de10 comp rx=0 tx=0).stop 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 shutdown_connections 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb1580a2cf0 0x7fb1580a51b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb16c106a80 0x7fb16c1a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb16c101420 0x7fb16c199700 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 --2- 172.21.15.86:0/2704718565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb16c100a80 0x7fb16c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:40.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 >> 172.21.15.86:0/2704718565 conn(0x7fb16c0fc7a0 msgr2=0x7fb16c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:40.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 shutdown_connections 2024-09-05T06:49:40.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:40.433+0000 7fb17455b640 1 -- 172.21.15.86:0/2704718565 wait complete. 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:49:40.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:49:40.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:39 smithi115 ceph-mon[34791]: pgmap v259: 129 pgs: 129 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 932 KiB/s wr, 71 op/s 2024-09-05T06:49:40.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 163035 1.94 MB/sec execute 300 sec latency 37.644 ms 2024-09-05T06:49:41.414 INFO:tasks.cephfs.filesystem:scrub status for tag:e94ea73b-87da-4da1-8817-5584fbaadc5f - {'path': '/', 'tag': 'e94ea73b-87da-4da1-8817-5584fbaadc5f', 'options': 'recursive,force'} 2024-09-05T06:49:41.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 163525 1.95 MB/sec execute 301 sec latency 13.946 ms 2024-09-05T06:49:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:41 smithi086 ceph-mon[30943]: pgmap v260: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-05T06:49:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:41 smithi096 ceph-mon[34468]: pgmap v260: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-05T06:49:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:41 smithi115 ceph-mon[34791]: pgmap v260: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-05T06:49:42.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 163851 1.94 MB/sec execute 302 sec latency 16.065 ms 2024-09-05T06:49:43.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 164391 1.94 MB/sec execute 303 sec latency 13.386 ms 2024-09-05T06:49:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:43 smithi096 ceph-mon[34468]: pgmap v261: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:43 smithi115 ceph-mon[34791]: pgmap v261: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:43 smithi086 ceph-mon[30943]: pgmap v261: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:44.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 164962 1.95 MB/sec execute 304 sec latency 21.363 ms 2024-09-05T06:49:45.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 165555 1.95 MB/sec execute 305 sec latency 21.196 ms 2024-09-05T06:49:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:45 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:49:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:45 smithi096 ceph-mon[34468]: pgmap v262: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:45 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:49:45.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:45 smithi115 ceph-mon[34791]: pgmap v262: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:45.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:45 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:49:45.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:45 smithi086 ceph-mon[30943]: pgmap v262: 129 pgs: 129 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:49:46.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 166189 1.96 MB/sec execute 306 sec latency 13.714 ms 2024-09-05T06:49:46.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:46 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:49:46.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:46 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:49:46.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:46 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:49:46.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:46 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:49:46.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:46 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:49:46.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:46 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:49:47.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 166740 1.97 MB/sec execute 307 sec latency 36.231 ms 2024-09-05T06:49:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:47 smithi096 ceph-mon[34468]: pgmap v263: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-05T06:49:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:47 smithi115 ceph-mon[34791]: pgmap v263: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-05T06:49:47.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:47 smithi086 ceph-mon[30943]: pgmap v263: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-05T06:49:48.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 167110 1.97 MB/sec execute 308 sec latency 13.234 ms 2024-09-05T06:49:49.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 167441 1.96 MB/sec execute 309 sec latency 19.862 ms 2024-09-05T06:49:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:49 smithi096 ceph-mon[34468]: pgmap v264: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:49:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:49 smithi115 ceph-mon[34791]: pgmap v264: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:49:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:49 smithi086 ceph-mon[30943]: pgmap v264: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:49:50.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 167974 1.96 MB/sec execute 310 sec latency 15.398 ms 2024-09-05T06:49:51.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 168541 1.97 MB/sec execute 311 sec latency 24.004 ms 2024-09-05T06:49:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:51 smithi096 ceph-mon[34468]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:51 smithi115 ceph-mon[34791]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:51.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:51 smithi086 ceph-mon[30943]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-09-05T06:49:52.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 169140 1.97 MB/sec execute 312 sec latency 19.689 ms 2024-09-05T06:49:53.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 169902 1.98 MB/sec execute 313 sec latency 14.537 ms 2024-09-05T06:49:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:53 smithi096 ceph-mon[34468]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:53.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:53 smithi115 ceph-mon[34791]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:53.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:53 smithi086 ceph-mon[30943]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:54.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 170358 1.99 MB/sec execute 314 sec latency 13.380 ms 2024-09-05T06:49:55.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 170684 1.99 MB/sec execute 315 sec latency 15.048 ms 2024-09-05T06:49:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:55 smithi096 ceph-mon[34468]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:55.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:55 smithi115 ceph-mon[34791]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:55.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:55 smithi086 ceph-mon[30943]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-05T06:49:56.368 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T06:49:56.368 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T06:49:56.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:49:56.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 171062 1.98 MB/sec execute 316 sec latency 17.044 ms 2024-09-05T06:49:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:57.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.127+0000 7ff6d86b0640 1 -- 172.21.15.86:0/1410883164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d01082a0 msgr2=0x7ff6d010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.127+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/1410883164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d01082a0 0x7ff6d010ec40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009cd0 tx=0x7ff6c802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 -- 172.21.15.86:0/1410883164 shutdown_connections 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/1410883164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d01082a0 0x7ff6d010ec40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/1410883164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01036e0 0x7ff6d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/1410883164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d0102d30 0x7ff6d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 -- 172.21.15.86:0/1410883164 >> 172.21.15.86:0/1410883164 conn(0x7ff6d00fc7d0 msgr2=0x7ff6d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 -- 172.21.15.86:0/1410883164 shutdown_connections 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 -- 172.21.15.86:0/1410883164 wait complete. 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.128+0000 7ff6d86b0640 1 Processor -- start 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 -- start start 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:57.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d01036e0 0x7ff6d01a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01082a0 0x7ff6d010dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0111f50 con 0x7ff6d0102d30 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0111dd0 con 0x7ff6d01082a0 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d86b0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01120d0 con 0x7ff6d01036e0 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46682/0 (socket says 172.21.15.86:46682) 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d5c4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d01036e0 0x7ff6d01a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 -- 172.21.15.86:0/434988406 learned_addr learned my addr 172.21.15.86:0/434988406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d6c4d640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01082a0 0x7ff6d010dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 -- 172.21.15.86:0/434988406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d01036e0 msgr2=0x7ff6d01a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d01036e0 0x7ff6d01a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 -- 172.21.15.86:0/434988406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01082a0 msgr2=0x7ff6d010dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01082a0 0x7ff6d010dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 -- 172.21.15.86:0/434988406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d010e4f0 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.129+0000 7ff6d644c640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c920 tx=0x7ff6cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.130+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc007c70 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.130+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6cc007e10 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.130+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d010e7e0 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.130+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc018820 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.130+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01ab550 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.131+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d0103110 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.133+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff6cc0189c0 con 0x7ff6d0102d30 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.133+0000 7ff6c77fe640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 0x7ff6bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.134+0000 7ff6d5c4b640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 0x7ff6bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.134+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7ff6cc0cfb70 con 0x7ff6d0102d30 2024-09-05T06:49:57.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.134+0000 7ff6d5c4b640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 0x7ff6bc0a5230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6d010f0a0 tx=0x7ff6c0005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:57.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.137+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6cc014030 con 0x7ff6d0102d30 2024-09-05T06:49:57.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.384+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6d0107270 con 0x7ff6d0102d30 2024-09-05T06:49:57.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.385+0000 7ff6c77fe640 1 -- 172.21.15.86:0/434988406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7ff6cc09c3d0 con 0x7ff6d0102d30 2024-09-05T06:49:57.385 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:49:57.386 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:49:57.386 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:49:57.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 msgr2=0x7ff6bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:57.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 0x7ff6bc0a5230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6d010f0a0 tx=0x7ff6c0005f90 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 msgr2=0x7ff6d01a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c920 tx=0x7ff6cc00cde0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 shutdown_connections 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6bc0a2d70 0x7ff6bc0a5230 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6d01082a0 0x7ff6d010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d01036e0 0x7ff6d01a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 --2- 172.21.15.86:0/434988406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d0102d30 0x7ff6d01a3620 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 >> 172.21.15.86:0/434988406 conn(0x7ff6d00fc7d0 msgr2=0x7ff6d00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 shutdown_connections 2024-09-05T06:49:57.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:57.387+0000 7ff6d86b0640 1 -- 172.21.15.86:0/434988406 wait complete. 2024-09-05T06:49:57.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 171558 1.98 MB/sec execute 317 sec latency 20.122 ms 2024-09-05T06:49:57.699 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:57 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/434988406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:57 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/434988406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:57 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/434988406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:49:58.226 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T06:49:58.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T06:49:58.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:58 smithi086 ceph-mon[30943]: pgmap v268: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-09-05T06:49:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:49:58.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172105 1.98 MB/sec execute 318 sec latency 21.296 ms 2024-09-05T06:49:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:58 smithi096 ceph-mon[34468]: pgmap v268: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-09-05T06:49:58.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:58 smithi115 ceph-mon[34791]: pgmap v268: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-09-05T06:49:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- 172.21.15.86:0/383830508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8106f20 msgr2=0x7fa0c810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/383830508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8106f20 0x7fa0c810eaa0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009ca0 tx=0x7fa0c002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- 172.21.15.86:0/383830508 shutdown_connections 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/383830508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8106f20 0x7fa0c810eaa0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/383830508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c81027a0 0x7fa0c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/383830508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c8101df0 0x7fa0c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- 172.21.15.86:0/383830508 >> 172.21.15.86:0/383830508 conn(0x7fa0c80fd8f0 msgr2=0x7fa0c80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- 172.21.15.86:0/383830508 shutdown_connections 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- 172.21.15.86:0/383830508 wait complete. 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 Processor -- start 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.049+0000 7fa0cf99d640 1 -- start start 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c81027a0 0x7fa0c8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 0x7fa0c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111e30 con 0x7fa0c8101df0 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111cb0 con 0x7fa0c8106f20 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cf99d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111fb0 con 0x7fa0c81027a0 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46702/0 (socket says 172.21.15.86:46702) 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 -- 172.21.15.86:0/3957284290 learned_addr learned my addr 172.21.15.86:0/3957284290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 0x7fa0c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c81027a0 0x7fa0c8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 -- 172.21.15.86:0/3957284290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c81027a0 msgr2=0x7fa0c8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c81027a0 0x7fa0c8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 -- 172.21.15.86:0/3957284290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 msgr2=0x7fa0c819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 0x7fa0c819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cd739640 1 -- 172.21.15.86:0/3957284290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819d2e0 con 0x7fa0c8101df0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.050+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 0x7fa0c819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0cd739640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d950 tx=0x7fa0c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0be7fc640 1 -- 172.21.15.86:0/3957284290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4017030 con 0x7fa0c8101df0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c819d5d0 con 0x7fa0c8101df0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0be7fc640 1 -- 172.21.15.86:0/3957284290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c4004020 con 0x7fa0c8101df0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0be7fc640 1 -- 172.21.15.86:0/3957284290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40047a0 con 0x7fa0c8101df0 2024-09-05T06:49:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.051+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a4eb0 con 0x7fa0c8101df0 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.053+0000 7fa0be7fc640 1 -- 172.21.15.86:0/3957284290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa0c40041d0 con 0x7fa0c8101df0 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 Processor -- start 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 -- start start 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6acd0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c6ac2b2a0 con 0x561c6acc5760 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c6abd2b10 con 0x561c6acd0940 2024-09-05T06:49:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c6ac27260 con 0x561c6acc6080 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0be7fc640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 0x7fa0b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0cdf3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0cdf3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46710/0 (socket says 172.21.15.86:46710) 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.054+0000 7fa0cdf3a640 1 -- 172.21.15.86:0/1839183920 learned_addr learned my addr 172.21.15.86:0/1839183920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0be7fc640 1 -- 172.21.15.86:0/3957284290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fa0c40cf320 con 0x7fa0c8101df0 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 0x7fa0b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0cd739640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6acd0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 -- 172.21.15.86:0/1839183920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 msgr2=0x561c6acd0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6acd0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 -- 172.21.15.86:0/1839183920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 msgr2=0x561c6acc5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 -- 172.21.15.86:0/1839183920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6a99cb90 con 0x561c6acd0940 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd2d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009ac0 tx=0x7fa0b8009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9556154f37cc3c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 0x7fa0b40a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00319e0 tx=0x7fa0c002a8c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0b37fe640 1 -- 172.21.15.86:0/1839183920 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011da0 con 0x561c6acd0940 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0b37fe640 1 -- 172.21.15.86:0/1839183920 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0b8016070 con 0x561c6acd0940 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.055+0000 7fa0b37fe640 1 -- 172.21.15.86:0/1839183920 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8012450 con 0x561c6acd0940 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1839183920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 msgr2=0x561c6acd2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd2d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009ac0 tx=0x7fa0b8009f80 comp rx=0 tx=0).stop 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1839183920 shutdown_connections 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd2d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6acd0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1839183920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6acc5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1839183920 >> 172.21.15.86:0/1839183920 conn(0x561c6acc1cf0 msgr2=0x561c6acc20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1839183920 shutdown_connections 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.056+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1839183920 wait complete. 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 -- start start 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 Processor -- start 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6accdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acd0940 0x561c6acce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c6acd3510 con 0x561c6acd0940 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c6abb36e0 con 0x561c6acc5760 2024-09-05T06:49:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c6acd3690 con 0x561c6acc6080 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0d1d76740 10 client.0 _ll_register_callbacks cb 0x561c6abd7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0cdf3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acd0940 0x561c6acce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43424/0 (socket says 172.21.15.86:43424) 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.057+0000 7fa0cd739640 1 -- 172.21.15.86:0/483849457 learned_addr learned my addr 172.21.15.86:0/483849457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6accdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 -- 172.21.15.86:0/483849457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 msgr2=0x561c6accdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6accdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 -- 172.21.15.86:0/483849457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acd0940 msgr2=0x561c6acce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acd0940 0x561c6acce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 -- 172.21.15.86:0/483849457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6acdc980 con 0x561c6acc5760 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0cd739640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4000c00 tx=0x7fa0c40977c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5230dac5c213ea24 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40cf160 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fa0c40b4210 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b8016070 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0c409f7b0 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0d1d76740 10 client.26702 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.058+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c6acdf370 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0b8016070 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c4009a30 con 0x561c6acc5760 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 10 client.26702 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 10 client.26702 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 20 client.26702 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6ace0880 0x561c6ace2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 4 client.26702 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:49:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c6ac389d0 con 0x561c6ace0880 2024-09-05T06:49:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6ace0880 0x561c6ace2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6ace0880 0x561c6ace2cf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0e2ac80ffe6ef2 server_cookie=d9768081000697d7 in_seq=0 out_seq=0 2024-09-05T06:49:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.059+0000 7fa0b0ff9640 10 client.26702 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/483849457 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c6ac389d0 con 0x561c6ace0880 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0b0ff9640 10 client.26702 handle_command_reply: tid=0 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 client.26702 shutdown 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 msgr2=0x561c6accd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4000c00 tx=0x7fa0c40977c0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 shutdown_connections 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6ace0880 0x561c6ace2cf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acd0940 0x561c6acce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6accdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0b0ff9640 0 client.26702 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/483849457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acc5760 0x561c6accd7b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 >> 172.21.15.86:0/483849457 conn(0x561c6acc1cf0 msgr2=0x561c6acc7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 shutdown_connections 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.060+0000 7fa0d1d76740 1 -- 172.21.15.86:0/483849457 wait complete. 2024-09-05T06:49:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.061+0000 7fa0d1d76740 20 client.26702 trim_cache size 0 max 16384 2024-09-05T06:49:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.078+0000 7fa0d1d76740 1 Processor -- start 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.078+0000 7fa0d1d76740 1 -- start start 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 0x561c6ace85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6accaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81af120 con 0x561c6acc6080 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b0c60 con 0x561c6acd0940 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81af7f0 con 0x561c6acc5760 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0cdf3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6accaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46730/0 (socket says 172.21.15.86:46730) 2024-09-05T06:49:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 0x561c6ace85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 -- 172.21.15.86:0/632055246 learned_addr learned my addr 172.21.15.86:0/632055246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 -- 172.21.15.86:0/632055246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 msgr2=0x561c6ace85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 0x561c6ace85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 -- 172.21.15.86:0/632055246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 msgr2=0x561c6accaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6accaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 -- 172.21.15.86:0/632055246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6acc1b20 con 0x561c6acc6080 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0cd739640 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 0x561c6ace85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6accaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.079+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009a10 tx=0x7fa0b8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6a8619056ae128 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0b2ffd640 1 -- 172.21.15.86:0/632055246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8003e00 con 0x561c6acc6080 2024-09-05T06:49:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0b2ffd640 1 -- 172.21.15.86:0/632055246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0b8016040 con 0x561c6acc6080 2024-09-05T06:49:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0b2ffd640 1 -- 172.21.15.86:0/632055246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801b6d0 con 0x561c6acc6080 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 -- 172.21.15.86:0/632055246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 msgr2=0x561c6ace8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009a10 tx=0x7fa0b8014040 comp rx=0 tx=0).stop 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 -- 172.21.15.86:0/632055246 shutdown_connections 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6accaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc6080 0x561c6ace8b00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/632055246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc5760 0x561c6ace85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.080+0000 7fa0d1d76740 1 -- 172.21.15.86:0/632055246 >> 172.21.15.86:0/632055246 conn(0x561c6acc1cf0 msgr2=0x561c6accf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- 172.21.15.86:0/632055246 shutdown_connections 2024-09-05T06:49:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- 172.21.15.86:0/632055246 wait complete. 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- start start 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 Processor -- start 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6ad452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b0590 con 0x561c6acc5760 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81afec0 con 0x561c6acd0940 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81ad120 con 0x561c6acc6080 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0d1d76740 10 client.0 _ll_register_callbacks cb 0x7fa0c409af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.081+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46742/0 (socket says 172.21.15.86:46742) 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 -- 172.21.15.86:0/1813652373 learned_addr learned my addr 172.21.15.86:0/1813652373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:49:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6ad452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cdf3a640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 -- 172.21.15.86:0/1813652373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 msgr2=0x561c6ad452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6ad452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 -- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 msgr2=0x561c6acd6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 -- 172.21.15.86:0/1813652373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6acc2e80 con 0x561c6acc5760 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6ad452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0cd739640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409b100 tx=0x7fa0c4097ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea0f46a979b8421 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c409b1d0 con 0x561c6acc5760 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9605+0+0 (secure 0 0 0) 0x7fa0c4097150 con 0x561c6acc5760 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b8016040 con 0x561c6acc5760 2024-09-05T06:49:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0c4097390 con 0x561c6acc5760 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0d1d76740 10 client.17487 fetch_fsmap learned FSMap version 32 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.082+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c6acc2980 con 0x561c6acc5760 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0b8016040 con 0x561c6acc5760 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c409a2c0 con 0x561c6acc5760 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 10 client.17487 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 10 client.17487 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 20 client.17487 populate_metadata read hostname 'smithi086' 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6acd8670 0x561c6acd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 4 client.17487 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x561c6ac32d20 con 0x561c6acd8670 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6acd8670 0x561c6acd8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:49:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0b0ff9640 10 client.17487 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.083+0000 7fa0ccf38640 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6acd8670 0x561c6acd8a80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2590c97bf328222 server_cookie=898e79712b56464e in_seq=0 out_seq=0 2024-09-05T06:49:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0b0ff9640 1 -- 172.21.15.86:0/1813652373 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c6ac32d20 con 0x561c6acd8670 2024-09-05T06:49:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0b0ff9640 10 client.17487 handle_command_reply: tid=0 2024-09-05T06:49:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 client.17487 shutdown 2024-09-05T06:49:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 msgr2=0x561c6ad44d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409b100 tx=0x7fa0c4097ca0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 shutdown_connections 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561c6acd8670 0x561c6acd8a80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c6acd0940 0x561c6acd6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c6acc6080 0x561c6ad452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0b0ff9640 0 client.17487 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 --2- 172.21.15.86:0/1813652373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c6acc5760 0x561c6ad44d70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.137+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 >> 172.21.15.86:0/1813652373 conn(0x561c6acc1cf0 msgr2=0x561c6accf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 shutdown_connections 2024-09-05T06:49:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0d1d76740 1 -- 172.21.15.86:0/1813652373 wait complete. 2024-09-05T06:49:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0d1d76740 20 client.17487 trim_cache size 0 max 16384 2024-09-05T06:49:59.140 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 30, 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "24a7b065": { 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 30, 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:49:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:59.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 msgr2=0x7fa0b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 0x7fa0b40a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00319e0 tx=0x7fa0c002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 msgr2=0x7fa0c8198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d950 tx=0x7fa0c400de10 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 shutdown_connections 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa0b40a2cf0 0x7fa0b40a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0c8106f20 0x7fa0c819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0c81027a0 0x7fa0c8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 --2- 172.21.15.86:0/3957284290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c8101df0 0x7fa0c8198330 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 >> 172.21.15.86:0/3957284290 conn(0x7fa0c80fd8f0 msgr2=0x7fa0c80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 shutdown_connections 2024-09-05T06:49:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:49:59.138+0000 7fa0cf99d640 1 -- 172.21.15.86:0/3957284290 wait complete. 2024-09-05T06:49:59.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.98 MB/sec execute 319 sec latency 388.926 ms 2024-09-05T06:49:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:49:59 smithi096 ceph-mon[34468]: pgmap v269: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:49:59.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:49:59 smithi115 ceph-mon[34791]: pgmap v269: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:49:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:49:59 smithi086 ceph-mon[30943]: pgmap v269: 129 pgs: 129 active+clean; 180 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-09-05T06:50:00.073 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '24a7b065', quiesce took 3.7 seconds. Will release after: 26.3 2024-09-05T06:50:00.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.97 MB/sec execute 320 sec latency 1388.989 ms 2024-09-05T06:50:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:00 smithi096 ceph-mon[34468]: overall HEALTH_OK 2024-09-05T06:50:00.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:00 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T06:50:00.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:00 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T06:50:01.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.97 MB/sec execute 321 sec latency 2389.076 ms 2024-09-05T06:50:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:01 smithi096 ceph-mon[34468]: pgmap v270: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T06:50:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:01 smithi115 ceph-mon[34791]: pgmap v270: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T06:50:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:01 smithi086 ceph-mon[30943]: pgmap v270: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-09-05T06:50:02.522 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.96 MB/sec execute 322 sec latency 3389.119 ms 2024-09-05T06:50:03.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.95 MB/sec execute 323 sec latency 4389.156 ms 2024-09-05T06:50:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:03 smithi096 ceph-mon[34468]: pgmap v271: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T06:50:03.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:03 smithi115 ceph-mon[34791]: pgmap v271: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T06:50:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:03 smithi086 ceph-mon[30943]: pgmap v271: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T06:50:04.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.95 MB/sec execute 324 sec latency 5389.301 ms 2024-09-05T06:50:05.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.94 MB/sec execute 325 sec latency 6389.367 ms 2024-09-05T06:50:06.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.94 MB/sec execute 326 sec latency 7389.458 ms 2024-09-05T06:50:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:06 smithi086 ceph-mon[30943]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T06:50:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:06 smithi086 ceph-mon[30943]: pgmap v273: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-05T06:50:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:06 smithi096 ceph-mon[34468]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T06:50:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:06 smithi096 ceph-mon[34468]: pgmap v273: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-05T06:50:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:06 smithi115 ceph-mon[34791]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T06:50:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:06 smithi115 ceph-mon[34791]: pgmap v273: 129 pgs: 129 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-09-05T06:50:07.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.93 MB/sec execute 327 sec latency 8389.541 ms 2024-09-05T06:50:08.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:07 smithi086 ceph-mon[30943]: pgmap v274: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:08.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:07 smithi096 ceph-mon[34468]: pgmap v274: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:08.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:07 smithi115 ceph-mon[34791]: pgmap v274: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:08.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.92 MB/sec execute 328 sec latency 9389.573 ms 2024-09-05T06:50:09.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.92 MB/sec execute 329 sec latency 10389.601 ms 2024-09-05T06:50:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:09 smithi096 ceph-mon[34468]: pgmap v275: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:09 smithi115 ceph-mon[34791]: pgmap v275: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:09.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:09 smithi086 ceph-mon[30943]: pgmap v275: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 414 KiB/s wr, 80 op/s 2024-09-05T06:50:10.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.91 MB/sec execute 330 sec latency 11389.689 ms 2024-09-05T06:50:11.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:50:11.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.91 MB/sec execute 331 sec latency 12389.741 ms 2024-09-05T06:50:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:11 smithi086 ceph-mon[30943]: pgmap v276: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:11 smithi096 ceph-mon[34468]: pgmap v276: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:11 smithi115 ceph-mon[34791]: pgmap v276: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 -- 172.21.15.86:0/3427885129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 msgr2=0x7f30e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 --2- 172.21.15.86:0/3427885129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e4101200 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009900 tx=0x7f30d802a480 comp rx=0 tx=0).stop 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 -- 172.21.15.86:0/3427885129 shutdown_connections 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 --2- 172.21.15.86:0/3427885129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102ef0 0x7f30e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 --2- 172.21.15.86:0/3427885129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102570 0x7f30e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 --2- 172.21.15.86:0/3427885129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e4101200 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 -- 172.21.15.86:0/3427885129 >> 172.21.15.86:0/3427885129 conn(0x7f30e40fc7d0 msgr2=0x7f30e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 -- 172.21.15.86:0/3427885129 shutdown_connections 2024-09-05T06:50:12.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.321+0000 7f30ec53f640 1 -- 172.21.15.86:0/3427885129 wait complete. 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 Processor -- start 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 -- start start 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 0x7f30e41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 0x7f30e419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4110e30 con 0x7f30e4100e20 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4110cb0 con 0x7f30e4102ef0 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ec53f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4110fb0 con 0x7f30e4102570 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56582/0 (socket says 172.21.15.86:56582) 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 -- 172.21.15.86:0/72905983 learned_addr learned my addr 172.21.15.86:0/72905983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30e9ada640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 0x7f30e41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30eaadc640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 0x7f30e419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 -- 172.21.15.86:0/72905983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 msgr2=0x7f30e41977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 0x7f30e41977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 -- 172.21.15.86:0/72905983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 msgr2=0x7f30e419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 0x7f30e419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 -- 172.21.15.86:0/72905983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e419c250 con 0x7f30e4100e20 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30e9ada640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 0x7f30e41977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30eaadc640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 0x7f30e419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30ea2db640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006010 tx=0x7f30d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.322+0000 7f30d77fe640 1 -- 172.21.15.86:0/72905983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d803f070 con 0x7f30e4100e20 2024-09-05T06:50:12.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.323+0000 7f30d77fe640 1 -- 172.21.15.86:0/72905983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d8003aa0 con 0x7f30e4100e20 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.323+0000 7f30d77fe640 1 -- 172.21.15.86:0/72905983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8035030 con 0x7f30e4100e20 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.323+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e419c4e0 con 0x7f30e4100e20 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.323+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e41a3e20 con 0x7f30e4100e20 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30d77fe640 1 -- 172.21.15.86:0/72905983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f30d8042070 con 0x7f30e4100e20 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 Processor -- start 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 -- start start 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 0x5578271a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578270fc2e0 con 0x557827196380 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5578270a3ce0 con 0x557827196ca0 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30ee918740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578270f82a0 con 0x5578271a3890 2024-09-05T06:50:12.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30eaadc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.325+0000 7f30eaadc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56598/0 (socket says 172.21.15.86:56598) 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 -- 172.21.15.86:0/3558147998 learned_addr learned my addr 172.21.15.86:0/3558147998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30d77fe640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 0x7f30d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30e9ada640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 0x5578271a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30ea2db640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 -- 172.21.15.86:0/3558147998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 msgr2=0x5578271a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 0x5578271a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 -- 172.21.15.86:0/3558147998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 msgr2=0x5578271a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 -- 172.21.15.86:0/3558147998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827088440 con 0x557827196380 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30e9ada640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 0x5578271a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30ea2db640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:12.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30d77fe640 1 -- 172.21.15.86:0/72905983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f30d80f0210 con 0x7f30e4100e20 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 0x7f30d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00d340 tx=0x7f30dc00d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d78b72deab943a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30d4ff9640 1 -- 172.21.15.86:0/3558147998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc01aa60 con 0x557827196380 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30d4ff9640 1 -- 172.21.15.86:0/3558147998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30dc011500 con 0x557827196380 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30d4ff9640 1 -- 172.21.15.86:0/3558147998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00e3b0 con 0x557827196380 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.326+0000 7f30eaadc640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 0x7f30d00a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30dc01e000 tx=0x7f30dc01f000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- 172.21.15.86:0/3558147998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 msgr2=0x557827196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00d340 tx=0x7f30dc00d800 comp rx=0 tx=0).stop 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- 172.21.15.86:0/3558147998 shutdown_connections 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578271a3890 0x5578271a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 --2- 172.21.15.86:0/3558147998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x557827196760 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- 172.21.15.86:0/3558147998 >> 172.21.15.86:0/3558147998 conn(0x557827192a30 msgr2=0x557827192e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- 172.21.15.86:0/3558147998 shutdown_connections 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- 172.21.15.86:0/3558147998 wait complete. 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.327+0000 7f30ee918740 1 -- start start 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 Processor -- start 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578271a6460 con 0x5578271a3890 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5578270b7050 con 0x557827196ca0 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578271a65e0 con 0x557827196380 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ee918740 10 client.0 _ll_register_callbacks cb 0x5578270f7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56608/0 (socket says 172.21.15.86:56608) 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 -- 172.21.15.86:0/385659027 learned_addr learned my addr 172.21.15.86:0/385659027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30e9ada640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ea2db640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 -- 172.21.15.86:0/385659027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 msgr2=0x5578271a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 -- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 msgr2=0x5578271a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 -- 172.21.15.86:0/385659027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557826e6d3c0 con 0x5578271a3890 2024-09-05T06:50:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30ea2db640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30eaadc640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f30dc019b20 tx=0x7f30dc019b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8906f4b42bd021 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.328+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc008ae0 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f30dc004a80 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30dc011500 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30dc004c80 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 10 client.17499 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5578271b1d80 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30dc011500 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f30dc02c070 con 0x5578271a3890 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 10 client.17499 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 20 client.17499 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5578271b30d0 0x5578271b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 4 client.17499 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557827109850 con 0x5578271b30d0 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.329+0000 7f30ea2db640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5578271b30d0 0x5578271b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.330+0000 7f30cdffb640 10 client.17499 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.330+0000 7f30ea2db640 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5578271b30d0 0x5578271b54c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67030ae4557648d server_cookie=fd96f8430eb22445 in_seq=0 out_seq=0 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.330+0000 7f30cdffb640 1 -- 172.21.15.86:0/385659027 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557827109850 con 0x5578271b30d0 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.330+0000 7f30cdffb640 10 client.17499 handle_command_reply: tid=0 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.330+0000 7f30ee918740 1 client.17499 shutdown 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 msgr2=0x5578271a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f30dc019b20 tx=0x7f30dc019b50 comp rx=0 tx=0).stop 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 shutdown_connections 2024-09-05T06:50:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5578271b30d0 0x5578271b54c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578271a3890 0x5578271a0960 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557827196ca0 0x5578271a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 --2- 172.21.15.86:0/385659027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 >> 172.21.15.86:0/385659027 conn(0x557827192a30 msgr2=0x557827198790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30cdffb640 0 client.17499 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 shutdown_connections 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 1 -- 172.21.15.86:0/385659027 wait complete. 2024-09-05T06:50:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.331+0000 7f30ee918740 20 client.17499 trim_cache size 0 max 16384 2024-09-05T06:50:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 Processor -- start 2024-09-05T06:50:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 -- start start 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196ca0 0x55782719bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x55782719f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41aceb0 con 0x557827196380 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41ae9f0 con 0x5578271a3890 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ee918740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41ad580 con 0x557827196ca0 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56612/0 (socket says 172.21.15.86:56612) 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 -- 172.21.15.86:0/1761207360 learned_addr learned my addr 172.21.15.86:0/1761207360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30e9ada640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196ca0 0x55782719bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30eaadc640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x55782719f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 -- 172.21.15.86:0/1761207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196ca0 msgr2=0x55782719bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196ca0 0x55782719bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 -- 172.21.15.86:0/1761207360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 msgr2=0x55782719f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.350+0000 7f30ea2db640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x55782719f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30ea2db640 1 -- 172.21.15.86:0/1761207360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827196ad0 con 0x557827196380 2024-09-05T06:50:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30eaadc640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x55782719f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30ea2db640 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005940 tx=0x7f30d808f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9a84373655bf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30cffff640 1 -- 172.21.15.86:0/1761207360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8049040 con 0x557827196380 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30cffff640 1 -- 172.21.15.86:0/1761207360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d80bc2f0 con 0x557827196380 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.351+0000 7f30cffff640 1 -- 172.21.15.86:0/1761207360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80b6e60 con 0x557827196380 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- 172.21.15.86:0/1761207360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 msgr2=0x5578271af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005940 tx=0x7f30d808f360 comp rx=0 tx=0).stop 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- 172.21.15.86:0/1761207360 shutdown_connections 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x55782719f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196ca0 0x55782719bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- 172.21.15.86:0/1761207360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196380 0x5578271af700 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- 172.21.15.86:0/1761207360 >> 172.21.15.86:0/1761207360 conn(0x557827192a30 msgr2=0x55782719ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- 172.21.15.86:0/1761207360 shutdown_connections 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- 172.21.15.86:0/1761207360 wait complete. 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- start start 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 Processor -- start 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x5578271a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41ae320 con 0x557827196ca0 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41adc50 con 0x5578271a3890 2024-09-05T06:50:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41aaeb0 con 0x557827196380 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30ee918740 10 client.0 _ll_register_callbacks cb 0x7f30dc01d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30e9ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30e9ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56624/0 (socket says 172.21.15.86:56624) 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.352+0000 7f30e9ada640 1 -- 172.21.15.86:0/941130224 learned_addr learned my addr 172.21.15.86:0/941130224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30ea2db640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30eaadc640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x5578271a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 -- 172.21.15.86:0/941130224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 msgr2=0x5578271abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 -- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 msgr2=0x5578271a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x5578271a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 -- 172.21.15.86:0/941130224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578271a79d0 con 0x557827196ca0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30e9ada640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f30e40ffea0 tx=0x7f30e0010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b786b5cefc1fa7d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000cbf0 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d80bc2f0 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f30e000cd90 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30e0026050 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30ee918740 10 client.17505 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55782721d9c0 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.353+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d80bc2f0 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f30e0021030 con 0x557827196ca0 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 10 client.17505 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 20 client.17505 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55782721e140 0x557827220560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 4 client.17505 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557827103d60 con 0x55782721e140 2024-09-05T06:50:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ea2db640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55782721e140 0x557827220560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30cdffb640 10 client.17505 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ea2db640 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55782721e140 0x557827220560 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38728c22eed5359 server_cookie=ff95a0597251feac in_seq=0 out_seq=0 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30cdffb640 1 -- 172.21.15.86:0/941130224 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557827103d60 con 0x55782721e140 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30cdffb640 10 client.17505 handle_command_reply: tid=0 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.354+0000 7f30ee918740 1 client.17505 shutdown 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 msgr2=0x5578271ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f30e40ffea0 tx=0x7f30e0010ca0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 shutdown_connections 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55782721e140 0x557827220560 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578271a3890 0x5578271a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557827196ca0 0x5578271ac180 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30cdffb640 0 client.17505 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 --2- 172.21.15.86:0/941130224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557827196380 0x5578271abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 >> 172.21.15.86:0/941130224 conn(0x557827192a30 msgr2=0x55782719f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 shutdown_connections 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 1 -- 172.21.15.86:0/941130224 wait complete. 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ee918740 20 client.17505 trim_cache size 0 max 16384 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:50:12.373 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 msgr2=0x7f30d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 0x7f30d00a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30dc01e000 tx=0x7f30dc01f000 comp rx=0 tx=0).stop 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 msgr2=0x7f30e41972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006010 tx=0x7f30d80042f0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 shutdown_connections 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f30d00a2d70 0x7f30d00a5230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30e4102ef0 0x7f30e419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30e4102570 0x7f30e41977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.355+0000 7f30ec53f640 1 --2- 172.21.15.86:0/72905983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e4100e20 0x7f30e41972a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:12.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.356+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 >> 172.21.15.86:0/72905983 conn(0x7f30e40fc7d0 msgr2=0x7f30e40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:12.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.356+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 shutdown_connections 2024-09-05T06:50:12.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:12.356+0000 7f30ec53f640 1 -- 172.21.15.86:0/72905983 wait complete. 2024-09-05T06:50:12.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.90 MB/sec execute 332 sec latency 13389.848 ms 2024-09-05T06:50:12.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:12 smithi086 ceph-mon[30943]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T06:50:13.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:50:13.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:50:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:12 smithi096 ceph-mon[34468]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T06:50:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:12 smithi115 ceph-mon[34791]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T06:50:13.508 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:13.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.90 MB/sec execute 333 sec latency 14389.886 ms 2024-09-05T06:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 -- 172.21.15.86:0/2594269420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 msgr2=0x7f3b681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/2594269420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681021c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009900 tx=0x7f3b6402a410 comp rx=0 tx=0).stop 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 -- 172.21.15.86:0/2594269420 shutdown_connections 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/2594269420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 0x7f3b6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/2594269420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 0x7f3b681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/2594269420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681021c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 -- 172.21.15.86:0/2594269420 >> 172.21.15.86:0/2594269420 conn(0x7f3b680fd8c0 msgr2=0x7f3b680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 -- 172.21.15.86:0/2594269420 shutdown_connections 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.988+0000 7f3b6ff96640 1 -- 172.21.15.86:0/2594269420 wait complete. 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 Processor -- start 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 -- start start 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 0x7f3b681988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 0x7f3b6819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68111e70 con 0x7f3b68101de0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68111cf0 con 0x7f3b68102790 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6ff96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68111ff0 con 0x7f3b68106f10 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6dd32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6dd32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33316/0 (socket says 172.21.15.86:33316) 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6dd32640 1 -- 172.21.15.86:0/1391920387 learned_addr learned my addr 172.21.15.86:0/1391920387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6d531640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 0x7f3b681988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.989+0000 7f3b6e533640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 0x7f3b6819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 -- 172.21.15.86:0/1391920387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 msgr2=0x7f3b6819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 0x7f3b6819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 -- 172.21.15.86:0/1391920387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 msgr2=0x7f3b681988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 0x7f3b681988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 -- 172.21.15.86:0/1391920387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6819d360 con 0x7f3b68101de0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6dd32640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005ab0 tx=0x7f3b64003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6403f070 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b64031d60 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6819d5f0 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64035030 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.990+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a4f30 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.991+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b681021c0 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.992+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b64042070 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.993+0000 7f3b5effd640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 0x7f3b540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.993+0000 7f3b6d531640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 0x7f3b540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.993+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f3b640f06f0 con 0x7f3b68101de0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.993+0000 7f3b6d531640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 0x7f3b540a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ffde0 tx=0x7f3b5800b410 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:13.996+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b640bced0 con 0x7f3b68101de0 2024-09-05T06:50:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.200+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6810a0f0 con 0x7f3b68101de0 2024-09-05T06:50:14.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:13 smithi086 ceph-mon[30943]: pgmap v278: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-09-05T06:50:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.201+0000 7f3b5effd640 1 -- 172.21.15.86:0/1391920387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f3b640bc7d0 con 0x7f3b68101de0 2024-09-05T06:50:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:50:14.202 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:50:14.202 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:50:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 msgr2=0x7f3b540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 0x7f3b540a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ffde0 tx=0x7f3b5800b410 comp rx=0 tx=0).stop 2024-09-05T06:50:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 msgr2=0x7f3b681983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005ab0 tx=0x7f3b64003cd0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 shutdown_connections 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b540a2cf0 0x7f3b540a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b68106f10 0x7f3b6819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b68102790 0x7f3b681988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 --2- 172.21.15.86:0/1391920387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b68101de0 0x7f3b681983b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 >> 172.21.15.86:0/1391920387 conn(0x7f3b680fd8c0 msgr2=0x7f3b680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 shutdown_connections 2024-09-05T06:50:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:14.203+0000 7f3b6ff96640 1 -- 172.21.15.86:0/1391920387 wait complete. 2024-09-05T06:50:14.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:13 smithi096 ceph-mon[34468]: pgmap v278: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-09-05T06:50:14.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:13 smithi115 ceph-mon[34791]: pgmap v278: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-09-05T06:50:14.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.89 MB/sec execute 334 sec latency 15389.999 ms 2024-09-05T06:50:14.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:50:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:14 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1391920387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:14 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1391920387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:14 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1391920387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:15.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.88 MB/sec execute 335 sec latency 16390.048 ms 2024-09-05T06:50:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 -- 172.21.15.86:0/3767074382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c100e20 msgr2=0x7f862c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 --2- 172.21.15.86:0/3767074382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c100e20 0x7f862c101200 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f862802a480 comp rx=0 tx=0).stop 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 -- 172.21.15.86:0/3767074382 shutdown_connections 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 --2- 172.21.15.86:0/3767074382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c102ef0 0x7f862c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 --2- 172.21.15.86:0/3767074382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102570 0x7f862c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 --2- 172.21.15.86:0/3767074382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c100e20 0x7f862c101200 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 -- 172.21.15.86:0/3767074382 >> 172.21.15.86:0/3767074382 conn(0x7f862c0fc7d0 msgr2=0x7f862c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 -- 172.21.15.86:0/3767074382 shutdown_connections 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.592+0000 7f8633f65640 1 -- 172.21.15.86:0/3767074382 wait complete. 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 Processor -- start 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 -- start start 2024-09-05T06:50:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c100e20 0x7f862c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 0x7f862c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f862c110e30 con 0x7f862c102570 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f862c110cb0 con 0x7f862c100e20 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8633f65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f862c110fb0 con 0x7f862c102ef0 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8631500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8631500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33320/0 (socket says 172.21.15.86:33320) 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8631500640 1 -- 172.21.15.86:0/1358401593 learned_addr learned my addr 172.21.15.86:0/1358401593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.593+0000 7f8631d01640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c100e20 0x7f862c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8632502640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 0x7f862c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 -- 172.21.15.86:0/1358401593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 msgr2=0x7f862c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 0x7f862c19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 -- 172.21.15.86:0/1358401593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c100e20 msgr2=0x7f862c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c100e20 0x7f862c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 -- 172.21.15.86:0/1358401593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c19c2b0 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8632502640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 0x7f862c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8631500640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f861c00d770 tx=0x7f861c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8622ffd640 1 -- 172.21.15.86:0/1358401593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c017070 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8622ffd640 1 -- 172.21.15.86:0/1358401593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f861c0054d0 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c19c5a0 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8622ffd640 1 -- 172.21.15.86:0/1358401593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c004cf0 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.594+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c1a3e80 con 0x7f862c102570 2024-09-05T06:50:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.597+0000 7f863633e740 1 Processor -- start 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.597+0000 7f8622ffd640 1 -- 172.21.15.86:0/1358401593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f861c004e90 con 0x7f862c102570 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.597+0000 7f863633e740 1 -- start start 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.597+0000 7f863633e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.597+0000 7f863633e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f863633e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f863633e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af07aa12f0 con 0x55af07b48580 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f863633e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af07a48cf0 con 0x55af07b3cb40 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f863633e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af07a9d2b0 con 0x55af07b3b5d0 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33336/0 (socket says 172.21.15.86:33336) 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631500640 1 -- 172.21.15.86:0/4254824678 learned_addr learned my addr 172.21.15.86:0/4254824678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8632502640 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8622ffd640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 0x7f86180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 -- 172.21.15.86:0/4254824678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 msgr2=0x55af07b3b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b3b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 -- 172.21.15.86:0/4254824678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 msgr2=0x55af07b4a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 -- 172.21.15.86:0/4254824678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af07a2d3a0 con 0x55af07b3b5d0 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631500640 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8632502640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 0x7f86180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8622ffd640 1 -- 172.21.15.86:0/1358401593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f861c0cf300 con 0x7f862c102570 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.598+0000 7f8631d01640 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b3ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8628005940 tx=0x7f8628005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8a8113f61948ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.599+0000 7f8632502640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 0x7f86180a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8624005fd0 tx=0x7f8624004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.599+0000 7f8617fff640 1 -- 172.21.15.86:0/4254824678 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628040070 con 0x55af07b3b5d0 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.599+0000 7f8617fff640 1 -- 172.21.15.86:0/4254824678 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8628031e10 con 0x55af07b3b5d0 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.599+0000 7f8617fff640 1 -- 172.21.15.86:0/4254824678 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628035040 con 0x55af07b3b5d0 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- 172.21.15.86:0/4254824678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 msgr2=0x55af07b3ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b3ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8628005940 tx=0x7f8628005c00 comp rx=0 tx=0).stop 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- 172.21.15.86:0/4254824678 shutdown_connections 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b3ba10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 --2- 172.21.15.86:0/4254824678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- 172.21.15.86:0/4254824678 >> 172.21.15.86:0/4254824678 conn(0x55af07b04310 msgr2=0x55af07b37c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- 172.21.15.86:0/4254824678 shutdown_connections 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- 172.21.15.86:0/4254824678 wait complete. 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 -- start start 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.600+0000 7f863633e740 1 Processor -- start 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b48580 0x55af07b45620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af07b4b220 con 0x55af07b3b5d0 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af07a5c060 con 0x55af07b48580 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55af07b4b3a0 con 0x55af07b3cb40 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f863633e740 10 client.0 _ll_register_callbacks cb 0x55af077af8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33352/0 (socket says 172.21.15.86:33352) 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 -- 172.21.15.86:0/793470888 learned_addr learned my addr 172.21.15.86:0/793470888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8632502640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b48580 0x55af07b45620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631500640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 -- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 msgr2=0x55af07b450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 -- 172.21.15.86:0/793470888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b48580 msgr2=0x55af07b45620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b48580 0x55af07b45620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 -- 172.21.15.86:0/793470888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af078123c0 con 0x55af07b3b5d0 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f8631d01640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8628000c00 tx=0x7f8628009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4242c6ad52f93f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f86157fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.601+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862804c020 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628031e10 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f8628031840 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8628057050 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 10 client.17523 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af07b56b80 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8628031e10 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8628004750 con 0x55af07b3b5d0 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 10 client.17523 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 10 client.17523 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 20 client.17523 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07b57ed0 0x55af07b5a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 4 client.17523 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af07aae860 con 0x55af07b57ed0 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.602+0000 7f8631500640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07b57ed0 0x55af07b5a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.603+0000 7f8631500640 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07b57ed0 0x55af07b5a2c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6fb106e35fb42313 server_cookie=d6a78b5c48c20ae9 in_seq=0 out_seq=0 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.603+0000 7f86157fa640 10 client.17523 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.603+0000 7f86157fa640 1 -- 172.21.15.86:0/793470888 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af07aae860 con 0x55af07b57ed0 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.603+0000 7f86157fa640 10 client.17523 handle_command_reply: tid=0 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.603+0000 7f863633e740 1 client.17523 shutdown 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 msgr2=0x55af07b47e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8628000c00 tx=0x7f8628009680 comp rx=0 tx=0).stop 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 shutdown_connections 2024-09-05T06:50:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07b57ed0 0x55af07b5a2c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b48580 0x55af07b45620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f86157fa640 0 client.17523 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 --2- 172.21.15.86:0/793470888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b3b5d0 0x55af07b47e30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 >> 172.21.15.86:0/793470888 conn(0x55af07b04310 msgr2=0x55af07b3d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 shutdown_connections 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 1 -- 172.21.15.86:0/793470888 wait complete. 2024-09-05T06:50:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.604+0000 7f863633e740 20 client.17523 trim_cache size 0 max 16384 2024-09-05T06:50:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 Processor -- start 2024-09-05T06:50:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 -- start start 2024-09-05T06:50:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b42360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1acf00 con 0x55af07b48580 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1aea40 con 0x55af07b3cb40 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f863633e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1ad5d0 con 0x55af07b3b5d0 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33356/0 (socket says 172.21.15.86:33356) 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 -- 172.21.15.86:0/384767288 learned_addr learned my addr 172.21.15.86:0/384767288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8631500640 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8631d01640 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b42360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 -- 172.21.15.86:0/384767288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 msgr2=0x55af07b42360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b42360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 -- 172.21.15.86:0/384767288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 msgr2=0x55af07b428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.617+0000 7f8632502640 1 -- 172.21.15.86:0/384767288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af07b43630 con 0x55af07b48580 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f8632502640 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f86240207a0 tx=0x7f8624015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efd9ff67e2d2b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f86177fe640 1 -- 172.21.15.86:0/384767288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624013b80 con 0x55af07b48580 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f86177fe640 1 -- 172.21.15.86:0/384767288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8624017070 con 0x55af07b48580 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f86177fe640 1 -- 172.21.15.86:0/384767288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862401e690 con 0x55af07b48580 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 -- 172.21.15.86:0/384767288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 msgr2=0x55af07b364e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f86240207a0 tx=0x7f8624015000 comp rx=0 tx=0).stop 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 -- 172.21.15.86:0/384767288 shutdown_connections 2024-09-05T06:50:15.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b364e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3cb40 0x55af07b428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 --2- 172.21.15.86:0/384767288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3b5d0 0x55af07b42360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 -- 172.21.15.86:0/384767288 >> 172.21.15.86:0/384767288 conn(0x55af07b04310 msgr2=0x55af07b441d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 -- 172.21.15.86:0/384767288 shutdown_connections 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.618+0000 7f863633e740 1 -- 172.21.15.86:0/384767288 wait complete. 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 -- start start 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 Processor -- start 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 0x55af07b58fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1ae370 con 0x55af07b48580 2024-09-05T06:50:15.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1adca0 con 0x55af07b3b5d0 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f862c1aaf00 con 0x55af07b3cb40 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f863633e740 10 client.0 _ll_register_callbacks cb 0x7f8628009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f8632502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f8632502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33372/0 (socket says 172.21.15.86:33372) 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f8632502640 1 -- 172.21.15.86:0/87777608 learned_addr learned my addr 172.21.15.86:0/87777608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f8631d01640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 0x55af07b58fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.619+0000 7f8631500640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 -- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 msgr2=0x55af07b594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 -- 172.21.15.86:0/87777608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 msgr2=0x55af07b58fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:15.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 0x55af07b58fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 -- 172.21.15.86:0/87777608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af07bc3c00 con 0x55af07b48580 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8631d01640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 0x55af07b58fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f8632502640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8624004af0 tx=0x7f862401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48034c760115cc93 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f86157fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624038060 con 0x55af07b48580 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f86240202d0 con 0x55af07b48580 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8624017070 con 0x55af07b48580 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f862403f050 con 0x55af07b48580 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f863633e740 10 client.17529 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:15.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af07bc4e40 con 0x55af07b48580 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.620+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8624017070 con 0x55af07b48580 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8624027020 con 0x55af07b48580 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 10 client.17529 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 10 client.17529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 20 client.17529 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07bc54e0 0x55af07bc7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 4 client.17529 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af07b459b0 con 0x55af07bc54e0 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f8631d01640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07bc54e0 0x55af07bc7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f8631d01640 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07bc54e0 0x55af07bc7900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2913e1860c27702b server_cookie=1a8ed59544959a9c in_seq=0 out_seq=0 2024-09-05T06:50:15.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.621+0000 7f86157fa640 10 client.17529 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f86157fa640 1 -- 172.21.15.86:0/87777608 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af07b459b0 con 0x55af07bc54e0 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f86157fa640 10 client.17529 handle_command_reply: tid=0 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 client.17529 shutdown 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 msgr2=0x55af07b59a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8624004af0 tx=0x7f862401cba0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 shutdown_connections 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55af07bc54e0 0x55af07bc7900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af07b48580 0x55af07b59a20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55af07b3cb40 0x55af07b594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 --2- 172.21.15.86:0/87777608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af07b3b5d0 0x55af07b58fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f86157fa640 0 client.17529 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:50:15.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 >> 172.21.15.86:0/87777608 conn(0x55af07b04310 msgr2=0x55af07b364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 shutdown_connections 2024-09-05T06:50:15.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 1 -- 172.21.15.86:0/87777608 wait complete. 2024-09-05T06:50:15.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.622+0000 7f863633e740 20 client.17529 trim_cache size 0 max 16384 2024-09-05T06:50:15.640 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:50:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 msgr2=0x7f86180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 0x7f86180a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8624005fd0 tx=0x7f8624004a40 comp rx=0 tx=0).stop 2024-09-05T06:50:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 msgr2=0x7f862c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f861c00d770 tx=0x7f861c00dc30 comp rx=0 tx=0).stop 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 shutdown_connections 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86180a2cf0 0x7f86180a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f862c102ef0 0x7f862c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c102570 0x7f862c197840 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 --2- 172.21.15.86:0/1358401593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f862c100e20 0x7f862c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 >> 172.21.15.86:0/1358401593 conn(0x7f862c0fc7d0 msgr2=0x7f862c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 shutdown_connections 2024-09-05T06:50:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:15.623+0000 7f8633f65640 1 -- 172.21.15.86:0/1358401593 wait complete. 2024-09-05T06:50:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:15 smithi096 ceph-mon[34468]: pgmap v279: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:16.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:15 smithi086 ceph-mon[30943]: pgmap v279: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:16.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:50:16.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:15 smithi115 ceph-mon[34791]: pgmap v279: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-05T06:50:16.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.88 MB/sec execute 336 sec latency 17390.101 ms 2024-09-05T06:50:16.659 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- 172.21.15.86:0/3864260002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 msgr2=0x7f1db41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3864260002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db41021c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009900 tx=0x7f1da802a410 comp rx=0 tx=0).stop 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- 172.21.15.86:0/3864260002 shutdown_connections 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3864260002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4106f10 0x7f1db410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3864260002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4102790 0x7f1db41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3864260002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db41021c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- 172.21.15.86:0/3864260002 >> 172.21.15.86:0/3864260002 conn(0x7f1db40fd8c0 msgr2=0x7f1db40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- 172.21.15.86:0/3864260002 shutdown_connections 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- 172.21.15.86:0/3864260002 wait complete. 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 Processor -- start 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.116+0000 7f1dbb165640 1 -- start start 2024-09-05T06:50:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4102790 0x7f1db41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4106f10 0x7f1db419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4111e20 con 0x7f1db4101de0 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4111ca0 con 0x7f1db4102790 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4111fa0 con 0x7f1db4106f10 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33404/0 (socket says 172.21.15.86:33404) 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 -- 172.21.15.86:0/3890697421 learned_addr learned my addr 172.21.15.86:0/3890697421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db9702640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4106f10 0x7f1db419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db3fff640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4102790 0x7f1db41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 -- 172.21.15.86:0/3890697421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4106f10 msgr2=0x7f1db419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4106f10 0x7f1db419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 -- 172.21.15.86:0/3890697421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4102790 msgr2=0x7f1db41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4102790 0x7f1db41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 -- 172.21.15.86:0/3890697421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db419d330 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db8f01640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005940 tx=0x7f1da8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db1ffb640 1 -- 172.21.15.86:0/3890697421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da803f030 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db1ffb640 1 -- 172.21.15.86:0/3890697421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da8003aa0 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1db1ffb640 1 -- 172.21.15.86:0/3890697421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8035070 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.117+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db419d5c0 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.118+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a4f00 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1db1ffb640 1 -- 172.21.15.86:0/3890697421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1da8042030 con 0x7f1db4101de0 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 Processor -- start 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 -- start start 2024-09-05T06:50:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 0x5593bb1e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 0x5593bb1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593bb14c2f0 con 0x5593bb1e65d0 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5593bb0f3cf0 con 0x5593bb1f5780 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5593bb1482b0 con 0x5593bb1e7b40 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1db1ffb640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 0x7f1da40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.120+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33418/0 (socket says 172.21.15.86:33418) 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 -- 172.21.15.86:0/292189132 learned_addr learned my addr 172.21.15.86:0/292189132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db9702640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 0x5593bb1e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db9702640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 0x7f1da40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db1ffb640 1 -- 172.21.15.86:0/3890697421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f1da80efdb0 con 0x7f1db4101de0 2024-09-05T06:50:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db3fff640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 0x5593bb1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 -- 172.21.15.86:0/292189132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 msgr2=0x5593bb1e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 0x5593bb1e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 -- 172.21.15.86:0/292189132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 msgr2=0x5593bb1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 0x5593bb1f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 -- 172.21.15.86:0/292189132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bb0d83a0 con 0x5593bb1e65d0 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db9702640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 0x5593bb1e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db3fff640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 0x5593bb1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db8f01640 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1da80dc070 tx=0x7f1da8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e519651e381ba0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1db9702640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 0x7f1da40a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009fd0 tx=0x7f1dac0096a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1d9effd640 1 -- 172.21.15.86:0/292189132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8049050 con 0x5593bb1e65d0 2024-09-05T06:50:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1d9effd640 1 -- 172.21.15.86:0/292189132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da80bbf40 con 0x5593bb1e65d0 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.121+0000 7f1d9effd640 1 -- 172.21.15.86:0/292189132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80b85b0 con 0x5593bb1e65d0 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- 172.21.15.86:0/292189132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 msgr2=0x5593bb1e6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1da80dc070 tx=0x7f1da8047000 comp rx=0 tx=0).stop 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- 172.21.15.86:0/292189132 shutdown_connections 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1f5780 0x5593bb1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1e6a10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/292189132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e7b40 0x5593bb1e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- 172.21.15.86:0/292189132 >> 172.21.15.86:0/292189132 conn(0x5593bb1af310 msgr2=0x5593bb1e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- 172.21.15.86:0/292189132 shutdown_connections 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- 172.21.15.86:0/292189132 wait complete. 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.122+0000 7f1dbd53e740 1 -- start start 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 Processor -- start 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e65d0 0x5593bb1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593bb1f8240 con 0x5593bb1f5780 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5593bb107060 con 0x5593bb1e7b40 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5593bb1f83c0 con 0x5593bb1e65d0 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1dbd53e740 10 client.0 _ll_register_callbacks cb 0x5593bae5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33428/0 (socket says 172.21.15.86:33428) 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 -- 172.21.15.86:0/1011769256 learned_addr learned my addr 172.21.15.86:0/1011769256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db3fff640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e65d0 0x5593bb1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 -- 172.21.15.86:0/1011769256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e65d0 msgr2=0x5593bb1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e65d0 0x5593bb1f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 -- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 msgr2=0x5593bb1f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.123+0000 7f1db9702640 1 -- 172.21.15.86:0/1011769256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593baebd3c0 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1db9702640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00bc30 tx=0x7f1dac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae0dff2379ae55b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac025030 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f1dac01b070 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da80bbf40 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1dac032050 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 10 client.17541 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593bb203ca0 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da80bbf40 con 0x5593bb1f5780 2024-09-05T06:50:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1dac01e050 con 0x5593bb1f5780 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 10 client.17541 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 10 client.17541 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 20 client.17541 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb205170 0x5593bb2075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 4 client.17541 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.124+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593bb159860 con 0x5593bb205170 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.125+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb205170 0x5593bb2075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.125+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb205170 0x5593bb2075d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23f316a648dc2628 server_cookie=9c4a1d059a48e198 in_seq=0 out_seq=0 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.125+0000 7f1d93fff640 10 client.17541 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1d93fff640 1 -- 172.21.15.86:0/1011769256 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593bb159860 con 0x5593bb205170 2024-09-05T06:50:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1d93fff640 10 client.17541 handle_command_reply: tid=0 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 client.17541 shutdown 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 msgr2=0x5593bb1f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00bc30 tx=0x7f1dac004ab0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 shutdown_connections 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb205170 0x5593bb2075d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1f5780 0x5593bb1f2c70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1d93fff640 0 client.17541 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1011769256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1e65d0 0x5593bb1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 >> 172.21.15.86:0/1011769256 conn(0x5593bb1af310 msgr2=0x5593bb1e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 shutdown_connections 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1011769256 wait complete. 2024-09-05T06:50:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.126+0000 7f1dbd53e740 20 client.17541 trim_cache size 0 max 16384 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.138+0000 7f1dbd53e740 1 Processor -- start 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 -- start start 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 0x5593bb1f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb20d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41acfc0 con 0x5593bb1e7b40 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41aeb00 con 0x5593bb1e65d0 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41ad690 con 0x5593bb1f5780 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33434/0 (socket says 172.21.15.86:33434) 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 -- 172.21.15.86:0/811044023 learned_addr learned my addr 172.21.15.86:0/811044023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db9702640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb20d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db8f01640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 0x5593bb1f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 -- 172.21.15.86:0/811044023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 msgr2=0x5593bb20d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb20d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 -- 172.21.15.86:0/811044023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 msgr2=0x5593bb1f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 0x5593bb1f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 -- 172.21.15.86:0/811044023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bb1e9120 con 0x5593bb1e7b40 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db8f01640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 0x5593bb1f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1db3fff640 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005fd0 tx=0x7f1da000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81ad42f925acd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1d9e7fc640 1 -- 172.21.15.86:0/811044023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00038b0 con 0x5593bb1e7b40 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1d9e7fc640 1 -- 172.21.15.86:0/811044023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da0005020 con 0x5593bb1e7b40 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.139+0000 7f1d9e7fc640 1 -- 172.21.15.86:0/811044023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004ad0 con 0x5593bb1e7b40 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- 172.21.15.86:0/811044023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 msgr2=0x5593bb20cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005fd0 tx=0x7f1da000bb00 comp rx=0 tx=0).stop 2024-09-05T06:50:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- 172.21.15.86:0/811044023 shutdown_connections 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb20d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e7b40 0x5593bb20cea0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/811044023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e65d0 0x5593bb1f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- 172.21.15.86:0/811044023 >> 172.21.15.86:0/811044023 conn(0x5593bb1af310 msgr2=0x5593bb1f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- 172.21.15.86:0/811044023 shutdown_connections 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- 172.21.15.86:0/811044023 wait complete. 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.140+0000 7f1dbd53e740 1 -- start start 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 Processor -- start 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1fda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb1fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41ae430 con 0x5593bb1e65d0 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41add60 con 0x5593bb1e7b40 2024-09-05T06:50:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41aafc0 con 0x5593bb1f5780 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1dbd53e740 10 client.0 _ll_register_callbacks cb 0x7f1dac009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33448/0 (socket says 172.21.15.86:33448) 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 -- 172.21.15.86:0/1101279101 learned_addr learned my addr 172.21.15.86:0/1101279101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db3fff640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db9702640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb1fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 -- 172.21.15.86:0/1101279101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 msgr2=0x5593bb1fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb1fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 -- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 msgr2=0x5593bb1fda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1fda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.141+0000 7f1db8f01640 1 -- 172.21.15.86:0/1101279101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593bb1faa50 con 0x5593bb1e65d0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1db8f01640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1da80b83a0 tx=0x7f1da8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afd0242654b5e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8049040 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f1da80ed340 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da0005020 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1da80bbf40 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 10 client.17547 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593bb270650 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da0005020 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1da80c4670 con 0x5593bb1e65d0 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 10 client.17547 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 10 client.17547 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 20 client.17547 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb270be0 0x5593bb272fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 4 client.17547 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593bb153d70 con 0x5593bb270be0 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.142+0000 7f1db3fff640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb270be0 0x5593bb272fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1d93fff640 10 client.17547 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1db3fff640 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb270be0 0x5593bb272fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3524c0f2ab5c9768 server_cookie=8e3759d2eef94d6a in_seq=0 out_seq=0 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1d93fff640 1 -- 172.21.15.86:0/1101279101 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593bb153d70 con 0x5593bb270be0 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1d93fff640 10 client.17547 handle_command_reply: tid=0 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 client.17547 shutdown 2024-09-05T06:50:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 msgr2=0x5593bb1fd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1da80b83a0 tx=0x7f1da8005c90 comp rx=0 tx=0).stop 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 shutdown_connections 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5593bb270be0 0x5593bb272fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1d93fff640 0 client.17547 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5593bb1f5780 0x5593bb1fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5593bb1e7b40 0x5593bb1fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 --2- 172.21.15.86:0/1101279101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593bb1e65d0 0x5593bb1fd540 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 >> 172.21.15.86:0/1101279101 conn(0x5593bb1af310 msgr2=0x5593bb1f15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 shutdown_connections 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.143+0000 7f1dbd53e740 1 -- 172.21.15.86:0/1101279101 wait complete. 2024-09-05T06:50:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbd53e740 20 client.17547 trim_cache size 0 max 16384 2024-09-05T06:50:17.163 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:50:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 msgr2=0x7f1da40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 0x7f1da40a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009fd0 tx=0x7f1dac0096a0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 msgr2=0x7f1db4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005940 tx=0x7f1da8004270 comp rx=0 tx=0).stop 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 shutdown_connections 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1da40a2cf0 0x7f1da40a51b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1db4106f10 0x7f1db419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1db4102790 0x7f1db41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 --2- 172.21.15.86:0/3890697421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1db4101de0 0x7f1db4198380 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 >> 172.21.15.86:0/3890697421 conn(0x7f1db40fd8c0 msgr2=0x7f1db40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 shutdown_connections 2024-09-05T06:50:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:17.144+0000 7f1dbb165640 1 -- 172.21.15.86:0/3890697421 wait complete. 2024-09-05T06:50:17.523 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.87 MB/sec execute 337 sec latency 18390.129 ms 2024-09-05T06:50:17.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:17 smithi086 ceph-mon[30943]: pgmap v280: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:17 smithi096 ceph-mon[34468]: pgmap v280: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:17 smithi115 ceph-mon[34791]: pgmap v280: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:17.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:50:18.185 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:18.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.87 MB/sec execute 338 sec latency 19390.255 ms 2024-09-05T06:50:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- 172.21.15.86:0/1007472236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214106f10 msgr2=0x7ff21410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 --2- 172.21.15.86:0/1007472236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214106f10 0x7ff21410ea90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009ca0 tx=0x7ff20c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- 172.21.15.86:0/1007472236 shutdown_connections 2024-09-05T06:50:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 --2- 172.21.15.86:0/1007472236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214106f10 0x7ff21410ea90 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 --2- 172.21.15.86:0/1007472236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 0x7ff2141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 --2- 172.21.15.86:0/1007472236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214101de0 0x7ff2141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- 172.21.15.86:0/1007472236 >> 172.21.15.86:0/1007472236 conn(0x7ff2140fd8c0 msgr2=0x7ff2140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- 172.21.15.86:0/1007472236 shutdown_connections 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- 172.21.15.86:0/1007472236 wait complete. 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 Processor -- start 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.660+0000 7ff21a764640 1 -- start start 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 0x7ff2141989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214106f10 0x7ff21419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff214111ec0 con 0x7ff214101de0 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff214111d40 con 0x7ff214106f10 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff21a764640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff214112040 con 0x7ff214102790 2024-09-05T06:50:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33474/0 (socket says 172.21.15.86:33474) 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 -- 172.21.15.86:0/4094942171 learned_addr learned my addr 172.21.15.86:0/4094942171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff218d01640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214106f10 0x7ff21419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff2137fe640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 0x7ff2141989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 -- 172.21.15.86:0/4094942171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 msgr2=0x7ff2141989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 0x7ff2141989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 -- 172.21.15.86:0/4094942171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214106f10 msgr2=0x7ff21419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214106f10 0x7ff21419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 -- 172.21.15.86:0/4094942171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419d430 con 0x7ff214101de0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.661+0000 7ff213fff640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b4a0 tx=0x7ff20400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.662+0000 7ff2117fa640 1 -- 172.21.15.86:0/4094942171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204014070 con 0x7ff214101de0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.662+0000 7ff2117fa640 1 -- 172.21.15.86:0/4094942171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff204010ac0 con 0x7ff214101de0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.662+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419d720 con 0x7ff214101de0 2024-09-05T06:50:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.662+0000 7ff2117fa640 1 -- 172.21.15.86:0/4094942171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204017c30 con 0x7ff214101de0 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.662+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2141a5000 con 0x7ff214101de0 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.664+0000 7ff2117fa640 1 -- 172.21.15.86:0/4094942171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff204017070 con 0x7ff214101de0 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.664+0000 7ff21cb3d740 1 Processor -- start 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 -- start start 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1d49362f0 con 0x55d1d49d1b40 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1d48ddcf0 con 0x55d1d49db370 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d1d49322b0 con 0x55d1d49d05d0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff2117fa640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 0x7ff1fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33488/0 (socket says 172.21.15.86:33488) 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 -- 172.21.15.86:0/1854313711 learned_addr learned my addr 172.21.15.86:0/1854313711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff213fff640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff2137fe640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 0x7ff1fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff2117fa640 1 -- 172.21.15.86:0/4094942171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ff2040ceeb0 con 0x7ff214101de0 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 -- 172.21.15.86:0/1854313711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 msgr2=0x55d1d49d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 -- 172.21.15.86:0/1854313711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 msgr2=0x55d1d49dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff218d01640 1 -- 172.21.15.86:0/1854313711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d48c23a0 con 0x55d1d49d1b40 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.665+0000 7ff213fff640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff218d01640 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff20c035000 tx=0x7ff20c031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa58b408a2004f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff1fa7fc640 1 -- 172.21.15.86:0/1854313711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c004b40 con 0x55d1d49d1b40 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff1fa7fc640 1 -- 172.21.15.86:0/1854313711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff20c036070 con 0x55d1d49d1b40 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff1fa7fc640 1 -- 172.21.15.86:0/1854313711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0305c0 con 0x55d1d49d1b40 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff218d01640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 0x7ff1fc0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff20c04e000 tx=0x7ff20c04c000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1854313711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 msgr2=0x55d1d49d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff20c035000 tx=0x7ff20c031f90 comp rx=0 tx=0).stop 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1854313711 shutdown_connections 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.666+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1854313711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d0090 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1854313711 >> 172.21.15.86:0/1854313711 conn(0x55d1d4999310 msgr2=0x55d1d49ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1854313711 shutdown_connections 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1854313711 wait complete. 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- start start 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 Processor -- start 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 0x55d1d49d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1d49ddef0 con 0x55d1d49d05d0 2024-09-05T06:50:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1d48f1060 con 0x55d1d49d1b40 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d1d49de070 con 0x55d1d49db370 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff21cb3d740 10 client.0 _ll_register_callbacks cb 0x55d1d46448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff213fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff213fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33494/0 (socket says 172.21.15.86:33494) 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.667+0000 7ff213fff640 1 -- 172.21.15.86:0/1014360082 learned_addr learned my addr 172.21.15.86:0/1014360082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff218d01640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff2137fe640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 0x55d1d49d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 -- 172.21.15.86:0/1014360082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 msgr2=0x55d1d49d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 -- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 msgr2=0x55d1d49d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 0x55d1d49d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 -- 172.21.15.86:0/1014360082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d46a73c0 con 0x55d1d49d05d0 2024-09-05T06:50:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff218d01640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff2137fe640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 0x55d1d49d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff213fff640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff20409afd0 tx=0x7ff20400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136eb292c3612d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20409f3f0 con 0x55d1d49d05d0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ff20409f110 con 0x55d1d49d05d0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff20c036070 con 0x55d1d49d05d0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2040a3cb0 con 0x55d1d49d05d0 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff21cb3d740 10 client.17559 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1d49e99b0 con 0x55d1d49d05d0 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.668+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff20c036070 con 0x55d1d49d05d0 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff204010c70 con 0x55d1d49d05d0 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 10 client.17559 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 10 client.17559 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 20 client.17559 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d49eae70 0x55d1d49ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 4 client.17559 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d4943860 con 0x55d1d49eae70 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff2137fe640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d49eae70 0x55d1d49ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff1f37fe640 10 client.17559 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:50:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.669+0000 7ff2137fe640 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d49eae70 0x55d1d49ed280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4191b22e024c5527 server_cookie=8d3a7b62400d2cbc in_seq=0 out_seq=0 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff1f37fe640 1 -- 172.21.15.86:0/1014360082 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1d4943860 con 0x55d1d49eae70 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff1f37fe640 10 client.17559 handle_command_reply: tid=0 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 client.17559 shutdown 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 msgr2=0x55d1d49dad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff20409afd0 tx=0x7ff20400ccb0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 shutdown_connections 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d49eae70 0x55d1d49ed280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d1b40 0x55d1d49d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff1f37fe640 0 client.17559 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:50:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/1014360082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d05d0 0x55d1d49dad30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 >> 172.21.15.86:0/1014360082 conn(0x55d1d4999310 msgr2=0x55d1d49d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 shutdown_connections 2024-09-05T06:50:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.670+0000 7ff21cb3d740 1 -- 172.21.15.86:0/1014360082 wait complete. 2024-09-05T06:50:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.671+0000 7ff21cb3d740 20 client.17559 trim_cache size 0 max 16384 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 Processor -- start 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 -- start start 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 0x55d1d49cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141af1d0 con 0x55d1d49d1b40 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141b0d10 con 0x55d1d49d05d0 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141af8a0 con 0x55d1d49db370 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33500/0 (socket says 172.21.15.86:33500) 2024-09-05T06:50:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 -- 172.21.15.86:0/2774877591 learned_addr learned my addr 172.21.15.86:0/2774877591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff213fff640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 0x55d1d49cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff218d01640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 -- 172.21.15.86:0/2774877591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 msgr2=0x55d1d49d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 -- 172.21.15.86:0/2774877591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 msgr2=0x55d1d49cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 0x55d1d49cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff2137fe640 1 -- 172.21.15.86:0/2774877591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d49d6f90 con 0x55d1d49d1b40 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.684+0000 7ff213fff640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 0x55d1d49cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.685+0000 7ff2137fe640 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55d1d49d1070 tx=0x7ff208016430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f45b31eb4cf05b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.685+0000 7ff1f9ffb640 1 -- 172.21.15.86:0/2774877591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208016b90 con 0x55d1d49d1b40 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.685+0000 7ff1f9ffb640 1 -- 172.21.15.86:0/2774877591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff208016d30 con 0x55d1d49d1b40 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.685+0000 7ff1f9ffb640 1 -- 172.21.15.86:0/2774877591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208014710 con 0x55d1d49d1b40 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- 172.21.15.86:0/2774877591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 msgr2=0x55d1d49d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55d1d49d1070 tx=0x7ff208016430 comp rx=0 tx=0).stop 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- 172.21.15.86:0/2774877591 shutdown_connections 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49db370 0x55d1d49d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49d3120 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/2774877591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49d05d0 0x55d1d49cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- 172.21.15.86:0/2774877591 >> 172.21.15.86:0/2774877591 conn(0x55d1d4999310 msgr2=0x55d1d49da060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- 172.21.15.86:0/2774877591 shutdown_connections 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- 172.21.15.86:0/2774877591 wait complete. 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- start start 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 Processor -- start 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141b0640 con 0x55d1d49d1b40 2024-09-05T06:50:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141aff70 con 0x55d1d49db370 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.686+0000 7ff21cb3d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141ad1d0 con 0x55d1d49d05d0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff21cb3d740 10 client.0 _ll_register_callbacks cb 0x7ff2040174e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33512/0 (socket says 172.21.15.86:33512) 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 -- 172.21.15.86:0/252118400 learned_addr learned my addr 172.21.15.86:0/252118400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff213fff640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff218d01640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 -- 172.21.15.86:0/252118400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 msgr2=0x55d1d49df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 -- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 msgr2=0x55d1d49dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 -- 172.21.15.86:0/252118400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d4a54df0 con 0x55d1d49d1b40 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff2137fe640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff208004700 tx=0x7ff208024590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa0260dc5c7ab34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff1f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20802e020 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ff20802a420 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208016d30 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff208039050 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff21cb3d740 10 client.17565 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.687+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1d4a560f0 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff208016d30 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff208032600 con 0x55d1d49d1b40 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 10 client.17565 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 10 client.17565 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:50:18.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 20 client.17565 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d4a56830 0x55d1d4a58c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 4 client.17565 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1d493dd70 con 0x55d1d4a56830 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff213fff640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d4a56830 0x55d1d4a58c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff1f37fe640 10 client.17565 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.688+0000 7ff213fff640 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d4a56830 0x55d1d4a58c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9eefe17231b7312c server_cookie=31bfcfe788d645bf in_seq=0 out_seq=0 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff1f37fe640 1 -- 172.21.15.86:0/252118400 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1d493dd70 con 0x55d1d4a56830 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff1f37fe640 10 client.17565 handle_command_reply: tid=0 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 client.17565 shutdown 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 msgr2=0x55d1d49df880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff208004700 tx=0x7ff208024590 comp rx=0 tx=0).stop 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 shutdown_connections 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55d1d4a56830 0x55d1d4a58c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1d49db370 0x55d1d49dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d49d1b40 0x55d1d49df880 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 --2- 172.21.15.86:0/252118400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1d49d05d0 0x55d1d49df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 >> 172.21.15.86:0/252118400 conn(0x55d1d4999310 msgr2=0x55d1d49cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff1f37fe640 0 client.17565 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 shutdown_connections 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 1 -- 172.21.15.86:0/252118400 wait complete. 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21cb3d740 20 client.17565 trim_cache size 0 max 16384 2024-09-05T06:50:18.703 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 msgr2=0x7ff1fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 0x7ff1fc0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff20c04e000 tx=0x7ff20c04c000 comp rx=0 tx=0).stop 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 msgr2=0x7ff214198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.689+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b4a0 tx=0x7ff20400b960 comp rx=0 tx=0).stop 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 shutdown_connections 2024-09-05T06:50:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff1fc0a2cf0 0x7ff1fc0a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff214106f10 0x7ff21419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff214102790 0x7ff2141989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 --2- 172.21.15.86:0/4094942171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff214101de0 0x7ff214198480 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 >> 172.21.15.86:0/4094942171 conn(0x7ff2140fd8c0 msgr2=0x7ff2140fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 shutdown_connections 2024-09-05T06:50:18.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:18.690+0000 7ff21a764640 1 -- 172.21.15.86:0/4094942171 wait complete. 2024-09-05T06:50:19.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:50:19.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.86 MB/sec execute 339 sec latency 20390.335 ms 2024-09-05T06:50:19.693 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:19 smithi086 ceph-mon[30943]: pgmap v281: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:19 smithi096 ceph-mon[34468]: pgmap v281: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:19.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:19 smithi115 ceph-mon[34791]: pgmap v281: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-05T06:50:20.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 -- 172.21.15.86:0/6946846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524106f10 msgr2=0x7f852410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 --2- 172.21.15.86:0/6946846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524106f10 0x7f852410ea90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f851c009ca0 tx=0x7f851c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 -- 172.21.15.86:0/6946846 shutdown_connections 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 --2- 172.21.15.86:0/6946846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524106f10 0x7f852410ea90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 --2- 172.21.15.86:0/6946846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524102790 0x7f85241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 --2- 172.21.15.86:0/6946846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524101de0 0x7f85241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 -- 172.21.15.86:0/6946846 >> 172.21.15.86:0/6946846 conn(0x7f85240fd8c0 msgr2=0x7f85240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 -- 172.21.15.86:0/6946846 shutdown_connections 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.195+0000 7f852c6a4640 1 -- 172.21.15.86:0/6946846 wait complete. 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 Processor -- start 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 -- start start 2024-09-05T06:50:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524102790 0x7f8524198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524106f10 0x7f852419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8524111e20 con 0x7f8524101de0 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8524111ca0 con 0x7f8524106f10 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852c6a4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8524111fa0 con 0x7f8524102790 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33524/0 (socket says 172.21.15.86:33524) 2024-09-05T06:50:20.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 -- 172.21.15.86:0/2644829626 learned_addr learned my addr 172.21.15.86:0/2644829626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f8529c3f640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524102790 0x7f8524198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852ac41640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524106f10 0x7f852419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 -- 172.21.15.86:0/2644829626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524102790 msgr2=0x7f8524198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524102790 0x7f8524198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 -- 172.21.15.86:0/2644829626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524106f10 msgr2=0x7f852419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524106f10 0x7f852419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.196+0000 7f852a440640 1 -- 172.21.15.86:0/2644829626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852419d1f0 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f852a440640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f852000c920 tx=0x7f852000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f851b7fe640 1 -- 172.21.15.86:0/2644829626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520007c70 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852419d4e0 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f851b7fe640 1 -- 172.21.15.86:0/2644829626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520007e10 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f851b7fe640 1 -- 172.21.15.86:0/2644829626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520018820 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.197+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a4dc0 con 0x7f8524101de0 2024-09-05T06:50:20.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.199+0000 7f851b7fe640 1 -- 172.21.15.86:0/2644829626 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f85200189c0 con 0x7f8524101de0 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.199+0000 7f852ea7d740 1 Processor -- start 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 -- start start 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a96da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a97a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d8a8d32f0 con 0x555d8a96eb40 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d8a87acf0 con 0x555d8a96d5d0 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ea7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d8a8cf2b0 con 0x555d8a978370 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f851b7fe640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 0x7f85100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33530/0 (socket says 172.21.15.86:33530) 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 -- 172.21.15.86:0/2116100506 learned_addr learned my addr 172.21.15.86:0/2116100506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f8529c3f640 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a97a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 0x7f85100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f851b7fe640 1 -- 172.21.15.86:0/2644829626 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f85200cfab0 con 0x7f8524101de0 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852a440640 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a96da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 -- 172.21.15.86:0/2116100506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 msgr2=0x555d8a97a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a97a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 -- 172.21.15.86:0/2116100506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 msgr2=0x555d8a96da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a96da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.200+0000 7f852ac41640 1 -- 172.21.15.86:0/2116100506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8a85f3a0 con 0x555d8a96eb40 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.201+0000 7f852ac41640 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f851c005cc0 tx=0x7f851c031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445213ef3fee5cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.201+0000 7f8518ff9640 1 -- 172.21.15.86:0/2116100506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c031bf0 con 0x555d8a96eb40 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.201+0000 7f8518ff9640 1 -- 172.21.15.86:0/2116100506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f851c04e070 con 0x555d8a96eb40 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.201+0000 7f8518ff9640 1 -- 172.21.15.86:0/2116100506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c030db0 con 0x555d8a96eb40 2024-09-05T06:50:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- 172.21.15.86:0/2116100506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 msgr2=0x555d8a96d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f851c005cc0 tx=0x7f851c031550 comp rx=0 tx=0).stop 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ac41640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 0x7f85100a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f851c04d000 tx=0x7f851c04b000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- 172.21.15.86:0/2116100506 shutdown_connections 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a97a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a96da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- 172.21.15.86:0/2116100506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a96d090 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- 172.21.15.86:0/2116100506 >> 172.21.15.86:0/2116100506 conn(0x555d8a936310 msgr2=0x555d8a969c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- 172.21.15.86:0/2116100506 shutdown_connections 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- 172.21.15.86:0/2116100506 wait complete. 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- start start 2024-09-05T06:50:20.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 Processor -- start 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96d5d0 0x555d8a977dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 0x555d8a975810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d8a97b070 con 0x555d8a96eb40 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d8a88e060 con 0x555d8a978370 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555d8a97b1f0 con 0x555d8a96d5d0 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f852ea7d740 10 client.0 _ll_register_callbacks cb 0x555d8a5e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f8529c3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f8529c3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33546/0 (socket says 172.21.15.86:33546) 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.202+0000 7f8529c3f640 1 -- 172.21.15.86:0/690511133 learned_addr learned my addr 172.21.15.86:0/690511133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852ac41640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 0x555d8a975810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852a440640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96d5d0 0x555d8a977dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 -- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96d5d0 msgr2=0x555d8a977dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96d5d0 0x555d8a977dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 -- 172.21.15.86:0/690511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 msgr2=0x555d8a975810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 0x555d8a975810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 -- 172.21.15.86:0/690511133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8a6443c0 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852ac41640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 0x555d8a975810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f8529c3f640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f851400be40 tx=0x7f8514012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2f0faf84a4ca99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f850dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851401f060 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851c04e070 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f85140108a0 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8514026050 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852ea7d740 10 client.17577 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.203+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d8a986bf0 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f851c04e070 con 0x555d8a96eb40 2024-09-05T06:50:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f8514019030 con 0x555d8a96eb40 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 10 client.17577 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 10 client.17577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 20 client.17577 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a988100 0x555d8a98a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 4 client.17577 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.204+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d8a8e0860 con 0x555d8a988100 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.205+0000 7f852a440640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a988100 0x555d8a98a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.205+0000 7f852a440640 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a988100 0x555d8a98a520 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b46915096c26b77e server_cookie=4b59650e068eb3ef in_seq=0 out_seq=0 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.205+0000 7f850dffb640 10 client.17577 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f850dffb640 1 -- 172.21.15.86:0/690511133 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d8a8e0860 con 0x555d8a988100 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f850dffb640 10 client.17577 handle_command_reply: tid=0 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 client.17577 shutdown 2024-09-05T06:50:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 msgr2=0x555d8a975080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f851400be40 tx=0x7f8514012820 comp rx=0 tx=0).stop 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 shutdown_connections 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a988100 0x555d8a98a520 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a978370 0x555d8a975810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a975080 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 --2- 172.21.15.86:0/690511133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96d5d0 0x555d8a977dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 >> 172.21.15.86:0/690511133 conn(0x555d8a936310 msgr2=0x555d8a96f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f850dffb640 0 client.17577 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 shutdown_connections 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 1 -- 172.21.15.86:0/690511133 wait complete. 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.206+0000 7f852ea7d740 20 client.17577 trim_cache size 0 max 16384 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 Processor -- start 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 -- start start 2024-09-05T06:50:20.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a975050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a990330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85241af060 con 0x555d8a96eb40 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85241b0ba0 con 0x555d8a96d5d0 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f852ea7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85241af730 con 0x555d8a978370 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f8529c3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f8529c3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33560/0 (socket says 172.21.15.86:33560) 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.219+0000 7f8529c3f640 1 -- 172.21.15.86:0/3013101199 learned_addr learned my addr 172.21.15.86:0/3013101199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f852a440640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a975050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f852ac41640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a990330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 -- 172.21.15.86:0/3013101199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 msgr2=0x555d8a990330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a990330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 -- 172.21.15.86:0/3013101199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 msgr2=0x555d8a975050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a975050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 -- 172.21.15.86:0/3013101199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8a9735d0 con 0x555d8a96eb40 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f852a440640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a975050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f8529c3f640 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f85140059f0 tx=0x7f8514005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accd1db0e97ee69b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f850ffff640 1 -- 172.21.15.86:0/3013101199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514004030 con 0x555d8a96eb40 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f850ffff640 1 -- 172.21.15.86:0/3013101199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8514005520 con 0x555d8a96eb40 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.220+0000 7f850ffff640 1 -- 172.21.15.86:0/3013101199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851400f860 con 0x555d8a96eb40 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- 172.21.15.86:0/3013101199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 msgr2=0x555d8a98fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f85140059f0 tx=0x7f8514005c50 comp rx=0 tx=0).stop 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- 172.21.15.86:0/3013101199 shutdown_connections 2024-09-05T06:50:20.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a978370 0x555d8a990330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a96eb40 0x555d8a98fdf0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 --2- 172.21.15.86:0/3013101199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a975050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- 172.21.15.86:0/3013101199 >> 172.21.15.86:0/3013101199 conn(0x555d8a936310 msgr2=0x555d8a974940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- 172.21.15.86:0/3013101199 shutdown_connections 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- 172.21.15.86:0/3013101199 wait complete. 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 -- start start 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.221+0000 7f852ea7d740 1 Processor -- start 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a980c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96eb40 0x555d8a97bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85241b04d0 con 0x555d8a978370 2024-09-05T06:50:20.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85241afe00 con 0x555d8a96d5d0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85241ad060 con 0x555d8a96eb40 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 10 client.0 _ll_register_callbacks cb 0x7f851400b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33562/0 (socket says 172.21.15.86:33562) 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 -- 172.21.15.86:0/21088213 learned_addr learned my addr 172.21.15.86:0/21088213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f8529c3f640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96eb40 0x555d8a97bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852a440640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a980c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 -- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96eb40 msgr2=0x555d8a97bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96eb40 0x555d8a97bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 -- 172.21.15.86:0/21088213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 msgr2=0x555d8a980c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a980c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 -- 172.21.15.86:0/21088213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8a97cd00 con 0x555d8a978370 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ac41640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f851c006010 tx=0x7f851c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b66b41539ef66a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:20.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f850dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c004620 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f851c0047c0 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.222+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8514005520 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f851c06b050 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 10 client.17583 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d8a9f7260 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8514005520 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f851c02f7b0 con 0x555d8a978370 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 10 client.17583 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 10 client.17583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 20 client.17583 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a9f75b0 0x555d8a9f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 4 client.17583 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:50:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d8a8dad70 con 0x555d8a9f75b0 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.223+0000 7f852a440640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a9f75b0 0x555d8a9f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f850dffb640 10 client.17583 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852a440640 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a9f75b0 0x555d8a9f99a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8d491f59428e15 server_cookie=2ad8840eae2a46f0 in_seq=0 out_seq=0 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f850dffb640 1 -- 172.21.15.86:0/21088213 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d8a8dad70 con 0x555d8a9f75b0 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f850dffb640 10 client.17583 handle_command_reply: tid=0 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 client.17583 shutdown 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 msgr2=0x555d8a97c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f851c006010 tx=0x7f851c005b50 comp rx=0 tx=0).stop 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 shutdown_connections 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555d8a9f75b0 0x555d8a9f99a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d8a978370 0x555d8a97c4d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555d8a96eb40 0x555d8a97bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 --2- 172.21.15.86:0/21088213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d8a96d5d0 0x555d8a980c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 >> 172.21.15.86:0/21088213 conn(0x555d8a936310 msgr2=0x555d8a983300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f850dffb640 0 client.17583 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 shutdown_connections 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 1 -- 172.21.15.86:0/21088213 wait complete. 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.224+0000 7f852ea7d740 20 client.17583 trim_cache size 0 max 16384 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 msgr2=0x7f85100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 0x7f85100a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f851c04d000 tx=0x7f851c04b000 comp rx=0 tx=0).stop 2024-09-05T06:50:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 msgr2=0x7f8524198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f852000c920 tx=0x7f852000cde0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 shutdown_connections 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85100a2cf0 0x7f85100a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8524106f10 0x7f852419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524102790 0x7f8524198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 --2- 172.21.15.86:0/2644829626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8524101de0 0x7f8524198240 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 >> 172.21.15.86:0/2644829626 conn(0x7f85240fd8c0 msgr2=0x7f85240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 shutdown_connections 2024-09-05T06:50:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:20.225+0000 7f852c6a4640 1 -- 172.21.15.86:0/2644829626 wait complete. 2024-09-05T06:50:20.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.86 MB/sec execute 340 sec latency 21390.394 ms 2024-09-05T06:50:21.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:50:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:21.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.85 MB/sec execute 341 sec latency 22390.419 ms 2024-09-05T06:50:21.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:21 smithi086 ceph-mon[30943]: pgmap v282: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-05T06:50:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 -- 172.21.15.86:0/1876840731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4106a80 msgr2=0x7f24c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 --2- 172.21.15.86:0/1876840731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4106a80 0x7f24c4106e60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009930 tx=0x7f24c002a420 comp rx=0 tx=0).stop 2024-09-05T06:50:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 -- 172.21.15.86:0/1876840731 shutdown_connections 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 --2- 172.21.15.86:0/1876840731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4101420 0x7f24c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 --2- 172.21.15.86:0/1876840731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4100a80 0x7f24c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 --2- 172.21.15.86:0/1876840731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4106a80 0x7f24c4106e60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.747+0000 7f24cb921640 1 -- 172.21.15.86:0/1876840731 >> 172.21.15.86:0/1876840731 conn(0x7f24c40fc7a0 msgr2=0x7f24c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- 172.21.15.86:0/1876840731 shutdown_connections 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- 172.21.15.86:0/1876840731 wait complete. 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 Processor -- start 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- start start 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4100a80 0x7f24c41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 0x7f24c419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110d50 con 0x7f24c4101420 2024-09-05T06:50:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110bd0 con 0x7f24c4106a80 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.748+0000 7f24cb921640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110ed0 con 0x7f24c4100a80 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33578/0 (socket says 172.21.15.86:33578) 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 -- 172.21.15.86:0/1551309456 learned_addr learned my addr 172.21.15.86:0/1551309456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4100a80 0x7f24c41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 0x7f24c419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 -- 172.21.15.86:0/1551309456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4100a80 msgr2=0x7f24c41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4100a80 0x7f24c41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 -- 172.21.15.86:0/1551309456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 msgr2=0x7f24c419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 0x7f24c419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 -- 172.21.15.86:0/1551309456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c419c280 con 0x7f24c4101420 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 0x7f24c419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4a0 tx=0x7f24b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24ba7fc640 1 -- 172.21.15.86:0/1551309456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4014070 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c419c570 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24ba7fc640 1 -- 172.21.15.86:0/1551309456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b4010ac0 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24ba7fc640 1 -- 172.21.15.86:0/1551309456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4017c30 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.749+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a3e50 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.752+0000 7f24ba7fc640 1 -- 172.21.15.86:0/1551309456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f24b4017070 con 0x7f24c4101420 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.752+0000 7f24cdcfa740 1 Processor -- start 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.752+0000 7f24cdcfa740 1 -- start start 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d0fea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d10d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b2d0642f0 con 0x560b2d0ffb40 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b2d00bcf0 con 0x560b2d10b580 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560b2d0602b0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33590/0 (socket says 172.21.15.86:33590) 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 -- 172.21.15.86:0/3918346496 learned_addr learned my addr 172.21.15.86:0/3918346496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d0fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 -- 172.21.15.86:0/3918346496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 msgr2=0x560b2d0fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d0fea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 -- 172.21.15.86:0/3918346496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 msgr2=0x560b2d10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24ba7fc640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 -- 172.21.15.86:0/3918346496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2cff03a0 con 0x560b2d0ffb40 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d0fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24ba7fc640 1 -- 172.21.15.86:0/1551309456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f24b40cee40 con 0x7f24c4101420 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.753+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007f40 tx=0x7f24bc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d3f0a31767b4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24af7fe640 1 -- 172.21.15.86:0/3918346496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc014070 con 0x560b2d0ffb40 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24af7fe640 1 -- 172.21.15.86:0/3918346496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc004d20 con 0x560b2d0ffb40 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24af7fe640 1 -- 172.21.15.86:0/3918346496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0054c0 con 0x560b2d0ffb40 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3918346496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 msgr2=0x560b2d0fe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f24c0006010 tx=0x7f24c0000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007f40 tx=0x7f24bc00e7e0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3918346496 shutdown_connections 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d0fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3918346496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d0fe090 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3918346496 >> 172.21.15.86:0/3918346496 conn(0x560b2d0c7310 msgr2=0x560b2d0fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3918346496 shutdown_connections 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.754+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3918346496 wait complete. 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 -- start start 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 Processor -- start 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d108240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d10b580 0x560b2d108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b2d10e280 con 0x560b2d0fe5d0 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b2d01f060 con 0x560b2d0ffb40 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560b2d10e400 con 0x560b2d10b580 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24cdcfa740 10 client.0 _ll_register_callbacks cb 0x560b2cd728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24c96bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24c96bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33592/0 (socket says 172.21.15.86:33592) 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24c96bd640 1 -- 172.21.15.86:0/1201384366 learned_addr learned my addr 172.21.15.86:0/1201384366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d10b580 0x560b2d108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.755+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 -- 172.21.15.86:0/1201384366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d10b580 msgr2=0x560b2d108850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d10b580 0x560b2d108850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 -- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 msgr2=0x560b2d108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d108240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 -- 172.21.15.86:0/1201384366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2cdd53c0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24c96bd640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009930 tx=0x7f24c002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c0043195aa5a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24acff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c004c030 con 0x560b2d0fe5d0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f24c0005020 con 0x560b2d0fe5d0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc004d20 con 0x560b2d0fe5d0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24c0058050 con 0x560b2d0fe5d0 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24cdcfa740 10 client.17595 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b2d119cc0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.756+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc004d20 con 0x560b2d0fe5d0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f24c00047a0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 10 client.17595 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 20 client.17595 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d11b0d0 0x560b2d11d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 4 client.17595 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2d071860 con 0x560b2d11b0d0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d11b0d0 0x560b2d11d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24acff9640 10 client.17595 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.757+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d11b0d0 0x560b2d11d530 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bac12e22d6ba43 server_cookie=b29cbdd7d6772cd7 in_seq=0 out_seq=0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24acff9640 1 -- 172.21.15.86:0/1201384366 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f24b4079e50 con 0x560b2d11b0d0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24acff9640 10 client.17595 handle_command_reply: tid=0 2024-09-05T06:50:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 client.17595 shutdown 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 msgr2=0x560b2d10af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009930 tx=0x7f24c002a420 comp rx=0 tx=0).stop 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 shutdown_connections 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d11b0d0 0x560b2d11d530 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d10b580 0x560b2d108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/1201384366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0fe5d0 0x560b2d10af90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24acff9640 0 client.17595 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 >> 172.21.15.86:0/1201384366 conn(0x560b2d0c7310 msgr2=0x560b2d100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 shutdown_connections 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 1 -- 172.21.15.86:0/1201384366 wait complete. 2024-09-05T06:50:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.758+0000 7f24cdcfa740 20 client.17595 trim_cache size 0 max 16384 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 Processor -- start 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 -- start start 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d117520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d113a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41acfc0 con 0x560b2d0ffb40 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41aeb00 con 0x560b2d10b580 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41ad690 con 0x560b2d0fe5d0 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33606/0 (socket says 172.21.15.86:33606) 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24c8ebc640 1 -- 172.21.15.86:0/3521545000 learned_addr learned my addr 172.21.15.86:0/3521545000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.771+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d117520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 -- 172.21.15.86:0/3521545000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 msgr2=0x560b2d117520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d117520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 -- 172.21.15.86:0/3521545000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 msgr2=0x560b2d113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 -- 172.21.15.86:0/3521545000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2d1071f0 con 0x560b2d0ffb40 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cc2c0 tx=0x7f24b40bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7997299383b332 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24aeffd640 1 -- 172.21.15.86:0/3521545000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b401e040 con 0x560b2d0ffb40 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24aeffd640 1 -- 172.21.15.86:0/3521545000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560b2d071860 con 0x560b2d0ffb40 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.772+0000 7f24aeffd640 1 -- 172.21.15.86:0/3521545000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4017e60 con 0x560b2d0ffb40 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3521545000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 msgr2=0x560b2d107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cc2c0 tx=0x7f24b40bb040 comp rx=0 tx=0).stop 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3521545000 shutdown_connections 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d10b580 0x560b2d113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d0ffb40 0x560b2d107b00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3521545000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d117520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3521545000 >> 172.21.15.86:0/3521545000 conn(0x560b2d0c7310 msgr2=0x560b2d0f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3521545000 shutdown_connections 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3521545000 wait complete. 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- start start 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 Processor -- start 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d101830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d101d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 0x560b2d180e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41ae430 con 0x560b2d10b580 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41add60 con 0x560b2d0ffb40 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41aafc0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.773+0000 7f24cdcfa740 10 client.0 _ll_register_callbacks cb 0x7f24c0040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c8ebc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:54174/0 (socket says 172.21.15.86:54174) 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c8ebc640 1 -- 172.21.15.86:0/3947284639 learned_addr learned my addr 172.21.15.86:0/3947284639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 0x560b2d180e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 -- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 msgr2=0x560b2d101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 -- 172.21.15.86:0/3947284639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 msgr2=0x560b2d180e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 0x560b2d180e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 -- 172.21.15.86:0/3947284639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2d1814c0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c9ebe640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 0x560b2d180e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24c96bd640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d101830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24c0004b70 tx=0x7f24c0031520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95fa72d883566fd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24acff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0061060 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560b2d071860 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f24c0003800 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.774+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24c0068050 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 10 client.26776 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b2d188830 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560b2d071860 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f24c00051b0 con 0x560b2d0fe5d0 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 10 client.26776 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 20 client.26776 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d188f80 0x560b2d18b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 4 client.26776 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:50:21.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b2d06bd70 con 0x560b2d188f80 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.775+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d188f80 0x560b2d18b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24c8ebc640 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d188f80 0x560b2d18b3a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9d65a6f0879c1e server_cookie=659552a4c675c17e in_seq=0 out_seq=0 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24acff9640 10 client.26776 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24acff9640 1 -- 172.21.15.86:0/3947284639 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b2d06bd70 con 0x560b2d188f80 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24acff9640 10 client.26776 handle_command_reply: tid=0 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 client.26776 shutdown 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 msgr2=0x560b2d101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d101830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24c0004b70 tx=0x7f24c0031520 comp rx=0 tx=0).stop 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 shutdown_connections 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560b2d188f80 0x560b2d18b3a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b2d10b580 0x560b2d180e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b2d0ffb40 0x560b2d101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24acff9640 0 client.26776 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 --2- 172.21.15.86:0/3947284639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560b2d0fe5d0 0x560b2d101830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 >> 172.21.15.86:0/3947284639 conn(0x560b2d0c7310 msgr2=0x560b2d109aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 shutdown_connections 2024-09-05T06:50:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 1 -- 172.21.15.86:0/3947284639 wait complete. 2024-09-05T06:50:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.776+0000 7f24cdcfa740 20 client.26776 trim_cache size 0 max 16384 2024-09-05T06:50:21.792 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 msgr2=0x7f24b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f24c0006010 tx=0x7f24c0000bc0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 msgr2=0x7f24c4197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4a0 tx=0x7f24b400b960 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 shutdown_connections 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4106a80 0x7f24c419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4101420 0x7f24c4197810 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 --2- 172.21.15.86:0/1551309456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4100a80 0x7f24c41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 >> 172.21.15.86:0/1551309456 conn(0x7f24c40fc7a0 msgr2=0x7f24c40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 shutdown_connections 2024-09-05T06:50:21.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:21.777+0000 7f24cb921640 1 -- 172.21.15.86:0/1551309456 wait complete. 2024-09-05T06:50:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:21 smithi115 ceph-mon[34791]: pgmap v282: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-05T06:50:22.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:21 smithi096 ceph-mon[34468]: pgmap v282: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-05T06:50:22.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.85 MB/sec execute 342 sec latency 23390.452 ms 2024-09-05T06:50:23.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.84 MB/sec execute 343 sec latency 24390.841 ms 2024-09-05T06:50:23.535 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:50:23.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:50:23.791 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:23 smithi096 ceph-mon[34468]: pgmap v283: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-05T06:50:23.911 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:23 smithi086 ceph-mon[30943]: pgmap v283: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-05T06:50:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:23 smithi115 ceph-mon[34791]: pgmap v283: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 -- 172.21.15.86:0/3461802988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 msgr2=0x7fe328106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 --2- 172.21.15.86:0/3461802988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe328106e60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009930 tx=0x7fe31c02a420 comp rx=0 tx=0).stop 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 -- 172.21.15.86:0/3461802988 shutdown_connections 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 --2- 172.21.15.86:0/3461802988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe32810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 --2- 172.21.15.86:0/3461802988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 0x7fe328100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 --2- 172.21.15.86:0/3461802988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe328106e60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 -- 172.21.15.86:0/3461802988 >> 172.21.15.86:0/3461802988 conn(0x7fe3280fc7a0 msgr2=0x7fe3280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 -- 172.21.15.86:0/3461802988 shutdown_connections 2024-09-05T06:50:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.275+0000 7fe32ef77640 1 -- 172.21.15.86:0/3461802988 wait complete. 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 Processor -- start 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 -- start start 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 0x7fe3280fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe3280fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe328110cf0 con 0x7fe328106a80 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe328110b70 con 0x7fe328100a80 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32ef77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe328110e70 con 0x7fe328101420 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37916/0 (socket says 172.21.15.86:37916) 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 -- 172.21.15.86:0/1314814261 learned_addr learned my addr 172.21.15.86:0/1314814261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32cd13640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 0x7fe3280fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe327fff640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe3280fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 -- 172.21.15.86:0/1314814261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 msgr2=0x7fe3280fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe3280fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 -- 172.21.15.86:0/1314814261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 msgr2=0x7fe3280fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 0x7fe3280fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 -- 172.21.15.86:0/1314814261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281a66e0 con 0x7fe328106a80 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe327fff640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe3280fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.276+0000 7fe32d514640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe32000cce0 tx=0x7fe32000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.277+0000 7fe325ffb640 1 -- 172.21.15.86:0/1314814261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000f450 con 0x7fe328106a80 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.277+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281a69d0 con 0x7fe328106a80 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.277+0000 7fe325ffb640 1 -- 172.21.15.86:0/1314814261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe32000faa0 con 0x7fe328106a80 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.277+0000 7fe325ffb640 1 -- 172.21.15.86:0/1314814261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320018b70 con 0x7fe328106a80 2024-09-05T06:50:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.277+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281a6ec0 con 0x7fe328106a80 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.279+0000 7fe325ffb640 1 -- 172.21.15.86:0/1314814261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe320018d10 con 0x7fe328106a80 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 Processor -- start 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 -- start start 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 0x559f27f97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fa2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe325ffb640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 0x7fe3180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27fa2d40 0x559f27fa5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f27efd280 con 0x559f27f97b60 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f27ea4b10 con 0x559f27f98480 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe331350740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559f27ef9240 con 0x559f27fa2d40 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe327fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 0x559f27f97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe327fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 0x559f27f97f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37926/0 (socket says 172.21.15.86:37926) 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe327fff640 1 -- 172.21.15.86:0/935660155 learned_addr learned my addr 172.21.15.86:0/935660155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fa2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe327fff640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 0x7fe3180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe325ffb640 1 -- 172.21.15.86:0/1314814261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe3200cfcc0 con 0x7fe328106a80 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32d514640 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27fa2d40 0x559f27fa5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 -- 172.21.15.86:0/935660155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27fa2d40 msgr2=0x559f27fa5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27fa2d40 0x559f27fa5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 -- 172.21.15.86:0/935660155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 msgr2=0x559f27f97f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 0x559f27f97f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.280+0000 7fe32cd13640 1 -- 172.21.15.86:0/935660155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f27eb11f0 con 0x559f27f98480 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe327fff640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 0x7fe3180a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe314005fd0 tx=0x7fe314005be0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe32cd13640 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fa2800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031440 tx=0x7fe31c032d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc82a53c987018c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe312ffd640 1 -- 172.21.15.86:0/935660155 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035040 con 0x559f27f98480 2024-09-05T06:50:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe312ffd640 1 -- 172.21.15.86:0/935660155 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31c0316a0 con 0x559f27f98480 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe312ffd640 1 -- 172.21.15.86:0/935660155 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0047a0 con 0x559f27f98480 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe331350740 1 -- 172.21.15.86:0/935660155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 msgr2=0x559f27fa2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.281+0000 7fe331350740 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fa2800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031440 tx=0x7fe31c032d60 comp rx=0 tx=0).stop 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- 172.21.15.86:0/935660155 shutdown_connections 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27fa2d40 0x559f27fa5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fa2800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- 172.21.15.86:0/935660155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f97b60 0x559f27f97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- 172.21.15.86:0/935660155 >> 172.21.15.86:0/935660155 conn(0x559f27f940f0 msgr2=0x559f27f944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- 172.21.15.86:0/935660155 shutdown_connections 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- 172.21.15.86:0/935660155 wait complete. 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- start start 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 Processor -- start 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 0x559f27fa00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f27fa5910 con 0x559f27fa2d40 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f27eb7f10 con 0x559f27f97b60 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559f27fa5a90 con 0x559f27f98480 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.282+0000 7fe331350740 10 client.0 _ll_register_callbacks cb 0x559f27e80a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:36402/0 (socket says 172.21.15.86:36402) 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 -- 172.21.15.86:0/2450465500 learned_addr learned my addr 172.21.15.86:0/2450465500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32d514640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe327fff640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 0x559f27fa00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 -- 172.21.15.86:0/2450465500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 msgr2=0x559f27fa00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 0x559f27fa00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 -- 172.21.15.86:0/2450465500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 msgr2=0x559f27fa0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 -- 172.21.15.86:0/2450465500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f27c6e6a0 con 0x559f27f97b60 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe327fff640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 0x559f27fa00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe32cd13640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005460 tx=0x7fe31c032b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48004233f1f50f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe307fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c04e020 con 0x559f27f97b60 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c0316a0 con 0x559f27f97b60 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe31c004240 con 0x559f27f97b60 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe31c059050 con 0x559f27f97b60 2024-09-05T06:50:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe331350740 10 client.26813 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.283+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f27fb14a0 con 0x559f27f97b60 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31c0316a0 con 0x559f27f97b60 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe31c004830 con 0x559f27f97b60 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 10 client.26813 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 20 client.26813 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f27fb27f0 0x559f27fb4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 4 client.26813 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f27f0a7f0 con 0x559f27fb27f0 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.284+0000 7fe327fff640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f27fb27f0 0x559f27fb4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.285+0000 7fe307fff640 10 client.26813 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.285+0000 7fe327fff640 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f27fb27f0 0x559f27fb4be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7c4a26e1e4b0e server_cookie=313e0634791b5e7c in_seq=0 out_seq=0 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.285+0000 7fe307fff640 1 -- 172.21.15.86:0/2450465500 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f27f0a7f0 con 0x559f27fb27f0 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.285+0000 7fe307fff640 10 client.26813 handle_command_reply: tid=0 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.285+0000 7fe331350740 1 client.26813 shutdown 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 msgr2=0x559f27f9fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005460 tx=0x7fe31c032b40 comp rx=0 tx=0).stop 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 shutdown_connections 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f27fb27f0 0x559f27fb4be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f98480 0x559f27fa00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe307fff640 0 client.26813 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 --2- 172.21.15.86:0/2450465500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f97b60 0x559f27f9fba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 >> 172.21.15.86:0/2450465500 conn(0x559f27f940f0 msgr2=0x559f27f93270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 shutdown_connections 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 1 -- 172.21.15.86:0/2450465500 wait complete. 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.286+0000 7fe331350740 20 client.26813 trim_cache size 0 max 16384 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 Processor -- start 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 -- start start 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27f9ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27fa2d40 0x559f27fba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b11c0 con 0x559f27f98480 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b2d00 con 0x559f27fa2d40 2024-09-05T06:50:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe331350740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b1890 con 0x559f27f97b60 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37938/0 (socket says 172.21.15.86:37938) 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 -- 172.21.15.86:0/213159173 learned_addr learned my addr 172.21.15.86:0/213159173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe32cd13640 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27f9ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 -- 172.21.15.86:0/213159173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 msgr2=0x559f27f9ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27f9ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 -- 172.21.15.86:0/213159173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27fa2d40 msgr2=0x559f27fba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27fa2d40 0x559f27fba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe327fff640 1 -- 172.21.15.86:0/213159173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f27f93f20 con 0x559f27f98480 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.303+0000 7fe32cd13640 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27f9ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe327fff640 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe314014010 tx=0x7fe314014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22abb26e61f084a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe3127fc640 1 -- 172.21.15.86:0/213159173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31401cad0 con 0x559f27f98480 2024-09-05T06:50:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe3127fc640 1 -- 172.21.15.86:0/213159173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31401cc70 con 0x559f27f98480 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe3127fc640 1 -- 172.21.15.86:0/213159173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314026650 con 0x559f27f98480 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe331350740 1 -- 172.21.15.86:0/213159173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 msgr2=0x559f27f93160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.304+0000 7fe331350740 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe314014010 tx=0x7fe314014040 comp rx=0 tx=0).stop 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- 172.21.15.86:0/213159173 shutdown_connections 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27fa2d40 0x559f27fba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27f98480 0x559f27f93160 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- 172.21.15.86:0/213159173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27f9ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- 172.21.15.86:0/213159173 >> 172.21.15.86:0/213159173 conn(0x559f27f940f0 msgr2=0x559f27f92a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- 172.21.15.86:0/213159173 shutdown_connections 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- 172.21.15.86:0/213159173 wait complete. 2024-09-05T06:50:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- start start 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 Processor -- start 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27faace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b2630 con 0x559f27fa2d40 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b1f60 con 0x559f27f98480 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281af1c0 con 0x559f27f97b60 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.305+0000 7fe331350740 10 client.0 _ll_register_callbacks cb 0x7fe31c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37942/0 (socket says 172.21.15.86:37942) 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 -- 172.21.15.86:0/1751198227 learned_addr learned my addr 172.21.15.86:0/1751198227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32cd13640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27faace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe327fff640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 -- 172.21.15.86:0/1751198227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 msgr2=0x559f27faace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27faace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 -- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 msgr2=0x559f27fab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 -- 172.21.15.86:0/1751198227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f27fab910 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe32d514640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe320098710 tx=0x7fe3200cd4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ab662bd0d77447 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe307fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000fc50 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31401cc70 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe320099790 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe32009c440 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe331350740 10 client.17619 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.306+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f2801d7d0 con 0x559f27fa2d40 2024-09-05T06:50:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31401cc70 con 0x559f27fa2d40 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe32009b190 con 0x559f27fa2d40 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 10 client.17619 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 20 client.17619 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f2801dfa0 0x559f280203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 4 client.17619 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f27f04d00 con 0x559f2801dfa0 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe32cd13640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f2801dfa0 0x559f280203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe32cd13640 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f2801dfa0 0x559f280203c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9d9fb8784f6b1d server_cookie=5868f94bc5eb5541 in_seq=0 out_seq=0 2024-09-05T06:50:24.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.307+0000 7fe307fff640 10 client.17619 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:24.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.308+0000 7fe307fff640 1 -- 172.21.15.86:0/1751198227 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f27f04d00 con 0x559f2801dfa0 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.308+0000 7fe307fff640 10 client.17619 handle_command_reply: tid=0 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.308+0000 7fe331350740 1 client.17619 shutdown 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.308+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 msgr2=0x559f27fa7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.308+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe320098710 tx=0x7fe3200cd4e0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 shutdown_connections 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559f2801dfa0 0x559f280203c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f27fa2d40 0x559f27fa7cb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f27f98480 0x559f27fab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 --2- 172.21.15.86:0/1751198227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559f27f97b60 0x559f27faace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 >> 172.21.15.86:0/1751198227 conn(0x559f27f940f0 msgr2=0x559f27fa0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe307fff640 0 client.17619 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 shutdown_connections 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 1 -- 172.21.15.86:0/1751198227 wait complete. 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.309+0000 7fe331350740 20 client.17619 trim_cache size 0 max 16384 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "843e38fd-057f-4edf-b8bd-abfdf3cc7736", 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:50:24.326 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 msgr2=0x7fe3180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 0x7fe3180a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe314005fd0 tx=0x7fe314005be0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 msgr2=0x7fe3280fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe32000cce0 tx=0x7fe32000a620 comp rx=0 tx=0).stop 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 shutdown_connections 2024-09-05T06:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe3180a2cf0 0x7fe3180a51b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe328106a80 0x7fe3280fd860 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe328101420 0x7fe3280fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 --2- 172.21.15.86:0/1314814261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328100a80 0x7fe3280fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 >> 172.21.15.86:0/1314814261 conn(0x7fe3280fc7a0 msgr2=0x7fe32806a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 shutdown_connections 2024-09-05T06:50:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:24.310+0000 7fe32ef77640 1 -- 172.21.15.86:0/1314814261 wait complete. 2024-09-05T06:50:24.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.83 MB/sec execute 344 sec latency 25390.883 ms 2024-09-05T06:50:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:24 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:50:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:24 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:50:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:24 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:24.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:24 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:50:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:24 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:50:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:24 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:24.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:24 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:50:24.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:24.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:24 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:50:24.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:24 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:50:25.334 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:25.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.83 MB/sec execute 345 sec latency 26390.939 ms 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:25 smithi086 ceph-mon[30943]: pgmap v284: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:25 smithi096 ceph-mon[34468]: pgmap v284: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 -- 172.21.15.86:0/1553242611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074590 msgr2=0x7fe7d0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 --2- 172.21.15.86:0/1553242611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074590 0x7fe7d0074970 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009900 tx=0x7fe7cc02a480 comp rx=0 tx=0).stop 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 -- 172.21.15.86:0/1553242611 shutdown_connections 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 --2- 172.21.15.86:0/1553242611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0112b90 0x7fe7d0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 --2- 172.21.15.86:0/1553242611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0074eb0 0x7fe7d0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 --2- 172.21.15.86:0/1553242611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074590 0x7fe7d0074970 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 -- 172.21.15.86:0/1553242611 >> 172.21.15.86:0/1553242611 conn(0x7fe7d00fd8c0 msgr2=0x7fe7d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 -- 172.21.15.86:0/1553242611 shutdown_connections 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.799+0000 7fe7d7444640 1 -- 172.21.15.86:0/1553242611 wait complete. 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 Processor -- start 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 -- start start 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 0x7fe7d01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 0x7fe7d019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0118420 con 0x7fe7d0074eb0 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01182a0 con 0x7fe7d0112b90 2024-09-05T06:50:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d7444640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01185a0 con 0x7fe7d0074590 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d49df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d49df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37970/0 (socket says 172.21.15.86:37970) 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.800+0000 7fe7d49df640 1 -- 172.21.15.86:0/4194540548 learned_addr learned my addr 172.21.15.86:0/4194540548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 0x7fe7d01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 0x7fe7d019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 -- 172.21.15.86:0/4194540548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 msgr2=0x7fe7d01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 0x7fe7d01a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 -- 172.21.15.86:0/4194540548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 msgr2=0x7fe7d019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 0x7fe7d019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 -- 172.21.15.86:0/4194540548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d019ef90 con 0x7fe7d0074eb0 2024-09-05T06:50:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 0x7fe7d019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 0x7fe7d01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d49df640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000c890 tx=0x7fe7c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7c67fc640 1 -- 172.21.15.86:0/4194540548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004220 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7c67fc640 1 -- 172.21.15.86:0/4194540548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c00043c0 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d019f280 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7c67fc640 1 -- 172.21.15.86:0/4194540548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004a00 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.801+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01ab4d0 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.804+0000 7fe7c67fc640 1 -- 172.21.15.86:0/4194540548 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe7c0004ba0 con 0x7fe7d0074eb0 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.804+0000 7fe7d981d740 1 Processor -- start 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.804+0000 7fe7d981d740 1 -- start start 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca3212e0 con 0x55d9ca3c6680 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca2c8ce0 con 0x55d9ca3bbdc0 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d981d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca31d2a0 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37978/0 (socket says 172.21.15.86:37978) 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 -- 172.21.15.86:0/1332334420 learned_addr learned my addr 172.21.15.86:0/1332334420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7c67fc640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 0x7fe7bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 -- 172.21.15.86:0/1332334420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bb4a0 msgr2=0x55d9ca3bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 -- 172.21.15.86:0/1332334420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bbdc0 msgr2=0x55d9ca3c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 -- 172.21.15.86:0/1332334420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ca2ad440 con 0x55d9ca3c6680 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 0x7fe7bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7c67fc640 1 -- 172.21.15.86:0/4194540548 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe7c00cf600 con 0x7fe7d0074eb0 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.805+0000 7fe7d49df640 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00cc880 tx=0x7fe7c001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9875482a15d6c211 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7bb7fe640 1 -- 172.21.15.86:0/1332334420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c009b220 con 0x55d9ca3c6680 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7bb7fe640 1 -- 172.21.15.86:0/1332334420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c009bde0 con 0x55d9ca3c6680 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7bb7fe640 1 -- 172.21.15.86:0/1332334420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004e90 con 0x55d9ca3c6680 2024-09-05T06:50:25.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 -- 172.21.15.86:0/1332334420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 msgr2=0x55d9ca3c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 0x7fe7bc0a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0076600 tx=0x7fe7c8004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00cc880 tx=0x7fe7c001f000 comp rx=0 tx=0).stop 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 -- 172.21.15.86:0/1332334420 shutdown_connections 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c8a70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 --2- 172.21.15.86:0/1332334420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 -- 172.21.15.86:0/1332334420 >> 172.21.15.86:0/1332334420 conn(0x55d9ca3b7a30 msgr2=0x55d9ca3b7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 -- 172.21.15.86:0/1332334420 shutdown_connections 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.806+0000 7fe7d981d740 1 -- 172.21.15.86:0/1332334420 wait complete. 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 -- start start 2024-09-05T06:50:25.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 Processor -- start 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca3c9250 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca2dc050 con 0x55d9ca3c6680 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d9ca3c93d0 con 0x55d9ca3bbdc0 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d981d740 10 client.0 _ll_register_callbacks cb 0x55d9ca31c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d51e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d51e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37986/0 (socket says 172.21.15.86:37986) 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d51e0640 1 -- 172.21.15.86:0/2495377693 learned_addr learned my addr 172.21.15.86:0/2495377693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.807+0000 7fe7d49df640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 -- 172.21.15.86:0/2495377693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 msgr2=0x55d9ca3c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 -- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 msgr2=0x55d9ca3c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 -- 172.21.15.86:0/2495377693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ca0923c0 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc02a9a0 tx=0x7fe7cc031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2db7e86856981f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc035070 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe7cc004240 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c009bde0 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7cc04d050 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 10 client.17631 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9ca3d4c00 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c009bde0 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe7cc0049e0 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 10 client.17631 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.808+0000 7fe7d981d740 20 client.17631 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca3d60c0 0x55d9ca3d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7d981d740 4 client.17631 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9ca32e850 con 0x55d9ca3d60c0 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7d49df640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca3d60c0 0x55d9ca3d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7b8ff9640 10 client.17631 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:25.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.809+0000 7fe7d49df640 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca3d60c0 0x55d9ca3d8530 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a1fc3cb0e89406 server_cookie=720b538cc3090afc in_seq=0 out_seq=0 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/2495377693 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9ca32e850 con 0x55d9ca3d60c0 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7b8ff9640 10 client.17631 handle_command_reply: tid=0 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 client.17631 shutdown 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 msgr2=0x55d9ca3c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc02a9a0 tx=0x7fe7cc031890 comp rx=0 tx=0).stop 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 shutdown_connections 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca3d60c0 0x55d9ca3d8530 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7b8ff9640 0 client.17631 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2495377693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3c5fb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 >> 172.21.15.86:0/2495377693 conn(0x55d9ca3b7a30 msgr2=0x55d9ca3bd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 shutdown_connections 2024-09-05T06:50:25.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 1 -- 172.21.15.86:0/2495377693 wait complete. 2024-09-05T06:50:25.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.810+0000 7fe7d981d740 20 client.17631 trim_cache size 0 max 16384 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.827+0000 7fe7d981d740 1 Processor -- start 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.827+0000 7fe7d981d740 1 -- start start 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b57e0 con 0x55d9ca3c6680 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b7320 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d981d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b5eb0 con 0x55d9ca3bbdc0 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37990/0 (socket says 172.21.15.86:37990) 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 -- 172.21.15.86:0/2590623493 learned_addr learned my addr 172.21.15.86:0/2590623493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d49df640 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 -- 172.21.15.86:0/2590623493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 msgr2=0x55d9ca3dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 -- 172.21.15.86:0/2590623493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bb4a0 msgr2=0x55d9ca3d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 -- 172.21.15.86:0/2590623493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ca3b7860 con 0x55d9ca3c6680 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7d59e1640 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82971e691473def server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7baffd640 1 -- 172.21.15.86:0/2590623493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8004d20 con 0x55d9ca3c6680 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7baffd640 1 -- 172.21.15.86:0/2590623493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c8023070 con 0x55d9ca3c6680 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.828+0000 7fe7baffd640 1 -- 172.21.15.86:0/2590623493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80054f0 con 0x55d9ca3c6680 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- 172.21.15.86:0/2590623493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 msgr2=0x55d9ca3de340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8015070 comp rx=0 tx=0).stop 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- 172.21.15.86:0/2590623493 shutdown_connections 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3c6680 0x55d9ca3de340 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 --2- 172.21.15.86:0/2590623493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- 172.21.15.86:0/2590623493 >> 172.21.15.86:0/2590623493 conn(0x55d9ca3b7a30 msgr2=0x55d9ca3c1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- 172.21.15.86:0/2590623493 shutdown_connections 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- 172.21.15.86:0/2590623493 wait complete. 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.829+0000 7fe7d981d740 1 -- start start 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 Processor -- start 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b6c50 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b6580 con 0x55d9ca3c6680 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01b37e0 con 0x55d9ca3bbdc0 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d981d740 10 client.0 _ll_register_callbacks cb 0x7fe7cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38000/0 (socket says 172.21.15.86:38000) 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 -- 172.21.15.86:0/3871116007 learned_addr learned my addr 172.21.15.86:0/3871116007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 -- 172.21.15.86:0/3871116007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 msgr2=0x55d9ca3caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 -- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 msgr2=0x55d9ca3caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.830+0000 7fe7d51e0640 1 -- 172.21.15.86:0/3871116007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ca43ff30 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d51e0640 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc031440 tx=0x7fe7cc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c6f869b2a52554 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc035070 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fe7cc04a040 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c8023070 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7cc031900 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 10 client.17637 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9ca441200 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c8023070 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fe7cc044470 con 0x55d9ca3bb4a0 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 10 client.17637 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 20 client.17637 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca441910 0x55d9ca443db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 4 client.17637 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.831+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9ca328d60 con 0x55d9ca441910 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d49df640 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca441910 0x55d9ca443db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7b8ff9640 10 client.17637 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d49df640 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca441910 0x55d9ca443db0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51a50d730fd2049 server_cookie=697ee37e6d193286 in_seq=0 out_seq=0 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7b8ff9640 1 -- 172.21.15.86:0/3871116007 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9ca328d60 con 0x55d9ca441910 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7b8ff9640 10 client.17637 handle_command_reply: tid=0 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 client.17637 shutdown 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 msgr2=0x55d9ca3ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc031440 tx=0x7fe7cc003820 comp rx=0 tx=0).stop 2024-09-05T06:50:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 shutdown_connections 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d9ca441910 0x55d9ca443db0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ca3c6680 0x55d9ca3caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9ca3bbdc0 0x55d9ca3caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 --2- 172.21.15.86:0/3871116007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9ca3bb4a0 0x55d9ca3ca4c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 >> 172.21.15.86:0/3871116007 conn(0x55d9ca3b7a30 msgr2=0x55d9ca3b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7b8ff9640 0 client.17637 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 shutdown_connections 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 1 -- 172.21.15.86:0/3871116007 wait complete. 2024-09-05T06:50:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.832+0000 7fe7d981d740 20 client.17637 trim_cache size 0 max 16384 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "843e38fd-057f-4edf-b8bd-abfdf3cc7736": { 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "843e38fd-057f-4edf-b8bd-abfdf3cc7736", 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:25.841 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 msgr2=0x7fe7bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 0x7fe7bc0a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0076600 tx=0x7fe7c8004a40 comp rx=0 tx=0).stop 2024-09-05T06:50:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 msgr2=0x7fe7d01a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000c890 tx=0x7fe7c000cd50 comp rx=0 tx=0).stop 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 shutdown_connections 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe7bc0a2cf0 0x7fe7bc0a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7d0112b90 0x7fe7d019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d0074eb0 0x7fe7d01a4bb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 --2- 172.21.15.86:0/4194540548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7d0074590 0x7fe7d01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 >> 172.21.15.86:0/4194540548 conn(0x7fe7d00fd8c0 msgr2=0x7fe7d00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 shutdown_connections 2024-09-05T06:50:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:25.833+0000 7fe7d7444640 1 -- 172.21.15.86:0/4194540548 wait complete. 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:50:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:25 smithi115 ceph-mon[34791]: pgmap v284: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-05T06:50:26.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:50:26.524 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.82 MB/sec execute 346 sec latency 27390.988 ms 2024-09-05T06:50:27.262 INFO:tasks.cephfs.filesystem:scrub status for tag:843e38fd-057f-4edf-b8bd-abfdf3cc7736 - {'path': '/', 'tag': '843e38fd-057f-4edf-b8bd-abfdf3cc7736', 'options': 'recursive,force'} 2024-09-05T06:50:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:27.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.82 MB/sec execute 347 sec latency 28391.098 ms 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 -- 172.21.15.86:0/2554624262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4101de0 msgr2=0x7f95a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 --2- 172.21.15.86:0/2554624262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4101de0 0x7f95a41021c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9594009900 tx=0x7f959402a410 comp rx=0 tx=0).stop 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 -- 172.21.15.86:0/2554624262 shutdown_connections 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 --2- 172.21.15.86:0/2554624262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 --2- 172.21.15.86:0/2554624262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4102790 0x7f95a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 --2- 172.21.15.86:0/2554624262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4101de0 0x7f95a41021c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 -- 172.21.15.86:0/2554624262 >> 172.21.15.86:0/2554624262 conn(0x7f95a40fd8c0 msgr2=0x7f95a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 -- 172.21.15.86:0/2554624262 shutdown_connections 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.793+0000 7f95a9621640 1 -- 172.21.15.86:0/2554624262 wait complete. 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 Processor -- start 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 -- start start 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4101de0 0x7f95a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111ec0 con 0x7f95a4102790 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111d40 con 0x7f95a4106f10 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a9621640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4112040 con 0x7f95a4101de0 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38010/0 (socket says 172.21.15.86:38010) 2024-09-05T06:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a27fc640 1 -- 172.21.15.86:0/2066890574 learned_addr learned my addr 172.21.15.86:0/2066890574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a2ffd640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4101de0 0x7f95a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.794+0000 7f95a37fe640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 -- 172.21.15.86:0/2066890574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4101de0 msgr2=0x7f95a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4101de0 0x7f95a40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 -- 172.21.15.86:0/2066890574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 msgr2=0x7f95a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 -- 172.21.15.86:0/2066890574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a69e0 con 0x7f95a4102790 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a37fe640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a27fc640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f959800b4a0 tx=0x7f959800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598014070 con 0x7f95a4102790 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a6cd0 con 0x7f95a4102790 2024-09-05T06:50:27.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9598010ac0 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598017c30 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.795+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a71c0 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.796+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4102240 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.797+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9598017070 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.798+0000 7f958bfff640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 0x7f958c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.798+0000 7f95a2ffd640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 0x7f958c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.798+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f95980cebe0 con 0x7f95a4102790 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.798+0000 7f95a2ffd640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 0x7f958c0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9594031440 tx=0x7f95940095f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:27.802+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f959809b4d0 con 0x7f95a4102790 2024-09-05T06:50:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.005+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95a402d7c0 con 0x7f95a4102790 2024-09-05T06:50:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.005+0000 7f958bfff640 1 -- 172.21.15.86:0/2066890574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f959809add0 con 0x7f95a4102790 2024-09-05T06:50:28.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:50:28.006 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:50:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 msgr2=0x7f958c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 0x7f958c0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9594031440 tx=0x7f95940095f0 comp rx=0 tx=0).stop 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 msgr2=0x7f95a4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f959800b4a0 tx=0x7f959800b960 comp rx=0 tx=0).stop 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 shutdown_connections 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f958c0a2cf0 0x7f958c0a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a4106f10 0x7f95a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a4102790 0x7f95a4074a30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 --2- 172.21.15.86:0/2066890574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a4101de0 0x7f95a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 >> 172.21.15.86:0/2066890574 conn(0x7f95a40fd8c0 msgr2=0x7f95a40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 shutdown_connections 2024-09-05T06:50:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:28.008+0000 7f95a9621640 1 -- 172.21.15.86:0/2066890574 wait complete. 2024-09-05T06:50:28.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.81 MB/sec execute 348 sec latency 29391.164 ms 2024-09-05T06:50:28.805 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:28 smithi086 ceph-mon[30943]: pgmap v285: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-05T06:50:28.805 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:28 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2066890574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:28.807 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 24a7b065 --release --await' 2024-09-05T06:50:28.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 24a7b065 --release --await 2024-09-05T06:50:28.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:28 smithi096 ceph-mon[34468]: pgmap v285: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-05T06:50:28.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:28 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2066890574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:28.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:28 smithi115 ceph-mon[34791]: pgmap v285: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-05T06:50:28.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:28 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2066890574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:29.057 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:29.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 172462 1.81 MB/sec execute 349 sec latency 30391.453 ms 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3409576807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102d40 msgr2=0x7fb6e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3409576807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102d40 0x7fb6e8103120 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d802a480 comp rx=0 tx=0).stop 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3409576807 shutdown_connections 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3409576807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e810c170 0x7fb6e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3409576807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8103660 0x7fb6e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3409576807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8102d40 0x7fb6e8103120 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.594+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3409576807 >> 172.21.15.86:0/3409576807 conn(0x7fb6e80fc7a0 msgr2=0x7fb6e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.595+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3409576807 shutdown_connections 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.595+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3409576807 wait complete. 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.595+0000 7fb6ed1c4640 1 Processor -- start 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 -- start start 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8102d40 0x7fb6e81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e8103660 0x7fb6e8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8111e20 con 0x7fb6e810c170 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8111ca0 con 0x7fb6e8102d40 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6ed1c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8111fa0 con 0x7fb6e8103660 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38040/0 (socket says 172.21.15.86:38040) 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 -- 172.21.15.86:0/3677055658 learned_addr learned my addr 172.21.15.86:0/3677055658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e659c640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e8103660 0x7fb6e8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8102d40 0x7fb6e81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 -- 172.21.15.86:0/3677055658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e8103660 msgr2=0x7fb6e8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e8103660 0x7fb6e8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 -- 172.21.15.86:0/3677055658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8102d40 msgr2=0x7fb6e81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8102d40 0x7fb6e81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 -- 172.21.15.86:0/3677055658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e81a6770 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6e759e640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000eaa0 tx=0x7fb6e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.596+0000 7fb6cffff640 1 -- 172.21.15.86:0/3677055658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000cdb0 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.597+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e81a6a60 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.597+0000 7fb6cffff640 1 -- 172.21.15.86:0/3677055658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e0005020 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.597+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e81a6fa0 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.597+0000 7fb6cffff640 1 -- 172.21.15.86:0/3677055658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00116d0 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6cffff640 1 -- 172.21.15.86:0/3677055658 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb6e00051d0 con 0x7fb6e810c170 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6ef59d740 1 Processor -- start 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6cffff640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 0x7fb6d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6ef59d740 1 -- start start 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6e659c640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 0x7fb6d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.599+0000 7fb6cffff640 1 -- 172.21.15.86:0/3677055658 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fb6e00cf960 con 0x7fb6e810c170 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6aab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1b6a10620 con 0x55c1b6aab980 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c1b69b7e50 con 0x55c1b6ab6240 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c1b6a0c5e0 con 0x55c1b6aab060 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38046/0 (socket says 172.21.15.86:38046) 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 -- 172.21.15.86:0/4266470084 learned_addr learned my addr 172.21.15.86:0/4266470084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6aab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 0x7fb6d00a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8102430 tx=0x7fb6dc00f3c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e759e640 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 -- 172.21.15.86:0/4266470084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 msgr2=0x55c1b6aab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6aab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 -- 172.21.15.86:0/4266470084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 msgr2=0x55c1b6ab8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 -- 172.21.15.86:0/4266470084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b69a8d40 con 0x55c1b6aab980 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6e659c640 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00ffe0 tx=0x7fb6dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94272560dbae0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.600+0000 7fb6cd7fa640 1 -- 172.21.15.86:0/4266470084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc018070 con 0x55c1b6aab980 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6cd7fa640 1 -- 172.21.15.86:0/4266470084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6dc012e30 con 0x55c1b6aab980 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6cd7fa640 1 -- 172.21.15.86:0/4266470084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc01c8c0 con 0x55c1b6aab980 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 -- 172.21.15.86:0/4266470084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 msgr2=0x55c1b6ab5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00ffe0 tx=0x7fb6dc01a000 comp rx=0 tx=0).stop 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 -- 172.21.15.86:0/4266470084 shutdown_connections 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab5d00 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/4266470084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6aab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 -- 172.21.15.86:0/4266470084 >> 172.21.15.86:0/4266470084 conn(0x55c1b6aa75f0 msgr2=0x55c1b6aa79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 -- 172.21.15.86:0/4266470084 shutdown_connections 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.601+0000 7fb6ef59d740 1 -- 172.21.15.86:0/4266470084 wait complete. 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 -- start start 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 Processor -- start 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6ab30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1b6ab8e10 con 0x55c1b6aab980 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c1b6998f10 con 0x55c1b6ab6240 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c1b6ab8f90 con 0x55c1b6aab060 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6ef59d740 10 client.0 _ll_register_callbacks cb 0x55c1b69bcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38060/0 (socket says 172.21.15.86:38060) 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 -- 172.21.15.86:0/931726997 learned_addr learned my addr 172.21.15.86:0/931726997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e759e640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 -- 172.21.15.86:0/931726997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 msgr2=0x55c1b6ab30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6ab30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 -- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 msgr2=0x55c1b6ab3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 -- 172.21.15.86:0/931726997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b671f580 con 0x55c1b6aab980 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e759e640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6e659c640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc026040 tx=0x7fb6dc0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380424a6cff4ca08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.602+0000 7fb6caff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc031060 con 0x55c1b6aab980 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fb6dc004ad0 con 0x55c1b6aab980 2024-09-05T06:50:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6dc012e30 con 0x55c1b6aab980 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6dc038050 con 0x55c1b6aab980 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 10 client.17655 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1b67e9960 con 0x55c1b6aab980 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6dc012e30 con 0x55c1b6aab980 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb6dc01c640 con 0x55c1b6aab980 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 10 client.17655 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 10 client.17655 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 20 client.17655 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6ac5ce0 0x55c1b6ac80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 4 client.17655 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1b6a1db90 con 0x55c1b6ac5ce0 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.603+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6ac5ce0 0x55c1b6ac80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.604+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6ac5ce0 0x55c1b6ac80d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc8ecc9cd4333f7 server_cookie=be5710d312c9e812 in_seq=0 out_seq=0 2024-09-05T06:50:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.604+0000 7fb6caff5640 10 client.17655 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6caff5640 1 -- 172.21.15.86:0/931726997 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1b6a1db90 con 0x55c1b6ac5ce0 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6caff5640 10 client.17655 handle_command_reply: tid=0 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 client.17655 shutdown 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 msgr2=0x55c1b6ab3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc026040 tx=0x7fb6dc0056e0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 shutdown_connections 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6ac5ce0 0x55c1b6ac80d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6ab6240 0x55c1b6ab3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6aab980 0x55c1b6ab3610 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6caff5640 0 client.17655 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/931726997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6ab30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 >> 172.21.15.86:0/931726997 conn(0x55c1b6aa75f0 msgr2=0x55c1b6aad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 shutdown_connections 2024-09-05T06:50:29.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 1 -- 172.21.15.86:0/931726997 wait complete. 2024-09-05T06:50:29.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.605+0000 7fb6ef59d740 20 client.17655 trim_cache size 0 max 16384 2024-09-05T06:50:29.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 Processor -- start 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 -- start start 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab060 0x55c1b6aadac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab980 0x55c1b6aa6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b12e0 con 0x55c1b6ab6240 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b2e20 con 0x55c1b6aab060 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b19b0 con 0x55c1b6aab980 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38066/0 (socket says 172.21.15.86:38066) 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 -- 172.21.15.86:0/450848174 learned_addr learned my addr 172.21.15.86:0/450848174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab060 0x55c1b6aadac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 -- 172.21.15.86:0/450848174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab980 msgr2=0x55c1b6aa6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab980 0x55c1b6aa6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 -- 172.21.15.86:0/450848174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab060 msgr2=0x55c1b6aadac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab060 0x55c1b6aadac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.622+0000 7fb6e759e640 1 -- 172.21.15.86:0/450848174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1b6aa7090 con 0x55c1b6ab6240 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6e759e640 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000efd0 tx=0x7fb6e009fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2431be83eb07cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ccff9640 1 -- 172.21.15.86:0/450848174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e001f040 con 0x55c1b6ab6240 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ccff9640 1 -- 172.21.15.86:0/450848174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e00b4070 con 0x55c1b6ab6240 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ccff9640 1 -- 172.21.15.86:0/450848174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0097aa0 con 0x55c1b6ab6240 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 -- 172.21.15.86:0/450848174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 msgr2=0x55c1b6acd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000efd0 tx=0x7fb6e009fdf0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 -- 172.21.15.86:0/450848174 shutdown_connections 2024-09-05T06:50:29.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6acd760 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab980 0x55c1b6aa6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/450848174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab060 0x55c1b6aadac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 -- 172.21.15.86:0/450848174 >> 172.21.15.86:0/450848174 conn(0x55c1b6aa75f0 msgr2=0x55c1b6aa5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 -- 172.21.15.86:0/450848174 shutdown_connections 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.623+0000 7fb6ef59d740 1 -- 172.21.15.86:0/450848174 wait complete. 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 -- start start 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 Processor -- start 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6abab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab980 0x55c1b6abb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b2750 con 0x55c1b6ab6240 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b2080 con 0x55c1b6aab980 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81af2e0 con 0x55c1b6aab060 2024-09-05T06:50:29.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6ef59d740 10 client.0 _ll_register_callbacks cb 0x7fb6dc005370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38078/0 (socket says 172.21.15.86:38078) 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 -- 172.21.15.86:0/2637811925 learned_addr learned my addr 172.21.15.86:0/2637811925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e659c640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab980 0x55c1b6abb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6abab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 -- 172.21.15.86:0/2637811925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 msgr2=0x55c1b6abab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6abab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 -- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab980 msgr2=0x55c1b6abb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab980 0x55c1b6abb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e759e640 1 -- 172.21.15.86:0/2637811925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1b6b28f20 con 0x55c1b6ab6240 2024-09-05T06:50:29.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.624+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6abab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6e759e640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ccd20 tx=0x7fb6e00ccc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97965615bf87e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e007f060 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fb6e00ba3b0 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e00b4070 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6e001f070 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 10 client.17661 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1b6b31270 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e00b4070 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fb6e000f040 con 0x55c1b6ab6240 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 10 client.17661 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 10 client.17661 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 20 client.17661 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:29.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6b318e0 0x55c1b6b33cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:29.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 4 client.17661 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "24a7b065", "release": true, "await": true}] 2024-09-05T06:50:29.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.625+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "24a7b065", "release": true, "await": true}) -- 0x55c1b6aaecf0 con 0x55c1b6b318e0 2024-09-05T06:50:29.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.626+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6b318e0 0x55c1b6b33cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:29.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.626+0000 7fb6e6d9d640 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6b318e0 0x55c1b6b33cd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e91e5d8fecce9ec server_cookie=40fcfecdbf861929 in_seq=0 out_seq=0 2024-09-05T06:50:29.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.626+0000 7fb6caff5640 10 client.17661 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6caff5640 1 -- 172.21.15.86:0/2637811925 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1b6aaecf0 con 0x55c1b6b318e0 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6caff5640 10 client.17661 handle_command_reply: tid=0 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 client.17661 shutdown 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 msgr2=0x55c1b6b286f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ccd20 tx=0x7fb6e00ccc40 comp rx=0 tx=0).stop 2024-09-05T06:50:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 shutdown_connections 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c1b6b318e0 0x55c1b6b33cd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1b6ab6240 0x55c1b6b286f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6caff5640 0 client.17661 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1b6aab980 0x55c1b6abb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 --2- 172.21.15.86:0/2637811925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1b6aab060 0x55c1b6abab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 >> 172.21.15.86:0/2637811925 conn(0x55c1b6aa75f0 msgr2=0x55c1b6aa6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 shutdown_connections 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 1 -- 172.21.15.86:0/2637811925 wait complete. 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.632+0000 7fb6ef59d740 20 client.17661 trim_cache size 0 max 16384 2024-09-05T06:50:29.636 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 32, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "24a7b065": { 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 32, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:50:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:29.638 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.633+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 msgr2=0x7fb6d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.633+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 0x7fb6d00a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8102430 tx=0x7fb6dc00f3c0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.633+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 msgr2=0x7fb6e8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.633+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000eaa0 tx=0x7fb6e000ef60 comp rx=0 tx=0).stop 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 shutdown_connections 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6d00a2cf0 0x7fb6d00a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e810c170 0x7fb6e8101420 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6e8103660 0x7fb6e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 --2- 172.21.15.86:0/3677055658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6e8102d40 0x7fb6e81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 >> 172.21.15.86:0/3677055658 conn(0x7fb6e80fc7a0 msgr2=0x7fb6e80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 shutdown_connections 2024-09-05T06:50:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:29.634+0000 7fb6ed1c4640 1 -- 172.21.15.86:0/3677055658 wait complete. 2024-09-05T06:50:30.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 173032 1.82 MB/sec execute 350 sec latency 30497.116 ms 2024-09-05T06:50:30.564 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '24a7b065', total seconds elapsed: 34.2 2024-09-05T06:50:30.564 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T06:50:30.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:50:30.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:30 smithi096 ceph-mon[34468]: pgmap v286: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-05T06:50:30.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:30 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:50:30.836 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:30.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:30 smithi115 ceph-mon[34791]: pgmap v286: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-05T06:50:30.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:30 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:50:30.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:30 smithi086 ceph-mon[30943]: pgmap v286: 129 pgs: 129 active+clean; 178 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-05T06:50:30.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:30 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:50:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 -- 172.21.15.86:0/56885811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c108290 msgr2=0x7f890c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 --2- 172.21.15.86:0/56885811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c108290 0x7f890c10ec30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8904009cd0 tx=0x7f890402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 -- 172.21.15.86:0/56885811 shutdown_connections 2024-09-05T06:50:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 --2- 172.21.15.86:0/56885811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c108290 0x7f890c10ec30 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 --2- 172.21.15.86:0/56885811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c103740 0x7f890c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 --2- 172.21.15.86:0/56885811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c102d90 0x7f890c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 -- 172.21.15.86:0/56885811 >> 172.21.15.86:0/56885811 conn(0x7f890c0fc7d0 msgr2=0x7f890c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 -- 172.21.15.86:0/56885811 shutdown_connections 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 -- 172.21.15.86:0/56885811 wait complete. 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.433+0000 7f891118d640 1 Processor -- start 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 -- start start 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 0x7f890c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c108290 0x7f890c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f890c111f40 con 0x7f890c103740 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f890c111dc0 con 0x7f890c102d90 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f891118d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1120c0 con 0x7f890c108290 2024-09-05T06:50:31.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f8903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f8903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38102/0 (socket says 172.21.15.86:38102) 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f8903fff640 1 -- 172.21.15.86:0/381859885 learned_addr learned my addr 172.21.15.86:0/381859885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 0x7f890c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890b59e640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c108290 0x7f890c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 -- 172.21.15.86:0/381859885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c108290 msgr2=0x7f890c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c108290 0x7f890c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 -- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 msgr2=0x7f890c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f890ad9d640 1 -- 172.21.15.86:0/381859885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1a6950 con 0x7f890c102d90 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.434+0000 7f8903fff640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f890ad9d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 0x7f890c1009f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00e9f0 tx=0x7f88fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00ccc0 con 0x7f890c102d90 2024-09-05T06:50:31.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a6c40 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88fc005020 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc004a00 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.435+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a7180 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.436+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c103170 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.437+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f88fc004ba0 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.438+0000 7f8908d99640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 0x7f88f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.438+0000 7f8903fff640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 0x7f88f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.438+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f88fc0cf8a0 con 0x7f890c102d90 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.439+0000 7f8903fff640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 0x7f88f80a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f890c102480 tx=0x7f88f4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.441+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88fc09c0b0 con 0x7f890c102d90 2024-09-05T06:50:31.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 173606 1.82 MB/sec execute 351 sec latency 44.937 ms 2024-09-05T06:50:31.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:50:31.686 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:50:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.685+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890c10a2a0 con 0x7f890c102d90 2024-09-05T06:50:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.686+0000 7f8908d99640 1 -- 172.21.15.86:0/381859885 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f88fc09b9b0 con 0x7f890c102d90 2024-09-05T06:50:31.687 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:50:31.687 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:50:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.689+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 msgr2=0x7f88f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.689+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 0x7f88f80a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f890c102480 tx=0x7f88f4005f90 comp rx=0 tx=0).stop 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.689+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 msgr2=0x7f890c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.689+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 0x7f890c1009f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00e9f0 tx=0x7f88fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 shutdown_connections 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88f80a2cf0 0x7f88f80a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f890c108290 0x7f890c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890c103740 0x7f890c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 --2- 172.21.15.86:0/381859885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f890c102d90 0x7f890c1009f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 >> 172.21.15.86:0/381859885 conn(0x7f890c0fc7d0 msgr2=0x7f890c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 shutdown_connections 2024-09-05T06:50:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:31.690+0000 7f891118d640 1 -- 172.21.15.86:0/381859885 wait complete. 2024-09-05T06:50:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:50:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:50:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:50:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:50:32.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174105 1.83 MB/sec execute 352 sec latency 13.206 ms 2024-09-05T06:50:32.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T06:50:32.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T06:50:32.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:32 smithi096 ceph-mon[34468]: pgmap v287: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 383 KiB/s wr, 30 op/s 2024-09-05T06:50:32.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:32 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/381859885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:32.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:32 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:50:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:32.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:32 smithi115 ceph-mon[34791]: pgmap v287: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 383 KiB/s wr, 30 op/s 2024-09-05T06:50:32.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:32 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/381859885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:32.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:32 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:50:32.982 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:32 smithi086 ceph-mon[30943]: pgmap v287: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 383 KiB/s wr, 30 op/s 2024-09-05T06:50:32.982 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:32 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/381859885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:50:32.982 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:32 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 -- 172.21.15.86:0/3516762437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4102d30 msgr2=0x7fabd4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 --2- 172.21.15.86:0/3516762437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4102d30 0x7fabd4103110 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009930 tx=0x7fabc402a420 comp rx=0 tx=0).stop 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 -- 172.21.15.86:0/3516762437 shutdown_connections 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 --2- 172.21.15.86:0/3516762437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41082a0 0x7fabd410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 --2- 172.21.15.86:0/3516762437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd41036e0 0x7fabd4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 --2- 172.21.15.86:0/3516762437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd4102d30 0x7fabd4103110 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 -- 172.21.15.86:0/3516762437 >> 172.21.15.86:0/3516762437 conn(0x7fabd40fc7d0 msgr2=0x7fabd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 -- 172.21.15.86:0/3516762437 shutdown_connections 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.425+0000 7fabd9af8640 1 -- 172.21.15.86:0/3516762437 wait complete. 2024-09-05T06:50:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 Processor -- start 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 -- start start 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 0x7fabd406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 0x7fabd406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4111fb0 con 0x7fabd41082a0 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4111e30 con 0x7fabd41036e0 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.426+0000 7fabd9af8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4112130 con 0x7fabd4102d30 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48358/0 (socket says 172.21.15.86:48358) 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 0x7fabd406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 -- 172.21.15.86:0/2155081417 learned_addr learned my addr 172.21.15.86:0/2155081417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 0x7fabd406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 -- 172.21.15.86:0/2155081417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 msgr2=0x7fabd406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 0x7fabd406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 -- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 msgr2=0x7fabd406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 0x7fabd406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 -- 172.21.15.86:0/2155081417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd406d670 con 0x7fabd4102d30 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 0x7fabd406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd3fff640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 0x7fabd406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.427+0000 7fabd37fe640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fabc4031440 tx=0x7fabc4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.428+0000 7fabd0ff9640 1 -- 172.21.15.86:0/2155081417 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc403f070 con 0x7fabd4102d30 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.428+0000 7fabd0ff9640 1 -- 172.21.15.86:0/2155081417 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc4031b50 con 0x7fabd4102d30 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.428+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd406d900 con 0x7fabd4102d30 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.428+0000 7fabd0ff9640 1 -- 172.21.15.86:0/2155081417 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4035030 con 0x7fabd4102d30 2024-09-05T06:50:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.428+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd4071870 con 0x7fabd4102d30 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.430+0000 7fabd0ff9640 1 -- 172.21.15.86:0/2155081417 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fabc4042070 con 0x7fabd4102d30 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.430+0000 7fabdbed1740 1 Processor -- start 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.430+0000 7fabdbed1740 1 -- start start 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f20400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f22d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562440e7b2a0 con 0x562440f16080 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562440e22b10 con 0x562440f15760 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabdbed1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562440e77260 con 0x562440f20940 2024-09-05T06:50:33.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f20400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd0ff9640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 0x7fabbc0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34060/0 (socket says 172.21.15.86:34060) 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 -- 172.21.15.86:0/3104557186 learned_addr learned my addr 172.21.15.86:0/3104557186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f22d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 0x7fabbc0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 -- 172.21.15.86:0/3104557186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 msgr2=0x562440f22d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f22d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 -- 172.21.15.86:0/3104557186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 msgr2=0x562440f20400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f20400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 -- 172.21.15.86:0/3104557186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562440becb90 con 0x562440f15760 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd0ff9640 1 -- 172.21.15.86:0/2155081417 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fabc40efef0 con 0x7fabd4102d30 2024-09-05T06:50:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f22d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.431+0000 7fabd3fff640 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009fd0 tx=0x7fabcc00f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f44619a47b810f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabb9ffb640 1 -- 172.21.15.86:0/3104557186 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc023020 con 0x562440f15760 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabb9ffb640 1 -- 172.21.15.86:0/3104557186 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabcc0138a0 con 0x562440f15760 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabd3fff640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 0x7fabbc0a5200 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fabcc024000 tx=0x7fabcc022000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabb9ffb640 1 -- 172.21.15.86:0/3104557186 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc010650 con 0x562440f15760 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 -- 172.21.15.86:0/3104557186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 msgr2=0x562440f15b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009fd0 tx=0x7fabcc00f7d0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 -- 172.21.15.86:0/3104557186 shutdown_connections 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f22d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f20400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 --2- 172.21.15.86:0/3104557186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f15b40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.432+0000 7fabdbed1740 1 -- 172.21.15.86:0/3104557186 >> 172.21.15.86:0/3104557186 conn(0x562440f11cf0 msgr2=0x562440f120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 -- 172.21.15.86:0/3104557186 shutdown_connections 2024-09-05T06:50:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 -- 172.21.15.86:0/3104557186 wait complete. 2024-09-05T06:50:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 -- start start 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 Processor -- start 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.433+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f1e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabdbed1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562440f23510 con 0x562440f16080 2024-09-05T06:50:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabdbed1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562440e036e0 con 0x562440f15760 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabdbed1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562440f23690 con 0x562440f20940 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabdbed1740 10 client.0 _ll_register_callbacks cb 0x562440e27a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34072/0 (socket says 172.21.15.86:34072) 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 -- 172.21.15.86:0/1382866379 learned_addr learned my addr 172.21.15.86:0/1382866379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd3fff640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 -- 172.21.15.86:0/1382866379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 msgr2=0x562440f1e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f1e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 -- 172.21.15.86:0/1382866379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 msgr2=0x562440f1dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f1dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 -- 172.21.15.86:0/1382866379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562440f2c9d0 con 0x562440f15760 2024-09-05T06:50:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f1dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabc40b8a50 tx=0x7fabc4009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c3e2b85fa653e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.434+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40c0380 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562440e889d0 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fabc40ed480 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabc40c4c40 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabdbed1740 10 client.26852 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562440f2f360 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562440e889d0 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.435+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fabc40c4840 con 0x562440f15760 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 10 client.26852 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 10 client.26852 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:50:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 20 client.26852 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f32dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 4 client.26852 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562440e82d20 con 0x562440f30950 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f32dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f32dd0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347eee72a0b63dbe server_cookie=3916e87439938d09 in_seq=0 out_seq=0 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.436+0000 7fabb2ffd640 10 client.26852 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1382866379 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562440e82d20 con 0x562440f30950 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabb2ffd640 10 client.26852 handle_command_reply: tid=0 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 client.26852 shutdown 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 msgr2=0x562440f1d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fabc40b8a50 tx=0x7fabc4009620 comp rx=0 tx=0).stop 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 shutdown_connections 2024-09-05T06:50:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f32dd0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f20940 0x562440f1e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabb2ffd640 0 client.26852 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.437+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f1dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1382866379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f1d7f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 >> 172.21.15.86:0/1382866379 conn(0x562440f11cf0 msgr2=0x562440f17a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 shutdown_connections 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabdbed1740 1 -- 172.21.15.86:0/1382866379 wait complete. 2024-09-05T06:50:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.438+0000 7fabdbed1740 20 client.26852 trim_cache size 0 max 16384 2024-09-05T06:50:33.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 Processor -- start 2024-09-05T06:50:33.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 -- start start 2024-09-05T06:50:33.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f16080 0x562440f10d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 0x562440f386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b7b40 con 0x562440f20940 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b9680 con 0x562440f15760 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.456+0000 7fabdbed1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b8210 con 0x562440f16080 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 0x562440f386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34086/0 (socket says 172.21.15.86:34086) 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 0x562440f386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57830/0 (socket says 172.21.15.86:57830) 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 -- 172.21.15.86:0/694371666 learned_addr learned my addr 172.21.15.86:0/694371666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f16080 0x562440f10d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 -- 172.21.15.86:0/694371666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f16080 msgr2=0x562440f10d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f16080 0x562440f10d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 -- 172.21.15.86:0/694371666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 msgr2=0x562440f386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 0x562440f386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 -- 172.21.15.86:0/694371666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562440f12da0 con 0x562440f15760 2024-09-05T06:50:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabd37fe640 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabc4000c00 tx=0x7fabc40057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207402b2beeac3af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabb97fa640 1 -- 172.21.15.86:0/694371666 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40bb9e0 con 0x562440f15760 2024-09-05T06:50:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabb97fa640 1 -- 172.21.15.86:0/694371666 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc40c0380 con 0x562440f15760 2024-09-05T06:50:33.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.457+0000 7fabb97fa640 1 -- 172.21.15.86:0/694371666 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40b86f0 con 0x562440f15760 2024-09-05T06:50:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- 172.21.15.86:0/694371666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 msgr2=0x562440f181c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabc4000c00 tx=0x7fabc40057b0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- 172.21.15.86:0/694371666 shutdown_connections 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f20940 0x562440f386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f16080 0x562440f10d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 --2- 172.21.15.86:0/694371666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f15760 0x562440f181c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- 172.21.15.86:0/694371666 >> 172.21.15.86:0/694371666 conn(0x562440f11cf0 msgr2=0x562440f10650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- 172.21.15.86:0/694371666 shutdown_connections 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- 172.21.15.86:0/694371666 wait complete. 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.458+0000 7fabdbed1740 1 -- start start 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 Processor -- start 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f15760 0x562440f249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f20940 0x562440f9a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b8fb0 con 0x562440f16080 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b88e0 con 0x562440f20940 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41b5b40 con 0x562440f15760 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabd2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabdbed1740 10 client.0 _ll_register_callbacks cb 0x7fabc4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabd2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57842/0 (socket says 172.21.15.86:57842) 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f20940 0x562440f9a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabd2ffd640 1 -- 172.21.15.86:0/1787903598 learned_addr learned my addr 172.21.15.86:0/1787903598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.459+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f15760 0x562440f249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 -- 172.21.15.86:0/1787903598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f15760 msgr2=0x562440f249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f15760 0x562440f249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 -- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f20940 msgr2=0x562440f9a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f20940 0x562440f9a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 -- 172.21.15.86:0/1787903598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562440f12980 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabd2ffd640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fabd406e340 tx=0x7fabc80207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134b214a85c1147a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc802c070 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fabc8005220 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc40c0380 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabc8037050 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabdbed1740 10 client.17685 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562440f0d260 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.460+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc40c0380 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fabc80285d0 con 0x562440f16080 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 10 client.17685 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 10 client.17685 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:50:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 20 client.17685 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f9e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 4 client.17685 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562440dfeec0 con 0x562440f30950 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f9e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabd37fe640 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f9e180 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3f14b505d80fc7 server_cookie=afbb97c44ae62dc0 in_seq=0 out_seq=0 2024-09-05T06:50:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.461+0000 7fabb2ffd640 10 client.17685 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabb2ffd640 1 -- 172.21.15.86:0/1787903598 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562440dfeec0 con 0x562440f30950 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabb2ffd640 10 client.17685 handle_command_reply: tid=0 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 client.17685 shutdown 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 msgr2=0x562440f24f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fabd406e340 tx=0x7fabc80207d0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 shutdown_connections 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562440f30950 0x562440f9e180 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562440f20940 0x562440f9a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562440f16080 0x562440f24f30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 --2- 172.21.15.86:0/1787903598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562440f15760 0x562440f249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 >> 172.21.15.86:0/1787903598 conn(0x562440f11cf0 msgr2=0x562440f1dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabb2ffd640 0 client.17685 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 shutdown_connections 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 1 -- 172.21.15.86:0/1787903598 wait complete. 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.508+0000 7fabdbed1740 20 client.17685 trim_cache size 0 max 16384 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:33.510 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 34, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "137e9144": { 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 34, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:50:33.511 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:50:33.512 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 msgr2=0x7fabbc0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 0x7fabbc0a5200 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fabcc024000 tx=0x7fabcc022000 comp rx=0 tx=0).stop 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 msgr2=0x7fabd4073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fabc4031440 tx=0x7fabc4003b20 comp rx=0 tx=0).stop 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 shutdown_connections 2024-09-05T06:50:33.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fabbc0a2d40 0x7fabbc0a5200 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabd41082a0 0x7fabd406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabd41036e0 0x7fabd406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 --2- 172.21.15.86:0/2155081417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd4102d30 0x7fabd4073a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 >> 172.21.15.86:0/2155081417 conn(0x7fabd40fc7d0 msgr2=0x7fabd4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 shutdown_connections 2024-09-05T06:50:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:33.509+0000 7fabd9af8640 1 -- 172.21.15.86:0/2155081417 wait complete. 2024-09-05T06:50:33.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.83 MB/sec execute 353 sec latency 64.758 ms 2024-09-05T06:50:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:33 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:50:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:33 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:50:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:33 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:50:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:33 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:50:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:33 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:50:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:33 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:50:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:33 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:50:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:33 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:50:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:33 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:50:34.245 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '137e9144', quiesce took 3.7 seconds. Will release after: 26.3 2024-09-05T06:50:34.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.82 MB/sec execute 354 sec latency 1064.839 ms 2024-09-05T06:50:34.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:34 smithi086 ceph-mon[30943]: pgmap v288: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:35.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:34 smithi096 ceph-mon[34468]: pgmap v288: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:35.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:34 smithi115 ceph-mon[34791]: pgmap v288: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:35.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.82 MB/sec execute 355 sec latency 2064.921 ms 2024-09-05T06:50:36.525 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.81 MB/sec execute 356 sec latency 3065.040 ms 2024-09-05T06:50:36.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:36 smithi086 ceph-mon[30943]: pgmap v289: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:37.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:36 smithi096 ceph-mon[34468]: pgmap v289: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:37.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:36 smithi115 ceph-mon[34791]: pgmap v289: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 379 KiB/s wr, 25 op/s 2024-09-05T06:50:37.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.81 MB/sec execute 357 sec latency 4065.115 ms 2024-09-05T06:50:38.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.80 MB/sec execute 358 sec latency 5065.203 ms 2024-09-05T06:50:38.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:38 smithi086 ceph-mon[30943]: pgmap v290: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 749 KiB/s wr, 63 op/s 2024-09-05T06:50:39.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:38 smithi096 ceph-mon[34468]: pgmap v290: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 749 KiB/s wr, 63 op/s 2024-09-05T06:50:39.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:38 smithi115 ceph-mon[34791]: pgmap v290: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 749 KiB/s wr, 63 op/s 2024-09-05T06:50:39.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.80 MB/sec execute 359 sec latency 6065.278 ms 2024-09-05T06:50:40.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.79 MB/sec execute 360 sec latency 7065.331 ms 2024-09-05T06:50:40.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:40 smithi086 ceph-mon[30943]: pgmap v291: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 749 KiB/s wr, 58 op/s 2024-09-05T06:50:41.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:40 smithi096 ceph-mon[34468]: pgmap v291: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 749 KiB/s wr, 58 op/s 2024-09-05T06:50:41.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:40 smithi115 ceph-mon[34791]: pgmap v291: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 749 KiB/s wr, 58 op/s 2024-09-05T06:50:41.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.79 MB/sec execute 361 sec latency 8065.413 ms 2024-09-05T06:50:42.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.78 MB/sec execute 362 sec latency 9065.504 ms 2024-09-05T06:50:42.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:42 smithi086 ceph-mon[30943]: pgmap v292: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 766 KiB/s wr, 64 op/s 2024-09-05T06:50:43.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:42 smithi096 ceph-mon[34468]: pgmap v292: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 766 KiB/s wr, 64 op/s 2024-09-05T06:50:43.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:42 smithi115 ceph-mon[34791]: pgmap v292: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 766 KiB/s wr, 64 op/s 2024-09-05T06:50:43.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.78 MB/sec execute 363 sec latency 10065.605 ms 2024-09-05T06:50:44.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.77 MB/sec execute 364 sec latency 11065.707 ms 2024-09-05T06:50:44.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:44 smithi086 ceph-mon[30943]: pgmap v293: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:45.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:44 smithi096 ceph-mon[34468]: pgmap v293: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:45.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:44 smithi115 ceph-mon[34791]: pgmap v293: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:45.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.77 MB/sec execute 365 sec latency 12065.800 ms 2024-09-05T06:50:46.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.76 MB/sec execute 366 sec latency 13065.899 ms 2024-09-05T06:50:46.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:46 smithi086 ceph-mon[30943]: pgmap v294: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:47.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:46 smithi096 ceph-mon[34468]: pgmap v294: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:47.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:46 smithi115 ceph-mon[34791]: pgmap v294: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 386 KiB/s wr, 42 op/s 2024-09-05T06:50:47.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.76 MB/sec execute 367 sec latency 14065.998 ms 2024-09-05T06:50:48.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.75 MB/sec execute 368 sec latency 15066.116 ms 2024-09-05T06:50:49.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:48 smithi096 ceph-mon[34468]: pgmap v295: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 396 KiB/s wr, 47 op/s 2024-09-05T06:50:49.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:48 smithi115 ceph-mon[34791]: pgmap v295: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 396 KiB/s wr, 47 op/s 2024-09-05T06:50:49.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:48 smithi086 ceph-mon[30943]: pgmap v295: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 396 KiB/s wr, 47 op/s 2024-09-05T06:50:49.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.75 MB/sec execute 369 sec latency 16066.214 ms 2024-09-05T06:50:50.526 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.74 MB/sec execute 370 sec latency 17066.293 ms 2024-09-05T06:50:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:50 smithi086 ceph-mon[30943]: pgmap v296: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-05T06:50:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:50 smithi096 ceph-mon[34468]: pgmap v296: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-05T06:50:51.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:50 smithi115 ceph-mon[34791]: pgmap v296: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-09-05T06:50:51.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.74 MB/sec execute 371 sec latency 18066.393 ms 2024-09-05T06:50:52.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.73 MB/sec execute 372 sec latency 19066.491 ms 2024-09-05T06:50:53.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:52 smithi096 ceph-mon[34468]: pgmap v297: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-05T06:50:53.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:52 smithi115 ceph-mon[34791]: pgmap v297: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-05T06:50:53.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:52 smithi086 ceph-mon[30943]: pgmap v297: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-05T06:50:53.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.73 MB/sec execute 373 sec latency 20066.551 ms 2024-09-05T06:50:54.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.72 MB/sec execute 374 sec latency 21066.621 ms 2024-09-05T06:50:55.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:54 smithi096 ceph-mon[34468]: pgmap v298: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:55.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:54 smithi115 ceph-mon[34791]: pgmap v298: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:55.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:54 smithi086 ceph-mon[30943]: pgmap v298: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:55.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.72 MB/sec execute 375 sec latency 22066.710 ms 2024-09-05T06:50:56.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.71 MB/sec execute 376 sec latency 23066.799 ms 2024-09-05T06:50:57.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:56 smithi096 ceph-mon[34468]: pgmap v299: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:57.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:56 smithi115 ceph-mon[34791]: pgmap v299: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:57.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:56 smithi086 ceph-mon[30943]: pgmap v299: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-09-05T06:50:57.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:50:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:57.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.71 MB/sec execute 377 sec latency 24066.828 ms 2024-09-05T06:50:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2354702102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 msgr2=0x7ffaec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2354702102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1067e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009900 tx=0x7ffad802a410 comp rx=0 tx=0).stop 2024-09-05T06:50:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2354702102 shutdown_connections 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2354702102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec106f10 0x7ffaec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2354702102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1067e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2354702102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec101de0 0x7ffaec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2354702102 >> 172.21.15.86:0/2354702102 conn(0x7ffaec0fd8c0 msgr2=0x7ffaec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2354702102 shutdown_connections 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.008+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2354702102 wait complete. 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 Processor -- start 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 -- start start 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec101de0 0x7ffaec198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec106f10 0x7ffaec19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111e20 con 0x7ffaec102790 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111ca0 con 0x7ffaec106f10 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf31f1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111fa0 con 0x7ffaec101de0 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43142/0 (socket says 172.21.15.86:43142) 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 -- 172.21.15.86:0/2026840916 learned_addr learned my addr 172.21.15.86:0/2026840916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf178e640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec106f10 0x7ffaec19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec101de0 0x7ffaec198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 -- 172.21.15.86:0/2026840916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec101de0 msgr2=0x7ffaec198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec101de0 0x7ffaec198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 -- 172.21.15.86:0/2026840916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec106f10 msgr2=0x7ffaec19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec106f10 0x7ffaec19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.009+0000 7ffaebfff640 1 -- 172.21.15.86:0/2026840916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19d330 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ffad8005940 tx=0x7ffad8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffae9ffb640 1 -- 172.21.15.86:0/2026840916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad803f030 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19d5c0 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffae9ffb640 1 -- 172.21.15.86:0/2026840916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffad8003aa0 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffae9ffb640 1 -- 172.21.15.86:0/2026840916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8035070 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.010+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a4f00 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.012+0000 7ffae9ffb640 1 -- 172.21.15.86:0/2026840916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffad8042030 con 0x7ffaec102790 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 Processor -- start 2024-09-05T06:50:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 -- start start 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffae9ffb640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 0x7ffadc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381dd7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564381d322e0 con 0x564381dd7680 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564381cd9ce0 con 0x564381dcc4a0 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564381d2e2a0 con 0x564381dccdc0 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf178e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf178e640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 0x7ffadc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffae9ffb640 1 -- 172.21.15.86:0/2026840916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ffad80eff60 con 0x7ffaec102790 2024-09-05T06:50:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf178e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43144/0 (socket says 172.21.15.86:43144) 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf178e640 1 -- 172.21.15.86:0/1409595368 learned_addr learned my addr 172.21.15.86:0/1409595368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaebfff640 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381dd7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.013+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 -- 172.21.15.86:0/1409595368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 msgr2=0x564381dd7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381dd7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 -- 172.21.15.86:0/1409595368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 msgr2=0x564381dcc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dcc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 -- 172.21.15.86:0/1409595368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564381cbe440 con 0x564381dd7680 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 0x7ffadc0a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffae400bfd0 tx=0x7ffae400b570 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffaf178e640 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ffae4010d90 tx=0x7ffae401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0949e63b806094a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffad6ffd640 1 -- 172.21.15.86:0/1409595368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae400de40 con 0x564381dd7680 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffad6ffd640 1 -- 172.21.15.86:0/1409595368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffae4024070 con 0x564381dd7680 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.014+0000 7ffad6ffd640 1 -- 172.21.15.86:0/1409595368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4011be0 con 0x564381dd7680 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- 172.21.15.86:0/1409595368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 msgr2=0x564381dd9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ffae4010d90 tx=0x7ffae401a000 comp rx=0 tx=0).stop 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- 172.21.15.86:0/1409595368 shutdown_connections 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dd7680 0x564381dd9a70 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381dd7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/1409595368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- 172.21.15.86:0/1409595368 >> 172.21.15.86:0/1409595368 conn(0x564381dc8a30 msgr2=0x564381dc8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- 172.21.15.86:0/1409595368 shutdown_connections 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- 172.21.15.86:0/1409595368 wait complete. 2024-09-05T06:50:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 -- start start 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.015+0000 7ffaf55ca740 1 Processor -- start 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dd6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dd7680 0x564381dd47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564381dda250 con 0x564381dccdc0 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564381ced050 con 0x564381dcc4a0 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564381dda3d0 con 0x564381dd7680 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 10 client.0 _ll_register_callbacks cb 0x564381d2d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43158/0 (socket says 172.21.15.86:43158) 2024-09-05T06:50:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 -- 172.21.15.86:0/814749380 learned_addr learned my addr 172.21.15.86:0/814749380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dd6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf178e640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dd7680 0x564381dd47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 -- 172.21.15.86:0/814749380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dd7680 msgr2=0x564381dd47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dd7680 0x564381dd47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 -- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 msgr2=0x564381dd6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dd6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 -- 172.21.15.86:0/814749380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564381aa33c0 con 0x564381dccdc0 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dd6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaebfff640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ffad80bbce0 tx=0x7ffad80bba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd8958f0b25c45a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffacbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8049040 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ffad80dd410 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae4024070 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffad80c0390 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 10 client.17697 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.016+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564381de5c50 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffae4024070 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ffad80ed520 con 0x564381dccdc0 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 10 client.17697 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 20 client.17697 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381de7010 0x564381de9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 4 client.17697 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564381d3f850 con 0x564381de7010 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.017+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381de7010 0x564381de9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381de7010 0x564381de9480 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2639d20ca36b6d6 server_cookie=f51abc458815ca24 in_seq=0 out_seq=0 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffacbfff640 10 client.17697 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffacbfff640 1 -- 172.21.15.86:0/814749380 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564381d3f850 con 0x564381de7010 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffacbfff640 10 client.17697 handle_command_reply: tid=0 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 client.17697 shutdown 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 msgr2=0x564381dd4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ffad80bbce0 tx=0x7ffad80bba00 comp rx=0 tx=0).stop 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 shutdown_connections 2024-09-05T06:50:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381de7010 0x564381de9480 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dd7680 0x564381dd47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dccdc0 0x564381dd4270 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffacbfff640 0 client.17697 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/814749380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381dd6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.018+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 >> 172.21.15.86:0/814749380 conn(0x564381dc8a30 msgr2=0x564381dce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.019+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 shutdown_connections 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.019+0000 7ffaf55ca740 1 -- 172.21.15.86:0/814749380 wait complete. 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.019+0000 7ffaf55ca740 20 client.17697 trim_cache size 0 max 16384 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.035+0000 7ffaf55ca740 1 Processor -- start 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.035+0000 7ffaf55ca740 1 -- start start 2024-09-05T06:50:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dcc4a0 0x564381deed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 0x564381def290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1af060 con 0x564381dc8a30 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b0ba0 con 0x564381dccdc0 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1af730 con 0x564381dcc4a0 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43170/0 (socket says 172.21.15.86:43170) 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/4257720613 learned_addr learned my addr 172.21.15.86:0/4257720613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaebfff640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dcc4a0 0x564381deed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf178e640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 0x564381def290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/4257720613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dcc4a0 msgr2=0x564381deed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dcc4a0 0x564381deed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/4257720613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 msgr2=0x564381def290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 0x564381def290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/4257720613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564381dccbf0 con 0x564381dc8a30 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf178e640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 0x564381def290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x564381de9cc0 tx=0x7ffae000b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252da5ed1b1ae6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.036+0000 7ffad67fc640 1 -- 172.21.15.86:0/4257720613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae00142d0 con 0x564381dc8a30 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffad67fc640 1 -- 172.21.15.86:0/4257720613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffae00054d0 con 0x564381dc8a30 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffad67fc640 1 -- 172.21.15.86:0/4257720613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae001b330 con 0x564381dc8a30 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 -- 172.21.15.86:0/4257720613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 msgr2=0x564381de3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x564381de9cc0 tx=0x7ffae000b4d0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 -- 172.21.15.86:0/4257720613 shutdown_connections 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dccdc0 0x564381def290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dcc4a0 0x564381deed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/4257720613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381de3500 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 -- 172.21.15.86:0/4257720613 >> 172.21.15.86:0/4257720613 conn(0x564381dd4910 msgr2=0x564381dd2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 -- 172.21.15.86:0/4257720613 shutdown_connections 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.037+0000 7ffaf55ca740 1 -- 172.21.15.86:0/4257720613 wait complete. 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 -- start start 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 Processor -- start 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381ddfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381ddc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b04d0 con 0x564381dc8a30 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1afe00 con 0x564381dcc4a0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ad060 con 0x564381dccdc0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf55ca740 10 client.0 _ll_register_callbacks cb 0x7ffad80b8330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf0f8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf0f8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43176/0 (socket says 172.21.15.86:43176) 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/2867928792 learned_addr learned my addr 172.21.15.86:0/2867928792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaf178e640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381ddc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.038+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381ddfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/2867928792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 msgr2=0x564381ddc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381ddc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 msgr2=0x564381ddfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381ddfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 -- 172.21.15.86:0/2867928792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564381de0250 con 0x564381dc8a30 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381ddfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf0f8d640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ffae0004f10 tx=0x7ffae0004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eae47e88d2e6c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0004730 con 0x564381dc8a30 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae00054d0 con 0x564381dc8a30 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7ffae00048d0 con 0x564381dc8a30 2024-09-05T06:50:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffae0037050 con 0x564381dc8a30 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 10 client.17703 fetch_fsmap learned FSMap version 32 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564381e52240 con 0x564381dc8a30 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffae00054d0 con 0x564381dc8a30 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ffae00038f0 con 0x564381dc8a30 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 10 client.17703 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 20 client.17703 populate_metadata read hostname 'smithi086' 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.039+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381dd7680 0x564381e54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffaf55ca740 4 client.17703 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564381d39d60 con 0x564381dd7680 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381dd7680 0x564381e54a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffaebfff640 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381dd7680 0x564381e54a30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3b432db0857a27 server_cookie=2cfcc3ffd95d5972 in_seq=0 out_seq=0 2024-09-05T06:50:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffacbfff640 10 client.17703 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffacbfff640 1 -- 172.21.15.86:0/2867928792 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564381d39d60 con 0x564381dd7680 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffacbfff640 10 client.17703 handle_command_reply: tid=0 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.040+0000 7ffaf55ca740 1 client.17703 shutdown 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 msgr2=0x564381ddf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ffae0004f10 tx=0x7ffae0004f40 comp rx=0 tx=0).stop 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 shutdown_connections 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564381dd7680 0x564381e54a30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffacbfff640 0 client.17703 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564381dccdc0 0x564381ddc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564381dcc4a0 0x564381ddfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 --2- 172.21.15.86:0/2867928792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564381dc8a30 0x564381ddf670 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 >> 172.21.15.86:0/2867928792 conn(0x564381dd4910 msgr2=0x564381dc7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 shutdown_connections 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 1 -- 172.21.15.86:0/2867928792 wait complete. 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf55ca740 20 client.17703 trim_cache size 0 max 16384 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:50:58.061 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 msgr2=0x7ffadc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 0x7ffadc0a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffae400bfd0 tx=0x7ffae400b570 comp rx=0 tx=0).stop 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 msgr2=0x7ffaec1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.041+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ffad8005940 tx=0x7ffad8004270 comp rx=0 tx=0).stop 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 shutdown_connections 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffadc0a2cf0 0x7ffadc0a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaec106f10 0x7ffaec19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaec102790 0x7ffaec1988c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 --2- 172.21.15.86:0/2026840916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffaec101de0 0x7ffaec198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 >> 172.21.15.86:0/2026840916 conn(0x7ffaec0fd8c0 msgr2=0x7ffaec0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 shutdown_connections 2024-09-05T06:50:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:58.042+0000 7ffaf31f1640 1 -- 172.21.15.86:0/2026840916 wait complete. 2024-09-05T06:50:58.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.71 MB/sec execute 378 sec latency 25066.895 ms 2024-09-05T06:50:58.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:50:58.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:50:59.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:50:58 smithi096 ceph-mon[34468]: pgmap v300: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-09-05T06:50:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:50:59.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:50:58 smithi115 ceph-mon[34791]: pgmap v300: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-09-05T06:50:59.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:50:58 smithi086 ceph-mon[30943]: pgmap v300: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-09-05T06:50:59.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.70 MB/sec execute 379 sec latency 26066.933 ms 2024-09-05T06:50:59.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 -- 172.21.15.86:0/233910697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 msgr2=0x7f9218105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:59.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 --2- 172.21.15.86:0/233910697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f9218105f80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f920802a480 comp rx=0 tx=0).stop 2024-09-05T06:50:59.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 -- 172.21.15.86:0/233910697 shutdown_connections 2024-09-05T06:50:59.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 --2- 172.21.15.86:0/233910697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218102110 0x7f921810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 --2- 172.21.15.86:0/233910697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218101790 0x7f9218101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 --2- 172.21.15.86:0/233910697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f9218105f80 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 -- 172.21.15.86:0/233910697 >> 172.21.15.86:0/233910697 conn(0x7f9218077210 msgr2=0x7f9218077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 -- 172.21.15.86:0/233910697 shutdown_connections 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.591+0000 7f921e9b1640 1 -- 172.21.15.86:0/233910697 wait complete. 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 Processor -- start 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 -- start start 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 0x7f9218102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102110 0x7f9218102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9218110d50 con 0x7f9218105ba0 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9218110bd0 con 0x7f9218101790 2024-09-05T06:50:59.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921e9b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9218110ed0 con 0x7f9218102110 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921cf4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921cf4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43180/0 (socket says 172.21.15.86:43180) 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.592+0000 7f921cf4e640 1 -- 172.21.15.86:0/3851561712 learned_addr learned my addr 172.21.15.86:0/3851561712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f9217fff640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 0x7f9218102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f92177fe640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102110 0x7f9218102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 -- 172.21.15.86:0/3851561712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102110 msgr2=0x7f9218102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102110 0x7f9218102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 -- 172.21.15.86:0/3851561712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 msgr2=0x7f9218102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 0x7f9218102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 -- 172.21.15.86:0/3851561712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92181a6730 con 0x7f9218105ba0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f9217fff640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 0x7f9218102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921cf4e640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f921000cce0 tx=0x7f921000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000f450 con 0x7f9218105ba0 2024-09-05T06:50:59.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f921000faa0 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181a6a20 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018b70 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.593+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92181a6f10 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.594+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9218101bd0 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.595+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9210018d10 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.596+0000 7f92157fa640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 0x7f92000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.596+0000 7f9217fff640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 0x7f92000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.596+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f92100cfd70 con 0x7f9218105ba0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.596+0000 7f9217fff640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 0x7f92000a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9208005ab0 tx=0x7f9208005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:50:59.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.599+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9210014030 con 0x7f9218105ba0 2024-09-05T06:50:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.800+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9218103e80 con 0x7f9218105ba0 2024-09-05T06:50:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.801+0000 7f92157fa640 1 -- 172.21.15.86:0/3851561712 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f9218103e80 con 0x7f9218105ba0 2024-09-05T06:50:59.801 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:50:59.801 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:50:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:50:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 msgr2=0x7f92000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 0x7f92000a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9208005ab0 tx=0x7f9208005a20 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 msgr2=0x7f92181033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f921000cce0 tx=0x7f921000a620 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 shutdown_connections 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92000a2cf0 0x7f92000a51b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9218105ba0 0x7f92181033d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9218102110 0x7f9218102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 --2- 172.21.15.86:0/3851561712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9218101790 0x7f9218102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 >> 172.21.15.86:0/3851561712 conn(0x7f9218077210 msgr2=0x7f92180ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 shutdown_connections 2024-09-05T06:50:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:50:59.803+0000 7f921e9b1640 1 -- 172.21.15.86:0/3851561712 wait complete. 2024-09-05T06:51:00.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:51:00.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.70 MB/sec execute 380 sec latency 27067.216 ms 2024-09-05T06:51:00.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:51:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:01.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:00 smithi086 ceph-mon[30943]: pgmap v301: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T06:51:01.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:00 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3851561712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:00 smithi096 ceph-mon[34468]: pgmap v301: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T06:51:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:00 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3851561712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:01.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:00 smithi115 ceph-mon[34791]: pgmap v301: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T06:51:01.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:00 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3851561712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.259+0000 7f29083ac640 1 -- 172.21.15.86:0/170077974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 msgr2=0x7f2900106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.259+0000 7f29083ac640 1 --2- 172.21.15.86:0/170077974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900106e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009930 tx=0x7f28fc02a420 comp rx=0 tx=0).stop 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- 172.21.15.86:0/170077974 shutdown_connections 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- 172.21.15.86:0/170077974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 0x7f290010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- 172.21.15.86:0/170077974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 0x7f2900100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- 172.21.15.86:0/170077974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900106e70 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- 172.21.15.86:0/170077974 >> 172.21.15.86:0/170077974 conn(0x7f29000fc7d0 msgr2=0x7f29000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- 172.21.15.86:0/170077974 shutdown_connections 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- 172.21.15.86:0/170077974 wait complete. 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 Processor -- start 2024-09-05T06:51:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- start start 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 0x7f2900196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 0x7f29001974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2900110d40 con 0x7f2900106a90 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2900110bc0 con 0x7f2900100a90 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.260+0000 7f29083ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2900110ec0 con 0x7f2900101430 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43194/0 (socket says 172.21.15.86:43194) 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 -- 172.21.15.86:0/2165669944 learned_addr learned my addr 172.21.15.86:0/2165669944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2905947640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 0x7f29001974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906148640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 0x7f2900196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 -- 172.21.15.86:0/2165669944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 msgr2=0x7f29001974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 0x7f29001974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 -- 172.21.15.86:0/2165669944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 msgr2=0x7f2900196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 0x7f2900196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 -- 172.21.15.86:0/2165669944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29001a6820 con 0x7f2900106a90 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f2906949640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f28f800eaa0 tx=0x7f28f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f28f77fe640 1 -- 172.21.15.86:0/2165669944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800cdb0 con 0x7f2900106a90 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f28f77fe640 1 -- 172.21.15.86:0/2165669944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f8005020 con 0x7f2900106a90 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29001a6b10 con 0x7f2900106a90 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f28f77fe640 1 -- 172.21.15.86:0/2165669944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80116d0 con 0x7f2900106a90 2024-09-05T06:51:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.261+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29001a7000 con 0x7f2900106a90 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f28f77fe640 1 -- 172.21.15.86:0/2165669944 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f28f80051d0 con 0x7f2900106a90 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 Processor -- start 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 -- start start 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf25255d0 0x556bf2525a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2530370 0x556bf2532760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bf248b2f0 con 0x556bf2526b40 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bf2432cf0 con 0x556bf2530370 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f290a785740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bf24872b0 con 0x556bf25255d0 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.264+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43200/0 (socket says 172.21.15.86:43200) 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f28f77fe640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 0x7f28ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 -- 172.21.15.86:0/3686727058 learned_addr learned my addr 172.21.15.86:0/3686727058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2906148640 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf25255d0 0x556bf2525a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2906949640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 0x7f28ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f28f77fe640 1 -- 172.21.15.86:0/2165669944 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f28f80cf970 con 0x7f2900106a90 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 -- 172.21.15.86:0/3686727058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf25255d0 msgr2=0x556bf2525a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf25255d0 0x556bf2525a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 -- 172.21.15.86:0/3686727058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2530370 msgr2=0x556bf2532760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2530370 0x556bf2532760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 -- 172.21.15.86:0/3686727058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf24173a0 con 0x556bf2526b40 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2905947640 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f28f00058a0 tx=0x7f28f00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92615baba20fa3c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f2906949640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 0x7f28ec0a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28f809ba90 tx=0x7f28f8090f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f28f4dea640 1 -- 172.21.15.86:0/3686727058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f001b470 con 0x556bf2526b40 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f28f4dea640 1 -- 172.21.15.86:0/3686727058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f0016070 con 0x556bf2526b40 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.265+0000 7f28f4dea640 1 -- 172.21.15.86:0/3686727058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0010760 con 0x556bf2526b40 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 -- 172.21.15.86:0/3686727058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 msgr2=0x556bf2525090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f28f00058a0 tx=0x7f28f00127c0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 -- 172.21.15.86:0/3686727058 shutdown_connections 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2530370 0x556bf2532760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf25255d0 0x556bf2525a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 --2- 172.21.15.86:0/3686727058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf2525090 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 -- 172.21.15.86:0/3686727058 >> 172.21.15.86:0/3686727058 conn(0x556bf24ee310 msgr2=0x556bf2521c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 -- 172.21.15.86:0/3686727058 shutdown_connections 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.266+0000 7f290a785740 1 -- 172.21.15.86:0/3686727058 wait complete. 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 -- start start 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 Processor -- start 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf252fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf252d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bf2533070 con 0x556bf2526b40 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bf2446060 con 0x556bf25255d0 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bf25331f0 con 0x556bf2530370 2024-09-05T06:51:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f290a785740 10 client.0 _ll_register_callbacks cb 0x556bf21998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43206/0 (socket says 172.21.15.86:43206) 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 -- 172.21.15.86:0/1036838743 learned_addr learned my addr 172.21.15.86:0/1036838743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2905947640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf252fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2905947640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf252d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 -- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 msgr2=0x556bf252d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf252d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 -- 172.21.15.86:0/1036838743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 msgr2=0x556bf252fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf252fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 -- 172.21.15.86:0/1036838743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf21fc3c0 con 0x556bf2526b40 2024-09-05T06:51:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2905947640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf252d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f2906148640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f28fc02a940 tx=0x7f28fc031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72578b1d59800c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.267+0000 7f28e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc035040 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f28fc031720 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f0016070 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28fc004040 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 10 client.17721 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bf253eba0 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f0016070 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f28fc03c440 con 0x556bf2526b40 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 10 client.17721 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 10 client.17721 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 20 client.17721 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf2540060 0x556bf25424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 4 client.17721 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.268+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bf2498860 con 0x556bf2540060 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.269+0000 7f2905947640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf2540060 0x556bf25424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.269+0000 7f2905947640 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf2540060 0x556bf25424d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16d45cf644f55bd3 server_cookie=3cdf10673de07628 in_seq=0 out_seq=0 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.269+0000 7f28e9ffb640 10 client.17721 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f28e9ffb640 1 -- 172.21.15.86:0/1036838743 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bf2498860 con 0x556bf2540060 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f28e9ffb640 10 client.17721 handle_command_reply: tid=0 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 client.17721 shutdown 2024-09-05T06:51:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 msgr2=0x556bf252d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f28fc02a940 tx=0x7f28fc031ef0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 shutdown_connections 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf2540060 0x556bf25424d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf252d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2526b40 0x556bf252d090 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f28e9ffb640 0 client.17721 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 --2- 172.21.15.86:0/1036838743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf252fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 >> 172.21.15.86:0/1036838743 conn(0x556bf24ee310 msgr2=0x556bf2527960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 shutdown_connections 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 1 -- 172.21.15.86:0/1036838743 wait complete. 2024-09-05T06:51:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.270+0000 7f290a785740 20 client.17721 trim_cache size 0 max 16384 2024-09-05T06:51:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.283+0000 7f290a785740 1 Processor -- start 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 -- start start 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2526b40 0x556bf252f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf2547d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29001b13e0 con 0x556bf25255d0 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29001b2f20 con 0x556bf2526b40 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f290a785740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29001b1ab0 con 0x556bf2530370 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43216/0 (socket says 172.21.15.86:43216) 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 -- 172.21.15.86:0/4203908865 learned_addr learned my addr 172.21.15.86:0/4203908865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2906148640 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2526b40 0x556bf252f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 -- 172.21.15.86:0/4203908865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 msgr2=0x556bf2547d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf2547d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 -- 172.21.15.86:0/4203908865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2526b40 msgr2=0x556bf252f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2526b40 0x556bf252f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.284+0000 7f2905947640 1 -- 172.21.15.86:0/4203908865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf2528120 con 0x556bf25255d0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.285+0000 7f2905947640 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f28f0012cc0 tx=0x7f28f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dca442c5d837fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.285+0000 7f28ebfff640 1 -- 172.21.15.86:0/4203908865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0005240 con 0x556bf25255d0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.285+0000 7f28ebfff640 1 -- 172.21.15.86:0/4203908865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f0016040 con 0x556bf25255d0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.285+0000 7f28ebfff640 1 -- 172.21.15.86:0/4203908865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00273d0 con 0x556bf25255d0 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- 172.21.15.86:0/4203908865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 msgr2=0x556bf253c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f28f0012cc0 tx=0x7f28f000b840 comp rx=0 tx=0).stop 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- 172.21.15.86:0/4203908865 shutdown_connections 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2530370 0x556bf2547d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf2526b40 0x556bf252f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- 172.21.15.86:0/4203908865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf25255d0 0x556bf253c310 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- 172.21.15.86:0/4203908865 >> 172.21.15.86:0/4203908865 conn(0x556bf24ee310 msgr2=0x556bf252e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- 172.21.15.86:0/4203908865 shutdown_connections 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- 172.21.15.86:0/4203908865 wait complete. 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- start start 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 Processor -- start 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf2534a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2526b40 0x556bf2534f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29001b2850 con 0x556bf2530370 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.286+0000 7f290a785740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29001b2180 con 0x556bf25255d0 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f290a785740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29001af3e0 con 0x556bf2526b40 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f290a785740 10 client.0 _ll_register_callbacks cb 0x7f28fc031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43232/0 (socket says 172.21.15.86:43232) 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 -- 172.21.15.86:0/177815116 learned_addr learned my addr 172.21.15.86:0/177815116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2906148640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2526b40 0x556bf2534f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf2534a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 -- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2526b40 msgr2=0x556bf2534f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2526b40 0x556bf2534f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 -- 172.21.15.86:0/177815116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 msgr2=0x556bf2534a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf2534a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 -- 172.21.15.86:0/177815116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf25a42e0 con 0x556bf2530370 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f2905947640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c010 tx=0x7f28f00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f222fface6f73984 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.287+0000 7f28e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00277b0 con 0x556bf2530370 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f28f0027950 con 0x556bf2530370 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f0016040 con 0x556bf2530370 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28f0005620 con 0x556bf2530370 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 10 client.17727 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bf25ab660 con 0x556bf2530370 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f0016040 con 0x556bf2530370 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f28f002f070 con 0x556bf2530370 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 10 client.17727 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 10 client.17727 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 20 client.17727 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf25abb00 0x556bf25adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 4 client.17727 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.288+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bf2492d70 con 0x556bf25abb00 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f2906148640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf25abb00 0x556bf25adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f2906148640 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf25abb00 0x556bf25adf20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c80574369c122e6 server_cookie=90cd43fae52177a1 in_seq=0 out_seq=0 2024-09-05T06:51:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f28e9ffb640 10 client.17727 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f28e9ffb640 1 -- 172.21.15.86:0/177815116 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bf2492d70 con 0x556bf25abb00 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f28e9ffb640 10 client.17727 handle_command_reply: tid=0 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f290a785740 1 client.17727 shutdown 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 msgr2=0x556bf25a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.289+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c010 tx=0x7f28f00058d0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 shutdown_connections 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x556bf25abb00 0x556bf25adf20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bf2530370 0x556bf25a3cf0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bf2526b40 0x556bf2534f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 --2- 172.21.15.86:0/177815116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bf25255d0 0x556bf2534a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 >> 172.21.15.86:0/177815116 conn(0x556bf24ee310 msgr2=0x556bf252f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f28e9ffb640 0 client.17727 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 shutdown_connections 2024-09-05T06:51:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 1 -- 172.21.15.86:0/177815116 wait complete. 2024-09-05T06:51:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f290a785740 20 client.17727 trim_cache size 0 max 16384 2024-09-05T06:51:01.306 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 msgr2=0x7f28ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 0x7f28ec0a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28f809ba90 tx=0x7f28f8090f90 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 msgr2=0x7f2900197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.290+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f28f800eaa0 tx=0x7f28f800ef60 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 shutdown_connections 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f28ec0a2cf0 0x7f28ec0a51b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2900106a90 0x7f2900197a30 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101430 0x7f29001974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 --2- 172.21.15.86:0/2165669944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2900100a90 0x7f2900196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 >> 172.21.15.86:0/2165669944 conn(0x7f29000fc7d0 msgr2=0x7f29000fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 shutdown_connections 2024-09-05T06:51:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.291+0000 7f29083ac640 1 -- 172.21.15.86:0/2165669944 wait complete. 2024-09-05T06:51:01.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.69 MB/sec execute 381 sec latency 28067.320 ms 2024-09-05T06:51:02.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 -- 172.21.15.86:0/3663240663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4102ef0 msgr2=0x7f2eb410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/3663240663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4102ef0 0x7f2eb410da40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009ca0 tx=0x7f2eac02a950 comp rx=0 tx=0).stop 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 -- 172.21.15.86:0/3663240663 shutdown_connections 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/3663240663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4102ef0 0x7f2eb410da40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/3663240663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102570 0x7f2eb41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/3663240663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4100e20 0x7f2eb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 -- 172.21.15.86:0/3663240663 >> 172.21.15.86:0/3663240663 conn(0x7f2eb40fc7d0 msgr2=0x7f2eb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 -- 172.21.15.86:0/3663240663 shutdown_connections 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.964+0000 7f2eb8a75640 1 -- 172.21.15.86:0/3663240663 wait complete. 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 Processor -- start 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 -- start start 2024-09-05T06:51:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4102570 0x7f2eb41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102ef0 0x7f2eb419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4110e30 con 0x7f2eb4100e20 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4110cb0 con 0x7f2eb4102570 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb8a75640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4110fb0 con 0x7f2eb4102ef0 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43240/0 (socket says 172.21.15.86:43240) 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 -- 172.21.15.86:0/2528116900 learned_addr learned my addr 172.21.15.86:0/2528116900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb1d9b640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4102570 0x7f2eb41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb2d9d640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102ef0 0x7f2eb419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 -- 172.21.15.86:0/2528116900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102ef0 msgr2=0x7f2eb419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102ef0 0x7f2eb419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.965+0000 7f2eb259c640 1 -- 172.21.15.86:0/2528116900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4102570 msgr2=0x7f2eb41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2eb259c640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4102570 0x7f2eb41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2eb259c640 1 -- 172.21.15.86:0/2528116900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb419d360 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2eb259c640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d950 tx=0x7f2ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4017070 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb419d650 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ea4009c30 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400f620 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.966+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb41a4f30 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.967+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb4101200 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.968+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2ea40109a0 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.969+0000 7f2ea37fe640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 0x7f2e9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.969+0000 7f2eb1d9b640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 0x7f2e9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.969+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f2ea40cf4e0 con 0x7f2eb4100e20 2024-09-05T06:51:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.969+0000 7f2eb1d9b640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 0x7f2e9c0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4100030 tx=0x7f2ea800a250 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:01.973+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ea4009de0 con 0x7f2eb4100e20 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.175+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eb4199710 con 0x7f2eb4100e20 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.175+0000 7f2ea37fe640 1 -- 172.21.15.86:0/2528116900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11974 (secure 0 0 0) 0x7f2ea409bcc0 con 0x7f2eb4100e20 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"btime":"2024-09-05T06:45:30:442363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:45:30.389136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 msgr2=0x7f2e9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 0x7f2e9c0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4100030 tx=0x7f2ea800a250 comp rx=0 tx=0).stop 2024-09-05T06:51:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 msgr2=0x7f2eb41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d950 tx=0x7f2ea400de10 comp rx=0 tx=0).stop 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 shutdown_connections 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2e9c0a2cf0 0x7f2e9c0a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4102ef0 0x7f2eb419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb4102570 0x7f2eb41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 --2- 172.21.15.86:0/2528116900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2eb4100e20 0x7f2eb41983b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 >> 172.21.15.86:0/2528116900 conn(0x7f2eb40fc7d0 msgr2=0x7f2eb406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 shutdown_connections 2024-09-05T06:51:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:02.178+0000 7f2eb8a75640 1 -- 172.21.15.86:0/2528116900 wait complete. 2024-09-05T06:51:02.527 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.69 MB/sec execute 382 sec latency 29067.368 ms 2024-09-05T06:51:02.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:02 smithi086 ceph-mon[30943]: pgmap v302: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-05T06:51:02.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:02 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2528116900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:02 smithi096 ceph-mon[34468]: pgmap v302: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-05T06:51:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:02 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2528116900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:03.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:02 smithi115 ceph-mon[34791]: pgmap v302: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-05T06:51:03.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:02 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2528116900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:03.184 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 137e9144 --release --await' 2024-09-05T06:51:03.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 137e9144 --release --await 2024-09-05T06:51:03.222 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:03.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.68 MB/sec execute 383 sec latency 30067.407 ms 2024-09-05T06:51:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.704+0000 7fcd76580640 1 -- 172.21.15.86:0/3352192037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd70102e00 msgr2=0x7fcd701031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.704+0000 7fcd76580640 1 --2- 172.21.15.86:0/3352192037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd70102e00 0x7fcd701031e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6002a480 comp rx=0 tx=0).stop 2024-09-05T06:51:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.704+0000 7fcd76580640 1 -- 172.21.15.86:0/3352192037 shutdown_connections 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.704+0000 7fcd76580640 1 --2- 172.21.15.86:0/3352192037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd7010c1a0 0x7fcd7010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.704+0000 7fcd76580640 1 --2- 172.21.15.86:0/3352192037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd7010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 --2- 172.21.15.86:0/3352192037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd70102e00 0x7fcd701031e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 -- 172.21.15.86:0/3352192037 >> 172.21.15.86:0/3352192037 conn(0x7fcd700fc7a0 msgr2=0x7fcd700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 -- 172.21.15.86:0/3352192037 shutdown_connections 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 -- 172.21.15.86:0/3352192037 wait complete. 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 Processor -- start 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 -- start start 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.705+0000 7fcd76580640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 0x7fcd701983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd701988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111e60 con 0x7fcd7010c1a0 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111ce0 con 0x7fcd70103720 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111fe0 con 0x7fcd70102e00 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33058/0 (socket says 172.21.15.86:33058) 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 -- 172.21.15.86:0/4005811207 learned_addr learned my addr 172.21.15.86:0/4005811207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 0x7fcd701983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd701988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 -- 172.21.15.86:0/4005811207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 msgr2=0x7fcd701983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 0x7fcd701983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 -- 172.21.15.86:0/4005811207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 msgr2=0x7fcd701988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd701988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 -- 172.21.15.86:0/4005811207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019d350 con 0x7fcd7010c1a0 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd701988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 0x7fcd701983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd74b1d640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cce0 tx=0x7fcd6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6d7fa640 1 -- 172.21.15.86:0/4005811207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800f450 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6d7fa640 1 -- 172.21.15.86:0/4005811207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6800faa0 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019d640 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd6d7fa640 1 -- 172.21.15.86:0/4005811207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68018b70 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.706+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a4f20 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.708+0000 7fcd6d7fa640 1 -- 172.21.15.86:0/4005811207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd68018d10 con 0x7fcd7010c1a0 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 Processor -- start 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 -- start start 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc948165d0 0x55dc94816a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94821370 0x55dc94823760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc9477c2f0 con 0x55dc94817b40 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc94723cf0 con 0x55dc94821370 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd78959740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dc947782b0 con 0x55dc948165d0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6d7fa640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33068/0 (socket says 172.21.15.86:33068) 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1702110321 learned_addr learned my addr 172.21.15.86:0/1702110321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd74b1d640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc948165d0 0x55dc94816a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94821370 0x55dc94823760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.709+0000 7fcd6d7fa640 1 -- 172.21.15.86:0/4005811207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fcd680cfd80 con 0x7fcd7010c1a0 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1702110321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc948165d0 msgr2=0x55dc94816a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc948165d0 0x55dc94816a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1702110321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94821370 msgr2=0x55dc94823760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94821370 0x55dc94823760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1702110321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc947083a0 con 0x55dc94817b40 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400af90 tx=0x7fcd6400a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afa8fdca40d2137 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd74b1d640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 0x7fcd580a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd680cd140 tx=0x7fcd6800af30 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd567fc640 1 -- 172.21.15.86:0/1702110321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6401b070 con 0x55dc94817b40 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd567fc640 1 -- 172.21.15.86:0/1702110321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd640168a0 con 0x55dc94817b40 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.710+0000 7fcd567fc640 1 -- 172.21.15.86:0/1702110321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400f040 con 0x55dc94817b40 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- 172.21.15.86:0/1702110321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 msgr2=0x55dc94816090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400af90 tx=0x7fcd6400a410 comp rx=0 tx=0).stop 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- 172.21.15.86:0/1702110321 shutdown_connections 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94821370 0x55dc94823760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc948165d0 0x55dc94816a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- 172.21.15.86:0/1702110321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94817b40 0x55dc94816090 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- 172.21.15.86:0/1702110321 >> 172.21.15.86:0/1702110321 conn(0x55dc947df310 msgr2=0x55dc94812c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- 172.21.15.86:0/1702110321 shutdown_connections 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- 172.21.15.86:0/1702110321 wait complete. 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- start start 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 Processor -- start 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc948165d0 0x55dc94820d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94817b40 0x55dc9481dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc94824010 con 0x55dc94821370 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc94737060 con 0x55dc948165d0 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.711+0000 7fcd78959740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dc94824190 con 0x55dc94817b40 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd78959740 10 client.0 _ll_register_callbacks cb 0x55dc9448a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33074/0 (socket says 172.21.15.86:33074) 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/66935433 learned_addr learned my addr 172.21.15.86:0/66935433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94817b40 0x55dc9481dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc948165d0 0x55dc94820d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/66935433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94817b40 msgr2=0x55dc9481dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94817b40 0x55dc9481dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc948165d0 msgr2=0x55dc94820d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc948165d0 0x55dc94820d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/66935433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc944ed3c0 con 0x55dc94821370 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcd64004700 tx=0x7fcd6400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76c87540ea2255 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64028020 con 0x55dc94821370 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd640168a0 con 0x55dc94821370 2024-09-05T06:51:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fcd64016c60 con 0x55dc94821370 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd64033050 con 0x55dc94821370 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd78959740 10 client.17745 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.712+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc9482fa70 con 0x55dc94821370 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd640168a0 con 0x55dc94821370 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcd64021070 con 0x55dc94821370 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 10 client.17745 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 10 client.17745 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 20 client.17745 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc94830e80 0x55dc94833270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 4 client.17745 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc94789860 con 0x55dc94830e80 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.713+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc94830e80 0x55dc94833270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc94830e80 0x55dc94833270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a06d3e07c654766a server_cookie=11125b615411a1d9 in_seq=0 out_seq=0 2024-09-05T06:51:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd4f7fe640 10 client.17745 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/66935433 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc94789860 con 0x55dc94830e80 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd4f7fe640 10 client.17745 handle_command_reply: tid=0 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd78959740 1 client.17745 shutdown 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 msgr2=0x55dc9481e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcd64004700 tx=0x7fcd6400a650 comp rx=0 tx=0).stop 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 shutdown_connections 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.714+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc94830e80 0x55dc94833270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc94821370 0x55dc9481e5d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94817b40 0x55dc9481dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 --2- 172.21.15.86:0/66935433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc948165d0 0x55dc94820d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd4f7fe640 0 client.17745 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 >> 172.21.15.86:0/66935433 conn(0x55dc947df310 msgr2=0x55dc94818960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 shutdown_connections 2024-09-05T06:51:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 1 -- 172.21.15.86:0/66935433 wait complete. 2024-09-05T06:51:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.715+0000 7fcd78959740 20 client.17745 trim_cache size 0 max 16384 2024-09-05T06:51:03.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 Processor -- start 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 -- start start 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94815bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc9481ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701af110 con 0x55dc948165d0 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b0c50 con 0x55dc94817b40 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.730+0000 7fcd78959740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701af7e0 con 0x55dc94821370 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33090/0 (socket says 172.21.15.86:33090) 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/3522307944 learned_addr learned my addr 172.21.15.86:0/3522307944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94815bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/3522307944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 msgr2=0x55dc9481ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc9481ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/3522307944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 msgr2=0x55dc94815bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94815bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/3522307944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc94823f50 con 0x55dc948165d0 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94815bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000c00 tx=0x7fcd6400ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba47aa1084e3052 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd55ffb640 1 -- 172.21.15.86:0/3522307944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64028070 con 0x55dc948165d0 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd55ffb640 1 -- 172.21.15.86:0/3522307944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6401c040 con 0x55dc948165d0 2024-09-05T06:51:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.731+0000 7fcd55ffb640 1 -- 172.21.15.86:0/3522307944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6401b030 con 0x55dc948165d0 2024-09-05T06:51:03.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 -- 172.21.15.86:0/3522307944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 msgr2=0x55dc9482d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcd64000c00 tx=0x7fcd6400ad60 comp rx=0 tx=0).stop 2024-09-05T06:51:03.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 -- 172.21.15.86:0/3522307944 shutdown_connections 2024-09-05T06:51:03.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc9481ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94815bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 --2- 172.21.15.86:0/3522307944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc9482d2b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 -- 172.21.15.86:0/3522307944 >> 172.21.15.86:0/3522307944 conn(0x55dc947df310 msgr2=0x55dc9481f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 -- 172.21.15.86:0/3522307944 shutdown_connections 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.732+0000 7fcd78959740 1 -- 172.21.15.86:0/3522307944 wait complete. 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 -- start start 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 Processor -- start 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94894220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc948249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701b0580 con 0x55dc948165d0 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701afeb0 con 0x55dc94817b40 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ad110 con 0x55dc94821370 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd78959740 10 client.0 _ll_register_callbacks cb 0x7fcd64019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33098/0 (socket says 172.21.15.86:33098) 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1828372905 learned_addr learned my addr 172.21.15.86:0/1828372905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94894220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.733+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc948249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1828372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 msgr2=0x55dc948249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc948249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 msgr2=0x55dc94894220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94894220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 -- 172.21.15.86:0/1828372905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc94819120 con 0x55dc948165d0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd6f7fe640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcd64010010 tx=0x7fcd640046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55e5ab515b494bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64042060 con 0x55dc948165d0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7fcd64024ac0 con 0x55dc948165d0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6401c040 con 0x55dc948165d0 2024-09-05T06:51:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd64049050 con 0x55dc948165d0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd78959740 10 client.17751 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc9489b210 con 0x55dc948165d0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.734+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6401c040 con 0x55dc948165d0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcd64021030 con 0x55dc948165d0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 10 client.17751 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 10 client.17751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 20 client.17751 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc9489b870 0x55dc9489dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 4 client.17751 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc94783d70 con 0x55dc9489b870 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc9489b870 0x55dc9489dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd6ffff640 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc9489b870 0x55dc9489dc60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7593c0df18699e4f server_cookie=2de991e64b2dd286 in_seq=0 out_seq=0 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.735+0000 7fcd4f7fe640 10 client.17751 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd4f7fe640 1 -- 172.21.15.86:0/1828372905 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc94783d70 con 0x55dc9489b870 2024-09-05T06:51:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd4f7fe640 10 client.17751 handle_command_reply: tid=0 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 client.17751 shutdown 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 msgr2=0x55dc94893ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcd64010010 tx=0x7fcd640046a0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 shutdown_connections 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55dc9489b870 0x55dc9489dc60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dc94821370 0x55dc948249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc94817b40 0x55dc94894220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd4f7fe640 0 client.17751 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 --2- 172.21.15.86:0/1828372905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc948165d0 0x55dc94893ce0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 >> 172.21.15.86:0/1828372905 conn(0x55dc947df310 msgr2=0x55dc94818930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 shutdown_connections 2024-09-05T06:51:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 1 -- 172.21.15.86:0/1828372905 wait complete. 2024-09-05T06:51:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.736+0000 7fcd78959740 20 client.17751 trim_cache size 0 max 16384 2024-09-05T06:51:03.741 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 msgr2=0x7fcd580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 0x7fcd580a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd680cd140 tx=0x7fcd6800af30 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 msgr2=0x7fcd7019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cce0 tx=0x7fcd6800a620 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 shutdown_connections 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd580a2cf0 0x7fcd580a51b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7010c1a0 0x7fcd7019cc70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd70103720 0x7fcd701988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 --2- 172.21.15.86:0/4005811207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd70102e00 0x7fcd701983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 >> 172.21.15.86:0/4005811207 conn(0x7fcd700fc7a0 msgr2=0x7fcd700fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:03.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 shutdown_connections 2024-09-05T06:51:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:03.737+0000 7fcd76580640 1 -- 172.21.15.86:0/4005811207 wait complete. 2024-09-05T06:51:04.084 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:04.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174377 1.68 MB/sec execute 384 sec latency 31067.493 ms 2024-09-05T06:51:04.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:51:04.874 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:04 smithi086 ceph-mon[30943]: pgmap v303: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:05.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:04 smithi096 ceph-mon[34468]: pgmap v303: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 -- 172.21.15.86:0/3767039054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c100a70 msgr2=0x7f776c10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 --2- 172.21.15.86:0/3767039054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c100a70 0x7f776c10ea60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7764009ca0 tx=0x7f776402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 -- 172.21.15.86:0/3767039054 shutdown_connections 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 --2- 172.21.15.86:0/3767039054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c100a70 0x7f776c10ea60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 --2- 172.21.15.86:0/3767039054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c1000d0 0x7f776c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 --2- 172.21.15.86:0/3767039054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c106ba0 0x7f776c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 -- 172.21.15.86:0/3767039054 >> 172.21.15.86:0/3767039054 conn(0x7f776c077210 msgr2=0x7f776c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 -- 172.21.15.86:0/3767039054 shutdown_connections 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.096+0000 7f7771ac7640 1 -- 172.21.15.86:0/3767039054 wait complete. 2024-09-05T06:51:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 Processor -- start 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 -- start start 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 0x7f776c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c100a70 0x7f776c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f776c111f80 con 0x7f776c106ba0 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f776c111e00 con 0x7f776c100a70 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f7771ac7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112100 con 0x7f776c1000d0 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f776bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f776bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33118/0 (socket says 172.21.15.86:33118) 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.097+0000 7f776bfff640 1 -- 172.21.15.86:0/1733630551 learned_addr learned my addr 172.21.15.86:0/1733630551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f7763fff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c100a70 0x7f776c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776b7fe640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 0x7f776c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 -- 172.21.15.86:0/1733630551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 msgr2=0x7f776c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 0x7f776c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 -- 172.21.15.86:0/1733630551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c100a70 msgr2=0x7f776c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c100a70 0x7f776c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 -- 172.21.15.86:0/1733630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19d450 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776b7fe640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 0x7f776c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f776bfff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7764009dc0 tx=0x7f7764032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f77697fa640 1 -- 172.21.15.86:0/1733630551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764003cf0 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f77697fa640 1 -- 172.21.15.86:0/1733630551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77640044f0 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19d6e0 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f77697fa640 1 -- 172.21.15.86:0/1733630551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764030bf0 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.098+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1a5020 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.101+0000 7f7773ea0740 1 Processor -- start 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.101+0000 7f77697fa640 1 -- 172.21.15.86:0/1733630551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7764004040 con 0x7f776c106ba0 2024-09-05T06:51:05.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.101+0000 7f7773ea0740 1 -- start start 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.101+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.101+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c980 0x55bc6c176d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c179630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7773ea0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c0d1620 con 0x55bc6c16c060 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7773ea0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c078e50 con 0x55bc6c177240 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7773ea0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c0cd5e0 con 0x55bc6c16c980 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33132/0 (socket says 172.21.15.86:33132) 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 -- 172.21.15.86:0/4288469225 learned_addr learned my addr 172.21.15.86:0/4288469225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f776b7fe640 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c980 0x55bc6c176d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f77697fa640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 0x7f77580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 -- 172.21.15.86:0/4288469225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c980 msgr2=0x55bc6c176d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c980 0x55bc6c176d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 -- 172.21.15.86:0/4288469225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 msgr2=0x55bc6c179630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c179630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 -- 172.21.15.86:0/4288469225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6c069d40 con 0x55bc6c16c060 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f776bfff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 0x7f77580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7763fff640 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7754006e10 tx=0x7f7754009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9cec08f0263e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f77697fa640 1 -- 172.21.15.86:0/1733630551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f77640f0e80 con 0x7f776c106ba0 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.102+0000 7f7761ffb640 1 -- 172.21.15.86:0/4288469225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400fa60 con 0x55bc6c16c060 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7761ffb640 1 -- 172.21.15.86:0/4288469225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7754010040 con 0x55bc6c16c060 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7761ffb640 1 -- 172.21.15.86:0/4288469225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754013390 con 0x55bc6c16c060 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f776bfff640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 0x7f77580a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77640bd000 tx=0x7f776402a8c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 -- 172.21.15.86:0/4288469225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 msgr2=0x55bc6c16c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7754006e10 tx=0x7f7754009ea0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 -- 172.21.15.86:0/4288469225 shutdown_connections 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c179630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c980 0x55bc6c176d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 --2- 172.21.15.86:0/4288469225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c16c440 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 -- 172.21.15.86:0/4288469225 >> 172.21.15.86:0/4288469225 conn(0x55bc6c1685f0 msgr2=0x55bc6c1689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 -- 172.21.15.86:0/4288469225 shutdown_connections 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.103+0000 7f7773ea0740 1 -- 172.21.15.86:0/4288469225 wait complete. 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 -- start start 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 Processor -- start 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c1746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c174c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c179e10 con 0x55bc6c16c060 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c059f10 con 0x55bc6c16c980 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c179f90 con 0x55bc6c177240 2024-09-05T06:51:05.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7773ea0740 10 client.0 _ll_register_callbacks cb 0x55bc6c07ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33144/0 (socket says 172.21.15.86:33144) 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.104+0000 7f7763fff640 1 -- 172.21.15.86:0/2508155462 learned_addr learned my addr 172.21.15.86:0/2508155462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f776b7fe640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c1746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c174c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 -- 172.21.15.86:0/2508155462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 msgr2=0x55bc6c174c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c174c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 -- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 msgr2=0x55bc6c1746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c1746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 -- 172.21.15.86:0/2508155462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6bde0580 con 0x55bc6c16c060 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7763fff640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7754014c90 tx=0x7f7754006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cb390120eaa654 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f774affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754014430 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f77540145d0 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7754010040 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7754030050 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7773ea0740 10 client.17763 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.105+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc6beaa960 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7754010040 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f7754013070 con 0x55bc6c16c060 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 10 client.17763 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 10 client.17763 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 20 client.17763 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c186da0 0x55bc6c189190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 4 client.17763 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:05.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6c0deb90 con 0x55bc6c186da0 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f776b7fe640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c186da0 0x55bc6c189190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f774affd640 10 client.17763 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.106+0000 7f776b7fe640 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c186da0 0x55bc6c189190 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f9bdab7c403f4d server_cookie=54a7f1f7695ef57a in_seq=0 out_seq=0 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f774affd640 1 -- 172.21.15.86:0/2508155462 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc6c0deb90 con 0x55bc6c186da0 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f774affd640 10 client.17763 handle_command_reply: tid=0 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 client.17763 shutdown 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 msgr2=0x55bc6c174180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7754014c90 tx=0x7f7754006a30 comp rx=0 tx=0).stop 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 shutdown_connections 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c186da0 0x55bc6c189190 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c174c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c1746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 --2- 172.21.15.86:0/2508155462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c174180 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f774affd640 0 client.17763 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:05.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 >> 172.21.15.86:0/2508155462 conn(0x55bc6c1685f0 msgr2=0x55bc6c16e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 shutdown_connections 2024-09-05T06:51:05.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.107+0000 7f7773ea0740 1 -- 172.21.15.86:0/2508155462 wait complete. 2024-09-05T06:51:05.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.108+0000 7f7773ea0740 20 client.17763 trim_cache size 0 max 16384 2024-09-05T06:51:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 Processor -- start 2024-09-05T06:51:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 -- start start 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c060 0x55bc6c1730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c18ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1af110 con 0x55bc6c16c980 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b0c50 con 0x55bc6c177240 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f7773ea0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1af7e0 con 0x55bc6c16c060 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.124+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33152/0 (socket says 172.21.15.86:33152) 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 -- 172.21.15.86:0/3630332207 learned_addr learned my addr 172.21.15.86:0/3630332207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f7763fff640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c18ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f7763fff640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c060 0x55bc6c1730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 -- 172.21.15.86:0/3630332207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c060 msgr2=0x55bc6c1730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c060 0x55bc6c1730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 -- 172.21.15.86:0/3630332207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 msgr2=0x55bc6c18ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c18ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 -- 172.21.15.86:0/3630332207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6c1726f0 con 0x55bc6c16c980 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f7763fff640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c18ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55bc6c173a10 tx=0x7f775c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa64e6d4e371e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f77617fa640 1 -- 172.21.15.86:0/3630332207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c014190 con 0x55bc6c16c980 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f77617fa640 1 -- 172.21.15.86:0/3630332207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f775c012900 con 0x55bc6c16c980 2024-09-05T06:51:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.125+0000 7f77617fa640 1 -- 172.21.15.86:0/3630332207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c01d810 con 0x55bc6c16c980 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- 172.21.15.86:0/3630332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 msgr2=0x55bc6c18e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55bc6c173a10 tx=0x7f775c014840 comp rx=0 tx=0).stop 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- 172.21.15.86:0/3630332207 shutdown_connections 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c177240 0x55bc6c18ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c980 0x55bc6c18e920 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3630332207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c16c060 0x55bc6c1730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- 172.21.15.86:0/3630332207 >> 172.21.15.86:0/3630332207 conn(0x55bc6c1685f0 msgr2=0x55bc6c176200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- 172.21.15.86:0/3630332207 shutdown_connections 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- 172.21.15.86:0/3630332207 wait complete. 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.126+0000 7f7773ea0740 1 -- start start 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 Processor -- start 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c17e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c17ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b0580 con 0x55bc6c16c060 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1afeb0 con 0x55bc6c16c980 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1ad110 con 0x55bc6c177240 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7773ea0740 10 client.0 _ll_register_callbacks cb 0x7f77540095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33156/0 (socket says 172.21.15.86:33156) 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 -- 172.21.15.86:0/3459942399 learned_addr learned my addr 172.21.15.86:0/3459942399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c17e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 -- 172.21.15.86:0/3459942399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 msgr2=0x55bc6c17ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c17ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 -- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 msgr2=0x55bc6c17e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c17e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.127+0000 7f7763fff640 1 -- 172.21.15.86:0/3459942399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6c17f460 con 0x55bc6c16c060 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7763fff640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7754006e10 tx=0x7f7754009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6802b57b776b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754038070 con 0x55bc6c16c060 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9605+0+0 (secure 0 0 0) 0x7f7754009680 con 0x55bc6c16c060 2024-09-05T06:51:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c012900 con 0x55bc6c16c060 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f775401fd80 con 0x55bc6c16c060 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 10 client.17769 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc6c1f6390 con 0x55bc6c16c060 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f775c012900 con 0x55bc6c16c060 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f7754013040 con 0x55bc6c16c060 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 10 client.17769 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 10 client.17769 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.128+0000 7f7773ea0740 20 client.17769 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c1f6ab0 0x55bc6c1f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f7773ea0740 4 client.17769 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "137e9144", "release": true, "await": true}] 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "137e9144", "release": true, "await": true}) -- 0x55bc6c0d90a0 con 0x55bc6c1f6ab0 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c1f6ab0 0x55bc6c1f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f774affd640 10 client.17769 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.129+0000 7f776b7fe640 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c1f6ab0 0x55bc6c1f8ed0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d54366dd269a8d server_cookie=e241ae5f033abeae in_seq=0 out_seq=0 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f774affd640 1 -- 172.21.15.86:0/3459942399 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bc6c0d90a0 con 0x55bc6c1f6ab0 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f774affd640 10 client.17769 handle_command_reply: tid=0 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 client.17769 shutdown 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 msgr2=0x55bc6c17e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7754006e10 tx=0x7f7754009410 comp rx=0 tx=0).stop 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 shutdown_connections 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc6c1f6ab0 0x55bc6c1f8ed0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc6c177240 0x55bc6c17ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc6c16c980 0x55bc6c17e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 --2- 172.21.15.86:0/3459942399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6c16c060 0x55bc6c17e1e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 >> 172.21.15.86:0/3459942399 conn(0x55bc6c1685f0 msgr2=0x55bc6c16dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f774affd640 0 client.17769 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 shutdown_connections 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.135+0000 7f7773ea0740 1 -- 172.21.15.86:0/3459942399 wait complete. 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7773ea0740 20 client.17769 trim_cache size 0 max 16384 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 32, 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:51:05.137 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 36, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "137e9144": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 36, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:51:05.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:05.139 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 msgr2=0x7f77580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 0x7f77580a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77640bd000 tx=0x7f776402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 msgr2=0x7f776c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7764009dc0 tx=0x7f7764032fe0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 shutdown_connections 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f77580a2cf0 0x7f77580a51b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f776c106ba0 0x7f776c19cd70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f776c100a70 0x7f776c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 --2- 172.21.15.86:0/1733630551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f776c1000d0 0x7f776c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 >> 172.21.15.86:0/1733630551 conn(0x7f776c077210 msgr2=0x7f776c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 shutdown_connections 2024-09-05T06:51:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:05.136+0000 7f7771ac7640 1 -- 172.21.15.86:0/1733630551 wait complete. 2024-09-05T06:51:05.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:04 smithi115 ceph-mon[34791]: pgmap v303: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:05.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174510 1.67 MB/sec execute 385 sec latency 31678.269 ms 2024-09-05T06:51:05.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '137e9144', total seconds elapsed: 35.3 2024-09-05T06:51:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 -- 172.21.15.86:0/3098617735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101e20 msgr2=0x7fa1381056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 --2- 172.21.15.86:0/3098617735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101e20 0x7fa1381056b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa128009900 tx=0x7fa12802a410 comp rx=0 tx=0).stop 2024-09-05T06:51:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 -- 172.21.15.86:0/3098617735 shutdown_connections 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 --2- 172.21.15.86:0/3098617735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138105de0 0x7fa13810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 --2- 172.21.15.86:0/3098617735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101e20 0x7fa1381056b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 --2- 172.21.15.86:0/3098617735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138101470 0x7fa138101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 -- 172.21.15.86:0/3098617735 >> 172.21.15.86:0/3098617735 conn(0x7fa138077230 msgr2=0x7fa1380fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 -- 172.21.15.86:0/3098617735 shutdown_connections 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.348+0000 7fa13fad8640 1 -- 172.21.15.86:0/3098617735 wait complete. 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 Processor -- start 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 -- start start 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 0x7fa138198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138105de0 0x7fa13819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa138110cf0 con 0x7fa138101470 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa138110b70 con 0x7fa138101e20 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13fad8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa138110e70 con 0x7fa138105de0 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33160/0 (socket says 172.21.15.86:33160) 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 -- 172.21.15.86:0/2081075966 learned_addr learned my addr 172.21.15.86:0/2081075966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13e075640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138105de0 0x7fa13819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d073640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 0x7fa138198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 -- 172.21.15.86:0/2081075966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138105de0 msgr2=0x7fa13819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138105de0 0x7fa13819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 -- 172.21.15.86:0/2081075966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 msgr2=0x7fa138198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 0x7fa138198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d874640 1 -- 172.21.15.86:0/2081075966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13819d2f0 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.349+0000 7fa13d073640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 0x7fa138198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa13d874640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa13400d770 tx=0x7fa13400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa12effd640 1 -- 172.21.15.86:0/2081075966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134017070 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa12effd640 1 -- 172.21.15.86:0/2081075966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1340054d0 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13819d5e0 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa12effd640 1 -- 172.21.15.86:0/2081075966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134004cf0 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.350+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1381a4ec0 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa12effd640 1 -- 172.21.15.86:0/2081075966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa134004e90 con 0x7fa138101470 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 Processor -- start 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 -- start start 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654d9eb42f0 con 0x5654d9f4e5d0 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654d9e5bcf0 con 0x5654d9f59370 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.353+0000 7fa141eb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654d9eb02b0 con 0x5654d9f4fb40 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33162/0 (socket says 172.21.15.86:33162) 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d874640 1 -- 172.21.15.86:0/1517177266 learned_addr learned my addr 172.21.15.86:0/1517177266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13e075640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa12effd640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 0x7fa1240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 -- 172.21.15.86:0/1517177266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 msgr2=0x5654d9f4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 -- 172.21.15.86:0/1517177266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 msgr2=0x5654d9f4ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 -- 172.21.15.86:0/1517177266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d9e403a0 con 0x5654d9f59370 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d874640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13e075640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13e075640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 0x7fa1240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa12effd640 1 -- 172.21.15.86:0/2081075966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7fa1340cf2d0 con 0x7fa138101470 2024-09-05T06:51:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.354+0000 7fa13d073640 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f5b760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa128009230 tx=0x7fa128032b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64702b2e6e86f636 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa13e075640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 0x7fa1240a5230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa130005fd0 tx=0x7fa130014040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa123fff640 1 -- 172.21.15.86:0/1517177266 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128035050 con 0x5654d9f59370 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa123fff640 1 -- 172.21.15.86:0/1517177266 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa128032d70 con 0x5654d9f59370 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa123fff640 1 -- 172.21.15.86:0/1517177266 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280052d0 con 0x5654d9f59370 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 -- 172.21.15.86:0/1517177266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 msgr2=0x5654d9f5b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f5b760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa128009230 tx=0x7fa128032b80 comp rx=0 tx=0).stop 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 -- 172.21.15.86:0/1517177266 shutdown_connections 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f5b760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f4ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1517177266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.355+0000 7fa141eb1740 1 -- 172.21.15.86:0/1517177266 >> 172.21.15.86:0/1517177266 conn(0x5654d9f17310 msgr2=0x5654d9f4ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.356+0000 7fa141eb1740 1 -- 172.21.15.86:0/1517177266 shutdown_connections 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.356+0000 7fa141eb1740 1 -- 172.21.15.86:0/1517177266 wait complete. 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.356+0000 7fa141eb1740 1 -- start start 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.356+0000 7fa141eb1740 1 Processor -- start 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654d9f5bfb0 con 0x5654d9f4fb40 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654d9e6f060 con 0x5654d9f59370 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654d9f5c130 con 0x5654d9f4e5d0 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa141eb1740 10 client.0 _ll_register_callbacks cb 0x5654d9bc28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33176/0 (socket says 172.21.15.86:33176) 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 -- 172.21.15.86:0/1741455067 learned_addr learned my addr 172.21.15.86:0/1741455067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13e075640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 -- 172.21.15.86:0/1741455067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 msgr2=0x5654d9f58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 -- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 msgr2=0x5654d9f565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 -- 172.21.15.86:0/1741455067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d9c253c0 con 0x5654d9f4fb40 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13e075640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa13d073640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa12802a450 tx=0x7fa128005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18d62a1ac5af117 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.357+0000 7fa1217fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128052060 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa128032d70 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7fa12803b6e0 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa128035020 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 10 client.17781 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654d9f67a10 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa128032d70 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa128004a90 con 0x5654d9f4fb40 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 10 client.17781 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 10 client.17781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 20 client.17781 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9f68d60 0x5654d9f6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 4 client.17781 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d9ec1860 con 0x5654d9f68d60 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.358+0000 7fa13d874640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9f68d60 0x5654d9f6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.359+0000 7fa1217fa640 10 client.17781 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.359+0000 7fa13d874640 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9f68d60 0x5654d9f6b150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d00c607378f261c8 server_cookie=80123168548ccef9 in_seq=0 out_seq=0 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa1217fa640 1 -- 172.21.15.86:0/1741455067 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa134080e80 con 0x5654d9f68d60 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa1217fa640 10 client.17781 handle_command_reply: tid=0 2024-09-05T06:51:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 client.17781 shutdown 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 msgr2=0x5654d9f55fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa12802a450 tx=0x7fa128005360 comp rx=0 tx=0).stop 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 shutdown_connections 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9f68d60 0x5654d9f6b150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f55fe0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 --2- 172.21.15.86:0/1741455067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa1217fa640 0 client.17781 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 >> 172.21.15.86:0/1741455067 conn(0x5654d9f17310 msgr2=0x5654d9f50960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 shutdown_connections 2024-09-05T06:51:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 1 -- 172.21.15.86:0/1741455067 wait complete. 2024-09-05T06:51:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.360+0000 7fa141eb1740 20 client.17781 trim_cache size 0 max 16384 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 Processor -- start 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 -- start start 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381acfe0 con 0x5654d9f4fb40 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381aeb20 con 0x5654d9f59370 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa141eb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381ad6b0 con 0x5654d9f4e5d0 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa13d073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa13d073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33178/0 (socket says 172.21.15.86:33178) 2024-09-05T06:51:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa13d073640 1 -- 172.21.15.86:0/3553287323 learned_addr learned my addr 172.21.15.86:0/3553287323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.380+0000 7fa13e075640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d874640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 -- 172.21.15.86:0/3553287323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 msgr2=0x5654d9f51120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f51120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 -- 172.21.15.86:0/3553287323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 msgr2=0x5654d9f6e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 -- 172.21.15.86:0/3553287323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d9f590a0 con 0x5654d9f4fb40 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13e075640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa13d073640 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa128006010 tx=0x7fa128044700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01df8c41ef8e616 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa1237fe640 1 -- 172.21.15.86:0/3553287323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128052070 con 0x5654d9f4fb40 2024-09-05T06:51:06.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa1237fe640 1 -- 172.21.15.86:0/3553287323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa128004b70 con 0x5654d9f4fb40 2024-09-05T06:51:06.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.381+0000 7fa1237fe640 1 -- 172.21.15.86:0/3553287323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12803b6e0 con 0x5654d9f4fb40 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- 172.21.15.86:0/3553287323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 msgr2=0x5654d9f51660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa128006010 tx=0x7fa128044700 comp rx=0 tx=0).stop 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- 172.21.15.86:0/3553287323 shutdown_connections 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4fb40 0x5654d9f51660 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 --2- 172.21.15.86:0/3553287323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4e5d0 0x5654d9f51120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- 172.21.15.86:0/3553287323 >> 172.21.15.86:0/3553287323 conn(0x5654d9f17310 msgr2=0x5654d9f550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- 172.21.15.86:0/3553287323 shutdown_connections 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- 172.21.15.86:0/3553287323 wait complete. 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.382+0000 7fa141eb1740 1 -- start start 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 Processor -- start 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f6bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381ae450 con 0x5654d9f4e5d0 2024-09-05T06:51:06.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381add80 con 0x5654d9f59370 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381aafe0 con 0x5654d9f4fb40 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa141eb1740 10 client.0 _ll_register_callbacks cb 0x7fa1340d2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa13d874640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33188/0 (socket says 172.21.15.86:33188) 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa13d874640 1 -- 172.21.15.86:0/2604601894 learned_addr learned my addr 172.21.15.86:0/2604601894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa13d073640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f6bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.383+0000 7fa13e075640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 -- 172.21.15.86:0/2604601894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 msgr2=0x5654d9f6bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f6bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 -- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 msgr2=0x5654d9f6c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 -- 172.21.15.86:0/2604601894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d9f6c700 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13e075640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa13d874640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa13409b060 tx=0x7fa1340cf640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f232d4660565620e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134020040 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa128004b70 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7fa1340979d0 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa13409acc0 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa141eb1740 10 client.17787 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654d9fd3540 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa128004b70 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.384+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fa1340ccce0 con 0x5654d9f4e5d0 2024-09-05T06:51:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 10 client.17787 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 10 client.17787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 20 client.17787 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9fd3c10 0x5654d9fd6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 4 client.17787 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654d9ebbd70 con 0x5654d9fd3c10 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa13d073640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9fd3c10 0x5654d9fd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa1217fa640 10 client.17787 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa13d073640 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9fd3c10 0x5654d9fd6000 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7503cdbd4563318d server_cookie=d3e0a24931feb49 in_seq=0 out_seq=0 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa1217fa640 1 -- 172.21.15.86:0/2604601894 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654d9ebbd70 con 0x5654d9fd3c10 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa1217fa640 10 client.17787 handle_command_reply: tid=0 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 1 client.17787 shutdown 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 msgr2=0x5654d9f6e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.385+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa13409b060 tx=0x7fa1340cf640 comp rx=0 tx=0).stop 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 shutdown_connections 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5654d9fd3c10 0x5654d9fd6000 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654d9f59370 0x5654d9f6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654d9f4fb40 0x5654d9f6bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa1217fa640 0 client.17787 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 --2- 172.21.15.86:0/2604601894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d9f4e5d0 0x5654d9f6e780 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 >> 172.21.15.86:0/2604601894 conn(0x5654d9f17310 msgr2=0x5654d9f57840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 shutdown_connections 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 1 -- 172.21.15.86:0/2604601894 wait complete. 2024-09-05T06:51:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa141eb1740 20 client.17787 trim_cache size 0 max 16384 2024-09-05T06:51:06.396 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:06.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 msgr2=0x7fa1240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 0x7fa1240a5230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa130005fd0 tx=0x7fa130014040 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 msgr2=0x7fa138198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa13400d770 tx=0x7fa13400dc30 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 shutdown_connections 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1240a2d70 0x7fa1240a5230 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa138105de0 0x7fa13819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa138101e20 0x7fa138198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 --2- 172.21.15.86:0/2081075966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138101470 0x7fa138198340 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.386+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 >> 172.21.15.86:0/2081075966 conn(0x7fa138077230 msgr2=0x7fa138100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.387+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 shutdown_connections 2024-09-05T06:51:06.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:06.387+0000 7fa13fad8640 1 -- 172.21.15.86:0/2081075966 wait complete. 2024-09-05T06:51:06.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174999 1.68 MB/sec execute 386 sec latency 27.790 ms 2024-09-05T06:51:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:06 smithi086 ceph-mon[30943]: pgmap v304: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:06 smithi086 ceph-mon[30943]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T06:51:07.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:06 smithi096 ceph-mon[34468]: pgmap v304: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:06 smithi096 ceph-mon[34468]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T06:51:07.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:51:07.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:06 smithi115 ceph-mon[34791]: pgmap v304: 129 pgs: 129 active+clean; 184 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-05T06:51:07.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:06 smithi115 ceph-mon[34791]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T06:51:07.353 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:07.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 175514 1.68 MB/sec execute 387 sec latency 19.660 ms 2024-09-05T06:51:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 -- 172.21.15.86:0/1674843932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c106f40 msgr2=0x7ff88c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 --2- 172.21.15.86:0/1674843932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c106f40 0x7ff88c10eac0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff884009ca0 tx=0x7ff88402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 -- 172.21.15.86:0/1674843932 shutdown_connections 2024-09-05T06:51:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 --2- 172.21.15.86:0/1674843932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c106f40 0x7ff88c10eac0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 --2- 172.21.15.86:0/1674843932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 0x7ff88c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 --2- 172.21.15.86:0/1674843932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c101e10 0x7ff88c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 -- 172.21.15.86:0/1674843932 >> 172.21.15.86:0/1674843932 conn(0x7ff88c0fd8d0 msgr2=0x7ff88c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 -- 172.21.15.86:0/1674843932 shutdown_connections 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 -- 172.21.15.86:0/1674843932 wait complete. 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.856+0000 7ff893310640 1 Processor -- start 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 -- start start 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 0x7ff88c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c106f40 0x7ff88c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c111e50 con 0x7ff88c101e10 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c111cd0 con 0x7ff88c106f40 2024-09-05T06:51:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff893310640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c111fd0 con 0x7ff88c1027c0 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33208/0 (socket says 172.21.15.86:33208) 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 -- 172.21.15.86:0/1184490946 learned_addr learned my addr 172.21.15.86:0/1184490946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8918ad640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c106f40 0x7ff88c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8908ab640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 0x7ff88c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 -- 172.21.15.86:0/1184490946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 msgr2=0x7ff88c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 0x7ff88c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 -- 172.21.15.86:0/1184490946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c106f40 msgr2=0x7ff88c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c106f40 0x7ff88c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.857+0000 7ff8910ac640 1 -- 172.21.15.86:0/1184490946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c19d300 con 0x7ff88c101e10 2024-09-05T06:51:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff88800d950 tx=0x7ff88800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff8827fc640 1 -- 172.21.15.86:0/1184490946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017030 con 0x7ff88c101e10 2024-09-05T06:51:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c19d5f0 con 0x7ff88c101e10 2024-09-05T06:51:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff8827fc640 1 -- 172.21.15.86:0/1184490946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff888004020 con 0x7ff88c101e10 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c1a4ed0 con 0x7ff88c101e10 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.858+0000 7ff8827fc640 1 -- 172.21.15.86:0/1184490946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880047a0 con 0x7ff88c101e10 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.860+0000 7ff8827fc640 1 -- 172.21.15.86:0/1184490946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff8880041d0 con 0x7ff88c101e10 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 Processor -- start 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 -- start start 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8827fc640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 0x7ff8780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 0x562922d5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562922cc42f0 con 0x562922d69370 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562922c6bcf0 con 0x562922d5e5d0 2024-09-05T06:51:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8956e9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562922cc02b0 con 0x562922d5fb40 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8827fc640 1 -- 172.21.15.86:0/1184490946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7ff8880cf280 con 0x7ff88c101e10 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8918ad640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 0x7ff8780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44462/0 (socket says 172.21.15.86:44462) 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8908ab640 1 -- 172.21.15.86:0/3374957291 learned_addr learned my addr 172.21.15.86:0/3374957291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.861+0000 7ff8918ad640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8910ac640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 0x562922d5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 -- 172.21.15.86:0/3374957291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 msgr2=0x562922d5e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 0x562922d5e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 -- 172.21.15.86:0/3374957291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 msgr2=0x562922d6b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d6b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 -- 172.21.15.86:0/3374957291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562922c503a0 con 0x562922d5e5d0 2024-09-05T06:51:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8910ac640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 0x562922d5e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8918ad640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8918ad640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 0x7ff8780a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff884009430 tx=0x7ff884009950 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00b690 tx=0x7ff87c00bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3611007257f013 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8777fe640 1 -- 172.21.15.86:0/3374957291 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c003bd0 con 0x562922d5e5d0 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8777fe640 1 -- 172.21.15.86:0/3374957291 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff87c015070 con 0x562922d5e5d0 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.862+0000 7ff8777fe640 1 -- 172.21.15.86:0/3374957291 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c004d10 con 0x562922d5e5d0 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 -- 172.21.15.86:0/3374957291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 msgr2=0x562922d5ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00b690 tx=0x7ff87c00bb50 comp rx=0 tx=0).stop 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 -- 172.21.15.86:0/3374957291 shutdown_connections 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5e5d0 0x562922d5ea10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3374957291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5fb40 0x562922d5e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 -- 172.21.15.86:0/3374957291 >> 172.21.15.86:0/3374957291 conn(0x562922d27310 msgr2=0x562922d5ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 -- 172.21.15.86:0/3374957291 shutdown_connections 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.863+0000 7ff8956e9740 1 -- 172.21.15.86:0/3374957291 wait complete. 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 -- start start 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 Processor -- start 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d69370 0x562922d66640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562922d6c070 con 0x562922d5e5d0 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562922c7f060 con 0x562922d5fb40 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562922d6c1f0 con 0x562922d69370 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8956e9740 10 client.0 _ll_register_callbacks cb 0x5629229d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33224/0 (socket says 172.21.15.86:33224) 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8910ac640 1 -- 172.21.15.86:0/1512855623 learned_addr learned my addr 172.21.15.86:0/1512855623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.864+0000 7ff8908ab640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8918ad640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d69370 0x562922d66640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 -- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d69370 msgr2=0x562922d66640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d69370 0x562922d66640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 -- 172.21.15.86:0/1512855623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 msgr2=0x562922d66030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d66030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 -- 172.21.15.86:0/1512855623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922a353c0 con 0x562922d5e5d0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8908ab640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d66030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8910ac640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff888000c00 tx=0x7ff88809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fe57dafa7b1c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff874ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880a3f80 con 0x562922d5e5d0 2024-09-05T06:51:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7ff8880cc7c0 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87c015070 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff888097980 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8956e9740 10 client.17799 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.865+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562922d77ba0 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff87c015070 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff88809f160 con 0x562922d5e5d0 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 10 client.17799 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 10 client.17799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 20 client.17799 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922d7b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 4 client.17799 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562922cd1860 con 0x562922d79000 2024-09-05T06:51:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.866+0000 7ff8908ab640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922d7b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff874ff9640 10 client.17799 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8908ab640 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922d7b450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d698e2a8c4316c77 server_cookie=21b6b996bfca4ee2 in_seq=0 out_seq=0 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff874ff9640 1 -- 172.21.15.86:0/1512855623 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562922cd1860 con 0x562922d79000 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff874ff9640 10 client.17799 handle_command_reply: tid=0 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 client.17799 shutdown 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 msgr2=0x562922d68d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff888000c00 tx=0x7ff88809ba80 comp rx=0 tx=0).stop 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 shutdown_connections 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922d7b450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff874ff9640 0 client.17799 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d69370 0x562922d66640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d66030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 --2- 172.21.15.86:0/1512855623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5e5d0 0x562922d68d80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.867+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 >> 172.21.15.86:0/1512855623 conn(0x562922d27310 msgr2=0x562922d60960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.868+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 shutdown_connections 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.868+0000 7ff8956e9740 1 -- 172.21.15.86:0/1512855623 wait complete. 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.868+0000 7ff8956e9740 20 client.17799 trim_cache size 0 max 16384 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 Processor -- start 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 -- start start 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d59bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 0x562922d6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1af330 con 0x562922d5fb40 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b0e70 con 0x562922d69370 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.881+0000 7ff8956e9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1afa00 con 0x562922d5e5d0 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d59bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33228/0 (socket says 172.21.15.86:33228) 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 -- 172.21.15.86:0/300270143 learned_addr learned my addr 172.21.15.86:0/300270143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d59bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:46770/0 (socket says 172.21.15.86:46770) 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8918ad640 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 0x562922d6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 -- 172.21.15.86:0/300270143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 msgr2=0x562922d59bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d59bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 -- 172.21.15.86:0/300270143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 msgr2=0x562922d6a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 0x562922d6a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 -- 172.21.15.86:0/300270143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562922d64580 con 0x562922d5fb40 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8918ad640 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 0x562922d6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff8908ab640 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004920 tx=0x7ff87c004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8a3379d64bb7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff876ffd640 1 -- 172.21.15.86:0/300270143 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c011440 con 0x562922d5fb40 2024-09-05T06:51:07.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff876ffd640 1 -- 172.21.15.86:0/300270143 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff87c015040 con 0x562922d5fb40 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.882+0000 7ff876ffd640 1 -- 172.21.15.86:0/300270143 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c019680 con 0x562922d5fb40 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- 172.21.15.86:0/300270143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 msgr2=0x562922d5a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004920 tx=0x7ff87c004950 comp rx=0 tx=0).stop 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- 172.21.15.86:0/300270143 shutdown_connections 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d69370 0x562922d6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d5fb40 0x562922d5a130 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 --2- 172.21.15.86:0/300270143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d59bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- 172.21.15.86:0/300270143 >> 172.21.15.86:0/300270143 conn(0x562922d27310 msgr2=0x562922d594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- 172.21.15.86:0/300270143 shutdown_connections 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- 172.21.15.86:0/300270143 wait complete. 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.883+0000 7ff8956e9740 1 -- start start 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 Processor -- start 2024-09-05T06:51:07.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.963 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:07.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d80820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b07a0 con 0x562922d69370 2024-09-05T06:51:07.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b00d0 con 0x562922d5fb40 2024-09-05T06:51:07.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1ad330 con 0x562922d5e5d0 2024-09-05T06:51:07.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8956e9740 10 client.0 _ll_register_callbacks cb 0x7ff8880046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8908ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44494/0 (socket says 172.21.15.86:44494) 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8908ab640 1 -- 172.21.15.86:0/3941502563 learned_addr learned my addr 172.21.15.86:0/3941502563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8918ad640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d80820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 -- 172.21.15.86:0/3941502563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 msgr2=0x562922d802e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 -- 172.21.15.86:0/3941502563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 msgr2=0x562922d80820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d80820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8910ac640 1 -- 172.21.15.86:0/3941502563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922d80d60 con 0x562922d5e5d0 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.884+0000 7ff8918ad640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d80820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:07.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8910ac640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d718b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff888009610 tx=0x7ff88808ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9ddb024cb457b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880a3f80 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7ff88809a760 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87c015040 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff88809ac70 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 10 client.26917 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562922d70160 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff88807c030 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff888097060 con 0x562922d5e5d0 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 10 client.26917 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 10 client.26917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 20 client.26917 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922dea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:07.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 4 client.26917 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.885+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562922ccbd70 con 0x562922d79000 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922dea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff874ff9640 10 client.26917 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8908ab640 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922dea3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2da329e92f9d0a76 server_cookie=2d5b07603116f392 in_seq=0 out_seq=0 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff874ff9640 1 -- 172.21.15.86:0/3941502563 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562922ccbd70 con 0x562922d79000 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff874ff9640 10 client.26917 handle_command_reply: tid=0 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 client.26917 shutdown 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 msgr2=0x562922d718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d718b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff888009610 tx=0x7ff88808ade0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 shutdown_connections 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562922d79000 0x562922dea3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562922d69370 0x562922d80820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff874ff9640 0 client.26917 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562922d5fb40 0x562922d802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 --2- 172.21.15.86:0/3941502563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562922d5e5d0 0x562922d718b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 >> 172.21.15.86:0/3941502563 conn(0x562922d27310 msgr2=0x562922d59d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 shutdown_connections 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.886+0000 7ff8956e9740 1 -- 172.21.15.86:0/3941502563 wait complete. 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff8956e9740 20 client.26917 trim_cache size 0 max 16384 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 msgr2=0x7ff8780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 0x7ff8780a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff884009430 tx=0x7ff884009950 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 msgr2=0x7ff88c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff88800d950 tx=0x7ff88800de10 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 shutdown_connections 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8780a2cf0 0x7ff8780a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff88c106f40 0x7ff88c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c1027c0 0x7ff88c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 --2- 172.21.15.86:0/1184490946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff88c101e10 0x7ff88c198350 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:07.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 >> 172.21.15.86:0/1184490946 conn(0x7ff88c0fd8d0 msgr2=0x7ff88c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:07.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 shutdown_connections 2024-09-05T06:51:07.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:07.887+0000 7ff893310640 1 -- 172.21.15.86:0/1184490946 wait complete. 2024-09-05T06:51:08.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 176150 1.68 MB/sec execute 388 sec latency 33.426 ms 2024-09-05T06:51:08.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:51:09.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:08 smithi096 ceph-mon[34468]: pgmap v306: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:09.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:08 smithi115 ceph-mon[34791]: pgmap v306: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:09.209 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:08 smithi086 ceph-mon[30943]: pgmap v306: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:09.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 176757 1.69 MB/sec execute 389 sec latency 22.844 ms 2024-09-05T06:51:09.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 -- 172.21.15.86:0/697989437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc101420 msgr2=0x7f3ecc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 --2- 172.21.15.86:0/697989437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc101420 0x7f3ecc10d950 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009cd0 tx=0x7f3ec402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 -- 172.21.15.86:0/697989437 shutdown_connections 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 --2- 172.21.15.86:0/697989437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc101420 0x7f3ecc10d950 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 --2- 172.21.15.86:0/697989437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ecc100a80 0x7f3ecc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 --2- 172.21.15.86:0/697989437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 0x7f3ecc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 -- 172.21.15.86:0/697989437 >> 172.21.15.86:0/697989437 conn(0x7f3ecc0fc7a0 msgr2=0x7f3ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 -- 172.21.15.86:0/697989437 shutdown_connections 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.588+0000 7f3ed3559640 1 -- 172.21.15.86:0/697989437 wait complete. 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.589+0000 7f3ed3559640 1 Processor -- start 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.589+0000 7f3ed3559640 1 -- start start 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ecc101420 0x7f3ecc198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 0x7f3ecc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc110d30 con 0x7f3ecc100a80 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33262/0 (socket says 172.21.15.86:33262) 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 -- 172.21.15.86:0/4127257861 learned_addr learned my addr 172.21.15.86:0/4127257861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc110bb0 con 0x7f3ecc101420 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc110eb0 con 0x7f3ecc106a80 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 0x7f3ecc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 -- 172.21.15.86:0/4127257861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 msgr2=0x7f3ecc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 0x7f3ecc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 -- 172.21.15.86:0/4127257861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ecc101420 msgr2=0x7f3ecc198880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ecc101420 0x7f3ecc198880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.590+0000 7f3ed12f5640 1 -- 172.21.15.86:0/4127257861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc19d2f0 con 0x7f3ecc100a80 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d770 tx=0x7f3ec800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ec27fc640 1 -- 172.21.15.86:0/4127257861 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8017070 con 0x7f3ecc100a80 2024-09-05T06:51:09.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ec27fc640 1 -- 172.21.15.86:0/4127257861 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec80054d0 con 0x7f3ecc100a80 2024-09-05T06:51:09.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc19d5e0 con 0x7f3ecc100a80 2024-09-05T06:51:09.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ec27fc640 1 -- 172.21.15.86:0/4127257861 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8004cf0 con 0x7f3ecc100a80 2024-09-05T06:51:09.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.591+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc1029d0 con 0x7f3ecc100a80 2024-09-05T06:51:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.594+0000 7f3ec27fc640 1 -- 172.21.15.86:0/4127257861 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3ec8004e90 con 0x7f3ecc100a80 2024-09-05T06:51:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.594+0000 7f3ed5932740 1 Processor -- start 2024-09-05T06:51:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 -- start start 2024-09-05T06:51:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 0x5654cf3e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654cf3492f0 con 0x5654cf3e4b40 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654cf2f0cf0 con 0x5654cf3ee370 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed5932740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654cf3452b0 con 0x5654cf3e35d0 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed1af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 0x5654cf3e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ec27fc640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 0x7f3eb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed12f5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed1af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 0x5654cf3e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33264/0 (socket says 172.21.15.86:33264) 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed1af6640 1 -- 172.21.15.86:0/2212174724 learned_addr learned my addr 172.21.15.86:0/2212174724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 -- 172.21.15.86:0/2212174724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 msgr2=0x5654cf3e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 0x7f3eb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 -- 172.21.15.86:0/2212174724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 msgr2=0x5654cf3e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 0x5654cf3e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.595+0000 7f3ed0af4640 1 -- 172.21.15.86:0/2212174724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cf2d53a0 con 0x5654cf3ee370 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3ec27fc640 1 -- 172.21.15.86:0/4127257861 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7f3ec80cf2c0 con 0x7f3ecc100a80 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f0760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009aa0 tx=0x7f3ebc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba40c5ad7be3b8e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3eb77fe640 1 -- 172.21.15.86:0/2212174724 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00ece0 con 0x5654cf3ee370 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3eb77fe640 1 -- 172.21.15.86:0/2212174724 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc00ee80 con 0x5654cf3ee370 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3eb77fe640 1 -- 172.21.15.86:0/2212174724 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0049e0 con 0x5654cf3ee370 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.596+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 0x7f3eb80a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005e10 tx=0x7f3ec402a8e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- 172.21.15.86:0/2212174724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 msgr2=0x5654cf3f0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f0760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009aa0 tx=0x7f3ebc009f60 comp rx=0 tx=0).stop 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- 172.21.15.86:0/2212174724 shutdown_connections 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f0760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2212174724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e4b40 0x5654cf3e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- 172.21.15.86:0/2212174724 >> 172.21.15.86:0/2212174724 conn(0x5654cf3ac310 msgr2=0x5654cf3dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- 172.21.15.86:0/2212174724 shutdown_connections 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- 172.21.15.86:0/2212174724 wait complete. 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 -- start start 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.597+0000 7f3ed5932740 1 Processor -- start 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 0x5654cf3eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654cf3f1070 con 0x5654cf3e35d0 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654cf304060 con 0x5654cf3e4b40 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654cf3f11f0 con 0x5654cf3ee370 2024-09-05T06:51:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed5932740 10 client.0 _ll_register_callbacks cb 0x5654cf0578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33280/0 (socket says 172.21.15.86:33280) 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 -- 172.21.15.86:0/2819589017 learned_addr learned my addr 172.21.15.86:0/2819589017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 0x5654cf3eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 -- 172.21.15.86:0/2819589017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 msgr2=0x5654cf3eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 0x5654cf3eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 -- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 msgr2=0x5654cf3eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 -- 172.21.15.86:0/2819589017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cf0ba3c0 con 0x5654cf3e35d0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 0x5654cf3eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d740 tx=0x7f3ec800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bef6c33f7b752d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.598+0000 7f3eb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8020070 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7f3ec809a7a0 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654cf356860 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ec8005180 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 10 client.17817 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654cf3fcb10 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec8009c30 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3ec809f170 con 0x5654cf3e35d0 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 10 client.17817 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 20 client.17817 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf3fe020 0x5654cf400470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 4 client.17817 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.599+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cf356860 con 0x5654cf3fe020 2024-09-05T06:51:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.600+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf3fe020 0x5654cf400470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.600+0000 7f3eb4ff9640 10 client.17817 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.600+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf3fe020 0x5654cf400470 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26a8775361f5ebb server_cookie=6fa5c7691af81a1e in_seq=0 out_seq=0 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.600+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/2819589017 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654cf356860 con 0x5654cf3fe020 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3eb4ff9640 10 client.17817 handle_command_reply: tid=0 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 client.17817 shutdown 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 msgr2=0x5654cf3edd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d740 tx=0x7f3ec800df00 comp rx=0 tx=0).stop 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 shutdown_connections 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf3fe020 0x5654cf400470 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3ee370 0x5654cf3eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3eb4ff9640 0 client.17817 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 --2- 172.21.15.86:0/2819589017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3e35d0 0x5654cf3edd60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 >> 172.21.15.86:0/2819589017 conn(0x5654cf3ac310 msgr2=0x5654cf3e5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 shutdown_connections 2024-09-05T06:51:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 1 -- 172.21.15.86:0/2819589017 wait complete. 2024-09-05T06:51:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.601+0000 7f3ed5932740 20 client.17817 trim_cache size 0 max 16384 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.615+0000 7f3ed5932740 1 Processor -- start 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.615+0000 7f3ed5932740 1 -- start start 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 0x5654cf405d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ad0d0 con 0x5654cf3ee370 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1aec10 con 0x5654cf3e4b40 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed5932740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ad7a0 con 0x5654cf3e35d0 2024-09-05T06:51:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed1af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 0x5654cf405d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44536/0 (socket says 172.21.15.86:44536) 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 -- 172.21.15.86:0/673842116 learned_addr learned my addr 172.21.15.86:0/673842116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed1af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 0x5654cf405d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33288/0 (socket says 172.21.15.86:33288) 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 -- 172.21.15.86:0/673842116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 msgr2=0x5654cf3fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 -- 172.21.15.86:0/673842116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 msgr2=0x5654cf405d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 0x5654cf405d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 -- 172.21.15.86:0/673842116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cf3e3be0 con 0x5654cf3e4b40 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.616+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009a70 tx=0x7f3ebc005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a05723a2f090be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.617+0000 7f3eb6ffd640 1 -- 172.21.15.86:0/673842116 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0050c0 con 0x5654cf3e4b40 2024-09-05T06:51:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.617+0000 7f3eb6ffd640 1 -- 172.21.15.86:0/673842116 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc00eae0 con 0x5654cf3e4b40 2024-09-05T06:51:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.617+0000 7f3eb6ffd640 1 -- 172.21.15.86:0/673842116 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc018470 con 0x5654cf3e4b40 2024-09-05T06:51:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.617+0000 7f3ed5932740 1 -- 172.21.15.86:0/673842116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 msgr2=0x5654cf3daad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.617+0000 7f3ed5932740 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009a70 tx=0x7f3ebc005550 comp rx=0 tx=0).stop 2024-09-05T06:51:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 -- 172.21.15.86:0/673842116 shutdown_connections 2024-09-05T06:51:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3ee370 0x5654cf405d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3e4b40 0x5654cf3daad0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009a70 tx=0x7f3ebc005550 comp rx=0 tx=0).stop 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 --2- 172.21.15.86:0/673842116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 -- 172.21.15.86:0/673842116 >> 172.21.15.86:0/673842116 conn(0x5654cf3ac310 msgr2=0x5654cf3e98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 -- 172.21.15.86:0/673842116 shutdown_connections 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 -- 172.21.15.86:0/673842116 wait complete. 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 -- start start 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.618+0000 7f3ed5932740 1 Processor -- start 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 0x5654cf3f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ae540 con 0x5654cf3f1ad0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ade70 con 0x5654cf3ee370 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ab0d0 con 0x5654cf3e35d0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed5932740 10 client.0 _ll_register_callbacks cb 0x7f3ec80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed1af6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 0x5654cf3f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44552/0 (socket says 172.21.15.86:44552) 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 -- 172.21.15.86:0/1412082847 learned_addr learned my addr 172.21.15.86:0/1412082847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 -- 172.21.15.86:0/1412082847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 msgr2=0x5654cf3f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 -- 172.21.15.86:0/1412082847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 msgr2=0x5654cf3f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 0x5654cf3f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed0af4640 1 -- 172.21.15.86:0/1412082847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cf3f2760 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.619+0000 7f3ed1af6640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 0x5654cf3f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed0af4640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc003bf0 tx=0x7f3ebc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da21a3c4c48a962f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00ec90 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc00eae0 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9693+0+0 (secure 0 0 0) 0x7f3ebc003d50 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc0035f0 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 10 client.26960 fetch_fsmap learned FSMap version 32 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654cf468ce0 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc00eae0 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3ebc023040 con 0x5654cf3ee370 2024-09-05T06:51:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 10 client.26960 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.620+0000 7f3ed5932740 20 client.26960 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf4691d0 0x5654cf46b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 4 client.26960 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654cf350d70 con 0x5654cf4691d0 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf4691d0 0x5654cf46b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed12f5640 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf4691d0 0x5654cf46b5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfedecb75aebb951 server_cookie=b7b74f7268a7583c in_seq=0 out_seq=0 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3eb4ff9640 10 client.26960 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3eb4ff9640 1 -- 172.21.15.86:0/1412082847 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654cf350d70 con 0x5654cf4691d0 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3eb4ff9640 10 client.26960 handle_command_reply: tid=0 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 client.26960 shutdown 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 msgr2=0x5654cf3f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc003bf0 tx=0x7f3ebc005850 comp rx=0 tx=0).stop 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 shutdown_connections 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654cf4691d0 0x5654cf46b5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654cf3f1ad0 0x5654cf3f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf3ee370 0x5654cf3f6ce0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 --2- 172.21.15.86:0/1412082847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654cf3e35d0 0x5654cf3f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3eb4ff9640 0 client.26960 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.621+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 >> 172.21.15.86:0/1412082847 conn(0x5654cf3ac310 msgr2=0x5654cf3e5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 shutdown_connections 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed5932740 1 -- 172.21.15.86:0/1412082847 wait complete. 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed5932740 20 client.26960 trim_cache size 0 max 16384 2024-09-05T06:51:09.627 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 msgr2=0x7f3eb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 0x7f3eb80a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005e10 tx=0x7f3ec402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 msgr2=0x7f3ecc198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d770 tx=0x7f3ec800dc30 comp rx=0 tx=0).stop 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 shutdown_connections 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3eb80a2d70 0x7f3eb80a5230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ecc106a80 0x7f3ecc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ecc101420 0x7f3ecc198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 --2- 172.21.15.86:0/4127257861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc100a80 0x7f3ecc198340 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 >> 172.21.15.86:0/4127257861 conn(0x7f3ecc0fc7a0 msgr2=0x7f3ecc104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 shutdown_connections 2024-09-05T06:51:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:09.622+0000 7f3ed3559640 1 -- 172.21.15.86:0/4127257861 wait complete. 2024-09-05T06:51:10.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 177310 1.70 MB/sec execute 390 sec latency 17.946 ms 2024-09-05T06:51:11.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:10 smithi096 ceph-mon[34468]: pgmap v307: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:11.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:10 smithi115 ceph-mon[34791]: pgmap v307: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:11.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:10 smithi086 ceph-mon[30943]: pgmap v307: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-09-05T06:51:11.514 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:51:11.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:51:11.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 177683 1.70 MB/sec execute 391 sec latency 20.980 ms 2024-09-05T06:51:11.782 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:12.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 177960 1.69 MB/sec execute 392 sec latency 19.786 ms 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 -- 172.21.15.86:0/623745308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100aa0 msgr2=0x7ffa6c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/623745308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c10ea20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009ca0 tx=0x7ffa6402a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 -- 172.21.15.86:0/623745308 shutdown_connections 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/623745308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c10ea20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/623745308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c100100 0x7ffa6c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/623745308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c106b80 0x7ffa6c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.519+0000 7ffa71ef7640 1 -- 172.21.15.86:0/623745308 >> 172.21.15.86:0/623745308 conn(0x7ffa6c077230 msgr2=0x7ffa6c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.520+0000 7ffa71ef7640 1 -- 172.21.15.86:0/623745308 shutdown_connections 2024-09-05T06:51:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.520+0000 7ffa71ef7640 1 -- 172.21.15.86:0/623745308 wait complete. 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.520+0000 7ffa71ef7640 1 Processor -- start 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.520+0000 7ffa71ef7640 1 -- start start 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.520+0000 7ffa71ef7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c106b80 0x7ffa6c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c111f00 con 0x7ffa6c100100 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c111d80 con 0x7ffa6c106b80 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c112080 con 0x7ffa6c100aa0 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54134/0 (socket says 172.21.15.86:54134) 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/4048754666 learned_addr learned my addr 172.21.15.86:0/4048754666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c106b80 0x7ffa6c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6affd640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/4048754666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 msgr2=0x7ffa6c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/4048754666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c106b80 msgr2=0x7ffa6c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c106b80 0x7ffa6c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/4048754666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c19d350 con 0x7ffa6c100100 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6affd640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9f0 tx=0x7ffa5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa68ff9640 1 -- 172.21.15.86:0/4048754666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00cd80 con 0x7ffa6c100100 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa68ff9640 1 -- 172.21.15.86:0/4048754666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa5c0054d0 con 0x7ffa6c100100 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c19d640 con 0x7ffa6c100100 2024-09-05T06:51:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa68ff9640 1 -- 172.21.15.86:0/4048754666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c010490 con 0x7ffa6c100100 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.521+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c1a4f20 con 0x7ffa6c100100 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.524+0000 7ffa68ff9640 1 -- 172.21.15.86:0/4048754666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa5c005020 con 0x7ffa6c100100 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.524+0000 7ffa742d0740 1 Processor -- start 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.524+0000 7ffa742d0740 1 -- start start 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee50146280 con 0x55ee501e1480 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee500edb10 con 0x55ee501e0b60 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa742d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ee50142240 con 0x55ee501ebd40 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa68ff9640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 0x7ffa540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54144/0 (socket says 172.21.15.86:54144) 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/875011554 learned_addr learned my addr 172.21.15.86:0/875011554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 0x7ffa540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa68ff9640 1 -- 172.21.15.86:0/4048754666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ffa5c0cfbf0 con 0x7ffa6c100100 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 -- 172.21.15.86:0/875011554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 msgr2=0x55ee501e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 -- 172.21.15.86:0/875011554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 msgr2=0x55ee501eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.525+0000 7ffa6affd640 1 -- 172.21.15.86:0/875011554 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee500fa1f0 con 0x55ee501ebd40 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.526+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 0x7ffa540a5230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa64005e30 tx=0x7ffa64004a60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.526+0000 7ffa6affd640 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ee130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005fd0 tx=0x7ffa6000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f48d0e5ee0efda server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.526+0000 7ffa51ffb640 1 -- 172.21.15.86:0/875011554 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60005020 con 0x55ee501ebd40 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.526+0000 7ffa51ffb640 1 -- 172.21.15.86:0/875011554 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa600047a0 con 0x55ee501ebd40 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.526+0000 7ffa51ffb640 1 -- 172.21.15.86:0/875011554 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa600116a0 con 0x55ee501ebd40 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- 172.21.15.86:0/875011554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 msgr2=0x55ee501ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ee130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005fd0 tx=0x7ffa6000f730 comp rx=0 tx=0).stop 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- 172.21.15.86:0/875011554 shutdown_connections 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ee130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 --2- 172.21.15.86:0/875011554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- 172.21.15.86:0/875011554 >> 172.21.15.86:0/875011554 conn(0x55ee501dd0f0 msgr2=0x55ee501dd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- 172.21.15.86:0/875011554 shutdown_connections 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- 172.21.15.86:0/875011554 wait complete. 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 -- start start 2024-09-05T06:51:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 Processor -- start 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.527+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee501ee910 con 0x55ee501e1480 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee50100f10 con 0x55ee501e0b60 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ee501eea90 con 0x55ee501ebd40 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 10 client.0 _ll_register_callbacks cb 0x55ee500c9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54150/0 (socket says 172.21.15.86:54150) 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 -- 172.21.15.86:0/1867382645 learned_addr learned my addr 172.21.15.86:0/1867382645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 -- 172.21.15.86:0/1867382645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 msgr2=0x55ee501e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501e9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 -- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 msgr2=0x55ee501e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 -- 172.21.15.86:0/1867382645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4feb76a0 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa6affd640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa60005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c0560fd7a6794b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004b60 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ffa60004020 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa600047a0 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa60004220 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 10 client.17835 fetch_fsmap learned FSMap version 33 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.528+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee501fa4a0 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa600047a0 con 0x55ee501e1480 2024-09-05T06:51:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ffa600038c0 con 0x55ee501e1480 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 10 client.17835 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 20 client.17835 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee501fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 4 client.17835 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee501537f0 con 0x55ee501fb900 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.529+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee501fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.530+0000 7ffa4affd640 10 client.17835 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.530+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee501fdcf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349657b3a256eda0 server_cookie=ec4c952e7e7daf76 in_seq=0 out_seq=0 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.530+0000 7ffa4affd640 1 -- 172.21.15.86:0/1867382645 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee501537f0 con 0x55ee501fb900 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.530+0000 7ffa4affd640 10 client.17835 handle_command_reply: tid=0 2024-09-05T06:51:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.530+0000 7ffa742d0740 1 client.17835 shutdown 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 msgr2=0x55ee501e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa60005770 comp rx=0 tx=0).stop 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 shutdown_connections 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee501fdcf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa4affd640 0 client.17835 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501e90e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 --2- 172.21.15.86:0/1867382645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e0b60 0x55ee501e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 >> 172.21.15.86:0/1867382645 conn(0x55ee501dd0f0 msgr2=0x55ee501e2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 shutdown_connections 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 1 -- 172.21.15.86:0/1867382645 wait complete. 2024-09-05T06:51:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.531+0000 7ffa742d0740 20 client.17835 trim_cache size 0 max 16384 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 Processor -- start 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 -- start start 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 0x55ee502037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1af150 con 0x55ee501e0b60 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1b0cc0 con 0x55ee501e1480 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa742d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1af820 con 0x55ee501ebd40 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54160/0 (socket says 172.21.15.86:54160) 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/2917596515 learned_addr learned my addr 172.21.15.86:0/2917596515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6affd640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 0x55ee502037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/2917596515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 msgr2=0x55ee501ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/2917596515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 msgr2=0x55ee502037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 0x55ee502037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/2917596515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee501e12b0 con 0x55ee501e0b60 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.550+0000 7ffa6affd640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 0x55ee502037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0986a0 tx=0x7ffa5c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e2ee209f742f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa517fa640 1 -- 172.21.15.86:0/2917596515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0ba070 con 0x55ee501e0b60 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa517fa640 1 -- 172.21.15.86:0/2917596515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa5c005310 con 0x55ee501e0b60 2024-09-05T06:51:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa517fa640 1 -- 172.21.15.86:0/2917596515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0046c0 con 0x55ee501e0b60 2024-09-05T06:51:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa742d0740 1 -- 172.21.15.86:0/2917596515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 msgr2=0x55ee50203290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.551+0000 7ffa742d0740 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0986a0 tx=0x7ffa5c0cf040 comp rx=0 tx=0).stop 2024-09-05T06:51:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- 172.21.15.86:0/2917596515 shutdown_connections 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501ebd40 0x55ee501ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501e1480 0x55ee502037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- 172.21.15.86:0/2917596515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e0b60 0x55ee50203290 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- 172.21.15.86:0/2917596515 >> 172.21.15.86:0/2917596515 conn(0x55ee501dd0f0 msgr2=0x55ee501e2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- 172.21.15.86:0/2917596515 shutdown_connections 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- 172.21.15.86:0/2917596515 wait complete. 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- start start 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 Processor -- start 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 0x55ee501f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501ef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501ebd40 0x55ee501efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1b05f0 con 0x55ee501e1480 2024-09-05T06:51:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1afef0 con 0x55ee501ebd40 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.552+0000 7ffa742d0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c1ad150 con 0x55ee501e0b60 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa742d0740 10 client.0 _ll_register_callbacks cb 0x7ffa60015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 0x55ee501f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501ef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54162/0 (socket says 172.21.15.86:54162) 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6affd640 1 -- 172.21.15.86:0/896791881 learned_addr learned my addr 172.21.15.86:0/896791881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6bfff640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501ebd40 0x55ee501efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501ebd40 msgr2=0x55ee501efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501ebd40 0x55ee501efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/896791881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 msgr2=0x55ee501ef530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501ef530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 -- 172.21.15.86:0/896791881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee501f0030 con 0x55ee501e0b60 2024-09-05T06:51:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa6b7fe640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 0x55ee501f4200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0986a0 tx=0x7ffa5c0a0180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d237fc8f6a6fd81a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0ae060 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ffa5c098340 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.553+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee5014dd00 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c0ba040 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 10 client.26953 fetch_fsmap learned FSMap version 33 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee50266070 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ee5014dd00 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ffa5c097a20 con 0x55ee501e0b60 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 10 client.26953 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 20 client.26953 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee50268750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:12.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 4 client.26953 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:51:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.554+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee500ca110 con 0x55ee501fb900 2024-09-05T06:51:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.555+0000 7ffa6affd640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee50268750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.555+0000 7ffa4affd640 10 client.26953 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.555+0000 7ffa6affd640 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee50268750 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f069b76b98cf2a server_cookie=dbe1ed8a2bdffb92 in_seq=0 out_seq=0 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa4affd640 1 -- 172.21.15.86:0/896791881 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee500ca110 con 0x55ee501fb900 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa4affd640 10 client.26953 handle_command_reply: tid=0 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 client.26953 shutdown 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 msgr2=0x55ee501f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 0x55ee501f4200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0986a0 tx=0x7ffa5c0a0180 comp rx=0 tx=0).stop 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 shutdown_connections 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ee501fb900 0x55ee50268750 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee501ebd40 0x55ee501efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa4affd640 0 client.26953 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee501e1480 0x55ee501ef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 --2- 172.21.15.86:0/896791881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee501e0b60 0x55ee501f4200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 >> 172.21.15.86:0/896791881 conn(0x55ee501dd0f0 msgr2=0x55ee501e8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 shutdown_connections 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 1 -- 172.21.15.86:0/896791881 wait complete. 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.556+0000 7ffa742d0740 20 client.26953 trim_cache size 0 max 16384 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5d3b8c65-f660-4dfe-bd69-7e6a8378c934", 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:51:12.561 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 msgr2=0x7ffa540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 0x7ffa540a5230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa64005e30 tx=0x7ffa64004a60 comp rx=0 tx=0).stop 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 msgr2=0x7ffa6c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9f0 tx=0x7ffa5c00edf0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 shutdown_connections 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ffa540a2d70 0x7ffa540a5230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa6c106b80 0x7ffa6c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c100aa0 0x7ffa6c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 --2- 172.21.15.86:0/4048754666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6c100100 0x7ffa6c1983a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 >> 172.21.15.86:0/4048754666 conn(0x7ffa6c077230 msgr2=0x7ffa6c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 shutdown_connections 2024-09-05T06:51:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:12.557+0000 7ffa71ef7640 1 -- 172.21.15.86:0/4048754666 wait complete. 2024-09-05T06:51:13.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:12 smithi096 ceph-mon[34468]: pgmap v308: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-09-05T06:51:13.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:12 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:13.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:12 smithi096 ceph-mon[34468]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T06:51:13.173 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:12 smithi086 ceph-mon[30943]: pgmap v308: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-09-05T06:51:13.173 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:12 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:13.173 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:12 smithi086 ceph-mon[30943]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T06:51:13.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:12 smithi115 ceph-mon[34791]: pgmap v308: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-09-05T06:51:13.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:12 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:13.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:12 smithi115 ceph-mon[34791]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T06:51:13.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:51:13.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 178438 1.69 MB/sec execute 393 sec latency 18.292 ms 2024-09-05T06:51:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:14.044 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:51:14.044 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.044 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.045 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:13 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:51:14.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:13 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:51:14.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.184 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:13 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:51:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 -- 172.21.15.86:0/3433929894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134106f10 msgr2=0x7f413410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 --2- 172.21.15.86:0/3433929894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134106f10 0x7f413410ea90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f412c009ca0 tx=0x7f412c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 -- 172.21.15.86:0/3433929894 shutdown_connections 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 --2- 172.21.15.86:0/3433929894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134106f10 0x7f413410ea90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 --2- 172.21.15.86:0/3433929894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134102790 0x7f41341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 --2- 172.21.15.86:0/3433929894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134101de0 0x7f41341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 -- 172.21.15.86:0/3433929894 >> 172.21.15.86:0/3433929894 conn(0x7f41340fd8c0 msgr2=0x7f41340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 -- 172.21.15.86:0/3433929894 shutdown_connections 2024-09-05T06:51:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.321+0000 7f413c1bc640 1 -- 172.21.15.86:0/3433929894 wait complete. 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 Processor -- start 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 -- start start 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134101de0 0x7f41340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134106f10 0x7f4134074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111e20 con 0x7f4134101de0 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111ca0 con 0x7f4134106f10 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413c1bc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4134111fa0 con 0x7f4134102790 2024-09-05T06:51:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139f58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134101de0 0x7f41340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51598/0 (socket says 172.21.15.86:51598) 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 -- 172.21.15.86:0/1469330506 learned_addr learned my addr 172.21.15.86:0/1469330506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f413a759640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134106f10 0x7f4134074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 -- 172.21.15.86:0/1469330506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134106f10 msgr2=0x7f4134074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134106f10 0x7f4134074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 -- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134101de0 msgr2=0x7f41340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134101de0 0x7f41340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.322+0000 7f4139757640 1 -- 172.21.15.86:0/1469330506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341a67e0 con 0x7f4134102790 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f4139757640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f412400ea60 tx=0x7f412400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f412affd640 1 -- 172.21.15.86:0/1469330506 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412400cde0 con 0x7f4134102790 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a6ad0 con 0x7f4134102790 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f412affd640 1 -- 172.21.15.86:0/1469330506 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41240035f0 con 0x7f4134102790 2024-09-05T06:51:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f412affd640 1 -- 172.21.15.86:0/1469330506 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124010690 con 0x7f4134102790 2024-09-05T06:51:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.323+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a7060 con 0x7f4134102790 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.325+0000 7f412affd640 1 -- 172.21.15.86:0/1469330506 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4124004020 con 0x7f4134102790 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.326+0000 7f412affd640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 0x7f41200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.326+0000 7f412affd640 1 -- 172.21.15.86:0/1469330506 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f41240cfa60 con 0x7f4134102790 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.326+0000 7f4139f58640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 0x7f41200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.326+0000 7f413e595740 1 Processor -- start 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139f58640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 0x7f41200a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4130009bd0 tx=0x7f4130005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 -- start start 2024-09-05T06:51:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 0x55f02a4c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f02a4232e0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f02a3cace0 con 0x55f02a4c8680 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413e595740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f02a41f2a0 con 0x55f02a4bddc0 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139f58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139f58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54178/0 (socket says 172.21.15.86:54178) 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139f58640 1 -- 172.21.15.86:0/584339168 learned_addr learned my addr 172.21.15.86:0/584339168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 0x55f02a4c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413a759640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 -- 172.21.15.86:0/584339168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 msgr2=0x55f02a4caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 -- 172.21.15.86:0/584339168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 msgr2=0x55f02a4bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139757640 1 -- 172.21.15.86:0/584339168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02a3af440 con 0x55f02a4bddc0 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f413a759640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.327+0000 7f4139f58640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.328+0000 7f4139757640 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 0x55f02a4c8140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41240ccd10 tx=0x7f412409b440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33fc397318837d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.328+0000 7f411ffff640 1 -- 172.21.15.86:0/584339168 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240b3020 con 0x55f02a4bddc0 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.328+0000 7f411ffff640 1 -- 172.21.15.86:0/584339168 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f412409c1e0 con 0x55f02a4bddc0 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.328+0000 7f411ffff640 1 -- 172.21.15.86:0/584339168 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124098150 con 0x55f02a4bddc0 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- 172.21.15.86:0/584339168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 msgr2=0x55f02a4c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 0x55f02a4c8140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41240ccd10 tx=0x7f412409b440 comp rx=0 tx=0).stop 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- 172.21.15.86:0/584339168 shutdown_connections 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bddc0 0x55f02a4c8140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 --2- 172.21.15.86:0/584339168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bd4a0 0x55f02a4bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- 172.21.15.86:0/584339168 >> 172.21.15.86:0/584339168 conn(0x55f02a4b9a30 msgr2=0x55f02a4b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- 172.21.15.86:0/584339168 shutdown_connections 2024-09-05T06:51:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- 172.21.15.86:0/584339168 wait complete. 2024-09-05T06:51:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 -- start start 2024-09-05T06:51:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.329+0000 7f413e595740 1 Processor -- start 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f02a4cb250 con 0x55f02a4bddc0 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f02a3de050 con 0x55f02a4c8680 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f02a4cb3d0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51612/0 (socket says 172.21.15.86:51612) 2024-09-05T06:51:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 -- 172.21.15.86:0/2091086049 learned_addr learned my addr 172.21.15.86:0/2091086049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413e595740 10 client.0 _ll_register_callbacks cb 0x55f02a41e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139757640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 -- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 msgr2=0x55f02a4c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413a759640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 -- 172.21.15.86:0/2091086049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 msgr2=0x55f02a4c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f4139f58640 1 -- 172.21.15.86:0/2091086049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f02a1943c0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.330+0000 7f413a759640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f4139f58640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4130003a20 tx=0x7f4130004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a17ba80af61919 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413001fdc0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412409c1e0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f41300273e0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f41300275e0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 10 client.26965 fetch_fsmap learned FSMap version 33 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f02a4d6c80 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f412409c1e0 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4130026070 con 0x55f02a4bd4a0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 10 client.26965 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 10 client.26965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 20 client.26965 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.331+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a4d8140 0x55f02a4da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.332+0000 7f413e595740 4 client.26965 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.332+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f02a430850 con 0x55f02a4d8140 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.332+0000 7f4139757640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a4d8140 0x55f02a4da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.332+0000 7f411d7fa640 10 client.26965 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.332+0000 7f4139757640 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a4d8140 0x55f02a4da5d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4cecb3175fc736 server_cookie=b5d840b8b9a42832 in_seq=0 out_seq=0 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f411d7fa640 1 -- 172.21.15.86:0/2091086049 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f02a430850 con 0x55f02a4d8140 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f411d7fa640 10 client.26965 handle_command_reply: tid=0 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 client.26965 shutdown 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 msgr2=0x55f02a4c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4130003a20 tx=0x7f4130004730 comp rx=0 tx=0).stop 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 shutdown_connections 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a4d8140 0x55f02a4da5d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4c8680 0x55f02a4c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 --2- 172.21.15.86:0/2091086049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c8000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 >> 172.21.15.86:0/2091086049 conn(0x55f02a4b9a30 msgr2=0x55f02a4bf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f411d7fa640 0 client.26965 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 shutdown_connections 2024-09-05T06:51:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 1 -- 172.21.15.86:0/2091086049 wait complete. 2024-09-05T06:51:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.333+0000 7f413e595740 20 client.26965 trim_cache size 0 max 16384 2024-09-05T06:51:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 Processor -- start 2024-09-05T06:51:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 -- start start 2024-09-05T06:51:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4c8680 0x55f02a4dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41341b1390 con 0x55f02a4bddc0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41341b2f00 con 0x55f02a4bd4a0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f413e595740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f41341b1a60 con 0x55f02a4c8680 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f4139757640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f4139757640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54196/0 (socket says 172.21.15.86:54196) 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.353+0000 7f4139757640 1 -- 172.21.15.86:0/1031920650 learned_addr learned my addr 172.21.15.86:0/1031920650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f413a759640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4c8680 0x55f02a4dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139f58640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 -- 172.21.15.86:0/1031920650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4c8680 msgr2=0x55f02a4dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4c8680 0x55f02a4dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 -- 172.21.15.86:0/1031920650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 msgr2=0x55f02a4c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 -- 172.21.15.86:0/1031920650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02a4c4260 con 0x55f02a4bddc0 2024-09-05T06:51:14.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139f58640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f4139757640 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4124000c00 tx=0x7f412400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4e6cff4673007d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f411f7fe640 1 -- 172.21.15.86:0/1031920650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240b3040 con 0x55f02a4bddc0 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f411f7fe640 1 -- 172.21.15.86:0/1031920650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f412409c1e0 con 0x55f02a4bddc0 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.354+0000 7f411f7fe640 1 -- 172.21.15.86:0/1031920650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124003930 con 0x55f02a4bddc0 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- 172.21.15.86:0/1031920650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 msgr2=0x55f02a4b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4124000c00 tx=0x7f412400efb0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- 172.21.15.86:0/1031920650 shutdown_connections 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4c8680 0x55f02a4dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4bddc0 0x55f02a4b8a80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 --2- 172.21.15.86:0/1031920650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bd4a0 0x55f02a4c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- 172.21.15.86:0/1031920650 >> 172.21.15.86:0/1031920650 conn(0x55f02a4b9a30 msgr2=0x55f02a4b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- 172.21.15.86:0/1031920650 shutdown_connections 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- 172.21.15.86:0/1031920650 wait complete. 2024-09-05T06:51:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 -- start start 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.355+0000 7f413e595740 1 Processor -- start 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4c8680 0x55f02a539e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41341b2830 con 0x55f02a4c8680 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41341b2130 con 0x55f02a4bddc0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f41341af390 con 0x55f02a4bd4a0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413e595740 10 client.0 _ll_register_callbacks cb 0x7f41300036a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37854/0 (socket says 172.21.15.86:37854) 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 -- 172.21.15.86:0/907333780 learned_addr learned my addr 172.21.15.86:0/907333780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f413a759640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4c8680 0x55f02a539e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139f58640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 -- 172.21.15.86:0/907333780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 msgr2=0x55f02a4cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 -- 172.21.15.86:0/907333780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4c8680 msgr2=0x55f02a539e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4c8680 0x55f02a539e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.356+0000 7f4139757640 1 -- 172.21.15.86:0/907333780 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f02a4c5ba0 con 0x55f02a4bddc0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f4139757640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41240984b0 tx=0x7f4124009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716ea4746825672 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240b3040 con 0x55f02a4bddc0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f41240ccf90 con 0x55f02a4bddc0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412409c1e0 con 0x55f02a4bddc0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f41240037a0 con 0x55f02a4bddc0 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 10 client.26993 fetch_fsmap learned FSMap version 33 2024-09-05T06:51:14.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f02a5428c0 con 0x55f02a4bddc0 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f412409c1e0 con 0x55f02a4bddc0 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f41240b4ce0 con 0x55f02a4bddc0 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 10 client.26993 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 10 client.26993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 20 client.26993 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a542b30 0x55f02a544f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 4 client.26993 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.357+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f02a42ad60 con 0x55f02a542b30 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f4139f58640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a542b30 0x55f02a544f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:14.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f411d7fa640 10 client.26993 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f4139f58640 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a542b30 0x55f02a544f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664a00df6fd85046 server_cookie=a7267ed6840b2626 in_seq=0 out_seq=0 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f411d7fa640 1 -- 172.21.15.86:0/907333780 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f02a42ad60 con 0x55f02a542b30 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f411d7fa640 10 client.26993 handle_command_reply: tid=0 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 client.26993 shutdown 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 msgr2=0x55f02a4ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f41240984b0 tx=0x7f4124009540 comp rx=0 tx=0).stop 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 shutdown_connections 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f02a542b30 0x55f02a544f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f02a4c8680 0x55f02a539e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f411d7fa640 0 client.26993 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f02a4bddc0 0x55f02a4ce3a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 --2- 172.21.15.86:0/907333780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f02a4bd4a0 0x55f02a4cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.358+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 >> 172.21.15.86:0/907333780 conn(0x55f02a4b9a30 msgr2=0x55f02a4b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 shutdown_connections 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413e595740 1 -- 172.21.15.86:0/907333780 wait complete. 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413e595740 20 client.26993 trim_cache size 0 max 16384 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "5d3b8c65-f660-4dfe-bd69-7e6a8378c934": { 2024-09-05T06:51:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5d3b8c65-f660-4dfe-bd69-7e6a8378c934", 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:51:14.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 msgr2=0x7f41200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 0x7f41200a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4130009bd0 tx=0x7f4130005f90 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 msgr2=0x7f4134074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f412400ea60 tx=0x7f412400ef20 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 shutdown_connections 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f41200a2d70 0x7f41200a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4134106f10 0x7f4134074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4134102790 0x7f4134074a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 --2- 172.21.15.86:0/1469330506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4134101de0 0x7f41340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 >> 172.21.15.86:0/1469330506 conn(0x7f41340fd8c0 msgr2=0x7f41340fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 shutdown_connections 2024-09-05T06:51:14.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:14.359+0000 7f413c1bc640 1 -- 172.21.15.86:0/1469330506 wait complete. 2024-09-05T06:51:14.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 178988 1.70 MB/sec execute 394 sec latency 19.482 ms 2024-09-05T06:51:14.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:14 smithi086 ceph-mon[30943]: pgmap v310: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-05T06:51:15.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:14 smithi096 ceph-mon[34468]: pgmap v310: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-05T06:51:15.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:14 smithi115 ceph-mon[34791]: pgmap v310: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-09-05T06:51:15.237 INFO:tasks.cephfs.filesystem:scrub status for tag:5d3b8c65-f660-4dfe-bd69-7e6a8378c934 - {'path': '/', 'tag': '5d3b8c65-f660-4dfe-bd69-7e6a8378c934', 'options': 'recursive,force'} 2024-09-05T06:51:15.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 179662 1.70 MB/sec execute 395 sec latency 20.113 ms 2024-09-05T06:51:16.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180279 1.71 MB/sec execute 396 sec latency 13.900 ms 2024-09-05T06:51:17.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:16 smithi096 ceph-mon[34468]: pgmap v311: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-09-05T06:51:17.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:16 smithi115 ceph-mon[34791]: pgmap v311: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-09-05T06:51:17.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:16 smithi086 ceph-mon[30943]: pgmap v311: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-09-05T06:51:17.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180876 1.72 MB/sec execute 397 sec latency 12.991 ms 2024-09-05T06:51:18.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180982 1.71 MB/sec execute 398 sec latency 788.350 ms 2024-09-05T06:51:19.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:18 smithi096 ceph-mon[34468]: pgmap v312: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:19.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:18 smithi115 ceph-mon[34791]: pgmap v312: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:19.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:18 smithi086 ceph-mon[30943]: pgmap v312: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:19.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180982 1.71 MB/sec execute 399 sec latency 1788.407 ms 2024-09-05T06:51:20.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180982 1.70 MB/sec execute 400 sec latency 2788.489 ms 2024-09-05T06:51:21.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:20 smithi096 ceph-mon[34468]: pgmap v313: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:21.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:20 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:51:21.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:20 smithi115 ceph-mon[34791]: pgmap v313: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:21.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:20 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:51:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:20 smithi086 ceph-mon[30943]: pgmap v313: 129 pgs: 129 active+clean; 192 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-09-05T06:51:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:20 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:51:21.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 180982 1.70 MB/sec execute 401 sec latency 3788.580 ms 2024-09-05T06:51:22.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:51:22.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:51:22.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:21 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:22.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:51:22.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:51:22.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:21 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:22.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:51:22.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:51:22.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:21 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:22.528 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181049 1.70 MB/sec execute 402 sec latency 4005.210 ms 2024-09-05T06:51:23.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:22 smithi096 ceph-mon[34468]: pgmap v314: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-05T06:51:23.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:22 smithi115 ceph-mon[34791]: pgmap v314: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-05T06:51:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:22 smithi086 ceph-mon[30943]: pgmap v314: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-05T06:51:23.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181049 1.69 MB/sec execute 403 sec latency 1559.438 ms 2024-09-05T06:51:24.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181049 1.69 MB/sec execute 404 sec latency 2559.572 ms 2024-09-05T06:51:25.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:24 smithi096 ceph-mon[34468]: pgmap v315: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1019 KiB/s wr, 92 op/s 2024-09-05T06:51:25.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:24 smithi115 ceph-mon[34791]: pgmap v315: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1019 KiB/s wr, 92 op/s 2024-09-05T06:51:25.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:24 smithi086 ceph-mon[30943]: pgmap v315: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1019 KiB/s wr, 92 op/s 2024-09-05T06:51:25.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181049 1.68 MB/sec execute 405 sec latency 3559.689 ms 2024-09-05T06:51:26.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181049 1.68 MB/sec execute 406 sec latency 4559.792 ms 2024-09-05T06:51:27.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:26 smithi096 ceph-mon[34468]: pgmap v316: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 977 KiB/s wr, 88 op/s 2024-09-05T06:51:27.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:26 smithi115 ceph-mon[34791]: pgmap v316: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 977 KiB/s wr, 88 op/s 2024-09-05T06:51:27.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:26 smithi086 ceph-mon[30943]: pgmap v316: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 977 KiB/s wr, 88 op/s 2024-09-05T06:51:27.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181288 1.68 MB/sec execute 407 sec latency 4770.373 ms 2024-09-05T06:51:28.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181606 1.67 MB/sec execute 408 sec latency 13.803 ms 2024-09-05T06:51:29.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:28 smithi096 ceph-mon[34468]: pgmap v317: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1023 KiB/s wr, 98 op/s 2024-09-05T06:51:29.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:28 smithi115 ceph-mon[34791]: pgmap v317: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1023 KiB/s wr, 98 op/s 2024-09-05T06:51:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:28 smithi086 ceph-mon[30943]: pgmap v317: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1023 KiB/s wr, 98 op/s 2024-09-05T06:51:29.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 182150 1.67 MB/sec execute 409 sec latency 14.034 ms 2024-09-05T06:51:30.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 182726 1.68 MB/sec execute 410 sec latency 20.271 ms 2024-09-05T06:51:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:30 smithi096 ceph-mon[34468]: pgmap v318: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 355 KiB/s wr, 33 op/s 2024-09-05T06:51:31.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:30 smithi115 ceph-mon[34791]: pgmap v318: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 355 KiB/s wr, 33 op/s 2024-09-05T06:51:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:30 smithi086 ceph-mon[30943]: pgmap v318: 129 pgs: 129 active+clean; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 355 KiB/s wr, 33 op/s 2024-09-05T06:51:31.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 183342 1.68 MB/sec execute 411 sec latency 19.340 ms 2024-09-05T06:51:32.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:32 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:32.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:32 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:32 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:51:32.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184120 1.69 MB/sec execute 412 sec latency 14.237 ms 2024-09-05T06:51:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:33 smithi096 ceph-mon[34468]: pgmap v319: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1011 KiB/s wr, 93 op/s 2024-09-05T06:51:33.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:33 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:51:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:33 smithi115 ceph-mon[34791]: pgmap v319: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1011 KiB/s wr, 93 op/s 2024-09-05T06:51:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:33 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:51:33.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:33 smithi086 ceph-mon[30943]: pgmap v319: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1011 KiB/s wr, 93 op/s 2024-09-05T06:51:33.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:33 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:51:33.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184528 1.70 MB/sec execute 413 sec latency 33.403 ms 2024-09-05T06:51:34.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184528 1.69 MB/sec execute 414 sec latency 1033.428 ms 2024-09-05T06:51:35.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184528 1.69 MB/sec execute 415 sec latency 2033.482 ms 2024-09-05T06:51:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: pgmap v320: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:51:35.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:35 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: pgmap v320: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:51:35.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:35 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: pgmap v320: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T06:51:35.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:51:35.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:51:35.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:35 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:51:36.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184528 1.68 MB/sec execute 416 sec latency 3033.514 ms 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:51:36.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:36 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:51:36.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:36 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T06:51:36.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:36 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T06:51:37.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184595 1.68 MB/sec execute 417 sec latency 3246.223 ms 2024-09-05T06:51:37.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:37 smithi115 ceph-mon[34791]: pgmap v321: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:37 smithi086 ceph-mon[30943]: pgmap v321: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:37 smithi096 ceph-mon[34468]: pgmap v321: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 703 KiB/s wr, 70 op/s 2024-09-05T06:51:38.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184595 1.68 MB/sec execute 418 sec latency 1548.148 ms 2024-09-05T06:51:39.529 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184595 1.67 MB/sec execute 419 sec latency 2548.267 ms 2024-09-05T06:51:39.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:39 smithi115 ceph-mon[34791]: pgmap v322: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T06:51:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:39 smithi086 ceph-mon[30943]: pgmap v322: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T06:51:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:39 smithi096 ceph-mon[34468]: pgmap v322: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T06:51:40.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184595 1.67 MB/sec execute 420 sec latency 3548.367 ms 2024-09-05T06:51:41.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184595 1.66 MB/sec execute 421 sec latency 4548.473 ms 2024-09-05T06:51:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:41 smithi115 ceph-mon[34791]: pgmap v323: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:51:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:41 smithi086 ceph-mon[30943]: pgmap v323: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:51:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:41 smithi096 ceph-mon[34468]: pgmap v323: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:51:42.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184831 1.66 MB/sec execute 422 sec latency 4760.971 ms 2024-09-05T06:51:43.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 185159 1.66 MB/sec execute 423 sec latency 16.247 ms 2024-09-05T06:51:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:43 smithi115 ceph-mon[34791]: pgmap v324: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:43 smithi086 ceph-mon[30943]: pgmap v324: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:43.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:43 smithi096 ceph-mon[34468]: pgmap v324: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:44.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 185708 1.66 MB/sec execute 424 sec latency 13.235 ms 2024-09-05T06:51:45.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:51:45.518 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:45.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 186282 1.66 MB/sec execute 425 sec latency 26.061 ms 2024-09-05T06:51:45.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:45 smithi086 ceph-mon[30943]: pgmap v325: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:45 smithi115 ceph-mon[34791]: pgmap v325: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:45 smithi096 ceph-mon[34468]: pgmap v325: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 -- 172.21.15.86:0/3169278006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e81036e0 msgr2=0x7f70e8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 --2- 172.21.15.86:0/3169278006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e81036e0 0x7f70e8107b90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009930 tx=0x7f70dc02a420 comp rx=0 tx=0).stop 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 -- 172.21.15.86:0/3169278006 shutdown_connections 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 --2- 172.21.15.86:0/3169278006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81082a0 0x7f70e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 --2- 172.21.15.86:0/3169278006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e81036e0 0x7f70e8107b90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 --2- 172.21.15.86:0/3169278006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e8102d30 0x7f70e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 -- 172.21.15.86:0/3169278006 >> 172.21.15.86:0/3169278006 conn(0x7f70e80fc7d0 msgr2=0x7f70e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 -- 172.21.15.86:0/3169278006 shutdown_connections 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.036+0000 7f70ece31640 1 -- 172.21.15.86:0/3169278006 wait complete. 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 Processor -- start 2024-09-05T06:51:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 -- start start 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81036e0 0x7f70e8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e81082a0 0x7f70e819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111f10 con 0x7f70e8102d30 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111d90 con 0x7f70e81036e0 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70ece31640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8112090 con 0x7f70e81082a0 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57452/0 (socket says 172.21.15.86:57452) 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 -- 172.21.15.86:0/2825295034 learned_addr learned my addr 172.21.15.86:0/2825295034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81036e0 0x7f70e8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e81082a0 0x7f70e819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 -- 172.21.15.86:0/2825295034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e81082a0 msgr2=0x7f70e819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e81082a0 0x7f70e819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 -- 172.21.15.86:0/2825295034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81036e0 msgr2=0x7f70e8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81036e0 0x7f70e8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.037+0000 7f70e659c640 1 -- 172.21.15.86:0/2825295034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819d4a0 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70e659c640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f70d800d950 tx=0x7f70d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70d77fe640 1 -- 172.21.15.86:0/2825295034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8017070 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70d77fe640 1 -- 172.21.15.86:0/2825295034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70d8009c30 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819d790 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70d77fe640 1 -- 172.21.15.86:0/2825295034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d800f620 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.038+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a5070 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.040+0000 7f70d77fe640 1 -- 172.21.15.86:0/2825295034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f70d80109a0 con 0x7f70e8102d30 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.040+0000 7f70ef20a740 1 Processor -- start 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 -- start start 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 0x555a59a8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a9ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a599f52e0 con 0x555a59a8f4a0 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555a5999cce0 con 0x555a59a9a680 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70ef20a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555a599f12a0 con 0x555a59a8fdc0 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 0x555a59a8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70d77fe640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 0x7f70d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 0x555a59a8f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57462/0 (socket says 172.21.15.86:57462) 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a9ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e6d9d640 1 -- 172.21.15.86:0/1777588978 learned_addr learned my addr 172.21.15.86:0/1777588978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 -- 172.21.15.86:0/1777588978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 msgr2=0x555a59a9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 -- 172.21.15.86:0/1777588978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 msgr2=0x555a59a8f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 0x555a59a8f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.041+0000 7f70e5d9b640 1 -- 172.21.15.86:0/1777588978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a59981440 con 0x555a59a9a680 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70d77fe640 1 -- 172.21.15.86:0/2825295034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f70d80cf3b0 con 0x7f70e8102d30 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 0x7f70d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a9ca70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a940 tx=0x7f70dc032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19af397d971232c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70d4ff9640 1 -- 172.21.15.86:0/1777588978 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc040070 con 0x555a59a9a680 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70d4ff9640 1 -- 172.21.15.86:0/1777588978 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70dc0316a0 con 0x555a59a9a680 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.042+0000 7f70d4ff9640 1 -- 172.21.15.86:0/1777588978 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc035040 con 0x555a59a9a680 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- 172.21.15.86:0/1777588978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 msgr2=0x555a59a9ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 0x7f70d00a5230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70e0010690 tx=0x7f70e0004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a9ca70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a940 tx=0x7f70dc032bd0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- 172.21.15.86:0/1777588978 shutdown_connections 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a9ca70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 --2- 172.21.15.86:0/1777588978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8f4a0 0x555a59a8f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- 172.21.15.86:0/1777588978 >> 172.21.15.86:0/1777588978 conn(0x555a59a8ba30 msgr2=0x555a59a8be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- 172.21.15.86:0/1777588978 shutdown_connections 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- 172.21.15.86:0/1777588978 wait complete. 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 -- start start 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.043+0000 7f70ef20a740 1 Processor -- start 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59a99ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a59a9d250 con 0x555a59a8fdc0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555a599b0050 con 0x555a59a9a680 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555a59a9d3d0 con 0x555a59a8f4a0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70ef20a740 10 client.0 _ll_register_callbacks cb 0x555a599f0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57468/0 (socket says 172.21.15.86:57468) 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 -- 172.21.15.86:0/2545227399 learned_addr learned my addr 172.21.15.86:0/2545227399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e659c640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59a99ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 -- 172.21.15.86:0/2545227399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 msgr2=0x555a59a99ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59a99ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 -- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 msgr2=0x555a59a97800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a97800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 -- 172.21.15.86:0/2545227399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a597663c0 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a460 tx=0x7f70dc032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6deb1dab17c88b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc04b070 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.044+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f70dc005410 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc0316a0 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f70dc057050 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 10 client.17871 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555a59aa8c20 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70dc0316a0 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f70dc003b20 con 0x555a59a8fdc0 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 10 client.17871 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:46.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 20 client.17871 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59aac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 4 client.17871 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.045+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a59a02850 con 0x555a59aa9f70 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.046+0000 7f70e659c640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59aac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.046+0000 7f70cdffb640 10 client.17871 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.046+0000 7f70e659c640 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59aac360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ae57eea7673bf0 server_cookie=a95a8f16cb689ae8 in_seq=0 out_seq=0 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70cdffb640 1 -- 172.21.15.86:0/2545227399 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a59a02850 con 0x555a59aa9f70 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70cdffb640 10 client.17871 handle_command_reply: tid=0 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 client.17871 shutdown 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 msgr2=0x555a59a972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a460 tx=0x7f70dc032e80 comp rx=0 tx=0).stop 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 shutdown_connections 2024-09-05T06:51:46.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59aac360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59a97800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70cdffb640 0 client.17871 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59a972a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 --2- 172.21.15.86:0/2545227399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59a99ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 >> 172.21.15.86:0/2545227399 conn(0x555a59a8ba30 msgr2=0x555a59a91760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 shutdown_connections 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 1 -- 172.21.15.86:0/2545227399 wait complete. 2024-09-05T06:51:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.047+0000 7f70ef20a740 20 client.17871 trim_cache size 0 max 16384 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.070+0000 7f70ef20a740 1 Processor -- start 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.070+0000 7f70ef20a740 1 -- start start 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59aa64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59ab1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81af0f0 con 0x555a59a8fdc0 2024-09-05T06:51:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81b0cc0 con 0x555a59a9a680 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70ef20a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81af7c0 con 0x555a59a8f4a0 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57476/0 (socket says 172.21.15.86:57476) 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e5d9b640 1 -- 172.21.15.86:0/3418330405 learned_addr learned my addr 172.21.15.86:0/3418330405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59ab1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e659c640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59aa64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 -- 172.21.15.86:0/3418330405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 msgr2=0x555a59aa64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59aa64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 -- 172.21.15.86:0/3418330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 msgr2=0x555a59ab1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e6d9d640 1 -- 172.21.15.86:0/3418330405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a59a957f0 con 0x555a59a9a680 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e659c640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59aa64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.071+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.072+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59ab1e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009fd0 tx=0x7f70e0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36eab427fa34df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.072+0000 7f70cffff640 1 -- 172.21.15.86:0/3418330405 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0003b70 con 0x555a59a9a680 2024-09-05T06:51:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.072+0000 7f70cffff640 1 -- 172.21.15.86:0/3418330405 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e0021de0 con 0x555a59a9a680 2024-09-05T06:51:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.072+0000 7f70cffff640 1 -- 172.21.15.86:0/3418330405 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00047a0 con 0x555a59a9a680 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- 172.21.15.86:0/3418330405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 msgr2=0x555a59ab1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59ab1e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009fd0 tx=0x7f70e0019000 comp rx=0 tx=0).stop 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- 172.21.15.86:0/3418330405 shutdown_connections 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a9a680 0x555a59ab1e70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a8fdc0 0x555a59ab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3418330405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8f4a0 0x555a59aa64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- 172.21.15.86:0/3418330405 >> 172.21.15.86:0/3418330405 conn(0x555a59a8ba30 msgr2=0x555a59a96390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- 172.21.15.86:0/3418330405 shutdown_connections 2024-09-05T06:51:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- 172.21.15.86:0/3418330405 wait complete. 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- start start 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 Processor -- start 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 0x555a59a9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 0x555a59a9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.073+0000 7f70ef20a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81b0590 con 0x555a59a9a680 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70ef20a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81afe90 con 0x555a59a8f4a0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70ef20a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81ad0f0 con 0x555a59a8fdc0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70ef20a740 10 client.0 _ll_register_callbacks cb 0x7f70e0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 0x555a59a9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 0x555a59a9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:34934/0 (socket says 172.21.15.86:34934) 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 -- 172.21.15.86:0/3805958336 learned_addr learned my addr 172.21.15.86:0/3805958336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 0x555a59a9e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:56606/0 (socket says 172.21.15.86:56606) 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 -- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 msgr2=0x555a59a9e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 0x555a59a9e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 -- 172.21.15.86:0/3805958336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 msgr2=0x555a59a9ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 0x555a59a9ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 -- 172.21.15.86:0/3805958336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a59b13a60 con 0x555a59a8fdc0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e6d9d640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 0x555a59a9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70e5d9b640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a940 tx=0x7f70dc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4579679d3546162 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc04b070 con 0x555a59a8fdc0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e0021de0 con 0x555a59a8fdc0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.074+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f70dc031af0 con 0x555a59a8fdc0 2024-09-05T06:51:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f70dc035070 con 0x555a59a8fdc0 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 10 client.26989 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555a59b14d10 con 0x555a59a8fdc0 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e0021de0 con 0x555a59a8fdc0 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f70dc003fc0 con 0x555a59a8fdc0 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 10 client.26989 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 20 client.26989 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59b17350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 4 client.26989 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a599fcd60 con 0x555a59aa9f70 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.075+0000 7f70e659c640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59b17350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70e659c640 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59b17350 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d900f613b836ccdd server_cookie=61d18af180ad6ae4 in_seq=0 out_seq=0 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70cdffb640 10 client.26989 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70cdffb640 1 -- 172.21.15.86:0/3805958336 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a599fcd60 con 0x555a59aa9f70 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70cdffb640 10 client.26989 handle_command_reply: tid=0 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 client.26989 shutdown 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 msgr2=0x555a59a9e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70dc02a940 tx=0x7f70dc004770 comp rx=0 tx=0).stop 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 shutdown_connections 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555a59aa9f70 0x555a59b17350 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a59a9a680 0x555a59a9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a59a8fdc0 0x555a59a9e9f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 --2- 172.21.15.86:0/3805958336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555a59a8f4a0 0x555a59a9e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70cdffb640 0 client.26989 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.076+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 >> 172.21.15.86:0/3805958336 conn(0x555a59a8ba30 msgr2=0x555a59a98b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 shutdown_connections 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ef20a740 1 -- 172.21.15.86:0/3805958336 wait complete. 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ef20a740 20 client.26989 trim_cache size 0 max 16384 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:51:46.082 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 msgr2=0x7f70d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 0x7f70d00a5230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70e0010690 tx=0x7f70e0004ab0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 msgr2=0x7f70e81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f70d800d950 tx=0x7f70d800de10 comp rx=0 tx=0).stop 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 shutdown_connections 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f70d00a2d70 0x7f70d00a5230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e81082a0 0x7f70e819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70e81036e0 0x7f70e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 --2- 172.21.15.86:0/2825295034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e8102d30 0x7f70e81984f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.077+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 >> 172.21.15.86:0/2825295034 conn(0x7f70e80fc7d0 msgr2=0x7f70e8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.078+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 shutdown_connections 2024-09-05T06:51:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:46.078+0000 7f70ece31640 1 -- 172.21.15.86:0/2825295034 wait complete. 2024-09-05T06:51:46.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 186885 1.66 MB/sec execute 426 sec latency 18.829 ms 2024-09-05T06:51:46.902 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:51:46.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:51:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:47.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 187719 1.68 MB/sec execute 427 sec latency 13.619 ms 2024-09-05T06:51:47.597 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:47 smithi086 ceph-mon[30943]: pgmap v326: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:47 smithi115 ceph-mon[34791]: pgmap v326: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 -- 172.21.15.86:0/5838218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81036e0 msgr2=0x7fafc8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 --2- 172.21.15.86:0/5838218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81036e0 0x7fafc8107b90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009930 tx=0x7fafbc02a420 comp rx=0 tx=0).stop 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 -- 172.21.15.86:0/5838218 shutdown_connections 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 --2- 172.21.15.86:0/5838218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc81082a0 0x7fafc810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 --2- 172.21.15.86:0/5838218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81036e0 0x7fafc8107b90 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 --2- 172.21.15.86:0/5838218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc8102d30 0x7fafc8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.687+0000 7fafd0219640 1 -- 172.21.15.86:0/5838218 >> 172.21.15.86:0/5838218 conn(0x7fafc80fc7d0 msgr2=0x7fafc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 -- 172.21.15.86:0/5838218 shutdown_connections 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 -- 172.21.15.86:0/5838218 wait complete. 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 Processor -- start 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 -- start start 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 0x7fafc8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.688+0000 7fafd0219640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafd0219640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8111f10 con 0x7fafc81082a0 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafd0219640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8111d90 con 0x7fafc8102d30 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafd0219640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8112090 con 0x7fafc81036e0 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafce7b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcdfb5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcdfb5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:56638/0 (socket says 172.21.15.86:56638) 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafce7b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57506/0 (socket says 172.21.15.86:57506) 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcdfb5640 1 -- 172.21.15.86:0/3199672095 learned_addr learned my addr 172.21.15.86:0/3199672095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 0x7fafc8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 -- 172.21.15.86:0/3199672095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 msgr2=0x7fafc8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 -- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 msgr2=0x7fafc8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 -- 172.21.15.86:0/3199672095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc81a69b0 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcdfb5640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafce7b6640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafcd7b4640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 0x7fafc8100ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02a940 tx=0x7fafbc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03f070 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.689+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc81a6c40 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.690+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafbc003aa0 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.690+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc035070 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.690+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81a7140 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.691+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc8103110 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.692+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fafbc003c50 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.692+0000 7fafbaffd640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 0x7fafb40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.692+0000 7fafcdfb5640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 0x7fafb40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.693+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7fafbc0ef920 con 0x7fafc81036e0 2024-09-05T06:51:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.693+0000 7fafcdfb5640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 0x7fafb40a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafc4011550 tx=0x7fafc4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.697+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafbc0bba50 con 0x7fafc81036e0 2024-09-05T06:51:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:47 smithi096 ceph-mon[34468]: pgmap v326: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 471 KiB/s wr, 41 op/s 2024-09-05T06:51:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.915+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc810a2b0 con 0x7fafc81036e0 2024-09-05T06:51:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.916+0000 7fafbaffd640 1 -- 172.21.15.86:0/3199672095 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11978 (secure 0 0 0) 0x7fafbc0bfd50 con 0x7fafc81036e0 2024-09-05T06:51:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:51:47.916 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"btime":"2024-09-05T06:51:30:994870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:51:30.398269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:51:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-09-05T06:51:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 msgr2=0x7fafb40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 0x7fafb40a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafc4011550 tx=0x7fafc4005f90 comp rx=0 tx=0).stop 2024-09-05T06:51:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 msgr2=0x7fafc8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 0x7fafc8100ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02a940 tx=0x7fafbc004270 comp rx=0 tx=0).stop 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 shutdown_connections 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fafb40a2d70 0x7fafb40a5230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc81082a0 0x7fafc8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafc81036e0 0x7fafc8100ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 --2- 172.21.15.86:0/3199672095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafc8102d30 0x7fafc8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 >> 172.21.15.86:0/3199672095 conn(0x7fafc80fc7d0 msgr2=0x7fafc8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 shutdown_connections 2024-09-05T06:51:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:47.918+0000 7fafd0219640 1 -- 172.21.15.86:0/3199672095 wait complete. 2024-09-05T06:51:48.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188073 1.68 MB/sec execute 428 sec latency 158.304 ms 2024-09-05T06:51:48.611 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3199672095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:48.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:51:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3199672095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:48.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3199672095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:51:48.881 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:49.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 -- 172.21.15.86:0/2710809067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102ef0 msgr2=0x7f821c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 --2- 172.21.15.86:0/2710809067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102ef0 0x7f821c10da40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8214009ca0 tx=0x7f821402a950 comp rx=0 tx=0).stop 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 -- 172.21.15.86:0/2710809067 shutdown_connections 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 --2- 172.21.15.86:0/2710809067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102ef0 0x7f821c10da40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 --2- 172.21.15.86:0/2710809067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102570 0x7f821c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 --2- 172.21.15.86:0/2710809067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 0x7f821c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 -- 172.21.15.86:0/2710809067 >> 172.21.15.86:0/2710809067 conn(0x7f821c0fc7d0 msgr2=0x7f821c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 -- 172.21.15.86:0/2710809067 shutdown_connections 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.383+0000 7f8223b96640 1 -- 172.21.15.86:0/2710809067 wait complete. 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 Processor -- start 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 -- start start 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 0x7f821c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102ef0 0x7f821c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f821c110e30 con 0x7f821c102570 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f821c110cb0 con 0x7f821c100e20 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8223b96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f821c110fb0 con 0x7f821c102ef0 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8221131640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8221131640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57532/0 (socket says 172.21.15.86:57532) 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8221131640 1 -- 172.21.15.86:0/312829498 learned_addr learned my addr 172.21.15.86:0/312829498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8221932640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 0x7f821c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.384+0000 7f8222133640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102ef0 0x7f821c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 -- 172.21.15.86:0/312829498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102ef0 msgr2=0x7f821c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102ef0 0x7f821c19bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 -- 172.21.15.86:0/312829498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 msgr2=0x7f821c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 0x7f821c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 -- 172.21.15.86:0/312829498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c19c300 con 0x7f821c102570 2024-09-05T06:51:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8221131640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f820c009860 tx=0x7f820c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8212ffd640 1 -- 172.21.15.86:0/312829498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c005020 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8212ffd640 1 -- 172.21.15.86:0/312829498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f820c0047a0 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c19c5f0 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8212ffd640 1 -- 172.21.15.86:0/312829498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c007b00 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.385+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c069f70 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8212ffd640 1 -- 172.21.15.86:0/312829498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f820c007ca0 con 0x7f821c102570 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 Processor -- start 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 -- start start 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d51ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c4d4822f0 con 0x561c4d51db40 2024-09-05T06:51:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c4d429cf0 con 0x561c4d52b780 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8225f6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c4d47e2b0 con 0x561c4d51c5d0 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57540/0 (socket says 172.21.15.86:57540) 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.388+0000 7f8222133640 1 -- 172.21.15.86:0/2352601947 learned_addr learned my addr 172.21.15.86:0/2352601947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8221932640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d51ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8221131640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8212ffd640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 0x7f82080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 -- 172.21.15.86:0/2352601947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 msgr2=0x561c4d51ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d51ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 -- 172.21.15.86:0/2352601947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 msgr2=0x561c4d52db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 -- 172.21.15.86:0/2352601947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d40e3a0 con 0x561c4d51db40 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8221131640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8212ffd640 1 -- 172.21.15.86:0/312829498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f820c0d0c60 con 0x7f821c102570 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8214035000 tx=0x7f8214005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5526ba0d710e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8222133640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 0x7f82080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8207fff640 1 -- 172.21.15.86:0/2352601947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214048070 con 0x561c4d51db40 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8207fff640 1 -- 172.21.15.86:0/2352601947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214003970 con 0x561c4d51db40 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.389+0000 7f8207fff640 1 -- 172.21.15.86:0/2352601947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214034040 con 0x561c4d51db40 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- 172.21.15.86:0/2352601947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 msgr2=0x561c4d51c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8222133640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 0x7f82080a51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8214045000 tx=0x7f82140047c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8214035000 tx=0x7f8214005fa0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- 172.21.15.86:0/2352601947 shutdown_connections 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d51ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 --2- 172.21.15.86:0/2352601947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51db40 0x561c4d51c090 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- 172.21.15.86:0/2352601947 >> 172.21.15.86:0/2352601947 conn(0x561c4d4e5310 msgr2=0x561c4d518c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- 172.21.15.86:0/2352601947 shutdown_connections 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- 172.21.15.86:0/2352601947 wait complete. 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 -- start start 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.390+0000 7f8225f6f740 1 Processor -- start 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d52b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d528440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c4d52e420 con 0x561c4d52b780 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c4d43d060 con 0x561c4d51db40 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561c4d52e5a0 con 0x561c4d51c5d0 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8225f6f740 10 client.0 _ll_register_callbacks cb 0x561c4d1908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57546/0 (socket says 172.21.15.86:57546) 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 -- 172.21.15.86:0/3968418657 learned_addr learned my addr 172.21.15.86:0/3968418657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8221131640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d528440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8221932640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d52b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 -- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 msgr2=0x561c4d52b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d52b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 -- 172.21.15.86:0/3968418657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 msgr2=0x561c4d528440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d528440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8222133640 1 -- 172.21.15.86:0/3968418657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4d1f33c0 con 0x561c4d52b780 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.391+0000 7f8221131640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d528440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8222133640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f821402ae70 tx=0x7f8214046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfe638bd87ad17b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821405e060 con 0x561c4d52b780 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8214003970 con 0x561c4d52b780 2024-09-05T06:51:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f8214030940 con 0x561c4d52b780 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8214065050 con 0x561c4d52b780 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 10 client.17895 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561c4d539ef0 con 0x561c4d52b780 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214003970 con 0x561c4d52b780 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f821403a430 con 0x561c4d52b780 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 10 client.17895 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 10 client.17895 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 20 client.17895 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d53b240 0x561c4d53d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 4 client.17895 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.392+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c4d48f860 con 0x561c4d53b240 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.393+0000 7f8221932640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d53b240 0x561c4d53d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.393+0000 7f82057fa640 10 client.17895 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.393+0000 7f8221932640 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d53b240 0x561c4d53d630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f858162c5cd2ff3d server_cookie=a004dcb5be5b5e4b in_seq=0 out_seq=0 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f82057fa640 1 -- 172.21.15.86:0/3968418657 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c4d48f860 con 0x561c4d53b240 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f82057fa640 10 client.17895 handle_command_reply: tid=0 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 client.17895 shutdown 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 msgr2=0x561c4d528a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f821402ae70 tx=0x7f8214046000 comp rx=0 tx=0).stop 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 shutdown_connections 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d53b240 0x561c4d53d630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d528a50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d528440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3968418657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d52b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f82057fa640 0 client.17895 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 >> 172.21.15.86:0/3968418657 conn(0x561c4d4e5310 msgr2=0x561c4d51e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 shutdown_connections 2024-09-05T06:51:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 1 -- 172.21.15.86:0/3968418657 wait complete. 2024-09-05T06:51:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.394+0000 7f8225f6f740 20 client.17895 trim_cache size 0 max 16384 2024-09-05T06:51:49.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 Processor -- start 2024-09-05T06:51:49.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 -- start start 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d5376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d527420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1ace40 con 0x561c4d52b780 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1aea10 con 0x561c4d51db40 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8225f6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1ad510 con 0x561c4d51c5d0 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8222133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57550/0 (socket says 172.21.15.86:57550) 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.407+0000 7f8222133640 1 -- 172.21.15.86:0/3565789416 learned_addr learned my addr 172.21.15.86:0/3565789416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8221932640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d5376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8221131640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d527420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 -- 172.21.15.86:0/3565789416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 msgr2=0x561c4d5376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d5376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 -- 172.21.15.86:0/3565789416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 msgr2=0x561c4d527420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d527420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 -- 172.21.15.86:0/3565789416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d5293e0 con 0x561c4d52b780 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8221131640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d527420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f8222133640 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8214009430 tx=0x7f821402fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d614be2680570 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f82077fe640 1 -- 172.21.15.86:0/3565789416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821406e070 con 0x561c4d52b780 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f82077fe640 1 -- 172.21.15.86:0/3565789416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214009800 con 0x561c4d52b780 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.408+0000 7f82077fe640 1 -- 172.21.15.86:0/3565789416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821405e070 con 0x561c4d52b780 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- 172.21.15.86:0/3565789416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 msgr2=0x561c4d542bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8214009430 tx=0x7f821402fe10 comp rx=0 tx=0).stop 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- 172.21.15.86:0/3565789416 shutdown_connections 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d52b780 0x561c4d542bd0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d51db40 0x561c4d527420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3565789416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51c5d0 0x561c4d5376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- 172.21.15.86:0/3565789416 >> 172.21.15.86:0/3565789416 conn(0x561c4d4e5310 msgr2=0x561c4d526d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- 172.21.15.86:0/3565789416 shutdown_connections 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- 172.21.15.86:0/3565789416 wait complete. 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 -- start start 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 Processor -- start 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51db40 0x561c4d52f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.409+0000 7f8225f6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8225f6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1ae2e0 con 0x561c4d51c5d0 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8225f6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1adbe0 con 0x561c4d52b780 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8225f6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1aae40 con 0x561c4d51db40 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8225f6f740 10 client.0 _ll_register_callbacks cb 0x7f8214046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57558/0 (socket says 172.21.15.86:57558) 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 -- 172.21.15.86:0/3197722352 learned_addr learned my addr 172.21.15.86:0/3197722352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221131640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51db40 0x561c4d52f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8222133640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 -- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51db40 msgr2=0x561c4d52f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51db40 0x561c4d52f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 -- 172.21.15.86:0/3197722352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 msgr2=0x561c4d52f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 -- 172.21.15.86:0/3197722352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4d52fd30 con 0x561c4d51c5d0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8222133640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8221932640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x561c4d529970 tx=0x7f82180108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7149feea1cd72a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f82057fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218022070 con 0x561c4d51c5d0 2024-09-05T06:51:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f8218004230 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.410+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8214009800 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8218004430 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 10 client.17901 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561c4d5a5a00 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214009800 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f8218003c00 con 0x561c4d51c5d0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 10 client.17901 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 10 client.17901 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 20 client.17901 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d5a6150 0x561c4d5a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 4 client.17901 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c4d489d70 con 0x561c4d5a6150 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.411+0000 7f8221131640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d5a6150 0x561c4d5a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8221131640 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d5a6150 0x561c4d5a8570 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=822c4571485736bb server_cookie=f9ef7fcb4da3e5d5 in_seq=0 out_seq=0 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f82057fa640 10 client.17901 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f82057fa640 1 -- 172.21.15.86:0/3197722352 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c4d489d70 con 0x561c4d5a6150 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f82057fa640 10 client.17901 handle_command_reply: tid=0 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 client.17901 shutdown 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 msgr2=0x561c4d533e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x561c4d529970 tx=0x7f82180108c0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 shutdown_connections 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x561c4d5a6150 0x561c4d5a8570 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c4d52b780 0x561c4d52f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561c4d51db40 0x561c4d52f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 --2- 172.21.15.86:0/3197722352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c4d51c5d0 0x561c4d533e20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 >> 172.21.15.86:0/3197722352 conn(0x561c4d4e5310 msgr2=0x561c4d527530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f82057fa640 0 client.17901 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:51:49.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 shutdown_connections 2024-09-05T06:51:49.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 1 -- 172.21.15.86:0/3197722352 wait complete. 2024-09-05T06:51:49.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.412+0000 7f8225f6f740 20 client.17901 trim_cache size 0 max 16384 2024-09-05T06:51:49.420 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 msgr2=0x7f82080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 0x7f82080a51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8214045000 tx=0x7f82140047c0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 msgr2=0x7f821c197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f820c009860 tx=0x7f820c009d20 comp rx=0 tx=0).stop 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 shutdown_connections 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82080a2cf0 0x7f82080a51b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c102ef0 0x7f821c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821c102570 0x7f821c197890 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 --2- 172.21.15.86:0/312829498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f821c100e20 0x7f821c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:49.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 >> 172.21.15.86:0/312829498 conn(0x7f821c0fc7d0 msgr2=0x7f821c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:49.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 shutdown_connections 2024-09-05T06:51:49.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:49.413+0000 7f8223b96640 1 -- 172.21.15.86:0/312829498 wait complete. 2024-09-05T06:51:49.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188073 1.68 MB/sec execute 429 sec latency 1158.340 ms 2024-09-05T06:51:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:49 smithi115 ceph-mon[34791]: pgmap v327: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:49 smithi086 ceph-mon[30943]: pgmap v327: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:49.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:49 smithi096 ceph-mon[34468]: pgmap v327: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-09-05T06:51:50.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:51:50.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188073 1.67 MB/sec execute 430 sec latency 2158.380 ms 2024-09-05T06:51:50.550 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 -- 172.21.15.86:0/380648926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101440 msgr2=0x7fd598101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 --2- 172.21.15.86:0/380648926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101440 0x7fd598101820 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd588009ca0 tx=0x7fd58802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 -- 172.21.15.86:0/380648926 shutdown_connections 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 --2- 172.21.15.86:0/380648926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 0x7fd59810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 --2- 172.21.15.86:0/380648926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101df0 0x7fd598105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 --2- 172.21.15.86:0/380648926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101440 0x7fd598101820 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.015+0000 7fd59de15640 1 -- 172.21.15.86:0/380648926 >> 172.21.15.86:0/380648926 conn(0x7fd598077210 msgr2=0x7fd5980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.016+0000 7fd59de15640 1 -- 172.21.15.86:0/380648926 shutdown_connections 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.016+0000 7fd59de15640 1 -- 172.21.15.86:0/380648926 wait complete. 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.016+0000 7fd59de15640 1 Processor -- start 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.016+0000 7fd59de15640 1 -- start start 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 0x7fd598197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 0x7fd59819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd598110f20 con 0x7fd598101df0 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd598110da0 con 0x7fd598101440 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981110a0 con 0x7fd598106050 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57572/0 (socket says 172.21.15.86:57572) 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 -- 172.21.15.86:0/421254688 learned_addr learned my addr 172.21.15.86:0/421254688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd597fff640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 0x7fd59819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd5977fe640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 0x7fd598197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 -- 172.21.15.86:0/421254688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 msgr2=0x7fd59819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 0x7fd59819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 -- 172.21.15.86:0/421254688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 msgr2=0x7fd598197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 0x7fd598197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 -- 172.21.15.86:0/421254688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59819c430 con 0x7fd598101df0 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd5977fe640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 0x7fd598197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd596ffd640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c890 tx=0x7fd58c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd594ff9640 1 -- 172.21.15.86:0/421254688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004220 con 0x7fd598101df0 2024-09-05T06:51:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.017+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59819c720 con 0x7fd598101df0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.018+0000 7fd594ff9640 1 -- 172.21.15.86:0/421254688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd58c0043c0 con 0x7fd598101df0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.018+0000 7fd594ff9640 1 -- 172.21.15.86:0/421254688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004a00 con 0x7fd598101df0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.018+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5981a4000 con 0x7fd598101df0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.020+0000 7fd594ff9640 1 -- 172.21.15.86:0/421254688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd58c004ba0 con 0x7fd598101df0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.020+0000 7fd5a01ee740 1 Processor -- start 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 -- start start 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b701534b40 0x55b701533090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b701542970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b7014992f0 con 0x55b7015335d0 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b701440cf0 con 0x55b701540580 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b7014952b0 con 0x55b701534b40 2024-09-05T06:51:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57574/0 (socket says 172.21.15.86:57574) 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 -- 172.21.15.86:0/211023646 learned_addr learned my addr 172.21.15.86:0/211023646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd596ffd640 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b701542970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd597fff640 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b701534b40 0x55b701533090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 -- 172.21.15.86:0/211023646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b701534b40 msgr2=0x55b701533090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b701534b40 0x55b701533090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 -- 172.21.15.86:0/211023646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 msgr2=0x55b701542970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b701542970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 -- 172.21.15.86:0/211023646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7014253a0 con 0x55b7015335d0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd594ff9640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 0x7fd5800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd597fff640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 0x7fd5800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.021+0000 7fd5977fe640 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd588035000 tx=0x7fd588031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36134289fc9e94a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd594ff9640 1 -- 172.21.15.86:0/421254688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7fd58c0cf550 con 0x7fd598101df0 2024-09-05T06:51:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd57dffb640 1 -- 172.21.15.86:0/211023646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588031650 con 0x55b7015335d0 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd57dffb640 1 -- 172.21.15.86:0/211023646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd588036040 con 0x55b7015335d0 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd57dffb640 1 -- 172.21.15.86:0/211023646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588004680 con 0x55b7015335d0 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 -- 172.21.15.86:0/211023646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 msgr2=0x55b701533a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd597fff640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 0x7fd5800a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd59000b6c0 tx=0x7fd59000b2e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd588035000 tx=0x7fd588031c00 comp rx=0 tx=0).stop 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 -- 172.21.15.86:0/211023646 shutdown_connections 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b701542970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b7015335d0 0x55b701533a10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/211023646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b701534b40 0x55b701533090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.022+0000 7fd5a01ee740 1 -- 172.21.15.86:0/211023646 >> 172.21.15.86:0/211023646 conn(0x55b7014fc310 msgr2=0x55b70152fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- 172.21.15.86:0/211023646 shutdown_connections 2024-09-05T06:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- 172.21.15.86:0/211023646 wait complete. 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- start start 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 Processor -- start 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70153fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b701543220 con 0x55b701534b40 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b701454060 con 0x55b701540580 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b7015433a0 con 0x55b7015335d0 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd5a01ee740 10 client.0 _ll_register_callbacks cb 0x55b7011a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57578/0 (socket says 172.21.15.86:57578) 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.023+0000 7fd596ffd640 1 -- 172.21.15.86:0/3157976532 learned_addr learned my addr 172.21.15.86:0/3157976532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd5977fe640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70153fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 -- 172.21.15.86:0/3157976532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 msgr2=0x55b70153fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70153fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 -- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 msgr2=0x55b70153d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 -- 172.21.15.86:0/3157976532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70120a3c0 con 0x55b701534b40 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd596ffd640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd58c09b330 tx=0x7fd58c0053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08a751c01f68415 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c01d040 con 0x55b701534b40 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7fd58c097ca0 con 0x55b701534b40 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd588036040 con 0x55b701534b40 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd58c0978d0 con 0x55b701534b40 2024-09-05T06:51:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd5a01ee740 10 client.17913 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b70154ecd0 con 0x55b701534b40 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.024+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd588036040 con 0x55b701534b40 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd58c0cf130 con 0x55b701534b40 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 10 client.17913 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 10 client.17913 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 20 client.17913 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b701552580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 4 client.17913 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7014a6860 con 0x55b701550190 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5977fe640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b701552580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd576ffd640 10 client.17913 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.025+0000 7fd5977fe640 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b701552580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b17f9c481bdec812 server_cookie=5831c694f7030722 in_seq=0 out_seq=0 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.026+0000 7fd576ffd640 1 -- 172.21.15.86:0/3157976532 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7014a6860 con 0x55b701550190 2024-09-05T06:51:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.026+0000 7fd576ffd640 10 client.17913 handle_command_reply: tid=0 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.026+0000 7fd5a01ee740 1 client.17913 shutdown 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.026+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 msgr2=0x55b70153d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.026+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd58c09b330 tx=0x7fd58c0053f0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 shutdown_connections 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b701552580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd576ffd640 0 client.17913 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b70153d1e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/3157976532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70153fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 >> 172.21.15.86:0/3157976532 conn(0x55b7014fc310 msgr2=0x55b701535960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 shutdown_connections 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 1 -- 172.21.15.86:0/3157976532 wait complete. 2024-09-05T06:51:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.027+0000 7fd5a01ee740 20 client.17913 trim_cache size 0 max 16384 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 Processor -- start 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 -- start start 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70152fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701534b40 0x55b70154c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981acf90 con 0x55b701540580 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981aeb60 con 0x55b701534b40 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.039+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981ad660 con 0x55b7015335d0 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57584/0 (socket says 172.21.15.86:57584) 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 -- 172.21.15.86:0/4092380567 learned_addr learned my addr 172.21.15.86:0/4092380567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:51.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd5977fe640 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70152fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 -- 172.21.15.86:0/4092380567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 msgr2=0x55b70152fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70152fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 -- 172.21.15.86:0/4092380567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701534b40 msgr2=0x55b70154c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701534b40 0x55b70154c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 -- 172.21.15.86:0/4092380567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70153fae0 con 0x55b701540580 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd597fff640 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd590003c10 tx=0x7fd5900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf93459ddd355f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd57d7fa640 1 -- 172.21.15.86:0/4092380567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59001fa80 con 0x55b701540580 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd57d7fa640 1 -- 172.21.15.86:0/4092380567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd590004c50 con 0x55b701540580 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.040+0000 7fd57d7fa640 1 -- 172.21.15.86:0/4092380567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900053b0 con 0x55b701540580 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- 172.21.15.86:0/4092380567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 msgr2=0x55b70153caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd590003c10 tx=0x7fd5900046c0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- 172.21.15.86:0/4092380567 shutdown_connections 2024-09-05T06:51:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701540580 0x55b70153caa0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701534b40 0x55b70154c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/4092380567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b70152fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- 172.21.15.86:0/4092380567 >> 172.21.15.86:0/4092380567 conn(0x55b7014fc310 msgr2=0x55b70152f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- 172.21.15.86:0/4092380567 shutdown_connections 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- 172.21.15.86:0/4092380567 wait complete. 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.041+0000 7fd5a01ee740 1 -- start start 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 Processor -- start 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b701548d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981ae430 con 0x55b701534b40 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981add30 con 0x55b701540580 2024-09-05T06:51:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981aaf90 con 0x55b7015335d0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5a01ee740 10 client.0 _ll_register_callbacks cb 0x7fd58c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57594/0 (socket says 172.21.15.86:57594) 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 -- 172.21.15.86:0/1729720556 learned_addr learned my addr 172.21.15.86:0/1729720556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd597fff640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd5977fe640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b701548d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 -- 172.21.15.86:0/1729720556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 msgr2=0x55b701548d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b701548d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 -- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 msgr2=0x55b70153a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 -- 172.21.15.86:0/1729720556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70153acc0 con 0x55b701534b40 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd596ffd640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0a43d0 tx=0x7fd58c0a42a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67952fae488784ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.042+0000 7fd576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c01d070 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7fd58c0afb50 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd590004c50 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd58c09af90 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 10 client.17919 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b7015baeb0 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd590004c50 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd58c097380 con 0x55b701534b40 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 10 client.17919 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 10 client.17919 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 20 client.17919 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b7015bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 4 client.17919 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.043+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7014a0d70 con 0x55b701550190 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5977fe640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b7015bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd576ffd640 10 client.17919 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5977fe640 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b7015bd440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32a0f5c6804c561c server_cookie=c33ad3278913644 in_seq=0 out_seq=0 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd576ffd640 1 -- 172.21.15.86:0/1729720556 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7014a0d70 con 0x55b701550190 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd576ffd640 10 client.17919 handle_command_reply: tid=0 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 client.17919 shutdown 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 msgr2=0x55b701539f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0a43d0 tx=0x7fd58c0a42a0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 shutdown_connections 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55b701550190 0x55b7015bd440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b701540580 0x55b70153a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd576ffd640 0 client.17919 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b701534b40 0x55b701539f30 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 --2- 172.21.15.86:0/1729720556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7015335d0 0x55b701548d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.044+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 >> 172.21.15.86:0/1729720556 conn(0x55b7014fc310 msgr2=0x55b70153f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 shutdown_connections 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd5a01ee740 1 -- 172.21.15.86:0/1729720556 wait complete. 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd5a01ee740 20 client.17919 trim_cache size 0 max 16384 2024-09-05T06:51:51.062 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 msgr2=0x7fd5800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 0x7fd5800a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd59000b6c0 tx=0x7fd59000b2e0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 msgr2=0x7fd5981979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c890 tx=0x7fd58c00cd50 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 shutdown_connections 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5800a2cf0 0x7fd5800a51b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd598106050 0x7fd59819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd598101df0 0x7fd5981979c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.045+0000 7fd59de15640 1 --2- 172.21.15.86:0/421254688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd598101440 0x7fd598197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.046+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 >> 172.21.15.86:0/421254688 conn(0x7fd598077210 msgr2=0x7fd5981043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.046+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 shutdown_connections 2024-09-05T06:51:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:51.046+0000 7fd59de15640 1 -- 172.21.15.86:0/421254688 wait complete. 2024-09-05T06:51:51.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188073 1.67 MB/sec execute 431 sec latency 3158.463 ms 2024-09-05T06:51:51.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:51 smithi115 ceph-mon[34791]: pgmap v328: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 693 KiB/s wr, 70 op/s 2024-09-05T06:51:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:51 smithi086 ceph-mon[30943]: pgmap v328: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 693 KiB/s wr, 70 op/s 2024-09-05T06:51:51.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:51:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:51 smithi096 ceph-mon[34468]: pgmap v328: 129 pgs: 129 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 693 KiB/s wr, 70 op/s 2024-09-05T06:51:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:52.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- 172.21.15.86:0/3907806819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 msgr2=0x7f7ca8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 --2- 172.21.15.86:0/3907806819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8101200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009900 tx=0x7f7ca402a480 comp rx=0 tx=0).stop 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- 172.21.15.86:0/3907806819 shutdown_connections 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 --2- 172.21.15.86:0/3907806819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 0x7f7ca810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 --2- 172.21.15.86:0/3907806819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 --2- 172.21.15.86:0/3907806819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8101200 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- 172.21.15.86:0/3907806819 >> 172.21.15.86:0/3907806819 conn(0x7f7ca80fc7d0 msgr2=0x7f7ca80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- 172.21.15.86:0/3907806819 shutdown_connections 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- 172.21.15.86:0/3907806819 wait complete. 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 Processor -- start 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.473+0000 7f7cafabc640 1 -- start start 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 0x7f7ca819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8110e80 con 0x7f7ca8100e20 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8110d00 con 0x7f7ca8102570 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8111000 con 0x7f7ca8102ef0 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57610/0 (socket says 172.21.15.86:57610) 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 -- 172.21.15.86:0/980879788 learned_addr learned my addr 172.21.15.86:0/980879788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad057640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cae059640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 0x7f7ca819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 -- 172.21.15.86:0/980879788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 msgr2=0x7f7ca819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 0x7f7ca819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 -- 172.21.15.86:0/980879788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 msgr2=0x7f7ca8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 -- 172.21.15.86:0/980879788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca819c2d0 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad057640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cad858640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4006010 tx=0x7f7ca4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7c9effd640 1 -- 172.21.15.86:0/980879788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca403f070 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca819c560 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.474+0000 7f7c9effd640 1 -- 172.21.15.86:0/980879788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca403bd20 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.475+0000 7f7c9effd640 1 -- 172.21.15.86:0/980879788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4035030 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.475+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81a3ea0 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.477+0000 7f7c9effd640 1 -- 172.21.15.86:0/980879788 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7ca4042070 con 0x7f7ca8100e20 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.477+0000 7f7cb1e95740 1 Processor -- start 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.477+0000 7f7cb1e95740 1 -- start start 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.477+0000 7f7c9effd640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 0x7f7c940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.477+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e71970 0x555985e73d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555985dc82f0 con 0x555985e647e0 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555985d6fcf0 con 0x555985e65c30 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555985dc42b0 con 0x555985e71970 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7c9effd640 1 -- 172.21.15.86:0/980879788 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f7ca40f02f0 con 0x7f7ca8100e20 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cae059640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 0x7f7c940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57622/0 (socket says 172.21.15.86:57622) 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad057640 1 -- 172.21.15.86:0/3818861307 learned_addr learned my addr 172.21.15.86:0/3818861307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cae059640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e71970 0x555985e73d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 -- 172.21.15.86:0/3818861307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e71970 msgr2=0x555985e73d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e71970 0x555985e73d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 -- 172.21.15.86:0/3818861307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 msgr2=0x555985e64c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 -- 172.21.15.86:0/3818861307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555985d543a0 con 0x555985e65c30 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad057640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cae059640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 0x7f7c940a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000acd0 tx=0x7f7ca000e720 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7cad858640 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e642a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40ed5a0 tx=0x7f7ca40bc140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2bc3275583aa5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7c93fff640 1 -- 172.21.15.86:0/3818861307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4049070 con 0x555985e65c30 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.478+0000 7f7c93fff640 1 -- 172.21.15.86:0/3818861307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca4031e50 con 0x555985e65c30 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7c93fff640 1 -- 172.21.15.86:0/3818861307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40bc370 con 0x555985e65c30 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 -- 172.21.15.86:0/3818861307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 msgr2=0x555985e642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e642a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40ed5a0 tx=0x7f7ca40bc140 comp rx=0 tx=0).stop 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 -- 172.21.15.86:0/3818861307 shutdown_connections 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e71970 0x555985e73d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e647e0 0x555985e64c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/3818861307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e642a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 -- 172.21.15.86:0/3818861307 >> 172.21.15.86:0/3818861307 conn(0x555985e2b310 msgr2=0x555985e60e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 -- 172.21.15.86:0/3818861307 shutdown_connections 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.479+0000 7f7cb1e95740 1 -- 172.21.15.86:0/3818861307 wait complete. 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 -- start start 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 Processor -- start 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 0x555985e6eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555985e74300 con 0x555985e65c30 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555985d83060 con 0x555985e71970 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555985e74480 con 0x555985e647e0 2024-09-05T06:51:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cb1e95740 10 client.0 _ll_register_callbacks cb 0x555985ad68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cad057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cad057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48878/0 (socket says 172.21.15.86:48878) 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cad057640 1 -- 172.21.15.86:0/2061156674 learned_addr learned my addr 172.21.15.86:0/2061156674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cae059640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 0x555985e6eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.480+0000 7f7cad858640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 -- 172.21.15.86:0/2061156674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 msgr2=0x555985e712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 -- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 msgr2=0x555985e6eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 0x555985e6eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 -- 172.21.15.86:0/2061156674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555985b393c0 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cae059640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 0x555985e6eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cad057640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005880 tx=0x7f7c98010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab3e9268853ebec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800ed40 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca4031e50 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f7c98005020 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7c98024050 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cb1e95740 10 client.17931 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555985e7fd30 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca4031e50 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.481+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f7c98011440 con 0x555985e65c30 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 10 client.17931 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 10 client.17931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:51:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 20 client.17931 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985e81240 0x555985e83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 4 client.17931 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555985dd5860 con 0x555985e81240 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cad858640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985e81240 0x555985e83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7c917fa640 10 client.17931 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.482+0000 7f7cad858640 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985e81240 0x555985e83660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f22e121594ba6f6 server_cookie=52dbfd8a86568520 in_seq=0 out_seq=0 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7c917fa640 1 -- 172.21.15.86:0/2061156674 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555985dd5860 con 0x555985e81240 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7c917fa640 10 client.17931 handle_command_reply: tid=0 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 client.17931 shutdown 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 msgr2=0x555985e6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005880 tx=0x7f7c98010ca0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 shutdown_connections 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985e81240 0x555985e83660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e71970 0x555985e6eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e65c30 0x555985e6e580 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/2061156674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7c917fa640 0 client.17931 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 >> 172.21.15.86:0/2061156674 conn(0x555985e2b310 msgr2=0x555985e66bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 shutdown_connections 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 1 -- 172.21.15.86:0/2061156674 wait complete. 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.483+0000 7f7cb1e95740 20 client.17931 trim_cache size 0 max 16384 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 Processor -- start 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 -- start start 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e7d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e5e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81ace50 con 0x555985e71970 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81aea20 con 0x555985e65c30 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.497+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81ad520 con 0x555985e647e0 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48882/0 (socket says 172.21.15.86:48882) 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 -- 172.21.15.86:0/1616975095 learned_addr learned my addr 172.21.15.86:0/1616975095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cad858640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e7d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cad057640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e5e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 -- 172.21.15.86:0/1616975095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 msgr2=0x555985e7d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e7d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 -- 172.21.15.86:0/1616975095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 msgr2=0x555985e5e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e5e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 -- 172.21.15.86:0/1616975095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555985e71640 con 0x555985e71970 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cad057640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e5e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7cae059640 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00128d0 tx=0x7f7ca0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997a9e581c7b780f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7c937fe640 1 -- 172.21.15.86:0/1616975095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011dd0 con 0x555985e71970 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7c937fe640 1 -- 172.21.15.86:0/1616975095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca0023070 con 0x555985e71970 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.498+0000 7f7c937fe640 1 -- 172.21.15.86:0/1616975095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca001f440 con 0x555985e71970 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- 172.21.15.86:0/1616975095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 msgr2=0x555985e5ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00128d0 tx=0x7f7ca0019000 comp rx=0 tx=0).stop 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- 172.21.15.86:0/1616975095 shutdown_connections 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e5ea30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e5e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/1616975095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e7d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- 172.21.15.86:0/1616975095 >> 172.21.15.86:0/1616975095 conn(0x555985e2b310 msgr2=0x555985e70600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- 172.21.15.86:0/1616975095 shutdown_connections 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- 172.21.15.86:0/1616975095 wait complete. 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 -- start start 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.499+0000 7f7cb1e95740 1 Processor -- start 2024-09-05T06:51:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e79a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81ae2f0 con 0x555985e71970 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81adbf0 con 0x555985e65c30 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81aae50 con 0x555985e647e0 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cb1e95740 10 client.0 _ll_register_callbacks cb 0x7f7c9800c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48896/0 (socket says 172.21.15.86:48896) 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 -- 172.21.15.86:0/274037849 learned_addr learned my addr 172.21.15.86:0/274037849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cad057640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e79a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cad858640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 -- 172.21.15.86:0/274037849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 msgr2=0x555985e79510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e79510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 -- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 msgr2=0x555985e79a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e79a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 -- 172.21.15.86:0/274037849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555985e76b20 con 0x555985e71970 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cad057640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e79a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7cae059640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005640 tx=0x7f7ca0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb75f0860a323bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.500+0000 7f7c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011de0 con 0x555985e71970 2024-09-05T06:51:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca0023070 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f7ca0004a40 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7ca0004c40 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 10 client.17937 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555985eec7a0 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca0023070 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f7ca0027420 con 0x555985e71970 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 10 client.17937 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 10 client.17937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 20 client.17937 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985eece00 0x555985eef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 4 client.17937 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555985dcfd70 con 0x555985eece00 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.501+0000 7f7cad858640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985eece00 0x555985eef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cad858640 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985eece00 0x555985eef220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd9917d6e5775cf1 server_cookie=b8dfb6a56b24472a in_seq=0 out_seq=0 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7c917fa640 10 client.17937 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7c917fa640 1 -- 172.21.15.86:0/274037849 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555985dcfd70 con 0x555985eece00 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7c917fa640 10 client.17937 handle_command_reply: tid=0 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 client.17937 shutdown 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 msgr2=0x555985e764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005640 tx=0x7f7ca0005230 comp rx=0 tx=0).stop 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 shutdown_connections 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x555985eece00 0x555985eef220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555985e71970 0x555985e764e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555985e65c30 0x555985e79a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7c917fa640 0 client.17937 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 --2- 172.21.15.86:0/274037849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555985e647e0 0x555985e79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 >> 172.21.15.86:0/274037849 conn(0x555985e2b310 msgr2=0x555985e5f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 shutdown_connections 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 1 -- 172.21.15.86:0/274037849 wait complete. 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.502+0000 7f7cb1e95740 20 client.17937 trim_cache size 0 max 16384 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 msgr2=0x7f7c940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 0x7f7c940a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000acd0 tx=0x7f7ca000e720 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 msgr2=0x7f7ca8197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4006010 tx=0x7f7ca4003d50 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 shutdown_connections 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c940a2cf0 0x7f7c940a51b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca8102ef0 0x7f7ca819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ca8102570 0x7f7ca8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 --2- 172.21.15.86:0/980879788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca8100e20 0x7f7ca8197320 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 >> 172.21.15.86:0/980879788 conn(0x7f7ca80fc7d0 msgr2=0x7f7ca8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 shutdown_connections 2024-09-05T06:51:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:52.503+0000 7f7cafabc640 1 -- 172.21.15.86:0/980879788 wait complete. 2024-09-05T06:51:52.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188140 1.67 MB/sec execute 432 sec latency 3374.190 ms 2024-09-05T06:51:53.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:51:53.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:53 smithi086 ceph-mon[30943]: pgmap v329: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:51:53.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188140 1.66 MB/sec execute 433 sec latency 1551.400 ms 2024-09-05T06:51:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:53 smithi115 ceph-mon[34791]: pgmap v329: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:51:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:53 smithi096 ceph-mon[34468]: pgmap v329: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:51:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- 172.21.15.86:0/2228753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4100e20 msgr2=0x7f84f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- 172.21.15.86:0/2228753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4100e20 0x7f84f4101200 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e802a480 comp rx=0 tx=0).stop 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- 172.21.15.86:0/2228753832 shutdown_connections 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- 172.21.15.86:0/2228753832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4102ef0 0x7f84f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- 172.21.15.86:0/2228753832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102570 0x7f84f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- 172.21.15.86:0/2228753832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4100e20 0x7f84f4101200 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- 172.21.15.86:0/2228753832 >> 172.21.15.86:0/2228753832 conn(0x7f84f40fc7d0 msgr2=0x7f84f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- 172.21.15.86:0/2228753832 shutdown_connections 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- 172.21.15.86:0/2228753832 wait complete. 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 Processor -- start 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- start start 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4100e20 0x7f84f41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102ef0 0x7f84f419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4110e00 con 0x7f84f4102570 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4110c80 con 0x7f84f4102ef0 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.025+0000 7f84faeda640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4110f80 con 0x7f84f4100e20 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48908/0 (socket says 172.21.15.86:48908) 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 -- 172.21.15.86:0/4268542780 learned_addr learned my addr 172.21.15.86:0/4268542780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f9477640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102ef0 0x7f84f419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f8c76640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4100e20 0x7f84f41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 -- 172.21.15.86:0/4268542780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4100e20 msgr2=0x7f84f41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4100e20 0x7f84f41972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 -- 172.21.15.86:0/4268542780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102ef0 msgr2=0x7f84f419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102ef0 0x7f84f419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 -- 172.21.15.86:0/4268542780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f419c270 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d950 tx=0x7f84e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f1ffb640 1 -- 172.21.15.86:0/4268542780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0017030 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f1ffb640 1 -- 172.21.15.86:0/4268542780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e0004020 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f419c560 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.026+0000 7f84f1ffb640 1 -- 172.21.15.86:0/4268542780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00047a0 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.027+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f41a3e40 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f1ffb640 1 -- 172.21.15.86:0/4268542780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84e00041d0 con 0x7f84f4102570 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 Processor -- start 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 -- start start 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f1ffb640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 0x7f84e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 0x55f0cad57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad66970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0cacbd2f0 con 0x55f0cad58b40 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f0cac64cf0 con 0x55f0cad575d0 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f0cacb92b0 con 0x55f0cad64580 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f9477640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 0x7f84e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f1ffb640 1 -- 172.21.15.86:0/4268542780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f84e00cf330 con 0x7f84f4102570 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48914/0 (socket says 172.21.15.86:48914) 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.029+0000 7f84f8c76640 1 -- 172.21.15.86:0/2322596367 learned_addr learned my addr 172.21.15.86:0/2322596367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f9477640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad66970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f3fff640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 0x55f0cad57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 -- 172.21.15.86:0/2322596367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 msgr2=0x55f0cad66970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad66970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 -- 172.21.15.86:0/2322596367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 msgr2=0x55f0cad57a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 0x55f0cad57a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 -- 172.21.15.86:0/2322596367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0cac493a0 con 0x55f0cad58b40 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f3fff640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 0x55f0cad57a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f9477640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad66970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f9477640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 0x7f84e40a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84ec005fd0 tx=0x7f84ec014b70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84f8c76640 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20d7203d93ecb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84deffd640 1 -- 172.21.15.86:0/2322596367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8040070 con 0x55f0cad58b40 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84deffd640 1 -- 172.21.15.86:0/2322596367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e8031e10 con 0x55f0cad58b40 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.030+0000 7f84deffd640 1 -- 172.21.15.86:0/2322596367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8035040 con 0x55f0cad58b40 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- 172.21.15.86:0/2322596367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 msgr2=0x55f0cad57090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e8005c00 comp rx=0 tx=0).stop 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- 172.21.15.86:0/2322596367 shutdown_connections 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad66970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad575d0 0x55f0cad57a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/2322596367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad58b40 0x55f0cad57090 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- 172.21.15.86:0/2322596367 >> 172.21.15.86:0/2322596367 conn(0x55f0cad20310 msgr2=0x55f0cad53c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- 172.21.15.86:0/2322596367 shutdown_connections 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- 172.21.15.86:0/2322596367 wait complete. 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 -- start start 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.031+0000 7f84fd2b3740 1 Processor -- start 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0cad5a5b0 con 0x55f0cad575d0 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f0cac78060 con 0x55f0cad58b40 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f0cad5a730 con 0x55f0cad64580 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84fd2b3740 10 client.0 _ll_register_callbacks cb 0x55f0ca9cb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48920/0 (socket says 172.21.15.86:48920) 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 -- 172.21.15.86:0/4089646598 learned_addr learned my addr 172.21.15.86:0/4089646598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f9477640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 -- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 msgr2=0x55f0cad60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad60000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 -- 172.21.15.86:0/4089646598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 msgr2=0x55f0cad5fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad5fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 -- 172.21.15.86:0/4089646598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0caa2e3c0 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f9477640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84f8c76640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005940 tx=0x7f84e802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2b2278631f7314 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.032+0000 7f84d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e804b070 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f84e8004750 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e8031e10 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f84e8057050 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 10 client.17949 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f0cad74d40 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e8031e10 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f84e803c7e0 con 0x55f0cad575d0 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 10 client.17949 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 10 client.17949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 20 client.17949 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cad76090 0x55f0cad78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 4 client.17949 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.033+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0cacca860 con 0x55f0cad76090 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.034+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cad76090 0x55f0cad78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.034+0000 7f84d3fff640 10 client.17949 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.034+0000 7f84f3fff640 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cad76090 0x55f0cad78480 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60e45a06dc150502 server_cookie=63bac822e8693003 in_seq=0 out_seq=0 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84d3fff640 1 -- 172.21.15.86:0/4089646598 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0cacca860 con 0x55f0cad76090 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84d3fff640 10 client.17949 handle_command_reply: tid=0 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 client.17949 shutdown 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 msgr2=0x55f0cad5f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005940 tx=0x7f84e802a480 comp rx=0 tx=0).stop 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 shutdown_connections 2024-09-05T06:51:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cad76090 0x55f0cad78480 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84d3fff640 0 client.17949 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/4089646598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad5f580 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 >> 172.21.15.86:0/4089646598 conn(0x55f0cad20310 msgr2=0x55f0cad59960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 shutdown_connections 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 1 -- 172.21.15.86:0/4089646598 wait complete. 2024-09-05T06:51:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.035+0000 7f84fd2b3740 20 client.17949 trim_cache size 0 max 16384 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 Processor -- start 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 -- start start 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 0x55f0cad7df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 0x55f0cad5c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41acea0 con 0x55f0cad575d0 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41aea70 con 0x55f0cad64580 2024-09-05T06:51:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.048+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41ad570 con 0x55f0cad58b40 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48926/0 (socket says 172.21.15.86:48926) 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 -- 172.21.15.86:0/597747062 learned_addr learned my addr 172.21.15.86:0/597747062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f3fff640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 0x55f0cad7df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f9477640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 0x55f0cad5c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 -- 172.21.15.86:0/597747062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 msgr2=0x55f0cad7df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 0x55f0cad7df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 -- 172.21.15.86:0/597747062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 msgr2=0x55f0cad5c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 0x55f0cad5c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 -- 172.21.15.86:0/597747062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0cad787f0 con 0x55f0cad575d0 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f3fff640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 0x55f0cad7df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f9477640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 0x55f0cad5c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:51:54.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84f8c76640 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f84e802a9a0 tx=0x7f84e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38d46de721dc14e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84de7fc640 1 -- 172.21.15.86:0/597747062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e804b070 con 0x55f0cad575d0 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84de7fc640 1 -- 172.21.15.86:0/597747062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e803c8f0 con 0x55f0cad575d0 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.049+0000 7f84de7fc640 1 -- 172.21.15.86:0/597747062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8035030 con 0x55f0cad575d0 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- 172.21.15.86:0/597747062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 msgr2=0x55f0cad7da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f84e802a9a0 tx=0x7f84e8005850 comp rx=0 tx=0).stop 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- 172.21.15.86:0/597747062 shutdown_connections 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad64580 0x55f0cad5c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad58b40 0x55f0cad7df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/597747062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad7da20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- 172.21.15.86:0/597747062 >> 172.21.15.86:0/597747062 conn(0x55f0cad20310 msgr2=0x55f0cad5fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- 172.21.15.86:0/597747062 shutdown_connections 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- 172.21.15.86:0/597747062 wait complete. 2024-09-05T06:51:54.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.050+0000 7f84fd2b3740 1 -- start start 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 Processor -- start 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad6d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad6d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41ae340 con 0x55f0cad575d0 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41adc40 con 0x55f0cad58b40 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84f41aaea0 con 0x55f0cad64580 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84fd2b3740 10 client.0 _ll_register_callbacks cb 0x7f84e8009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48928/0 (socket says 172.21.15.86:48928) 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 -- 172.21.15.86:0/3642942320 learned_addr learned my addr 172.21.15.86:0/3642942320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f3fff640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad6d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f9477640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad6d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 -- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 msgr2=0x55f0cad6d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad6d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 -- 172.21.15.86:0/3642942320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 msgr2=0x55f0cad6d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad6d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.051+0000 7f84f8c76640 1 -- 172.21.15.86:0/3642942320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0cad6e110 con 0x55f0cad575d0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84f8c76640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005cf0 tx=0x7f84e8005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e64587002453dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8035040 con 0x55f0cad575d0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f84e803c440 con 0x55f0cad575d0 2024-09-05T06:51:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e803c8f0 con 0x55f0cad575d0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f84e806c050 con 0x55f0cad575d0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 10 client.17955 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f0cade0890 con 0x55f0cad575d0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e803c8f0 con 0x55f0cad575d0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f84e8050520 con 0x55f0cad575d0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 10 client.17955 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 10 client.17955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 20 client.17955 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cade0f10 0x55f0cade3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 4 client.17955 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.052+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0cacc4d70 con 0x55f0cade0f10 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84f3fff640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cade0f10 0x55f0cade3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84f3fff640 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cade0f10 0x55f0cade3330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53b3aa75a22ac2eb server_cookie=859ab52b496fb404 in_seq=0 out_seq=0 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84d3fff640 10 client.17955 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84d3fff640 1 -- 172.21.15.86:0/3642942320 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0cacc4d70 con 0x55f0cade0f10 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84d3fff640 10 client.17955 handle_command_reply: tid=0 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 client.17955 shutdown 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 msgr2=0x55f0cad71bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005cf0 tx=0x7f84e8005410 comp rx=0 tx=0).stop 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 shutdown_connections 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55f0cade0f10 0x55f0cade3330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f0cad64580 0x55f0cad6d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0cad58b40 0x55f0cad6d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 --2- 172.21.15.86:0/3642942320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0cad575d0 0x55f0cad71bd0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84d3fff640 0 client.17955 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:51:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.053+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 >> 172.21.15.86:0/3642942320 conn(0x55f0cad20310 msgr2=0x55f0cad59110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 shutdown_connections 2024-09-05T06:51:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84fd2b3740 1 -- 172.21.15.86:0/3642942320 wait complete. 2024-09-05T06:51:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84fd2b3740 20 client.17955 trim_cache size 0 max 16384 2024-09-05T06:51:54.066 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 msgr2=0x7f84e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 0x7f84e40a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84ec005fd0 tx=0x7f84ec014b70 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 msgr2=0x7f84f4197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d950 tx=0x7f84e000de10 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 shutdown_connections 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84e40a2cf0 0x7f84e40a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84f4102ef0 0x7f84f419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84f4102570 0x7f84f4197800 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 --2- 172.21.15.86:0/4268542780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84f4100e20 0x7f84f41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 >> 172.21.15.86:0/4268542780 conn(0x7f84f40fc7d0 msgr2=0x7f84f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 shutdown_connections 2024-09-05T06:51:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:54.054+0000 7f84faeda640 1 -- 172.21.15.86:0/4268542780 wait complete. 2024-09-05T06:51:54.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188140 1.66 MB/sec execute 434 sec latency 2551.481 ms 2024-09-05T06:51:54.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:51:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:55.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188140 1.65 MB/sec execute 435 sec latency 3551.616 ms 2024-09-05T06:51:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:55 smithi115 ceph-mon[34791]: pgmap v330: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:55 smithi086 ceph-mon[30943]: pgmap v330: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:55 smithi096 ceph-mon[34468]: pgmap v330: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:56.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188140 1.65 MB/sec execute 436 sec latency 4551.701 ms 2024-09-05T06:51:56.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 -- 172.21.15.86:0/2563905674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 msgr2=0x7f16d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 --2- 172.21.15.86:0/2563905674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d01029b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b802a480 comp rx=0 tx=0).stop 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 -- 172.21.15.86:0/2563905674 shutdown_connections 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 --2- 172.21.15.86:0/2563905674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 0x7f16d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 --2- 172.21.15.86:0/2563905674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d01029b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 --2- 172.21.15.86:0/2563905674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 -- 172.21.15.86:0/2563905674 >> 172.21.15.86:0/2563905674 conn(0x7f16d00fc7d0 msgr2=0x7f16d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 -- 172.21.15.86:0/2563905674 shutdown_connections 2024-09-05T06:51:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 -- 172.21.15.86:0/2563905674 wait complete. 2024-09-05T06:51:56.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.859+0000 7f16d69c4640 1 Processor -- start 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 -- start start 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 0x7f16d019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0110e80 con 0x7f16d0102570 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0110d00 con 0x7f16d0100e20 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d69c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0111000 con 0x7f16d0102ef0 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48948/0 (socket says 172.21.15.86:48948) 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16c7fff640 1 -- 172.21.15.86:0/1269606452 learned_addr learned my addr 172.21.15.86:0/1269606452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16d4f61640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 0x7f16d019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 -- 172.21.15.86:0/1269606452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 msgr2=0x7f16d019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 0x7f16d019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 -- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 msgr2=0x7f16d019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 -- 172.21.15.86:0/1269606452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019f5d0 con 0x7f16d0100e20 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16c7fff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.860+0000 7f16cffff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d019a620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16c000e9f0 tx=0x7f16c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.861+0000 7f16cdffb640 1 -- 172.21.15.86:0/1269606452 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000ccc0 con 0x7f16d0100e20 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.861+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019f8c0 con 0x7f16d0100e20 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.861+0000 7f16cdffb640 1 -- 172.21.15.86:0/1269606452 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c00108e0 con 0x7f16d0100e20 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.861+0000 7f16cdffb640 1 -- 172.21.15.86:0/1269606452 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0018880 con 0x7f16d0100e20 2024-09-05T06:51:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.861+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01a71a0 con 0x7f16d0100e20 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.863+0000 7f16d8d9d740 1 Processor -- start 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.863+0000 7f16d8d9d740 1 -- start start 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646c262d2f0 con 0x5646c26c75d0 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646c25d4cf0 con 0x5646c26c8b40 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5646c26292b0 con 0x5646c26d2370 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48434/0 (socket says 172.21.15.86:48434) 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 -- 172.21.15.86:0/72424160 learned_addr learned my addr 172.21.15.86:0/72424160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16c7fff640 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 -- 172.21.15.86:0/72424160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 msgr2=0x5646c26d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 -- 172.21.15.86:0/72424160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 msgr2=0x5646c26c7a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 -- 172.21.15.86:0/72424160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c25b93a0 con 0x5646c26c8b40 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16cdffb640 1 -- 172.21.15.86:0/1269606452 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f16c0010a90 con 0x7f16d0100e20 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.864+0000 7f16d4f61640 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16c80039f0 tx=0x7f16c8003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb56f8e388841fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16c67fc640 1 -- 172.21.15.86:0/72424160 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c801a470 con 0x5646c26c8b40 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16c67fc640 1 -- 172.21.15.86:0/72424160 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8015070 con 0x5646c26c8b40 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16c67fc640 1 -- 172.21.15.86:0/72424160 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8010780 con 0x5646c26c8b40 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16cdffb640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 0x7f16bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16cffff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 0x7f16bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16cdffb640 1 -- 172.21.15.86:0/1269606452 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f16c00cfbe0 con 0x7f16d0100e20 2024-09-05T06:51:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 -- 172.21.15.86:0/72424160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 msgr2=0x5646c26c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16c80039f0 tx=0x7f16c8003eb0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 -- 172.21.15.86:0/72424160 shutdown_connections 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/72424160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26c7090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.865+0000 7f16d8d9d740 1 -- 172.21.15.86:0/72424160 >> 172.21.15.86:0/72424160 conn(0x5646c2690310 msgr2=0x5646c26c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- 172.21.15.86:0/72424160 shutdown_connections 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- 172.21.15.86:0/72424160 wait complete. 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16cffff640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 0x7f16bc0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5646c26c4ac0 tx=0x7f16c0091200 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- start start 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 Processor -- start 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26c75d0 0x5646c26d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646c26d5070 con 0x5646c26d2370 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646c25e8060 con 0x5646c26c8b40 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5646c26d51f0 con 0x5646c26c75d0 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.866+0000 7f16d8d9d740 10 client.0 _ll_register_callbacks cb 0x5646c233b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48440/0 (socket says 172.21.15.86:48440) 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48960/0 (socket says 172.21.15.86:48960) 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16c7fff640 1 -- 172.21.15.86:0/1403570343 learned_addr learned my addr 172.21.15.86:0/1403570343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26c75d0 0x5646c26d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 -- 172.21.15.86:0/1403570343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26c75d0 msgr2=0x5646c26d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26c75d0 0x5646c26d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 -- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 msgr2=0x5646c26ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 -- 172.21.15.86:0/1403570343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c239e3c0 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16c7fff640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d4f61640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f16c8011f10 tx=0x7f16c8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b4f921ebcd253e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8023040 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f16c8010560 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.867+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c8015070 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f16c8030050 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 10 client.17964 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646c26e0b10 con 0x5646c26d2370 2024-09-05T06:51:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c8015070 con 0x5646c26d2370 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f16c801a670 con 0x5646c26d2370 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 10 client.17964 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 20 client.17964 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c26e1f70 0x5646c26e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 4 client.17964 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.868+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c263a860 con 0x5646c26e1f70 2024-09-05T06:51:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.869+0000 7f16c7fff640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c26e1f70 0x5646c26e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.869+0000 7f16c7fff640 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c26e1f70 0x5646c26e4360 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=814e4d687243372 server_cookie=12503eb6db4a1a56 in_seq=0 out_seq=0 2024-09-05T06:51:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.869+0000 7f16af7fe640 10 client.17964 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16af7fe640 1 -- 172.21.15.86:0/1403570343 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646c263a860 con 0x5646c26e1f70 2024-09-05T06:51:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16af7fe640 10 client.17964 handle_command_reply: tid=0 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 client.17964 shutdown 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 msgr2=0x5646c26cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f16c8011f10 tx=0x7f16c8005c50 comp rx=0 tx=0).stop 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 shutdown_connections 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c26e1f70 0x5646c26e4360 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26d2370 0x5646c26cf5e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16af7fe640 0 client.17964 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/1403570343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26c75d0 0x5646c26d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 >> 172.21.15.86:0/1403570343 conn(0x5646c2690310 msgr2=0x5646c26c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 shutdown_connections 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 1 -- 172.21.15.86:0/1403570343 wait complete. 2024-09-05T06:51:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.870+0000 7f16d8d9d740 20 client.17964 trim_cache size 0 max 16384 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.885+0000 7f16d8d9d740 1 Processor -- start 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.885+0000 7f16d8d9d740 1 -- start start 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b1400 con 0x5646c26c75d0 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b2fd0 con 0x5646c26c8b40 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b1ad0 con 0x5646c26d2370 2024-09-05T06:51:56.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48970/0 (socket says 172.21.15.86:48970) 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d4f61640 1 -- 172.21.15.86:0/3628936160 learned_addr learned my addr 172.21.15.86:0/3628936160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 -- 172.21.15.86:0/3628936160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 msgr2=0x5646c26e9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d4f61640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26e9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 -- 172.21.15.86:0/3628936160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 msgr2=0x5646c26c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 -- 172.21.15.86:0/3628936160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c26d20b0 con 0x5646c26c8b40 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16d4f61640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.886+0000 7f16c7fff640 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26de340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5646c26d0630 tx=0x7f16b8003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4961c0e03c31311 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16c5ffb640 1 -- 172.21.15.86:0/3628936160 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b804a070 con 0x5646c26c8b40 2024-09-05T06:51:56.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16c5ffb640 1 -- 172.21.15.86:0/3628936160 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b80054d0 con 0x5646c26c8b40 2024-09-05T06:51:56.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16c5ffb640 1 -- 172.21.15.86:0/3628936160 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8035040 con 0x5646c26c8b40 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 -- 172.21.15.86:0/3628936160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 msgr2=0x5646c26de340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26de340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5646c26d0630 tx=0x7f16b8003960 comp rx=0 tx=0).stop 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 -- 172.21.15.86:0/3628936160 shutdown_connections 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26e9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26de340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/3628936160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.887+0000 7f16d8d9d740 1 -- 172.21.15.86:0/3628936160 >> 172.21.15.86:0/3628936160 conn(0x5646c2690310 msgr2=0x5646c26c3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- 172.21.15.86:0/3628936160 shutdown_connections 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- 172.21.15.86:0/3628936160 wait complete. 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- start start 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 Processor -- start 2024-09-05T06:51:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b28a0 con 0x5646c26c75d0 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b21a0 con 0x5646c26c8b40 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.888+0000 7f16d8d9d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01af400 con 0x5646c26d2370 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d8d9d740 10 client.0 _ll_register_callbacks cb 0x7f16b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48978/0 (socket says 172.21.15.86:48978) 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 -- 172.21.15.86:0/2293355381 learned_addr learned my addr 172.21.15.86:0/2293355381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16c7fff640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 -- 172.21.15.86:0/2293355381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 msgr2=0x5646c26d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 -- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 msgr2=0x5646c26d68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26d68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 -- 172.21.15.86:0/2293355381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c274bc40 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16c7fff640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26d68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d4f61640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f16c801ab90 tx=0x7f16c801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b90d8daf70ffbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8004410 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f16c8011930 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.889+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b80054d0 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f16c8045050 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 10 client.17970 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646c274cf20 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b80054d0 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f16c8004ad0 con 0x5646c26c75d0 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 10 client.17970 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:56.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 20 client.17970 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c274d610 0x5646c274fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 4 client.17970 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646c2634d70 con 0x5646c274d610 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.890+0000 7f16c7fff640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c274d610 0x5646c274fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16c7fff640 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c274d610 0x5646c274fa30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69317974b9756d52 server_cookie=632b6a06e1cfa758 in_seq=0 out_seq=0 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16af7fe640 10 client.17970 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16af7fe640 1 -- 172.21.15.86:0/2293355381 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646c2634d70 con 0x5646c274d610 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16af7fe640 10 client.17970 handle_command_reply: tid=0 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 client.17970 shutdown 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 msgr2=0x5646c26d63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f16c801ab90 tx=0x7f16c801d000 comp rx=0 tx=0).stop 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 shutdown_connections 2024-09-05T06:51:56.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5646c274d610 0x5646c274fa30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5646c26d2370 0x5646c26d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646c26c8b40 0x5646c26d68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16af7fe640 0 client.17970 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 --2- 172.21.15.86:0/2293355381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c26c75d0 0x5646c26d63a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 >> 172.21.15.86:0/2293355381 conn(0x5646c2690310 msgr2=0x5646c26c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 shutdown_connections 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.891+0000 7f16d8d9d740 1 -- 172.21.15.86:0/2293355381 wait complete. 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d8d9d740 20 client.17970 trim_cache size 0 max 16384 2024-09-05T06:51:56.897 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 msgr2=0x7f16bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 0x7f16bc0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5646c26c4ac0 tx=0x7f16c0091200 comp rx=0 tx=0).stop 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 msgr2=0x7f16d019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d019a620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16c000e9f0 tx=0x7f16c000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 shutdown_connections 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f16bc0a2cf0 0x7f16bc0a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16d0102ef0 0x7f16d019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d0102570 0x7f16d019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 --2- 172.21.15.86:0/1269606452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16d0100e20 0x7f16d019a620 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:56.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 >> 172.21.15.86:0/1269606452 conn(0x7f16d00fc7d0 msgr2=0x7f16d006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:56.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 shutdown_connections 2024-09-05T06:51:56.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:56.892+0000 7f16d69c4640 1 -- 172.21.15.86:0/1269606452 wait complete. 2024-09-05T06:51:57.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188371 1.65 MB/sec execute 437 sec latency 4764.156 ms 2024-09-05T06:51:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:57 smithi115 ceph-mon[34791]: pgmap v331: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:57 smithi086 ceph-mon[30943]: pgmap v331: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:57 smithi096 ceph-mon[34468]: pgmap v331: 129 pgs: 129 active+clean; 205 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-05T06:51:58.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188687 1.65 MB/sec execute 438 sec latency 16.639 ms 2024-09-05T06:51:58.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:51:58.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:51:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:51:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:51:59 smithi086 ceph-mon[30943]: pgmap v332: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:51:59.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 189189 1.65 MB/sec execute 439 sec latency 28.043 ms 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 -- 172.21.15.86:0/2857927932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044102d90 msgr2=0x7ff044103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 --2- 172.21.15.86:0/2857927932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044102d90 0x7ff044103170 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff034009930 tx=0x7ff03402a420 comp rx=0 tx=0).stop 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 -- 172.21.15.86:0/2857927932 shutdown_connections 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 --2- 172.21.15.86:0/2857927932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044108290 0x7ff04410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 --2- 172.21.15.86:0/2857927932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044103740 0x7ff044107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 --2- 172.21.15.86:0/2857927932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044102d90 0x7ff044103170 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 -- 172.21.15.86:0/2857927932 >> 172.21.15.86:0/2857927932 conn(0x7ff0440fc7d0 msgr2=0x7ff0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 -- 172.21.15.86:0/2857927932 shutdown_connections 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.617+0000 7ff04a898640 1 -- 172.21.15.86:0/2857927932 wait complete. 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 Processor -- start 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 -- start start 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044102d90 0x7ff0441009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 0x7ff044101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff044111f70 con 0x7ff044103740 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff044111df0 con 0x7ff044108290 2024-09-05T06:51:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff04a898640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441120f0 con 0x7ff044102d90 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48996/0 (socket says 172.21.15.86:48996) 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff0437fe640 1 -- 172.21.15.86:0/3440760679 learned_addr learned my addr 172.21.15.86:0/3440760679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff048e35640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 0x7ff044101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff043fff640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044102d90 0x7ff0441009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff0437fe640 1 -- 172.21.15.86:0/3440760679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044102d90 msgr2=0x7ff0441009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.618+0000 7ff0437fe640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044102d90 0x7ff0441009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0437fe640 1 -- 172.21.15.86:0/3440760679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 msgr2=0x7ff044101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0437fe640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 0x7ff044101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0437fe640 1 -- 172.21.15.86:0/3440760679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0441a6990 con 0x7ff044103740 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff048e35640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 0x7ff044101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0437fe640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e9c0 tx=0x7ff03800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0417fa640 1 -- 172.21.15.86:0/3440760679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03800cd80 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0441a6c80 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0417fa640 1 -- 172.21.15.86:0/3440760679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0380054d0 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff0417fa640 1 -- 172.21.15.86:0/3440760679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038010430 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.619+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441a71c0 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.621+0000 7ff0417fa640 1 -- 172.21.15.86:0/3440760679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff038004fd0 con 0x7ff044103740 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.621+0000 7ff04cc71740 1 Processor -- start 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.621+0000 7ff04cc71740 1 -- start start 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a20477cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a20478a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2046e2280 con 0x55a20477d480 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a204689b10 con 0x55a20477cb60 2024-09-05T06:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff04cc71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a2046de240 con 0x55a204787d40 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff048e35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a20477cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49008/0 (socket says 172.21.15.86:49008) 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 -- 172.21.15.86:0/2010435832 learned_addr learned my addr 172.21.15.86:0/2010435832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff0417fa640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 0x7ff02c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff048e35640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 0x7ff02c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 -- 172.21.15.86:0/2010435832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 msgr2=0x55a20478a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a20478a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 -- 172.21.15.86:0/2010435832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 msgr2=0x55a20477cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a20477cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff0417fa640 1 -- 172.21.15.86:0/3440760679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ff0380cfd30 con 0x7ff044103740 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.622+0000 7ff043fff640 1 -- 172.21.15.86:0/2010435832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2046961f0 con 0x55a20477d480 2024-09-05T06:51:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff043fff640 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff034031440 tx=0x7ff0340040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a01532291ef0893 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff048e35640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 0x7ff02c0a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0094d0 tx=0x7ff03c004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff02a7fc640 1 -- 172.21.15.86:0/2010435832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034035030 con 0x55a20477d480 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff02a7fc640 1 -- 172.21.15.86:0/2010435832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0340044a0 con 0x55a20477d480 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff02a7fc640 1 -- 172.21.15.86:0/2010435832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340035f0 con 0x55a20477d480 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 -- 172.21.15.86:0/2010435832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 msgr2=0x55a204787800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff034031440 tx=0x7ff0340040b0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 -- 172.21.15.86:0/2010435832 shutdown_connections 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a20478a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a204787800 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2010435832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a20477cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 -- 172.21.15.86:0/2010435832 >> 172.21.15.86:0/2010435832 conn(0x55a2047790f0 msgr2=0x55a2047794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.623+0000 7ff04cc71740 1 -- 172.21.15.86:0/2010435832 shutdown_connections 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 -- 172.21.15.86:0/2010435832 wait complete. 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 -- start start 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 Processor -- start 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204784c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 0x55a204785180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a20478a910 con 0x55a204787d40 2024-09-05T06:51:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a20469cf10 con 0x55a20477d480 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a20478aa90 con 0x55a20477cb60 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff04cc71740 10 client.0 _ll_register_callbacks cb 0x55a204665a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff048e35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff043fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204784c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 0x55a204785180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff048e35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49016/0 (socket says 172.21.15.86:49016) 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.624+0000 7ff048e35640 1 -- 172.21.15.86:0/2222101134 learned_addr learned my addr 172.21.15.86:0/2222101134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 -- 172.21.15.86:0/2222101134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 msgr2=0x55a204784c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204784c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 -- 172.21.15.86:0/2222101134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 msgr2=0x55a2047856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 -- 172.21.15.86:0/2222101134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2044536a0 con 0x55a20477d480 2024-09-05T06:51:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff048e35640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff043fff640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204784c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0437fe640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 0x55a204785180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0380cd040 tx=0x7ff0380981e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7906ee2587dfcbba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0237fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380a4a80 con 0x55a20477d480 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ff0380a45f0 con 0x55a20477d480 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0340044a0 con 0x55a20477d480 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff03809be10 con 0x55a20477d480 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff04cc71740 10 client.27095 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2047965a0 con 0x55a20477d480 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.625+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0340044a0 con 0x55a20477d480 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff038097d60 con 0x55a20477d480 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 10 client.27095 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 20 client.27095 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a204797a50 0x55a204799e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 4 client.27095 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2046ef7f0 con 0x55a204797a50 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff043fff640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a204797a50 0x55a204799e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff043fff640 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a204797a50 0x55a204799e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af2ffcf432694c8 server_cookie=f268cb1d323e582b in_seq=0 out_seq=0 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.626+0000 7ff0237fe640 10 client.27095 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff0237fe640 1 -- 172.21.15.86:0/2222101134 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2046ef7f0 con 0x55a204797a50 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff0237fe640 10 client.27095 handle_command_reply: tid=0 2024-09-05T06:51:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 client.27095 shutdown 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 msgr2=0x55a204785180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 0x55a204785180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0380cd040 tx=0x7ff0380981e0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 shutdown_connections 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a204797a50 0x55a204799e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a204787d40 0x55a2047856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477d480 0x55a204785180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 --2- 172.21.15.86:0/2222101134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204784c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff0237fe640 0 client.27095 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 >> 172.21.15.86:0/2222101134 conn(0x55a2047790f0 msgr2=0x55a20477ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 shutdown_connections 2024-09-05T06:51:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 1 -- 172.21.15.86:0/2222101134 wait complete. 2024-09-05T06:51:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.627+0000 7ff04cc71740 20 client.27095 trim_cache size 0 max 16384 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 Processor -- start 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 -- start start 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204785510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a204787d40 0x55a20479fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b12d0 con 0x55a20477d480 2024-09-05T06:51:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b2ea0 con 0x55a204787d40 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff04cc71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b19a0 con 0x55a20477cb60 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204785510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49018/0 (socket says 172.21.15.86:49018) 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff0437fe640 1 -- 172.21.15.86:0/3758022257 learned_addr learned my addr 172.21.15.86:0/3758022257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff048e35640 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a204787d40 0x55a20479fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 -- 172.21.15.86:0/3758022257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a204787d40 msgr2=0x55a20479fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a204787d40 0x55a20479fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 -- 172.21.15.86:0/3758022257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 msgr2=0x55a20479f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff043fff640 1 -- 172.21.15.86:0/3758022257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20477d2b0 con 0x55a20477cb60 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.648+0000 7ff0437fe640 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.649+0000 7ff043fff640 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204785510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034004b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aab2447c909b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.649+0000 7ff029ffb640 1 -- 172.21.15.86:0/3758022257 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034048070 con 0x55a20477cb60 2024-09-05T06:51:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.649+0000 7ff029ffb640 1 -- 172.21.15.86:0/3758022257 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff034052030 con 0x55a20477cb60 2024-09-05T06:51:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.649+0000 7ff029ffb640 1 -- 172.21.15.86:0/3758022257 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034035040 con 0x55a20477cb60 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- 172.21.15.86:0/3758022257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 msgr2=0x55a204785510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204785510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034004b90 comp rx=0 tx=0).stop 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- 172.21.15.86:0/3758022257 shutdown_connections 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a204787d40 0x55a20479fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a20479f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 --2- 172.21.15.86:0/3758022257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a20477cb60 0x55a204785510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- 172.21.15.86:0/3758022257 >> 172.21.15.86:0/3758022257 conn(0x55a2047790f0 msgr2=0x55a20477ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- 172.21.15.86:0/3758022257 shutdown_connections 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- 172.21.15.86:0/3758022257 wait complete. 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 -- start start 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.650+0000 7ff04cc71740 1 Processor -- start 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a2047902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a204781d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b2770 con 0x55a20477d480 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b2070 con 0x55a20477cb60 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441af2d0 con 0x55a204787d40 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49024/0 (socket says 172.21.15.86:49024) 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff0437fe640 1 -- 172.21.15.86:0/106890991 learned_addr learned my addr 172.21.15.86:0/106890991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:51:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff04cc71740 10 client.0 _ll_register_callbacks cb 0x7ff03800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a2047902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff048e35640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a204781d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 -- 172.21.15.86:0/106890991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 msgr2=0x55a204781d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a204781d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 -- 172.21.15.86:0/106890991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 msgr2=0x55a2047817e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 -- 172.21.15.86:0/106890991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a204782390 con 0x55a20477cb60 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff0437fe640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff043fff640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a2047902f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff03402aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a023f329afd537f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.651+0000 7ff0237fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03405c070 con 0x55a20477cb60 2024-09-05T06:51:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7ff034031860 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff034052030 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff034068050 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 10 client.27101 fetch_fsmap learned FSMap version 35 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2048025c0 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff034052030 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff034003cd0 con 0x55a20477cb60 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 10 client.27101 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 20 client.27101 populate_metadata read hostname 'smithi086' 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a2048027f0 0x55a204804be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 4 client.27101 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a2046e9d00 con 0x55a2048027f0 2024-09-05T06:51:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.652+0000 7ff0437fe640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a2048027f0 0x55a204804be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.653+0000 7ff0437fe640 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a2048027f0 0x55a204804be0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b8444f283a7f84 server_cookie=234fc39c065ace in_seq=0 out_seq=0 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.653+0000 7ff0237fe640 10 client.27101 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff0237fe640 1 -- 172.21.15.86:0/106890991 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a2046e9d00 con 0x55a2048027f0 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff0237fe640 10 client.27101 handle_command_reply: tid=0 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 client.27101 shutdown 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 msgr2=0x55a2047902f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a2047902f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff03402aae0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 shutdown_connections 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a2048027f0 0x55a204804be0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a204787d40 0x55a204781d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a20477d480 0x55a2047817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff0237fe640 0 client.27101 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 --2- 172.21.15.86:0/106890991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a20477cb60 0x55a2047902f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.654+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 >> 172.21.15.86:0/106890991 conn(0x55a2047790f0 msgr2=0x55a204777a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 shutdown_connections 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04cc71740 1 -- 172.21.15.86:0/106890991 wait complete. 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04cc71740 20 client.27101 trim_cache size 0 max 16384 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:51:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5c03675c-6ef0-4b31-9a10-ba44b291d4da", 2024-09-05T06:51:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:51:59.661 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:51:59.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 msgr2=0x7ff02c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 0x7ff02c0a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0094d0 tx=0x7ff03c004ab0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 msgr2=0x7ff044100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e9c0 tx=0x7ff03800ee80 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 shutdown_connections 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff02c0a2cf0 0x7ff02c0a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff044108290 0x7ff044101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff044103740 0x7ff044100f30 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 --2- 172.21.15.86:0/3440760679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff044102d90 0x7ff0441009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 >> 172.21.15.86:0/3440760679 conn(0x7ff0440fc7d0 msgr2=0x7ff044105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 shutdown_connections 2024-09-05T06:51:59.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:51:59.655+0000 7ff04a898640 1 -- 172.21.15.86:0/3440760679 wait complete. 2024-09-05T06:51:59.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:51:59 smithi115 ceph-mon[34791]: pgmap v332: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:51:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:51:59 smithi096 ceph-mon[34468]: pgmap v332: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-09-05T06:52:00.530 INFO:tasks.workunit.client.0.smithi086.stdout: 1 189698 1.65 MB/sec execute 440 sec latency 28.335 ms 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:00.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:52:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- 172.21.15.86:0/1065586585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 msgr2=0x7f49a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 --2- 172.21.15.86:0/1065586585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a8101200 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f499c009890 tx=0x7f499c02a410 comp rx=0 tx=0).stop 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- 172.21.15.86:0/1065586585 shutdown_connections 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 --2- 172.21.15.86:0/1065586585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102ef0 0x7f49a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 --2- 172.21.15.86:0/1065586585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102570 0x7f49a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 --2- 172.21.15.86:0/1065586585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a8101200 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- 172.21.15.86:0/1065586585 >> 172.21.15.86:0/1065586585 conn(0x7f49a80fc7d0 msgr2=0x7f49a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- 172.21.15.86:0/1065586585 shutdown_connections 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- 172.21.15.86:0/1065586585 wait complete. 2024-09-05T06:52:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 Processor -- start 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.389+0000 7f49aeb73640 1 -- start start 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 0x7f49a8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 0x7f49a819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8110e30 con 0x7f49a8102570 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8110cb0 con 0x7f49a8100e20 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49aeb73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8110fb0 con 0x7f49a8102ef0 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48528/0 (socket says 172.21.15.86:48528) 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 -- 172.21.15.86:0/2426709876 learned_addr learned my addr 172.21.15.86:0/2426709876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 0x7f49a8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ad110640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 0x7f49a819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 -- 172.21.15.86:0/2426709876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 msgr2=0x7f49a819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 0x7f49a819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 -- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 msgr2=0x7f49a8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 0x7f49a8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 -- 172.21.15.86:0/2426709876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819d470 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ad110640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 0x7f49a819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49a7fff640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 0x7f49a8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.390+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f499c000c00 tx=0x7f499c030ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.391+0000 7f49a5ffb640 1 -- 172.21.15.86:0/2426709876 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c005460 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.391+0000 7f49a5ffb640 1 -- 172.21.15.86:0/2426709876 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f499c005600 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.391+0000 7f49a5ffb640 1 -- 172.21.15.86:0/2426709876 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004e10 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.391+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819d700 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.391+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a819db90 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.393+0000 7f49a5ffb640 1 -- 172.21.15.86:0/2426709876 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f499c039040 con 0x7f49a8100e20 2024-09-05T06:52:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.393+0000 7f49b0f4c740 1 Processor -- start 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.393+0000 7f49b0f4c740 1 -- start start 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e832550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e834e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e7892e0 con 0x55ae3e823380 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e730ce0 con 0x55ae3e832a90 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e7852a0 con 0x55ae3e823ca0 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ad110640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ad110640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49050/0 (socket says 172.21.15.86:49050) 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ad110640 1 -- 172.21.15.86:0/3028630006 learned_addr learned my addr 172.21.15.86:0/3028630006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ac90f640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e832550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e834e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 -- 172.21.15.86:0/3028630006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 msgr2=0x55ae3e832550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e832550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 -- 172.21.15.86:0/3028630006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 msgr2=0x55ae3e823760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a5ffb640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 0x7f49980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 -- 172.21.15.86:0/3028630006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3e715440 con 0x55ae3e832a90 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ad110640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49ad110640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 0x7f49980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e834e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fa0 tx=0x7f49940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5916985c50f25d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.394+0000 7f49a5ffb640 1 -- 172.21.15.86:0/2426709876 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f499c0f0710 con 0x7f49a8100e20 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f4992ffd640 1 -- 172.21.15.86:0/3028630006 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994004310 con 0x55ae3e832a90 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f4992ffd640 1 -- 172.21.15.86:0/3028630006 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49940044b0 con 0x55ae3e832a90 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f4992ffd640 1 -- 172.21.15.86:0/3028630006 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940035f0 con 0x55ae3e832a90 2024-09-05T06:52:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3028630006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 msgr2=0x55ae3e834e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49ad110640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 0x7f49980a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009a60 tx=0x7f49a00095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e834e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fa0 tx=0x7f49940118d0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3028630006 shutdown_connections 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e834e80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e832550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3028630006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e823760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3028630006 >> 172.21.15.86:0/3028630006 conn(0x55ae3e81fa30 msgr2=0x55ae3e81fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3028630006 shutdown_connections 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.395+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3028630006 wait complete. 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 -- start start 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 Processor -- start 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 0x55ae3e82f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e832a90 0x55ae3e82fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e835660 con 0x55ae3e823380 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e744050 con 0x55ae3e823ca0 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae3e8357e0 con 0x55ae3e832a90 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49056/0 (socket says 172.21.15.86:49056) 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49ac90f640 1 -- 172.21.15.86:0/2633586874 learned_addr learned my addr 172.21.15.86:0/2633586874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49b0f4c740 10 client.0 _ll_register_callbacks cb 0x55ae3e784160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.396+0000 7f49a7fff640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 0x55ae3e82f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ad110640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e832a90 0x55ae3e82fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 -- 172.21.15.86:0/2633586874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e832a90 msgr2=0x55ae3e82fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e832a90 0x55ae3e82fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 -- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 msgr2=0x55ae3e82f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 0x55ae3e82f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 -- 172.21.15.86:0/2633586874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3e4fa3c0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49a7fff640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 0x55ae3e82f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49ac90f640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f499c04a000 tx=0x7f499c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84148e7bb2dcb539 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f4987fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0bc0f0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49940044b0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f499c0b8e00 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f499c0bce90 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49b0f4c740 10 client.18000 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.397+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae3e8410c0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49940044b0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f499c0bc9f0 con 0x55ae3e823380 2024-09-05T06:52:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 10 client.18000 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 20 client.18000 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e842500 0x55ae3e844990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 4 client.18000 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae3e796850 con 0x55ae3e842500 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49a7fff640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e842500 0x55ae3e844990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f49a7fff640 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e842500 0x55ae3e844990 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa6f5bab603322e server_cookie=5cbcfba0036501f9 in_seq=0 out_seq=0 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.398+0000 7f4987fff640 10 client.18000 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f4987fff640 1 -- 172.21.15.86:0/2633586874 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae3e796850 con 0x55ae3e842500 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f4987fff640 10 client.18000 handle_command_reply: tid=0 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 client.18000 shutdown 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 msgr2=0x55ae3e832410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f499c04a000 tx=0x7f499c009550 comp rx=0 tx=0).stop 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 shutdown_connections 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e842500 0x55ae3e844990 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e832a90 0x55ae3e82fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e823ca0 0x55ae3e82f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/2633586874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e832410 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f4987fff640 0 client.18000 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 >> 172.21.15.86:0/2633586874 conn(0x55ae3e81fa30 msgr2=0x55ae3e825790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 shutdown_connections 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 1 -- 172.21.15.86:0/2633586874 wait complete. 2024-09-05T06:52:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.399+0000 7f49b0f4c740 20 client.18000 trim_cache size 0 max 16384 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 Processor -- start 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 -- start start 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e82ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e82ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81acf90 con 0x55ae3e823380 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81aeb60 con 0x55ae3e832a90 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.418+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ad660 con 0x55ae3e823ca0 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e82ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e82ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49072/0 (socket says 172.21.15.86:49072) 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49ac90f640 1 -- 172.21.15.86:0/312935259 learned_addr learned my addr 172.21.15.86:0/312935259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49ad110640 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e82ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 -- 172.21.15.86:0/312935259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 msgr2=0x55ae3e82ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e82ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 -- 172.21.15.86:0/312935259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 msgr2=0x55ae3e82ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e82ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 -- 172.21.15.86:0/312935259 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3e823ad0 con 0x55ae3e823ca0 2024-09-05T06:52:01.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49a7fff640 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83e930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f499401b040 tx=0x7f4994005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42286d0a4f7ced59 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49927fc640 1 -- 172.21.15.86:0/312935259 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499401d040 con 0x55ae3e823ca0 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49927fc640 1 -- 172.21.15.86:0/312935259 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4994009b30 con 0x55ae3e823ca0 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.419+0000 7f49927fc640 1 -- 172.21.15.86:0/312935259 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400f740 con 0x55ae3e823ca0 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 -- 172.21.15.86:0/312935259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 msgr2=0x55ae3e83e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83e930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f499401b040 tx=0x7f4994005d50 comp rx=0 tx=0).stop 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 -- 172.21.15.86:0/312935259 shutdown_connections 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e82ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83e930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/312935259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e82ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 -- 172.21.15.86:0/312935259 >> 172.21.15.86:0/312935259 conn(0x55ae3e81fa30 msgr2=0x55ae3e81fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 -- 172.21.15.86:0/312935259 shutdown_connections 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.420+0000 7f49b0f4c740 1 -- 172.21.15.86:0/312935259 wait complete. 2024-09-05T06:52:01.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 -- start start 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 Processor -- start 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e8361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ae430 con 0x55ae3e823380 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81add30 con 0x55ae3e832a90 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81aaf90 con 0x55ae3e823ca0 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49b0f4c740 10 client.0 _ll_register_callbacks cb 0x7f499c0f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49ac90f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49088/0 (socket says 172.21.15.86:49088) 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49ad110640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e8361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49ac90f640 1 -- 172.21.15.86:0/3536536094 learned_addr learned my addr 172.21.15.86:0/3536536094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.421+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 -- 172.21.15.86:0/3536536094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 msgr2=0x55ae3e83b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 -- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 msgr2=0x55ae3e8361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e8361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 -- 172.21.15.86:0/3536536094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3e8369d0 con 0x55ae3e823380 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ad110640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e8361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49ac90f640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f499c0b8bf0 tx=0x7f499c031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd05bc53d30f7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f4987fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0bc0f0 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9605+0+0 (secure 0 0 0) 0x7f499c0f0240 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4994009b30 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f499c0bc560 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49b0f4c740 10 client.18006 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.422+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae3e8369d0 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4994009b30 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f499c0b84e0 con 0x55ae3e823380 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 10 client.18006 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 20 client.18006 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e8ad210 0x55ae3e8af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 4 client.18006 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae3e790d60 con 0x55ae3e8ad210 2024-09-05T06:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e8ad210 0x55ae3e8af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f4987fff640 10 client.18006 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.423+0000 7f49a7fff640 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e8ad210 0x55ae3e8af630 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c935cc2db10bac88 server_cookie=bc22f47145338b7b in_seq=0 out_seq=0 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f4987fff640 1 -- 172.21.15.86:0/3536536094 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae3e790d60 con 0x55ae3e8ad210 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f4987fff640 10 client.18006 handle_command_reply: tid=0 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 client.18006 shutdown 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 msgr2=0x55ae3e83ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f499c0b8bf0 tx=0x7f499c031bb0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 shutdown_connections 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3e8ad210 0x55ae3e8af630 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3e832a90 0x55ae3e8361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3e823ca0 0x55ae3e83b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f4987fff640 0 client.18006 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 --2- 172.21.15.86:0/3536536094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3e823380 0x55ae3e83ada0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.424+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 >> 172.21.15.86:0/3536536094 conn(0x55ae3e81fa30 msgr2=0x55ae3e82ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 shutdown_connections 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49b0f4c740 1 -- 172.21.15.86:0/3536536094 wait complete. 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49b0f4c740 20 client.18006 trim_cache size 0 max 16384 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout: "5c03675c-6ef0-4b31-9a10-ba44b291d4da": { 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:52:01.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5c03675c-6ef0-4b31-9a10-ba44b291d4da", 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 msgr2=0x7f49980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 0x7f49980a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009a60 tx=0x7f49a00095c0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 msgr2=0x7f49a81984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f499c000c00 tx=0x7f499c030ef0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 shutdown_connections 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49980a2d70 0x7f49980a5230 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.425+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49a8102ef0 0x7f49a819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.426+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8102570 0x7f49a8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.426+0000 7f49aeb73640 1 --2- 172.21.15.86:0/2426709876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49a8100e20 0x7f49a81984c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.426+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 >> 172.21.15.86:0/2426709876 conn(0x7f49a80fc7d0 msgr2=0x7f49a806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.426+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 shutdown_connections 2024-09-05T06:52:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:01.426+0000 7f49aeb73640 1 -- 172.21.15.86:0/2426709876 wait complete. 2024-09-05T06:52:01.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 190309 1.65 MB/sec execute 441 sec latency 43.867 ms 2024-09-05T06:52:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:01 smithi115 ceph-mon[34791]: pgmap v333: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 476 KiB/s wr, 42 op/s 2024-09-05T06:52:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:01 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:52:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:01 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:01 smithi096 ceph-mon[34468]: pgmap v333: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 476 KiB/s wr, 42 op/s 2024-09-05T06:52:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:01 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:52:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:01 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:01 smithi086 ceph-mon[30943]: pgmap v333: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 476 KiB/s wr, 42 op/s 2024-09-05T06:52:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:01 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:52:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:01 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:02.281 INFO:tasks.cephfs.filesystem:scrub status for tag:5c03675c-6ef0-4b31-9a10-ba44b291d4da - {'path': '/', 'tag': '5c03675c-6ef0-4b31-9a10-ba44b291d4da', 'options': 'recursive,force'} 2024-09-05T06:52:02.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 190939 1.66 MB/sec execute 442 sec latency 14.176 ms 2024-09-05T06:52:03.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191399 1.66 MB/sec execute 443 sec latency 14.010 ms 2024-09-05T06:52:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:03 smithi115 ceph-mon[34791]: pgmap v334: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-09-05T06:52:03.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:03 smithi086 ceph-mon[30943]: pgmap v334: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-09-05T06:52:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:03 smithi096 ceph-mon[34468]: pgmap v334: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-09-05T06:52:04.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191620 1.66 MB/sec execute 444 sec latency 744.583 ms 2024-09-05T06:52:05.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191620 1.66 MB/sec execute 445 sec latency 1744.629 ms 2024-09-05T06:52:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:05 smithi115 ceph-mon[34791]: pgmap v335: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:05 smithi086 ceph-mon[30943]: pgmap v335: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:05 smithi096 ceph-mon[34468]: pgmap v335: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:06.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191620 1.66 MB/sec execute 446 sec latency 2744.705 ms 2024-09-05T06:52:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:06.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:06 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:06 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:06.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:06 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:07.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191687 1.65 MB/sec execute 447 sec latency 2958.478 ms 2024-09-05T06:52:07.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:07 smithi115 ceph-mon[34791]: pgmap v336: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:07 smithi115 ceph-mon[34791]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T06:52:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:07 smithi086 ceph-mon[30943]: pgmap v336: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:07 smithi086 ceph-mon[30943]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T06:52:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:07 smithi096 ceph-mon[34468]: pgmap v336: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 665 KiB/s wr, 86 op/s 2024-09-05T06:52:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:07 smithi096 ceph-mon[34468]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T06:52:08.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191687 1.65 MB/sec execute 448 sec latency 1560.350 ms 2024-09-05T06:52:09.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191687 1.65 MB/sec execute 449 sec latency 2560.450 ms 2024-09-05T06:52:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:09 smithi115 ceph-mon[34791]: pgmap v338: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:09 smithi086 ceph-mon[30943]: pgmap v338: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:09 smithi096 ceph-mon[34468]: pgmap v338: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:10.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191687 1.64 MB/sec execute 450 sec latency 3560.548 ms 2024-09-05T06:52:11.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191687 1.64 MB/sec execute 451 sec latency 4560.652 ms 2024-09-05T06:52:11.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:11 smithi115 ceph-mon[34791]: pgmap v339: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:11 smithi086 ceph-mon[30943]: pgmap v339: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:11 smithi096 ceph-mon[34468]: pgmap v339: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-05T06:52:12.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191914 1.64 MB/sec execute 452 sec latency 4771.348 ms 2024-09-05T06:52:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:13 smithi096 ceph-mon[34468]: pgmap v340: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 690 KiB/s wr, 52 op/s 2024-09-05T06:52:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:13 smithi096 ceph-mon[34468]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T06:52:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:13 smithi115 ceph-mon[34791]: pgmap v340: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 690 KiB/s wr, 52 op/s 2024-09-05T06:52:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:13 smithi115 ceph-mon[34791]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T06:52:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:13 smithi086 ceph-mon[30943]: pgmap v340: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 690 KiB/s wr, 52 op/s 2024-09-05T06:52:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:13 smithi086 ceph-mon[30943]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T06:52:13.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 192230 1.63 MB/sec execute 453 sec latency 13.738 ms 2024-09-05T06:52:14.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 192788 1.63 MB/sec execute 454 sec latency 13.423 ms 2024-09-05T06:52:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:15 smithi096 ceph-mon[34468]: pgmap v342: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 863 KiB/s wr, 65 op/s 2024-09-05T06:52:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:15 smithi115 ceph-mon[34791]: pgmap v342: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 863 KiB/s wr, 65 op/s 2024-09-05T06:52:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:15 smithi086 ceph-mon[30943]: pgmap v342: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 863 KiB/s wr, 65 op/s 2024-09-05T06:52:15.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 193359 1.64 MB/sec execute 455 sec latency 22.572 ms 2024-09-05T06:52:16.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 193961 1.64 MB/sec execute 456 sec latency 19.816 ms 2024-09-05T06:52:17.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:17 smithi096 ceph-mon[34468]: pgmap v343: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 755 KiB/s wr, 57 op/s 2024-09-05T06:52:17.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:17 smithi115 ceph-mon[34791]: pgmap v343: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 755 KiB/s wr, 57 op/s 2024-09-05T06:52:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:17 smithi086 ceph-mon[30943]: pgmap v343: 129 pgs: 129 active+clean; 209 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 755 KiB/s wr, 57 op/s 2024-09-05T06:52:17.531 INFO:tasks.workunit.client.0.smithi086.stdout: 1 194809 1.65 MB/sec execute 457 sec latency 13.652 ms 2024-09-05T06:52:18.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195165 1.65 MB/sec execute 458 sec latency 164.384 ms 2024-09-05T06:52:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:19 smithi096 ceph-mon[34468]: pgmap v344: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:19.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:19 smithi115 ceph-mon[34791]: pgmap v344: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:19 smithi086 ceph-mon[30943]: pgmap v344: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:19.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195165 1.65 MB/sec execute 459 sec latency 1164.490 ms 2024-09-05T06:52:20.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195165 1.65 MB/sec execute 460 sec latency 2164.585 ms 2024-09-05T06:52:21.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:21 smithi096 ceph-mon[34468]: pgmap v345: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:21 smithi115 ceph-mon[34791]: pgmap v345: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:21 smithi086 ceph-mon[30943]: pgmap v345: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 863 KiB/s wr, 85 op/s 2024-09-05T06:52:21.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195165 1.64 MB/sec execute 461 sec latency 3164.675 ms 2024-09-05T06:52:22.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195390 1.64 MB/sec execute 462 sec latency 3378.651 ms 2024-09-05T06:52:23.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:23 smithi096 ceph-mon[34468]: pgmap v346: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-05T06:52:23.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:23 smithi115 ceph-mon[34791]: pgmap v346: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-05T06:52:23.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:23 smithi086 ceph-mon[30943]: pgmap v346: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-05T06:52:23.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195658 1.64 MB/sec execute 463 sec latency 18.535 ms 2024-09-05T06:52:24.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 196164 1.64 MB/sec execute 464 sec latency 20.883 ms 2024-09-05T06:52:25.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:25 smithi096 ceph-mon[34468]: pgmap v347: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-05T06:52:25.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:25 smithi115 ceph-mon[34791]: pgmap v347: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-05T06:52:25.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:25 smithi086 ceph-mon[30943]: pgmap v347: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-05T06:52:25.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 196719 1.64 MB/sec execute 465 sec latency 20.270 ms 2024-09-05T06:52:26.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 197324 1.64 MB/sec execute 466 sec latency 20.897 ms 2024-09-05T06:52:27.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:27 smithi096 ceph-mon[34468]: pgmap v348: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:52:27.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:27 smithi115 ceph-mon[34791]: pgmap v348: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:52:27.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:27 smithi086 ceph-mon[30943]: pgmap v348: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:52:27.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 197989 1.65 MB/sec execute 467 sec latency 16.152 ms 2024-09-05T06:52:28.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 198594 1.66 MB/sec execute 468 sec latency 16.385 ms 2024-09-05T06:52:29.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:29 smithi096 ceph-mon[34468]: pgmap v349: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-05T06:52:29.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:29 smithi115 ceph-mon[34791]: pgmap v349: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-05T06:52:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:29 smithi086 ceph-mon[30943]: pgmap v349: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.7 MiB/s wr, 150 op/s 2024-09-05T06:52:29.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 198710 1.66 MB/sec execute 469 sec latency 758.809 ms 2024-09-05T06:52:30.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 198710 1.65 MB/sec execute 470 sec latency 1758.906 ms 2024-09-05T06:52:31.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:31 smithi096 ceph-mon[34468]: pgmap v350: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-09-05T06:52:31.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:31 smithi115 ceph-mon[34791]: pgmap v350: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-09-05T06:52:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:31 smithi086 ceph-mon[30943]: pgmap v350: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-09-05T06:52:31.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 198710 1.65 MB/sec execute 471 sec latency 2759.011 ms 2024-09-05T06:52:32.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:52:32.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 198933 1.65 MB/sec execute 472 sec latency 2971.661 ms 2024-09-05T06:52:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 -- 172.21.15.86:0/201252287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c102790 msgr2=0x7fa76c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 --2- 172.21.15.86:0/201252287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c102790 0x7fa76c1067e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa760009860 tx=0x7fa76002a340 comp rx=0 tx=0).stop 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 -- 172.21.15.86:0/201252287 shutdown_connections 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 --2- 172.21.15.86:0/201252287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c106f10 0x7fa76c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 --2- 172.21.15.86:0/201252287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c102790 0x7fa76c1067e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 --2- 172.21.15.86:0/201252287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 -- 172.21.15.86:0/201252287 >> 172.21.15.86:0/201252287 conn(0x7fa76c0fd8c0 msgr2=0x7fa76c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 -- 172.21.15.86:0/201252287 shutdown_connections 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.027+0000 7fa772773640 1 -- 172.21.15.86:0/201252287 wait complete. 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.028+0000 7fa772773640 1 Processor -- start 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.028+0000 7fa772773640 1 -- start start 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.028+0000 7fa772773640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa772773640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c102790 0x7fa76c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa772773640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 0x7fa76c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa772773640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c111de0 con 0x7fa76c101de0 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa772773640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c111c60 con 0x7fa76c106f10 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa772773640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c111f60 con 0x7fa76c102790 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46100/0 (socket says 172.21.15.86:46100) 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 -- 172.21.15.86:0/1395674656 learned_addr learned my addr 172.21.15.86:0/1395674656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c102790 0x7fa76c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa770d10640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 0x7fa76c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 -- 172.21.15.86:0/1395674656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c102790 msgr2=0x7fa76c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c102790 0x7fa76c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 -- 172.21.15.86:0/1395674656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 msgr2=0x7fa76c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 0x7fa76c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 -- 172.21.15.86:0/1395674656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c19d310 con 0x7fa76c101de0 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa770d10640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 0x7fa76c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa76bfff640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00e9c0 tx=0x7fa75c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa7697fa640 1 -- 172.21.15.86:0/1395674656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00cde0 con 0x7fa76c101de0 2024-09-05T06:52:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa7697fa640 1 -- 172.21.15.86:0/1395674656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa75c0035f0 con 0x7fa76c101de0 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.029+0000 7fa7697fa640 1 -- 172.21.15.86:0/1395674656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c010690 con 0x7fa76c101de0 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.030+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c19d600 con 0x7fa76c101de0 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.030+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1a4ee0 con 0x7fa76c101de0 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.032+0000 7fa7697fa640 1 -- 172.21.15.86:0/1395674656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa75c0037a0 con 0x7fa76c101de0 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.032+0000 7fa774b4c740 1 Processor -- start 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.032+0000 7fa774b4c740 1 -- start start 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.032+0000 7fa7697fa640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 0x7fa7540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.032+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 0x558436d0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d0ca90 0x558436d0ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa774b4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558436c632e0 con 0x558436d0ca90 2024-09-05T06:52:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa774b4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558436c0ace0 con 0x558436d021d0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa774b4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558436c5f2a0 con 0x558436d018b0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa7697fa640 1 -- 172.21.15.86:0/1395674656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fa75c0cfb40 con 0x7fa76c101de0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa770d10640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 0x7fa7540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36316/0 (socket says 172.21.15.86:36316) 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 -- 172.21.15.86:0/2642274847 learned_addr learned my addr 172.21.15.86:0/2642274847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa770d10640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d0ca90 0x558436d0ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 0x558436d0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 -- 172.21.15.86:0/2642274847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 msgr2=0x558436d0c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 0x558436d0c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 -- 172.21.15.86:0/2642274847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d0ca90 msgr2=0x558436d0ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d0ca90 0x558436d0ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 -- 172.21.15.86:0/2642274847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558436bef440 con 0x558436d018b0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 0x558436d0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa770d10640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 0x7fa7540a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa764009fd0 tx=0x7fa7640116b0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa75c098590 tx=0x7fa75c09b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34ca8f68ea4d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa7527fc640 1 -- 172.21.15.86:0/2642274847 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c04a520 con 0x558436d018b0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa7527fc640 1 -- 172.21.15.86:0/2642274847 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa75c003b10 con 0x558436d018b0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.033+0000 7fa7527fc640 1 -- 172.21.15.86:0/2642274847 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0a4830 con 0x558436d018b0 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 -- 172.21.15.86:0/2642274847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 msgr2=0x558436d01c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa75c098590 tx=0x7fa75c09b830 comp rx=0 tx=0).stop 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 -- 172.21.15.86:0/2642274847 shutdown_connections 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d0ca90 0x558436d0ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d021d0 0x558436d0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2642274847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d01c90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 -- 172.21.15.86:0/2642274847 >> 172.21.15.86:0/2642274847 conn(0x558436cfde40 msgr2=0x558436cfe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 -- 172.21.15.86:0/2642274847 shutdown_connections 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.034+0000 7fa774b4c740 1 -- 172.21.15.86:0/2642274847 wait complete. 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 -- start start 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 Processor -- start 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d021d0 0x558436d09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558436d0f660 con 0x558436d018b0 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558436c1e050 con 0x558436d0ca90 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558436d0f7e0 con 0x558436d021d0 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa774b4c740 10 client.0 _ll_register_callbacks cb 0x558436c5e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46110/0 (socket says 172.21.15.86:46110) 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 -- 172.21.15.86:0/3117459967 learned_addr learned my addr 172.21.15.86:0/3117459967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d021d0 0x558436d09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa770d10640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 -- 172.21.15.86:0/3117459967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d021d0 msgr2=0x558436d09750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d021d0 0x558436d09750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 -- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 msgr2=0x558436d09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d09d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.035+0000 7fa76bfff640 1 -- 172.21.15.86:0/3117459967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584369d43c0 con 0x558436d018b0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa76bfff640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c6a0 tx=0x7fa75c048de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18713d460316e2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c09bbc0 con 0x558436d018b0 2024-09-05T06:52:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fa75c0cd070 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558436c70850 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa75c09c2c0 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 10 client.18018 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558436d1b060 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558436c70850 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fa75c09aa80 con 0x558436d018b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 10 client.18018 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 20 client.18018 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d1c3b0 0x558436d1e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 4 client.18018 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.036+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558436c6ad60 con 0x558436d1c3b0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.037+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d1c3b0 0x558436d1e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.037+0000 7fa74b7fe640 10 client.18018 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.037+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d1c3b0 0x558436d1e7a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8eaa82787f75bc server_cookie=5406d8f74b02bad5 in_seq=0 out_seq=0 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa74b7fe640 1 -- 172.21.15.86:0/3117459967 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558436c6ad60 con 0x558436d1c3b0 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa74b7fe640 10 client.18018 handle_command_reply: tid=0 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 client.18018 shutdown 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 msgr2=0x558436d0c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c6a0 tx=0x7fa75c048de0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 shutdown_connections 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d1c3b0 0x558436d1e7a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d021d0 0x558436d09750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 --2- 172.21.15.86:0/3117459967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d018b0 0x558436d0c4a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 >> 172.21.15.86:0/3117459967 conn(0x558436cfde40 msgr2=0x558436cf8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa74b7fe640 0 client.18018 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 shutdown_connections 2024-09-05T06:52:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 1 -- 172.21.15.86:0/3117459967 wait complete. 2024-09-05T06:52:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.038+0000 7fa774b4c740 20 client.18018 trim_cache size 0 max 16384 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 Processor -- start 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 -- start start 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d18a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d06ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1acf70 con 0x558436d021d0 2024-09-05T06:52:33.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1aeb40 con 0x558436d0ca90 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa774b4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ad640 con 0x558436d018b0 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46112/0 (socket says 172.21.15.86:46112) 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.057+0000 7fa76b7fe640 1 -- 172.21.15.86:0/1134594401 learned_addr learned my addr 172.21.15.86:0/1134594401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa770d10640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d06ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76bfff640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d18a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 -- 172.21.15.86:0/1134594401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 msgr2=0x558436d18a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d18a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 -- 172.21.15.86:0/1134594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 msgr2=0x558436d06ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d06ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 -- 172.21.15.86:0/1134594401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558436d016e0 con 0x558436d021d0 2024-09-05T06:52:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76bfff640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d18a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa770d10640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d06ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa760009780 tx=0x7fa760005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d128fe56706d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa751ffb640 1 -- 172.21.15.86:0/1134594401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76004c070 con 0x558436d021d0 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa751ffb640 1 -- 172.21.15.86:0/1134594401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa760038040 con 0x558436d021d0 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.058+0000 7fa751ffb640 1 -- 172.21.15.86:0/1134594401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760037040 con 0x558436d021d0 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- 172.21.15.86:0/1134594401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 msgr2=0x558436d069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa760009780 tx=0x7fa760005ae0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- 172.21.15.86:0/1134594401 shutdown_connections 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d0ca90 0x558436d06ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d069b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 --2- 172.21.15.86:0/1134594401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d018b0 0x558436d18a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- 172.21.15.86:0/1134594401 >> 172.21.15.86:0/1134594401 conn(0x558436cfde40 msgr2=0x558436d08840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- 172.21.15.86:0/1134594401 shutdown_connections 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- 172.21.15.86:0/1134594401 wait complete. 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.059+0000 7fa774b4c740 1 -- start start 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 Processor -- start 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d21aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 0x558436d1e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ae410 con 0x558436d021d0 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1add10 con 0x558436d018b0 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1aaf70 con 0x558436d0ca90 2024-09-05T06:52:33.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa774b4c740 10 client.0 _ll_register_callbacks cb 0x7fa764011b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d21aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55554/0 (socket says 172.21.15.86:55554) 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 -- 172.21.15.86:0/2422013307 learned_addr learned my addr 172.21.15.86:0/2422013307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa770d10640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 0x558436d1e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 -- 172.21.15.86:0/2422013307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 msgr2=0x558436d1e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 0x558436d1e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 -- 172.21.15.86:0/2422013307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 msgr2=0x558436d21aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d21aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa76bfff640 1 -- 172.21.15.86:0/2422013307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558436d0a6f0 con 0x558436d018b0 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.060+0000 7fa770d10640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 0x558436d1e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d21aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa76bfff640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa75c09bb00 tx=0x7fa75c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a6fc8bcd8367e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c09b700 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa760038040 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fa75c0a1310 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa75c097e60 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 10 client.27134 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558436d0a6f0 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa760038040 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fa75c0cd070 con 0x558436d018b0 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 10 client.27134 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 10 client.27134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.061+0000 7fa774b4c740 20 client.27134 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d86950 0x558436d88d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 4 client.27134 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558436be7210 con 0x558436d86950 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d86950 0x558436d88d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa74b7fe640 10 client.27134 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d86950 0x558436d88d60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e766b5bc733e38 server_cookie=fd6dc3736cc26ffa in_seq=0 out_seq=0 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa74b7fe640 1 -- 172.21.15.86:0/2422013307 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558436be7210 con 0x558436d86950 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa74b7fe640 10 client.27134 handle_command_reply: tid=0 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 client.27134 shutdown 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 msgr2=0x558436d21560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa75c09bb00 tx=0x7fa75c00c7d0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.062+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 shutdown_connections 2024-09-05T06:52:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558436d86950 0x558436d88d60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558436d0ca90 0x558436d1e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558436d021d0 0x558436d21aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 --2- 172.21.15.86:0/2422013307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558436d018b0 0x558436d21560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa74b7fe640 0 client.27134 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 >> 172.21.15.86:0/2422013307 conn(0x558436cfde40 msgr2=0x558436cf8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 shutdown_connections 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 1 -- 172.21.15.86:0/2422013307 wait complete. 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa774b4c740 20 client.27134 trim_cache size 0 max 16384 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:52:33.070 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:52:33.071 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:52:33.071 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:52:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 msgr2=0x7fa7540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 0x7fa7540a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa764009fd0 tx=0x7fa7640116b0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 msgr2=0x7fa76c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00e9c0 tx=0x7fa75c00ee80 comp rx=0 tx=0).stop 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 shutdown_connections 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7540a2d70 0x7fa7540a5230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa76c106f10 0x7fa76c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa76c102790 0x7fa76c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 --2- 172.21.15.86:0/1395674656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa76c101de0 0x7fa76c198360 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.063+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 >> 172.21.15.86:0/1395674656 conn(0x7fa76c0fd8c0 msgr2=0x7fa76c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.064+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 shutdown_connections 2024-09-05T06:52:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:33.064+0000 7fa772773640 1 -- 172.21.15.86:0/1395674656 wait complete. 2024-09-05T06:52:33.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:33 smithi086 ceph-mon[30943]: pgmap v351: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-05T06:52:33.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:33 smithi096 ceph-mon[34468]: pgmap v351: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-05T06:52:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:33 smithi115 ceph-mon[34791]: pgmap v351: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-05T06:52:33.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 199196 1.65 MB/sec execute 473 sec latency 47.127 ms 2024-09-05T06:52:33.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:52:33.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:52:34.237 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:34.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 199657 1.64 MB/sec execute 474 sec latency 45.899 ms 2024-09-05T06:52:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 -- 172.21.15.86:0/439825829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84100e20 msgr2=0x7f3b84101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 --2- 172.21.15.86:0/439825829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84100e20 0x7f3b84101200 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009900 tx=0x7f3b7402a480 comp rx=0 tx=0).stop 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 -- 172.21.15.86:0/439825829 shutdown_connections 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 --2- 172.21.15.86:0/439825829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 0x7f3b8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 --2- 172.21.15.86:0/439825829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84102570 0x7f3b841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 --2- 172.21.15.86:0/439825829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84100e20 0x7f3b84101200 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 -- 172.21.15.86:0/439825829 >> 172.21.15.86:0/439825829 conn(0x7f3b840fc7d0 msgr2=0x7f3b840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 -- 172.21.15.86:0/439825829 shutdown_connections 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.708+0000 7f3b88bca640 1 -- 172.21.15.86:0/439825829 wait complete. 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 Processor -- start 2024-09-05T06:52:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 -- start start 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84100e20 0x7f3b841972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 0x7f3b8419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84110e30 con 0x7f3b84102570 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84110cb0 con 0x7f3b84100e20 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b88bca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84110fb0 con 0x7f3b84102ef0 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b8259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84100e20 0x7f3b841972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46142/0 (socket says 172.21.15.86:46142) 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 -- 172.21.15.86:0/1498107486 learned_addr learned my addr 172.21.15.86:0/1498107486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b82d9d640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 0x7f3b8419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:34.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 -- 172.21.15.86:0/1498107486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 msgr2=0x7f3b8419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 0x7f3b8419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 -- 172.21.15.86:0/1498107486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84100e20 msgr2=0x7f3b841972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84100e20 0x7f3b841972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.709+0000 7f3b7bfff640 1 -- 172.21.15.86:0/1498107486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8419c250 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b7bfff640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00d950 tx=0x7f3b6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c017030 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8419c540 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b6c004020 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0047a0 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.710+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b84069f70 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.711+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b84101200 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.712+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b6c0041d0 con 0x7f3b84102570 2024-09-05T06:52:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.713+0000 7f3b7b7fe640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 0x7f3b700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.713+0000 7f3b8259c640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 0x7f3b700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.713+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7f3b6c0cf9c0 con 0x7f3b84102570 2024-09-05T06:52:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.714+0000 7f3b8259c640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 0x7f3b700a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009230 tx=0x7f3b7403a040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.717+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b6c09c1a0 con 0x7f3b84102570 2024-09-05T06:52:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.948+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b84198600 con 0x7f3b84102570 2024-09-05T06:52:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.948+0000 7f3b7b7fe640 1 -- 172.21.15.86:0/1498107486 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11978 (secure 0 0 0) 0x7f3b6c09baa0 con 0x7f3b84102570 2024-09-05T06:52:34.949 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:52:34.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"btime":"2024-09-05T06:51:30:994870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:51:30.398269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:52:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-09-05T06:52:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 msgr2=0x7f3b700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 0x7f3b700a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009230 tx=0x7f3b7403a040 comp rx=0 tx=0).stop 2024-09-05T06:52:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 msgr2=0x7f3b841977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00d950 tx=0x7f3b6c00de10 comp rx=0 tx=0).stop 2024-09-05T06:52:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 shutdown_connections 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b700a2cf0 0x7f3b700a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b84102ef0 0x7f3b8419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b84102570 0x7f3b841977e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 --2- 172.21.15.86:0/1498107486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b84100e20 0x7f3b841972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 >> 172.21.15.86:0/1498107486 conn(0x7f3b840fc7d0 msgr2=0x7f3b840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 shutdown_connections 2024-09-05T06:52:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:34.951+0000 7f3b88bca640 1 -- 172.21.15.86:0/1498107486 wait complete. 2024-09-05T06:52:35.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:35 smithi086 ceph-mon[30943]: pgmap v352: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:35.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:35 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1498107486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:35.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:35 smithi096 ceph-mon[34468]: pgmap v352: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:35.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:35 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1498107486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:35.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:35 smithi115 ceph-mon[34791]: pgmap v352: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:35 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1498107486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:35.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 200198 1.65 MB/sec execute 475 sec latency 22.068 ms 2024-09-05T06:52:35.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:52:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:36.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:36 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:52:36.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:36 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:36 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.348 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:36 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:52:36.348 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:36 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.348 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:36 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:36 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:52:36.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:36 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:36 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 -- 172.21.15.86:0/1176349893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100430 msgr2=0x7ff758100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 --2- 172.21.15.86:0/1176349893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100430 0x7ff758100810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff748009860 tx=0x7ff74802a340 comp rx=0 tx=0).stop 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 -- 172.21.15.86:0/1176349893 shutdown_connections 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 --2- 172.21.15.86:0/1176349893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 --2- 172.21.15.86:0/1176349893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100de0 0x7ff758104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 --2- 172.21.15.86:0/1176349893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100430 0x7ff758100810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 -- 172.21.15.86:0/1176349893 >> 172.21.15.86:0/1176349893 conn(0x7ff758077230 msgr2=0x7ff7580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 -- 172.21.15.86:0/1176349893 shutdown_connections 2024-09-05T06:52:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.453+0000 7ff75e2de640 1 -- 172.21.15.86:0/1176349893 wait complete. 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.454+0000 7ff75e2de640 1 Processor -- start 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.454+0000 7ff75e2de640 1 -- start start 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.454+0000 7ff75e2de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.454+0000 7ff75e2de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100de0 0x7ff75819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.454+0000 7ff75e2de640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75e2de640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff758113fb0 con 0x7ff758100430 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75e2de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff758113e30 con 0x7ff758100de0 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75e2de640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff758114130 con 0x7ff758105560 2024-09-05T06:52:36.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46166/0 (socket says 172.21.15.86:46166) 2024-09-05T06:52:36.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff757fff640 1 -- 172.21.15.86:0/2965086356 learned_addr learned my addr 172.21.15.86:0/2965086356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:36.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100de0 0x7ff75819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 -- 172.21.15.86:0/2965086356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100de0 msgr2=0x7ff75819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100de0 0x7ff75819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 -- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 msgr2=0x7ff75819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 -- 172.21.15.86:0/2965086356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75819f4e0 con 0x7ff758105560 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff757fff640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75819ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff750009fd0 tx=0x7ff75000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.455+0000 7ff7557fa640 1 -- 172.21.15.86:0/2965086356 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75000cd60 con 0x7ff758105560 2024-09-05T06:52:36.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.456+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75819f7d0 con 0x7ff758105560 2024-09-05T06:52:36.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.456+0000 7ff7557fa640 1 -- 172.21.15.86:0/2965086356 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff750005020 con 0x7ff758105560 2024-09-05T06:52:36.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.456+0000 7ff7557fa640 1 -- 172.21.15.86:0/2965086356 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750010690 con 0x7ff758105560 2024-09-05T06:52:36.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.456+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a70b0 con 0x7ff758105560 2024-09-05T06:52:36.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.458+0000 7ff7557fa640 1 -- 172.21.15.86:0/2965086356 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff750004b20 con 0x7ff758105560 2024-09-05T06:52:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 Processor -- start 2024-09-05T06:52:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 -- start start 2024-09-05T06:52:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7557fa640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 0x7ff7400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4986072f0 con 0x55e4986a15d0 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4985aecf0 con 0x55e4986a2b40 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7606b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e4986032b0 con 0x55e4986ac370 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 0x7ff7400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7557fa640 1 -- 172.21.15.86:0/2965086356 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff7500cfde0 con 0x7ff758105560 2024-09-05T06:52:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55600/0 (socket says 172.21.15.86:55600) 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff757fff640 1 -- 172.21.15.86:0/566925461 learned_addr learned my addr 172.21.15.86:0/566925461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.459+0000 7ff7577fe640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff75c87b640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 -- 172.21.15.86:0/566925461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 msgr2=0x55e4986ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 -- 172.21.15.86:0/566925461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 msgr2=0x55e4986a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 -- 172.21.15.86:0/566925461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4985933a0 con 0x55e4986a2b40 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff7577fe640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff75c87b640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff757fff640 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff748035040 tx=0x7ff748005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c9ea64e951d669 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff73e7fc640 1 -- 172.21.15.86:0/566925461 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748043020 con 0x55e4986a2b40 2024-09-05T06:52:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff73e7fc640 1 -- 172.21.15.86:0/566925461 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff748038040 con 0x55e4986a2b40 2024-09-05T06:52:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff73e7fc640 1 -- 172.21.15.86:0/566925461 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748037040 con 0x55e4986a2b40 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.460+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 0x7ff7400a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e49869eac0 tx=0x7ff74c00a900 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- 172.21.15.86:0/566925461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 msgr2=0x55e4986a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff748035040 tx=0x7ff748005dc0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- 172.21.15.86:0/566925461 shutdown_connections 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 --2- 172.21.15.86:0/566925461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a1090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- 172.21.15.86:0/566925461 >> 172.21.15.86:0/566925461 conn(0x55e49866a310 msgr2=0x55e49869dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- 172.21.15.86:0/566925461 shutdown_connections 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- 172.21.15.86:0/566925461 wait complete. 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.461+0000 7ff7606b7740 1 -- start start 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 Processor -- start 2024-09-05T06:52:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986a15d0 0x55e4986abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4986af010 con 0x55e4986ac370 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4985c2060 con 0x55e4986a2b40 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e4986af190 con 0x55e4986a15d0 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7606b7740 10 client.0 _ll_register_callbacks cb 0x55e4983158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff75c87b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff75c87b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46182/0 (socket says 172.21.15.86:46182) 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55614/0 (socket says 172.21.15.86:55614) 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff75c87b640 1 -- 172.21.15.86:0/2904808317 learned_addr learned my addr 172.21.15.86:0/2904808317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff757fff640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986a15d0 0x55e4986abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff75c87b640 1 -- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986a15d0 msgr2=0x55e4986abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.462+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986a15d0 0x55e4986abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff75c87b640 1 -- 172.21.15.86:0/2904808317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 msgr2=0x55e4986a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff75c87b640 1 -- 172.21.15.86:0/2904808317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4983783c0 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff7500cd150 tx=0x7ff75009b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577814123afca52b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b8070 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff7500051d0 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff748038040 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff7500a0270 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7606b7740 10 client.18042 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.463+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4986c3210 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff748038040 con 0x55e4986ac370 2024-09-05T06:52:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff75007ace0 con 0x55e4986ac370 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 10 client.18042 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 10 client.18042 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 20 client.18042 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e4986c4670 0x55e4986c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 4 client.18042 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e498614860 con 0x55e4986c4670 2024-09-05T06:52:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.464+0000 7ff757fff640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e4986c4670 0x55e4986c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7377fe640 10 client.18042 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:52:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff757fff640 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e4986c4670 0x55e4986c6b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4cd35d1addbf1035 server_cookie=412509ef48cb7581 in_seq=0 out_seq=0 2024-09-05T06:52:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7377fe640 1 -- 172.21.15.86:0/2904808317 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e498614860 con 0x55e4986c4670 2024-09-05T06:52:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7377fe640 10 client.18042 handle_command_reply: tid=0 2024-09-05T06:52:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7606b7740 1 client.18042 shutdown 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 msgr2=0x55e4986a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff7500cd150 tx=0x7ff75009b8e0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 shutdown_connections 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.465+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e4986c4670 0x55e4986c6b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986ac370 0x55e4986a94a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2904808317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986a15d0 0x55e4986abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7377fe640 0 client.18042 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 >> 172.21.15.86:0/2904808317 conn(0x55e49866a310 msgr2=0x55e4986a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 shutdown_connections 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 1 -- 172.21.15.86:0/2904808317 wait complete. 2024-09-05T06:52:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.466+0000 7ff7606b7740 20 client.18042 trim_cache size 0 max 16384 2024-09-05T06:52:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.482+0000 7ff7606b7740 1 Processor -- start 2024-09-05T06:52:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.482+0000 7ff7606b7740 1 -- start start 2024-09-05T06:52:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.482+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.482+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a2b40 0x55e4986a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff7606b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ace30 con 0x55e4986a2b40 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff7606b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581aea00 con 0x55e4986a15d0 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff7606b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ad500 con 0x55e4986ac370 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55620/0 (socket says 172.21.15.86:55620) 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 -- 172.21.15.86:0/4278560439 learned_addr learned my addr 172.21.15.86:0/4278560439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a2b40 0x55e4986a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff75c87b640 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 -- 172.21.15.86:0/4278560439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 msgr2=0x55e4986a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 -- 172.21.15.86:0/4278560439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a2b40 msgr2=0x55e4986a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a2b40 0x55e4986a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 -- 172.21.15.86:0/4278560439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4986a1d50 con 0x55e4986a15d0 2024-09-05T06:52:36.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff757fff640 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff748030860 tx=0x7ff7480307f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2debca3d6ea6a0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.483+0000 7ff73dffb640 1 -- 172.21.15.86:0/4278560439 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74804e070 con 0x55e4986a15d0 2024-09-05T06:52:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff73dffb640 1 -- 172.21.15.86:0/4278560439 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff748030dd0 con 0x55e4986a15d0 2024-09-05T06:52:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff73dffb640 1 -- 172.21.15.86:0/4278560439 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748037040 con 0x55e4986a15d0 2024-09-05T06:52:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 -- 172.21.15.86:0/4278560439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 msgr2=0x55e4986c0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff748030860 tx=0x7ff7480307f0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 -- 172.21.15.86:0/4278560439 shutdown_connections 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a2b40 0x55e4986a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 --2- 172.21.15.86:0/4278560439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a15d0 0x55e4986c0b70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 -- 172.21.15.86:0/4278560439 >> 172.21.15.86:0/4278560439 conn(0x55e49866a310 msgr2=0x55e4986a8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 -- 172.21.15.86:0/4278560439 shutdown_connections 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.484+0000 7ff7606b7740 1 -- 172.21.15.86:0/4278560439 wait complete. 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 -- start start 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 Processor -- start 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ae2d0 con 0x55e4986a15d0 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581adbd0 con 0x55e4986a2b40 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581aae30 con 0x55e4986ac370 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff7606b7740 10 client.0 _ll_register_callbacks cb 0x7ff750005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.485+0000 7ff757fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46212/0 (socket says 172.21.15.86:46212) 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff757fff640 1 -- 172.21.15.86:0/2140343774 learned_addr learned my addr 172.21.15.86:0/2140343774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff75c87b640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 -- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 msgr2=0x55e4986b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 -- 172.21.15.86:0/2140343774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 msgr2=0x55e4986b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 -- 172.21.15.86:0/2140343774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49872d5c0 con 0x55e4986a2b40 2024-09-05T06:52:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7577fe640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986b81e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55e4986a87f0 tx=0x7ff74c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12cef5d358c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7377fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c02d060 con 0x55e4986a2b40 2024-09-05T06:52:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff74c01d6f0 con 0x55e4986a2b40 2024-09-05T06:52:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.486+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff748030dd0 con 0x55e4986a2b40 2024-09-05T06:52:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff74c034050 con 0x55e4986a2b40 2024-09-05T06:52:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 10 client.27158 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e49872e900 con 0x55e4986a2b40 2024-09-05T06:52:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff748030dd0 con 0x55e4986a2b40 2024-09-05T06:52:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff74c00b040 con 0x55e4986a2b40 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 10 client.27158 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 10 client.27158 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 20 client.27158 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e49872f020 0x55e498731440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 4 client.27158 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.487+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e49860ed70 con 0x55e49872f020 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff757fff640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e49872f020 0x55e498731440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff757fff640 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e49872f020 0x55e498731440 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f4439c4b34a8e7 server_cookie=87939ba639534698 in_seq=0 out_seq=0 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7377fe640 10 client.27158 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7377fe640 1 -- 172.21.15.86:0/2140343774 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e49860ed70 con 0x55e49872f020 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7377fe640 10 client.27158 handle_command_reply: tid=0 2024-09-05T06:52:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 client.27158 shutdown 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 msgr2=0x55e4986b81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986b81e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55e4986a87f0 tx=0x7ff74c0046c0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 shutdown_connections 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e49872f020 0x55e498731440 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4986ac370 0x55e4986b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4986a2b40 0x55e4986b81e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 --2- 172.21.15.86:0/2140343774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4986a15d0 0x55e4986b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.488+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 >> 172.21.15.86:0/2140343774 conn(0x55e49866a310 msgr2=0x55e49869c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff7377fe640 0 client.27158 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 shutdown_connections 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff7606b7740 1 -- 172.21.15.86:0/2140343774 wait complete. 2024-09-05T06:52:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff7606b7740 20 client.27158 trim_cache size 0 max 16384 2024-09-05T06:52:36.493 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:52:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 msgr2=0x7ff7400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 0x7ff7400a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e49869eac0 tx=0x7ff74c00a900 comp rx=0 tx=0).stop 2024-09-05T06:52:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 msgr2=0x7ff75819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75819ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff750009fd0 tx=0x7ff75000ee50 comp rx=0 tx=0).stop 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 shutdown_connections 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff7400a2d70 0x7ff7400a5230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff758105560 0x7ff75819ee00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff758100de0 0x7ff75819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 --2- 172.21.15.86:0/2965086356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758100430 0x7ff75819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 >> 172.21.15.86:0/2965086356 conn(0x7ff758077230 msgr2=0x7ff758103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.489+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 shutdown_connections 2024-09-05T06:52:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:36.490+0000 7ff75e2de640 1 -- 172.21.15.86:0/2965086356 wait complete. 2024-09-05T06:52:36.532 INFO:tasks.workunit.client.0.smithi086.stdout: 1 200704 1.65 MB/sec execute 476 sec latency 28.339 ms 2024-09-05T06:52:37.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:37 smithi086 ceph-mon[30943]: pgmap v353: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:37.164 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:52:37.164 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:52:37.164 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:37.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:52:37.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:37 smithi096 ceph-mon[34468]: pgmap v353: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:37.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:52:37.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:52:37.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:37 smithi115 ceph-mon[34791]: pgmap v353: 129 pgs: 129 active+clean; 220 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:52:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:52:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:52:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:37.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 201239 1.65 MB/sec execute 477 sec latency 19.056 ms 2024-09-05T06:52:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 -- 172.21.15.86:0/3759001094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8100e20 msgr2=0x7fd5c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/3759001094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8100e20 0x7fd5c8101200 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009900 tx=0x7fd5b802a480 comp rx=0 tx=0).stop 2024-09-05T06:52:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 -- 172.21.15.86:0/3759001094 shutdown_connections 2024-09-05T06:52:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/3759001094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102ef0 0x7fd5c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/3759001094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8102570 0x7fd5c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/3759001094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8100e20 0x7fd5c8101200 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 -- 172.21.15.86:0/3759001094 >> 172.21.15.86:0/3759001094 conn(0x7fd5c80fc7d0 msgr2=0x7fd5c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 -- 172.21.15.86:0/3759001094 shutdown_connections 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.928+0000 7fd5cdb54640 1 -- 172.21.15.86:0/3759001094 wait complete. 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 Processor -- start 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 -- start start 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102570 0x7fd5c8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8110e00 con 0x7fd5c8102ef0 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8110c80 con 0x7fd5c8100e20 2024-09-05T06:52:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5cdb54640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8110f80 con 0x7fd5c8102570 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55654/0 (socket says 172.21.15.86:55654) 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5c77fe640 1 -- 172.21.15.86:0/1463295623 learned_addr learned my addr 172.21.15.86:0/1463295623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.929+0000 7fd5bffff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102570 0x7fd5c8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 -- 172.21.15.86:0/1463295623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102570 msgr2=0x7fd5c8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102570 0x7fd5c8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 -- 172.21.15.86:0/1463295623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 msgr2=0x7fd5c8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 -- 172.21.15.86:0/1463295623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c819d400 con 0x7fd5c8102ef0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c7fff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009fd0 tx=0x7fd5c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c57fa640 1 -- 172.21.15.86:0/1463295623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000ce20 con 0x7fd5c8102ef0 2024-09-05T06:52:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c57fa640 1 -- 172.21.15.86:0/1463295623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5c0005020 con 0x7fd5c8102ef0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c819d6f0 con 0x7fd5c8102ef0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5c57fa640 1 -- 172.21.15.86:0/1463295623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0010690 con 0x7fd5c8102ef0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.930+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81a4fd0 con 0x7fd5c8102ef0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.933+0000 7fd5cff2d740 1 Processor -- start 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.933+0000 7fd5c57fa640 1 -- 172.21.15.86:0/1463295623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd5c00051d0 con 0x7fd5c8102ef0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.933+0000 7fd5cff2d740 1 -- start start 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.933+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.933+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82446a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82453760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae823ac2f0 con 0x55ae82447b40 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae82353cf0 con 0x55ae824465d0 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae823a82b0 con 0x55ae82451370 2024-09-05T06:52:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82446a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46248/0 (socket says 172.21.15.86:46248) 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 -- 172.21.15.86:0/453279802 learned_addr learned my addr 172.21.15.86:0/453279802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c57fa640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 0x7fd5b40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82453760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 -- 172.21.15.86:0/453279802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 msgr2=0x55ae82453760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82453760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c7fff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 0x7fd5b40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 -- 172.21.15.86:0/453279802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 msgr2=0x55ae82446a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82446a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 -- 172.21.15.86:0/453279802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae823383a0 con 0x55ae82447b40 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82446a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c57fa640 1 -- 172.21.15.86:0/1463295623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd5c00cfe50 con 0x7fd5c8102ef0 2024-09-05T06:52:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bffff640 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005bc0 tx=0x7fd5b0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee19d8ca8d6c1067 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bdffb640 1 -- 172.21.15.86:0/453279802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0004050 con 0x55ae82447b40 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5c7fff640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 0x7fd5b40a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5c009bc30 tx=0x7fd5c00bc040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.934+0000 7fd5bdffb640 1 -- 172.21.15.86:0/453279802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b00041f0 con 0x55ae82447b40 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5bdffb640 1 -- 172.21.15.86:0/453279802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000f760 con 0x55ae82447b40 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 -- 172.21.15.86:0/453279802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 msgr2=0x55ae82446090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005bc0 tx=0x7fd5b0009850 comp rx=0 tx=0).stop 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 -- 172.21.15.86:0/453279802 shutdown_connections 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82453760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82446a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/453279802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae82446090 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 -- 172.21.15.86:0/453279802 >> 172.21.15.86:0/453279802 conn(0x55ae8240f310 msgr2=0x55ae82442c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 -- 172.21.15.86:0/453279802 shutdown_connections 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.935+0000 7fd5cff2d740 1 -- 172.21.15.86:0/453279802 wait complete. 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 -- start start 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 Processor -- start 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae824465d0 0x55ae82450ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae82454070 con 0x55ae82451370 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae82367060 con 0x55ae82447b40 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae824541f0 con 0x55ae824465d0 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5cff2d740 10 client.0 _ll_register_callbacks cb 0x55ae820ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46260/0 (socket says 172.21.15.86:46260) 2024-09-05T06:52:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5bffff640 1 -- 172.21.15.86:0/2697747045 learned_addr learned my addr 172.21.15.86:0/2697747045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.936+0000 7fd5bffff640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae824465d0 0x55ae82450ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 -- 172.21.15.86:0/2697747045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae824465d0 msgr2=0x55ae82450ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae824465d0 0x55ae82450ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 -- 172.21.15.86:0/2697747045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 msgr2=0x55ae8244e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 -- 172.21.15.86:0/2697747045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8211d3c0 con 0x55ae82447b40 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5bffff640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244df90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009230 tx=0x7fd5b8032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77937eef0ad37a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8035040 con 0x55ae82447b40 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b00041f0 con 0x55ae82447b40 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd5b80317f0 con 0x55ae82447b40 2024-09-05T06:52:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5b804f050 con 0x55ae82447b40 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5cff2d740 10 client.27170 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae8245fad0 con 0x55ae82447b40 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.937+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b00041f0 con 0x55ae82447b40 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd5b80046c0 con 0x55ae82447b40 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 10 client.27170 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 10 client.27170 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 20 client.27170 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae82460ee0 0x55ae824632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 4 client.27170 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae823b9860 con 0x55ae82460ee0 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5bffff640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae82460ee0 0x55ae824632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5a6ffd640 10 client.27170 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.938+0000 7fd5bffff640 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae82460ee0 0x55ae824632d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ebfa913f6ee8a29 server_cookie=c8d95876c71b9443 in_seq=0 out_seq=0 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/2697747045 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae823b9860 con 0x55ae82460ee0 2024-09-05T06:52:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5a6ffd640 10 client.27170 handle_command_reply: tid=0 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 client.27170 shutdown 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 msgr2=0x55ae8244df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244df90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009230 tx=0x7fd5b8032eb0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 shutdown_connections 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae82460ee0 0x55ae824632d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82451370 0x55ae8244e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244df90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5a6ffd640 0 client.27170 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/2697747045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae824465d0 0x55ae82450ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 >> 172.21.15.86:0/2697747045 conn(0x55ae8240f310 msgr2=0x55ae82448930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 shutdown_connections 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 1 -- 172.21.15.86:0/2697747045 wait complete. 2024-09-05T06:52:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.939+0000 7fd5cff2d740 20 client.27170 trim_cache size 0 max 16384 2024-09-05T06:52:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 Processor -- start 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 -- start start 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae824465d0 0x55ae8245d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82468b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81af120 con 0x55ae824465d0 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81b0cf0 con 0x55ae82447b40 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81af7f0 con 0x55ae82451370 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae824465d0 0x55ae8245d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55678/0 (socket says 172.21.15.86:55678) 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5c77fe640 1 -- 172.21.15.86:0/3348891860 learned_addr learned my addr 172.21.15.86:0/3348891860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5bffff640 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82468b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.953+0000 7fd5c77fe640 1 -- 172.21.15.86:0/3348891860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 msgr2=0x55ae82468b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82468b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5c77fe640 1 -- 172.21.15.86:0/3348891860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae824465d0 msgr2=0x55ae8245d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae824465d0 0x55ae8245d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5c77fe640 1 -- 172.21.15.86:0/3348891860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8244e860 con 0x55ae82447b40 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8004ac0 tx=0x7fd5b8004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c3d5920ee6cba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5bd7fa640 1 -- 172.21.15.86:0/3348891860 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8035070 con 0x55ae82447b40 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5bd7fa640 1 -- 172.21.15.86:0/3348891860 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b80048c0 con 0x55ae82447b40 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.954+0000 7fd5bd7fa640 1 -- 172.21.15.86:0/3348891860 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8005000 con 0x55ae82447b40 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3348891860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 msgr2=0x55ae8244cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8004ac0 tx=0x7fd5b8004f20 comp rx=0 tx=0).stop 2024-09-05T06:52:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3348891860 shutdown_connections 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae82468b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae82447b40 0x55ae8244cf70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3348891860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae824465d0 0x55ae8245d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3348891860 >> 172.21.15.86:0/3348891860 conn(0x55ae8240f310 msgr2=0x55ae8244c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3348891860 shutdown_connections 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3348891860 wait complete. 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 -- start start 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.955+0000 7fd5cff2d740 1 Processor -- start 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae8244aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae8244b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81b05c0 con 0x55ae82447b40 2024-09-05T06:52:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81afec0 con 0x55ae824465d0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81ad120 con 0x55ae82451370 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5cff2d740 10 client.0 _ll_register_callbacks cb 0x7fd5b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55684/0 (socket says 172.21.15.86:55684) 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 -- 172.21.15.86:0/3922983468 learned_addr learned my addr 172.21.15.86:0/3922983468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae8244aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae8244b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 -- 172.21.15.86:0/3922983468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 msgr2=0x55ae8244b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae8244b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 -- 172.21.15.86:0/3922983468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 msgr2=0x55ae8244aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae8244aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5bffff640 1 -- 172.21.15.86:0/3922983468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8244b5e0 con 0x55ae824465d0 2024-09-05T06:52:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.956+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae8244aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5bffff640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009ed0 tx=0x7fd5b0021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5043db126cb923c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b002a040 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd5b0003810 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b80048c0 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5b0003a10 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 10 client.27176 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ae824cb8e0 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b80048c0 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd5b003a040 con 0x55ae824465d0 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 10 client.27176 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:37.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 10 client.27176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 20 client.27176 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae824cbe10 0x55ae824ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 4 client.27176 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae823b3d70 con 0x55ae824cbe10 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.957+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae824cbe10 0x55ae824ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5c77fe640 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae824cbe10 0x55ae824ce200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d7aa95bca2175ac server_cookie=afbab2fcd591f594 in_seq=0 out_seq=0 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5a6ffd640 10 client.27176 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5a6ffd640 1 -- 172.21.15.86:0/3922983468 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae823b3d70 con 0x55ae824cbe10 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5a6ffd640 10 client.27176 handle_command_reply: tid=0 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 client.27176 shutdown 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 msgr2=0x55ae82459960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009ed0 tx=0x7fd5b0021040 comp rx=0 tx=0).stop 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 shutdown_connections 2024-09-05T06:52:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55ae824cbe10 0x55ae824ce200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae82451370 0x55ae8244b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae82447b40 0x55ae8244aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5a6ffd640 0 client.27176 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 --2- 172.21.15.86:0/3922983468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae824465d0 0x55ae82459960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 >> 172.21.15.86:0/3922983468 conn(0x55ae8240f310 msgr2=0x55ae824414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 shutdown_connections 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.958+0000 7fd5cff2d740 1 -- 172.21.15.86:0/3922983468 wait complete. 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cff2d740 20 client.27176 trim_cache size 0 max 16384 2024-09-05T06:52:37.964 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 msgr2=0x7fd5b40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 0x7fd5b40a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5c009bc30 tx=0x7fd5c00bc040 comp rx=0 tx=0).stop 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 msgr2=0x7fd5c819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009fd0 tx=0x7fd5c000ef10 comp rx=0 tx=0).stop 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 shutdown_connections 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd5b40a2d70 0x7fd5b40a5230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd20 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5c8102570 0x7fd5c8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 --2- 172.21.15.86:0/1463295623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:37.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 >> 172.21.15.86:0/1463295623 conn(0x7fd5c80fc7d0 msgr2=0x7fd5c806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:37.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 shutdown_connections 2024-09-05T06:52:37.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:37.959+0000 7fd5cdb54640 1 -- 172.21.15.86:0/1463295623 wait complete. 2024-09-05T06:52:38.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 201959 1.66 MB/sec execute 478 sec latency 14.081 ms 2024-09-05T06:52:38.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:52:39.066 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:39.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:39 smithi096 ceph-mon[34468]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-05T06:52:39.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:39 smithi086 ceph-mon[30943]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-05T06:52:39.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:39 smithi115 ceph-mon[34791]: pgmap v354: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-09-05T06:52:39.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202255 1.66 MB/sec execute 479 sec latency 451.682 ms 2024-09-05T06:52:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1000435603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 msgr2=0x7fb2f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1000435603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f810d950 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009cd0 tx=0x7fb2f002a8e0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1000435603 shutdown_connections 2024-09-05T06:52:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1000435603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f810d950 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1000435603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8100a80 0x7fb2f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1000435603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8106a80 0x7fb2f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1000435603 >> 172.21.15.86:0/1000435603 conn(0x7fb2f80fc7a0 msgr2=0x7fb2f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1000435603 shutdown_connections 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.526+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1000435603 wait complete. 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 Processor -- start 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 -- start start 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8100a80 0x7fb2f8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8106a80 0x7fb2f81a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8110d70 con 0x7fb2f8101420 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8110bf0 con 0x7fb2f8100a80 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2ffff2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8110ef0 con 0x7fb2f8106a80 2024-09-05T06:52:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2fd58d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2fd58d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46302/0 (socket says 172.21.15.86:46302) 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2fd58d640 1 -- 172.21.15.86:0/1731124201 learned_addr learned my addr 172.21.15.86:0/1731124201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8106a80 0x7fb2f81a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.527+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8100a80 0x7fb2f8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 -- 172.21.15.86:0/1731124201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8106a80 msgr2=0x7fb2f81a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8106a80 0x7fb2f81a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 -- 172.21.15.86:0/1731124201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8100a80 msgr2=0x7fb2f8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8100a80 0x7fb2f8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 -- 172.21.15.86:0/1731124201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f81a4840 con 0x7fb2f8101420 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d770 tx=0x7fb2e800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2eeffd640 1 -- 172.21.15.86:0/1731124201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8017070 con 0x7fb2f8101420 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2eeffd640 1 -- 172.21.15.86:0/1731124201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2e8009820 con 0x7fb2f8101420 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f81a4a10 con 0x7fb2f8101420 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2eeffd640 1 -- 172.21.15.86:0/1731124201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8004ba0 con 0x7fb2f8101420 2024-09-05T06:52:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.528+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f81a4ed0 con 0x7fb2f8101420 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.530+0000 7fb2eeffd640 1 -- 172.21.15.86:0/1731124201 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb2e80099d0 con 0x7fb2f8101420 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.530+0000 7fb3023cb740 1 Processor -- start 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 -- start start 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaacc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560deaa322f0 con 0x560deaacc5d0 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560dea9d9cf0 con 0x560deaadb780 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb3023cb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560deaa2e2b0 con 0x560deaacdb40 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2eeffd640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 0x7fb2e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46304/0 (socket says 172.21.15.86:46304) 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/229269363 learned_addr learned my addr 172.21.15.86:0/229269363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 0x7fb2e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaacc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2eeffd640 1 -- 172.21.15.86:0/1731124201 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb2e80cf2e0 con 0x7fb2f8101420 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/229269363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 msgr2=0x560deaacc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaacc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/229269363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 msgr2=0x560deaaddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/229269363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dea9be3a0 con 0x560deaacc5d0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.531+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a70 tx=0x7fb2f4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae5dd4606b08b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 0x7fb2e40a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0004af0 tx=0x7fb2f0004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb2e3fff640 1 -- 172.21.15.86:0/229269363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4017070 con 0x560deaacc5d0 2024-09-05T06:52:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb2e3fff640 1 -- 172.21.15.86:0/229269363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f4013470 con 0x560deaacc5d0 2024-09-05T06:52:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb2e3fff640 1 -- 172.21.15.86:0/229269363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4012630 con 0x560deaacc5d0 2024-09-05T06:52:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 -- 172.21.15.86:0/229269363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 msgr2=0x560deaacca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a70 tx=0x7fb2f4009f30 comp rx=0 tx=0).stop 2024-09-05T06:52:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 -- 172.21.15.86:0/229269363 shutdown_connections 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaacca10 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 --2- 172.21.15.86:0/229269363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaacc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.532+0000 7fb3023cb740 1 -- 172.21.15.86:0/229269363 >> 172.21.15.86:0/229269363 conn(0x560deaa95310 msgr2=0x560deaac8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- 172.21.15.86:0/229269363 shutdown_connections 2024-09-05T06:52:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- 172.21.15.86:0/229269363 wait complete. 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- start start 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 Processor -- start 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacc5d0 0x560deaadb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaad89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560deaade480 con 0x560deaacdb40 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560dea9ed060 con 0x560deaadb780 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560deaade600 con 0x560deaacc5d0 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.533+0000 7fb3023cb740 10 client.0 _ll_register_callbacks cb 0x560dea7408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacc5d0 0x560deaadb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46314/0 (socket says 172.21.15.86:46314) 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 -- 172.21.15.86:0/163629324 learned_addr learned my addr 172.21.15.86:0/163629324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaad89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 -- 172.21.15.86:0/163629324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacc5d0 msgr2=0x560deaadb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacc5d0 0x560deaadb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 -- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 msgr2=0x560deaad89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaad89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 -- 172.21.15.86:0/163629324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dea7a33c0 con 0x560deaacdb40 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaad89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb2e809b060 tx=0x7fb2e8017a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46485de3592a61c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80cf0d0 con 0x560deaacdb40 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f4013470 con 0x560deaacdb40 2024-09-05T06:52:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb2e80979d0 con 0x560deaacdb40 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2e809ba60 con 0x560deaacdb40 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb3023cb740 10 client.18078 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560deaae9f20 con 0x560deaacdb40 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.534+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f4013470 con 0x560deaacdb40 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fb2e80ccce0 con 0x560deaacdb40 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 10 client.18078 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 10 client.18078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 20 client.18078 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deaaeb270 0x560deaaed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 4 client.18078 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560deaa3f860 con 0x560deaaeb270 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deaaeb270 0x560deaaed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb2e17fa640 10 client.18078 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.535+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deaaeb270 0x560deaaed660 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=273b0aa88d85f9bf server_cookie=b3374a2598ef645c in_seq=0 out_seq=0 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb2e17fa640 1 -- 172.21.15.86:0/163629324 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560deaa3f860 con 0x560deaaeb270 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb2e17fa640 10 client.18078 handle_command_reply: tid=0 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 client.18078 shutdown 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 msgr2=0x560deaad8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb2e809b060 tx=0x7fb2e8017a70 comp rx=0 tx=0).stop 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 shutdown_connections 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deaaeb270 0x560deaaed660 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaad89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacdb40 0x560deaad8400 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 --2- 172.21.15.86:0/163629324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacc5d0 0x560deaadb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb2e17fa640 0 client.18078 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 >> 172.21.15.86:0/163629324 conn(0x560deaa95310 msgr2=0x560deaace930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 shutdown_connections 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 1 -- 172.21.15.86:0/163629324 wait complete. 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.536+0000 7fb3023cb740 20 client.18078 trim_cache size 0 max 16384 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 Processor -- start 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 -- start start 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaae7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaf2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81ad060 con 0x560deaacc5d0 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81aec30 con 0x560deaadb780 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb3023cb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81ad730 con 0x560deaacdb40 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46324/0 (socket says 172.21.15.86:46324) 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.549+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/700015592 learned_addr learned my addr 172.21.15.86:0/700015592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaae7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaf2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/700015592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 msgr2=0x560deaae7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaae7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/700015592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 msgr2=0x560deaaf2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaf2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/700015592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560deaad3fd0 con 0x560deaacc5d0 2024-09-05T06:52:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaae7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaf2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a70 tx=0x7fb2f4009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc4af9194b33aea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2e37fe640 1 -- 172.21.15.86:0/700015592 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f402b070 con 0x560deaacc5d0 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2e37fe640 1 -- 172.21.15.86:0/700015592 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f400edc0 con 0x560deaacc5d0 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.550+0000 7fb2e37fe640 1 -- 172.21.15.86:0/700015592 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4017040 con 0x560deaacc5d0 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- 172.21.15.86:0/700015592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 msgr2=0x560deaac8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a70 tx=0x7fb2f4009110 comp rx=0 tx=0).stop 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- 172.21.15.86:0/700015592 shutdown_connections 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaadb780 0x560deaaf2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaacdb40 0x560deaae7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 --2- 172.21.15.86:0/700015592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaac8e10 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- 172.21.15.86:0/700015592 >> 172.21.15.86:0/700015592 conn(0x560deaa95310 msgr2=0x560deaac8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- 172.21.15.86:0/700015592 shutdown_connections 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- 172.21.15.86:0/700015592 wait complete. 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 -- start start 2024-09-05T06:52:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.551+0000 7fb3023cb740 1 Processor -- start 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 0x560deaae0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaadb780 0x560deab54f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81ae500 con 0x560deaacc5d0 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81ade00 con 0x560deaacdb40 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81ab060 con 0x560deaadb780 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb3023cb740 10 client.0 _ll_register_callbacks cb 0x7fb2e80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fdd8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46340/0 (socket says 172.21.15.86:46340) 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fdd8e640 1 -- 172.21.15.86:0/3056934997 learned_addr learned my addr 172.21.15.86:0/3056934997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 0x560deaae0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fe58f640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaadb780 0x560deab54f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 -- 172.21.15.86:0/3056934997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaadb780 msgr2=0x560deab54f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaadb780 0x560deab54f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 -- 172.21.15.86:0/3056934997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 msgr2=0x560deaadfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 -- 172.21.15.86:0/3056934997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560deab55540 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2fd58d640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 0x560deaae0360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560deaad99b0 tx=0x7fb2e80053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eade68ff340899c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.552+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80b4350 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb2e80035f0 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f400edc0 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2e80037f0 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 10 client.27188 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560deab56770 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f400edc0 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fb2e80ccce0 con 0x560deaacdb40 2024-09-05T06:52:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 10 client.27188 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 10 client.27188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 20 client.27188 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deab56d90 0x560deab591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 4 client.27188 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560deaa39d70 con 0x560deab56d90 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.553+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deab56d90 0x560deab591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb2e17fa640 10 client.27188 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb2fdd8e640 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deab56d90 0x560deab591b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e94c9765a1a305d server_cookie=eac6e1d4a310c7e2 in_seq=0 out_seq=0 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb2e17fa640 1 -- 172.21.15.86:0/3056934997 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560deaa39d70 con 0x560deab56d90 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb2e17fa640 10 client.27188 handle_command_reply: tid=0 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 client.27188 shutdown 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 msgr2=0x560deaae0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 0x560deaae0360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560deaad99b0 tx=0x7fb2e80053b0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 shutdown_connections 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560deab56d90 0x560deab591b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560deaadb780 0x560deab54f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560deaacdb40 0x560deaae0360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb2e17fa640 0 client.27188 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 --2- 172.21.15.86:0/3056934997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560deaacc5d0 0x560deaadfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 >> 172.21.15.86:0/3056934997 conn(0x560deaa95310 msgr2=0x560deaac74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 shutdown_connections 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 1 -- 172.21.15.86:0/3056934997 wait complete. 2024-09-05T06:52:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.554+0000 7fb3023cb740 20 client.27188 trim_cache size 0 max 16384 2024-09-05T06:52:39.576 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:52:39.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 msgr2=0x7fb2e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 0x7fb2e40a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0004af0 tx=0x7fb2f0004a40 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 msgr2=0x7fb2f81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d770 tx=0x7fb2e800db70 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 shutdown_connections 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb2e40a2cf0 0x7fb2e40a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2f8106a80 0x7fb2f81a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2f8101420 0x7fb2f81997c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 --2- 172.21.15.86:0/1731124201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2f8100a80 0x7fb2f8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 >> 172.21.15.86:0/1731124201 conn(0x7fb2f80fc7a0 msgr2=0x7fb2f80fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 shutdown_connections 2024-09-05T06:52:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:39.555+0000 7fb2ffff2640 1 -- 172.21.15.86:0/1731124201 wait complete. 2024-09-05T06:52:40.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:52:40.512 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:40.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202255 1.66 MB/sec execute 480 sec latency 1451.719 ms 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 -- 172.21.15.86:0/3049315056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 msgr2=0x7fcb9010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/3049315056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb9010ea90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009ca0 tx=0x7fcb8802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 -- 172.21.15.86:0/3049315056 shutdown_connections 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/3049315056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb9010ea90 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/3049315056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90102790 0x7fcb901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/3049315056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90101de0 0x7fcb901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 -- 172.21.15.86:0/3049315056 >> 172.21.15.86:0/3049315056 conn(0x7fcb900fd8c0 msgr2=0x7fcb900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 -- 172.21.15.86:0/3049315056 shutdown_connections 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.968+0000 7fcb94c7f640 1 -- 172.21.15.86:0/3049315056 wait complete. 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.969+0000 7fcb94c7f640 1 Processor -- start 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.969+0000 7fcb94c7f640 1 -- start start 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.969+0000 7fcb94c7f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90101de0 0x7fcb900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.969+0000 7fcb94c7f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90102790 0x7fcb90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.969+0000 7fcb94c7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb94c7f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb90111e20 con 0x7fcb90106f10 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb94c7f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb90111ca0 con 0x7fcb90101de0 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb94c7f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb90111fa0 con 0x7fcb90102790 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46350/0 (socket says 172.21.15.86:46350) 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 -- 172.21.15.86:0/1406780945 learned_addr learned my addr 172.21.15.86:0/1406780945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:41.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90101de0 0x7fcb900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb87fff640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90102790 0x7fcb90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 -- 172.21.15.86:0/1406780945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90102790 msgr2=0x7fcb90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90102790 0x7fcb90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 -- 172.21.15.86:0/1406780945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90101de0 msgr2=0x7fcb900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90101de0 0x7fcb900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 -- 172.21.15.86:0/1406780945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb901a67e0 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb8ed9d640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009dc0 tx=0x7fcb88032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb877fe640 1 -- 172.21.15.86:0/1406780945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88003cf0 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb901a6a70 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb877fe640 1 -- 172.21.15.86:0/1406780945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb880044f0 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb877fe640 1 -- 172.21.15.86:0/1406780945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88030bf0 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.970+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb901a6fb0 con 0x7fcb90106f10 2024-09-05T06:52:41.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb877fe640 1 -- 172.21.15.86:0/1406780945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb88004040 con 0x7fcb90106f10 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 Processor -- start 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 -- start start 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18083090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d18083a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d17fe92f0 con 0x557d1808e370 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d17f90cf0 con 0x557d18084b40 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.973+0000 7fcb97058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d17fe52b0 con 0x557d180835d0 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46362/0 (socket says 172.21.15.86:46362) 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d18083a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb87fff640 1 -- 172.21.15.86:0/3997387262 learned_addr learned my addr 172.21.15.86:0/3997387262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:41.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb877fe640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 0x7fcb7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8ed9d640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 0x7fcb7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 -- 172.21.15.86:0/3997387262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 msgr2=0x557d18083090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18083090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 -- 172.21.15.86:0/3997387262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 msgr2=0x557d18090760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 -- 172.21.15.86:0/3997387262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d17f753a0 con 0x557d180835d0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb877fe640 1 -- 172.21.15.86:0/1406780945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fcb880f0d10 con 0x7fcb90106f10 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d18083a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb80004630 tx=0x7fcb8000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b7076891b2f28e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb8ed9d640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 0x7fcb7c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ee620 tx=0x7fcb8808eec0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.974+0000 7fcb84ff9640 1 -- 172.21.15.86:0/3997387262 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80003bd0 con 0x557d180835d0 2024-09-05T06:52:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb84ff9640 1 -- 172.21.15.86:0/3997387262 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb80015070 con 0x557d180835d0 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb84ff9640 1 -- 172.21.15.86:0/3997387262 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80004d10 con 0x557d180835d0 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 -- 172.21.15.86:0/3997387262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 msgr2=0x557d18083a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d18083a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb80004630 tx=0x7fcb8000bb20 comp rx=0 tx=0).stop 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 -- 172.21.15.86:0/3997387262 shutdown_connections 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d18090760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d18083a10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 --2- 172.21.15.86:0/3997387262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18083090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 -- 172.21.15.86:0/3997387262 >> 172.21.15.86:0/3997387262 conn(0x557d1804c310 msgr2=0x557d1807fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 -- 172.21.15.86:0/3997387262 shutdown_connections 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.975+0000 7fcb97058740 1 -- 172.21.15.86:0/3997387262 wait complete. 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 -- start start 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 Processor -- start 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d18084b40 0x557d1808afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d1808e370 0x557d1808b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d18091070 con 0x557d180835d0 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d17fa4060 con 0x557d1808e370 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d180911f0 con 0x557d18084b40 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb97058740 10 client.0 _ll_register_callbacks cb 0x557d17cf78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46368/0 (socket says 172.21.15.86:46368) 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb87fff640 1 -- 172.21.15.86:0/2534454166 learned_addr learned my addr 172.21.15.86:0/2534454166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.976+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d18084b40 0x557d1808afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 -- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d18084b40 msgr2=0x557d1808afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d18084b40 0x557d1808afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 -- 172.21.15.86:0/2534454166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d1808e370 msgr2=0x557d1808b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d1808e370 0x557d1808b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 -- 172.21.15.86:0/2534454166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d17d5a3c0 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb87fff640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800adc0 tx=0x7fcb7800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6fa3d7632078e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004020 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fcb780041c0 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb80015070 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcb78026050 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb97058740 10 client.18096 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557d1809cb60 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb80015070 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.977+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fcb78004880 con 0x557d180835d0 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 10 client.18096 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 10 client.18096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 20 client.18096 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d1809e020 0x557d180a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 4 client.18096 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d17ff6860 con 0x557d1809e020 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d1809e020 0x557d180a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d1809e020 0x557d180a04a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9052f47dab19445c server_cookie=b1011e6ebceaec6 in_seq=0 out_seq=0 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.978+0000 7fcb75ffb640 10 client.18096 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb75ffb640 1 -- 172.21.15.86:0/2534454166 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d17ff6860 con 0x557d1809e020 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb75ffb640 10 client.18096 handle_command_reply: tid=0 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 client.18096 shutdown 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 msgr2=0x557d1808dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800adc0 tx=0x7fcb7800f790 comp rx=0 tx=0).stop 2024-09-05T06:52:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 shutdown_connections 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d1809e020 0x557d180a04a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d1808e370 0x557d1808b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d18084b40 0x557d1808afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 --2- 172.21.15.86:0/2534454166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d1808dd00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb75ffb640 0 client.18096 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 >> 172.21.15.86:0/2534454166 conn(0x557d1804c310 msgr2=0x557d18085960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 shutdown_connections 2024-09-05T06:52:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 1 -- 172.21.15.86:0/2534454166 wait complete. 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.979+0000 7fcb97058740 20 client.18096 trim_cache size 0 max 16384 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 Processor -- start 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 -- start start 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d1807fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d1809a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901b12f0 con 0x557d1808e370 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901b2ec0 con 0x557d18084b40 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb97058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901b19c0 con 0x557d180835d0 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46376/0 (socket says 172.21.15.86:46376) 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb87fff640 1 -- 172.21.15.86:0/3005912264 learned_addr learned my addr 172.21.15.86:0/3005912264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d1807fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.992+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d1809a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 -- 172.21.15.86:0/3005912264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 msgr2=0x557d1807fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d1807fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 -- 172.21.15.86:0/3005912264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 msgr2=0x557d1809a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d1809a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 -- 172.21.15.86:0/3005912264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1808b8a0 con 0x557d1808e370 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb87fff640 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcb78010ac0 tx=0x7fcb78011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883cf7711440b0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb77fff640 1 -- 172.21.15.86:0/3005912264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800fe90 con 0x557d1808e370 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb77fff640 1 -- 172.21.15.86:0/3005912264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb780047a0 con 0x557d1808e370 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.993+0000 7fcb77fff640 1 -- 172.21.15.86:0/3005912264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801c690 con 0x557d1808e370 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- 172.21.15.86:0/3005912264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 msgr2=0x557d180a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcb78010ac0 tx=0x7fcb78011f90 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- 172.21.15.86:0/3005912264 shutdown_connections 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d1808e370 0x557d180a5db0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d1809a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- 172.21.15.86:0/3005912264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d180835d0 0x557d1807fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- 172.21.15.86:0/3005912264 >> 172.21.15.86:0/3005912264 conn(0x557d1804c310 msgr2=0x557d1807f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- 172.21.15.86:0/3005912264 shutdown_connections 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- 172.21.15.86:0/3005912264 wait complete. 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- start start 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 Processor -- start 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18091d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d1808e370 0x557d180922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901b2790 con 0x557d180835d0 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901b2090 con 0x557d18084b40 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901af2f0 con 0x557d1808e370 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb97058740 10 client.0 _ll_register_callbacks cb 0x7fcb7800a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46392/0 (socket says 172.21.15.86:46392) 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb8e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18091d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.994+0000 7fcb87fff640 1 -- 172.21.15.86:0/945933313 learned_addr learned my addr 172.21.15.86:0/945933313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d1808e370 0x557d180922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 -- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d1808e370 msgr2=0x557d180922c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d1808e370 0x557d180922c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 -- 172.21.15.86:0/945933313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 msgr2=0x557d18091d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18091d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 -- 172.21.15.86:0/945933313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d18092900 con 0x557d180835d0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18091d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb87fff640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005fa0 tx=0x7fcb7800ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ea29d9ab1cb733 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7803f060 con 0x557d180835d0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fcb780050c0 con 0x557d180835d0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb780047a0 con 0x557d180835d0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcb780052c0 con 0x557d180835d0 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb97058740 10 client.18102 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:41.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557d18108b90 con 0x557d180835d0 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.995+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb780047a0 con 0x557d180835d0 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fcb78037070 con 0x557d180835d0 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 10 client.18102 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 10 client.18102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 20 client.18102 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d18109190 0x557d1810b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 4 client.18102 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d17ff0d70 con 0x557d18109190 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d18109190 0x557d1810b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:41.056 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb75ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.996+0000 7fcb8e59c640 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d18109190 0x557d1810b5b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ddd007eb9248897 server_cookie=cc29e98e76a4d09a in_seq=0 out_seq=0 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb75ffb640 1 -- 172.21.15.86:0/945933313 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d17ff0d70 con 0x557d18109190 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb75ffb640 10 client.18102 handle_command_reply: tid=0 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 client.18102 shutdown 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 msgr2=0x557d18096a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005fa0 tx=0x7fcb7800ac60 comp rx=0 tx=0).stop 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 shutdown_connections 2024-09-05T06:52:41.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x557d18109190 0x557d1810b5b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d1808e370 0x557d180922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d18084b40 0x557d18091d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb75ffb640 0 client.18102 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 --2- 172.21.15.86:0/945933313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d180835d0 0x557d18096a50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 >> 172.21.15.86:0/945933313 conn(0x557d1804c310 msgr2=0x557d1807e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 shutdown_connections 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 1 -- 172.21.15.86:0/945933313 wait complete. 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb97058740 20 client.18102 trim_cache size 0 max 16384 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 msgr2=0x7fcb7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 0x7fcb7c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ee620 tx=0x7fcb8808eec0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 msgr2=0x7fcb90074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009dc0 tx=0x7fcb88032ff0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 shutdown_connections 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcb7c0a2cf0 0x7fcb7c0a51b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb90106f10 0x7fcb90074f70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb90102790 0x7fcb90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 --2- 172.21.15.86:0/1406780945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb90101de0 0x7fcb900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.997+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 >> 172.21.15.86:0/1406780945 conn(0x7fcb900fd8c0 msgr2=0x7fcb900fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.998+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 shutdown_connections 2024-09-05T06:52:41.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:40.998+0000 7fcb94c7f640 1 -- 172.21.15.86:0/1406780945 wait complete. 2024-09-05T06:52:41.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:41 smithi086 ceph-mon[30943]: pgmap v355: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-05T06:52:41.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:41 smithi096 ceph-mon[34468]: pgmap v355: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-05T06:52:41.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:41 smithi115 ceph-mon[34791]: pgmap v355: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-05T06:52:41.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202255 1.65 MB/sec execute 481 sec latency 2451.803 ms 2024-09-05T06:52:41.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:52:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:42.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202322 1.65 MB/sec execute 482 sec latency 2660.780 ms 2024-09-05T06:52:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 -- 172.21.15.86:0/2683386272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48102790 msgr2=0x7fca481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 --2- 172.21.15.86:0/2683386272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48102790 0x7fca481067e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fca34009900 tx=0x7fca3402a410 comp rx=0 tx=0).stop 2024-09-05T06:52:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 -- 172.21.15.86:0/2683386272 shutdown_connections 2024-09-05T06:52:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 --2- 172.21.15.86:0/2683386272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48106f10 0x7fca4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 --2- 172.21.15.86:0/2683386272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48102790 0x7fca481067e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 --2- 172.21.15.86:0/2683386272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 -- 172.21.15.86:0/2683386272 >> 172.21.15.86:0/2683386272 conn(0x7fca480fd8c0 msgr2=0x7fca480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 -- 172.21.15.86:0/2683386272 shutdown_connections 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.528+0000 7fca4ef05640 1 -- 172.21.15.86:0/2683386272 wait complete. 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 Processor -- start 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 -- start start 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca48198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48102790 0x7fca48198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca48111e90 con 0x7fca48106f10 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca48111d10 con 0x7fca48102790 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4ef05640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca48112010 con 0x7fca48101de0 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49740/0 (socket says 172.21.15.86:49740) 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 -- 172.21.15.86:0/673899763 learned_addr learned my addr 172.21.15.86:0/673899763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4cca1640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca48198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca47fff640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48102790 0x7fca48198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 -- 172.21.15.86:0/673899763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 msgr2=0x7fca48198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca48198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 -- 172.21.15.86:0/673899763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48102790 msgr2=0x7fca48198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48102790 0x7fca48198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4d4a2640 1 -- 172.21.15.86:0/673899763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4819d3e0 con 0x7fca48106f10 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.529+0000 7fca4cca1640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca48198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fca40009fd0 tx=0x7fca4000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca45ffb640 1 -- 172.21.15.86:0/673899763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40017070 con 0x7fca48106f10 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca45ffb640 1 -- 172.21.15.86:0/673899763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca40003e20 con 0x7fca48106f10 2024-09-05T06:52:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4819d6d0 con 0x7fca48106f10 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca45ffb640 1 -- 172.21.15.86:0/673899763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40010460 con 0x7fca48106f10 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.530+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca481a4fb0 con 0x7fca48106f10 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.532+0000 7fca45ffb640 1 -- 172.21.15.86:0/673899763 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fca40003970 con 0x7fca48106f10 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.532+0000 7fca512de740 1 Processor -- start 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.532+0000 7fca512de740 1 -- start start 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca45ffb640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 0x7fca380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d747a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d754760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f2d6ad2f0 con 0x555f2d748b40 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f2d654cf0 con 0x555f2d752370 2024-09-05T06:52:42.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca512de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f2d6a92b0 con 0x555f2d7475d0 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca45ffb640 1 -- 172.21.15.86:0/673899763 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fca400cf770 con 0x7fca48106f10 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49748/0 (socket says 172.21.15.86:49748) 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 -- 172.21.15.86:0/2105520508 learned_addr learned my addr 172.21.15.86:0/2105520508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca47fff640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 0x7fca380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d754760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 -- 172.21.15.86:0/2105520508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 msgr2=0x555f2d747a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d747a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 -- 172.21.15.86:0/2105520508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 msgr2=0x555f2d754760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d754760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 -- 172.21.15.86:0/2105520508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2d6393a0 con 0x555f2d748b40 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fca3c007850 tx=0x7fca3c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e0af69f6b84889 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca47fff640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 0x7fca380a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca34000c00 tx=0x7fca34031400 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.533+0000 7fca32ffd640 1 -- 172.21.15.86:0/2105520508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00d950 con 0x555f2d748b40 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca32ffd640 1 -- 172.21.15.86:0/2105520508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca3c018070 con 0x555f2d748b40 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca32ffd640 1 -- 172.21.15.86:0/2105520508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c013da0 con 0x555f2d748b40 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 -- 172.21.15.86:0/2105520508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 msgr2=0x555f2d747090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fca3c007850 tx=0x7fca3c007d10 comp rx=0 tx=0).stop 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 -- 172.21.15.86:0/2105520508 shutdown_connections 2024-09-05T06:52:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d754760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d747a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 --2- 172.21.15.86:0/2105520508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d747090 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 -- 172.21.15.86:0/2105520508 >> 172.21.15.86:0/2105520508 conn(0x555f2d710310 msgr2=0x555f2d743c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 -- 172.21.15.86:0/2105520508 shutdown_connections 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.534+0000 7fca512de740 1 -- 172.21.15.86:0/2105520508 wait complete. 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 -- start start 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 Processor -- start 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d748b40 0x555f2d74ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f2d755070 con 0x555f2d752370 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f2d668060 con 0x555f2d7475d0 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f2d7551f0 con 0x555f2d748b40 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca512de740 10 client.0 _ll_register_callbacks cb 0x555f2d3bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca4d4a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca4cca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca4cca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57990/0 (socket says 172.21.15.86:57990) 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca4d4a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49760/0 (socket says 172.21.15.86:49760) 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca4cca1640 1 -- 172.21.15.86:0/2795933737 learned_addr learned my addr 172.21.15.86:0/2795933737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.535+0000 7fca47fff640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d748b40 0x555f2d74ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 -- 172.21.15.86:0/2795933737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d748b40 msgr2=0x555f2d74ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d748b40 0x555f2d74ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 -- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 msgr2=0x555f2d751cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 -- 172.21.15.86:0/2795933737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2d41e3c0 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fca400cca80 tx=0x7fca4009b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0dfa5b4d2b06a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40020040 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fca400b46f0 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3c018070 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca4009b850 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca512de740 10 client.18114 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555f2d760b30 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4009fc00 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.536+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fca40086070 con 0x555f2d752370 2024-09-05T06:52:42.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 10 client.18114 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 20 client.18114 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 4 client.18114 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca3c018070 con 0x555f2d761f90 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca27fff640 10 client.18114 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.537+0000 7fca4cca1640 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7643f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6905abfac2a699e1 server_cookie=2c5cf9526675fa27 in_seq=0 out_seq=0 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca27fff640 1 -- 172.21.15.86:0/2795933737 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fca3c018070 con 0x555f2d761f90 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca27fff640 10 client.18114 handle_command_reply: tid=0 2024-09-05T06:52:42.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 client.18114 shutdown 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 msgr2=0x555f2d74f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fca400cca80 tx=0x7fca4009b2e0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 shutdown_connections 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7643f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca27fff640 0 client.18114 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d752370 0x555f2d74f4e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d748b40 0x555f2d74ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 --2- 172.21.15.86:0/2795933737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d751cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 >> 172.21.15.86:0/2795933737 conn(0x555f2d710310 msgr2=0x555f2d749960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 shutdown_connections 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 1 -- 172.21.15.86:0/2795933737 wait complete. 2024-09-05T06:52:42.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.538+0000 7fca512de740 20 client.18114 trim_cache size 0 max 16384 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 Processor -- start 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 -- start start 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d75e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d752370 0x555f2d74d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca481af190 con 0x555f2d748b40 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b0d60 con 0x555f2d7475d0 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca512de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca481af860 con 0x555f2d752370 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49766/0 (socket says 172.21.15.86:49766) 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 -- 172.21.15.86:0/967808146 learned_addr learned my addr 172.21.15.86:0/967808146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca4cca1640 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d75e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 -- 172.21.15.86:0/967808146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d752370 msgr2=0x555f2d74d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d752370 0x555f2d74d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 -- 172.21.15.86:0/967808146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 msgr2=0x555f2d75e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d75e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca47fff640 1 -- 172.21.15.86:0/967808146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2d74d4d0 con 0x555f2d748b40 2024-09-05T06:52:42.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.551+0000 7fca4cca1640 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d75e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca47fff640 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fca34033000 tx=0x7fca3403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7899d20412f0851b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca327fc640 1 -- 172.21.15.86:0/967808146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3404d070 con 0x555f2d748b40 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca327fc640 1 -- 172.21.15.86:0/967808146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca34004b70 con 0x555f2d748b40 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca327fc640 1 -- 172.21.15.86:0/967808146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34035050 con 0x555f2d748b40 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 -- 172.21.15.86:0/967808146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 msgr2=0x555f2d74e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fca34033000 tx=0x7fca3403c000 comp rx=0 tx=0).stop 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 -- 172.21.15.86:0/967808146 shutdown_connections 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d752370 0x555f2d74d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d74e840 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 --2- 172.21.15.86:0/967808146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d7475d0 0x555f2d75e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 -- 172.21.15.86:0/967808146 >> 172.21.15.86:0/967808146 conn(0x555f2d710310 msgr2=0x555f2d74ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 -- 172.21.15.86:0/967808146 shutdown_connections 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.552+0000 7fca512de740 1 -- 172.21.15.86:0/967808146 wait complete. 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 -- start start 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 Processor -- start 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d75abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d769c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b0630 con 0x555f2d748b40 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca481aff30 con 0x555f2d752370 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca481ad190 con 0x555f2d7475d0 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca512de740 10 client.0 _ll_register_callbacks cb 0x7fca40004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49770/0 (socket says 172.21.15.86:49770) 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 -- 172.21.15.86:0/1534359445 learned_addr learned my addr 172.21.15.86:0/1534359445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d769c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 -- 172.21.15.86:0/1534359445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 msgr2=0x555f2d75abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d75abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 -- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 msgr2=0x555f2d769c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d769c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca47fff640 1 -- 172.21.15.86:0/1534359445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2d76a430 con 0x555f2d748b40 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.553+0000 7fca4d4a2640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d769c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca47fff640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fca3402a930 tx=0x7fca340325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b194b9586fd8a06d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3405a030 con 0x555f2d748b40 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca34004b70 con 0x555f2d748b40 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fca34005450 con 0x555f2d748b40 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca34067050 con 0x555f2d748b40 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 10 client.18120 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555f2d7ccd50 con 0x555f2d748b40 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca34004b70 con 0x555f2d748b40 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fca340614e0 con 0x555f2d748b40 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 10 client.18120 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 20 client.18120 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 4 client.18120 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.554+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f2d6ba860 con 0x555f2d761f90 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca4cca1640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca27fff640 10 client.18120 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca4cca1640 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7cf540 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3692ba5f5bc97962 server_cookie=fc0f6c85410183c7 in_seq=0 out_seq=0 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca27fff640 1 -- 172.21.15.86:0/1534359445 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f2d6ba860 con 0x555f2d761f90 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca27fff640 10 client.18120 handle_command_reply: tid=0 2024-09-05T06:52:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 client.18120 shutdown 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 msgr2=0x555f2d769660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fca3402a930 tx=0x7fca340325e0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 shutdown_connections 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f2d761f90 0x555f2d7cf540 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f2d752370 0x555f2d769c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca27fff640 0 client.18120 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f2d748b40 0x555f2d769660 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 --2- 172.21.15.86:0/1534359445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f2d7475d0 0x555f2d75abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.583 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 >> 172.21.15.86:0/1534359445 conn(0x555f2d710310 msgr2=0x555f2d74ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 shutdown_connections 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.555+0000 7fca512de740 1 -- 172.21.15.86:0/1534359445 wait complete. 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca512de740 20 client.18120 trim_cache size 0 max 16384 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 msgr2=0x7fca380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 0x7fca380a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca34000c00 tx=0x7fca34031400 comp rx=0 tx=0).stop 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 msgr2=0x7fca4819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fca40009fd0 tx=0x7fca4000b260 comp rx=0 tx=0).stop 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 shutdown_connections 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fca380a2cf0 0x7fca380a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca48106f10 0x7fca4819cd00 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca48102790 0x7fca48198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 --2- 172.21.15.86:0/673899763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca48101de0 0x7fca48198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 >> 172.21.15.86:0/673899763 conn(0x7fca480fd8c0 msgr2=0x7fca480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 shutdown_connections 2024-09-05T06:52:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:42.556+0000 7fca4ef05640 1 -- 172.21.15.86:0/673899763 wait complete. 2024-09-05T06:52:43.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:43 smithi086 ceph-mon[30943]: pgmap v356: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:52:43.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:43 smithi096 ceph-mon[34468]: pgmap v356: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:52:43.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:43 smithi115 ceph-mon[34791]: pgmap v356: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-09-05T06:52:43.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202322 1.65 MB/sec execute 483 sec latency 1560.285 ms 2024-09-05T06:52:44.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:52:44.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:52:44.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202322 1.64 MB/sec execute 484 sec latency 2560.364 ms 2024-09-05T06:52:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.100+0000 7ff53ad2a640 1 -- 172.21.15.86:0/3030309905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 msgr2=0x7ff534101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.100+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/3030309905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff534101200 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff524009ca0 tx=0x7ff52402a950 comp rx=0 tx=0).stop 2024-09-05T06:52:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.100+0000 7ff53ad2a640 1 -- 172.21.15.86:0/3030309905 shutdown_connections 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/3030309905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff53410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/3030309905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 0x7ff5341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/3030309905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff534101200 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- 172.21.15.86:0/3030309905 >> 172.21.15.86:0/3030309905 conn(0x7ff5340fc7d0 msgr2=0x7ff5340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- 172.21.15.86:0/3030309905 shutdown_connections 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- 172.21.15.86:0/3030309905 wait complete. 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 Processor -- start 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- start start 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 0x7ff534199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff5341a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff534110e30 con 0x7ff534100e20 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff534110cb0 con 0x7ff534102570 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff53ad2a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff534110fb0 con 0x7ff534102ef0 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.101+0000 7ff538ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49788/0 (socket says 172.21.15.86:49788) 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 -- 172.21.15.86:0/61504738 learned_addr learned my addr 172.21.15.86:0/61504738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff533fff640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 0x7ff534199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff5392c7640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff5341a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 -- 172.21.15.86:0/61504738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 msgr2=0x7ff5341a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff5341a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 -- 172.21.15.86:0/61504738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 msgr2=0x7ff534199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 0x7ff534199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 -- 172.21.15.86:0/61504738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5341a4880 con 0x7ff534100e20 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff5392c7640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff5341a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff538ac6640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff524005cc0 tx=0x7ff52402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff531ffb640 1 -- 172.21.15.86:0/61504738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524034040 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff531ffb640 1 -- 172.21.15.86:0/61504738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5240308f0 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5341a4a50 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff531ffb640 1 -- 172.21.15.86:0/61504738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240047c0 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.102+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341a4ea0 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.104+0000 7ff531ffb640 1 -- 172.21.15.86:0/61504738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff524037020 con 0x7ff534100e20 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 Processor -- start 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 -- start start 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c946fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9472540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642c93c6280 con 0x5642c9460a40 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642c936db10 con 0x5642c9461360 2024-09-05T06:52:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff531ffb640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 0x7ff51c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff53d103740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5642c93c2240 con 0x5642c9470150 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49802/0 (socket says 172.21.15.86:49802) 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff5392c7640 1 -- 172.21.15.86:0/2852755932 learned_addr learned my addr 172.21.15.86:0/2852755932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff531ffb640 1 -- 172.21.15.86:0/61504738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff5240f07a0 con 0x7ff534100e20 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff533fff640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9472540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.105+0000 7ff538ac6640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c946fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 -- 172.21.15.86:0/2852755932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 msgr2=0x5642c946fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c946fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 -- 172.21.15.86:0/2852755932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 msgr2=0x5642c9460e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 -- 172.21.15.86:0/2852755932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c937a1f0 con 0x5642c9470150 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff5392c7640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff5392c7640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 0x7ff51c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff533fff640 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9472540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff52800bbc0 tx=0x7ff5280107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114e6db822d50b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff5392c7640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 0x7ff51c0a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff52c005fd0 tx=0x7ff52c0046c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff51affd640 1 -- 172.21.15.86:0/2852755932 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528004030 con 0x5642c9470150 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff51affd640 1 -- 172.21.15.86:0/2852755932 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5280035f0 con 0x5642c9470150 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.106+0000 7ff51affd640 1 -- 172.21.15.86:0/2852755932 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280116a0 con 0x5642c9470150 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- 172.21.15.86:0/2852755932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 msgr2=0x5642c9472540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9472540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff52800bbc0 tx=0x7ff5280107a0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- 172.21.15.86:0/2852755932 shutdown_connections 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9472540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c946fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 --2- 172.21.15.86:0/2852755932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9460e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- 172.21.15.86:0/2852755932 >> 172.21.15.86:0/2852755932 conn(0x5642c945d0f0 msgr2=0x5642c945d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- 172.21.15.86:0/2852755932 shutdown_connections 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- 172.21.15.86:0/2852755932 wait complete. 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 -- start start 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.107+0000 7ff53d103740 1 Processor -- start 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c946d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642c9472d20 con 0x5642c9470150 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642c9380f10 con 0x5642c9460a40 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5642c9472ea0 con 0x5642c9461360 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff53d103740 10 client.0 _ll_register_callbacks cb 0x5642c9349a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49812/0 (socket says 172.21.15.86:49812) 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 -- 172.21.15.86:0/1798052841 learned_addr learned my addr 172.21.15.86:0/1798052841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff533fff640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c946d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 -- 172.21.15.86:0/1798052841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 msgr2=0x5642c946d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c946d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 -- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 msgr2=0x5642c946d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 -- 172.21.15.86:0/1798052841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c91376a0 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff5392c7640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff52c011720 tx=0x7ff52c00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c7edfc0ebc104 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.108+0000 7ff513fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005020 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff52c0051c0 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5280035f0 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff52c0053c0 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 10 client.18132 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642c947ea00 con 0x5642c9470150 2024-09-05T06:52:45.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5280035f0 con 0x5642c9470150 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff52c024070 con 0x5642c9470150 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 10 client.18132 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 20 client.18132 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c947ff10 0x5642c94823b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 4 client.18132 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642c93d37f0 con 0x5642c947ff10 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.109+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c947ff10 0x5642c94823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.110+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c947ff10 0x5642c94823b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317bca1a26346f92 server_cookie=7bd2a9a617b704e3 in_seq=0 out_seq=0 2024-09-05T06:52:45.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.110+0000 7ff513fff640 10 client.18132 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.110+0000 7ff513fff640 1 -- 172.21.15.86:0/1798052841 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642c93d37f0 con 0x5642c947ff10 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.110+0000 7ff513fff640 10 client.18132 handle_command_reply: tid=0 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 client.18132 shutdown 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 msgr2=0x5642c946dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff52c011720 tx=0x7ff52c00b430 comp rx=0 tx=0).stop 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 shutdown_connections 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c947ff10 0x5642c94823b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c946dab0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c946d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 --2- 172.21.15.86:0/1798052841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 >> 172.21.15.86:0/1798052841 conn(0x5642c945d0f0 msgr2=0x5642c9462e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff513fff640 0 client.18132 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:45.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 shutdown_connections 2024-09-05T06:52:45.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 1 -- 172.21.15.86:0/1798052841 wait complete. 2024-09-05T06:52:45.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.111+0000 7ff53d103740 20 client.18132 trim_cache size 0 max 16384 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 Processor -- start 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 -- start start 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c9487c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341ace30 con 0x5642c9470150 2024-09-05T06:52:45.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341aea00 con 0x5642c9460a40 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff53d103740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341ad500 con 0x5642c9461360 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.128+0000 7ff5392c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49818/0 (socket says 172.21.15.86:49818) 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 -- 172.21.15.86:0/3680503337 learned_addr learned my addr 172.21.15.86:0/3680503337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff533fff640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c9487c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff538ac6640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 -- 172.21.15.86:0/3680503337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 msgr2=0x5642c9487c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c9487c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 -- 172.21.15.86:0/3680503337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 msgr2=0x5642c946ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 -- 172.21.15.86:0/3680503337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c94635b0 con 0x5642c9470150 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff538ac6640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff5392c7640 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff52c01b920 tx=0x7ff52c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a9e776ca8fd8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff51a7fc640 1 -- 172.21.15.86:0/3680503337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005560 con 0x5642c9470150 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff51a7fc640 1 -- 172.21.15.86:0/3680503337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff52c01bb90 con 0x5642c9470150 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.129+0000 7ff51a7fc640 1 -- 172.21.15.86:0/3680503337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0253f0 con 0x5642c9470150 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- 172.21.15.86:0/3680503337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 msgr2=0x5642c94881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff52c01b920 tx=0x7ff52c011f10 comp rx=0 tx=0).stop 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- 172.21.15.86:0/3680503337 shutdown_connections 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9470150 0x5642c94881c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9461360 0x5642c9487c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 --2- 172.21.15.86:0/3680503337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9460a40 0x5642c946ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- 172.21.15.86:0/3680503337 >> 172.21.15.86:0/3680503337 conn(0x5642c945d0f0 msgr2=0x5642c946e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- 172.21.15.86:0/3680503337 shutdown_connections 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- 172.21.15.86:0/3680503337 wait complete. 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 -- start start 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.130+0000 7ff53d103740 1 Processor -- start 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c9473b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9474090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341ae2d0 con 0x5642c9460a40 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341adbd0 con 0x5642c9461360 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341aae30 con 0x5642c9470150 2024-09-05T06:52:45.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 10 client.0 _ll_register_callbacks cb 0x7ff52c00be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49830/0 (socket says 172.21.15.86:49830) 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 -- 172.21.15.86:0/1560181438 learned_addr learned my addr 172.21.15.86:0/1560181438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff533fff640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c9473b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff5392c7640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9474090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 -- 172.21.15.86:0/1560181438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 msgr2=0x5642c9474090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9474090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 -- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 msgr2=0x5642c9473b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c9473b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 -- 172.21.15.86:0/1560181438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c9474870 con 0x5642c9460a40 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff5392c7640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9474090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:45.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff538ac6640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff5240ee0b0 tx=0x7ff524036d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e489dacc9ae71e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff513fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240c4300 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7ff524036d80 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.131+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff52c01bb90 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff5240bd090 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 10 client.18138 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642c94eac10 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff52c01bb90 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7ff52404d030 con 0x5642c9460a40 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 10 client.18138 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 20 client.18138 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:45.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c94eb360 0x5642c94ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 4 client.18138 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:52:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642c93cdd00 con 0x5642c94eb360 2024-09-05T06:52:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.132+0000 7ff533fff640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c94eb360 0x5642c94ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.133+0000 7ff513fff640 10 client.18138 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.133+0000 7ff533fff640 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c94eb360 0x5642c94ed780 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467bd0d3bba7e688 server_cookie=d792f91785f674e8 in_seq=0 out_seq=0 2024-09-05T06:52:45.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:45 smithi086 ceph-mon[30943]: pgmap v357: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff513fff640 1 -- 172.21.15.86:0/1560181438 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642c93cdd00 con 0x5642c94eb360 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff513fff640 10 client.18138 handle_command_reply: tid=0 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 client.18138 shutdown 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 msgr2=0x5642c9478820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff5240ee0b0 tx=0x7ff524036d10 comp rx=0 tx=0).stop 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 shutdown_connections 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5642c94eb360 0x5642c94ed780 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5642c9470150 0x5642c9474090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642c9461360 0x5642c9473b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff513fff640 0 client.18138 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 --2- 172.21.15.86:0/1560181438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642c9460a40 0x5642c9478820 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 >> 172.21.15.86:0/1560181438 conn(0x5642c945d0f0 msgr2=0x5642c9462dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 shutdown_connections 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.134+0000 7ff53d103740 1 -- 172.21.15.86:0/1560181438 wait complete. 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53d103740 20 client.18138 trim_cache size 0 max 16384 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "79ed5763-6ad5-423a-b140-e9698bf49538", 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:52:45.147 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:52:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 msgr2=0x7ff51c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 0x7ff51c0a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff52c005fd0 tx=0x7ff52c0046c0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 msgr2=0x7ff5341992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff524005cc0 tx=0x7ff52402f650 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 shutdown_connections 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff51c0a2cf0 0x7ff51c0a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534102ef0 0x7ff5341a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff534102570 0x7ff534199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 --2- 172.21.15.86:0/61504738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff534100e20 0x7ff5341992c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 >> 172.21.15.86:0/61504738 conn(0x7ff5340fc7d0 msgr2=0x7ff5340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 shutdown_connections 2024-09-05T06:52:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:45.135+0000 7ff53ad2a640 1 -- 172.21.15.86:0/61504738 wait complete. 2024-09-05T06:52:45.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:45 smithi096 ceph-mon[34468]: pgmap v357: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:45.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:45 smithi115 ceph-mon[34791]: pgmap v357: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:45.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202447 1.64 MB/sec execute 485 sec latency 3182.489 ms 2024-09-05T06:52:46.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:52:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:46.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:46 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:52:46.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:46.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:46.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:46.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:52:46.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:46 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:46 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:52:46.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202744 1.64 MB/sec execute 486 sec latency 12.148 ms 2024-09-05T06:52:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 -- 172.21.15.86:0/3391151306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102570 msgr2=0x7fb6741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/3391151306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102570 0x7fb6741029b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb664009900 tx=0x7fb66402a480 comp rx=0 tx=0).stop 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 -- 172.21.15.86:0/3391151306 shutdown_connections 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/3391151306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674102ef0 0x7fb67410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/3391151306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102570 0x7fb6741029b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/3391151306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674100e20 0x7fb674101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 -- 172.21.15.86:0/3391151306 >> 172.21.15.86:0/3391151306 conn(0x7fb6740fc7d0 msgr2=0x7fb6740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 -- 172.21.15.86:0/3391151306 shutdown_connections 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.853+0000 7fb67bb9c640 1 -- 172.21.15.86:0/3391151306 wait complete. 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 Processor -- start 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 -- start start 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 0x7fb6741997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 0x7fb6741a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb674110e30 con 0x7fb674102ef0 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb674110cb0 con 0x7fb674100e20 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67bb9c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb674110fb0 con 0x7fb674102570 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 0x7fb6741a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb679137640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 0x7fb6741997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58068/0 (socket says 172.21.15.86:58068) 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.854+0000 7fb679938640 1 -- 172.21.15.86:0/4065187542 learned_addr learned my addr 172.21.15.86:0/4065187542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 -- 172.21.15.86:0/4065187542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 msgr2=0x7fb6741997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 0x7fb6741997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 -- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 msgr2=0x7fb6741a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 0x7fb6741a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 -- 172.21.15.86:0/4065187542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6741a4840 con 0x7fb674100e20 2024-09-05T06:52:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679137640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 0x7fb6741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb67a139640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 0x7fb6741a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb679938640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb67000e9f0 tx=0x7fb67000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb66affd640 1 -- 172.21.15.86:0/4065187542 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67000cd80 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb674197070 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb66affd640 1 -- 172.21.15.86:0/4065187542 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6700054d0 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb674197530 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.855+0000 7fb66affd640 1 -- 172.21.15.86:0/4065187542 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670010490 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.857+0000 7fb66affd640 1 -- 172.21.15.86:0/4065187542 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb670005020 con 0x7fb674100e20 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 Processor -- start 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 -- start start 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 0x55b2b52e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb66affd640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 0x7fb6600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2b523f2e0 con 0x55b2b52d94a0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2b51e6ce0 con 0x55b2b52e4680 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67df75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b2b523b2a0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49856/0 (socket says 172.21.15.86:49856) 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb67a139640 1 -- 172.21.15.86:0/2527987340 learned_addr learned my addr 172.21.15.86:0/2527987340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb679938640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 0x7fb6600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.858+0000 7fb679137640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 0x55b2b52e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb66affd640 1 -- 172.21.15.86:0/4065187542 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb6700cfb40 con 0x7fb674100e20 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 -- 172.21.15.86:0/2527987340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 msgr2=0x55b2b52e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 -- 172.21.15.86:0/2527987340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 msgr2=0x55b2b52e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 0x55b2b52e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 -- 172.21.15.86:0/2527987340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b51cb440 con 0x55b2b52d94a0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb679137640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 0x55b2b52e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb679938640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00f5c0 tx=0x7fb66c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e0b2394e284fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb67a139640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 0x7fb6600a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0099e0 tx=0x7fb66c01e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb65ffff640 1 -- 172.21.15.86:0/2527987340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c02b070 con 0x55b2b52d94a0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb65ffff640 1 -- 172.21.15.86:0/2527987340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb66c0268c0 con 0x55b2b52d94a0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.859+0000 7fb65ffff640 1 -- 172.21.15.86:0/2527987340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c022070 con 0x55b2b52d94a0 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 -- 172.21.15.86:0/2527987340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 msgr2=0x55b2b52d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00f5c0 tx=0x7fb66c00fa80 comp rx=0 tx=0).stop 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 -- 172.21.15.86:0/2527987340 shutdown_connections 2024-09-05T06:52:46.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52e4680 0x55b2b52e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 --2- 172.21.15.86:0/2527987340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52d94a0 0x55b2b52d9880 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 -- 172.21.15.86:0/2527987340 >> 172.21.15.86:0/2527987340 conn(0x55b2b52d5a30 msgr2=0x55b2b52d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 -- 172.21.15.86:0/2527987340 shutdown_connections 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.860+0000 7fb67df75740 1 -- 172.21.15.86:0/2527987340 wait complete. 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 -- start start 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 Processor -- start 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2b52e7250 con 0x55b2b52e4680 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2b51fa050 con 0x55b2b52d94a0 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b2b52e73d0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67df75740 10 client.0 _ll_register_callbacks cb 0x55b2b523a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb679137640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb679137640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44654/0 (socket says 172.21.15.86:44654) 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58094/0 (socket says 172.21.15.86:58094) 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.861+0000 7fb679137640 1 -- 172.21.15.86:0/2221473842 learned_addr learned my addr 172.21.15.86:0/2221473842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 -- 172.21.15.86:0/2221473842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 msgr2=0x55b2b52e11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 -- 172.21.15.86:0/2221473842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 msgr2=0x55b2b52e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 -- 172.21.15.86:0/2221473842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b4fb03c0 con 0x55b2b52d94a0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb679938640 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6700cce50 tx=0x7fb67000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7d4c06241fad48 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb65d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:52:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700a1160 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb67009c080 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66c0268c0 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb670005310 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb67df75740 10 client.27242 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b2b52f2c20 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.862+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb66c0268c0 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fb6700a4400 con 0x55b2b52d94a0 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 10 client.27242 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 10 client.27242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 20 client.27242 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b52f3fc0 0x55b2b52f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 4 client.27242 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b524c850 con 0x55b2b52f3fc0 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb679137640 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b52f3fc0 0x55b2b52f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb679137640 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b52f3fc0 0x55b2b52f6420 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0bf15a2bf628ab server_cookie=a137613570c2eae0 in_seq=0 out_seq=0 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.863+0000 7fb65d7fa640 10 client.27242 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.864+0000 7fb65d7fa640 1 -- 172.21.15.86:0/2221473842 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2b524c850 con 0x55b2b52f3fc0 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.864+0000 7fb65d7fa640 10 client.27242 handle_command_reply: tid=0 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.864+0000 7fb67df75740 1 client.27242 shutdown 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.864+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 msgr2=0x55b2b52e3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.864+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6700cce50 tx=0x7fb67000cf90 comp rx=0 tx=0).stop 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 shutdown_connections 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b52f3fc0 0x55b2b52f6420 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 --2- 172.21.15.86:0/2221473842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52e3f10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 >> 172.21.15.86:0/2221473842 conn(0x55b2b52d5a30 msgr2=0x55b2b52db710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb65d7fa640 0 client.27242 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:46.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 shutdown_connections 2024-09-05T06:52:46.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 1 -- 172.21.15.86:0/2221473842 wait complete. 2024-09-05T06:52:46.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.865+0000 7fb67df75740 20 client.27242 trim_cache size 0 max 16384 2024-09-05T06:52:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.882+0000 7fb67df75740 1 Processor -- start 2024-09-05T06:52:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.882+0000 7fb67df75740 1 -- start start 2024-09-05T06:52:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb65800fff0 con 0x55b2b52e4680 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb658011bc0 con 0x55b2b52d94a0 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67df75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6580106c0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49878/0 (socket says 172.21.15.86:49878) 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67a139640 1 -- 172.21.15.86:0/3384853241 learned_addr learned my addr 172.21.15.86:0/3384853241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679137640 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 -- 172.21.15.86:0/3384853241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 msgr2=0x55b2b52fc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52fc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 -- 172.21.15.86:0/3384853241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 msgr2=0x55b2b52ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 -- 172.21.15.86:0/3384853241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b52e4280 con 0x55b2b52d94a0 2024-09-05T06:52:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb67a139640 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb679938640 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52fbd30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6700985f0 tx=0x7fb67000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40585fd5d2fd0c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.883+0000 7fb65f7fe640 1 -- 172.21.15.86:0/3384853241 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700a1160 con 0x55b2b52d94a0 2024-09-05T06:52:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb65f7fe640 1 -- 172.21.15.86:0/3384853241 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb67009c320 con 0x55b2b52d94a0 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb65f7fe640 1 -- 172.21.15.86:0/3384853241 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67009b760 con 0x55b2b52d94a0 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 -- 172.21.15.86:0/3384853241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 msgr2=0x55b2b52fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52fbd30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6700985f0 tx=0x7fb67000cf20 comp rx=0 tx=0).stop 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 -- 172.21.15.86:0/3384853241 shutdown_connections 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 --2- 172.21.15.86:0/3384853241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52fbd30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 -- 172.21.15.86:0/3384853241 >> 172.21.15.86:0/3384853241 conn(0x55b2b52d5a30 msgr2=0x55b2b52d4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 -- 172.21.15.86:0/3384853241 shutdown_connections 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.884+0000 7fb67df75740 1 -- 172.21.15.86:0/3384853241 wait complete. 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 -- start start 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 Processor -- start 2024-09-05T06:52:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb658011490 con 0x55b2b52e4680 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb658010d90 con 0x55b2b52d94a0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb65800dff0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67df75740 10 client.0 _ll_register_callbacks cb 0x7fb67000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.885+0000 7fb67a139640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44668/0 (socket says 172.21.15.86:44668) 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58098/0 (socket says 172.21.15.86:58098) 2024-09-05T06:52:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 -- 172.21.15.86:0/3659021961 learned_addr learned my addr 172.21.15.86:0/3659021961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 -- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 msgr2=0x55b2b52ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 -- 172.21.15.86:0/3659021961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 msgr2=0x55b2b52e85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 -- 172.21.15.86:0/3659021961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b52e8e00 con 0x55b2b52d9dc0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679938640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb679137640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb66402a9a0 tx=0x7fb664032660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96aa1c6019b3b9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb65d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664056060 con 0x55b2b52d9dc0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fb664004230 con 0x55b2b52d9dc0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb67009c320 con 0x55b2b52d9dc0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb66405d050 con 0x55b2b52d9dc0 2024-09-05T06:52:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb67df75740 10 client.27229 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.886+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b2b535ecd0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb67009c320 con 0x55b2b52d9dc0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fb664004ac0 con 0x55b2b52d9dc0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 10 client.27229 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 10 client.27229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 20 client.27229 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b535f4e0 0x55b2b5361900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 4 client.27229 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2b5246d60 con 0x55b2b535f4e0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.887+0000 7fb679938640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b535f4e0 0x55b2b5361900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb65d7fa640 10 client.27229 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb679938640 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b535f4e0 0x55b2b5361900 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59211ccbb55018fe server_cookie=25a3eb9dc2f522f in_seq=0 out_seq=0 2024-09-05T06:52:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb65d7fa640 1 -- 172.21.15.86:0/3659021961 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b2b5246d60 con 0x55b2b535f4e0 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb65d7fa640 10 client.27229 handle_command_reply: tid=0 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 client.27229 shutdown 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 msgr2=0x55b2b52e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb66402a9a0 tx=0x7fb664032660 comp rx=0 tx=0).stop 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 shutdown_connections 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b2b535f4e0 0x55b2b5361900 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b52e4680 0x55b2b52e85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2b52d9dc0 0x55b2b52e8090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 --2- 172.21.15.86:0/3659021961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2b52d94a0 0x55b2b52ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb65d7fa640 0 client.27229 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 >> 172.21.15.86:0/3659021961 conn(0x55b2b52d5a30 msgr2=0x55b2b52e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 shutdown_connections 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 1 -- 172.21.15.86:0/3659021961 wait complete. 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.888+0000 7fb67df75740 20 client.27229 trim_cache size 0 max 16384 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "79ed5763-6ad5-423a-b140-e9698bf49538": { 2024-09-05T06:52:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "79ed5763-6ad5-423a-b140-e9698bf49538", 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:52:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 msgr2=0x7fb6600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 0x7fb6600a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0099e0 tx=0x7fb66c01e000 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 msgr2=0x7fb674199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb67000e9f0 tx=0x7fb67000edf0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 shutdown_connections 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb6600a2cf0 0x7fb6600a51b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb674102ef0 0x7fb6741a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb674102570 0x7fb6741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 --2- 172.21.15.86:0/4065187542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb674100e20 0x7fb674199280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 >> 172.21.15.86:0/4065187542 conn(0x7fb6740fc7d0 msgr2=0x7fb6740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:46.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 shutdown_connections 2024-09-05T06:52:46.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:46.889+0000 7fb67bb9c640 1 -- 172.21.15.86:0/4065187542 wait complete. 2024-09-05T06:52:47.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:47 smithi096 ceph-mon[34468]: pgmap v358: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:47.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:47 smithi115 ceph-mon[34791]: pgmap v358: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:47.457 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:47 smithi086 ceph-mon[30943]: pgmap v358: 129 pgs: 129 active+clean; 226 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-05T06:52:47.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 203222 1.64 MB/sec execute 487 sec latency 19.617 ms 2024-09-05T06:52:47.773 INFO:tasks.cephfs.filesystem:scrub status for tag:79ed5763-6ad5-423a-b140-e9698bf49538 - {'path': '/', 'tag': '79ed5763-6ad5-423a-b140-e9698bf49538', 'options': 'recursive,force'} 2024-09-05T06:52:48.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 203747 1.64 MB/sec execute 488 sec latency 22.585 ms 2024-09-05T06:52:49.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:49 smithi096 ceph-mon[34468]: pgmap v359: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-05T06:52:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:49 smithi115 ceph-mon[34791]: pgmap v359: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-05T06:52:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:49 smithi086 ceph-mon[30943]: pgmap v359: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-09-05T06:52:49.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 204382 1.64 MB/sec execute 489 sec latency 18.747 ms 2024-09-05T06:52:50.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205065 1.65 MB/sec execute 490 sec latency 13.148 ms 2024-09-05T06:52:51.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:51 smithi096 ceph-mon[34468]: pgmap v360: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 840 KiB/s wr, 63 op/s 2024-09-05T06:52:51.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:51 smithi115 ceph-mon[34791]: pgmap v360: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 840 KiB/s wr, 63 op/s 2024-09-05T06:52:51.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:51 smithi086 ceph-mon[30943]: pgmap v360: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 840 KiB/s wr, 63 op/s 2024-09-05T06:52:51.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205581 1.65 MB/sec execute 491 sec latency 14.140 ms 2024-09-05T06:52:52.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205802 1.66 MB/sec execute 492 sec latency 744.716 ms 2024-09-05T06:52:53.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:53 smithi096 ceph-mon[34468]: pgmap v361: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:52:53.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:53 smithi115 ceph-mon[34791]: pgmap v361: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:52:53.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:53 smithi086 ceph-mon[30943]: pgmap v361: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-05T06:52:53.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205802 1.65 MB/sec execute 493 sec latency 1744.830 ms 2024-09-05T06:52:54.533 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205802 1.65 MB/sec execute 494 sec latency 2744.918 ms 2024-09-05T06:52:55.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:55 smithi115 ceph-mon[34791]: pgmap v362: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:55.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:55 smithi086 ceph-mon[30943]: pgmap v362: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:55.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205802 1.65 MB/sec execute 495 sec latency 3745.011 ms 2024-09-05T06:52:55.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:55 smithi096 ceph-mon[34468]: pgmap v362: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:56.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:56.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:56.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:56.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205802 1.64 MB/sec execute 496 sec latency 4745.126 ms 2024-09-05T06:52:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:57.301 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T06:52:57.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:52:57.326 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:57 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:52:57.326 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:57 smithi086 ceph-mon[30943]: pgmap v363: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:57.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:57 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:52:57.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:57 smithi115 ceph-mon[34791]: pgmap v363: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:57.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205869 1.64 MB/sec execute 497 sec latency 4954.235 ms 2024-09-05T06:52:57.548 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:57.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:57 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:52:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:57 smithi096 ceph-mon[34468]: pgmap v363: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-05T06:52:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- 172.21.15.86:0/3702115091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074105ec0 msgr2=0x7f6074106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3702115091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074105ec0 0x7f6074106340 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f606c009ca0 tx=0x7f606c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- 172.21.15.86:0/3702115091 shutdown_connections 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3702115091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074105ec0 0x7f6074106340 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3702115091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60741011e0 0x7f6074105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3702115091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 0x7f6074100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- 172.21.15.86:0/3702115091 >> 172.21.15.86:0/3702115091 conn(0x7f60740fc7d0 msgr2=0x7f60740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- 172.21.15.86:0/3702115091 shutdown_connections 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- 172.21.15.86:0/3702115091 wait complete. 2024-09-05T06:52:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 Processor -- start 2024-09-05T06:52:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.026+0000 7f607b6c7640 1 -- start start 2024-09-05T06:52:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 0x7f60741a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6074105ec0 0x7f607419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60741184e0 con 0x7f60741011e0 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6074118360 con 0x7f6074105ec0 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6074118660 con 0x7f60741008c0 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33048/0 (socket says 172.21.15.86:33048) 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 -- 172.21.15.86:0/3445675789 learned_addr learned my addr 172.21.15.86:0/3445675789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6079463640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 0x7f60741a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6079c64640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6074105ec0 0x7f607419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 -- 172.21.15.86:0/3445675789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 msgr2=0x7f60741a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 0x7f60741a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 -- 172.21.15.86:0/3445675789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6074105ec0 msgr2=0x7f607419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6074105ec0 0x7f607419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 -- 172.21.15.86:0/3445675789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607419efa0 con 0x7f60741011e0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f6078c62640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f606400e9f0 tx=0x7f606400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606400cd80 con 0x7f60741011e0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60640108e0 con 0x7f60741011e0 2024-09-05T06:52:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607419f290 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.027+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064018920 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.028+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60741ab580 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.028+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607406a850 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.030+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6064010a90 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.030+0000 7f606a7fc640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 0x7f60600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.030+0000 7f6079463640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 0x7f60600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.030+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7f60640cffe0 con 0x7f60741011e0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.031+0000 7f6079463640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 0x7f60600a5230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f607419f9b0 tx=0x7f6070005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.033+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f606409c7a0 con 0x7f60741011e0 2024-09-05T06:52:58.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.234+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60741ab7b0 con 0x7f60741011e0 2024-09-05T06:52:58.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.235+0000 7f606a7fc640 1 -- 172.21.15.86:0/3445675789 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11978 (secure 0 0 0) 0x7f606409c0a0 con 0x7f60741011e0 2024-09-05T06:52:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:52:58.236 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"btime":"2024-09-05T06:51:30:994870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:51:30.398269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:52:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 msgr2=0x7f60600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 0x7f60600a5230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f607419f9b0 tx=0x7f6070005f90 comp rx=0 tx=0).stop 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 msgr2=0x7f60741a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f606400e9f0 tx=0x7f606400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 shutdown_connections 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60600a2d70 0x7f60600a5230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6074105ec0 0x7f607419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60741011e0 0x7f60741a4bf0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 --2- 172.21.15.86:0/3445675789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60741008c0 0x7f60741a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 >> 172.21.15.86:0/3445675789 conn(0x7f60740fc7d0 msgr2=0x7f6074103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 shutdown_connections 2024-09-05T06:52:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:58.237+0000 7f607b6c7640 1 -- 172.21.15.86:0/3445675789 wait complete. 2024-09-05T06:52:58.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205869 1.64 MB/sec execute 498 sec latency 1563.443 ms 2024-09-05T06:52:59.000 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T06:52:59.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T06:52:59.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:59 smithi086 ceph-mon[30943]: pgmap v364: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-05T06:52:59.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:52:59 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3445675789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:59.290 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:52:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:59 smithi115 ceph-mon[34791]: pgmap v364: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-05T06:52:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:52:59 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3445675789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:59.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205869 1.63 MB/sec execute 499 sec latency 2563.472 ms 2024-09-05T06:52:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:59 smithi096 ceph-mon[34468]: pgmap v364: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-05T06:52:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:52:59 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3445675789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 -- 172.21.15.86:0/2798397582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 msgr2=0x7fd4281089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 --2- 172.21.15.86:0/2798397582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd4281089f0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd418009900 tx=0x7fd41802a410 comp rx=0 tx=0).stop 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 -- 172.21.15.86:0/2798397582 shutdown_connections 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 --2- 172.21.15.86:0/2798397582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428109120 0x7fd428110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 --2- 172.21.15.86:0/2798397582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd4281089f0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 --2- 172.21.15.86:0/2798397582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428103ff0 0x7fd4281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 -- 172.21.15.86:0/2798397582 >> 172.21.15.86:0/2798397582 conn(0x7fd4280ffa90 msgr2=0x7fd428101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 -- 172.21.15.86:0/2798397582 shutdown_connections 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 -- 172.21.15.86:0/2798397582 wait complete. 2024-09-05T06:52:59.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.788+0000 7fd4301c4640 1 Processor -- start 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 -- start start 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 0x7fd42819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 0x7fd42819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd428114050 con 0x7fd4281049a0 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd428113ed0 con 0x7fd428109120 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281141d0 con 0x7fd428103ff0 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33078/0 (socket says 172.21.15.86:33078) 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 -- 172.21.15.86:0/569627290 learned_addr learned my addr 172.21.15.86:0/569627290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42e761640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 0x7fd42819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42df60640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 0x7fd42819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 -- 172.21.15.86:0/569627290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 msgr2=0x7fd42819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 0x7fd42819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 -- 172.21.15.86:0/569627290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 msgr2=0x7fd42819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 0x7fd42819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 -- 172.21.15.86:0/569627290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42819f4e0 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42e761640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 0x7fd42819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42df60640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 0x7fd42819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd42d75f640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd418006010 tx=0x7fd418004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd41effd640 1 -- 172.21.15.86:0/569627290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41803f030 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42819f770 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd41effd640 1 -- 172.21.15.86:0/569627290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4180035f0 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.789+0000 7fd41effd640 1 -- 172.21.15.86:0/569627290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418035070 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.790+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4281a70b0 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.792+0000 7fd41effd640 1 -- 172.21.15.86:0/569627290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd418003b10 con 0x7fd4281049a0 2024-09-05T06:52:59.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.792+0000 7fd43259d740 1 Processor -- start 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 -- start start 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd41effd640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 0x7fd4140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e71d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c11dca2a0 con 0x558c11e65080 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c11d71b10 con 0x558c11e6f940 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd43259d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c11dc6260 con 0x558c11e64760 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42e761640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 0x7fd4140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42df60640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42df60640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35794/0 (socket says 172.21.15.86:35794) 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42df60640 1 -- 172.21.15.86:0/3861076411 learned_addr learned my addr 172.21.15.86:0/3861076411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:59.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd42e761640 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e71d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.793+0000 7fd41effd640 1 -- 172.21.15.86:0/569627290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd4180f02a0 con 0x7fd4281049a0 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 -- 172.21.15.86:0/3861076411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 msgr2=0x558c11e71d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e71d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 -- 172.21.15.86:0/3861076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 msgr2=0x558c11e6f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 -- 172.21.15.86:0/3861076411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c11b3bb90 con 0x558c11e64760 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42e761640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 0x7fd4140a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd42000cd10 tx=0x7fd420009eb0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd42df60640 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd42400bc20 tx=0x7fd42400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc690a1b76b2236 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd413fff640 1 -- 172.21.15.86:0/3861076411 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424017620 con 0x558c11e64760 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd413fff640 1 -- 172.21.15.86:0/3861076411 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd424017c70 con 0x558c11e64760 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.794+0000 7fd413fff640 1 -- 172.21.15.86:0/3861076411 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400fc30 con 0x558c11e64760 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- 172.21.15.86:0/3861076411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 msgr2=0x558c11e64b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd42400bc20 tx=0x7fd42400d710 comp rx=0 tx=0).stop 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- 172.21.15.86:0/3861076411 shutdown_connections 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e71d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 --2- 172.21.15.86:0/3861076411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e64b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- 172.21.15.86:0/3861076411 >> 172.21.15.86:0/3861076411 conn(0x558c11e60cf0 msgr2=0x558c11e610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- 172.21.15.86:0/3861076411 shutdown_connections 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- 172.21.15.86:0/3861076411 wait complete. 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 -- start start 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.795+0000 7fd43259d740 1 Processor -- start 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e6c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e6d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c11e72510 con 0x558c11e65080 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c11d526e0 con 0x558c11e6f940 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c11e72690 con 0x558c11e64760 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 10 client.0 _ll_register_callbacks cb 0x558c11d76a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33086/0 (socket says 172.21.15.86:33086) 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 -- 172.21.15.86:0/3287531720 learned_addr learned my addr 172.21.15.86:0/3287531720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42df60640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e6c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42e761640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e6d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 -- 172.21.15.86:0/3287531720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 msgr2=0x558c11e6c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e6c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 -- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 msgr2=0x558c11e6d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e6d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 -- 172.21.15.86:0/3287531720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c11e7b980 con 0x558c11e65080 2024-09-05T06:52:59.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42df60640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e6c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd418005ec0 tx=0x7fd4180bbce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b0ba05484a03d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418049040 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd4180e6260 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.796+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd424017c70 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd4180eda30 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 10 client.18174 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558c11e7e2e0 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd424017c70 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd4180bc5e0 con 0x558c11e65080 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 10 client.18174 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 10 client.18174 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 20 client.18174 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11e7f740 0x558c11e81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 4 client.18174 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c11dd79d0 con 0x558c11e7f740 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.797+0000 7fd42df60640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11e7f740 0x558c11e81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd4117fa640 10 client.18174 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd42df60640 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11e7f740 0x558c11e81c00 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9405a1b32b61ba1e server_cookie=9f85abad037fd7d4 in_seq=0 out_seq=0 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd4117fa640 1 -- 172.21.15.86:0/3287531720 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c11dd79d0 con 0x558c11e7f740 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd4117fa640 10 client.18174 handle_command_reply: tid=0 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd43259d740 1 client.18174 shutdown 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 msgr2=0x558c11e6ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.798+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd418005ec0 tx=0x7fd4180bbce0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 shutdown_connections 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11e7f740 0x558c11e81c00 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11e6d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e6ccc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 --2- 172.21.15.86:0/3287531720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e6c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd4117fa640 0 client.18174 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 >> 172.21.15.86:0/3287531720 conn(0x558c11e60cf0 msgr2=0x558c11e66a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 shutdown_connections 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 1 -- 172.21.15.86:0/3287531720 wait complete. 2024-09-05T06:52:59.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.799+0000 7fd43259d740 20 client.18174 trim_cache size 0 max 16384 2024-09-05T06:52:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.815+0000 7fd43259d740 1 Processor -- start 2024-09-05T06:52:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.815+0000 7fd43259d740 1 -- start start 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.815+0000 7fd43259d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 0x558c11e671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd43259d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd43259d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 0x558c11e87a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd43259d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b1370 con 0x558c11e65080 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd43259d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b2f40 con 0x558c11e64760 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd43259d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b1a40 con 0x558c11e6f940 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33102/0 (socket says 172.21.15.86:33102) 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 -- 172.21.15.86:0/3065117599 learned_addr learned my addr 172.21.15.86:0/3065117599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42df60640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 0x558c11e671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42e761640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 0x558c11e87a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 -- 172.21.15.86:0/3065117599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 msgr2=0x558c11e87a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 0x558c11e87a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 -- 172.21.15.86:0/3065117599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 msgr2=0x558c11e671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 0x558c11e671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 -- 172.21.15.86:0/3065117599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c11e60b20 con 0x558c11e65080 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42e761640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 0x558c11e87a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42df60640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 0x558c11e671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd42d75f640 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd4180c4ac0 tx=0x7fd4180ba890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb424e6baf8b51f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd4137fe640 1 -- 172.21.15.86:0/3065117599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418085070 con 0x558c11e65080 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd4137fe640 1 -- 172.21.15.86:0/3065117599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4180d5350 con 0x558c11e65080 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.816+0000 7fd4137fe640 1 -- 172.21.15.86:0/3065117599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180bc5e0 con 0x558c11e65080 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- 172.21.15.86:0/3065117599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 msgr2=0x558c11e87510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd4180c4ac0 tx=0x7fd4180ba890 comp rx=0 tx=0).stop 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- 172.21.15.86:0/3065117599 shutdown_connections 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e6f940 0x558c11e87a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e87510 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 --2- 172.21.15.86:0/3065117599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e64760 0x558c11e671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- 172.21.15.86:0/3065117599 >> 172.21.15.86:0/3065117599 conn(0x558c11e60cf0 msgr2=0x558c11e5fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- 172.21.15.86:0/3065117599 shutdown_connections 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- 172.21.15.86:0/3065117599 wait complete. 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 -- start start 2024-09-05T06:52:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.817+0000 7fd43259d740 1 Processor -- start 2024-09-05T06:52:59.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e74230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11ee24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b2810 con 0x558c11e65080 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281b2110 con 0x558c11e6f940 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4281af370 con 0x558c11e64760 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd43259d740 10 client.0 _ll_register_callbacks cb 0x7fd4180bc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33116/0 (socket says 172.21.15.86:33116) 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 -- 172.21.15.86:0/2009272392 learned_addr learned my addr 172.21.15.86:0/2009272392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42df60640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e74230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42e761640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11ee24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 -- 172.21.15.86:0/2009272392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 msgr2=0x558c11e74230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e74230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 -- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 msgr2=0x558c11ee24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11ee24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 -- 172.21.15.86:0/2009272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c11e61980 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd42d75f640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd4180092a0 tx=0x7fd4180bc280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f816c04d5d7a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.818+0000 7fd4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418085050 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9605+0+0 (secure 0 0 0) 0x7fd4180ed830 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4180d5350 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd4180e4050 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 10 client.18180 fetch_fsmap learned FSMap version 35 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558c11e5c260 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4180d5350 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fd4180dc250 con 0x558c11e65080 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 10 client.18180 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 10 client.18180 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:52:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 20 client.18180 populate_metadata read hostname 'smithi086' 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11eeafc0 0x558c11eed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 4 client.18180 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x558c11dd1d20 con 0x558c11eeafc0 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.819+0000 7fd42df60640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11eeafc0 0x558c11eed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.820+0000 7fd4117fa640 10 client.18180 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.820+0000 7fd42df60640 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11eeafc0 0x558c11eed3e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cbe83232c61b1d server_cookie=9bb99e4f7facdcc2 in_seq=0 out_seq=0 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd4117fa640 1 -- 172.21.15.86:0/2009272392 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558c11dd1d20 con 0x558c11eeafc0 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd4117fa640 10 client.18180 handle_command_reply: tid=0 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 client.18180 shutdown 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 msgr2=0x558c11e74770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd4180092a0 tx=0x7fd4180bc280 comp rx=0 tx=0).stop 2024-09-05T06:52:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 shutdown_connections 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558c11eeafc0 0x558c11eed3e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c11e6f940 0x558c11ee24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c11e65080 0x558c11e74770 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd4117fa640 0 client.18180 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 --2- 172.21.15.86:0/2009272392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c11e64760 0x558c11e74230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.868+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 >> 172.21.15.86:0/2009272392 conn(0x558c11e60cf0 msgr2=0x558c11e6dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 shutdown_connections 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd43259d740 1 -- 172.21.15.86:0/2009272392 wait complete. 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd43259d740 20 client.18180 trim_cache size 0 max 16384 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:52:59.870 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 35, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 38, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "19149df2": { 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 38, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:52:59.871 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:52:59.872 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 msgr2=0x7fd4140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 0x7fd4140a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd42000cd10 tx=0x7fd420009eb0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 msgr2=0x7fd42819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd418006010 tx=0x7fd418004270 comp rx=0 tx=0).stop 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 shutdown_connections 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd4140a2cf0 0x7fd4140a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd428109120 0x7fd42819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4281049a0 0x7fd42819aa70 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.869+0000 7fd4301c4640 1 --2- 172.21.15.86:0/569627290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd428103ff0 0x7fd42819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:52:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.870+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 >> 172.21.15.86:0/569627290 conn(0x7fd4280ffa90 msgr2=0x7fd428100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:52:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.870+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 shutdown_connections 2024-09-05T06:52:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:52:59.870+0000 7fd4301c4640 1 -- 172.21.15.86:0/569627290 wait complete. 2024-09-05T06:53:00.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.63 MB/sec execute 500 sec latency 2884.170 ms 2024-09-05T06:53:00.791 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '19149df2', quiesce took 3.5 seconds. Will release after: 6.5 2024-09-05T06:53:01.358 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:01 smithi096 ceph-mon[34468]: pgmap v365: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 889 KiB/s wr, 78 op/s 2024-09-05T06:53:01.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:01 smithi115 ceph-mon[34791]: pgmap v365: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 889 KiB/s wr, 78 op/s 2024-09-05T06:53:01.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:01 smithi086 ceph-mon[30943]: pgmap v365: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 889 KiB/s wr, 78 op/s 2024-09-05T06:53:01.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.63 MB/sec execute 501 sec latency 1652.461 ms 2024-09-05T06:53:02.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.62 MB/sec execute 502 sec latency 2652.548 ms 2024-09-05T06:53:03.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:03 smithi115 ceph-mon[34791]: pgmap v366: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-09-05T06:53:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:03 smithi086 ceph-mon[30943]: pgmap v366: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-09-05T06:53:03.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.62 MB/sec execute 503 sec latency 3652.655 ms 2024-09-05T06:53:03.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:03 smithi096 ceph-mon[34468]: pgmap v366: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 981 KiB/s wr, 106 op/s 2024-09-05T06:53:04.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.62 MB/sec execute 504 sec latency 4652.772 ms 2024-09-05T06:53:05.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:05 smithi115 ceph-mon[34791]: pgmap v367: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:05 smithi086 ceph-mon[30943]: pgmap v367: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:05.534 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.61 MB/sec execute 505 sec latency 5652.883 ms 2024-09-05T06:53:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:05 smithi096 ceph-mon[34468]: pgmap v367: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:06.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.61 MB/sec execute 506 sec latency 6652.984 ms 2024-09-05T06:53:07.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:53:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:07 smithi115 ceph-mon[34791]: pgmap v368: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:07 smithi115 ceph-mon[34791]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T06:53:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:07 smithi086 ceph-mon[30943]: pgmap v368: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:07 smithi086 ceph-mon[30943]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T06:53:07.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.61 MB/sec execute 507 sec latency 7653.026 ms 2024-09-05T06:53:07.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:07 smithi096 ceph-mon[34468]: pgmap v368: 129 pgs: 129 active+clean; 233 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 162 KiB/s wr, 44 op/s 2024-09-05T06:53:07.619 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:07 smithi096 ceph-mon[34468]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T06:53:07.619 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:08.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 -- 172.21.15.86:0/830371380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278106a80 msgr2=0x7fa278106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:08.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/830371380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278106a80 0x7fa278106e60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa260009930 tx=0x7fa26002a420 comp rx=0 tx=0).stop 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 -- 172.21.15.86:0/830371380 shutdown_connections 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/830371380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278101420 0x7fa27810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/830371380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 0x7fa278100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/830371380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278106a80 0x7fa278106e60 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 -- 172.21.15.86:0/830371380 >> 172.21.15.86:0/830371380 conn(0x7fa2780fc7a0 msgr2=0x7fa2780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 -- 172.21.15.86:0/830371380 shutdown_connections 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.037+0000 7fa27ccf9640 1 -- 172.21.15.86:0/830371380 wait complete. 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 Processor -- start 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 -- start start 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 0x7fa2781983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:08.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278106a80 0x7fa27819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa278110d30 con 0x7fa278101420 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa278110bb0 con 0x7fa278106a80 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27ccf9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa278110eb0 con 0x7fa278100a80 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40934/0 (socket says 172.21.15.86:40934) 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 -- 172.21.15.86:0/1797867114 learned_addr learned my addr 172.21.15.86:0/1797867114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa27659c640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 0x7fa2781983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa276d9d640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278106a80 0x7fa27819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 -- 172.21.15.86:0/1797867114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 msgr2=0x7fa2781983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 0x7fa2781983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 -- 172.21.15.86:0/1797867114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278106a80 msgr2=0x7fa27819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278106a80 0x7fa27819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.038+0000 7fa275d9b640 1 -- 172.21.15.86:0/1797867114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27819d360 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa275d9b640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c970 tx=0x7fa26800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268007bb0 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa268007d50 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27819d650 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26800f450 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.039+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781a4f30 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.040+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa278100ee0 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.041+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa268004f20 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.041+0000 7fa26f7fe640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 0x7fa2640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.042+0000 7fa27659c640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 0x7fa2640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.042+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9693+0+0 (secure 0 0 0) 0x7fa2680cf660 con 0x7fa278101420 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.042+0000 7fa27659c640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 0x7fa2640a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa260009900 tx=0x7fa260000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:08.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.046+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa268014030 con 0x7fa278101420 2024-09-05T06:53:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.246+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa278199710 con 0x7fa278101420 2024-09-05T06:53:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.247+0000 7fa26f7fe640 1 -- 172.21.15.86:0/1797867114 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11978 (secure 0 0 0) 0x7fa26809bdf0 con 0x7fa278101420 2024-09-05T06:53:08.249 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:53:08.249 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"btime":"2024-09-05T06:51:30:994870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:51:30.398269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:53:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 msgr2=0x7fa2640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 0x7fa2640a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa260009900 tx=0x7fa260000bc0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 msgr2=0x7fa2781988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c970 tx=0x7fa26800ce30 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 shutdown_connections 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2640a2cf0 0x7fa2640a51b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa278106a80 0x7fa27819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa278101420 0x7fa2781988f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 --2- 172.21.15.86:0/1797867114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa278100a80 0x7fa2781983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 >> 172.21.15.86:0/1797867114 conn(0x7fa2780fc7a0 msgr2=0x7fa27806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.249+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 shutdown_connections 2024-09-05T06:53:08.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:08.250+0000 7fa27ccf9640 1 -- 172.21.15.86:0/1797867114 wait complete. 2024-09-05T06:53:08.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.60 MB/sec execute 508 sec latency 8653.096 ms 2024-09-05T06:53:09.070 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 19149df2 --release --await' 2024-09-05T06:53:09.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 19149df2 --release --await 2024-09-05T06:53:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:09.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:09 smithi115 ceph-mon[34791]: pgmap v370: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:09.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:09 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1797867114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:09.440 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:09 smithi086 ceph-mon[30943]: pgmap v370: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:09.440 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:09 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1797867114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:09.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 205871 1.60 MB/sec execute 509 sec latency 9653.198 ms 2024-09-05T06:53:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:09 smithi096 ceph-mon[34468]: pgmap v370: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:09.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:09 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1797867114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:09.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- 172.21.15.86:0/562307251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc41027a0 msgr2=0x7fcbc41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 --2- 172.21.15.86:0/562307251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc41027a0 0x7fcbc41067f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009900 tx=0x7fcbac02a410 comp rx=0 tx=0).stop 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- 172.21.15.86:0/562307251 shutdown_connections 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 --2- 172.21.15.86:0/562307251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc4106f20 0x7fcbc410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 --2- 172.21.15.86:0/562307251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc41027a0 0x7fcbc41067f0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 --2- 172.21.15.86:0/562307251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4101df0 0x7fcbc41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- 172.21.15.86:0/562307251 >> 172.21.15.86:0/562307251 conn(0x7fcbc40fd8b0 msgr2=0x7fcbc40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- 172.21.15.86:0/562307251 shutdown_connections 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- 172.21.15.86:0/562307251 wait complete. 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 Processor -- start 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.813+0000 7fcbca060640 1 -- start start 2024-09-05T06:53:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc41027a0 0x7fcbc4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 0x7fcbc4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111e70 con 0x7fcbc4101df0 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111cf0 con 0x7fcbc41027a0 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbca060640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111ff0 con 0x7fcbc4106f20 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40952/0 (socket says 172.21.15.86:40952) 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1688849459 learned_addr learned my addr 172.21.15.86:0/1688849459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc41027a0 0x7fcbc4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 0x7fcbc4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1688849459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 msgr2=0x7fcbc4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 0x7fcbc4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1688849459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc41027a0 msgr2=0x7fcbc4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc41027a0 0x7fcbc4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1688849459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc41a68b0 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 0x7fcbc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c970 tx=0x7fcbb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.814+0000 7fcbc17fa640 1 -- 172.21.15.86:0/1688849459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4007bb0 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.815+0000 7fcbc17fa640 1 -- 172.21.15.86:0/1688849459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb4007d50 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.815+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc41a6ba0 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.815+0000 7fcbc17fa640 1 -- 172.21.15.86:0/1688849459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400f450 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.815+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41a7060 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbc17fa640 1 -- 172.21.15.86:0/1688849459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcbb4004f20 con 0x7fcbc4101df0 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 Processor -- start 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 -- start start 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb110dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1110630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdb1068620 con 0x55cdb1103060 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdb100fe50 con 0x55cdb1103980 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.817+0000 7fcbcc439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cdb10645e0 con 0x55cdb110e240 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40956/0 (socket says 172.21.15.86:40956) 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 -- 172.21.15.86:0/2158088779 learned_addr learned my addr 172.21.15.86:0/2158088779 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc17fa640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 0x7fcbb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb110dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 -- 172.21.15.86:0/2158088779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 msgr2=0x55cdb1110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1110630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 -- 172.21.15.86:0/2158088779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 msgr2=0x55cdb110dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb110dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 -- 172.21.15.86:0/2158088779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb1000d40 con 0x55cdb1103060 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb110dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc17fa640 1 -- 172.21.15.86:0/1688849459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9693+0+0 (secure 0 0 0) 0x7fcbb40cf670 con 0x7fcbc4101df0 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 0x7fcbb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c550 tx=0x7fcbbc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00d7d2c0af7189e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbb9ffb640 1 -- 172.21.15.86:0/2158088779 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00a600 con 0x55cdb1103060 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbb9ffb640 1 -- 172.21.15.86:0/2158088779 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbbc00ac50 con 0x55cdb1103060 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.818+0000 7fcbb9ffb640 1 -- 172.21.15.86:0/2158088779 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc018aa0 con 0x55cdb1103060 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- 172.21.15.86:0/2158088779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 msgr2=0x55cdb1103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 0x7fcbb00a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc01f000 tx=0x7fcbbc018000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c550 tx=0x7fcbbc00ca10 comp rx=0 tx=0).stop 2024-09-05T06:53:09.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- 172.21.15.86:0/2158088779 shutdown_connections 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb110dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 --2- 172.21.15.86:0/2158088779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb1103440 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- 172.21.15.86:0/2158088779 >> 172.21.15.86:0/2158088779 conn(0x55cdb10ff5f0 msgr2=0x55cdb10ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- 172.21.15.86:0/2158088779 shutdown_connections 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- 172.21.15.86:0/2158088779 wait complete. 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.819+0000 7fcbcc439740 1 -- start start 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 Processor -- start 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103060 0x55cdb110b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb110bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdb1110e10 con 0x55cdb1103980 2024-09-05T06:53:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdb0ff0f10 con 0x55cdb1103060 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cdb1110f90 con 0x55cdb110e240 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbcc439740 10 client.0 _ll_register_callbacks cb 0x55cdb1014db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40966/0 (socket says 172.21.15.86:40966) 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1440038576 learned_addr learned my addr 172.21.15.86:0/1440038576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103060 0x55cdb110b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb110bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1440038576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 msgr2=0x55cdb110bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb110bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103060 msgr2=0x55cdb110b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103060 0x55cdb110b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1440038576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb0d77580 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009900 tx=0x7fcbac0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337be465c7b0ef98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.820+0000 7fcba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac035040 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9693+0+0 (secure 0 0 0) 0x7fcbac031840 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbbc00ac50 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbac04d050 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 10 client.18198 fetch_fsmap learned FSMap version 35 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cdb0e41960 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbbc00ac50 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fcbac03c440 con 0x55cdb1103980 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 10 client.18198 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:53:09.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 10 client.18198 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 20 client.18198 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb111dda0 0x55cdb1120210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 4 client.18198 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdb1075b90 con 0x55cdb111dda0 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.821+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb111dda0 0x55cdb1120210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.822+0000 7fcba2ffd640 10 client.18198 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.822+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb111dda0 0x55cdb1120210 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac232c65c20070b server_cookie=39b6a4c2251be639 in_seq=0 out_seq=0 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.822+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1440038576 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdb1075b90 con 0x55cdb111dda0 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.822+0000 7fcba2ffd640 10 client.18198 handle_command_reply: tid=0 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.822+0000 7fcbcc439740 1 client.18198 shutdown 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 msgr2=0x55cdb110b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009900 tx=0x7fcbac0316d0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 shutdown_connections 2024-09-05T06:53:09.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb111dda0 0x55cdb1120210 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb110bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb110b6f0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcba2ffd640 0 client.18198 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1440038576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103060 0x55cdb110b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 >> 172.21.15.86:0/1440038576 conn(0x55cdb10ff5f0 msgr2=0x55cdb10fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 shutdown_connections 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 1 -- 172.21.15.86:0/1440038576 wait complete. 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.823+0000 7fcbcc439740 20 client.18198 trim_cache size 0 max 16384 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.839+0000 7fcbcc439740 1 Processor -- start 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.839+0000 7fcbcc439740 1 -- start start 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 0x55cdb110a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb110e240 0x55cdb1126060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41b1490 con 0x55cdb1103980 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41b3060 con 0x55cdb110e240 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbcc439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41b1b60 con 0x55cdb1103060 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40970/0 (socket says 172.21.15.86:40970) 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1365163136 learned_addr learned my addr 172.21.15.86:0/1365163136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 0x55cdb110a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb110e240 0x55cdb1126060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1365163136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 msgr2=0x55cdb110a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 0x55cdb110a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1365163136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb110e240 msgr2=0x55cdb1126060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb110e240 0x55cdb1126060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 -- 172.21.15.86:0/1365163136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb1102e90 con 0x55cdb1103980 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 0x55cdb110a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcbac03c840 tx=0x7fcbac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fa02378b69bee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbb97fa640 1 -- 172.21.15.86:0/1365163136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac000e60 con 0x55cdb1103980 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbb97fa640 1 -- 172.21.15.86:0/1365163136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbac03c440 con 0x55cdb1103980 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.840+0000 7fcbb97fa640 1 -- 172.21.15.86:0/1365163136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac035030 con 0x55cdb1103980 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- 172.21.15.86:0/1365163136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 msgr2=0x55cdb1125b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcbac03c840 tx=0x7fcbac02a410 comp rx=0 tx=0).stop 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- 172.21.15.86:0/1365163136 shutdown_connections 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb110e240 0x55cdb1126060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103980 0x55cdb1125b20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1365163136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb1103060 0x55cdb110a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- 172.21.15.86:0/1365163136 >> 172.21.15.86:0/1365163136 conn(0x55cdb10ff5f0 msgr2=0x55cdb110d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- 172.21.15.86:0/1365163136 shutdown_connections 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- 172.21.15.86:0/1365163136 wait complete. 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 -- start start 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.841+0000 7fcbcc439740 1 Processor -- start 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb1116b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1112ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41b2930 con 0x55cdb1103060 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41b2230 con 0x55cdb1103980 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41af490 con 0x55cdb110e240 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 10 client.0 _ll_register_callbacks cb 0x7fcbac031cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40980/0 (socket says 172.21.15.86:40980) 2024-09-05T06:53:09.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1486982173 learned_addr learned my addr 172.21.15.86:0/1486982173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb1116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc3fff640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1112ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1486982173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 msgr2=0x55cdb1112ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1112ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 msgr2=0x55cdb1116b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb1116b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1486982173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdb11133d0 con 0x55cdb1103060 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb1116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbc37fe640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40ccf30 tx=0x7fcbb406b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8de510513bf8da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401d040 con 0x55cdb1103060 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9693+0+0 (secure 0 0 0) 0x7fcbb40ccbb0 con 0x55cdb1103060 2024-09-05T06:53:09.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbac03c440 con 0x55cdb1103060 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.842+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcbb409fab0 con 0x55cdb1103060 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 10 client.18204 fetch_fsmap learned FSMap version 35 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cdb11897f0 con 0x55cdb1103060 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbac03c440 con 0x55cdb1103060 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fcbb40a3ee0 con 0x55cdb1103060 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 10 client.18204 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 10 client.18204 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 20 client.18204 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb1189ec0 0x55cdb118c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 4 client.18204 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "19149df2", "release": true, "await": true}] 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "19149df2", "release": true, "await": true}) -- 0x55cdb10700a0 con 0x55cdb1189ec0 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.843+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb1189ec0 0x55cdb118c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:09.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.844+0000 7fcba2ffd640 10 client.18204 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:09.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.844+0000 7fcbbbfff640 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb1189ec0 0x55cdb118c2b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc54940a584837d server_cookie=e5e150523f3eeba3 in_seq=0 out_seq=0 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.851+0000 7fcba2ffd640 1 -- 172.21.15.86:0/1486982173 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cdb10700a0 con 0x55cdb1189ec0 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.851+0000 7fcba2ffd640 10 client.18204 handle_command_reply: tid=0 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 client.18204 shutdown 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 msgr2=0x55cdb11165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40ccf30 tx=0x7fcbb406b560 comp rx=0 tx=0).stop 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 shutdown_connections 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cdb1189ec0 0x55cdb118c2b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdb110e240 0x55cdb1112ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdb1103980 0x55cdb1116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcba2ffd640 0 client.18204 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 --2- 172.21.15.86:0/1486982173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdb1103060 0x55cdb11165c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 >> 172.21.15.86:0/1486982173 conn(0x55cdb10ff5f0 msgr2=0x55cdb110b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 shutdown_connections 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 1 -- 172.21.15.86:0/1486982173 wait complete. 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbcc439740 20 client.18204 trim_cache size 0 max 16384 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 35, 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 40, 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:53:09.863 INFO:teuthology.orchestra.run.smithi086.stdout: "19149df2": { 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 40, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:09.865 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 msgr2=0x7fcbb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 0x7fcbb00a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc01f000 tx=0x7fcbbc018000 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 msgr2=0x7fcbc40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.852+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c970 tx=0x7fcbb400ce30 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 shutdown_connections 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbb00a2cf0 0x7fcbb00a51b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc4106f20 0x7fcbc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbc41027a0 0x7fcbc4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 --2- 172.21.15.86:0/1688849459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc4101df0 0x7fcbc40744f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 >> 172.21.15.86:0/1688849459 conn(0x7fcbc40fd8b0 msgr2=0x7fcbc40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 shutdown_connections 2024-09-05T06:53:09.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:09.853+0000 7fcbca060640 1 -- 172.21.15.86:0/1688849459 wait complete. 2024-09-05T06:53:10.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 206088 1.60 MB/sec execute 510 sec latency 9972.871 ms 2024-09-05T06:53:10.605 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '19149df2', total seconds elapsed: 13.3 2024-09-05T06:53:11.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:11 smithi115 ceph-mon[34791]: pgmap v371: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:11 smithi086 ceph-mon[30943]: pgmap v371: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:11.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 206408 1.60 MB/sec execute 511 sec latency 17.020 ms 2024-09-05T06:53:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:11 smithi096 ceph-mon[34468]: pgmap v371: 129 pgs: 129 active+clean; 232 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 137 KiB/s wr, 40 op/s 2024-09-05T06:53:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:12 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:53:12.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:12 smithi115 ceph-mon[34791]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T06:53:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:12 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:53:12.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:12 smithi086 ceph-mon[30943]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T06:53:12.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 206947 1.60 MB/sec execute 512 sec latency 14.583 ms 2024-09-05T06:53:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:12 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:53:12.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:12 smithi096 ceph-mon[34468]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T06:53:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:13 smithi115 ceph-mon[34791]: pgmap v372: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 359 KiB/s wr, 35 op/s 2024-09-05T06:53:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:13 smithi086 ceph-mon[30943]: pgmap v372: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 359 KiB/s wr, 35 op/s 2024-09-05T06:53:13.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 207505 1.60 MB/sec execute 513 sec latency 21.243 ms 2024-09-05T06:53:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:13 smithi096 ceph-mon[34468]: pgmap v372: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 359 KiB/s wr, 35 op/s 2024-09-05T06:53:14.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 208128 1.60 MB/sec execute 514 sec latency 20.185 ms 2024-09-05T06:53:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:15 smithi115 ceph-mon[34791]: pgmap v374: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 449 KiB/s wr, 44 op/s 2024-09-05T06:53:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:15 smithi086 ceph-mon[30943]: pgmap v374: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 449 KiB/s wr, 44 op/s 2024-09-05T06:53:15.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 208900 1.61 MB/sec execute 515 sec latency 16.871 ms 2024-09-05T06:53:15.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:15 smithi096 ceph-mon[34468]: pgmap v374: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 449 KiB/s wr, 44 op/s 2024-09-05T06:53:16.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209346 1.62 MB/sec execute 516 sec latency 62.434 ms 2024-09-05T06:53:17.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:17 smithi115 ceph-mon[34791]: pgmap v375: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 380 KiB/s wr, 38 op/s 2024-09-05T06:53:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:17 smithi086 ceph-mon[30943]: pgmap v375: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 380 KiB/s wr, 38 op/s 2024-09-05T06:53:17.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209413 1.61 MB/sec execute 517 sec latency 570.014 ms 2024-09-05T06:53:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:17 smithi096 ceph-mon[34468]: pgmap v375: 129 pgs: 129 active+clean; 235 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 380 KiB/s wr, 38 op/s 2024-09-05T06:53:17.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:53:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 -- 172.21.15.86:0/2993230223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0106f40 msgr2=0x7f20b010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 --2- 172.21.15.86:0/2993230223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0106f40 0x7f20b010eac0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009ca0 tx=0x7f20a802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 -- 172.21.15.86:0/2993230223 shutdown_connections 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 --2- 172.21.15.86:0/2993230223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0106f40 0x7f20b010eac0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 --2- 172.21.15.86:0/2993230223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 0x7f20b0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 --2- 172.21.15.86:0/2993230223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0101e10 0x7f20b01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 -- 172.21.15.86:0/2993230223 >> 172.21.15.86:0/2993230223 conn(0x7f20b00fd8d0 msgr2=0x7f20b00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 -- 172.21.15.86:0/2993230223 shutdown_connections 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.530+0000 7f20b4950640 1 -- 172.21.15.86:0/2993230223 wait complete. 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 Processor -- start 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 -- start start 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 0x7f20b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 0x7f20b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0111e70 con 0x7f20b0101e10 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0111cf0 con 0x7f20b0106f40 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.531+0000 7f20b4950640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0111ff0 con 0x7f20b01027c0 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42780/0 (socket says 172.21.15.86:42780) 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 -- 172.21.15.86:0/3837575163 learned_addr learned my addr 172.21.15.86:0/3837575163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20add9b640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 0x7f20b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20aed9d640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 0x7f20b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 -- 172.21.15.86:0/3837575163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 msgr2=0x7f20b0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 0x7f20b0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 -- 172.21.15.86:0/3837575163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 msgr2=0x7f20b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 0x7f20b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 -- 172.21.15.86:0/3837575163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01a6830 con 0x7f20b0101e10 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20aed9d640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 0x7f20b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20a77fe640 1 -- 172.21.15.86:0/3837575163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b0101e10 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20a77fe640 1 -- 172.21.15.86:0/3837575163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a000de80 con 0x7f20b0101e10 2024-09-05T06:53:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01a6b20 con 0x7f20b0101e10 2024-09-05T06:53:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20a77fe640 1 -- 172.21.15.86:0/3837575163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000f620 con 0x7f20b0101e10 2024-09-05T06:53:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.532+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01a7060 con 0x7f20b0101e10 2024-09-05T06:53:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20a77fe640 1 -- 172.21.15.86:0/3837575163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20a000f7c0 con 0x7f20b0101e10 2024-09-05T06:53:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 Processor -- start 2024-09-05T06:53:18.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209413 1.61 MB/sec execute 518 sec latency 1570.054 ms 2024-09-05T06:53:18.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 -- start start 2024-09-05T06:53:18.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247cdc0 0x55aec2487140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec2489a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aec23e22e0 con 0x55aec247c4a0 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aec2389ce0 con 0x55aec247cdc0 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20b6d29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aec23de2a0 con 0x55aec2487680 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42796/0 (socket says 172.21.15.86:42796) 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.535+0000 7f20aed9d640 1 -- 172.21.15.86:0/2982240605 learned_addr learned my addr 172.21.15.86:0/2982240605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20add9b640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec2489a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20a77fe640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 0x7f209c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20ae59c640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247cdc0 0x55aec2487140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 -- 172.21.15.86:0/2982240605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 msgr2=0x55aec2489a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec2489a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 -- 172.21.15.86:0/2982240605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247cdc0 msgr2=0x55aec2487140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247cdc0 0x55aec2487140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 -- 172.21.15.86:0/2982240605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec236e440 con 0x55aec247c4a0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20add9b640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec2489a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 0x7f209c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20a77fe640 1 -- 172.21.15.86:0/3837575163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f20a00cf520 con 0x7f20b0101e10 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009a60 tx=0x7f20a8031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e71a2ef533f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20a4ff9640 1 -- 172.21.15.86:0/2982240605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004040 con 0x55aec247c4a0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20a4ff9640 1 -- 172.21.15.86:0/2982240605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a8036070 con 0x55aec247c4a0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20aed9d640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 0x7f209c0a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f20a804b000 tx=0x7f20a8048000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.536+0000 7f20a4ff9640 1 -- 172.21.15.86:0/2982240605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8030c30 con 0x55aec247c4a0 2024-09-05T06:53:18.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- 172.21.15.86:0/2982240605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 msgr2=0x55aec247c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009a60 tx=0x7f20a8031b90 comp rx=0 tx=0).stop 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- 172.21.15.86:0/2982240605 shutdown_connections 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec2489a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247cdc0 0x55aec2487140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 --2- 172.21.15.86:0/2982240605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec247c880 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- 172.21.15.86:0/2982240605 >> 172.21.15.86:0/2982240605 conn(0x55aec2478a30 msgr2=0x55aec2478e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- 172.21.15.86:0/2982240605 shutdown_connections 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- 172.21.15.86:0/2982240605 wait complete. 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.537+0000 7f20b6d29740 1 -- start start 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 Processor -- start 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec2484270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec2487680 0x55aec24847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aec248a120 con 0x55aec247c4a0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aec239d050 con 0x55aec2487680 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aec248a2a0 con 0x55aec247cdc0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20b6d29740 10 client.0 _ll_register_callbacks cb 0x55aec23dd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42810/0 (socket says 172.21.15.86:42810) 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 -- 172.21.15.86:0/3921185881 learned_addr learned my addr 172.21.15.86:0/3921185881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20aed9d640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec2487680 0x55aec24847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20add9b640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec2484270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 -- 172.21.15.86:0/3921185881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 msgr2=0x55aec2484270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec2484270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 -- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec2487680 msgr2=0x55aec24847b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec2487680 0x55aec24847b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 -- 172.21.15.86:0/3921185881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec21533c0 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f20ae59c640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f20a0097a60 tx=0x7f20a009b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eff0a90d8239f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.538+0000 7f2095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0020050 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f20a0009c30 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a8036070 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f20a0097040 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 10 client.18216 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aec2495b20 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a8036070 con 0x55aec247c4a0 2024-09-05T06:53:18.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f20a00cf120 con 0x55aec247c4a0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 10 client.18216 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 20 client.18216 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2496ff0 0x55aec24994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 4 client.18216 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aec23ef850 con 0x55aec2496ff0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.539+0000 7f20add9b640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2496ff0 0x55aec24994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.540+0000 7f20add9b640 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2496ff0 0x55aec24994b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7da107b1b8639c server_cookie=6611d56da1606b18 in_seq=0 out_seq=0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.540+0000 7f2095ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f2095ffb640 1 -- 172.21.15.86:0/3921185881 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aec23ef850 con 0x55aec2496ff0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f2095ffb640 10 client.18216 handle_command_reply: tid=0 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 client.18216 shutdown 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 msgr2=0x55aec2486fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f20a0097a60 tx=0x7f20a009b060 comp rx=0 tx=0).stop 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 shutdown_connections 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2496ff0 0x55aec24994b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec2487680 0x55aec24847b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec2484270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 --2- 172.21.15.86:0/3921185881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247c4a0 0x55aec2486fc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 >> 172.21.15.86:0/3921185881 conn(0x55aec2478a30 msgr2=0x55aec247e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f2095ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 shutdown_connections 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 1 -- 172.21.15.86:0/3921185881 wait complete. 2024-09-05T06:53:18.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.541+0000 7f20b6d29740 20 client.18216 trim_cache size 0 max 16384 2024-09-05T06:53:18.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.557+0000 7f20b6d29740 1 Processor -- start 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.557+0000 7f20b6d29740 1 -- start start 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec2477a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec249f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01b1340 con 0x55aec247cdc0 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01b2ee0 con 0x55aec247c4a0 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20b6d29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01b1a10 con 0x55aec2487680 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42812/0 (socket says 172.21.15.86:42812) 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 -- 172.21.15.86:0/1451048545 learned_addr learned my addr 172.21.15.86:0/1451048545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20ae59c640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec2477a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20aed9d640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec249f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 -- 172.21.15.86:0/1451048545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 msgr2=0x55aec249f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec249f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 -- 172.21.15.86:0/1451048545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 msgr2=0x55aec2477a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec2477a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 -- 172.21.15.86:0/1451048545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec24784d0 con 0x55aec247cdc0 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20ae59c640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec2477a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20aed9d640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec249f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f20add9b640 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x55aec2479db0 tx=0x7f2098008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a121390a978221 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f2097fff640 1 -- 172.21.15.86:0/1451048545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098009da0 con 0x55aec247cdc0 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f2097fff640 1 -- 172.21.15.86:0/1451048545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209801c700 con 0x55aec247cdc0 2024-09-05T06:53:18.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.558+0000 7f2097fff640 1 -- 172.21.15.86:0/1451048545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098014900 con 0x55aec247cdc0 2024-09-05T06:53:18.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- 172.21.15.86:0/1451048545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 msgr2=0x55aec249ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x55aec2479db0 tx=0x7f2098008040 comp rx=0 tx=0).stop 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- 172.21.15.86:0/1451048545 shutdown_connections 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec2487680 0x55aec249f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec247cdc0 0x55aec249ed30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 --2- 172.21.15.86:0/1451048545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec2477a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- 172.21.15.86:0/1451048545 >> 172.21.15.86:0/1451048545 conn(0x55aec2478a30 msgr2=0x55aec2477370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- 172.21.15.86:0/1451048545 shutdown_connections 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- 172.21.15.86:0/1451048545 wait complete. 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 -- start start 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.559+0000 7f20b6d29740 1 Processor -- start 2024-09-05T06:53:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec248fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec248cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01b27e0 con 0x55aec2487680 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01b20e0 con 0x55aec247c4a0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01af340 con 0x55aec247cdc0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 10 client.0 _ll_register_callbacks cb 0x7f20a009b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42824/0 (socket says 172.21.15.86:42824) 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 -- 172.21.15.86:0/67171872 learned_addr learned my addr 172.21.15.86:0/67171872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20add9b640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec248cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20ae59c640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec248fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 -- 172.21.15.86:0/67171872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 msgr2=0x55aec248cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec248cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 -- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 msgr2=0x55aec248fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec248fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 -- 172.21.15.86:0/67171872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec248d960 con 0x55aec2487680 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20aed9d640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f20a8004af0 tx=0x7f20a8031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134ba8734b122155 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f2095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8060060 con 0x55aec2487680 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f20a8004250 con 0x55aec2487680 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.560+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209801c700 con 0x55aec2487680 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f20a8067050 con 0x55aec2487680 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 10 client.18222 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aec2501ae0 con 0x55aec2487680 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209801c700 con 0x55aec2487680 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f20a805a030 con 0x55aec2487680 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 10 client.18222 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 10 client.18222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 20 client.18222 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2502080 0x55aec25044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 4 client.18222 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aec2484b40 con 0x55aec2502080 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.561+0000 7f20ae59c640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2502080 0x55aec25044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f2095ffb640 10 client.18222 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20ae59c640 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2502080 0x55aec25044a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30d2bac9f2b3ded server_cookie=a0b50353950962ac in_seq=0 out_seq=0 2024-09-05T06:53:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f2095ffb640 1 -- 172.21.15.86:0/67171872 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aec2484b40 con 0x55aec2502080 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f2095ffb640 10 client.18222 handle_command_reply: tid=0 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 client.18222 shutdown 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 msgr2=0x55aec248d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f20a8004af0 tx=0x7f20a8031980 comp rx=0 tx=0).stop 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 shutdown_connections 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55aec2502080 0x55aec25044a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec2487680 0x55aec248d320 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec247cdc0 0x55aec248cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 --2- 172.21.15.86:0/67171872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec247c4a0 0x55aec248fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f2095ffb640 0 client.18222 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 >> 172.21.15.86:0/67171872 conn(0x55aec2478a30 msgr2=0x55aec2477b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 shutdown_connections 2024-09-05T06:53:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 1 -- 172.21.15.86:0/67171872 wait complete. 2024-09-05T06:53:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.562+0000 7f20b6d29740 20 client.18222 trim_cache size 0 max 16384 2024-09-05T06:53:18.569 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:18.569 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:53:18.569 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:53:18.569 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 msgr2=0x7f209c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 0x7f209c0a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f20a804b000 tx=0x7f20a8048000 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 msgr2=0x7f20b00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000dd50 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 shutdown_connections 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f209c0a2cf0 0x7f209c0a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b0106f40 0x7f20b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b01027c0 0x7f20b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 --2- 172.21.15.86:0/3837575163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b0101e10 0x7f20b00744f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 >> 172.21.15.86:0/3837575163 conn(0x7f20b00fd8d0 msgr2=0x7f20b00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 shutdown_connections 2024-09-05T06:53:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:18.563+0000 7f20b4950640 1 -- 172.21.15.86:0/3837575163 wait complete. 2024-09-05T06:53:19.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:53:19.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:53:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:19 smithi115 ceph-mon[34791]: pgmap v376: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:19 smithi086 ceph-mon[30943]: pgmap v376: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:19.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209413 1.61 MB/sec execute 519 sec latency 2570.076 ms 2024-09-05T06:53:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:19 smithi096 ceph-mon[34468]: pgmap v376: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 -- 172.21.15.86:0/2037084348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40102570 msgr2=0x7f4f401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 --2- 172.21.15.86:0/2037084348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40102570 0x7f4f401029b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a480 comp rx=0 tx=0).stop 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 -- 172.21.15.86:0/2037084348 shutdown_connections 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 --2- 172.21.15.86:0/2037084348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102ef0 0x7f4f4010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 --2- 172.21.15.86:0/2037084348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40102570 0x7f4f401029b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 --2- 172.21.15.86:0/2037084348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40100e20 0x7f4f40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 -- 172.21.15.86:0/2037084348 >> 172.21.15.86:0/2037084348 conn(0x7f4f400fc7d0 msgr2=0x7f4f400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 -- 172.21.15.86:0/2037084348 shutdown_connections 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.993+0000 7f4f448e5640 1 -- 172.21.15.86:0/2037084348 wait complete. 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 Processor -- start 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 -- start start 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 0x7f4f40198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40102ef0 0x7f4f4019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40110e30 con 0x7f4f40100e20 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40110cb0 con 0x7f4f40102570 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f448e5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40110fb0 con 0x7f4f40102ef0 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42834/0 (socket says 172.21.15.86:42834) 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f3e59c640 1 -- 172.21.15.86:0/3859759653 learned_addr learned my addr 172.21.15.86:0/3859759653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f3ed9d640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40102ef0 0x7f4f4019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.994+0000 7f4f37fff640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 0x7f4f40198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 -- 172.21.15.86:0/3859759653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40102ef0 msgr2=0x7f4f4019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40102ef0 0x7f4f4019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 -- 172.21.15.86:0/3859759653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 msgr2=0x7f4f40198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 0x7f4f40198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 -- 172.21.15.86:0/3859759653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019d3a0 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f37fff640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 0x7f4f40198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f3e59c640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d950 tx=0x7f4f3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017070 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4019d690 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f30009c30 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000f620 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.995+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a4f70 con 0x7f4f40100e20 2024-09-05T06:53:20.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.996+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40101200 con 0x7f4f40100e20 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.997+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4f300109a0 con 0x7f4f40100e20 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.998+0000 7f4f377fe640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 0x7f4f2c0a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.998+0000 7f4f37fff640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 0x7f4f2c0a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.998+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f4f300cf4d0 con 0x7f4f40100e20 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:19.998+0000 7f4f37fff640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 0x7f4f2c0a95c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4f400fef00 tx=0x7f4f28005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:20.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.002+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f30009de0 con 0x7f4f40100e20 2024-09-05T06:53:20.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.203+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f40199750 con 0x7f4f40100e20 2024-09-05T06:53:20.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.204+0000 7f4f377fe640 1 -- 172.21.15.86:0/3859759653 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7f4f3009bcb0 con 0x7f4f40100e20 2024-09-05T06:53:20.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:53:20.204 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:53:20.205 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:53:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 msgr2=0x7f4f2c0a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 0x7f4f2c0a95c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4f400fef00 tx=0x7f4f28005a20 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 msgr2=0x7f4f401983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d950 tx=0x7f4f3000de10 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 shutdown_connections 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4f2c0a7100 0x7f4f2c0a95c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f40102ef0 0x7f4f4019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40102570 0x7f4f40198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 --2- 172.21.15.86:0/3859759653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40100e20 0x7f4f401983f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 >> 172.21.15.86:0/3859759653 conn(0x7f4f400fc7d0 msgr2=0x7f4f40069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 shutdown_connections 2024-09-05T06:53:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:20.206+0000 7f4f448e5640 1 -- 172.21.15.86:0/3859759653 wait complete. 2024-09-05T06:53:20.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209413 1.60 MB/sec execute 520 sec latency 3570.120 ms 2024-09-05T06:53:20.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:53:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:21 smithi086 ceph-mon[30943]: pgmap v377: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:21 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3859759653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:21 smithi115 ceph-mon[34791]: pgmap v377: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:21 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3859759653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:21.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209413 1.60 MB/sec execute 521 sec latency 4570.171 ms 2024-09-05T06:53:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:21 smithi096 ceph-mon[34468]: pgmap v377: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-05T06:53:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:21 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3859759653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- 172.21.15.86:0/2682514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4102790 msgr2=0x7f40b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 --2- 172.21.15.86:0/2682514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4102790 0x7f40b41067e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a802a410 comp rx=0 tx=0).stop 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- 172.21.15.86:0/2682514868 shutdown_connections 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 --2- 172.21.15.86:0/2682514868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40b4106f10 0x7f40b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 --2- 172.21.15.86:0/2682514868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4102790 0x7f40b41067e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 --2- 172.21.15.86:0/2682514868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 0x7f40b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- 172.21.15.86:0/2682514868 >> 172.21.15.86:0/2682514868 conn(0x7f40b40fd8c0 msgr2=0x7f40b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- 172.21.15.86:0/2682514868 shutdown_connections 2024-09-05T06:53:21.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- 172.21.15.86:0/2682514868 wait complete. 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 Processor -- start 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.698+0000 7f40ba69a640 1 -- start start 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 0x7f40b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40b4102790 0x7f40b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4111ec0 con 0x7f40b4106f10 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4111d40 con 0x7f40b4101de0 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40ba69a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112040 con 0x7f40b4102790 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42854/0 (socket says 172.21.15.86:42854) 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 -- 172.21.15.86:0/854236615 learned_addr learned my addr 172.21.15.86:0/854236615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b3fff640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 0x7f40b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 -- 172.21.15.86:0/854236615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40b4102790 msgr2=0x7f40b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40b4102790 0x7f40b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 -- 172.21.15.86:0/854236615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 msgr2=0x7f40b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 0x7f40b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 -- 172.21.15.86:0/854236615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b41a69e0 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b8c37640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00b970 tx=0x7f40ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.699+0000 7f40b17fa640 1 -- 172.21.15.86:0/854236615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac019070 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.700+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b41a6cd0 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.700+0000 7f40b17fa640 1 -- 172.21.15.86:0/854236615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40ac014440 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.700+0000 7f40b17fa640 1 -- 172.21.15.86:0/854236615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0075d0 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.700+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a7260 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40b17fa640 1 -- 172.21.15.86:0/854236615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f40ac01c070 con 0x7f40b4106f10 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 Processor -- start 2024-09-05T06:53:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 -- start start 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40b17fa640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 0x7f409c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea0073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea007e370 0x559ea0080760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e9ffd92f0 con 0x559ea0074b40 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e9ff80cf0 con 0x559ea00735d0 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40bca73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559e9ffd52b0 con 0x559ea007e370 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40b37fe640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 0x7f409c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.702+0000 7f40b17fa640 1 -- 172.21.15.86:0/854236615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f40ac0cedc0 con 0x7f40b4106f10 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42860/0 (socket says 172.21.15.86:42860) 2024-09-05T06:53:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 -- 172.21.15.86:0/1609030222 learned_addr learned my addr 172.21.15.86:0/1609030222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea007e370 0x559ea0080760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 -- 172.21.15.86:0/1609030222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea007e370 msgr2=0x559ea0080760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea007e370 0x559ea0080760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 -- 172.21.15.86:0/1609030222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 msgr2=0x559ea0073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea0073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea0073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 -- 172.21.15.86:0/1609030222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9ff653a0 con 0x559ea0074b40 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea0073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b820 tx=0x7f40a400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dab398651f3c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f40b37fe640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 0x7f409c0a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559ea0070ac0 tx=0x7f40a8000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f409a7fc640 1 -- 172.21.15.86:0/1609030222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a400d950 con 0x559ea0074b40 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f409a7fc640 1 -- 172.21.15.86:0/1609030222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40a4019070 con 0x559ea0074b40 2024-09-05T06:53:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.703+0000 7f409a7fc640 1 -- 172.21.15.86:0/1609030222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40123e0 con 0x559ea0074b40 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- 172.21.15.86:0/1609030222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 msgr2=0x559ea0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b820 tx=0x7f40a400bce0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- 172.21.15.86:0/1609030222 shutdown_connections 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea007e370 0x559ea0080760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea0073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 --2- 172.21.15.86:0/1609030222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea0074b40 0x559ea0073090 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- 172.21.15.86:0/1609030222 >> 172.21.15.86:0/1609030222 conn(0x559ea003c310 msgr2=0x559ea006fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- 172.21.15.86:0/1609030222 shutdown_connections 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- 172.21.15.86:0/1609030222 wait complete. 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.704+0000 7f40bca73740 1 -- start start 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 Processor -- start 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea007dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea007af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ea0081070 con 0x559ea007e370 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e9ff94060 con 0x559ea00735d0 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559ea00811f0 con 0x559ea0074b40 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40bca73740 10 client.0 _ll_register_callbacks cb 0x559e9fce78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42866/0 (socket says 172.21.15.86:42866) 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 -- 172.21.15.86:0/1679813053 learned_addr learned my addr 172.21.15.86:0/1679813053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea007dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea007af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 -- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 msgr2=0x559ea007af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea007af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 -- 172.21.15.86:0/1679813053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 msgr2=0x559ea007dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea007dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 -- 172.21.15.86:0/1679813053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9fd4a3c0 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f40ac097410 tx=0x7f40ac0bb170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b076bf48463826a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.705+0000 7f40937fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac09f360 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f40ac0a3790 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a4019070 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f40ac0a3c20 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 10 client.18240 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559ea008c9d0 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40a4019070 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f40ac095b20 con 0x559ea007e370 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 10 client.18240 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:53:21.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 20 client.18240 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea008de90 0x559ea0090280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 4 client.18240 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9ffe6860 con 0x559ea008de90 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.706+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea008de90 0x559ea0090280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.707+0000 7f40937fe640 10 client.18240 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.707+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea008de90 0x559ea0090280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfd2c7406ee6ffcf server_cookie=9550958565d0edcb in_seq=0 out_seq=0 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.707+0000 7f40937fe640 1 -- 172.21.15.86:0/1679813053 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e9ffe6860 con 0x559ea008de90 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.707+0000 7f40937fe640 10 client.18240 handle_command_reply: tid=0 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.707+0000 7f40bca73740 1 client.18240 shutdown 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 msgr2=0x559ea007b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f40ac097410 tx=0x7f40ac0bb170 comp rx=0 tx=0).stop 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 shutdown_connections 2024-09-05T06:53:21.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea008de90 0x559ea0090280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea007e370 0x559ea007b4a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40937fe640 0 client.18240 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea007af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 --2- 172.21.15.86:0/1679813053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea00735d0 0x559ea007dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 >> 172.21.15.86:0/1679813053 conn(0x559ea003c310 msgr2=0x559ea0075930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 shutdown_connections 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 1 -- 172.21.15.86:0/1679813053 wait complete. 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.708+0000 7f40bca73740 20 client.18240 trim_cache size 0 max 16384 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 Processor -- start 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 -- start start 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea003c310 0x559ea008a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea0095b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b1510 con 0x559ea00735d0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b30b0 con 0x559ea003c310 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40bca73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b1be0 con 0x559ea0074b40 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42872/0 (socket says 172.21.15.86:42872) 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 -- 172.21.15.86:0/1874047700 learned_addr learned my addr 172.21.15.86:0/1874047700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea003c310 0x559ea008a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea0095b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 -- 172.21.15.86:0/1874047700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 msgr2=0x559ea0095b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea0095b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 -- 172.21.15.86:0/1874047700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea003c310 msgr2=0x559ea008a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea003c310 0x559ea008a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.721+0000 7f40b37fe640 1 -- 172.21.15.86:0/1874047700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea0090900 con 0x559ea00735d0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f40a8033000 tx=0x7f40a8037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16e6e5eec6999b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f4099ffb640 1 -- 172.21.15.86:0/1874047700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8045040 con 0x559ea00735d0 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f4099ffb640 1 -- 172.21.15.86:0/1874047700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40a8004250 con 0x559ea00735d0 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f4099ffb640 1 -- 172.21.15.86:0/1874047700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035040 con 0x559ea00735d0 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 -- 172.21.15.86:0/1874047700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 msgr2=0x559ea0072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f40a8033000 tx=0x7f40a8037000 comp rx=0 tx=0).stop 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 -- 172.21.15.86:0/1874047700 shutdown_connections 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea0074b40 0x559ea0095b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea00735d0 0x559ea0072bd0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 --2- 172.21.15.86:0/1874047700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea003c310 0x559ea008a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 -- 172.21.15.86:0/1874047700 >> 172.21.15.86:0/1874047700 conn(0x559ea007b600 msgr2=0x559ea0079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 -- 172.21.15.86:0/1874047700 shutdown_connections 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.722+0000 7f40bca73740 1 -- 172.21.15.86:0/1874047700 wait complete. 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 -- start start 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 Processor -- start 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea00735d0 0x559ea00820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 0x559ea0082840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b29b0 con 0x559ea003c310 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b22b0 con 0x559ea0074b40 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41af510 con 0x559ea00735d0 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40bca73740 10 client.0 _ll_register_callbacks cb 0x7f40ac097250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42888/0 (socket says 172.21.15.86:42888) 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40b3fff640 1 -- 172.21.15.86:0/1312773400 learned_addr learned my addr 172.21.15.86:0/1312773400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.723+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea00735d0 0x559ea00820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 0x559ea0082840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 -- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea00735d0 msgr2=0x559ea00820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea00735d0 0x559ea00820b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 -- 172.21.15.86:0/1312773400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 msgr2=0x559ea0082840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 0x559ea0082840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 -- 172.21.15.86:0/1312773400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea00f7b20 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b8c37640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 0x559ea0082840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40b3fff640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b7f0 tx=0x7f40a400b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436425a83a392649 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a400dd60 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f40a4014430 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a8004250 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f40a4014630 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 10 client.18246 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559ea00f8ce0 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40a8004250 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f40a4013070 con 0x559ea003c310 2024-09-05T06:53:21.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 10 client.18246 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 10 client.18246 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.724+0000 7f40bca73740 20 client.18246 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea007e370 0x559ea00fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40bca73740 4 client.18246 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e9ffe0d70 con 0x559ea007e370 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea007e370 0x559ea00fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40937fe640 10 client.18246 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40b37fe640 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea007e370 0x559ea00fb190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94fb488826c2f3ac server_cookie=4096aceb460b85c7 in_seq=0 out_seq=0 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40937fe640 1 -- 172.21.15.86:0/1312773400 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e9ffe0d70 con 0x559ea007e370 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40937fe640 10 client.18246 handle_command_reply: tid=0 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.725+0000 7f40bca73740 1 client.18246 shutdown 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 msgr2=0x559ea0086d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b7f0 tx=0x7f40a400b320 comp rx=0 tx=0).stop 2024-09-05T06:53:21.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 shutdown_connections 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559ea007e370 0x559ea00fb190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ea0074b40 0x559ea0082840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ea00735d0 0x559ea00820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40937fe640 0 client.18246 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 --2- 172.21.15.86:0/1312773400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea003c310 0x559ea0086d80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 >> 172.21.15.86:0/1312773400 conn(0x559ea007b600 msgr2=0x559ea007c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 shutdown_connections 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 1 -- 172.21.15.86:0/1312773400 wait complete. 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40bca73740 20 client.18246 trim_cache size 0 max 16384 2024-09-05T06:53:21.742 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 msgr2=0x7f409c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 0x7f409c0a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559ea0070ac0 tx=0x7f40a8000bc0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 msgr2=0x7f40b4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.726+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00b970 tx=0x7f40ac00be30 comp rx=0 tx=0).stop 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 shutdown_connections 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f409c0a2cf0 0x7f409c0a51b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b4106f10 0x7f40b4074f70 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40b4102790 0x7f40b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 --2- 172.21.15.86:0/854236615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b4101de0 0x7f40b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:21.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 >> 172.21.15.86:0/854236615 conn(0x7f40b40fd8c0 msgr2=0x7f40b40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:21.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 shutdown_connections 2024-09-05T06:53:21.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:21.727+0000 7f40ba69a640 1 -- 172.21.15.86:0/854236615 wait complete. 2024-09-05T06:53:22.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209662 1.60 MB/sec execute 522 sec latency 4779.056 ms 2024-09-05T06:53:22.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:53:22.817 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:23.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:23 smithi086 ceph-mon[30943]: pgmap v378: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:53:23.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- 172.21.15.86:0/3243059736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0100e20 msgr2=0x7f99c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 --2- 172.21.15.86:0/3243059736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0100e20 0x7f99c0101200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b002a480 comp rx=0 tx=0).stop 2024-09-05T06:53:23.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- 172.21.15.86:0/3243059736 shutdown_connections 2024-09-05T06:53:23.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 --2- 172.21.15.86:0/3243059736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0102ef0 0x7f99c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 --2- 172.21.15.86:0/3243059736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 0x7f99c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 --2- 172.21.15.86:0/3243059736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0100e20 0x7f99c0101200 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- 172.21.15.86:0/3243059736 >> 172.21.15.86:0/3243059736 conn(0x7f99c00fc7d0 msgr2=0x7f99c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- 172.21.15.86:0/3243059736 shutdown_connections 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- 172.21.15.86:0/3243059736 wait complete. 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 Processor -- start 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.299+0000 7f99c5698640 1 -- start start 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 0x7f99c01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 0x7f99c0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0110e30 con 0x7f99c0102ef0 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0110cb0 con 0x7f99c0100e20 2024-09-05T06:53:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0110fb0 con 0x7f99c0102570 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 0x7f99c01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55400/0 (socket says 172.21.15.86:55400) 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1349337514 learned_addr learned my addr 172.21.15.86:0/1349337514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99b7fff640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 0x7f99c0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1349337514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 msgr2=0x7f99c0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 0x7f99c0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1349337514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 msgr2=0x7f99c01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 0x7f99c01992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1349337514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c01a48a0 con 0x7f99c0102ef0 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99beffd640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 0x7f99c01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:23.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f99b800eb10 tx=0x7f99b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bcff9640 1 -- 172.21.15.86:0/1349337514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004b20 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bcff9640 1 -- 172.21.15.86:0/1349337514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b8004cc0 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99bcff9640 1 -- 172.21.15.86:0/1349337514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010690 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c01a4a70 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.300+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c01a4f30 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 Processor -- start 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99bcff9640 1 -- 172.21.15.86:0/1349337514 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f99b8005020 con 0x7f99c0102ef0 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 -- start start 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 0x56281d499090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56281d3ff2f0 con 0x56281d4995d0 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56281d3a6cf0 con 0x56281d4a4370 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.303+0000 7f99c7a71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56281d3fb2b0 con 0x56281d49ab40 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55412/0 (socket says 172.21.15.86:55412) 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 -- 172.21.15.86:0/465025590 learned_addr learned my addr 172.21.15.86:0/465025590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 0x56281d499090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99b7fff640 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 -- 172.21.15.86:0/465025590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 msgr2=0x56281d499090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 0x56281d499090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 -- 172.21.15.86:0/465025590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 msgr2=0x56281d4a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 -- 172.21.15.86:0/465025590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281d38b3a0 con 0x56281d4995d0 2024-09-05T06:53:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99b7fff640 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 0x56281d499090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99bcff9640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 0x7f99ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99b7fff640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 0x7f99ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99bcff9640 1 -- 172.21.15.86:0/1349337514 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f99b80cfe50 con 0x7f99c0102ef0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.304+0000 7f99beffd640 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c4a05050e25483 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99b57fa640 1 -- 172.21.15.86:0/465025590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x56281d4995d0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99b57fa640 1 -- 172.21.15.86:0/465025590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b00317b0 con 0x56281d4995d0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99b57fa640 1 -- 172.21.15.86:0/465025590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0004a00 con 0x56281d4995d0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 -- 172.21.15.86:0/465025590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 msgr2=0x56281d499a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99b7fff640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 0x7f99ac0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x56281d49a070 tx=0x7f99a8005d20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b0032f90 comp rx=0 tx=0).stop 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 -- 172.21.15.86:0/465025590 shutdown_connections 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d499a10 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.305+0000 7f99c7a71740 1 --2- 172.21.15.86:0/465025590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d49ab40 0x56281d499090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- 172.21.15.86:0/465025590 >> 172.21.15.86:0/465025590 conn(0x56281d462310 msgr2=0x56281d495c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- 172.21.15.86:0/465025590 shutdown_connections 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- 172.21.15.86:0/465025590 wait complete. 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- start start 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 Processor -- start 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4995d0 0x56281d4a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d4a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56281d4a7010 con 0x56281d49ab40 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56281d3ba060 con 0x56281d4995d0 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56281d4a7190 con 0x56281d4a4370 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99c7a71740 10 client.0 _ll_register_callbacks cb 0x56281d10d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55414/0 (socket says 172.21.15.86:55414) 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.306+0000 7f99b7fff640 1 -- 172.21.15.86:0/391724458 learned_addr learned my addr 172.21.15.86:0/391724458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d4a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99beffd640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4995d0 0x56281d4a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 -- 172.21.15.86:0/391724458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 msgr2=0x56281d4a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d4a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 -- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4995d0 msgr2=0x56281d4a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4995d0 0x56281d4a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 -- 172.21.15.86:0/391724458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281d1703c0 con 0x56281d49ab40 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99b7fff640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f99a8013e60 tx=0x7f99a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3916dbf536908a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80295f0 con 0x56281d49ab40 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b00317b0 con 0x56281d49ab40 2024-09-05T06:53:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.307+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f99a8029790 con 0x56281d49ab40 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99a8034050 con 0x56281d49ab40 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 10 client.18258 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56281d4b2af0 con 0x56281d49ab40 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b00317b0 con 0x56281d49ab40 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f99a801d760 con 0x56281d49ab40 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 10 client.18258 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 10 client.18258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 20 client.18258 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d4b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 4 client.18258 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56281d40c860 con 0x56281d4b3fc0 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.308+0000 7f99beffd640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d4b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.309+0000 7f99beffd640 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d4b6400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=facd2ccbfd3f2a5d server_cookie=598535e90c2aa in_seq=0 out_seq=0 2024-09-05T06:53:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.309+0000 7f99a67fc640 10 client.18258 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99a67fc640 1 -- 172.21.15.86:0/391724458 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56281d40c860 con 0x56281d4b3fc0 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99a67fc640 10 client.18258 handle_command_reply: tid=0 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 client.18258 shutdown 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 msgr2=0x56281d4a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f99a8013e60 tx=0x7f99a80046c0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 shutdown_connections 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d4b6400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99a67fc640 0 client.18258 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d4a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d4a0fe0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 --2- 172.21.15.86:0/391724458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4995d0 0x56281d4a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 >> 172.21.15.86:0/391724458 conn(0x56281d462310 msgr2=0x56281d49b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 shutdown_connections 2024-09-05T06:53:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 1 -- 172.21.15.86:0/391724458 wait complete. 2024-09-05T06:53:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.310+0000 7f99c7a71740 20 client.18258 trim_cache size 0 max 16384 2024-09-05T06:53:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.324+0000 7f99c7a71740 1 Processor -- start 2024-09-05T06:53:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.324+0000 7f99c7a71740 1 -- start start 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4995d0 0x56281d4b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d49ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01af180 con 0x56281d49ab40 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01b0d20 con 0x56281d4a4370 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99c7a71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01af850 con 0x56281d4995d0 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d49ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d49ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55418/0 (socket says 172.21.15.86:55418) 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99b7fff640 1 -- 172.21.15.86:0/1594960892 learned_addr learned my addr 172.21.15.86:0/1594960892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99beffd640 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4995d0 0x56281d4b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1594960892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4995d0 msgr2=0x56281d4b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4995d0 0x56281d4b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1594960892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 msgr2=0x56281d49ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d49ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 -- 172.21.15.86:0/1594960892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281d49cbc0 con 0x56281d4a4370 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4bbcd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f99b80cd160 tx=0x7f99b809bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685dc3c0264788cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.325+0000 7f99b4ff9640 1 -- 172.21.15.86:0/1594960892 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b809bf30 con 0x56281d4a4370 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99b4ff9640 1 -- 172.21.15.86:0/1594960892 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b80a02e0 con 0x56281d4a4370 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99b4ff9640 1 -- 172.21.15.86:0/1594960892 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b80a4710 con 0x56281d4a4370 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 -- 172.21.15.86:0/1594960892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 msgr2=0x56281d4bbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4bbcd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f99b80cd160 tx=0x7f99b809bdc0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 -- 172.21.15.86:0/1594960892 shutdown_connections 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d4a4370 0x56281d4bbcd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d49ab40 0x56281d49ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 --2- 172.21.15.86:0/1594960892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4995d0 0x56281d4b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 -- 172.21.15.86:0/1594960892 >> 172.21.15.86:0/1594960892 conn(0x56281d462310 msgr2=0x56281d49f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.326+0000 7f99c7a71740 1 -- 172.21.15.86:0/1594960892 shutdown_connections 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 -- 172.21.15.86:0/1594960892 wait complete. 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 -- start start 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 Processor -- start 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d49ab40 0x56281d49dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d49e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01b0620 con 0x56281d4995d0 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01aff20 con 0x56281d49ab40 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99c01ad180 con 0x56281d4a4370 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d49ab40 0x56281d49dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55422/0 (socket says 172.21.15.86:55422) 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99beffd640 1 -- 172.21.15.86:0/856631960 learned_addr learned my addr 172.21.15.86:0/856631960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.327+0000 7f99c7a71740 10 client.0 _ll_register_callbacks cb 0x7f99a8013ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99bf7fe640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d49e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 -- 172.21.15.86:0/856631960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 msgr2=0x56281d49e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d49e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 -- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d49ab40 msgr2=0x56281d49dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d49ab40 0x56281d49dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 -- 172.21.15.86:0/856631960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281d49ea30 con 0x56281d4995d0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99beffd640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f99b00096c0 tx=0x7f99b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52809893a7799cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x56281d4995d0 2024-09-05T06:53:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f99b0004040 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b80a02e0 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99b0062050 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99c7a71740 10 client.18264 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.328+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56281d51ec90 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b80a02e0 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f99b005d030 con 0x56281d4995d0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 10 client.18264 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 10 client.18264 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 20 client.18264 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d5210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 4 client.18264 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56281d406d70 con 0x56281d4b3fc0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.329+0000 7f99b7fff640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d5210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99b7fff640 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d5210a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25c0e86ea0b46577 server_cookie=7acaf309d90a2517 in_seq=0 out_seq=0 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99a67fc640 10 client.18264 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99a67fc640 1 -- 172.21.15.86:0/856631960 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56281d406d70 con 0x56281d4b3fc0 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99a67fc640 10 client.18264 handle_command_reply: tid=0 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 client.18264 shutdown 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 msgr2=0x56281d4acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f99b00096c0 tx=0x7f99b0009780 comp rx=0 tx=0).stop 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 shutdown_connections 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56281d4b3fc0 0x56281d5210a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56281d4a4370 0x56281d49e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56281d49ab40 0x56281d49dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 --2- 172.21.15.86:0/856631960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56281d4995d0 0x56281d4acb60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 >> 172.21.15.86:0/856631960 conn(0x56281d462310 msgr2=0x56281d4a00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99a67fc640 0 client.18264 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:53:23.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 shutdown_connections 2024-09-05T06:53:23.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 1 -- 172.21.15.86:0/856631960 wait complete. 2024-09-05T06:53:23.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.330+0000 7f99c7a71740 20 client.18264 trim_cache size 0 max 16384 2024-09-05T06:53:23.345 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 msgr2=0x7f99ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 0x7f99ac0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x56281d49a070 tx=0x7f99a8005d20 comp rx=0 tx=0).stop 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 msgr2=0x7f99c01a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f99b800eb10 tx=0x7f99b800c2d0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 shutdown_connections 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99ac0a2cf0 0x7f99ac0a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4360 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c0102570 0x7f99c0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 --2- 172.21.15.86:0/1349337514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c0100e20 0x7f99c01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 >> 172.21.15.86:0/1349337514 conn(0x7f99c00fc7d0 msgr2=0x7f99c00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 shutdown_connections 2024-09-05T06:53:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:23.331+0000 7f99c5698640 1 -- 172.21.15.86:0/1349337514 wait complete. 2024-09-05T06:53:23.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:23 smithi115 ceph-mon[34791]: pgmap v378: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:53:23.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 210013 1.60 MB/sec execute 523 sec latency 19.844 ms 2024-09-05T06:53:23.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:23 smithi096 ceph-mon[34468]: pgmap v378: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-05T06:53:24.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:53:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:24.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 210566 1.60 MB/sec execute 524 sec latency 13.832 ms 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 -- 172.21.15.86:0/3770193049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c101420 msgr2=0x7f967c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 --2- 172.21.15.86:0/3770193049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c101420 0x7f967c10d950 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9674009cd0 tx=0x7f967402a8e0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 -- 172.21.15.86:0/3770193049 shutdown_connections 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 --2- 172.21.15.86:0/3770193049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c101420 0x7f967c10d950 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 --2- 172.21.15.86:0/3770193049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 0x7f967c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 --2- 172.21.15.86:0/3770193049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c106a80 0x7f967c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 -- 172.21.15.86:0/3770193049 >> 172.21.15.86:0/3770193049 conn(0x7f967c0fc7a0 msgr2=0x7f967c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 -- 172.21.15.86:0/3770193049 shutdown_connections 2024-09-05T06:53:24.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.855+0000 7f9682461640 1 -- 172.21.15.86:0/3770193049 wait complete. 2024-09-05T06:53:24.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 Processor -- start 2024-09-05T06:53:24.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 -- start start 2024-09-05T06:53:24.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 0x7f967c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c101420 0x7f967c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f967c110d30 con 0x7f967c106a80 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f967c110bb0 con 0x7f967c101420 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.856+0000 7f9682461640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f967c110eb0 con 0x7f967c100a80 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55440/0 (socket says 172.21.15.86:55440) 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 -- 172.21.15.86:0/1348601331 learned_addr learned my addr 172.21.15.86:0/1348601331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f967bfff640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 0x7f967c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f9673fff640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c101420 0x7f967c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 -- 172.21.15.86:0/1348601331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 msgr2=0x7f967c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 0x7f967c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 -- 172.21.15.86:0/1348601331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c101420 msgr2=0x7f967c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c101420 0x7f967c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 -- 172.21.15.86:0/1348601331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c19d2a0 con 0x7f967c106a80 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f96809fe640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9674033040 tx=0x7f9674003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f9679ffb640 1 -- 172.21.15.86:0/1348601331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674004040 con 0x7f967c106a80 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f9679ffb640 1 -- 172.21.15.86:0/1348601331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9674004690 con 0x7f967c106a80 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.857+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967c19d530 con 0x7f967c106a80 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.858+0000 7f9679ffb640 1 -- 172.21.15.86:0/1348601331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674030e10 con 0x7f967c106a80 2024-09-05T06:53:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.858+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f967c1a4e70 con 0x7f967c106a80 2024-09-05T06:53:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.860+0000 7f9679ffb640 1 -- 172.21.15.86:0/1348601331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f96740305a0 con 0x7f967c106a80 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.860+0000 7f968483a740 1 Processor -- start 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.860+0000 7f9679ffb640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 0x7f96680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 -- start start 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9673fff640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 0x7f96680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9679ffb640 1 -- 172.21.15.86:0/1348601331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f967402e030 con 0x7f967c106a80 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 0x5557fe203090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe203a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe214b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557fe1692f0 con 0x5557fe212780 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557fe110cf0 con 0x5557fe2035d0 2024-09-05T06:53:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f968483a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557fe1652b0 con 0x5557fe204b40 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9673fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe203a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 0x5557fe203090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9673fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe203a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50270/0 (socket says 172.21.15.86:50270) 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f96809fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe214b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9673fff640 1 -- 172.21.15.86:0/3769296761 learned_addr learned my addr 172.21.15.86:0/3769296761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 -- 172.21.15.86:0/3769296761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 msgr2=0x5557fe203a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe203a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 -- 172.21.15.86:0/3769296761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 msgr2=0x5557fe214b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe214b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f967bfff640 1 -- 172.21.15.86:0/3769296761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fe0f53a0 con 0x5557fe204b40 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.861+0000 7f9673fff640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 0x7f96680a5230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9664014660 tx=0x7f9664014a60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.862+0000 7f967bfff640 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 0x5557fe203090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f966c009c20 tx=0x7f966c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e9a3392c1007c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.862+0000 7f96727fc640 1 -- 172.21.15.86:0/3769296761 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c004050 con 0x5557fe204b40 2024-09-05T06:53:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.862+0000 7f96727fc640 1 -- 172.21.15.86:0/3769296761 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f966c0054d0 con 0x5557fe204b40 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.862+0000 7f96727fc640 1 -- 172.21.15.86:0/3769296761 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c004cc0 con 0x5557fe204b40 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- 172.21.15.86:0/3769296761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 msgr2=0x5557fe203090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 0x5557fe203090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f966c009c20 tx=0x7f966c011810 comp rx=0 tx=0).stop 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- 172.21.15.86:0/3769296761 shutdown_connections 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe214b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe203a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 --2- 172.21.15.86:0/3769296761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe204b40 0x5557fe203090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- 172.21.15.86:0/3769296761 >> 172.21.15.86:0/3769296761 conn(0x5557fe1cc310 msgr2=0x5557fe1ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- 172.21.15.86:0/3769296761 shutdown_connections 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- 172.21.15.86:0/3769296761 wait complete. 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 -- start start 2024-09-05T06:53:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.863+0000 7f968483a740 1 Processor -- start 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe212110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 0x5557fe20f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557fe215480 con 0x5557fe204b40 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557fe124060 con 0x5557fe2035d0 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557fe215600 con 0x5557fe212780 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f968483a740 10 client.0 _ll_register_callbacks cb 0x5557fde778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55460/0 (socket says 172.21.15.86:55460) 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f967bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe212110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 -- 172.21.15.86:0/499164569 learned_addr learned my addr 172.21.15.86:0/499164569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f96809fe640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 0x5557fe20f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 -- 172.21.15.86:0/499164569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 msgr2=0x5557fe20f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 0x5557fe20f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 -- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 msgr2=0x5557fe212110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe212110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 -- 172.21.15.86:0/499164569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fdeda3c0 con 0x5557fe204b40 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f96809fe640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 0x5557fe20f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f9673fff640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9664005940 tx=0x7f9664004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40ac6978ec0bac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f965b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.864+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664020850 con 0x5557fe204b40 2024-09-05T06:53:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f96640209f0 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f966c0054d0 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9664031050 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 10 client.18276 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5557fe220f00 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f966c0054d0 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f9664011c20 con 0x5557fe204b40 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 10 client.18276 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 10 client.18276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 20 client.18276 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe222410 0x5557fe224850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 4 client.18276 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.865+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557fe176860 con 0x5557fe222410 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.866+0000 7f967bfff640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe222410 0x5557fe224850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.866+0000 7f967bfff640 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe222410 0x5557fe224850 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b027eaee58c4b653 server_cookie=868ff88a91cf0d1b in_seq=0 out_seq=0 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.866+0000 7f965b7fe640 10 client.18276 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f965b7fe640 1 -- 172.21.15.86:0/499164569 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557fe176860 con 0x5557fe222410 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f965b7fe640 10 client.18276 handle_command_reply: tid=0 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 client.18276 shutdown 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 msgr2=0x5557fe20f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9664005940 tx=0x7f9664004730 comp rx=0 tx=0).stop 2024-09-05T06:53:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 shutdown_connections 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe222410 0x5557fe224850 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe212780 0x5557fe20f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe204b40 0x5557fe20f3c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 --2- 172.21.15.86:0/499164569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe212110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 >> 172.21.15.86:0/499164569 conn(0x5557fe1cc310 msgr2=0x5557fe205960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f965b7fe640 0 client.18276 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 shutdown_connections 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 1 -- 172.21.15.86:0/499164569 wait complete. 2024-09-05T06:53:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.867+0000 7f968483a740 20 client.18276 trim_cache size 0 max 16384 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.881+0000 7f968483a740 1 Processor -- start 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.881+0000 7f968483a740 1 -- start start 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe204410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe22a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1ace40 con 0x5557fe212780 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1ae9e0 con 0x5557fe2035d0 2024-09-05T06:53:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f968483a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1ad510 con 0x5557fe20e3a0 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55474/0 (socket says 172.21.15.86:55474) 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f967bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe204410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 -- 172.21.15.86:0/3026052255 learned_addr learned my addr 172.21.15.86:0/3026052255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f96809fe640 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe22a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 -- 172.21.15.86:0/3026052255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 msgr2=0x5557fe22a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe22a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 -- 172.21.15.86:0/3026052255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 msgr2=0x5557fe204410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe204410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 -- 172.21.15.86:0/3026052255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fe206120 con 0x5557fe212780 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9673fff640 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x5557fe210a00 tx=0x7f966400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaa49b700f46231 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9671ffb640 1 -- 172.21.15.86:0/3026052255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664039070 con 0x5557fe212780 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9671ffb640 1 -- 172.21.15.86:0/3026052255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9664010e50 con 0x5557fe212780 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.882+0000 7f9671ffb640 1 -- 172.21.15.86:0/3026052255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96640208b0 con 0x5557fe212780 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 -- 172.21.15.86:0/3026052255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 msgr2=0x5557fe21e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x5557fe210a00 tx=0x7f966400bf80 comp rx=0 tx=0).stop 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 -- 172.21.15.86:0/3026052255 shutdown_connections 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe22a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe212780 0x5557fe21e720 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 --2- 172.21.15.86:0/3026052255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe2035d0 0x5557fe204410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 -- 172.21.15.86:0/3026052255 >> 172.21.15.86:0/3026052255 conn(0x5557fe1cc310 msgr2=0x5557fe20dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 -- 172.21.15.86:0/3026052255 shutdown_connections 2024-09-05T06:53:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.883+0000 7f968483a740 1 -- 172.21.15.86:0/3026052255 wait complete. 2024-09-05T06:53:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 -- start start 2024-09-05T06:53:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 Processor -- start 2024-09-05T06:53:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe216c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe212780 0x5557fe217140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1ae2e0 con 0x5557fe2035d0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1adbe0 con 0x5557fe212780 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1aae40 con 0x5557fe20e3a0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f968483a740 10 client.0 _ll_register_callbacks cb 0x7f9664005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.884+0000 7f967bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f967bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55490/0 (socket says 172.21.15.86:55490) 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f967bfff640 1 -- 172.21.15.86:0/1732210363 learned_addr learned my addr 172.21.15.86:0/1732210363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe216c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f96809fe640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe212780 0x5557fe217140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 -- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe212780 msgr2=0x5557fe217140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe212780 0x5557fe217140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 -- 172.21.15.86:0/1732210363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 msgr2=0x5557fe2166c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 -- 172.21.15.86:0/1732210363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fe28c1c0 con 0x5557fe20e3a0 2024-09-05T06:53:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f967bfff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f9673fff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe216c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5557fe210a00 tx=0x7f9664020880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aef2a00eba1670a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f965b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664039040 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f96640287e0 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.885+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9664010e50 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9664028590 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 10 client.27322 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5557fe28d4a0 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9664010e50 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f9664011640 con 0x5557fe20e3a0 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 10 client.27322 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 10 client.27322 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 20 client.27322 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe28db50 0x5557fe28ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 4 client.27322 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557fe170d70 con 0x5557fe28db50 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.886+0000 7f967bfff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe28db50 0x5557fe28ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f965b7fe640 10 client.27322 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f967bfff640 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe28db50 0x5557fe28ff70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39d32eb547f31f94 server_cookie=1e382a5379e55749 in_seq=0 out_seq=0 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f965b7fe640 1 -- 172.21.15.86:0/1732210363 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557fe170d70 con 0x5557fe28db50 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f965b7fe640 10 client.27322 handle_command_reply: tid=0 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 client.27322 shutdown 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 msgr2=0x5557fe216c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe216c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5557fe210a00 tx=0x7f9664020880 comp rx=0 tx=0).stop 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 shutdown_connections 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5557fe28db50 0x5557fe28ff70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fe212780 0x5557fe217140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557fe20e3a0 0x5557fe216c00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f965b7fe640 0 client.27322 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 --2- 172.21.15.86:0/1732210363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557fe2035d0 0x5557fe2166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 >> 172.21.15.86:0/1732210363 conn(0x5557fe1cc310 msgr2=0x5557fe1fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 shutdown_connections 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 1 -- 172.21.15.86:0/1732210363 wait complete. 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.887+0000 7f968483a740 20 client.27322 trim_cache size 0 max 16384 2024-09-05T06:53:24.893 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 msgr2=0x7f96680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 0x7f96680a5230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9664014660 tx=0x7f9664014a60 comp rx=0 tx=0).stop 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 msgr2=0x7f967c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9674033040 tx=0x7f9674003970 comp rx=0 tx=0).stop 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 shutdown_connections 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96680a2d70 0x7f96680a5230 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c106a80 0x7f967c19cbc0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f967c101420 0x7f967c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 --2- 172.21.15.86:0/1348601331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f967c100a80 0x7f967c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.888+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 >> 172.21.15.86:0/1348601331 conn(0x7f967c0fc7a0 msgr2=0x7f967c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.889+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 shutdown_connections 2024-09-05T06:53:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:24.889+0000 7f9682461640 1 -- 172.21.15.86:0/1348601331 wait complete. 2024-09-05T06:53:25.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:25 smithi115 ceph-mon[34791]: pgmap v379: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 937 KiB/s wr, 79 op/s 2024-09-05T06:53:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:25 smithi086 ceph-mon[30943]: pgmap v379: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 937 KiB/s wr, 79 op/s 2024-09-05T06:53:25.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 211127 1.60 MB/sec execute 525 sec latency 27.618 ms 2024-09-05T06:53:25.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:25 smithi096 ceph-mon[34468]: pgmap v379: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 937 KiB/s wr, 79 op/s 2024-09-05T06:53:25.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:53:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 -- 172.21.15.86:0/2833036030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 msgr2=0x7f58e4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 --2- 172.21.15.86:0/2833036030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4103170 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009930 tx=0x7f58e002a420 comp rx=0 tx=0).stop 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 -- 172.21.15.86:0/2833036030 shutdown_connections 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 --2- 172.21.15.86:0/2833036030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4108290 0x7f58e410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 --2- 172.21.15.86:0/2833036030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4103740 0x7f58e4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 --2- 172.21.15.86:0/2833036030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4103170 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 -- 172.21.15.86:0/2833036030 >> 172.21.15.86:0/2833036030 conn(0x7f58e40fc7d0 msgr2=0x7f58e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 -- 172.21.15.86:0/2833036030 shutdown_connections 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.435+0000 7f58ebc76640 1 -- 172.21.15.86:0/2833036030 wait complete. 2024-09-05T06:53:26.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 Processor -- start 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 -- start start 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4103740 0x7f58e4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 0x7f58e419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4111f40 con 0x7f58e4102d90 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4111dc0 con 0x7f58e4103740 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ebc76640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41120c0 con 0x7f58e4108290 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55516/0 (socket says 172.21.15.86:55516) 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 -- 172.21.15.86:0/1416938298 learned_addr learned my addr 172.21.15.86:0/1416938298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9211640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4103740 0x7f58e4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 -- 172.21.15.86:0/1416938298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 msgr2=0x7f58e419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ea213640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 0x7f58e419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 0x7f58e419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 -- 172.21.15.86:0/1416938298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4103740 msgr2=0x7f58e4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4103740 0x7f58e4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58e9a12640 1 -- 172.21.15.86:0/1416938298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e419d4b0 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.436+0000 7f58ea213640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 0x7f58e419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58e9a12640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006010 tx=0x7f58e0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58daffd640 1 -- 172.21.15.86:0/1416938298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e003f030 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58daffd640 1 -- 172.21.15.86:0/1416938298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e0032de0 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e419d740 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a5080 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.437+0000 7f58daffd640 1 -- 172.21.15.86:0/1416938298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0035070 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58daffd640 1 -- 172.21.15.86:0/1416938298 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f58e0042030 con 0x7f58e4102d90 2024-09-05T06:53:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 Processor -- start 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 -- start start 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2905172f0 con 0x55b2905b15d0 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2904becf0 con 0x55b2905b2b40 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.440+0000 7f58ee04f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b2905132b0 con 0x55b2905c0780 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9211640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55518/0 (socket says 172.21.15.86:55518) 2024-09-05T06:53:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9211640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44094/0 (socket says 172.21.15.86:44094) 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 -- 172.21.15.86:0/618010839 learned_addr learned my addr 172.21.15.86:0/618010839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 -- 172.21.15.86:0/618010839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 msgr2=0x55b2905c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 -- 172.21.15.86:0/618010839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 msgr2=0x55b2905b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58ea213640 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58e9a12640 1 -- 172.21.15.86:0/618010839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2904a33a0 con 0x55b2905b15d0 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.441+0000 7f58daffd640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 0x7f58d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.442+0000 7f58ea213640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 0x7f58d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.442+0000 7f58e9a12640 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ece70 tx=0x7f58e00f3ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0378112971b2cea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.442+0000 7f58daffd640 1 -- 172.21.15.86:0/1416938298 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f58e00efbf0 con 0x7f58e4102d90 2024-09-05T06:53:26.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.442+0000 7f58cffff640 1 -- 172.21.15.86:0/618010839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0049070 con 0x55b2905b15d0 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.442+0000 7f58cffff640 1 -- 172.21.15.86:0/618010839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e00bc3d0 con 0x55b2905b15d0 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58cffff640 1 -- 172.21.15.86:0/618010839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00bbaa0 con 0x55b2905b15d0 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- 172.21.15.86:0/618010839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 msgr2=0x55b2905b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ea213640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 0x7f58d00a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f58dc005fd0 tx=0x7f58dc009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ece70 tx=0x7f58e00f3ec0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- 172.21.15.86:0/618010839 shutdown_connections 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b1a10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 --2- 172.21.15.86:0/618010839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- 172.21.15.86:0/618010839 >> 172.21.15.86:0/618010839 conn(0x55b29057a310 msgr2=0x55b2905adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- 172.21.15.86:0/618010839 shutdown_connections 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- 172.21.15.86:0/618010839 wait complete. 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.443+0000 7f58ee04f740 1 -- start start 2024-09-05T06:53:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 Processor -- start 2024-09-05T06:53:26.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 0x55b2905c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b2b40 0x55b2905b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2905b4310 con 0x55b2905c0780 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2904d2060 con 0x55b2905b15d0 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b2905b4490 con 0x55b2905b2b40 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ee04f740 10 client.0 _ll_register_callbacks cb 0x55b2902258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ea213640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ea213640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55520/0 (socket says 172.21.15.86:55520) 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58ea213640 1 -- 172.21.15.86:0/3836490655 learned_addr learned my addr 172.21.15.86:0/3836490655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 0x55b2905c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.444+0000 7f58e9211640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b2b40 0x55b2905b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 -- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b2b40 msgr2=0x55b2905b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b2b40 0x55b2905b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 -- 172.21.15.86:0/3836490655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 msgr2=0x55b2905c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 0x55b2905c0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 -- 172.21.15.86:0/3836490655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2902883c0 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 0x55b2905c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ea213640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0127a0 tx=0x7f58dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7f2391ebf73fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc023070 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f58dc011b60 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e00bc3d0 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58dc02f050 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ee04f740 10 client.18294 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b2905ca8c0 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.445+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e00bc3d0 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f58dc00f040 con 0x55b2905c0780 2024-09-05T06:53:26.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 10 client.18294 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 10 client.18294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 20 client.18294 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b2905cbd20 0x55b2905ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 4 client.18294 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b290524860 con 0x55b2905cbd20 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b2905cbd20 0x55b2905ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58cd7fa640 10 client.18294 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.446+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b2905cbd20 0x55b2905ce190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=356f2815d3a59278 server_cookie=1b2afed4dad50617 in_seq=0 out_seq=0 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3836490655 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b290524860 con 0x55b2905cbd20 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58cd7fa640 10 client.18294 handle_command_reply: tid=0 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 client.18294 shutdown 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 msgr2=0x55b2905b9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0127a0 tx=0x7f58dc004ab0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 shutdown_connections 2024-09-05T06:53:26.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b2905cbd20 0x55b2905ce190 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905b9a30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b2b40 0x55b2905b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3836490655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b15d0 0x55b2905c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58cd7fa640 0 client.18294 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 >> 172.21.15.86:0/3836490655 conn(0x55b29057a310 msgr2=0x55b2905b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 shutdown_connections 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 1 -- 172.21.15.86:0/3836490655 wait complete. 2024-09-05T06:53:26.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.447+0000 7f58ee04f740 20 client.18294 trim_cache size 0 max 16384 2024-09-05T06:53:26.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 Processor -- start 2024-09-05T06:53:26.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 -- start start 2024-09-05T06:53:26.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ad010 con 0x55b2905b15d0 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41aebb0 con 0x55b2905b2b40 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ee04f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ad6e0 con 0x55b2905c0780 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55532/0 (socket says 172.21.15.86:55532) 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58e9211640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58e9a12640 1 -- 172.21.15.86:0/2582763769 learned_addr learned my addr 172.21.15.86:0/2582763769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.461+0000 7f58ea213640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 -- 172.21.15.86:0/2582763769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 msgr2=0x55b2905d3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905d3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 -- 172.21.15.86:0/2582763769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 msgr2=0x55b2905d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 -- 172.21.15.86:0/2582763769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2905ba030 con 0x55b2905b15d0 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9211640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58e9a12640 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f58e00bbc60 tx=0x7f58e0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678be1c802fa4e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58cf7fe640 1 -- 172.21.15.86:0/2582763769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0049040 con 0x55b2905b15d0 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58cf7fe640 1 -- 172.21.15.86:0/2582763769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e00bbcd0 con 0x55b2905b15d0 2024-09-05T06:53:26.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.462+0000 7f58cf7fe640 1 -- 172.21.15.86:0/2582763769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00bb630 con 0x55b2905b15d0 2024-09-05T06:53:26.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- 172.21.15.86:0/2582763769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 msgr2=0x55b2905b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f58e00bbc60 tx=0x7f58e0005b50 comp rx=0 tx=0).stop 2024-09-05T06:53:26.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- 172.21.15.86:0/2582763769 shutdown_connections 2024-09-05T06:53:26.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905c0780 0x55b2905d3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 --2- 172.21.15.86:0/2582763769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905b15d0 0x55b2905b93d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- 172.21.15.86:0/2582763769 >> 172.21.15.86:0/2582763769 conn(0x55b29057a310 msgr2=0x55b2905b8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- 172.21.15.86:0/2582763769 shutdown_connections 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- 172.21.15.86:0/2582763769 wait complete. 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 -- start start 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.463+0000 7f58ee04f740 1 Processor -- start 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ae4b0 con 0x55b2905c0780 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41addb0 con 0x55b2905b2b40 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ab010 con 0x55b2905b15d0 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ee04f740 10 client.0 _ll_register_callbacks cb 0x7f58dc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58e9a12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44112/0 (socket says 172.21.15.86:44112) 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58e9a12640 1 -- 172.21.15.86:0/3258258469 learned_addr learned my addr 172.21.15.86:0/3258258469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58e9211640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 -- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 msgr2=0x55b2905c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 -- 172.21.15.86:0/3258258469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 msgr2=0x55b2905c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58ea213640 1 -- 172.21.15.86:0/3258258469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2905c4640 con 0x55b2905c0780 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.464+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58e9211640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58ea213640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905c4100 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0274f0 tx=0x7f58dc0110f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20bfe871ab96bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:26.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc037070 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f58dc0119c0 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e00bbcd0 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58dc043050 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58ee04f740 10 client.18300 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b290636780 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e00bbcd0 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.465+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f58dc027520 con 0x55b2905c0780 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 10 client.18300 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:26.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 10 client.18300 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 20 client.18300 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b290636f20 0x55b290639340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 4 client.18300 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b29051ed70 con 0x55b290636f20 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b290636f20 0x55b290639340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58cd7fa640 10 client.18300 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58e9a12640 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b290636f20 0x55b290639340 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af9fc229848d662 server_cookie=5356ebb291bdd52e in_seq=0 out_seq=0 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58cd7fa640 1 -- 172.21.15.86:0/3258258469 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b29051ed70 con 0x55b290636f20 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58cd7fa640 10 client.18300 handle_command_reply: tid=0 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 1 client.18300 shutdown 2024-09-05T06:53:26.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 msgr2=0x55b2905c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.466+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905c4100 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0274f0 tx=0x7f58dc0110f0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 shutdown_connections 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55b290636f20 0x55b290639340 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2905c0780 0x55b2905c4100 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2905b2b40 0x55b2905c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58cd7fa640 0 client.18300 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 --2- 172.21.15.86:0/3258258469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b2905b15d0 0x55b2905c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 >> 172.21.15.86:0/3258258469 conn(0x55b29057a310 msgr2=0x55b2905b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 shutdown_connections 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 1 -- 172.21.15.86:0/3258258469 wait complete. 2024-09-05T06:53:26.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ee04f740 20 client.18300 trim_cache size 0 max 16384 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 msgr2=0x7f58d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 0x7f58d00a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f58dc005fd0 tx=0x7f58dc009500 comp rx=0 tx=0).stop 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 msgr2=0x7f58e4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.467+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006010 tx=0x7f58e0004610 comp rx=0 tx=0).stop 2024-09-05T06:53:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 shutdown_connections 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f58d00a2cf0 0x7f58d00a51b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58e4108290 0x7f58e419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4103740 0x7f58e4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 --2- 172.21.15.86:0/1416938298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4102d90 0x7f58e4198500 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 >> 172.21.15.86:0/1416938298 conn(0x7f58e40fc7d0 msgr2=0x7f58e4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 shutdown_connections 2024-09-05T06:53:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:26.468+0000 7f58ebc76640 1 -- 172.21.15.86:0/1416938298 wait complete. 2024-09-05T06:53:26.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 211692 1.60 MB/sec execute 526 sec latency 37.567 ms 2024-09-05T06:53:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:27 smithi115 ceph-mon[34791]: pgmap v380: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 899 KiB/s wr, 75 op/s 2024-09-05T06:53:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:27 smithi086 ceph-mon[30943]: pgmap v380: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 899 KiB/s wr, 75 op/s 2024-09-05T06:53:27.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212487 1.61 MB/sec execute 527 sec latency 13.558 ms 2024-09-05T06:53:27.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:27 smithi096 ceph-mon[34468]: pgmap v380: 129 pgs: 129 active+clean; 241 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 899 KiB/s wr, 75 op/s 2024-09-05T06:53:28.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212851 1.62 MB/sec execute 528 sec latency 63.590 ms 2024-09-05T06:53:28.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:53:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:29.178 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T06:53:29.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:53:29.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:29 smithi086 ceph-mon[30943]: pgmap v381: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-05T06:53:29.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:29 smithi115 ceph-mon[34791]: pgmap v381: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-05T06:53:29.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212892 1.61 MB/sec execute 529 sec latency 972.140 ms 2024-09-05T06:53:29.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:29 smithi096 ceph-mon[34468]: pgmap v381: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- 172.21.15.86:0/4211519945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 msgr2=0x7f84c010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 --2- 172.21.15.86:0/4211519945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c010da40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009900 tx=0x7f84b002a480 comp rx=0 tx=0).stop 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- 172.21.15.86:0/4211519945 shutdown_connections 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 --2- 172.21.15.86:0/4211519945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c010da40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 --2- 172.21.15.86:0/4211519945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0102570 0x7f84c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 --2- 172.21.15.86:0/4211519945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0100e20 0x7f84c0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- 172.21.15.86:0/4211519945 >> 172.21.15.86:0/4211519945 conn(0x7f84c00fc7d0 msgr2=0x7f84c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- 172.21.15.86:0/4211519945 shutdown_connections 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- 172.21.15.86:0/4211519945 wait complete. 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 Processor -- start 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.634+0000 7f84c5881640 1 -- start start 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 0x7f84c0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0102570 0x7f84c01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0110e30 con 0x7f84c0102ef0 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0110cb0 con 0x7f84c0100e20 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c5881640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0110fb0 con 0x7f84c0102570 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55568/0 (socket says 172.21.15.86:55568) 2024-09-05T06:53:29.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 -- 172.21.15.86:0/4091535643 learned_addr learned my addr 172.21.15.86:0/4091535643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84be7fc640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0102570 0x7f84c01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84beffd640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 0x7f84c0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 -- 172.21.15.86:0/4091535643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0102570 msgr2=0x7f84c01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0102570 0x7f84c01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 -- 172.21.15.86:0/4091535643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 msgr2=0x7f84c0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 0x7f84c0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 -- 172.21.15.86:0/4091535643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c019d320 con 0x7f84c0102ef0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84beffd640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 0x7f84c0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84bf7fe640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f84b0005ab0 tx=0x7f84b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.635+0000 7f84c487f640 1 -- 172.21.15.86:0/4091535643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b003f030 con 0x7f84c0102ef0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.636+0000 7f84c487f640 1 -- 172.21.15.86:0/4091535643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b0003cb0 con 0x7f84c0102ef0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.636+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c019d5b0 con 0x7f84c0102ef0 2024-09-05T06:53:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.636+0000 7f84c487f640 1 -- 172.21.15.86:0/4091535643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0035070 con 0x7f84c0102ef0 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.636+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a4ef0 con 0x7f84c0102ef0 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.638+0000 7f84c487f640 1 -- 172.21.15.86:0/4091535643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84b0042030 con 0x7f84c0102ef0 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.638+0000 7f84c7c5a740 1 Processor -- start 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.638+0000 7f84c7c5a740 1 -- start start 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b4b40 0x561fdb7b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fdb7192f0 con 0x561fdb7be370 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561fdb6c0cf0 con 0x561fdb7b4b40 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561fdb7152b0 con 0x561fdb7b35d0 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55574/0 (socket says 172.21.15.86:55574) 2024-09-05T06:53:29.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 -- 172.21.15.86:0/4091251226 learned_addr learned my addr 172.21.15.86:0/4091251226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c487f640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84beffd640 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b4b40 0x561fdb7b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 -- 172.21.15.86:0/4091251226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 msgr2=0x561fdb7b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 -- 172.21.15.86:0/4091251226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b4b40 msgr2=0x561fdb7b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b4b40 0x561fdb7b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 -- 172.21.15.86:0/4091251226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdb6a53a0 con 0x561fdb7be370 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84c487f640 1 -- 172.21.15.86:0/4091535643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f84b00efff0 con 0x7f84c0102ef0 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84bf7fe640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84be7fc640 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b6c0 tx=0x7f84a800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd028530ed08c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.639+0000 7f84b5ffb640 1 -- 172.21.15.86:0/4091251226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004220 con 0x561fdb7be370 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84b5ffb640 1 -- 172.21.15.86:0/4091251226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80043c0 con 0x561fdb7be370 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84b5ffb640 1 -- 172.21.15.86:0/4091251226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8010960 con 0x561fdb7be370 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 -- 172.21.15.86:0/4091251226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 msgr2=0x561fdb7c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b6c0 tx=0x7f84a800b650 comp rx=0 tx=0).stop 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84bf7fe640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84b0032a80 tx=0x7f84b002a480 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 -- 172.21.15.86:0/4091251226 shutdown_connections 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7c0760 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/4091251226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b4b40 0x561fdb7b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 -- 172.21.15.86:0/4091251226 >> 172.21.15.86:0/4091251226 conn(0x561fdb77c310 msgr2=0x561fdb7afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 -- 172.21.15.86:0/4091251226 shutdown_connections 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.640+0000 7f84c7c5a740 1 -- 172.21.15.86:0/4091251226 wait complete. 2024-09-05T06:53:29.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 -- start start 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 Processor -- start 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7be370 0x561fdb7bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fdb7c1070 con 0x561fdb7b4b40 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561fdb6d4060 con 0x561fdb7be370 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561fdb7c11f0 con 0x561fdb7b35d0 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84c7c5a740 10 client.0 _ll_register_callbacks cb 0x561fdb4278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55580/0 (socket says 172.21.15.86:55580) 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 -- 172.21.15.86:0/1006042126 learned_addr learned my addr 172.21.15.86:0/1006042126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84be7fc640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7be370 0x561fdb7bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84be7fc640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 -- 172.21.15.86:0/1006042126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 msgr2=0x561fdb7bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.641+0000 7f84beffd640 1 -- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7be370 msgr2=0x561fdb7bb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84beffd640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7be370 0x561fdb7bb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84beffd640 1 -- 172.21.15.86:0/1006042126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdb48a3c0 con 0x561fdb7b4b40 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84beffd640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009bb0 tx=0x7f84b8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4d53a7162a5995 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8012860 con 0x561fdb7b4b40 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f84b8012a00 con 0x561fdb7b4b40 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a80043c0 con 0x561fdb7b4b40 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f84b8024050 con 0x561fdb7b4b40 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 10 client.18312 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:29.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561fdb7ccb70 con 0x561fdb7b4b40 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a80043c0 con 0x561fdb7b4b40 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f84b801bcb0 con 0x561fdb7b4b40 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 10 client.18312 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 20 client.18312 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb7ce0d0 0x561fdb7d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 4 client.18312 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.642+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdb726860 con 0x561fdb7ce0d0 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.643+0000 7f84be7fc640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb7ce0d0 0x561fdb7d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.643+0000 7f84be7fc640 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb7ce0d0 0x561fdb7d04f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2925180be03fac6 server_cookie=84a97d781435004f in_seq=0 out_seq=0 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.643+0000 7f849effd640 10 client.18312 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f849effd640 1 -- 172.21.15.86:0/1006042126 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fdb726860 con 0x561fdb7ce0d0 2024-09-05T06:53:29.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f849effd640 10 client.18312 handle_command_reply: tid=0 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 client.18312 shutdown 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 msgr2=0x561fdb7bb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009bb0 tx=0x7f84b8004410 comp rx=0 tx=0).stop 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 shutdown_connections 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb7ce0d0 0x561fdb7d04f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7be370 0x561fdb7bb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7bb020 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/1006042126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b35d0 0x561fdb7bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f849effd640 0 client.18312 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 >> 172.21.15.86:0/1006042126 conn(0x561fdb77c310 msgr2=0x561fdb7b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 shutdown_connections 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 1 -- 172.21.15.86:0/1006042126 wait complete. 2024-09-05T06:53:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.644+0000 7f84c7c5a740 20 client.18312 trim_cache size 0 max 16384 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 Processor -- start 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 -- start start 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7baff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7be370 0x561fdb7d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01af060 con 0x561fdb7b4b40 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0c00 con 0x561fdb7b35d0 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01af730 con 0x561fdb7be370 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55586/0 (socket says 172.21.15.86:55586) 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 -- 172.21.15.86:0/481840101 learned_addr learned my addr 172.21.15.86:0/481840101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84be7fc640 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7baff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 -- 172.21.15.86:0/481840101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7be370 msgr2=0x561fdb7d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7be370 0x561fdb7d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 -- 172.21.15.86:0/481840101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 msgr2=0x561fdb7baff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7baff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 -- 172.21.15.86:0/481840101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdb7b0960 con 0x561fdb7b4b40 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.657+0000 7f84beffd640 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f84b80116c0 tx=0x7f84b8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f5cc007875668 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84b57fa640 1 -- 172.21.15.86:0/481840101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8005b50 con 0x561fdb7b4b40 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84b57fa640 1 -- 172.21.15.86:0/481840101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b8016070 con 0x561fdb7b4b40 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84b57fa640 1 -- 172.21.15.86:0/481840101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8012660 con 0x561fdb7b4b40 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 -- 172.21.15.86:0/481840101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 msgr2=0x561fdb7ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f84b80116c0 tx=0x7f84b8003f40 comp rx=0 tx=0).stop 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 -- 172.21.15.86:0/481840101 shutdown_connections 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7be370 0x561fdb7d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7b4b40 0x561fdb7ca310 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/481840101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7baff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 -- 172.21.15.86:0/481840101 >> 172.21.15.86:0/481840101 conn(0x561fdb77c310 msgr2=0x561fdb7ba8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 -- 172.21.15.86:0/481840101 shutdown_connections 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.658+0000 7f84c7c5a740 1 -- 172.21.15.86:0/481840101 wait complete. 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 -- start start 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 Processor -- start 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b4b40 0x561fdb7b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0500 con 0x561fdb7be370 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01afe00 con 0x561fdb7b35d0 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ad060 con 0x561fdb7b4b40 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84c7c5a740 10 client.0 _ll_register_callbacks cb 0x7f84b8003a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84be7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55596/0 (socket says 172.21.15.86:55596) 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84be7fc640 1 -- 172.21.15.86:0/2162488193 learned_addr learned my addr 172.21.15.86:0/2162488193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.659+0000 7f84beffd640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b4b40 0x561fdb7b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 -- 172.21.15.86:0/2162488193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b4b40 msgr2=0x561fdb7b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b4b40 0x561fdb7b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 -- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 msgr2=0x561fdb7b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 -- 172.21.15.86:0/2162488193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdb7ca850 con 0x561fdb7be370 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84be7fc640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f84a80057c0 tx=0x7f84a802f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7f4e5c95e7b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8022040 con 0x561fdb7be370 2024-09-05T06:53:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f84a8005230 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8016070 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f84a8005430 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84c7c5a740 10 client.18318 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561fdb7b6120 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b8016070 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.660+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f84a801c070 con 0x561fdb7be370 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 10 client.18318 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 20 client.18318 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb839190 0x561fdb83b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 4 client.18318 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fdb720d70 con 0x561fdb839190 2024-09-05T06:53:29.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84beffd640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb839190 0x561fdb83b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f849effd640 10 client.18318 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84beffd640 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb839190 0x561fdb83b580 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160d8851ccabb5fa server_cookie=a01612ef1214777c in_seq=0 out_seq=0 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f849effd640 1 -- 172.21.15.86:0/2162488193 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fdb720d70 con 0x561fdb839190 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f849effd640 10 client.18318 handle_command_reply: tid=0 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.661+0000 7f84c7c5a740 1 client.18318 shutdown 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 msgr2=0x561fdb7b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f84a80057c0 tx=0x7f84a802f040 comp rx=0 tx=0).stop 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 shutdown_connections 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561fdb839190 0x561fdb83b580 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fdb7be370 0x561fdb7b8b10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fdb7b4b40 0x561fdb7b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 --2- 172.21.15.86:0/2162488193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561fdb7b35d0 0x561fdb7b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 >> 172.21.15.86:0/2162488193 conn(0x561fdb77c310 msgr2=0x561fdb7ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f849effd640 0 client.18318 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:29.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 shutdown_connections 2024-09-05T06:53:29.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 1 -- 172.21.15.86:0/2162488193 wait complete. 2024-09-05T06:53:29.679 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.662+0000 7f84c7c5a740 20 client.18318 trim_cache size 0 max 16384 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 msgr2=0x7f84ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84b0032a80 tx=0x7f84b002a480 comp rx=0 tx=0).stop 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 msgr2=0x7f84c019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f84b0005ab0 tx=0x7f84b00042f0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 shutdown_connections 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c0102ef0 0x7f84c019cc40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84c0102570 0x7f84c01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 --2- 172.21.15.86:0/4091535643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c0100e20 0x7f84c0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:29.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 >> 172.21.15.86:0/4091535643 conn(0x7f84c00fc7d0 msgr2=0x7f84c006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:29.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 shutdown_connections 2024-09-05T06:53:29.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:29.663+0000 7f84c5881640 1 -- 172.21.15.86:0/4091535643 wait complete. 2024-09-05T06:53:30.032 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:30.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212892 1.61 MB/sec execute 530 sec latency 1972.223 ms 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 -- 172.21.15.86:0/2556122598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 msgr2=0x7fa7e010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/2556122598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e010e610 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802a480 comp rx=0 tx=0).stop 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 -- 172.21.15.86:0/2556122598 shutdown_connections 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/2556122598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e010e610 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/2556122598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0103660 0x7fa7e010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/2556122598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0102d40 0x7fa7e0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 -- 172.21.15.86:0/2556122598 >> 172.21.15.86:0/2556122598 conn(0x7fa7e00fc7a0 msgr2=0x7fa7e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 -- 172.21.15.86:0/2556122598 shutdown_connections 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.142+0000 7fa7e60f0640 1 -- 172.21.15.86:0/2556122598 wait complete. 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 Processor -- start 2024-09-05T06:53:31.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 -- start start 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0102d40 0x7fa7e01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0103660 0x7fa7e0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0111de0 con 0x7fa7e010c170 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0111c60 con 0x7fa7e0103660 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.143+0000 7fa7e60f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0111f60 con 0x7fa7e0102d40 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55622/0 (socket says 172.21.15.86:55622) 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 -- 172.21.15.86:0/68221747 learned_addr learned my addr 172.21.15.86:0/68221747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7deffd640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0103660 0x7fa7e0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7df7fe640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0102d40 0x7fa7e01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:31.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 -- 172.21.15.86:0/68221747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0102d40 msgr2=0x7fa7e01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0102d40 0x7fa7e01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 -- 172.21.15.86:0/68221747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0103660 msgr2=0x7fa7e0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0103660 0x7fa7e0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 -- 172.21.15.86:0/68221747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e019d2a0 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dffff640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009230 tx=0x7fa7d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d803f030 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d8003aa0 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e019d530 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8035070 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.144+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e01a4e70 con 0x7fa7e010c170 2024-09-05T06:53:31.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.145+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e0103120 con 0x7fa7e010c170 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.147+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa7d8042030 con 0x7fa7e010c170 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.147+0000 7fa7dcff9640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 0x7fa7c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.147+0000 7fa7df7fe640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 0x7fa7c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.147+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7fa7d80f0390 con 0x7fa7e010c170 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.148+0000 7fa7df7fe640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 0x7fa7c80a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007ec0 tx=0x7fa7d00075b0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:31.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.150+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7d80bc520 con 0x7fa7e010c170 2024-09-05T06:53:31.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.352+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7e01071e0 con 0x7fa7e010c170 2024-09-05T06:53:31.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.352+0000 7fa7dcff9640 1 -- 172.21.15.86:0/68221747 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7fa7d80c0820 con 0x7fa7e010c170 2024-09-05T06:53:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:53:31.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:53:31.353 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 msgr2=0x7fa7c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 0x7fa7c80a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007ec0 tx=0x7fa7d00075b0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 msgr2=0x7fa7e019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009230 tx=0x7fa7d8004270 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 shutdown_connections 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7c80a2cf0 0x7fa7c80a51b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e010c170 0x7fa7e019cbc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e0103660 0x7fa7e0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 --2- 172.21.15.86:0/68221747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e0102d40 0x7fa7e01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.354+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 >> 172.21.15.86:0/68221747 conn(0x7fa7e00fc7a0 msgr2=0x7fa7e00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.355+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 shutdown_connections 2024-09-05T06:53:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:31.355+0000 7fa7e60f0640 1 -- 172.21.15.86:0/68221747 wait complete. 2024-09-05T06:53:31.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:31 smithi115 ceph-mon[34791]: pgmap v382: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 744 KiB/s wr, 74 op/s 2024-09-05T06:53:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:31 smithi086 ceph-mon[30943]: pgmap v382: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 744 KiB/s wr, 74 op/s 2024-09-05T06:53:31.499 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:53:31.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:53:31.535 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212892 1.61 MB/sec execute 531 sec latency 2972.275 ms 2024-09-05T06:53:31.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:31 smithi096 ceph-mon[34468]: pgmap v382: 129 pgs: 129 active+clean; 243 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 744 KiB/s wr, 74 op/s 2024-09-05T06:53:32.080 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T06:53:32.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T06:53:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:32.314 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:32 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/68221747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:32.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:32 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/68221747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:32.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213116 1.61 MB/sec execute 532 sec latency 3178.300 ms 2024-09-05T06:53:32.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:32 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/68221747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 -- 172.21.15.86:0/1600159654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 msgr2=0x7f15f81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 --2- 172.21.15.86:0/1600159654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f81021f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009900 tx=0x7f15f402a410 comp rx=0 tx=0).stop 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 -- 172.21.15.86:0/1600159654 shutdown_connections 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 --2- 172.21.15.86:0/1600159654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 0x7f15f810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 --2- 172.21.15.86:0/1600159654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 0x7f15f8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 --2- 172.21.15.86:0/1600159654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f81021f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 -- 172.21.15.86:0/1600159654 >> 172.21.15.86:0/1600159654 conn(0x7f15f80fd8d0 msgr2=0x7f15f80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 -- 172.21.15.86:0/1600159654 shutdown_connections 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.673+0000 7f1600631640 1 -- 172.21.15.86:0/1600159654 wait complete. 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 Processor -- start 2024-09-05T06:53:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 -- start start 2024-09-05T06:53:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 0x7f15f81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 0x7f15f819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111e50 con 0x7f15f8101e10 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111cd0 con 0x7f15f8106f40 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f1600631640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111fd0 con 0x7f15f81027c0 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52238/0 (socket says 172.21.15.86:52238) 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f15fe3cd640 1 -- 172.21.15.86:0/742360150 learned_addr learned my addr 172.21.15.86:0/742360150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 0x7f15f81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.674+0000 7f15febce640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 0x7f15f819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 -- 172.21.15.86:0/742360150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 msgr2=0x7f15f81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 0x7f15f81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 -- 172.21.15.86:0/742360150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 msgr2=0x7f15f819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 0x7f15f819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 -- 172.21.15.86:0/742360150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f819d340 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f15f4006010 tx=0x7f15f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15ef7fe640 1 -- 172.21.15.86:0/742360150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f403f030 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15ef7fe640 1 -- 172.21.15.86:0/742360150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15f40035f0 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f819d5d0 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f15ef7fe640 1 -- 172.21.15.86:0/742360150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4035070 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.675+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f81a4f10 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.677+0000 7f15ef7fe640 1 -- 172.21.15.86:0/742360150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f15f4003b10 con 0x7f15f8101e10 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 Processor -- start 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 -- start start 2024-09-05T06:53:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2d8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f15ef7fe640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f2e23e280 con 0x564f2e2e3d40 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f2e1e5b10 con 0x564f2e2d8b60 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.678+0000 7f1602a0a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f2e23a240 con 0x564f2e2d9480 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48450/0 (socket says 172.21.15.86:48450) 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15febce640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4257891045 learned_addr learned my addr 172.21.15.86:0/4257891045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15ef7fe640 1 -- 172.21.15.86:0/742360150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f15f40efdb0 con 0x7f15f8101e10 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15febce640 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2d8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4257891045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 msgr2=0x564f2e2d8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2d8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4257891045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 msgr2=0x564f2e2e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4257891045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2e1f21f0 con 0x564f2e2d9480 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15febce640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 0x7f15e40a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x564f2e2d6470 tx=0x7f15f0005bd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15e800abf0 tx=0x7f15e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74cfa95207ada7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15ecff9640 1 -- 172.21.15.86:0/4257891045 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801d070 con 0x564f2e2d9480 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15ecff9640 1 -- 172.21.15.86:0/4257891045 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15e8018af0 con 0x564f2e2d9480 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.679+0000 7f15ecff9640 1 -- 172.21.15.86:0/4257891045 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8010a30 con 0x564f2e2d9480 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 -- 172.21.15.86:0/4257891045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 msgr2=0x564f2e2e3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15e800abf0 tx=0x7f15e800f790 comp rx=0 tx=0).stop 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 -- 172.21.15.86:0/4257891045 shutdown_connections 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e3800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4257891045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2d8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 -- 172.21.15.86:0/4257891045 >> 172.21.15.86:0/4257891045 conn(0x564f2e2d50f0 msgr2=0x564f2e2d54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 -- 172.21.15.86:0/4257891045 shutdown_connections 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.680+0000 7f1602a0a740 1 -- 172.21.15.86:0/4257891045 wait complete. 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 -- start start 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 Processor -- start 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d9480 0x564f2e2e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f2e2e6910 con 0x564f2e2e3d40 2024-09-05T06:53:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f2e1f8f10 con 0x564f2e2d9480 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f2e2e6a90 con 0x564f2e2d8b60 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f1602a0a740 10 client.0 _ll_register_callbacks cb 0x564f2e1c1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f15febce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48460/0 (socket says 172.21.15.86:48460) 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f15fe3cd640 1 -- 172.21.15.86:0/4045277540 learned_addr learned my addr 172.21.15.86:0/4045277540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.681+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d9480 0x564f2e2e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 -- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d9480 msgr2=0x564f2e2e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d9480 0x564f2e2e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 -- 172.21.15.86:0/4045277540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 msgr2=0x564f2e2e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 -- 172.21.15.86:0/4045277540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2dfaf6a0 con 0x564f2e2d8b60 2024-09-05T06:53:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15febce640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15f40b8340 tx=0x7f15f40bb840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413be0abff483f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4049040 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f15f40ed2e0 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8018af0 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f15f40ed4e0 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f1602a0a740 10 client.27367 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f2e2e6970 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15e8018af0 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.682+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f15f40bc1d0 con 0x564f2e2d8b60 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 10 client.27367 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 10 client.27367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 20 client.27367 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:32.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e2f3990 0x564f2e2f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 4 client.27367 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f2e24b7f0 con 0x564f2e2f3990 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e2f3990 0x564f2e2f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f15e1ffb640 10 client.27367 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.683+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e2f3990 0x564f2e2f5d80 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fced6d7c9e63a8 server_cookie=6c1c99eae3097612 in_seq=0 out_seq=0 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4045277540 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f2e24b7f0 con 0x564f2e2f3990 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f15e1ffb640 10 client.27367 handle_command_reply: tid=0 2024-09-05T06:53:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 client.27367 shutdown 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 msgr2=0x564f2e2e0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15f40b8340 tx=0x7f15f40bb840 comp rx=0 tx=0).stop 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 shutdown_connections 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e2f3990 0x564f2e2f5d80 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d9480 0x564f2e2e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f15e1ffb640 0 client.27367 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4045277540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d8b60 0x564f2e2e0c00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 >> 172.21.15.86:0/4045277540 conn(0x564f2e2d50f0 msgr2=0x564f2e2d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 shutdown_connections 2024-09-05T06:53:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 1 -- 172.21.15.86:0/4045277540 wait complete. 2024-09-05T06:53:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.684+0000 7f1602a0a740 20 client.27367 trim_cache size 0 max 16384 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 Processor -- start 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 -- start start 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81af1a0 con 0x564f2e2e3d40 2024-09-05T06:53:32.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81b0d40 con 0x564f2e2d8b60 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f1602a0a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81af870 con 0x564f2e2d9480 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f15fdbcc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f15febce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f15fdbcc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48468/0 (socket says 172.21.15.86:48468) 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4155786382 learned_addr learned my addr 172.21.15.86:0/4155786382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.705+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4155786382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 msgr2=0x564f2e2e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4155786382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 msgr2=0x564f2e2fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4155786382 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2e2d92b0 con 0x564f2e2d9480 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15e800abc0 tx=0x7f15e800a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3d7a95bd81fc1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15e3fff640 1 -- 172.21.15.86:0/4155786382 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e801d040 con 0x564f2e2d9480 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15e3fff640 1 -- 172.21.15.86:0/4155786382 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15e8028070 con 0x564f2e2d9480 2024-09-05T06:53:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.706+0000 7f15e3fff640 1 -- 172.21.15.86:0/4155786382 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80186a0 con 0x564f2e2d9480 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- 172.21.15.86:0/4155786382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 msgr2=0x564f2e2e14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15e800abc0 tx=0x7f15e800a7f0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- 172.21.15.86:0/4155786382 shutdown_connections 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2e3d40 0x564f2e2fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e14d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4155786382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2d8b60 0x564f2e2e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- 172.21.15.86:0/4155786382 >> 172.21.15.86:0/4155786382 conn(0x564f2e2d50f0 msgr2=0x564f2e2e2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- 172.21.15.86:0/4155786382 shutdown_connections 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- 172.21.15.86:0/4155786382 wait complete. 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 -- start start 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.707+0000 7f1602a0a740 1 Processor -- start 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2e3d40 0x564f2e2e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81b0640 con 0x564f2e2d8b60 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81aff40 con 0x564f2e2e3d40 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81ad1a0 con 0x564f2e2d9480 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f1602a0a740 10 client.0 _ll_register_callbacks cb 0x7f15f40bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fe3cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52286/0 (socket says 172.21.15.86:52286) 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fe3cd640 1 -- 172.21.15.86:0/4033028982 learned_addr learned my addr 172.21.15.86:0/4033028982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15febce640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2e3d40 0x564f2e2e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2e3d40 msgr2=0x564f2e2e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2e3d40 0x564f2e2e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4033028982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 msgr2=0x564f2e2ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 -- 172.21.15.86:0/4033028982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2e2e8450 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15fdbcc640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e76e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f15e80058c0 tx=0x7f15e800a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdce74fb83a425d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.708+0000 7f15e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8032070 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f15e8005040 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8028070 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f15e803b050 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 10 client.27373 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f2e35e570 con 0x564f2e2d9480 2024-09-05T06:53:32.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15e8028070 con 0x564f2e2d9480 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f15e8035020 con 0x564f2e2d9480 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 10 client.27373 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 10 client.27373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 20 client.27373 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e35ea60 0x564f2e360e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 4 client.27373 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f2e245d00 con 0x564f2e35ea60 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.709+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e35ea60 0x564f2e360e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.710+0000 7f15fe3cd640 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e35ea60 0x564f2e360e80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ba8d362a2fc97f server_cookie=462122a3ba4efc4b in_seq=0 out_seq=0 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.710+0000 7f15e1ffb640 10 client.27373 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f15e1ffb640 1 -- 172.21.15.86:0/4033028982 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f2e245d00 con 0x564f2e35ea60 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f15e1ffb640 10 client.27373 handle_command_reply: tid=0 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 client.27373 shutdown 2024-09-05T06:53:32.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 msgr2=0x564f2e2e76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e76e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f15e80058c0 tx=0x7f15e800a310 comp rx=0 tx=0).stop 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 shutdown_connections 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564f2e35ea60 0x564f2e360e80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f2e2e3d40 0x564f2e2e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f2e2d9480 0x564f2e2e76e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f15e1ffb640 0 client.27373 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 --2- 172.21.15.86:0/4033028982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e2d8b60 0x564f2e2ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 >> 172.21.15.86:0/4033028982 conn(0x564f2e2d50f0 msgr2=0x564f2e2da5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 shutdown_connections 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 1 -- 172.21.15.86:0/4033028982 wait complete. 2024-09-05T06:53:32.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.711+0000 7f1602a0a740 20 client.27373 trim_cache size 0 max 16384 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2d0a65a3-cbf1-4978-b517-8380274f397d", 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:32.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 msgr2=0x7f15e40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 0x7f15e40a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x564f2e2d6470 tx=0x7f15f0005bd0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 msgr2=0x7f15f8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f15f4006010 tx=0x7f15f4004270 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 shutdown_connections 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f15e40a2d70 0x7f15e40a5230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15f8106f40 0x7f15f819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15f81027c0 0x7f15f81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 --2- 172.21.15.86:0/742360150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101e10 0x7f15f8198390 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 >> 172.21.15.86:0/742360150 conn(0x7f15f80fd8d0 msgr2=0x7f15f80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 shutdown_connections 2024-09-05T06:53:32.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:32.712+0000 7f1600631640 1 -- 172.21.15.86:0/742360150 wait complete. 2024-09-05T06:53:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:33 smithi115 ceph-mon[34791]: pgmap v383: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-05T06:53:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:33 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:33 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:33 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:33 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:33 smithi086 ceph-mon[30943]: pgmap v383: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-05T06:53:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:33 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:33 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:33 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:33 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213395 1.60 MB/sec execute 533 sec latency 37.864 ms 2024-09-05T06:53:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:33 smithi096 ceph-mon[34468]: pgmap v383: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-05T06:53:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:33 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:33 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:33 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:33 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:33.588 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:33.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:53:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 -- 172.21.15.86:0/3471746810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301082a0 msgr2=0x7f383010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 --2- 172.21.15.86:0/3471746810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301082a0 0x7f383010ec40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3828009cd0 tx=0x7f382802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 -- 172.21.15.86:0/3471746810 shutdown_connections 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 --2- 172.21.15.86:0/3471746810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301082a0 0x7f383010ec40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 --2- 172.21.15.86:0/3471746810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38301036e0 0x7f3830107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 --2- 172.21.15.86:0/3471746810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830102d30 0x7f3830103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.112+0000 7f383781e640 1 -- 172.21.15.86:0/3471746810 >> 172.21.15.86:0/3471746810 conn(0x7f38300fc7d0 msgr2=0x7f38300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 -- 172.21.15.86:0/3471746810 shutdown_connections 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 -- 172.21.15.86:0/3471746810 wait complete. 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 Processor -- start 2024-09-05T06:53:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 -- start start 2024-09-05T06:53:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 0x7f3830100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.113+0000 7f383781e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 0x7f3830101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f383781e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3830111f50 con 0x7f38301036e0 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f383781e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3830111dd0 con 0x7f38301082a0 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f383781e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38301120d0 con 0x7f3830102d30 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52304/0 (socket says 172.21.15.86:52304) 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 -- 172.21.15.86:0/1418180057 learned_addr learned my addr 172.21.15.86:0/1418180057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f38355ba640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 0x7f3830100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 0x7f3830101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 -- 172.21.15.86:0/1418180057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 msgr2=0x7f3830100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 0x7f3830100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 -- 172.21.15.86:0/1418180057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 msgr2=0x7f3830101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 0x7f3830101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 -- 172.21.15.86:0/1418180057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38301a6a10 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 0x7f3830101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f38355ba640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 0x7f3830100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f3834db9640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f382000d740 tx=0x7f382000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f38267fc640 1 -- 172.21.15.86:0/1418180057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820017070 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.114+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38301a6d00 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.115+0000 7f38267fc640 1 -- 172.21.15.86:0/1418180057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f382000de50 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.115+0000 7f38267fc640 1 -- 172.21.15.86:0/1418180057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382000f620 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.115+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301a7230 con 0x7f38301036e0 2024-09-05T06:53:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.117+0000 7f38267fc640 1 -- 172.21.15.86:0/1418180057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f382001a070 con 0x7f38301036e0 2024-09-05T06:53:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 Processor -- start 2024-09-05T06:53:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 -- start start 2024-09-05T06:53:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 0x55e759a31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 0x55e759a3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7599972a0 con 0x55e759a31760 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e75993eb10 con 0x55e759a3c940 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3839bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e759993260 con 0x55e759a32080 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f38355ba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f38267fc640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 0x7f381c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f38355ba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48498/0 (socket says 172.21.15.86:48498) 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f3835dbb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 0x55e759a31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.118+0000 7f38355ba640 1 -- 172.21.15.86:0/1706582384 learned_addr learned my addr 172.21.15.86:0/1706582384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f3834db9640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 0x55e759a3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 0x7f381c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 -- 172.21.15.86:0/1706582384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 msgr2=0x55e759a3ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 0x55e759a3ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 -- 172.21.15.86:0/1706582384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 msgr2=0x55e759a31b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 0x55e759a31b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 -- 172.21.15.86:0/1706582384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e759708b90 con 0x55e759a32080 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 0x55e759a31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f3834db9640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 0x55e759a3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38267fc640 1 -- 172.21.15.86:0/1418180057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f38200cf1f0 con 0x7f38301036e0 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.119+0000 7f38355ba640 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f382c0115f0 tx=0x7f382c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1819e8ca3a2635 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.120+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 0x7f381c0a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f382802a8e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.120+0000 7f381b7fe640 1 -- 172.21.15.86:0/1706582384 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c01e070 con 0x55e759a32080 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.120+0000 7f381b7fe640 1 -- 172.21.15.86:0/1706582384 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f382c00faf0 con 0x55e759a32080 2024-09-05T06:53:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.120+0000 7f381b7fe640 1 -- 172.21.15.86:0/1706582384 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c019bc0 con 0x55e759a32080 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- 172.21.15.86:0/1706582384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 msgr2=0x55e759a3c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f382c0115f0 tx=0x7f382c011ab0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- 172.21.15.86:0/1706582384 shutdown_connections 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a3c940 0x55e759a3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a3c400 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1706582384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a31760 0x55e759a31b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- 172.21.15.86:0/1706582384 >> 172.21.15.86:0/1706582384 conn(0x55e759a2dcf0 msgr2=0x55e759a2e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- 172.21.15.86:0/1706582384 shutdown_connections 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- 172.21.15.86:0/1706582384 wait complete. 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.121+0000 7f3839bf7740 1 -- start start 2024-09-05T06:53:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 Processor -- start 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e759a3f510 con 0x55e759a32080 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e75991f6e0 con 0x55e759a31760 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e759a3f690 con 0x55e759a3c940 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3839bf7740 10 client.0 _ll_register_callbacks cb 0x55e759943a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3835dbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3835dbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48512/0 (socket says 172.21.15.86:48512) 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3835dbb640 1 -- 172.21.15.86:0/1151509529 learned_addr learned my addr 172.21.15.86:0/1151509529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f38355ba640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.122+0000 7f3834db9640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a39d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 -- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 msgr2=0x55e759a397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 -- 172.21.15.86:0/1151509529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 msgr2=0x55e759a39d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a39d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 -- 172.21.15.86:0/1151509529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e759a489d0 con 0x55e759a3c940 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3835dbb640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3828035000 tx=0x7f3828004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02ac8aad1104453 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3818ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280307a0 con 0x55e759a3c940 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c00faf0 con 0x55e759a3c940 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f3828030940 con 0x55e759a3c940 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3828003e20 con 0x55e759a3c940 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3839bf7740 10 client.27385 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.123+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e759a4b390 con 0x55e759a3c940 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f382c00faf0 con 0x55e759a3c940 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f3828047070 con 0x55e759a3c940 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 10 client.27385 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 10 client.27385 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 20 client.27385 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759a4c7f0 0x55e759a4ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 4 client.27385 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.124+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7599a49d0 con 0x55e759a4c7f0 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.125+0000 7f38355ba640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759a4c7f0 0x55e759a4ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.125+0000 7f3818ff9640 10 client.27385 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.125+0000 7f38355ba640 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759a4c7f0 0x55e759a4ec60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcda9d9ef611d12 server_cookie=bea98c535fd41b66 in_seq=0 out_seq=0 2024-09-05T06:53:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3818ff9640 1 -- 172.21.15.86:0/1151509529 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7599a49d0 con 0x55e759a4c7f0 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3818ff9640 10 client.27385 handle_command_reply: tid=0 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3839bf7740 1 client.27385 shutdown 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 msgr2=0x55e759a3a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3828035000 tx=0x7f3828004a60 comp rx=0 tx=0).stop 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 shutdown_connections 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.126+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759a4c7f0 0x55e759a4ec60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a3a260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a39d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 --2- 172.21.15.86:0/1151509529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 >> 172.21.15.86:0/1151509529 conn(0x55e759a2dcf0 msgr2=0x55e759a2ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3818ff9640 0 client.27385 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 shutdown_connections 2024-09-05T06:53:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 1 -- 172.21.15.86:0/1151509529 wait complete. 2024-09-05T06:53:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.127+0000 7f3839bf7740 20 client.27385 trim_cache size 0 max 16384 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.146+0000 7f3839bf7740 1 Processor -- start 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.146+0000 7f3839bf7740 1 -- start start 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.146+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.146+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:40762/0 (socket says 172.21.15.86:40762) 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3834db9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3834db9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52348/0 (socket says 172.21.15.86:52348) 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a54a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3834db9640 1 -- 172.21.15.86:0/3054827135 learned_addr learned my addr 172.21.15.86:0/3054827135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b1490 con 0x55e759a32080 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b3030 con 0x55e759a31760 2024-09-05T06:53:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b1b60 con 0x55e759a3c940 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3835dbb640 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a54a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 -- 172.21.15.86:0/3054827135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 msgr2=0x55e759a54a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a54a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 -- 172.21.15.86:0/3054827135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 msgr2=0x55e759a54530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 -- 172.21.15.86:0/3054827135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e759a29260 con 0x55e759a31760 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f3834db9640 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.147+0000 7f38355ba640 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f382c01c040 tx=0x7f382c0197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8f9588d1086da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.148+0000 7f381affd640 1 -- 172.21.15.86:0/3054827135 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c01e040 con 0x55e759a31760 2024-09-05T06:53:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.148+0000 7f381affd640 1 -- 172.21.15.86:0/3054827135 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f382c029070 con 0x55e759a31760 2024-09-05T06:53:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.148+0000 7f381affd640 1 -- 172.21.15.86:0/3054827135 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c005470 con 0x55e759a31760 2024-09-05T06:53:34.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 msgr2=0x55e759a380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f382c01c040 tx=0x7f382c0197a0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 shutdown_connections 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a3c940 0x55e759a54a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a32080 0x55e759a54530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 --2- 172.21.15.86:0/3054827135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a380b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 >> 172.21.15.86:0/3054827135 conn(0x55e759a2dcf0 msgr2=0x55e759a3b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 shutdown_connections 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.149+0000 7f3839bf7740 1 -- 172.21.15.86:0/3054827135 wait complete. 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.150+0000 7f3839bf7740 1 -- start start 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.150+0000 7f3839bf7740 1 Processor -- start 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.150+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.150+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.150+0000 7f3839bf7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a3c940 0x55e759a41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:40772/0 (socket says 172.21.15.86:40772) 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 -- 172.21.15.86:0/649759866 learned_addr learned my addr 172.21.15.86:0/649759866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3834db9640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b2930 con 0x55e759a3c940 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b2230 con 0x55e759a31760 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38301af490 con 0x55e759a32080 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3835dbb640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a3c940 0x55e759a41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f3839bf7740 10 client.0 _ll_register_callbacks cb 0x7f3828003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 -- 172.21.15.86:0/649759866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 msgr2=0x55e759a40dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a40dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 -- 172.21.15.86:0/649759866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a3c940 msgr2=0x55e759a41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a3c940 0x55e759a41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.151+0000 7f38355ba640 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e759ab6700 con 0x55e759a31760 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f38355ba640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382c011af0 tx=0x7f382c00f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b27f2434aaefde server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c01e070 con 0x55e759a31760 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c029070 con 0x55e759a31760 2024-09-05T06:53:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f382c00fd20 con 0x55e759a31760 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f382c00f640 con 0x55e759a31760 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 10 client.27416 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e759ab7980 con 0x55e759a31760 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f382c029070 con 0x55e759a31760 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f382c0199a0 con 0x55e759a31760 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 10 client.27416 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 10 client.27416 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 20 client.27416 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759ab7f70 0x55e759aba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 4 client.27416 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T06:53:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.152+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55e75999ed20 con 0x55e759ab7f70 2024-09-05T06:53:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.153+0000 7f3834db9640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759ab7f70 0x55e759aba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.153+0000 7f3818ff9640 10 client.27416 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.153+0000 7f3834db9640 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759ab7f70 0x55e759aba360 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a5be451c4a1e14 server_cookie=3aa0014a52ea3e78 in_seq=0 out_seq=0 2024-09-05T06:53:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3818ff9640 1 -- 172.21.15.86:0/649759866 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e75999ed20 con 0x55e759ab7f70 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3818ff9640 10 client.27416 handle_command_reply: tid=0 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 client.27416 shutdown 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 msgr2=0x55e759a40890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382c011af0 tx=0x7f382c00f450 comp rx=0 tx=0).stop 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 shutdown_connections 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e759ab7f70 0x55e759aba360 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e759a3c940 0x55e759a41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e759a32080 0x55e759a40dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3818ff9640 0 client.27416 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 --2- 172.21.15.86:0/649759866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e759a31760 0x55e759a40890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 >> 172.21.15.86:0/649759866 conn(0x55e759a2dcf0 msgr2=0x55e759a39780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 shutdown_connections 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 1 -- 172.21.15.86:0/649759866 wait complete. 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.208+0000 7f3839bf7740 20 client.27416 trim_cache size 0 max 16384 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 36, 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 42, 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "1dbea106": { 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 42, 2024-09-05T06:53:34.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 msgr2=0x7f381c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 0x7f381c0a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f382802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 msgr2=0x7f3830100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f382000d740 tx=0x7f382000dc00 comp rx=0 tx=0).stop 2024-09-05T06:53:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 shutdown_connections 2024-09-05T06:53:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2cf0 0x7f381c0a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38301082a0 0x7f3830101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38301036e0 0x7f3830100ed0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 --2- 172.21.15.86:0/1418180057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830102d30 0x7f3830100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 >> 172.21.15.86:0/1418180057 conn(0x7f38300fc7d0 msgr2=0x7f3830106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 shutdown_connections 2024-09-05T06:53:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.209+0000 7f383781e640 1 -- 172.21.15.86:0/1418180057 wait complete. 2024-09-05T06:53:34.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:53:34.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.401 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:34 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:34 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.60 MB/sec execute 534 sec latency 326.281 ms 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:34 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 -- 172.21.15.86:0/2880227700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 msgr2=0x7f39481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 --2- 172.21.15.86:0/2880227700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f39481067f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f393c009900 tx=0x7f393c02a410 comp rx=0 tx=0).stop 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 -- 172.21.15.86:0/2880227700 shutdown_connections 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 --2- 172.21.15.86:0/2880227700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948106f20 0x7f394810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 --2- 172.21.15.86:0/2880227700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f39481067f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 --2- 172.21.15.86:0/2880227700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948101df0 0x7f39481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 -- 172.21.15.86:0/2880227700 >> 172.21.15.86:0/2880227700 conn(0x7f39480fd8b0 msgr2=0x7f39480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 -- 172.21.15.86:0/2880227700 shutdown_connections 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.831+0000 7f394cbed640 1 -- 172.21.15.86:0/2880227700 wait complete. 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.832+0000 7f394cbed640 1 Processor -- start 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.832+0000 7f394cbed640 1 -- start start 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 0x7f3948198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 0x7f394819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111df0 con 0x7f39481027a0 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111c70 con 0x7f3948106f20 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111f70 con 0x7f3948101df0 2024-09-05T06:53:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52368/0 (socket says 172.21.15.86:52368) 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 -- 172.21.15.86:0/1974567558 learned_addr learned my addr 172.21.15.86:0/1974567558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3946d9d640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 0x7f394819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394659c640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 0x7f3948198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 -- 172.21.15.86:0/1974567558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 msgr2=0x7f3948198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 0x7f3948198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 -- 172.21.15.86:0/1974567558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 msgr2=0x7f394819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 0x7f394819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 -- 172.21.15.86:0/1974567558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819d300 con 0x7f39481027a0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394659c640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 0x7f3948198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3946d9d640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 0x7f394819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f393c005940 tx=0x7f393c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f39377fe640 1 -- 172.21.15.86:0/1974567558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c03f030 con 0x7f39481027a0 2024-09-05T06:53:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f39377fe640 1 -- 172.21.15.86:0/1974567558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f393c003aa0 con 0x7f39481027a0 2024-09-05T06:53:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819d590 con 0x7f39481027a0 2024-09-05T06:53:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.833+0000 7f39377fe640 1 -- 172.21.15.86:0/1974567558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c035070 con 0x7f39481027a0 2024-09-05T06:53:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.834+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a4ed0 con 0x7f39481027a0 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f39377fe640 1 -- 172.21.15.86:0/1974567558 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f393c042030 con 0x7f39481027a0 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 Processor -- start 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 -- start start 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d542780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d542cc0 0x55ca7d551550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 0x55ca7d553e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d4a82e0 con 0x55ca7d551a90 2024-09-05T06:53:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d44fce0 con 0x55ca7d542cc0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.836+0000 7f394efc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d4a42a0 con 0x55ca7d5423a0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f39377fe640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 0x7f39300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 0x55ca7d553e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 0x55ca7d553e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52382/0 (socket says 172.21.15.86:52382) 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3945d9b640 1 -- 172.21.15.86:0/106213181 learned_addr learned my addr 172.21.15.86:0/106213181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d542780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f394659c640 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d542cc0 0x55ca7d551550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 0x7f39300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 -- 172.21.15.86:0/106213181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d542cc0 msgr2=0x55ca7d551550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d542cc0 0x55ca7d551550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 -- 172.21.15.86:0/106213181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 msgr2=0x55ca7d553e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f39377fe640 1 -- 172.21.15.86:0/1974567558 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f393c0effd0 con 0x7f39481027a0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 0x55ca7d553e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 -- 172.21.15.86:0/106213181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7d434440 con 0x55ca7d5423a0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d542780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3940005fd0 tx=0x7f3940010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1489acbe636ec9a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3946d9d640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 0x7f39300a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f394001b000 tx=0x7f394001d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3934ff9640 1 -- 172.21.15.86:0/106213181 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394000cc80 con 0x55ca7d5423a0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3934ff9640 1 -- 172.21.15.86:0/106213181 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3940022070 con 0x55ca7d5423a0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.837+0000 7f3934ff9640 1 -- 172.21.15.86:0/106213181 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39400056e0 con 0x55ca7d5423a0 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 -- 172.21.15.86:0/106213181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 msgr2=0x55ca7d542780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d542780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3940005fd0 tx=0x7f3940010740 comp rx=0 tx=0).stop 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 -- 172.21.15.86:0/106213181 shutdown_connections 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d551a90 0x55ca7d553e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d542cc0 0x55ca7d551550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 --2- 172.21.15.86:0/106213181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d542780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 -- 172.21.15.86:0/106213181 >> 172.21.15.86:0/106213181 conn(0x55ca7d53ea30 msgr2=0x55ca7d53ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 -- 172.21.15.86:0/106213181 shutdown_connections 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.838+0000 7f394efc6740 1 -- 172.21.15.86:0/106213181 wait complete. 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 -- start start 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 Processor -- start 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d551470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d54ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d5546c0 con 0x55ca7d542cc0 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d463050 con 0x55ca7d551a90 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ca7d554840 con 0x55ca7d5423a0 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394efc6740 10 client.0 _ll_register_callbacks cb 0x55ca7d4a3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52386/0 (socket says 172.21.15.86:52386) 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 -- 172.21.15.86:0/1201275246 learned_addr learned my addr 172.21.15.86:0/1201275246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3946d9d640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d54ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f394659c640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d551470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 -- 172.21.15.86:0/1201275246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 msgr2=0x55ca7d551470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d551470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 -- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 msgr2=0x55ca7d54ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d54ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.839+0000 7f3945d9b640 1 -- 172.21.15.86:0/1201275246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7d2193c0 con 0x55ca7d542cc0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f3945d9b640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f393c0bbe10 tx=0x7f393c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ab5048859c0b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c049070 con 0x55ca7d542cc0 2024-09-05T06:53:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f393c0c0110 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3940022070 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f393c0bc160 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 10 client.18372 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca7d560140 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3940022070 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f393c0ed560 con 0x55ca7d542cc0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 10 client.18372 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 20 client.18372 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d563970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 4 client.18372 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.840+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7d4b5850 con 0x55ca7d561550 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.841+0000 7f394659c640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d563970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.841+0000 7f394659c640 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d563970 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98af08a83409225e server_cookie=1bc473e40e178a0a in_seq=0 out_seq=0 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.841+0000 7f392dffb640 10 client.18372 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f392dffb640 1 -- 172.21.15.86:0/1201275246 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca7d4b5850 con 0x55ca7d561550 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f392dffb640 10 client.18372 handle_command_reply: tid=0 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 client.18372 shutdown 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 msgr2=0x55ca7d54e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f393c0bbe10 tx=0x7f393c009620 comp rx=0 tx=0).stop 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 shutdown_connections 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d563970 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d54ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f392dffb640 0 client.18372 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54e720 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 --2- 172.21.15.86:0/1201275246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d5423a0 0x55ca7d551470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 >> 172.21.15.86:0/1201275246 conn(0x55ca7d53ea30 msgr2=0x55ca7d544760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 shutdown_connections 2024-09-05T06:53:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 1 -- 172.21.15.86:0/1201275246 wait complete. 2024-09-05T06:53:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.842+0000 7f394efc6740 20 client.18372 trim_cache size 0 max 16384 2024-09-05T06:53:34.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 Processor -- start 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 -- start start 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 0x55ca7d55d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d551a90 0x55ca7d549d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39481acf70 con 0x55ca7d542cc0 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39481aeb10 con 0x55ca7d5423a0 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394efc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ad640 con 0x55ca7d551a90 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52394/0 (socket says 172.21.15.86:52394) 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 -- 172.21.15.86:0/3313873238 learned_addr learned my addr 172.21.15.86:0/3313873238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3946d9d640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d551a90 0x55ca7d549d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394659c640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 0x55ca7d55d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 -- 172.21.15.86:0/3313873238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d551a90 msgr2=0x55ca7d549d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d551a90 0x55ca7d549d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 -- 172.21.15.86:0/3313873238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 msgr2=0x55ca7d55d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 0x55ca7d55d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 -- 172.21.15.86:0/3313873238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7d5517e0 con 0x55ca7d542cc0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f394659c640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 0x55ca7d55d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f3945d9b640 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f393c0c47f0 tx=0x7f393c0ba670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200dccb953c07ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f392ffff640 1 -- 172.21.15.86:0/3313873238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c049070 con 0x55ca7d542cc0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.859+0000 7f392ffff640 1 -- 172.21.15.86:0/3313873238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f393c0c4d20 con 0x55ca7d542cc0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f392ffff640 1 -- 172.21.15.86:0/3313873238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0bc3c0 con 0x55ca7d542cc0 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- 172.21.15.86:0/3313873238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 msgr2=0x55ca7d54dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f393c0c47f0 tx=0x7f393c0ba670 comp rx=0 tx=0).stop 2024-09-05T06:53:34.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- 172.21.15.86:0/3313873238 shutdown_connections 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d551a90 0x55ca7d549d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d542cc0 0x55ca7d54dfe0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 --2- 172.21.15.86:0/3313873238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d5423a0 0x55ca7d55d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- 172.21.15.86:0/3313873238 >> 172.21.15.86:0/3313873238 conn(0x55ca7d53ea30 msgr2=0x55ca7d53ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- 172.21.15.86:0/3313873238 shutdown_connections 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- 172.21.15.86:0/3313873238 wait complete. 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.860+0000 7f394efc6740 1 -- start start 2024-09-05T06:53:34.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 Processor -- start 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d542cc0 0x55ca7d5553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d555930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ae410 con 0x55ca7d5423a0 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39481add10 con 0x55ca7d551a90 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39481aaf70 con 0x55ca7d542cc0 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394efc6740 10 client.0 _ll_register_callbacks cb 0x7f393c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52404/0 (socket says 172.21.15.86:52404) 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 -- 172.21.15.86:0/432239903 learned_addr learned my addr 172.21.15.86:0/432239903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f3945d9b640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d542cc0 0x55ca7d5553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f3946d9d640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d555930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 -- 172.21.15.86:0/432239903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d542cc0 msgr2=0x55ca7d5553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d542cc0 0x55ca7d5553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 -- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 msgr2=0x55ca7d555930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d555930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 -- 172.21.15.86:0/432239903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7d555fd0 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f3946d9d640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d555930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f394659c640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f39380118c0 tx=0x7f3938011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967b75882cd738dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.861+0000 7f392dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393802a060 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f3938010de0 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f393c0c4d20 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3938031050 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 10 client.18378 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca7d5cc0f0 con 0x55ca7d5423a0 2024-09-05T06:53:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f393c0c4d20 con 0x55ca7d5423a0 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f393801d580 con 0x55ca7d5423a0 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 10 client.18378 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 20 client.18378 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d5ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 4 client.18378 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.862+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca7d4afd60 con 0x55ca7d561550 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f3945d9b640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d5ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f392dffb640 10 client.18378 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f3945d9b640 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d5ce8e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767f118446b39f98 server_cookie=64b671f5790f4104 in_seq=0 out_seq=0 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f392dffb640 1 -- 172.21.15.86:0/432239903 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ca7d4afd60 con 0x55ca7d561550 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f392dffb640 10 client.18378 handle_command_reply: tid=0 2024-09-05T06:53:34.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 client.18378 shutdown 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 msgr2=0x55ca7d55a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f39380118c0 tx=0x7f3938011d80 comp rx=0 tx=0).stop 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 shutdown_connections 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ca7d561550 0x55ca7d5ce8e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ca7d551a90 0x55ca7d555930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f392dffb640 0 client.18378 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca7d542cc0 0x55ca7d5553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 --2- 172.21.15.86:0/432239903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7d5423a0 0x55ca7d55a0c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 >> 172.21.15.86:0/432239903 conn(0x55ca7d53ea30 msgr2=0x55ca7d54dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 shutdown_connections 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 1 -- 172.21.15.86:0/432239903 wait complete. 2024-09-05T06:53:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.863+0000 7f394efc6740 20 client.18378 trim_cache size 0 max 16384 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "2d0a65a3-cbf1-4978-b517-8380274f397d": { 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2d0a65a3-cbf1-4978-b517-8380274f397d", 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:34.870 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 msgr2=0x7f39300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 0x7f39300a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f394001b000 tx=0x7f394001d000 comp rx=0 tx=0).stop 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 msgr2=0x7f3948198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f393c005940 tx=0x7f393c004270 comp rx=0 tx=0).stop 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 shutdown_connections 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f39300a2cf0 0x7f39300a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3948106f20 0x7f394819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39481027a0 0x7f3948198890 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 --2- 172.21.15.86:0/1974567558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3948101df0 0x7f3948198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:34.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 >> 172.21.15.86:0/1974567558 conn(0x7f39480fd8b0 msgr2=0x7f39480fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:34.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 shutdown_connections 2024-09-05T06:53:34.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:34.864+0000 7f394cbed640 1 -- 172.21.15.86:0/1974567558 wait complete. 2024-09-05T06:53:35.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:35 smithi115 ceph-mon[34791]: pgmap v384: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:35 smithi086 ceph-mon[30943]: pgmap v384: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:35.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.60 MB/sec execute 535 sec latency 1326.349 ms 2024-09-05T06:53:35.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:35 smithi096 ceph-mon[34468]: pgmap v384: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:35.694 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1dbea106', quiesce took 6.5 seconds. Will release after: 3.5 2024-09-05T06:53:36.223 INFO:tasks.cephfs.filesystem:scrub status for tag:2d0a65a3-cbf1-4978-b517-8380274f397d - {'path': '/', 'tag': '2d0a65a3-cbf1-4978-b517-8380274f397d', 'options': 'recursive,force'} 2024-09-05T06:53:36.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.60 MB/sec execute 536 sec latency 2326.445 ms 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: pgmap v385: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:37 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: pgmap v385: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:37 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.59 MB/sec execute 537 sec latency 3326.521 ms 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: pgmap v385: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:37 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:38.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:53:38.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:53:38.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:53:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:53:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:38.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.59 MB/sec execute 538 sec latency 4326.563 ms 2024-09-05T06:53:38.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:53:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:53:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:53:39.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:53:39.456 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:39.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:39 smithi086 ceph-mon[30943]: pgmap v386: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-09-05T06:53:39.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.59 MB/sec execute 539 sec latency 5326.672 ms 2024-09-05T06:53:39.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:39 smithi096 ceph-mon[34468]: pgmap v386: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-09-05T06:53:39.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:39 smithi115 ceph-mon[34791]: pgmap v386: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-09-05T06:53:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 -- 172.21.15.86:0/1905197047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c100a80 msgr2=0x7fbe1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 --2- 172.21.15.86:0/1905197047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c100a80 0x7fbe1c100ee0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009930 tx=0x7fbe1002a420 comp rx=0 tx=0).stop 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 -- 172.21.15.86:0/1905197047 shutdown_connections 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 --2- 172.21.15.86:0/1905197047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c101420 0x7fbe1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 --2- 172.21.15.86:0/1905197047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c100a80 0x7fbe1c100ee0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 --2- 172.21.15.86:0/1905197047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c106a80 0x7fbe1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 -- 172.21.15.86:0/1905197047 >> 172.21.15.86:0/1905197047 conn(0x7fbe1c0fc7a0 msgr2=0x7fbe1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 -- 172.21.15.86:0/1905197047 shutdown_connections 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 -- 172.21.15.86:0/1905197047 wait complete. 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.940+0000 7fbe2273b640 1 Processor -- start 2024-09-05T06:53:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 -- start start 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 0x7fbe1c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c101420 0x7fbe1c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c110d30 con 0x7fbe1c106a80 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c110bb0 con 0x7fbe1c100a80 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe2273b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c110eb0 con 0x7fbe1c101420 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52418/0 (socket says 172.21.15.86:52418) 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 -- 172.21.15.86:0/3144998736 learned_addr learned my addr 172.21.15.86:0/3144998736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe1bfff640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 0x7fbe1c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe1b7fe640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c101420 0x7fbe1c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 -- 172.21.15.86:0/3144998736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c101420 msgr2=0x7fbe1c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c101420 0x7fbe1c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 -- 172.21.15.86:0/3144998736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 msgr2=0x7fbe1c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 0x7fbe1c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe20cd8640 1 -- 172.21.15.86:0/3144998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c19c200 con 0x7fbe1c106a80 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.941+0000 7fbe1bfff640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 0x7fbe1c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe20cd8640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400cce0 tx=0x7fbe1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400f450 con 0x7fbe1c106a80 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c19c4f0 con 0x7fbe1c106a80 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1400faa0 con 0x7fbe1c106a80 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14018b70 con 0x7fbe1c106a80 2024-09-05T06:53:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.942+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a3dd0 con 0x7fbe1c106a80 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.943+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c100ee0 con 0x7fbe1c106a80 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.944+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbe14018d10 con 0x7fbe1c106a80 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.945+0000 7fbe197fa640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 0x7fbe040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.945+0000 7fbe1bfff640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 0x7fbe040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.945+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7fbe140cfe50 con 0x7fbe1c106a80 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.945+0000 7fbe1bfff640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 0x7fbe040a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c005fd0 tx=0x7fbe0c005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:39.948+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe14014030 con 0x7fbe1c106a80 2024-09-05T06:53:40.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.147+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe1c1985b0 con 0x7fbe1c106a80 2024-09-05T06:53:40.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.147+0000 7fbe197fa640 1 -- 172.21.15.86:0/3144998736 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7fbe1c1985b0 con 0x7fbe1c106a80 2024-09-05T06:53:40.148 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:53:40.148 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:53:40.148 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.149+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 msgr2=0x7fbe040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.149+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 0x7fbe040a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c005fd0 tx=0x7fbe0c005850 comp rx=0 tx=0).stop 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.149+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 msgr2=0x7fbe1c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.149+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400cce0 tx=0x7fbe1400a620 comp rx=0 tx=0).stop 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 shutdown_connections 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe040a2cf0 0x7fbe040a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1c106a80 0x7fbe1c19bb20 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe1c101420 0x7fbe1c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 --2- 172.21.15.86:0/3144998736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c100a80 0x7fbe1c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:40.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 >> 172.21.15.86:0/3144998736 conn(0x7fbe1c0fc7a0 msgr2=0x7fbe1c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:40.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 shutdown_connections 2024-09-05T06:53:40.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:40.150+0000 7fbe2273b640 1 -- 172.21.15.86:0/3144998736 wait complete. 2024-09-05T06:53:40.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:40 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3144998736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:40.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.58 MB/sec execute 540 sec latency 6326.719 ms 2024-09-05T06:53:40.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:40 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3144998736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:40.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:40 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3144998736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:53:40.952 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 1dbea106 --release --await' 2024-09-05T06:53:40.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 1dbea106 --release --await 2024-09-05T06:53:41.205 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:53:41.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213686 1.58 MB/sec execute 541 sec latency 7326.767 ms 2024-09-05T06:53:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:41 smithi096 ceph-mon[34468]: pgmap v387: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 855 KiB/s wr, 75 op/s 2024-09-05T06:53:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:41 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:53:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:41 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:53:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:41 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:53:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:41 smithi086 ceph-mon[30943]: pgmap v387: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 855 KiB/s wr, 75 op/s 2024-09-05T06:53:41.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:41 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:53:41.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:41 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:53:41.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:41 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:53:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.656+0000 7f21262ca640 1 -- 172.21.15.86:0/2008951013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2120106f20 msgr2=0x7f212010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.656+0000 7f21262ca640 1 --2- 172.21.15.86:0/2008951013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2120106f20 0x7f212010eaa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2118009ca0 tx=0x7f211802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- 172.21.15.86:0/2008951013 shutdown_connections 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- 172.21.15.86:0/2008951013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2120106f20 0x7f212010eaa0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- 172.21.15.86:0/2008951013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21201027a0 0x7f21201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- 172.21.15.86:0/2008951013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120101df0 0x7f21201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- 172.21.15.86:0/2008951013 >> 172.21.15.86:0/2008951013 conn(0x7f21200fd8f0 msgr2=0x7f21200ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- 172.21.15.86:0/2008951013 shutdown_connections 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- 172.21.15.86:0/2008951013 wait complete. 2024-09-05T06:53:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 Processor -- start 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- start start 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2120101df0 0x7f21200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 0x7f2120074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2120111ed0 con 0x7f21201027a0 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2120111d50 con 0x7f2120101df0 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f21262ca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2120112050 con 0x7f2120106f20 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f2117fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.657+0000 7f2117fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52442/0 (socket says 172.21.15.86:52442) 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 -- 172.21.15.86:0/349322867 learned_addr learned my addr 172.21.15.86:0/349322867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2124867640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 0x7f2120074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f211ffff640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2120101df0 0x7f21200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 -- 172.21.15.86:0/349322867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 msgr2=0x7f2120074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 0x7f2120074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 -- 172.21.15.86:0/349322867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2120101df0 msgr2=0x7f21200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2120101df0 0x7f21200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 -- 172.21.15.86:0/349322867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21201a68e0 con 0x7f21201027a0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2124867640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 0x7f2120074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f2117fff640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f210800b4a0 tx=0x7f210800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f211dffb640 1 -- 172.21.15.86:0/349322867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108014070 con 0x7f21201027a0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f211dffb640 1 -- 172.21.15.86:0/349322867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2108010ac0 con 0x7f21201027a0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21201a6bd0 con 0x7f21201027a0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f211dffb640 1 -- 172.21.15.86:0/349322867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108017c30 con 0x7f21201027a0 2024-09-05T06:53:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.658+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21201a7110 con 0x7f21201027a0 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.660+0000 7f211dffb640 1 -- 172.21.15.86:0/349322867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2108017070 con 0x7f21201027a0 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 Processor -- start 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 -- start start 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e56d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 0x55717e57a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55717e4d2620 con 0x55717e56d980 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55717e479e50 con 0x55717e56d060 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f21286a3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55717e4ce5e0 con 0x55717e578240 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f211dffb640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 0x7f210c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f211ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f211ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52454/0 (socket says 172.21.15.86:52454) 2024-09-05T06:53:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f211ffff640 1 -- 172.21.15.86:0/1265636629 learned_addr learned my addr 172.21.15.86:0/1265636629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f2124867640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 0x7f210c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f2124867640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e56d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f211dffb640 1 -- 172.21.15.86:0/349322867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f21080cef00 con 0x7f21201027a0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.661+0000 7f2117fff640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 0x55717e57a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 -- 172.21.15.86:0/1265636629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 msgr2=0x55717e57a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 0x55717e57a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 -- 172.21.15.86:0/1265636629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 msgr2=0x55717e56d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e56d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 -- 172.21.15.86:0/1265636629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717e46ad40 con 0x55717e56d980 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f2117fff640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 0x55717e57a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f211ffff640 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f211000b820 tx=0x7f211000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25309a109f1f63d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f2124867640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 0x7f210c0a51b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2118009a60 tx=0x7f2118009800 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f21167fc640 1 -- 172.21.15.86:0/1265636629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000d950 con 0x55717e56d980 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f21167fc640 1 -- 172.21.15.86:0/1265636629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2110019070 con 0x55717e56d980 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.662+0000 7f21167fc640 1 -- 172.21.15.86:0/1265636629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100123e0 con 0x55717e56d980 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- 172.21.15.86:0/1265636629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 msgr2=0x55717e577d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f211000b820 tx=0x7f211000bce0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- 172.21.15.86:0/1265636629 shutdown_connections 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e578240 0x55717e57a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e577d00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 --2- 172.21.15.86:0/1265636629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e56d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- 172.21.15.86:0/1265636629 >> 172.21.15.86:0/1265636629 conn(0x55717e5695f0 msgr2=0x55717e5699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- 172.21.15.86:0/1265636629 shutdown_connections 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- 172.21.15.86:0/1265636629 wait complete. 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 -- start start 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.663+0000 7f21286a3740 1 Processor -- start 2024-09-05T06:53:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d060 0x55717e575060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e575ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55717e57ae10 con 0x55717e56d980 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55717e45af10 con 0x55717e578240 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55717e57af90 con 0x55717e56d060 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f21286a3740 10 client.0 _ll_register_callbacks cb 0x55717e47edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52466/0 (socket says 172.21.15.86:52466) 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 -- 172.21.15.86:0/31163308 learned_addr learned my addr 172.21.15.86:0/31163308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2124867640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e575ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f211ffff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d060 0x55717e575060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 -- 172.21.15.86:0/31163308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d060 msgr2=0x55717e575060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d060 0x55717e575060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 -- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 msgr2=0x55717e575ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e575ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 -- 172.21.15.86:0/31163308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55717e1e1580 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f2117fff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f210800be60 tx=0x7f210809b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b0c150ba055dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f20ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.664+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210801e050 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f210809f120 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2110019070 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f21080a3d60 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 10 client.18396 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55717e2ab960 con 0x55717e56d980 2024-09-05T06:53:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2110019070 con 0x55717e56d980 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f21080b5080 con 0x55717e56d980 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 10 client.18396 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 10 client.18396 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 20 client.18396 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e58a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 4 client.18396 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55717e4dfb90 con 0x55717e587cb0 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.665+0000 7f211ffff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e58a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.666+0000 7f20ff7fe640 10 client.18396 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.666+0000 7f211ffff640 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e58a0c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2c53a391a03cc2 server_cookie=363e32b581ee43e7 in_seq=0 out_seq=0 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.666+0000 7f20ff7fe640 1 -- 172.21.15.86:0/31163308 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55717e4dfb90 con 0x55717e587cb0 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.666+0000 7f20ff7fe640 10 client.18396 handle_command_reply: tid=0 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.666+0000 7f21286a3740 1 client.18396 shutdown 2024-09-05T06:53:41.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 msgr2=0x55717e5755a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f210800be60 tx=0x7f210809b7f0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 shutdown_connections 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e58a0c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e575ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f20ff7fe640 0 client.18396 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d980 0x55717e5755a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 --2- 172.21.15.86:0/31163308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d060 0x55717e575060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 >> 172.21.15.86:0/31163308 conn(0x55717e5695f0 msgr2=0x55717e56f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 shutdown_connections 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 1 -- 172.21.15.86:0/31163308 wait complete. 2024-09-05T06:53:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.667+0000 7f21286a3740 20 client.18396 trim_cache size 0 max 16384 2024-09-05T06:53:41.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:41 smithi115 ceph-mon[34791]: pgmap v387: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 855 KiB/s wr, 75 op/s 2024-09-05T06:53:41.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:41 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:53:41.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:41 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:53:41.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:41 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:53:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 Processor -- start 2024-09-05T06:53:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 -- start start 2024-09-05T06:53:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e58f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e58fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21201b1390 con 0x55717e578240 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21201b2f30 con 0x55717e56d060 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f21286a3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21201b1a60 con 0x55717e56d980 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52476/0 (socket says 172.21.15.86:52476) 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 -- 172.21.15.86:0/1345178894 learned_addr learned my addr 172.21.15.86:0/1345178894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f211ffff640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e58f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2117fff640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e58fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 -- 172.21.15.86:0/1345178894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 msgr2=0x55717e58fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e58fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 -- 172.21.15.86:0/1345178894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 msgr2=0x55717e58f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e58f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 -- 172.21.15.86:0/1345178894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717e569420 con 0x55717e578240 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2117fff640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e58fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.684+0000 7f2124867640 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2118031d00 tx=0x7f211803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c7ec2ac7a4a8b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f2115ffb640 1 -- 172.21.15.86:0/1345178894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118051070 con 0x55717e578240 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f2115ffb640 1 -- 172.21.15.86:0/1345178894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2118036040 con 0x55717e578240 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f2115ffb640 1 -- 172.21.15.86:0/1345178894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180309e0 con 0x55717e578240 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 -- 172.21.15.86:0/1345178894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 msgr2=0x55717e5722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2118031d00 tx=0x7f211803e000 comp rx=0 tx=0).stop 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 -- 172.21.15.86:0/1345178894 shutdown_connections 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e578240 0x55717e5722e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e58fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 --2- 172.21.15.86:0/1345178894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e56d060 0x55717e58f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 -- 172.21.15.86:0/1345178894 >> 172.21.15.86:0/1345178894 conn(0x55717e5695f0 msgr2=0x55717e5770e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 -- 172.21.15.86:0/1345178894 shutdown_connections 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.685+0000 7f21286a3740 1 -- 172.21.15.86:0/1345178894 wait complete. 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 -- start start 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 Processor -- start 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e580960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e57d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21201b2830 con 0x55717e56d060 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21201b2130 con 0x55717e578240 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21201af390 con 0x55717e56d980 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f21286a3740 10 client.0 _ll_register_callbacks cb 0x7f210809ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f211ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f211ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52492/0 (socket says 172.21.15.86:52492) 2024-09-05T06:53:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f211ffff640 1 -- 172.21.15.86:0/2886244834 learned_addr learned my addr 172.21.15.86:0/2886244834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f2124867640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e57d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.686+0000 7f2117fff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e580960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 -- 172.21.15.86:0/2886244834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 msgr2=0x55717e580960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e580960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 -- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 msgr2=0x55717e57d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e57d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 -- 172.21.15.86:0/2886244834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55717e56fed0 con 0x55717e56d060 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f2124867640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e57d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f211ffff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f211000b740 tx=0x7f211000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ea31cd8f4b392 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000db50 con 0x55717e56d060 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f211000dcf0 con 0x55717e56d060 2024-09-05T06:53:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2118036040 con 0x55717e56d060 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f21100147c0 con 0x55717e56d060 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 10 client.18402 fetch_fsmap learned FSMap version 36 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55717e5f3240 con 0x55717e56d060 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2118036040 con 0x55717e56d060 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f2110013070 con 0x55717e56d060 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 10 client.18402 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 10 client.18402 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.687+0000 7f21286a3740 20 client.18402 populate_metadata read hostname 'smithi086' 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e5f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f21286a3740 4 client.18402 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1dbea106", "release": true, "await": true}] 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1dbea106", "release": true, "await": true}) -- 0x55717e4da0a0 con 0x55717e587cb0 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f2117fff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e5f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:53:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f2117fff640 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e5f5720 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1510aaf9a0c020 server_cookie=8daad2acdd31c9de in_seq=0 out_seq=0 2024-09-05T06:53:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.688+0000 7f20ff7fe640 10 client.18402 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.694+0000 7f20ff7fe640 1 -- 172.21.15.86:0/2886244834 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55717e4da0a0 con 0x55717e587cb0 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.694+0000 7f20ff7fe640 10 client.18402 handle_command_reply: tid=0 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.694+0000 7f21286a3740 1 client.18402 shutdown 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 msgr2=0x55717e580420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f211000b740 tx=0x7f211000bfb0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 shutdown_connections 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55717e587cb0 0x55717e5f5720 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55717e578240 0x55717e57d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55717e56d980 0x55717e580960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 --2- 172.21.15.86:0/2886244834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55717e56d060 0x55717e580420 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f20ff7fe640 0 client.18402 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 >> 172.21.15.86:0/2886244834 conn(0x55717e5695f0 msgr2=0x55717e5768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 shutdown_connections 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 1 -- 172.21.15.86:0/2886244834 wait complete. 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.695+0000 7f21286a3740 20 client.18402 trim_cache size 0 max 16384 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:53:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 36, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 44, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "1dbea106": { 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 44, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:53:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:53:41.704 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 msgr2=0x7f210c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 0x7f210c0a51b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2118009a60 tx=0x7f2118009800 comp rx=0 tx=0).stop 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 msgr2=0x7f2120074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f210800b4a0 tx=0x7f210800b960 comp rx=0 tx=0).stop 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 shutdown_connections 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f210c0a2cf0 0x7f210c0a51b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2120106f20 0x7f2120074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21201027a0 0x7f2120074a30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 --2- 172.21.15.86:0/349322867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2120101df0 0x7f21200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:53:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 >> 172.21.15.86:0/349322867 conn(0x7f21200fd8f0 msgr2=0x7f21200fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:53:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 shutdown_connections 2024-09-05T06:53:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:53:41.696+0000 7f21262ca640 1 -- 172.21.15.86:0/349322867 wait complete. 2024-09-05T06:53:42.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 214205 1.59 MB/sec execute 542 sec latency 7487.308 ms 2024-09-05T06:53:42.656 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1dbea106', total seconds elapsed: 13.5 2024-09-05T06:53:43.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 214697 1.58 MB/sec execute 543 sec latency 19.188 ms 2024-09-05T06:53:43.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:43 smithi096 ceph-mon[34468]: pgmap v388: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 877 KiB/s wr, 81 op/s 2024-09-05T06:53:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:43 smithi115 ceph-mon[34791]: pgmap v388: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 877 KiB/s wr, 81 op/s 2024-09-05T06:53:43.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:43 smithi086 ceph-mon[30943]: pgmap v388: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 877 KiB/s wr, 81 op/s 2024-09-05T06:53:44.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 215343 1.59 MB/sec execute 544 sec latency 18.731 ms 2024-09-05T06:53:45.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 216155 1.59 MB/sec execute 545 sec latency 13.778 ms 2024-09-05T06:53:45.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:45 smithi096 ceph-mon[34468]: pgmap v389: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:45 smithi115 ceph-mon[34791]: pgmap v389: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:45 smithi086 ceph-mon[30943]: pgmap v389: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:46.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 216537 1.60 MB/sec execute 546 sec latency 49.246 ms 2024-09-05T06:53:47.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 216810 1.60 MB/sec execute 547 sec latency 155.168 ms 2024-09-05T06:53:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:47 smithi096 ceph-mon[34468]: pgmap v390: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:47 smithi115 ceph-mon[34791]: pgmap v390: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:47 smithi086 ceph-mon[30943]: pgmap v390: 129 pgs: 129 active+clean; 248 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 424 KiB/s wr, 48 op/s 2024-09-05T06:53:48.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 217164 1.60 MB/sec execute 548 sec latency 16.137 ms 2024-09-05T06:53:49.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 217707 1.60 MB/sec execute 549 sec latency 14.198 ms 2024-09-05T06:53:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:49 smithi096 ceph-mon[34468]: pgmap v391: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-05T06:53:49.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:49 smithi115 ceph-mon[34791]: pgmap v391: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-05T06:53:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:49 smithi086 ceph-mon[30943]: pgmap v391: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-05T06:53:50.536 INFO:tasks.workunit.client.0.smithi086.stdout: 1 218269 1.60 MB/sec execute 550 sec latency 20.305 ms 2024-09-05T06:53:51.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 218853 1.60 MB/sec execute 551 sec latency 19.449 ms 2024-09-05T06:53:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:51 smithi096 ceph-mon[34468]: pgmap v392: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 806 KiB/s wr, 59 op/s 2024-09-05T06:53:51.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:51 smithi115 ceph-mon[34791]: pgmap v392: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 806 KiB/s wr, 59 op/s 2024-09-05T06:53:51.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:51 smithi086 ceph-mon[30943]: pgmap v392: 129 pgs: 129 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 806 KiB/s wr, 59 op/s 2024-09-05T06:53:52.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 219690 1.61 MB/sec execute 552 sec latency 12.884 ms 2024-09-05T06:53:53.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220037 1.61 MB/sec execute 553 sec latency 190.979 ms 2024-09-05T06:53:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:53 smithi096 ceph-mon[34468]: pgmap v393: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-05T06:53:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:53 smithi115 ceph-mon[34791]: pgmap v393: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-05T06:53:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:53 smithi086 ceph-mon[30943]: pgmap v393: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-09-05T06:53:54.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220037 1.61 MB/sec execute 554 sec latency 1191.075 ms 2024-09-05T06:53:55.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220037 1.61 MB/sec execute 555 sec latency 2191.154 ms 2024-09-05T06:53:55.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:55 smithi096 ceph-mon[34468]: pgmap v394: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:55 smithi115 ceph-mon[34791]: pgmap v394: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:55 smithi086 ceph-mon[30943]: pgmap v394: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:56.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220037 1.60 MB/sec execute 556 sec latency 3191.253 ms 2024-09-05T06:53:57.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220104 1.60 MB/sec execute 557 sec latency 3396.794 ms 2024-09-05T06:53:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:57 smithi096 ceph-mon[34468]: pgmap v395: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:57 smithi115 ceph-mon[34791]: pgmap v395: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:57 smithi086 ceph-mon[30943]: pgmap v395: 129 pgs: 129 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-05T06:53:58.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220104 1.60 MB/sec execute 558 sec latency 1557.751 ms 2024-09-05T06:53:59.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220104 1.60 MB/sec execute 559 sec latency 2557.788 ms 2024-09-05T06:53:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:53:59 smithi096 ceph-mon[34468]: pgmap v396: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-05T06:53:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:53:59 smithi115 ceph-mon[34791]: pgmap v396: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-05T06:53:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:53:59 smithi086 ceph-mon[30943]: pgmap v396: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-09-05T06:54:00.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220104 1.59 MB/sec execute 560 sec latency 3557.844 ms 2024-09-05T06:54:01.229 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-09-05T06:54:01.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:54:01.490 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:01.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220104 1.59 MB/sec execute 561 sec latency 4557.955 ms 2024-09-05T06:54:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:01 smithi096 ceph-mon[34468]: pgmap v397: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-05T06:54:01.611 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:01 smithi086 ceph-mon[30943]: pgmap v397: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-05T06:54:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:01 smithi115 ceph-mon[34791]: pgmap v397: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 -- 172.21.15.86:0/4048727266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc102d90 msgr2=0x7f66bc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/4048727266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc102d90 0x7f66bc103170 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009930 tx=0x7f66ac02a420 comp rx=0 tx=0).stop 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 -- 172.21.15.86:0/4048727266 shutdown_connections 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/4048727266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc108290 0x7f66bc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/4048727266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc103740 0x7f66bc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/4048727266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc102d90 0x7f66bc103170 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 -- 172.21.15.86:0/4048727266 >> 172.21.15.86:0/4048727266 conn(0x7f66bc0fc7d0 msgr2=0x7f66bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 -- 172.21.15.86:0/4048727266 shutdown_connections 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.096+0000 7f66c0e7b640 1 -- 172.21.15.86:0/4048727266 wait complete. 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 Processor -- start 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 -- start start 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc102d90 0x7f66bc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc103740 0x7f66bc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc111f60 con 0x7f66bc108290 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc111de0 con 0x7f66bc102d90 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66c0e7b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1120e0 con 0x7f66bc103740 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51092/0 (socket says 172.21.15.86:51092) 2024-09-05T06:54:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66bad9d640 1 -- 172.21.15.86:0/1745419181 learned_addr learned my addr 172.21.15.86:0/1745419181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66ba59c640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc102d90 0x7f66bc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.097+0000 7f66b3fff640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc103740 0x7f66bc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 -- 172.21.15.86:0/1745419181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc103740 msgr2=0x7f66bc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc103740 0x7f66bc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 -- 172.21.15.86:0/1745419181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc102d90 msgr2=0x7f66bc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc102d90 0x7f66bc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 -- 172.21.15.86:0/1745419181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc19d480 con 0x7f66bc108290 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66bad9d640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f66b400eb10 tx=0x7f66b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4004b20 con 0x7f66bc108290 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc19d770 con 0x7f66bc108290 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b4004cc0 con 0x7f66bc108290 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4010690 con 0x7f66bc108290 2024-09-05T06:54:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.098+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1a5050 con 0x7f66bc108290 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.099+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc06a7f0 con 0x7f66bc108290 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.101+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f66b4005020 con 0x7f66bc108290 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.101+0000 7f66b37fe640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 0x7f66a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.101+0000 7f66ba59c640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 0x7f66a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.101+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f66b40cfd80 con 0x7f66bc108290 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.102+0000 7f66ba59c640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 0x7f66a80a5230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66ac031440 tx=0x7f66ac005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.106+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66b409c500 con 0x7f66bc108290 2024-09-05T06:54:02.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.328+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66bc10a2a0 con 0x7f66bc108290 2024-09-05T06:54:02.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.329+0000 7f66b37fe640 1 -- 172.21.15.86:0/1745419181 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7f66b409be00 con 0x7f66bc108290 2024-09-05T06:54:02.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:54:02.331 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:54:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:54:02.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 msgr2=0x7f66a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:02.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 0x7f66a80a5230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66ac031440 tx=0x7f66ac005a20 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 msgr2=0x7f66bc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f66b400eb10 tx=0x7f66b400c2d0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 shutdown_connections 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66a80a2d70 0x7f66a80a5230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66bc108290 0x7f66bc19cda0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66bc103740 0x7f66bc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 --2- 172.21.15.86:0/1745419181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66bc102d90 0x7f66bc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 >> 172.21.15.86:0/1745419181 conn(0x7f66bc0fc7d0 msgr2=0x7f66bc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 shutdown_connections 2024-09-05T06:54:02.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:02.332+0000 7f66c0e7b640 1 -- 172.21.15.86:0/1745419181 wait complete. 2024-09-05T06:54:02.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220349 1.59 MB/sec execute 562 sec latency 4763.047 ms 2024-09-05T06:54:03.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-09-05T06:54:03.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-09-05T06:54:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:03 smithi086 ceph-mon[30943]: pgmap v398: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-05T06:54:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:03 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1745419181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:03.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220705 1.59 MB/sec execute 563 sec latency 16.785 ms 2024-09-05T06:54:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:03 smithi115 ceph-mon[34791]: pgmap v398: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-05T06:54:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:03 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1745419181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:03 smithi096 ceph-mon[34468]: pgmap v398: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-05T06:54:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:03 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1745419181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 -- 172.21.15.86:0/2509315140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 msgr2=0x7f57bc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 --2- 172.21.15.86:0/2509315140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc10e600 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009ca0 tx=0x7f57b402a950 comp rx=0 tx=0).stop 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 -- 172.21.15.86:0/2509315140 shutdown_connections 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 --2- 172.21.15.86:0/2509315140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc10e600 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 --2- 172.21.15.86:0/2509315140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 0x7f57bc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 --2- 172.21.15.86:0/2509315140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 -- 172.21.15.86:0/2509315140 >> 172.21.15.86:0/2509315140 conn(0x7f57bc0fc7d0 msgr2=0x7f57bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 -- 172.21.15.86:0/2509315140 shutdown_connections 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.933+0000 7f57c107a640 1 -- 172.21.15.86:0/2509315140 wait complete. 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 Processor -- start 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 -- start start 2024-09-05T06:54:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 0x7f57bc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc111df0 con 0x7f57bc10c160 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc111c70 con 0x7f57bc102d30 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57c107a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc111f70 con 0x7f57bc103650 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57bad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38390/0 (socket says 172.21.15.86:38390) 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.934+0000 7f57bb59e640 1 -- 172.21.15.86:0/2871967900 learned_addr learned my addr 172.21.15.86:0/2871967900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57b3fff640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 0x7f57bc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 -- 172.21.15.86:0/2871967900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 msgr2=0x7f57bc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 0x7f57bc198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 -- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 msgr2=0x7f57bc19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 -- 172.21.15.86:0/2871967900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc19d300 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bb59e640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57bad9d640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc198350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e9f0 tx=0x7f57ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57b8d99640 1 -- 172.21.15.86:0/2871967900 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00ccc0 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc19d5f0 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57b8d99640 1 -- 172.21.15.86:0/2871967900 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ac0108e0 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57b8d99640 1 -- 172.21.15.86:0/2871967900 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac018880 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.935+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc1a4ed0 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57b8d99640 1 -- 172.21.15.86:0/2871967900 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f57ac010a90 con 0x7f57bc102d30 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 Processor -- start 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 -- start start 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a6080 0x55db2b3b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3b0940 0x55db2b3b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db2b30b2a0 con 0x55db2b3a6080 2024-09-05T06:54:03.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55db2b2b2b10 con 0x55db2b3a5760 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.938+0000 7f57c3453740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db2b307260 con 0x55db2b3b0940 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a6080 0x55db2b3b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:56534/0 (socket says 172.21.15.86:56534) 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 -- 172.21.15.86:0/3845396699 learned_addr learned my addr 172.21.15.86:0/3845396699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b3fff640 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3b0940 0x55db2b3b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b8d99640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 0x7f57a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 -- 172.21.15.86:0/3845396699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3b0940 msgr2=0x55db2b3b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3b0940 0x55db2b3b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 -- 172.21.15.86:0/3845396699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a6080 msgr2=0x55db2b3b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a6080 0x55db2b3b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 -- 172.21.15.86:0/3845396699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2b07cb90 con 0x55db2b3a5760 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 0x7f57a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b8d99640 1 -- 172.21.15.86:0/2871967900 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f57ac0cfc40 con 0x7f57bc102d30 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57b402fd70 tx=0x7f57b402fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2d4f7e9ec7574b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b17fa640 1 -- 172.21.15.86:0/3845396699 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4053070 con 0x55db2b3a5760 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b17fa640 1 -- 172.21.15.86:0/3845396699 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b4003e40 con 0x55db2b3a5760 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57b17fa640 1 -- 172.21.15.86:0/3845396699 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4034040 con 0x55db2b3a5760 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.939+0000 7f57bb59e640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 0x7f57a80a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57b4050000 tx=0x7f57b404a000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 -- 172.21.15.86:0/3845396699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 msgr2=0x55db2b3a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57b402fd70 tx=0x7f57b402fda0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 -- 172.21.15.86:0/3845396699 shutdown_connections 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3b0940 0x55db2b3b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a6080 0x55db2b3b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 --2- 172.21.15.86:0/3845396699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3a5b40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 -- 172.21.15.86:0/3845396699 >> 172.21.15.86:0/3845396699 conn(0x55db2b3a1cf0 msgr2=0x55db2b3a20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:03.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 -- 172.21.15.86:0/3845396699 shutdown_connections 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.940+0000 7f57c3453740 1 -- 172.21.15.86:0/3845396699 wait complete. 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 -- start start 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 Processor -- start 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db2b3b3510 con 0x55db2b3b0940 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55db2b2936e0 con 0x55db2b3a5760 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db2b3b3690 con 0x55db2b3a6080 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57c3453740 10 client.0 _ll_register_callbacks cb 0x55db2b2b7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38400/0 (socket says 172.21.15.86:38400) 2024-09-05T06:54:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57bb59e640 1 -- 172.21.15.86:0/915633244 learned_addr learned my addr 172.21.15.86:0/915633244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57bad9d640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57b3fff640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.941+0000 7f57bb59e640 1 -- 172.21.15.86:0/915633244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 msgr2=0x55db2b3add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57bb59e640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57bb59e640 1 -- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 msgr2=0x55db2b3ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57bb59e640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57bb59e640 1 -- 172.21.15.86:0/915633244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2b3bca30 con 0x55db2b3b0940 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57bb59e640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f57b402ae70 tx=0x7f57b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67927f8be7cc157 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b405b070 con 0x55db2b3b0940 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f57b4057670 con 0x55db2b3b0940 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db2b3189d0 con 0x55db2b3b0940 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f57b4067050 con 0x55db2b3b0940 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57c3453740 10 client.18420 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:03.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55db2b3b3510 con 0x55db2b3b0940 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.942+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55db2b3189d0 con 0x55db2b3b0940 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f57b404c470 con 0x55db2b3b0940 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 10 client.18420 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 10 client.18420 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 20 client.18420 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b3c0540 0x55db2b3c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 4 client.18420 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db2b312d20 con 0x55db2b3c0540 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57bad9d640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b3c0540 0x55db2b3c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57a27fc640 10 client.18420 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.943+0000 7f57bad9d640 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b3c0540 0x55db2b3c2930 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9ab87e3d23218b server_cookie=129215281f0dea43 in_seq=0 out_seq=0 2024-09-05T06:54:03.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57a27fc640 1 -- 172.21.15.86:0/915633244 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db2b312d20 con 0x55db2b3c0540 2024-09-05T06:54:03.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57a27fc640 10 client.18420 handle_command_reply: tid=0 2024-09-05T06:54:03.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 client.18420 shutdown 2024-09-05T06:54:03.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 msgr2=0x55db2b3ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f57b402ae70 tx=0x7f57b40046c0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 shutdown_connections 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b3c0540 0x55db2b3c2930 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3ae2b0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 --2- 172.21.15.86:0/915633244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a5760 0x55db2b3ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57a27fc640 0 client.18420 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 >> 172.21.15.86:0/915633244 conn(0x55db2b3a1cf0 msgr2=0x55db2b3a0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 shutdown_connections 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 1 -- 172.21.15.86:0/915633244 wait complete. 2024-09-05T06:54:03.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.944+0000 7f57c3453740 20 client.18420 trim_cache size 0 max 16384 2024-09-05T06:54:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 Processor -- start 2024-09-05T06:54:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 -- start start 2024-09-05T06:54:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 0x55db2b3ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1ad040 con 0x55db2b3b0940 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1aebe0 con 0x55db2b3a6080 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57c3453740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1ad710 con 0x55db2b3a5760 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57bad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 0x55db2b3ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:56552/0 (socket says 172.21.15.86:56552) 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.965+0000 7f57b3fff640 1 -- 172.21.15.86:0/3841956172 learned_addr learned my addr 172.21.15.86:0/3841956172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 -- 172.21.15.86:0/3841956172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 msgr2=0x55db2b3ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 0x55db2b3ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 -- 172.21.15.86:0/3841956172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 msgr2=0x55db2b3c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 -- 172.21.15.86:0/3841956172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2b39d260 con 0x55db2b3a6080 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57bad9d640 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 0x55db2b3ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57bb59e640 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b3fff640 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009cf0 tx=0x7f57a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f813f01fe9c079c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b0ff9640 1 -- 172.21.15.86:0/3841956172 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004030 con 0x55db2b3a6080 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b0ff9640 1 -- 172.21.15.86:0/3841956172 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a40041d0 con 0x55db2b3a6080 2024-09-05T06:54:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.966+0000 7f57b0ff9640 1 -- 172.21.15.86:0/3841956172 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4010430 con 0x55db2b3a6080 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- 172.21.15.86:0/3841956172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 msgr2=0x55db2b3a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009cf0 tx=0x7f57a4005f90 comp rx=0 tx=0).stop 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- 172.21.15.86:0/3841956172 shutdown_connections 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3b0940 0x55db2b3c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3a6080 0x55db2b3a8210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 --2- 172.21.15.86:0/3841956172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a5760 0x55db2b3ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- 172.21.15.86:0/3841956172 >> 172.21.15.86:0/3841956172 conn(0x55db2b3a1cf0 msgr2=0x55db2b3af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- 172.21.15.86:0/3841956172 shutdown_connections 2024-09-05T06:54:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- 172.21.15.86:0/3841956172 wait complete. 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.967+0000 7f57c3453740 1 -- start start 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 Processor -- start 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 0x55db2b3b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1ae4e0 con 0x55db2b3a5760 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1adde0 con 0x55db2b3b0940 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1ab040 con 0x55db2b3a6080 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57c3453740 10 client.0 _ll_register_callbacks cb 0x7f57a4009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38426/0 (socket says 172.21.15.86:38426) 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 -- 172.21.15.86:0/4062175677 learned_addr learned my addr 172.21.15.86:0/4062175677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57b3fff640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bb59e640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 0x55db2b3b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 -- 172.21.15.86:0/4062175677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 msgr2=0x55db2b3b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 -- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 msgr2=0x55db2b3b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 0x55db2b3b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bad9d640 1 -- 172.21.15.86:0/4062175677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2b3b9590 con 0x55db2b3a5760 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.968+0000 7f57bb59e640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 0x55db2b3b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57bad9d640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0ccf50 tx=0x7f57ac0d36f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6105f22616a31cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0d3020 con 0x55db2b3a5760 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a40041d0 con 0x55db2b3a5760 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9605+0+0 (secure 0 0 0) 0x7f57ac0cd170 con 0x55db2b3a5760 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f57ac0cd370 con 0x55db2b3a5760 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 10 client.18426 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55db2b42b700 con 0x55db2b3a5760 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a40041d0 con 0x55db2b3a5760 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f57ac0a44a0 con 0x55db2b3a5760 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 10 client.18426 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 10 client.18426 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 20 client.18426 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.969+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b42be40 0x55db2b42e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.970+0000 7f57c3453740 4 client.18426 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.970+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x55db2b28eec0 con 0x55db2b42be40 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.970+0000 7f57b3fff640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b42be40 0x55db2b42e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.970+0000 7f57a27fc640 10 client.18426 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:03.970+0000 7f57b3fff640 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b42be40 0x55db2b42e260 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4774d707f1d09035 server_cookie=4206db1e981d3ca4 in_seq=0 out_seq=0 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57a27fc640 1 -- 172.21.15.86:0/4062175677 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55db2b28eec0 con 0x55db2b42be40 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57a27fc640 10 client.18426 handle_command_reply: tid=0 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 client.18426 shutdown 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 msgr2=0x55db2b3b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0ccf50 tx=0x7f57ac0d36f0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 shutdown_connections 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55db2b42be40 0x55db2b42e260 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2b3b0940 0x55db2b3b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db2b3a6080 0x55db2b3b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 --2- 172.21.15.86:0/4062175677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2b3a5760 0x55db2b3b8910 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 >> 172.21.15.86:0/4062175677 conn(0x55db2b3a1cf0 msgr2=0x55db2b3a0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57a27fc640 0 client.18426 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 shutdown_connections 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 1 -- 172.21.15.86:0/4062175677 wait complete. 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.010+0000 7f57c3453740 20 client.18426 trim_cache size 0 max 16384 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 36, 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:54:04.012 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 46, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "b43bdd40": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 46, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 146.4, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:54:04.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:04.014 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 msgr2=0x7f57a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 0x7f57a80a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57b4050000 tx=0x7f57b404a000 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 msgr2=0x7f57bc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc198350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e9f0 tx=0x7f57ac00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 shutdown_connections 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a80a2cf0 0x7f57a80a51b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc10c160 0x7f57bc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57bc103650 0x7f57bc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 --2- 172.21.15.86:0/2871967900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57bc102d30 0x7f57bc198350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 >> 172.21.15.86:0/2871967900 conn(0x7f57bc0fc7d0 msgr2=0x7f57bc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 shutdown_connections 2024-09-05T06:54:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:04.011+0000 7f57c107a640 1 -- 172.21.15.86:0/2871967900 wait complete. 2024-09-05T06:54:04.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.59 MB/sec execute 564 sec latency 565.901 ms 2024-09-05T06:54:04.691 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b43bdd40', quiesce took 3.5 seconds. Will release after: 22.9 2024-09-05T06:54:05.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.58 MB/sec execute 565 sec latency 1565.982 ms 2024-09-05T06:54:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:05 smithi115 ceph-mon[34791]: pgmap v399: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:05 smithi086 ceph-mon[30943]: pgmap v399: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:05 smithi096 ceph-mon[34468]: pgmap v399: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:06.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:54:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:06.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.58 MB/sec execute 566 sec latency 2566.048 ms 2024-09-05T06:54:07.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 -- 172.21.15.86:0/2335036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0106f10 msgr2=0x7f72f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 --2- 172.21.15.86:0/2335036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0106f10 0x7f72f010ea90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009ca0 tx=0x7f72e802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 -- 172.21.15.86:0/2335036617 shutdown_connections 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 --2- 172.21.15.86:0/2335036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0106f10 0x7f72f010ea90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 --2- 172.21.15.86:0/2335036617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0102790 0x7f72f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 --2- 172.21.15.86:0/2335036617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0101de0 0x7f72f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 -- 172.21.15.86:0/2335036617 >> 172.21.15.86:0/2335036617 conn(0x7f72f00fd8c0 msgr2=0x7f72f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 -- 172.21.15.86:0/2335036617 shutdown_connections 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 -- 172.21.15.86:0/2335036617 wait complete. 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.017+0000 7f72f6439640 1 Processor -- start 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 -- start start 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0101de0 0x7f72f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 0x7f72f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111e20 con 0x7f72f0102790 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111ca0 con 0x7f72f0101de0 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f6439640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111fa0 con 0x7f72f0106f10 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38450/0 (socket says 172.21.15.86:38450) 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 -- 172.21.15.86:0/29692958 learned_addr learned my addr 172.21.15.86:0/29692958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72effff640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0101de0 0x7f72f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f49d6640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 0x7f72f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 -- 172.21.15.86:0/29692958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 msgr2=0x7f72f0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 0x7f72f0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 -- 172.21.15.86:0/29692958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0101de0 msgr2=0x7f72f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0101de0 0x7f72f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 -- 172.21.15.86:0/29692958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f01a67e0 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72f49d6640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 0x7f72f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.018+0000 7f72e7fff640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f72d800c890 tx=0x7f72d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.019+0000 7f72edffb640 1 -- 172.21.15.86:0/29692958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004220 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.019+0000 7f72edffb640 1 -- 172.21.15.86:0/29692958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72d80043c0 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.019+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f01a6ad0 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.019+0000 7f72edffb640 1 -- 172.21.15.86:0/29692958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004a00 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.019+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a7010 con 0x7f72f0102790 2024-09-05T06:54:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72edffb640 1 -- 172.21.15.86:0/29692958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f72d8004ba0 con 0x7f72f0102790 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 Processor -- start 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 -- start start 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d2b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d36680 0x55f088d38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f088c912e0 con 0x55f088d2bdc0 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f088c38ce0 con 0x55f088d36680 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.021+0000 7f72f8812740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f088c8d2a0 con 0x55f088d2b4a0 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38458/0 (socket says 172.21.15.86:38458) 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 -- 172.21.15.86:0/72788397 learned_addr learned my addr 172.21.15.86:0/72788397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72edffb640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 0x7f72dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72f49d6640 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d2b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72f49d6640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 0x7f72dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 -- 172.21.15.86:0/72788397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 msgr2=0x55f088d2b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d2b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 -- 172.21.15.86:0/72788397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d36680 msgr2=0x55f088d38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d36680 0x55f088d38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 -- 172.21.15.86:0/72788397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f088c1d440 con 0x55f088d2bdc0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72edffb640 1 -- 172.21.15.86:0/29692958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f72d80cf570 con 0x7f72f0102790 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72effff640 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f72e000ac10 tx=0x7f72e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6149119e835ab99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72f49d6640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 0x7f72dc0a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009ca0 tx=0x7f72e80095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72e67fc640 1 -- 172.21.15.86:0/72788397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e001d070 con 0x55f088d2bdc0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72e67fc640 1 -- 172.21.15.86:0/72788397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e0018af0 con 0x55f088d2bdc0 2024-09-05T06:54:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.022+0000 7f72e67fc640 1 -- 172.21.15.86:0/72788397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0010b20 con 0x55f088d2bdc0 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- 172.21.15.86:0/72788397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 msgr2=0x55f088d36140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f72e000ac10 tx=0x7f72e000f790 comp rx=0 tx=0).stop 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- 172.21.15.86:0/72788397 shutdown_connections 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d36680 0x55f088d38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d36140 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 --2- 172.21.15.86:0/72788397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d2b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- 172.21.15.86:0/72788397 >> 172.21.15.86:0/72788397 conn(0x55f088d27a30 msgr2=0x55f088d27e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- 172.21.15.86:0/72788397 shutdown_connections 2024-09-05T06:54:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- 172.21.15.86:0/72788397 wait complete. 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.023+0000 7f72f8812740 1 -- start start 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 Processor -- start 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d36090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088d33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f088d39250 con 0x55f088d2bdc0 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f088c4c050 con 0x55f088d2b4a0 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f088d393d0 con 0x55f088d36680 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f8812740 10 client.0 _ll_register_callbacks cb 0x55f088c8c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38464/0 (socket says 172.21.15.86:38464) 2024-09-05T06:54:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 -- 172.21.15.86:0/787690763 learned_addr learned my addr 172.21.15.86:0/787690763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72effff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d36090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72f49d6640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088d33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 -- 172.21.15.86:0/787690763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 msgr2=0x55f088d33950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088d33950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 -- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 msgr2=0x55f088d36090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d36090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72e7fff640 1 -- 172.21.15.86:0/787690763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f088a023c0 con 0x55f088d2bdc0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.024+0000 7f72effff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72e7fff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f72d809b300 tx=0x7f72d809b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9b6489955ebf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d801d040 con 0x55f088d2bdc0 2024-09-05T06:54:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f72d8097c70 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e0018af0 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f72d8097040 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 10 client.18438 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f088d44d40 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e0018af0 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f72d80cf1e0 con 0x55f088d2bdc0 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 10 client.18438 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 20 client.18438 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088d46150 0x55f088d48540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 4 client.18438 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f088c9e850 con 0x55f088d46150 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.025+0000 7f72effff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088d46150 0x55f088d48540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.026+0000 7f72cf7fe640 10 client.18438 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.026+0000 7f72effff640 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088d46150 0x55f088d48540 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47db769b1322ea1c server_cookie=64854bb04444a03a in_seq=0 out_seq=0 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.026+0000 7f72cf7fe640 1 -- 172.21.15.86:0/787690763 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f088c9e850 con 0x55f088d46150 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.026+0000 7f72cf7fe640 10 client.18438 handle_command_reply: tid=0 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.026+0000 7f72f8812740 1 client.18438 shutdown 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 msgr2=0x55f088d33340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f72d809b300 tx=0x7f72d809b200 comp rx=0 tx=0).stop 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 shutdown_connections 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088d46150 0x55f088d48540 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088d33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d33340 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72cf7fe640 0 client.18438 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 --2- 172.21.15.86:0/787690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 >> 172.21.15.86:0/787690763 conn(0x55f088d27a30 msgr2=0x55f088d2d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 shutdown_connections 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 1 -- 172.21.15.86:0/787690763 wait complete. 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.027+0000 7f72f8812740 20 client.18438 trim_cache size 0 max 16384 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 Processor -- start 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 -- start start 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 0x55f088d4ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b13a0 con 0x55f088d36680 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b2f40 con 0x55f088d2bdc0 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f8812740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b1a70 con 0x55f088d2b4a0 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38478/0 (socket says 172.21.15.86:38478) 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 -- 172.21.15.86:0/452288696 learned_addr learned my addr 172.21.15.86:0/452288696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72effff640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72e7fff640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 0x55f088d4ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 -- 172.21.15.86:0/452288696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 msgr2=0x55f088d42510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d42510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 -- 172.21.15.86:0/452288696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 msgr2=0x55f088d4ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 0x55f088d4ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 -- 172.21.15.86:0/452288696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f088d31890 con 0x55f088d36680 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72e7fff640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 0x55f088d4ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:07.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72effff640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d42510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.044+0000 7f72f49d6640 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f72e8031d90 tx=0x7f72e803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd5416e342292e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72e5ffb640 1 -- 172.21.15.86:0/452288696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8043070 con 0x55f088d36680 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72e5ffb640 1 -- 172.21.15.86:0/452288696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e8036040 con 0x55f088d36680 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72e5ffb640 1 -- 172.21.15.86:0/452288696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8030900 con 0x55f088d36680 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 -- 172.21.15.86:0/452288696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 msgr2=0x55f088d4e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f72e8031d90 tx=0x7f72e803b000 comp rx=0 tx=0).stop 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 -- 172.21.15.86:0/452288696 shutdown_connections 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d36680 0x55f088d4e330 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2bdc0 0x55f088d4ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 --2- 172.21.15.86:0/452288696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d2b4a0 0x55f088d42510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 -- 172.21.15.86:0/452288696 >> 172.21.15.86:0/452288696 conn(0x55f088d27a30 msgr2=0x55f088d353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 -- 172.21.15.86:0/452288696 shutdown_connections 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.045+0000 7f72f8812740 1 -- 172.21.15.86:0/452288696 wait complete. 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 -- start start 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 Processor -- start 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d3ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088da9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b2840 con 0x55f088d2bdc0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01b2140 con 0x55f088d2b4a0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01af3a0 con 0x55f088d36680 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72f8812740 10 client.0 _ll_register_callbacks cb 0x7f72d8004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38488/0 (socket says 172.21.15.86:38488) 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.046+0000 7f72e7fff640 1 -- 172.21.15.86:0/2179992171 learned_addr learned my addr 172.21.15.86:0/2179992171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72effff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d3ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f49d6640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088da9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 -- 172.21.15.86:0/2179992171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 msgr2=0x55f088da9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088da9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 -- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 msgr2=0x55f088d3ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d3ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 -- 172.21.15.86:0/2179992171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f088daa4f0 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72effff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d3ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f49d6640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088da9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72e7fff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f72d809aad0 tx=0x7f72d8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd26114627dae117 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d801d040 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f72d8084b40 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e8036040 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f72d80cf360 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 10 client.18444 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f088db17f0 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e8036040 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f72d80b4330 con 0x55f088d2bdc0 2024-09-05T06:54:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 10 client.18444 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 10 client.18444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 20 client.18444 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088db1d80 0x55f088db41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.047+0000 7f72f8812740 4 client.18444 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f088c98d60 con 0x55f088db1d80 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72effff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088db1d80 0x55f088db41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72effff640 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088db1d80 0x55f088db41a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3fbcd1d20eda04 server_cookie=5c66cf61d1d87db5 in_seq=0 out_seq=0 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72cf7fe640 10 client.18444 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72cf7fe640 1 -- 172.21.15.86:0/2179992171 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f088c98d60 con 0x55f088db1d80 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72cf7fe640 10 client.18444 handle_command_reply: tid=0 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.048+0000 7f72f8812740 1 client.18444 shutdown 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 msgr2=0x55f088d3b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f72d809aad0 tx=0x7f72d8005760 comp rx=0 tx=0).stop 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 shutdown_connections 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f088db1d80 0x55f088db41a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f088d36680 0x55f088da9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f088d2bdc0 0x55f088d3b2c0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 --2- 172.21.15.86:0/2179992171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f088d2b4a0 0x55f088d3ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 >> 172.21.15.86:0/2179992171 conn(0x55f088d27a30 msgr2=0x55f088d26350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72cf7fe640 0 client.18444 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 shutdown_connections 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 1 -- 172.21.15.86:0/2179992171 wait complete. 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f8812740 20 client.18444 trim_cache size 0 max 16384 2024-09-05T06:54:07.068 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 msgr2=0x7f72dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 0x7f72dc0a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009ca0 tx=0x7f72e80095c0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 msgr2=0x7f72f0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f72d800c890 tx=0x7f72d800cd50 comp rx=0 tx=0).stop 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 shutdown_connections 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f72dc0a2cf0 0x7f72dc0a51b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72f0106f10 0x7f72f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72f0102790 0x7f72f0074a30 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 --2- 172.21.15.86:0/29692958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72f0101de0 0x7f72f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 >> 172.21.15.86:0/29692958 conn(0x7f72f00fd8c0 msgr2=0x7f72f00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 shutdown_connections 2024-09-05T06:54:07.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:07.049+0000 7f72f6439640 1 -- 172.21.15.86:0/29692958 wait complete. 2024-09-05T06:54:07.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.58 MB/sec execute 567 sec latency 3566.075 ms 2024-09-05T06:54:07.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:07 smithi086 ceph-mon[30943]: pgmap v400: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:07.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:07 smithi086 ceph-mon[30943]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T06:54:07.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:07 smithi115 ceph-mon[34791]: pgmap v400: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:07 smithi115 ceph-mon[34791]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T06:54:07.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:54:07.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:54:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:07 smithi096 ceph-mon[34468]: pgmap v400: 129 pgs: 129 active+clean; 260 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 497 KiB/s wr, 43 op/s 2024-09-05T06:54:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:07 smithi096 ceph-mon[34468]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T06:54:07.980 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 -- 172.21.15.86:0/340358601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 msgr2=0x7f6f781021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/340358601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f781021d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009900 tx=0x7f6f6802a410 comp rx=0 tx=0).stop 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 -- 172.21.15.86:0/340358601 shutdown_connections 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/340358601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f78106f20 0x7f6f7810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/340358601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/340358601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f781021d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 -- 172.21.15.86:0/340358601 >> 172.21.15.86:0/340358601 conn(0x7f6f780fd8f0 msgr2=0x7f6f780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 -- 172.21.15.86:0/340358601 shutdown_connections 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 -- 172.21.15.86:0/340358601 wait complete. 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.481+0000 7f6f7dd51640 1 Processor -- start 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 -- start start 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f78106f20 0x7f6f7819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111df0 con 0x7f6f78101df0 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111c70 con 0x7f6f78106f20 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f7dd51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111f70 con 0x7f6f781027a0 2024-09-05T06:54:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38506/0 (socket says 172.21.15.86:38506) 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 -- 172.21.15.86:0/1320381264 learned_addr learned my addr 172.21.15.86:0/1320381264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f76ffd640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 -- 172.21.15.86:0/1320381264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 msgr2=0x7f6f781987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 -- 172.21.15.86:0/1320381264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f78106f20 msgr2=0x7f6f7819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f78106f20 0x7f6f7819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f777fe640 1 -- 172.21.15.86:0/1320381264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7819d220 con 0x7f6f78101df0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.482+0000 7f6f76ffd640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f777fe640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006010 tx=0x7f6f68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6803f030 con 0x7f6f78101df0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f680035f0 con 0x7f6f78101df0 2024-09-05T06:54:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7819d4b0 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68035070 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.483+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781a4df0 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.484+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f781032d0 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.485+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f68003b10 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.485+0000 7f6f74ff9640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.486+0000 7f6f76ffd640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.486+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f6f680f0510 con 0x7f6f78101df0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.486+0000 7f6f76ffd640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 0x7f6f600a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f781088d0 tx=0x7f6f6c005d90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.490+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f680bcca0 con 0x7f6f78101df0 2024-09-05T06:54:08.537 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.57 MB/sec execute 568 sec latency 4566.128 ms 2024-09-05T06:54:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.690+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f781021d0 con 0x7f6f78101df0 2024-09-05T06:54:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.690+0000 7f6f74ff9640 1 -- 172.21.15.86:0/1320381264 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7f6f680bc5a0 con 0x7f6f78101df0 2024-09-05T06:54:08.691 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:54:08.691 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:54:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 msgr2=0x7f6f600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 0x7f6f600a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f781088d0 tx=0x7f6f6c005d90 comp rx=0 tx=0).stop 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 msgr2=0x7f6f78198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006010 tx=0x7f6f68004270 comp rx=0 tx=0).stop 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 shutdown_connections 2024-09-05T06:54:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f78106f20 0x7f6f7819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f781027a0 0x7f6f781987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 --2- 172.21.15.86:0/1320381264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78101df0 0x7f6f78198270 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 >> 172.21.15.86:0/1320381264 conn(0x7f6f780fd8f0 msgr2=0x7f6f780fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 shutdown_connections 2024-09-05T06:54:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:08.693+0000 7f6f7dd51640 1 -- 172.21.15.86:0/1320381264 wait complete. 2024-09-05T06:54:09.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:54:09.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.57 MB/sec execute 569 sec latency 5566.184 ms 2024-09-05T06:54:09.657 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:09.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:09 smithi115 ceph-mon[34791]: pgmap v402: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:09 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1320381264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:09 smithi086 ceph-mon[30943]: pgmap v402: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:09 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1320381264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:09.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:09 smithi096 ceph-mon[34468]: pgmap v402: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:09.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:09 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1320381264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:10.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 -- 172.21.15.86:0/3558725807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b41027a0 msgr2=0x7f60b41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 --2- 172.21.15.86:0/3558725807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b41027a0 0x7f60b41067f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009900 tx=0x7f60a002a410 comp rx=0 tx=0).stop 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 -- 172.21.15.86:0/3558725807 shutdown_connections 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 --2- 172.21.15.86:0/3558725807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b4106f20 0x7f60b410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 --2- 172.21.15.86:0/3558725807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b41027a0 0x7f60b41067f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 --2- 172.21.15.86:0/3558725807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4101df0 0x7f60b41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 -- 172.21.15.86:0/3558725807 >> 172.21.15.86:0/3558725807 conn(0x7f60b40fd8f0 msgr2=0x7f60b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 -- 172.21.15.86:0/3558725807 shutdown_connections 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.143+0000 7f60bb23e640 1 -- 172.21.15.86:0/3558725807 wait complete. 2024-09-05T06:54:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 Processor -- start 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 -- start start 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b41027a0 0x7f60b4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 0x7f60b419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4111e30 con 0x7f60b4101df0 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4111cb0 con 0x7f60b4106f20 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60bb23e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4111fb0 con 0x7f60b41027a0 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38518/0 (socket says 172.21.15.86:38518) 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 -- 172.21.15.86:0/2519882742 learned_addr learned my addr 172.21.15.86:0/2519882742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b3fff640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b41027a0 0x7f60b4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 -- 172.21.15.86:0/2519882742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b41027a0 msgr2=0x7f60b4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b41027a0 0x7f60b4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b97db640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 0x7f60b419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 -- 172.21.15.86:0/2519882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 msgr2=0x7f60b419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 0x7f60b419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 -- 172.21.15.86:0/2519882742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b419d2c0 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b97db640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 0x7f60b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.144+0000 7f60b8fda640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f60a800c890 tx=0x7f60a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.145+0000 7f60b1ffb640 1 -- 172.21.15.86:0/2519882742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8004220 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.145+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b419d5b0 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.145+0000 7f60b1ffb640 1 -- 172.21.15.86:0/2519882742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a80043c0 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.145+0000 7f60b1ffb640 1 -- 172.21.15.86:0/2519882742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8004a00 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.145+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41a4e90 con 0x7f60b4101df0 2024-09-05T06:54:10.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.147+0000 7f60b1ffb640 1 -- 172.21.15.86:0/2519882742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60a8004ba0 con 0x7f60b4101df0 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.147+0000 7f60bd617740 1 Processor -- start 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.147+0000 7f60bd617740 1 -- start start 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.147+0000 7f60b1ffb640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 0x7f60a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 0x5612bd3b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612bd31d2f0 con 0x5612bd3c6780 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612bd2c4cf0 con 0x5612bd3b75d0 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 0x7f60a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60bd617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5612bd3192b0 con 0x5612bd3b8b40 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b1ffb640 1 -- 172.21.15.86:0/2519882742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f60a80cf650 con 0x7f60b4101df0 2024-09-05T06:54:10.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38534/0 (socket says 172.21.15.86:38534) 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 -- 172.21.15.86:0/88992225 learned_addr learned my addr 172.21.15.86:0/88992225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b8fda640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 0x5612bd3b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 0x7f60a40a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0136b0 tx=0x7f60ac013ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 -- 172.21.15.86:0/88992225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 msgr2=0x5612bd3b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 0x5612bd3b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 -- 172.21.15.86:0/88992225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 msgr2=0x5612bd3c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 -- 172.21.15.86:0/88992225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bd2a93a0 con 0x5612bd3b75d0 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b8fda640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 0x5612bd3b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b97db640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.148+0000 7f60b3fff640 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3b7a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a0032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611b644a163e873e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f609effd640 1 -- 172.21.15.86:0/88992225 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a003f030 con 0x5612bd3b75d0 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f609effd640 1 -- 172.21.15.86:0/88992225 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a0004020 con 0x5612bd3b75d0 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f609effd640 1 -- 172.21.15.86:0/88992225 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0035070 con 0x5612bd3b75d0 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 -- 172.21.15.86:0/88992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 msgr2=0x5612bd3b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3b7a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a0032c30 comp rx=0 tx=0).stop 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 -- 172.21.15.86:0/88992225 shutdown_connections 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3b7a10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 --2- 172.21.15.86:0/88992225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b8b40 0x5612bd3b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 -- 172.21.15.86:0/88992225 >> 172.21.15.86:0/88992225 conn(0x5612bd380310 msgr2=0x5612bd3b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 -- 172.21.15.86:0/88992225 shutdown_connections 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.149+0000 7f60bd617740 1 -- 172.21.15.86:0/88992225 wait complete. 2024-09-05T06:54:10.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 -- start start 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 Processor -- start 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b75d0 0x5612bd3c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612bd3c9420 con 0x5612bd3c6780 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612bd2d8060 con 0x5612bd3b8b40 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5612bd3c95a0 con 0x5612bd3b75d0 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60bd617740 10 client.0 _ll_register_callbacks cb 0x5612bd02b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b97db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b97db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38548/0 (socket says 172.21.15.86:38548) 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b97db640 1 -- 172.21.15.86:0/3242481399 learned_addr learned my addr 172.21.15.86:0/3242481399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:10.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b75d0 0x5612bd3c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b3fff640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b97db640 1 -- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b75d0 msgr2=0x5612bd3c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.150+0000 7f60b97db640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b75d0 0x5612bd3c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60b97db640 1 -- 172.21.15.86:0/3242481399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 msgr2=0x5612bd3c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60b97db640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60b97db640 1 -- 172.21.15.86:0/3242481399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bd08e3c0 con 0x5612bd3c6780 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60b97db640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0119c0 tx=0x7f60ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f825d4d816dddf1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac013220 con 0x5612bd3c6780 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a0004020 con 0x5612bd3c6780 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f60ac004b20 con 0x5612bd3c6780 2024-09-05T06:54:10.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f60ac032050 con 0x5612bd3c6780 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60bd617740 10 client.18462 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612bd3d4ec0 con 0x5612bd3c6780 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a0004020 con 0x5612bd3c6780 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f60ac005380 con 0x5612bd3c6780 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60bd617740 10 client.18462 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.151+0000 7f60bd617740 10 client.18462 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60bd617740 20 client.18462 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd3d62d0 0x5612bd3d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60bd617740 4 client.18462 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612bd32a860 con 0x5612bd3d62d0 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd3d62d0 0x5612bd3d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f6093fff640 10 client.18462 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.152+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd3d62d0 0x5612bd3d8740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5485073f012ab281 server_cookie=6a2081374f863400 in_seq=0 out_seq=0 2024-09-05T06:54:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f6093fff640 1 -- 172.21.15.86:0/3242481399 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612bd32a860 con 0x5612bd3d62d0 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f6093fff640 10 client.18462 handle_command_reply: tid=0 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 client.18462 shutdown 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 msgr2=0x5612bd3c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0119c0 tx=0x7f60ac004ab0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 shutdown_connections 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd3d62d0 0x5612bd3d8740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3c6780 0x5612bd3c3970 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 --2- 172.21.15.86:0/3242481399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3b75d0 0x5612bd3c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f6093fff640 0 client.18462 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 >> 172.21.15.86:0/3242481399 conn(0x5612bd380310 msgr2=0x5612bd3b99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 shutdown_connections 2024-09-05T06:54:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 1 -- 172.21.15.86:0/3242481399 wait complete. 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.153+0000 7f60bd617740 20 client.18462 trim_cache size 0 max 16384 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 Processor -- start 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 -- start start 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41ad080 con 0x5612bd3b75d0 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41aec20 con 0x5612bd3b8b40 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60bd617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41ad750 con 0x5612bd3c6780 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38562/0 (socket says 172.21.15.86:38562) 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 -- 172.21.15.86:0/4061189115 learned_addr learned my addr 172.21.15.86:0/4061189115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 -- 172.21.15.86:0/4061189115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 msgr2=0x5612bd3de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 -- 172.21.15.86:0/4061189115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 msgr2=0x5612bd3d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.166+0000 7f60b8fda640 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60b8fda640 1 -- 172.21.15.86:0/4061189115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bd3b34f0 con 0x5612bd3b75d0 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60b8fda640 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097b40 tx=0x7f60a8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a22a50de87b9529 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f609e7fc640 1 -- 172.21.15.86:0/4061189115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a801d040 con 0x5612bd3b75d0 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f609e7fc640 1 -- 172.21.15.86:0/4061189115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a809b6e0 con 0x5612bd3b75d0 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f609e7fc640 1 -- 172.21.15.86:0/4061189115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a80050b0 con 0x5612bd3b75d0 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 -- 172.21.15.86:0/4061189115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 msgr2=0x5612bd3b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097b40 tx=0x7f60a8007f90 comp rx=0 tx=0).stop 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 -- 172.21.15.86:0/4061189115 shutdown_connections 2024-09-05T06:54:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b8b40 0x5612bd3d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 --2- 172.21.15.86:0/4061189115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b75d0 0x5612bd3b3e10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 -- 172.21.15.86:0/4061189115 >> 172.21.15.86:0/4061189115 conn(0x5612bd380310 msgr2=0x5612bd3b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 -- 172.21.15.86:0/4061189115 shutdown_connections 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.167+0000 7f60bd617740 1 -- 172.21.15.86:0/4061189115 wait complete. 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 -- start start 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 Processor -- start 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41ae520 con 0x5612bd3b8b40 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41ade20 con 0x5612bd3b75d0 2024-09-05T06:54:10.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41ab080 con 0x5612bd3c6780 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60bd617740 10 client.0 _ll_register_callbacks cb 0x7f60ac011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.168+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38568/0 (socket says 172.21.15.86:38568) 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 -- 172.21.15.86:0/3039578096 learned_addr learned my addr 172.21.15.86:0/3039578096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b97db640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 -- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 msgr2=0x5612bd3c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 -- 172.21.15.86:0/3039578096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 msgr2=0x5612bd3cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 -- 172.21.15.86:0/3039578096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bd3c0dc0 con 0x5612bd3b8b40 2024-09-05T06:54:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60b3fff640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009230 tx=0x7f60a00316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdddb1f25427578b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f6093fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a003f040 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f60a0004020 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a809b6e0 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f60a0053050 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60bd617740 10 client.18468 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612bd440f40 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.169+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a809b6e0 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f60a00046c0 con 0x5612bd3b8b40 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 10 client.18468 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 10 client.18468 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:54:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 20 client.18468 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd441230 0x5612bd443650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 4 client.18468 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612bd324d70 con 0x5612bd441230 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd441230 0x5612bd443650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f6093fff640 10 client.18468 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60b8fda640 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd441230 0x5612bd443650 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d42c9a0627d32a0 server_cookie=ccab566ad6c31e5c in_seq=0 out_seq=0 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f6093fff640 1 -- 172.21.15.86:0/3039578096 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612bd324d70 con 0x5612bd441230 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f6093fff640 10 client.18468 handle_command_reply: tid=0 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.170+0000 7f60bd617740 1 client.18468 shutdown 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 msgr2=0x5612bd3c0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009230 tx=0x7f60a00316b0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 shutdown_connections 2024-09-05T06:54:10.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5612bd441230 0x5612bd443650 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612bd3c6780 0x5612bd3c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612bd3b8b40 0x5612bd3c0080 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f6093fff640 0 client.18468 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 --2- 172.21.15.86:0/3039578096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612bd3b75d0 0x5612bd3cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 >> 172.21.15.86:0/3039578096 conn(0x5612bd380310 msgr2=0x5612bd3c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 shutdown_connections 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 1 -- 172.21.15.86:0/3039578096 wait complete. 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bd617740 20 client.18468 trim_cache size 0 max 16384 2024-09-05T06:54:10.202 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 msgr2=0x7f60a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 0x7f60a40a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0136b0 tx=0x7f60ac013ab0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 msgr2=0x7f60b4198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.171+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f60a800c890 tx=0x7f60a800cd50 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 shutdown_connections 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60a40a2cf0 0x7f60a40a51b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60b4106f20 0x7f60b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b41027a0 0x7f60b4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 --2- 172.21.15.86:0/2519882742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b4101df0 0x7f60b4198310 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 >> 172.21.15.86:0/2519882742 conn(0x7f60b40fd8f0 msgr2=0x7f60b40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 shutdown_connections 2024-09-05T06:54:10.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:10.172+0000 7f60bb23e640 1 -- 172.21.15.86:0/2519882742 wait complete. 2024-09-05T06:54:10.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.57 MB/sec execute 570 sec latency 6566.210 ms 2024-09-05T06:54:10.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:54:11.155 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:11.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.57 MB/sec execute 571 sec latency 7566.524 ms 2024-09-05T06:54:11.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:11 smithi086 ceph-mon[30943]: pgmap v403: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 -- 172.21.15.86:0/3066449503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c102d40 msgr2=0x7f041c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 --2- 172.21.15.86:0/3066449503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c102d40 0x7f041c103120 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0418009900 tx=0x7f041802a480 comp rx=0 tx=0).stop 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 -- 172.21.15.86:0/3066449503 shutdown_connections 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 --2- 172.21.15.86:0/3066449503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c10c170 0x7f041c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 --2- 172.21.15.86:0/3066449503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c103660 0x7f041c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 --2- 172.21.15.86:0/3066449503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c102d40 0x7f041c103120 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 -- 172.21.15.86:0/3066449503 >> 172.21.15.86:0/3066449503 conn(0x7f041c0fc7a0 msgr2=0x7f041c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 -- 172.21.15.86:0/3066449503 shutdown_connections 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.637+0000 7f0424370640 1 -- 172.21.15.86:0/3066449503 wait complete. 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 Processor -- start 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 -- start start 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 0x7f041c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103660 0x7f041c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041c111e60 con 0x7f041c10c170 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041c111ce0 con 0x7f041c103660 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f0424370640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f041c111fe0 con 0x7f041c102d40 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38586/0 (socket says 172.21.15.86:38586) 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 -- 172.21.15.86:0/1426513120 learned_addr learned my addr 172.21.15.86:0/1426513120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042190b640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103660 0x7f041c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042210c640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 0x7f041c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 -- 172.21.15.86:0/1426513120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 msgr2=0x7f041c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 0x7f041c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 -- 172.21.15.86:0/1426513120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103660 msgr2=0x7f041c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103660 0x7f041c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042290d640 1 -- 172.21.15.86:0/1426513120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a6800 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.638+0000 7f042210c640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 0x7f041c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f042290d640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f041400b970 tx=0x7f041400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f04137fe640 1 -- 172.21.15.86:0/1426513120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414019070 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f04137fe640 1 -- 172.21.15.86:0/1426513120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0414014440 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f04137fe640 1 -- 172.21.15.86:0/1426513120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140075d0 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a6af0 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.639+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a6fe0 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.641+0000 7f04137fe640 1 -- 172.21.15.86:0/1426513120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f041401c070 con 0x7f041c10c170 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.641+0000 7f0426749740 1 Processor -- start 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.641+0000 7f0426749740 1 -- start start 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba31ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba32b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba2842f0 con 0x55f6ba31fb40 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba22bcf0 con 0x55f6ba31e5d0 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f0426749740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba2802b0 con 0x55f6ba329370 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f04137fe640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 0x7f04080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 -- 172.21.15.86:0/3494711335 learned_addr learned my addr 172.21.15.86:0/3494711335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042210c640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba31ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042290d640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 0x7f04080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba32b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f04137fe640 1 -- 172.21.15.86:0/1426513120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f04140cee50 con 0x7f041c10c170 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 -- 172.21.15.86:0/3494711335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 msgr2=0x55f6ba32b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba32b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 -- 172.21.15.86:0/3494711335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 msgr2=0x55f6ba31ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba31ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042190b640 1 -- 172.21.15.86:0/3494711335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ba2103a0 con 0x55f6ba31fb40 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.642+0000 7f042210c640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba31ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f042190b640 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f040c010370 tx=0x7f040c008570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b35ef36771674a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f042290d640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 0x7f04080a51b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04140cc2d0 tx=0x7f0414004b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0410dea640 1 -- 172.21.15.86:0/3494711335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c005020 con 0x55f6ba31fb40 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0410dea640 1 -- 172.21.15.86:0/3494711335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c0047a0 con 0x55f6ba31fb40 2024-09-05T06:54:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0410dea640 1 -- 172.21.15.86:0/3494711335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c012880 con 0x55f6ba31fb40 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 -- 172.21.15.86:0/3494711335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 msgr2=0x55f6ba31e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f040c010370 tx=0x7f040c008570 comp rx=0 tx=0).stop 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 -- 172.21.15.86:0/3494711335 shutdown_connections 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba32b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba31ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 --2- 172.21.15.86:0/3494711335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31fb40 0x55f6ba31e090 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.643+0000 7f0426749740 1 -- 172.21.15.86:0/3494711335 >> 172.21.15.86:0/3494711335 conn(0x55f6ba2e7310 msgr2=0x55f6ba31ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- 172.21.15.86:0/3494711335 shutdown_connections 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- 172.21.15.86:0/3494711335 wait complete. 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- start start 2024-09-05T06:54:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 Processor -- start 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba328d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba325fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba32c070 con 0x55f6ba329370 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba23f060 con 0x55f6ba31e5d0 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6ba32c1f0 con 0x55f6ba31fb40 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f0426749740 10 client.0 _ll_register_callbacks cb 0x55f6b9f928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38594/0 (socket says 172.21.15.86:38594) 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f042190b640 1 -- 172.21.15.86:0/1610825461 learned_addr learned my addr 172.21.15.86:0/1610825461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.644+0000 7f042190b640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba328d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042210c640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba325fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 -- 172.21.15.86:0/1610825461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 msgr2=0x55f6ba325fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba325fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 -- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 msgr2=0x55f6ba328d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba328d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 -- 172.21.15.86:0/1610825461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b9ff53c0 con 0x55f6ba329370 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f042190b640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f040c005750 tx=0x7f040c010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3db9db8c01312 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c012c60 con 0x55f6ba329370 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f040c012e00 con 0x55f6ba329370 2024-09-05T06:54:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c0047a0 con 0x55f6ba329370 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f040c004020 con 0x55f6ba329370 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0426749740 10 client.18480 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f6ba337af0 con 0x55f6ba329370 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c0047a0 con 0x55f6ba329370 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.645+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f040c0035f0 con 0x55f6ba329370 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 10 client.18480 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 10 client.18480 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 20 client.18480 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba33b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 4 client.18480 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6ba291860 con 0x55f6ba338f50 2024-09-05T06:54:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f042210c640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba33b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f042210c640 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba33b340 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=994f41d95f9dbe2c server_cookie=7c0c55d72b7fde48 in_seq=0 out_seq=0 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.646+0000 7f0405ffb640 10 client.18480 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0405ffb640 1 -- 172.21.15.86:0/1610825461 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6ba291860 con 0x55f6ba338f50 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0405ffb640 10 client.18480 handle_command_reply: tid=0 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 client.18480 shutdown 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 msgr2=0x55f6ba326560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f040c005750 tx=0x7f040c010f30 comp rx=0 tx=0).stop 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 shutdown_connections 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba33b340 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba326560 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba325fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0405ffb640 0 client.18480 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:54:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 --2- 172.21.15.86:0/1610825461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba328d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 >> 172.21.15.86:0/1610825461 conn(0x55f6ba2e7310 msgr2=0x55f6ba320930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 shutdown_connections 2024-09-05T06:54:11.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 1 -- 172.21.15.86:0/1610825461 wait complete. 2024-09-05T06:54:11.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.647+0000 7f0426749740 20 client.18480 trim_cache size 0 max 16384 2024-09-05T06:54:11.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 Processor -- start 2024-09-05T06:54:11.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 -- start start 2024-09-05T06:54:11.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 0x55f6ba327f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba340a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b14a0 con 0x55f6ba31e5d0 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b3040 con 0x55f6ba31fb40 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.660+0000 7f0426749740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b1b70 con 0x55f6ba329370 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38598/0 (socket says 172.21.15.86:38598) 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 -- 172.21.15.86:0/1317589691 learned_addr learned my addr 172.21.15.86:0/1317589691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042210c640 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 0x55f6ba327f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 -- 172.21.15.86:0/1317589691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 msgr2=0x55f6ba340a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba340a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 -- 172.21.15.86:0/1317589691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 msgr2=0x55f6ba327f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 0x55f6ba327f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 -- 172.21.15.86:0/1317589691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ba31b960 con 0x55f6ba31e5d0 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042210c640 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 0x55f6ba327f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f042190b640 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f040c005e10 tx=0x7f040c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f29dabfed09dd1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f0407fff640 1 -- 172.21.15.86:0/1317589691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c004730 con 0x55f6ba31e5d0 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f0407fff640 1 -- 172.21.15.86:0/1317589691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c003c80 con 0x55f6ba31e5d0 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.661+0000 7f0407fff640 1 -- 172.21.15.86:0/1317589691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c012a40 con 0x55f6ba31e5d0 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- 172.21.15.86:0/1317589691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 msgr2=0x55f6ba335340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f040c005e10 tx=0x7f040c004d70 comp rx=0 tx=0).stop 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- 172.21.15.86:0/1317589691 shutdown_connections 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba329370 0x55f6ba340a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31fb40 0x55f6ba327f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 --2- 172.21.15.86:0/1317589691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba31e5d0 0x55f6ba335340 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- 172.21.15.86:0/1317589691 >> 172.21.15.86:0/1317589691 conn(0x55f6ba2e7310 msgr2=0x55f6ba327820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- 172.21.15.86:0/1317589691 shutdown_connections 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- 172.21.15.86:0/1317589691 wait complete. 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 -- start start 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.662+0000 7f0426749740 1 Processor -- start 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba331740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba331c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b2940 con 0x55f6ba329370 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1b2240 con 0x55f6ba31e5d0 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1af4a0 con 0x55f6ba31fb40 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0426749740 10 client.0 _ll_register_callbacks cb 0x7f0418009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38604/0 (socket says 172.21.15.86:38604) 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 -- 172.21.15.86:0/3077034938 learned_addr learned my addr 172.21.15.86:0/3077034938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042210c640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba331c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba331740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 -- 172.21.15.86:0/3077034938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 msgr2=0x55f6ba331c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba331c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 -- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 msgr2=0x55f6ba331740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba331740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 -- 172.21.15.86:0/3077034938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ba321120 con 0x55f6ba329370 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042210c640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba331c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f042190b640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x55f6ba324050 tx=0x7f040c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbab4616e46565e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.663+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c012e30 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9693+0+0 (secure 0 0 0) 0x7f040c0041c0 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c003c80 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f040c0043c0 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 10 client.18486 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f6ba3a3380 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c003c80 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f040c043040 con 0x55f6ba329370 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 10 client.18486 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 10 client.18486 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 20 client.18486 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba3a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 4 client.18486 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6ba28bd70 con 0x55f6ba338f50 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.664+0000 7f042210c640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba3a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0405ffb640 10 client.18486 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f042210c640 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba3a5810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76b8b247549fe131 server_cookie=2cb47f52c954b680 in_seq=0 out_seq=0 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0405ffb640 1 -- 172.21.15.86:0/3077034938 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6ba28bd70 con 0x55f6ba338f50 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0405ffb640 10 client.18486 handle_command_reply: tid=0 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 client.18486 shutdown 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 msgr2=0x55f6ba322960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x55f6ba324050 tx=0x7f040c014040 comp rx=0 tx=0).stop 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 shutdown_connections 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f6ba338f50 0x55f6ba3a5810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6ba329370 0x55f6ba322960 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ba31fb40 0x55f6ba331c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0405ffb640 0 client.18486 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 --2- 172.21.15.86:0/3077034938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6ba31e5d0 0x55f6ba331740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 >> 172.21.15.86:0/3077034938 conn(0x55f6ba2e7310 msgr2=0x55f6ba320930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 shutdown_connections 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 1 -- 172.21.15.86:0/3077034938 wait complete. 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.665+0000 7f0426749740 20 client.18486 trim_cache size 0 max 16384 2024-09-05T06:54:11.678 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 msgr2=0x7f04080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 0x7f04080a51b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04140cc2d0 tx=0x7f0414004b50 comp rx=0 tx=0).stop 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 msgr2=0x7f041c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f041400b970 tx=0x7f041400be30 comp rx=0 tx=0).stop 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 shutdown_connections 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f04080a2cf0 0x7f04080a51b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f041c10c170 0x7f041c101420 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103660 0x7f041c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 --2- 172.21.15.86:0/1426513120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f041c102d40 0x7f041c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 >> 172.21.15.86:0/1426513120 conn(0x7f041c0fc7a0 msgr2=0x7f041c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 shutdown_connections 2024-09-05T06:54:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:11.666+0000 7f0424370640 1 -- 172.21.15.86:0/1426513120 wait complete. 2024-09-05T06:54:11.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:11 smithi115 ceph-mon[34791]: pgmap v403: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:11.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:11 smithi096 ceph-mon[34468]: pgmap v403: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 617 KiB/s wr, 82 op/s 2024-09-05T06:54:12.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:54:12.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.56 MB/sec execute 572 sec latency 8566.589 ms 2024-09-05T06:54:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:13.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:13 smithi086 ceph-mon[30943]: pgmap v404: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 574 KiB/s wr, 74 op/s 2024-09-05T06:54:13.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:13 smithi086 ceph-mon[30943]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.153+0000 7f29d78a0640 1 -- 172.21.15.86:0/1091642992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01036e0 msgr2=0x7f29d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.153+0000 7f29d78a0640 1 --2- 172.21.15.86:0/1091642992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01036e0 0x7f29d0107b90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009930 tx=0x7f29c002a420 comp rx=0 tx=0).stop 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- 172.21.15.86:0/1091642992 shutdown_connections 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- 172.21.15.86:0/1091642992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d01082a0 0x7f29d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- 172.21.15.86:0/1091642992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01036e0 0x7f29d0107b90 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- 172.21.15.86:0/1091642992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d0102d30 0x7f29d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- 172.21.15.86:0/1091642992 >> 172.21.15.86:0/1091642992 conn(0x7f29d00fc7d0 msgr2=0x7f29d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- 172.21.15.86:0/1091642992 shutdown_connections 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- 172.21.15.86:0/1091642992 wait complete. 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 Processor -- start 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- start start 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 0x7f29d01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 0x7f29d0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0111f70 con 0x7f29d01082a0 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0111df0 con 0x7f29d0102d30 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.154+0000 7f29d78a0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01120f0 con 0x7f29d01036e0 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36230/0 (socket says 172.21.15.86:36230) 2024-09-05T06:54:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 -- 172.21.15.86:0/2439663534 learned_addr learned my addr 172.21.15.86:0/2439663534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 0x7f29d0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d563c640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 0x7f29d01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 -- 172.21.15.86:0/2439663534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 msgr2=0x7f29d0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 0x7f29d0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 -- 172.21.15.86:0/2439663534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 msgr2=0x7f29d01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 0x7f29d01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 -- 172.21.15.86:0/2439663534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d019d560 con 0x7f29d01082a0 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 0x7f29d0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d563c640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 0x7f29d01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d5e3d640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f29c800cad0 tx=0x7f29c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29c67fc640 1 -- 172.21.15.86:0/2439663534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800ae20 con 0x7f29d01082a0 2024-09-05T06:54:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d019d850 con 0x7f29d01082a0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29c67fc640 1 -- 172.21.15.86:0/2439663534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29c8005020 con 0x7f29d01082a0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29c67fc640 1 -- 172.21.15.86:0/2439663534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800f6b0 con 0x7f29d01082a0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.155+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a5130 con 0x7f29d01082a0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29c67fc640 1 -- 172.21.15.86:0/2439663534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f29c80051d0 con 0x7f29d01082a0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 Processor -- start 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 -- start start 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8eb40 0x560d98f8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98f9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d98ef32f0 con 0x560d98f8d5d0 2024-09-05T06:54:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d98e9acf0 con 0x560d98f98370 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d9c79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d98eef2b0 con 0x560d98f8eb40 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29c67fc640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36240/0 (socket says 172.21.15.86:36240) 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.158+0000 7f29d563c640 1 -- 172.21.15.86:0/3202474414 learned_addr learned my addr 172.21.15.86:0/3202474414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d5e3d640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98f9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8eb40 0x560d98f8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29c67fc640 1 -- 172.21.15.86:0/2439663534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f29c80cf9e0 con 0x7f29d01082a0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 -- 172.21.15.86:0/3202474414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8eb40 msgr2=0x560d98f8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8eb40 0x560d98f8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 -- 172.21.15.86:0/3202474414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 msgr2=0x560d98f9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98f9a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 -- 172.21.15.86:0/3202474414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98e7f3a0 con 0x560d98f8d5d0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d563c640 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007850 tx=0x7f29cc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a46e761011ac5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29d5e3d640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ccda0 tx=0x7f29c801baf0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29bb7fe640 1 -- 172.21.15.86:0/3202474414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00d8d0 con 0x560d98f8d5d0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29bb7fe640 1 -- 172.21.15.86:0/3202474414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc018070 con 0x560d98f8d5d0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.159+0000 7f29bb7fe640 1 -- 172.21.15.86:0/3202474414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc013d70 con 0x560d98f8d5d0 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- 172.21.15.86:0/3202474414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 msgr2=0x560d98f8da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007850 tx=0x7f29cc007d10 comp rx=0 tx=0).stop 2024-09-05T06:54:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- 172.21.15.86:0/3202474414 shutdown_connections 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98f9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8d5d0 0x560d98f8da10 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 --2- 172.21.15.86:0/3202474414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8eb40 0x560d98f8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- 172.21.15.86:0/3202474414 >> 172.21.15.86:0/3202474414 conn(0x560d98f56310 msgr2=0x560d98f89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- 172.21.15.86:0/3202474414 shutdown_connections 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- 172.21.15.86:0/3202474414 wait complete. 2024-09-05T06:54:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 -- start start 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.160+0000 7f29d9c79740 1 Processor -- start 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98f97d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f95570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d98f9b070 con 0x560d98f8eb40 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d98eae060 con 0x560d98f8d5d0 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d98f9b1f0 con 0x560d98f98370 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d9c79740 10 client.0 _ll_register_callbacks cb 0x560d98c018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d4e3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98f97d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36246/0 (socket says 172.21.15.86:36246) 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 -- 172.21.15.86:0/311394290 learned_addr learned my addr 172.21.15.86:0/311394290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f95570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 -- 172.21.15.86:0/311394290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 msgr2=0x560d98f95570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f95570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 -- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 msgr2=0x560d98f97d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98f97d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 -- 172.21.15.86:0/311394290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98c643c0 con 0x560d98f8eb40 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29d563c640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f29cc006f40 tx=0x7f29cc0139b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceba97993dae82f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.161+0000 7f29b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00d8d0 con 0x560d98f8eb40 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f29cc00da70 con 0x560d98f8eb40 2024-09-05T06:54:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29cc018070 con 0x560d98f8eb40 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f29cc02f050 con 0x560d98f8eb40 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 10 client.18498 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d98fa6ac0 con 0x560d98f8eb40 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc018070 con 0x560d98f8eb40 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f29cc024c40 con 0x560d98f8eb40 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 10 client.18498 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 10 client.18498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 20 client.18498 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d98fa7f20 0x560d98faa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 4 client.18498 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d98f00860 con 0x560d98fa7f20 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.162+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d98fa7f20 0x560d98faa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.163+0000 7f29b8ff9640 10 client.18498 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.163+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d98fa7f20 0x560d98faa310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75f07300a25c40dd server_cookie=72edce0ae92eee50 in_seq=0 out_seq=0 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.163+0000 7f29b8ff9640 1 -- 172.21.15.86:0/311394290 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d98f00860 con 0x560d98fa7f20 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.163+0000 7f29b8ff9640 10 client.18498 handle_command_reply: tid=0 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.163+0000 7f29d9c79740 1 client.18498 shutdown 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 msgr2=0x560d98f94fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f29cc006f40 tx=0x7f29cc0139b0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 shutdown_connections 2024-09-05T06:54:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d98fa7f20 0x560d98faa310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29b8ff9640 0 client.18498 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f94fd0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 --2- 172.21.15.86:0/311394290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98f97d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 >> 172.21.15.86:0/311394290 conn(0x560d98f56310 msgr2=0x560d98f8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 shutdown_connections 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 1 -- 172.21.15.86:0/311394290 wait complete. 2024-09-05T06:54:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.164+0000 7f29d9c79740 20 client.18498 trim_cache size 0 max 16384 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 Processor -- start 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 -- start start 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8d5d0 0x560d98f90120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98faf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01af2c0 con 0x560d98f8eb40 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b0e60 con 0x560d98f98370 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d9c79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01af990 con 0x560d98f8d5d0 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36248/0 (socket says 172.21.15.86:36248) 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 -- 172.21.15.86:0/2725350873 learned_addr learned my addr 172.21.15.86:0/2725350873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98faf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8d5d0 0x560d98f90120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 -- 172.21.15.86:0/2725350873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8d5d0 msgr2=0x560d98f90120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8d5d0 0x560d98f90120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 -- 172.21.15.86:0/2725350873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 msgr2=0x560d98faf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98faf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 -- 172.21.15.86:0/2725350873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98f93540 con 0x560d98f8eb40 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.177+0000 7f29d563c640 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3f18b451e4f729 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29baffd640 1 -- 172.21.15.86:0/2725350873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc038070 con 0x560d98f8eb40 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29baffd640 1 -- 172.21.15.86:0/2725350873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc018040 con 0x560d98f8eb40 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29baffd640 1 -- 172.21.15.86:0/2725350873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00dbe0 con 0x560d98f8eb40 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 -- 172.21.15.86:0/2725350873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 msgr2=0x560d98f90660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc0076d0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 -- 172.21.15.86:0/2725350873 shutdown_connections 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f98370 0x560d98faf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98f90660 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 --2- 172.21.15.86:0/2725350873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f8d5d0 0x560d98f90120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 -- 172.21.15.86:0/2725350873 >> 172.21.15.86:0/2725350873 conn(0x560d98f56310 msgr2=0x560d98f97050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 -- 172.21.15.86:0/2725350873 shutdown_connections 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.178+0000 7f29d9c79740 1 -- 172.21.15.86:0/2725350873 wait complete. 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 -- start start 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 Processor -- start 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98fa0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f9d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b0760 con 0x560d98f8eb40 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b0060 con 0x560d98f8d5d0 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01ad2c0 con 0x560d98f98370 2024-09-05T06:54:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d9c79740 10 client.0 _ll_register_callbacks cb 0x7f29c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36258/0 (socket says 172.21.15.86:36258) 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 -- 172.21.15.86:0/1560391772 learned_addr learned my addr 172.21.15.86:0/1560391772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98fa0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f9d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 -- 172.21.15.86:0/1560391772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 msgr2=0x560d98f9d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f9d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 -- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 msgr2=0x560d98fa0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98fa0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d563c640 1 -- 172.21.15.86:0/1560391772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98fa10b0 con 0x560d98f8eb40 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.179+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f9d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d563c640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007010 tx=0x7f29cc01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39409d729d7a922 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc040060 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f29cc00d940 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29cc018040 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f29cc047050 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 10 client.18504 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d99012f80 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc018040 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f29cc0248d0 con 0x560d98f8eb40 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 10 client.18504 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 10 client.18504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 20 client.18504 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d99013670 0x560d99015a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 4 client.18504 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d98efad70 con 0x560d99013670 2024-09-05T06:54:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.180+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d99013670 0x560d99015a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29b8ff9640 10 client.18504 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:54:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d4e3b640 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d99013670 0x560d99015a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c102460ff4858572 server_cookie=fd9cfb90a41fc132 in_seq=0 out_seq=0 2024-09-05T06:54:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29b8ff9640 1 -- 172.21.15.86:0/1560391772 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d98efad70 con 0x560d99013670 2024-09-05T06:54:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29b8ff9640 10 client.18504 handle_command_reply: tid=0 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 client.18504 shutdown 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 msgr2=0x560d98fa09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007010 tx=0x7f29cc01c830 comp rx=0 tx=0).stop 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 shutdown_connections 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x560d99013670 0x560d99015a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d98f98370 0x560d98f9d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d98f8eb40 0x560d98fa09c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 --2- 172.21.15.86:0/1560391772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d98f8d5d0 0x560d98fa0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29b8ff9640 0 client.18504 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 >> 172.21.15.86:0/1560391772 conn(0x560d98f56310 msgr2=0x560d98f96830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 shutdown_connections 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 1 -- 172.21.15.86:0/1560391772 wait complete. 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.181+0000 7f29d9c79740 20 client.18504 trim_cache size 0 max 16384 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 msgr2=0x7f29bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ccda0 tx=0x7f29c801baf0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 msgr2=0x7f29d019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f29c800cad0 tx=0x7f29c800cf90 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 shutdown_connections 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29bc0a2cf0 0x7f29bc0a51b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29d01082a0 0x7f29d019ce80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29d01036e0 0x7f29d0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 --2- 172.21.15.86:0/2439663534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29d0102d30 0x7f29d01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 >> 172.21.15.86:0/2439663534 conn(0x7f29d00fc7d0 msgr2=0x7f29d0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 shutdown_connections 2024-09-05T06:54:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:13.182+0000 7f29d78a0640 1 -- 172.21.15.86:0/2439663534 wait complete. 2024-09-05T06:54:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:13 smithi096 ceph-mon[34468]: pgmap v404: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 574 KiB/s wr, 74 op/s 2024-09-05T06:54:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:13 smithi096 ceph-mon[34468]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T06:54:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:13 smithi115 ceph-mon[34791]: pgmap v404: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 574 KiB/s wr, 74 op/s 2024-09-05T06:54:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:13 smithi115 ceph-mon[34791]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T06:54:13.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.56 MB/sec execute 573 sec latency 9566.654 ms 2024-09-05T06:54:13.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:54:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:14.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.56 MB/sec execute 574 sec latency 10566.682 ms 2024-09-05T06:54:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 -- 172.21.15.86:0/1012522552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080102ef0 msgr2=0x7fb08010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1012522552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080102ef0 0x7fb08010da40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb070009ca0 tx=0x7fb07002a950 comp rx=0 tx=0).stop 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 -- 172.21.15.86:0/1012522552 shutdown_connections 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1012522552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080102ef0 0x7fb08010da40 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1012522552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb080102570 0x7fb0801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1012522552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080100e20 0x7fb080101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 -- 172.21.15.86:0/1012522552 >> 172.21.15.86:0/1012522552 conn(0x7fb0800fc7d0 msgr2=0x7fb0800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 -- 172.21.15.86:0/1012522552 shutdown_connections 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.732+0000 7fb0887b3640 1 -- 172.21.15.86:0/1012522552 wait complete. 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 Processor -- start 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 -- start start 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 0x7fb0801989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb080102ef0 0x7fb08019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111f40 con 0x7fb080100e20 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111dc0 con 0x7fb080102ef0 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb0887b3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801120c0 con 0x7fb080102570 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb08654f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb08654f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36284/0 (socket says 172.21.15.86:36284) 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb08654f640 1 -- 172.21.15.86:0/1288630539 learned_addr learned my addr 172.21.15.86:0/1288630539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb085d4e640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 0x7fb0801989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb08654f640 1 -- 172.21.15.86:0/1288630539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 msgr2=0x7fb0801989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.733+0000 7fb08654f640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 0x7fb0801989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb08654f640 1 -- 172.21.15.86:0/1288630539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb080102ef0 msgr2=0x7fb08019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb08654f640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb080102ef0 0x7fb08019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb08654f640 1 -- 172.21.15.86:0/1288630539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019d410 con 0x7fb080100e20 2024-09-05T06:54:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb085d4e640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 0x7fb0801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb08654f640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9f0 tx=0x7fb07800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb07f7fe640 1 -- 172.21.15.86:0/1288630539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07800cd80 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019d700 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb07f7fe640 1 -- 172.21.15.86:0/1288630539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0780054d0 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb07f7fe640 1 -- 172.21.15.86:0/1288630539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078010490 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.734+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a4fe0 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.736+0000 7fb07f7fe640 1 -- 172.21.15.86:0/1288630539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb078005020 con 0x7fb080100e20 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.736+0000 7fb08ab8c740 1 Processor -- start 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 -- start start 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc963ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc9647370 0x55ffc9649760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffc95a22f0 con 0x55ffc963db40 2024-09-05T06:54:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ffc9549cf0 con 0x55ffc963c5d0 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ffc959e2b0 con 0x55ffc9647370 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb07f7fe640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 0x7fb0680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08654f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc963ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36292/0 (socket says 172.21.15.86:36292) 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 -- 172.21.15.86:0/2810352584 learned_addr learned my addr 172.21.15.86:0/2810352584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 0x7fb0680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb085d4e640 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc9647370 0x55ffc9649760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb07f7fe640 1 -- 172.21.15.86:0/1288630539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb0780cfd70 con 0x7fb080100e20 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 -- 172.21.15.86:0/2810352584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc9647370 msgr2=0x55ffc9649760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc9647370 0x55ffc9649760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 -- 172.21.15.86:0/2810352584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 msgr2=0x55ffc963ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc963ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb086d50640 1 -- 172.21.15.86:0/2810352584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc952e3a0 con 0x55ffc963db40 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.737+0000 7fb08654f640 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc963ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.738+0000 7fb086d50640 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb070004af0 tx=0x7fb07002fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139321557296b8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.738+0000 7fb086d50640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 0x7fb0680a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb07002ae70 tx=0x7fb070034000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.738+0000 7fb07cff9640 1 -- 172.21.15.86:0/2810352584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070034070 con 0x55ffc963db40 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.738+0000 7fb07cff9640 1 -- 172.21.15.86:0/2810352584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb07002fe10 con 0x55ffc963db40 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.738+0000 7fb07cff9640 1 -- 172.21.15.86:0/2810352584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07003be60 con 0x55ffc963db40 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- 172.21.15.86:0/2810352584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 msgr2=0x55ffc963c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb070004af0 tx=0x7fb07002fc90 comp rx=0 tx=0).stop 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- 172.21.15.86:0/2810352584 shutdown_connections 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc9647370 0x55ffc9649760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc963ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/2810352584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc963c090 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- 172.21.15.86:0/2810352584 >> 172.21.15.86:0/2810352584 conn(0x55ffc9605310 msgr2=0x55ffc9638c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- 172.21.15.86:0/2810352584 shutdown_connections 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- 172.21.15.86:0/2810352584 wait complete. 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- start start 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 Processor -- start 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc9646d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc9644010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffc964a010 con 0x55ffc9647370 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ffc955d060 con 0x55ffc963c5d0 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.739+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ffc964a190 con 0x55ffc963db40 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08ab8c740 10 client.0 _ll_register_callbacks cb 0x55ffc92b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36308/0 (socket says 172.21.15.86:36308) 2024-09-05T06:54:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 -- 172.21.15.86:0/914241614 learned_addr learned my addr 172.21.15.86:0/914241614 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb085d4e640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc9644010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08654f640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc9646d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 -- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 msgr2=0x55ffc9644010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc9644010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 -- 172.21.15.86:0/914241614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 msgr2=0x55ffc9646d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc9646d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 -- 172.21.15.86:0/914241614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc93133c0 con 0x55ffc9647370 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08654f640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc9646d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb086d50640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb070004a80 tx=0x7fb070004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a282d0f1ed1c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070003970 con 0x55ffc9647370 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb070003b10 con 0x55ffc9647370 2024-09-05T06:54:14.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07002fe10 con 0x55ffc9647370 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb070003d10 con 0x55ffc9647370 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08ab8c740 10 client.18516 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ffc9655ae0 con 0x55ffc9647370 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.740+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb07002fe10 con 0x55ffc9647370 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb07005a450 con 0x55ffc9647370 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 10 client.18516 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 10 client.18516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 20 client.18516 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc9656e30 0x55ffc9659220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 4 client.18516 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc95af860 con 0x55ffc9656e30 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08654f640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc9656e30 0x55ffc9659220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb08654f640 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc9656e30 0x55ffc9659220 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=741a5c158cbf5e0d server_cookie=380b599602206425 in_seq=0 out_seq=0 2024-09-05T06:54:14.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.741+0000 7fb065ffb640 10 client.18516 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb065ffb640 1 -- 172.21.15.86:0/914241614 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffc95af860 con 0x55ffc9656e30 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb065ffb640 10 client.18516 handle_command_reply: tid=0 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 client.18516 shutdown 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 msgr2=0x55ffc9644600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb070004a80 tx=0x7fb070004d70 comp rx=0 tx=0).stop 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 shutdown_connections 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc9656e30 0x55ffc9659220 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc9647370 0x55ffc9644600 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb065ffb640 0 client.18516 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc9644010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/914241614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc963c5d0 0x55ffc9646d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 >> 172.21.15.86:0/914241614 conn(0x55ffc9605310 msgr2=0x55ffc963e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 shutdown_connections 2024-09-05T06:54:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.742+0000 7fb08ab8c740 1 -- 172.21.15.86:0/914241614 wait complete. 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.743+0000 7fb08ab8c740 20 client.18516 trim_cache size 0 max 16384 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.755+0000 7fb08ab8c740 1 Processor -- start 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.755+0000 7fb08ab8c740 1 -- start start 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc963f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc963f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801af190 con 0x55ffc963c5d0 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801b0d30 con 0x55ffc9647370 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801af860 con 0x55ffc963db40 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36322/0 (socket says 172.21.15.86:36322) 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 -- 172.21.15.86:0/3339276691 learned_addr learned my addr 172.21.15.86:0/3339276691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 -- 172.21.15.86:0/3339276691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 msgr2=0x55ffc963f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc963f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 -- 172.21.15.86:0/3339276691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 msgr2=0x55ffc963f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc963f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 -- 172.21.15.86:0/3339276691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc9642560 con 0x55ffc963c5d0 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb08654f640 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb0780cd130 tx=0x7fb07806ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9005e93ca811c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb067fff640 1 -- 172.21.15.86:0/3339276691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078066fa0 con 0x55ffc963c5d0 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb067fff640 1 -- 172.21.15.86:0/3339276691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb07809e040 con 0x55ffc963c5d0 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.756+0000 7fb067fff640 1 -- 172.21.15.86:0/3339276691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07809a150 con 0x55ffc963c5d0 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- 172.21.15.86:0/3339276691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 msgr2=0x55ffc9646340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb0780cd130 tx=0x7fb07806ef80 comp rx=0 tx=0).stop 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- 172.21.15.86:0/3339276691 shutdown_connections 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc963f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963db40 0x55ffc963f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/3339276691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963c5d0 0x55ffc9646340 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- 172.21.15.86:0/3339276691 >> 172.21.15.86:0/3339276691 conn(0x55ffc9605310 msgr2=0x55ffc9637d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- 172.21.15.86:0/3339276691 shutdown_connections 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- 172.21.15.86:0/3339276691 wait complete. 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- start start 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 Processor -- start 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963c5d0 0x55ffc964f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc964b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801b0630 con 0x55ffc963db40 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801aff30 con 0x55ffc9647370 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ad190 con 0x55ffc963c5d0 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb08ab8c740 10 client.0 _ll_register_callbacks cb 0x7fb0780981c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:14.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.757+0000 7fb085d4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36334/0 (socket says 172.21.15.86:36334) 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 -- 172.21.15.86:0/1481250198 learned_addr learned my addr 172.21.15.86:0/1481250198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb08654f640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963c5d0 0x55ffc964f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 -- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963c5d0 msgr2=0x55ffc964f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963c5d0 0x55ffc964f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 -- 172.21.15.86:0/1481250198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 msgr2=0x55ffc964b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc964b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 -- 172.21.15.86:0/1481250198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc964c1b0 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb085d4e640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x55ffc9647340 tx=0x7fb074013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600f58e11eec7ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074021060 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07809e040 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb074011ae0 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb074028050 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb08ab8c740 10 client.18522 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ffc96c2430 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.758+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb07809e040 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fb07400f040 con 0x55ffc963db40 2024-09-05T06:54:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 10 client.18522 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 10 client.18522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 20 client.18522 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc96c2be0 0x55ffc96c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 4 client.18522 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffc95a9d70 con 0x55ffc96c2be0 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08654f640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc96c2be0 0x55ffc96c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb08654f640 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc96c2be0 0x55ffc96c5000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2447bb7331aabe0 server_cookie=f71a541166a51129 in_seq=0 out_seq=0 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.759+0000 7fb065ffb640 10 client.18522 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb065ffb640 1 -- 172.21.15.86:0/1481250198 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffc95a9d70 con 0x55ffc96c2be0 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb065ffb640 10 client.18522 handle_command_reply: tid=0 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 client.18522 shutdown 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 msgr2=0x55ffc964faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x55ffc9647340 tx=0x7fb074013b70 comp rx=0 tx=0).stop 2024-09-05T06:54:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 shutdown_connections 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ffc96c2be0 0x55ffc96c5000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ffc9647370 0x55ffc964b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc963db40 0x55ffc964faf0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 --2- 172.21.15.86:0/1481250198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ffc963c5d0 0x55ffc964f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 >> 172.21.15.86:0/1481250198 conn(0x55ffc9605310 msgr2=0x55ffc96438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb065ffb640 0 client.18522 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 shutdown_connections 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 1 -- 172.21.15.86:0/1481250198 wait complete. 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb08ab8c740 20 client.18522 trim_cache size 0 max 16384 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 msgr2=0x7fb0680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 0x7fb0680a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb07002ae70 tx=0x7fb070034000 comp rx=0 tx=0).stop 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 msgr2=0x7fb080198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:14.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9f0 tx=0x7fb07800edf0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 shutdown_connections 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0680a2cf0 0x7fb0680a51b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb080102ef0 0x7fb08019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb080102570 0x7fb0801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 --2- 172.21.15.86:0/1288630539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080100e20 0x7fb080198460 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 >> 172.21.15.86:0/1288630539 conn(0x7fb0800fc7d0 msgr2=0x7fb0800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 shutdown_connections 2024-09-05T06:54:14.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:14.760+0000 7fb0887b3640 1 -- 172.21.15.86:0/1288630539 wait complete. 2024-09-05T06:54:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:15 smithi096 ceph-mon[34468]: pgmap v406: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 717 KiB/s wr, 93 op/s 2024-09-05T06:54:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:15 smithi115 ceph-mon[34791]: pgmap v406: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 717 KiB/s wr, 93 op/s 2024-09-05T06:54:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:15 smithi086 ceph-mon[30943]: pgmap v406: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 717 KiB/s wr, 93 op/s 2024-09-05T06:54:15.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.55 MB/sec execute 575 sec latency 11566.712 ms 2024-09-05T06:54:15.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:54:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.330+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/3637864671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac10c2f0 msgr2=0x7fc0ac10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.330+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/3637864671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac10e790 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009ca0 tx=0x7fc09c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/3637864671 shutdown_connections 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/3637864671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac10e790 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/3637864671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/3637864671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac102df0 0x7fc0ac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/3637864671 >> 172.21.15.86:0/3637864671 conn(0x7fc0ac0fc7d0 msgr2=0x7fc0ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/3637864671 shutdown_connections 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/3637864671 wait complete. 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.331+0000 7fc0b0ddd640 1 Processor -- start 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 -- start start 2024-09-05T06:54:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac102df0 0x7fc0ac1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac111fa0 con 0x7fc0ac1037a0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac111e20 con 0x7fc0ac102df0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0b0ddd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac112120 con 0x7fc0ac10c2f0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36352/0 (socket says 172.21.15.86:36352) 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/314174693 learned_addr learned my addr 172.21.15.86:0/314174693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac102df0 0x7fc0ac1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/314174693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac10c2f0 msgr2=0x7fc0ac19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/314174693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac102df0 msgr2=0x7fc0ac1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac102df0 0x7fc0ac1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/314174693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac19d570 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c970 tx=0x7fc09400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.332+0000 7fc0a37fe640 1 -- 172.21.15.86:0/314174693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094007bb0 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.333+0000 7fc0a37fe640 1 -- 172.21.15.86:0/314174693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc094007d50 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.333+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac19d860 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.333+0000 7fc0a37fe640 1 -- 172.21.15.86:0/314174693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09400f450 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.333+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1a5140 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.335+0000 7fc0a37fe640 1 -- 172.21.15.86:0/314174693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc094004eb0 con 0x7fc0ac1037a0 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.335+0000 7fc0b31b6740 1 Processor -- start 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.335+0000 7fc0b31b6740 1 -- start start 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe3b40 0x562850fe2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 0x562850fe2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562850f482f0 con 0x562850ff1780 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562850eefcf0 con 0x562850fe25d0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562850f442b0 con 0x562850fe3b40 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a37fe640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 0x7fc0980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 0x7fc0980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36364/0 (socket says 172.21.15.86:36364) 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/797390194 learned_addr learned my addr 172.21.15.86:0/797390194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 0x562850fe2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe3b40 0x562850fe2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a37fe640 1 -- 172.21.15.86:0/314174693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc0940cf580 con 0x7fc0ac1037a0 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 -- 172.21.15.86:0/797390194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe3b40 msgr2=0x562850fe2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe3b40 0x562850fe2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 -- 172.21.15.86:0/797390194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 msgr2=0x562850ff3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 -- 172.21.15.86:0/797390194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562850ed43a0 con 0x562850fe25d0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 0x7fc0980a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009430 tx=0x7fc09c009800 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.336+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 0x562850fe2a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400bbf0 tx=0x7fc0a40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0008f8ea9a33302 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0a0ff9640 1 -- 172.21.15.86:0/797390194 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4004030 con 0x562850fe25d0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0a0ff9640 1 -- 172.21.15.86:0/797390194 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0a40035f0 con 0x562850fe25d0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0a0ff9640 1 -- 172.21.15.86:0/797390194 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40116a0 con 0x562850fe25d0 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 -- 172.21.15.86:0/797390194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 msgr2=0x562850fe2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 0x562850fe2a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400bbf0 tx=0x7fc0a40107a0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 -- 172.21.15.86:0/797390194 shutdown_connections 2024-09-05T06:54:16.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850ff1780 0x562850ff3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe25d0 0x562850fe2a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/797390194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe3b40 0x562850fe2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.337+0000 7fc0b31b6740 1 -- 172.21.15.86:0/797390194 >> 172.21.15.86:0/797390194 conn(0x562850fab310 msgr2=0x562850fdec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- 172.21.15.86:0/797390194 shutdown_connections 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- 172.21.15.86:0/797390194 wait complete. 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- start start 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 Processor -- start 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850ff1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562850ff4480 con 0x562850fe3b40 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562850f03060 con 0x562850ff1780 2024-09-05T06:54:16.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562850ff4600 con 0x562850fe25d0 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0b31b6740 10 client.0 _ll_register_callbacks cb 0x562850c568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36380/0 (socket says 172.21.15.86:36380) 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/350668610 learned_addr learned my addr 172.21.15.86:0/350668610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850ff1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.338+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/350668610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 msgr2=0x562850ff1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850ff1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 msgr2=0x562850fee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/350668610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562850cb93c0 con 0x562850fe3b40 2024-09-05T06:54:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850ff1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc094005940 tx=0x7fc094098de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0b7e625d004493 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09401d040 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc0940055d0 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a40035f0 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0940046c0 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 10 client.18534 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562850ffff60 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0a40035f0 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc094097cd0 con 0x562850fe3b40 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 10 client.18534 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 20 client.18534 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5628510013c0 0x562851003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 4 client.18534 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.339+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562850f55860 con 0x5628510013c0 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.340+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5628510013c0 0x562851003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.340+0000 7fc091ffb640 10 client.18534 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.340+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5628510013c0 0x562851003800 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921d39b6b0fd73ac server_cookie=eccaaf0baa2d40d9 in_seq=0 out_seq=0 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc091ffb640 1 -- 172.21.15.86:0/350668610 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562850f55860 con 0x5628510013c0 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc091ffb640 10 client.18534 handle_command_reply: tid=0 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 client.18534 shutdown 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 msgr2=0x562850fee410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc094005940 tx=0x7fc094098de0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 shutdown_connections 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5628510013c0 0x562851003800 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850fee410 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc091ffb640 0 client.18534 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/350668610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850ff1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 >> 172.21.15.86:0/350668610 conn(0x562850fab310 msgr2=0x562850fe4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 shutdown_connections 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 1 -- 172.21.15.86:0/350668610 wait complete. 2024-09-05T06:54:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.341+0000 7fc0b31b6740 20 client.18534 trim_cache size 0 max 16384 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 Processor -- start 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 -- start start 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850fdedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fedcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1acf90 con 0x562850fe3b40 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1aeb30 con 0x562850ff1780 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ad660 con 0x562850fe25d0 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0a9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36382/0 (socket says 172.21.15.86:36382) 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.354+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/3288922332 learned_addr learned my addr 172.21.15.86:0/3288922332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850fdedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/3288922332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 msgr2=0x562850fdedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850fdedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fedcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/3288922332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 msgr2=0x562850fedcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fedcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 -- 172.21.15.86:0/3288922332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562850fe5120 con 0x562850fe3b40 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850fdedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc09409b5f0 tx=0x7fc0940971e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79de9a8e239cf63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc093fff640 1 -- 172.21.15.86:0/3288922332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09401d070 con 0x562850fe3b40 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc093fff640 1 -- 172.21.15.86:0/3288922332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc09409bd60 con 0x562850fe3b40 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.355+0000 7fc093fff640 1 -- 172.21.15.86:0/3288922332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094097cd0 con 0x562850fe3b40 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- 172.21.15.86:0/3288922332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 msgr2=0x562850ffd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc09409b5f0 tx=0x7fc0940971e0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- 172.21.15.86:0/3288922332 shutdown_connections 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850ff1780 0x562850fedcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe3b40 0x562850ffd720 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/3288922332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850fe25d0 0x562850fdedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- 172.21.15.86:0/3288922332 >> 172.21.15.86:0/3288922332 conn(0x562850fab310 msgr2=0x562850fde6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- 172.21.15.86:0/3288922332 shutdown_connections 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- 172.21.15.86:0/3288922332 wait complete. 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- start start 2024-09-05T06:54:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 Processor -- start 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 0x562850ff53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850ff1780 0x562850ff5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ae430 con 0x562850fe25d0 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1add30 con 0x562850fe3b40 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0b31b6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1aaf90 con 0x562850ff1780 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.356+0000 7fc0aa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0b31b6740 10 client.0 _ll_register_callbacks cb 0x7fc0940cc830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36396/0 (socket says 172.21.15.86:36396) 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 -- 172.21.15.86:0/2316026915 learned_addr learned my addr 172.21.15.86:0/2316026915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aad9d640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850ff1780 0x562850ff5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 0x562850ff53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 -- 172.21.15.86:0/2316026915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850ff1780 msgr2=0x562850ff5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850ff1780 0x562850ff5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 -- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 msgr2=0x562850ff53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 0x562850ff53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 -- 172.21.15.86:0/2316026915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562850ff6170 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 0x562850ff53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0aa59c640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4000c00 tx=0x7fc0a400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c57b50a93005b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4005040 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc0a40051e0 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09409bd60 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0a4038050 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0b31b6740 10 client.18540 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56285106c140 con 0x562850fe25d0 2024-09-05T06:54:16.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.357+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc09409bd60 con 0x562850fe25d0 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc0a4011c50 con 0x562850fe25d0 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 10 client.18540 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 20 client.18540 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56285106c610 0x56285106ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 4 client.18540 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562850f4fd70 con 0x56285106c610 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56285106c610 0x56285106ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc0a9d9b640 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56285106c610 0x56285106ea30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3443a99bbb88397 server_cookie=9d809e3a58281983 in_seq=0 out_seq=0 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.358+0000 7fc091ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc091ffb640 1 -- 172.21.15.86:0/2316026915 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562850f4fd70 con 0x56285106c610 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc091ffb640 10 client.18540 handle_command_reply: tid=0 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 client.18540 shutdown 2024-09-05T06:54:16.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 msgr2=0x562850ffa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4000c00 tx=0x7fc0a400bf00 comp rx=0 tx=0).stop 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 shutdown_connections 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56285106c610 0x56285106ea30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562850ff1780 0x562850ff5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562850fe3b40 0x562850ff53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 --2- 172.21.15.86:0/2316026915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562850fe25d0 0x562850ffa090 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 >> 172.21.15.86:0/2316026915 conn(0x562850fab310 msgr2=0x562850fe4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc091ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 shutdown_connections 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 1 -- 172.21.15.86:0/2316026915 wait complete. 2024-09-05T06:54:16.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b31b6740 20 client.18540 trim_cache size 0 max 16384 2024-09-05T06:54:16.374 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 msgr2=0x7fc0980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 0x7fc0980a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009430 tx=0x7fc09c009800 comp rx=0 tx=0).stop 2024-09-05T06:54:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 msgr2=0x7fc0ac198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.359+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c970 tx=0x7fc09400ce30 comp rx=0 tx=0).stop 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 shutdown_connections 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0980a2cf0 0x7fc0980a51b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac10c2f0 0x7fc0ac19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ac1037a0 0x7fc0ac198b00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 --2- 172.21.15.86:0/314174693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac102df0 0x7fc0ac1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 >> 172.21.15.86:0/314174693 conn(0x7fc0ac0fc7d0 msgr2=0x7fc0ac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 shutdown_connections 2024-09-05T06:54:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:16.360+0000 7fc0b0ddd640 1 -- 172.21.15.86:0/314174693 wait complete. 2024-09-05T06:54:16.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.55 MB/sec execute 576 sec latency 12566.787 ms 2024-09-05T06:54:17.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:17 smithi096 ceph-mon[34468]: pgmap v407: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 622 KiB/s wr, 80 op/s 2024-09-05T06:54:17.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:17 smithi115 ceph-mon[34791]: pgmap v407: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 622 KiB/s wr, 80 op/s 2024-09-05T06:54:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:17 smithi086 ceph-mon[30943]: pgmap v407: 129 pgs: 129 active+clean; 263 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 622 KiB/s wr, 80 op/s 2024-09-05T06:54:17.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.55 MB/sec execute 577 sec latency 13566.883 ms 2024-09-05T06:54:18.040 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:54:18.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:54:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:18.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.55 MB/sec execute 578 sec latency 14566.924 ms 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 -- 172.21.15.86:0/3999368335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f9810e3c0 msgr2=0x7f5f98110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 --2- 172.21.15.86:0/3999368335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f9810e3c0 0x7f5f98110810 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009ca0 tx=0x7f5f9002a950 comp rx=0 tx=0).stop 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 -- 172.21.15.86:0/3999368335 shutdown_connections 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 --2- 172.21.15.86:0/3999368335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f9810e3c0 0x7f5f98110810 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 --2- 172.21.15.86:0/3999368335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f98074f40 0x7f5f9810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 --2- 172.21.15.86:0/3999368335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074620 0x7f5f98074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 -- 172.21.15.86:0/3999368335 >> 172.21.15.86:0/3999368335 conn(0x7f5f980fd8c0 msgr2=0x7f5f980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 -- 172.21.15.86:0/3999368335 shutdown_connections 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.754+0000 7f5fa003a640 1 -- 172.21.15.86:0/3999368335 wait complete. 2024-09-05T06:54:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 Processor -- start 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 -- start start 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 0x7f5f9819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f9810e3c0 0x7f5f9819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98114060 con 0x7f5f98074620 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113ee0 con 0x7f5f98074f40 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5fa003a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981141e0 con 0x7f5f9810e3c0 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5f9ddd6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5f9ddd6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36406/0 (socket says 172.21.15.86:36406) 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.755+0000 7f5f9ddd6640 1 -- 172.21.15.86:0/21923129 learned_addr learned my addr 172.21.15.86:0/21923129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 0x7f5f9819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 -- 172.21.15.86:0/21923129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f9810e3c0 msgr2=0x7f5f9819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f9810e3c0 0x7f5f9819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 -- 172.21.15.86:0/21923129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 msgr2=0x7f5f9819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 0x7f5f9819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 -- 172.21.15.86:0/21923129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9819f520 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 0x7f5f9819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c890 tx=0x7f5f9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f8effd640 1 -- 172.21.15.86:0/21923129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004220 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9819f810 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f8effd640 1 -- 172.21.15.86:0/21923129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f940043c0 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5f8effd640 1 -- 172.21.15.86:0/21923129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004a00 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.756+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f981a70f0 con 0x7f5f98074620 2024-09-05T06:54:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.758+0000 7f5f8effd640 1 -- 172.21.15.86:0/21923129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f94004ba0 con 0x7f5f98074620 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.758+0000 7f5fa2413740 1 Processor -- start 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 -- start start 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb135f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb136480 0x557cdb140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cdb09b280 con 0x557cdb140d40 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557cdb042b10 con 0x557cdb135b60 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5fa2413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557cdb097240 con 0x557cdb136480 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36420/0 (socket says 172.21.15.86:36420) 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/2945033841 learned_addr learned my addr 172.21.15.86:0/2945033841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f8effd640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 0x7f5f840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb136480 0x557cdb140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 0x7f5f840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/2945033841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb136480 msgr2=0x557cdb140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f8effd640 1 -- 172.21.15.86:0/21923129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f5f940cf660 con 0x7f5f98074620 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb136480 0x557cdb140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/2945033841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 msgr2=0x557cdb135f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.759+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb135f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/2945033841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cdb04f1f0 con 0x557cdb140d40 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800a9d0 tx=0x7f5f8800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1bd1bcb260d2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 0x7f5f840a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009430 tx=0x7f5f900095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f83fff640 1 -- 172.21.15.86:0/2945033841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88015070 con 0x557cdb140d40 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f83fff640 1 -- 172.21.15.86:0/2945033841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8800cc60 con 0x557cdb140d40 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.760+0000 7f5f83fff640 1 -- 172.21.15.86:0/2945033841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f880185c0 con 0x557cdb140d40 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- 172.21.15.86:0/2945033841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 msgr2=0x557cdb143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800a9d0 tx=0x7f5f8800ae90 comp rx=0 tx=0).stop 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- 172.21.15.86:0/2945033841 shutdown_connections 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb143130 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb136480 0x557cdb140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 --2- 172.21.15.86:0/2945033841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb135f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- 172.21.15.86:0/2945033841 >> 172.21.15.86:0/2945033841 conn(0x557cdb1320f0 msgr2=0x557cdb1324d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- 172.21.15.86:0/2945033841 shutdown_connections 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- 172.21.15.86:0/2945033841 wait complete. 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 -- start start 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.761+0000 7f5fa2413740 1 Processor -- start 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb13db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 0x557cdb13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cdb143910 con 0x557cdb136480 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557cdb055f10 con 0x557cdb140d40 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557cdb143a90 con 0x557cdb135b60 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5fa2413740 10 client.0 _ll_register_callbacks cb 0x557cdb01ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36432/0 (socket says 172.21.15.86:36432) 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/1558228631 learned_addr learned my addr 172.21.15.86:0/1558228631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb13db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 0x557cdb13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/1558228631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 msgr2=0x557cdb13db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb13db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 msgr2=0x557cdb13e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 0x557cdb13e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/1558228631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cdae0c6a0 con 0x557cdb136480 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 0x557cdb13e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5f88004700 tx=0x7f5f88006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc540b0371f1ce79 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.762+0000 7f5f817fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88020070 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f5f8801a650 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8800cc60 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5f88015040 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 10 client.18552 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557cdb14f460 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8800cc60 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5f88024750 con 0x557cdb136480 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 10 client.18552 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 20 client.18552 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:18.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb150810 0x557cdb152c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 4 client.18552 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:18.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.763+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cdb0a87f0 con 0x557cdb150810 2024-09-05T06:54:18.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.764+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb150810 0x557cdb152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.764+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb150810 0x557cdb152c00 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ff7ed01bc06372 server_cookie=c1ac43c9227eca40 in_seq=0 out_seq=0 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.764+0000 7f5f817fa640 10 client.18552 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5f817fa640 1 -- 172.21.15.86:0/1558228631 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cdb0a87f0 con 0x557cdb150810 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5f817fa640 10 client.18552 handle_command_reply: tid=0 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 client.18552 shutdown 2024-09-05T06:54:18.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 msgr2=0x557cdb13e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5f88004700 tx=0x7f5f88006c90 comp rx=0 tx=0).stop 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 shutdown_connections 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb150810 0x557cdb152c00 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb140d40 0x557cdb13e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb13e0b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 --2- 172.21.15.86:0/1558228631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb13db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5f817fa640 0 client.18552 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 >> 172.21.15.86:0/1558228631 conn(0x557cdb1320f0 msgr2=0x557cdb137dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 shutdown_connections 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 1 -- 172.21.15.86:0/1558228631 wait complete. 2024-09-05T06:54:18.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.765+0000 7f5fa2413740 20 client.18552 trim_cache size 0 max 16384 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 Processor -- start 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 -- start start 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb137cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 0x557cdb158770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981b1450 con 0x557cdb136480 2024-09-05T06:54:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981b2ff0 con 0x557cdb135b60 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5fa2413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981b1b20 con 0x557cdb140d40 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5f9d5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36436/0 (socket says 172.21.15.86:36436) 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.782+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/3153841736 learned_addr learned my addr 172.21.15.86:0/3153841736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb137cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/3153841736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 msgr2=0x557cdb158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 0x557cdb158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 0x557cdb158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/3153841736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 msgr2=0x557cdb137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb137cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 -- 172.21.15.86:0/3153841736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cdb13cb80 con 0x557cdb136480 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 0x557cdb158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f8800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a210ec1c81b9892b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f837fe640 1 -- 172.21.15.86:0/3153841736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88020040 con 0x557cdb136480 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f837fe640 1 -- 172.21.15.86:0/3153841736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8800c520 con 0x557cdb136480 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.783+0000 7f5f837fe640 1 -- 172.21.15.86:0/3153841736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88024640 con 0x557cdb136480 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- 172.21.15.86:0/3153841736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 msgr2=0x557cdb158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f8800ae90 comp rx=0 tx=0).stop 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- 172.21.15.86:0/3153841736 shutdown_connections 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb140d40 0x557cdb158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb136480 0x557cdb158230 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3153841736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb135b60 0x557cdb137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- 172.21.15.86:0/3153841736 >> 172.21.15.86:0/3153841736 conn(0x557cdb1320f0 msgr2=0x557cdb1375b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- 172.21.15.86:0/3153841736 shutdown_connections 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- 172.21.15.86:0/3153841736 wait complete. 2024-09-05T06:54:18.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 -- start start 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 Processor -- start 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb136480 0x557cdb149400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.784+0000 7f5fa2413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5fa2413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981b28f0 con 0x557cdb140d40 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5fa2413740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981b21f0 con 0x557cdb136480 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5fa2413740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981af450 con 0x557cdb135b60 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5fa2413740 10 client.0 _ll_register_callbacks cb 0x7f5f8800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36446/0 (socket says 172.21.15.86:36446) 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 -- 172.21.15.86:0/3754538213 learned_addr learned my addr 172.21.15.86:0/3754538213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9d5d5640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb136480 0x557cdb149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 -- 172.21.15.86:0/3754538213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 msgr2=0x557cdb148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 -- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb136480 msgr2=0x557cdb149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb136480 0x557cdb149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 -- 172.21.15.86:0/3754538213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cdb1362b0 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f9e5d7640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5f9002f770 tx=0x7f5f90009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b44f69b6c4f82a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f817fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90041070 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f5f90004500 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8800c520 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.785+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5f90034040 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 10 client.18558 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557cdb1bbac0 con 0x557cdb140d40 2024-09-05T06:54:18.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8800c520 con 0x557cdb140d40 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5f90051020 con 0x557cdb140d40 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 10 client.18558 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 10 client.18558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 20 client.18558 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb1bc2a0 0x557cdb1be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 4 client.18558 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557cdb0a2d00 con 0x557cdb1bc2a0 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.786+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb1bc2a0 0x557cdb1be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.787+0000 7f5f9ddd6640 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb1bc2a0 0x557cdb1be6c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9316246ad93369e server_cookie=332cfe48d12193e5 in_seq=0 out_seq=0 2024-09-05T06:54:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.787+0000 7f5f817fa640 10 client.18558 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5f817fa640 1 -- 172.21.15.86:0/3754538213 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557cdb0a2d00 con 0x557cdb1bc2a0 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5f817fa640 10 client.18558 handle_command_reply: tid=0 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 client.18558 shutdown 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 msgr2=0x557cdb145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5f9002f770 tx=0x7f5f90009800 comp rx=0 tx=0).stop 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 shutdown_connections 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557cdb1bc2a0 0x557cdb1be6c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cdb140d40 0x557cdb145cb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557cdb136480 0x557cdb149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5f817fa640 0 client.18558 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 --2- 172.21.15.86:0/3754538213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557cdb135b60 0x557cdb148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 >> 172.21.15.86:0/3754538213 conn(0x557cdb1320f0 msgr2=0x557cdb131270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 shutdown_connections 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 1 -- 172.21.15.86:0/3754538213 wait complete. 2024-09-05T06:54:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.788+0000 7f5fa2413740 20 client.18558 trim_cache size 0 max 16384 2024-09-05T06:54:18.795 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:18.795 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:54:18.795 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "99f98ccd-ac9b-474d-a173-f23488918247", 2024-09-05T06:54:18.795 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:54:18.795 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 msgr2=0x7f5f840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 0x7f5f840a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009430 tx=0x7f5f900095c0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 msgr2=0x7f5f9819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c890 tx=0x7f5f9400cd50 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 shutdown_connections 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f840a2cf0 0x7f5f840a51b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f9810e3c0 0x7f5f9819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f98074f40 0x7f5f9819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 --2- 172.21.15.86:0/21923129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f98074620 0x7f5f9819a570 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 >> 172.21.15.86:0/21923129 conn(0x7f5f980fd8c0 msgr2=0x7f5f980ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 shutdown_connections 2024-09-05T06:54:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:18.789+0000 7f5fa003a640 1 -- 172.21.15.86:0/21923129 wait complete. 2024-09-05T06:54:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:19 smithi096 ceph-mon[34468]: pgmap v408: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:19 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:54:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:19 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:19 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:54:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:19 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:19 smithi086 ceph-mon[30943]: pgmap v408: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:19.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:19 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:54:19.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:19 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.430 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:19 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:54:19.430 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:19 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:19 smithi115 ceph-mon[34791]: pgmap v408: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:19 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:54:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:19 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:19 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:54:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:19 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:19.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:54:19.538 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.54 MB/sec execute 579 sec latency 15566.951 ms 2024-09-05T06:54:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.205 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:20 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/2389698737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b010c170 msgr2=0x7ff6b010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/2389698737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b010c170 0x7ff6b010e610 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009ca0 tx=0x7ff6a802a950 comp rx=0 tx=0).stop 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/2389698737 shutdown_connections 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/2389698737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b010c170 0x7ff6b010e610 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/2389698737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b0103660 0x7ff6b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/2389698737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0102d40 0x7ff6b0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/2389698737 >> 172.21.15.86:0/2389698737 conn(0x7ff6b00fc7a0 msgr2=0x7ff6b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/2389698737 shutdown_connections 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/2389698737 wait complete. 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 Processor -- start 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.286+0000 7ff6b5f5b640 1 -- start start 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 0x7ff6b0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b010c170 0x7ff6b019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0111de0 con 0x7ff6b0102d40 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0111c60 con 0x7ff6b0103660 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0111f60 con 0x7ff6b010c170 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36464/0 (socket says 172.21.15.86:36464) 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 -- 172.21.15.86:0/3406589905 learned_addr learned my addr 172.21.15.86:0/3406589905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6affff640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b010c170 0x7ff6b019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 -- 172.21.15.86:0/3406589905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b010c170 msgr2=0x7ff6b019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b010c170 0x7ff6b019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 -- 172.21.15.86:0/3406589905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 msgr2=0x7ff6b0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 0x7ff6b0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 0x7ff6b0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 -- 172.21.15.86:0/3406589905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b019d2a0 con 0x7ff6b0102d40 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 0x7ff6b0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c920 tx=0x7ff6a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6ad7fa640 1 -- 172.21.15.86:0/3406589905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0007c70 con 0x7ff6b0102d40 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b019d590 con 0x7ff6b0102d40 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6ad7fa640 1 -- 172.21.15.86:0/3406589905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6a0007e10 con 0x7ff6b0102d40 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6ad7fa640 1 -- 172.21.15.86:0/3406589905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0018820 con 0x7ff6b0102d40 2024-09-05T06:54:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.287+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b01a4e70 con 0x7ff6b0102d40 2024-09-05T06:54:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.290+0000 7ff6ad7fa640 1 -- 172.21.15.86:0/3406589905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff6a00189c0 con 0x7ff6b0102d40 2024-09-05T06:54:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.290+0000 7ff6b8334740 1 Processor -- start 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.290+0000 7ff6b8334740 1 -- start start 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bd40352e0 con 0x564bd40da680 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bd3fdcce0 con 0x564bd40cfdc0 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6b8334740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564bd40312a0 con 0x564bd40cf4a0 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36478/0 (socket says 172.21.15.86:36478) 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2004280529 learned_addr learned my addr 172.21.15.86:0/2004280529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6affff640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6ad7fa640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 0x7ff69c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2004280529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 msgr2=0x564bd40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2004280529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 msgr2=0x564bd40da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2004280529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd3fc1440 con 0x564bd40da680 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6affff640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 0x7ff69c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6ad7fa640 1 -- 172.21.15.86:0/3406589905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7ff6a00cf9c0 con 0x7ff6b0102d40 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.291+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff698009c10 tx=0x7ff69800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d13100068b3a397 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6a5ffb640 1 -- 172.21.15.86:0/2004280529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698016070 con 0x564bd40da680 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6a5ffb640 1 -- 172.21.15.86:0/2004280529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff698011920 con 0x564bd40da680 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6a5ffb640 1 -- 172.21.15.86:0/2004280529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980107e0 con 0x564bd40da680 2024-09-05T06:54:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 -- 172.21.15.86:0/2004280529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 msgr2=0x564bd40dca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6affff640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 0x7ff69c0a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005fe0 tx=0x7ff6a803c040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff698009c10 tx=0x7ff69800e6c0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 -- 172.21.15.86:0/2004280529 shutdown_connections 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40dca70 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2004280529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.292+0000 7ff6b8334740 1 -- 172.21.15.86:0/2004280529 >> 172.21.15.86:0/2004280529 conn(0x564bd40cba30 msgr2=0x564bd40cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- 172.21.15.86:0/2004280529 shutdown_connections 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- 172.21.15.86:0/2004280529 wait complete. 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- start start 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 Processor -- start 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40da680 0x564bd40d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bd40dd250 con 0x564bd40cf4a0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bd3ff0050 con 0x564bd40cfdc0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564bd40dd3d0 con 0x564bd40da680 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6b8334740 10 client.0 _ll_register_callbacks cb 0x564bd4030160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36480/0 (socket says 172.21.15.86:36480) 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.293+0000 7ff6af7fe640 1 -- 172.21.15.86:0/4242546777 learned_addr learned my addr 172.21.15.86:0/4242546777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6affff640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40da680 0x564bd40d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 -- 172.21.15.86:0/4242546777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40da680 msgr2=0x564bd40d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40da680 0x564bd40d7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 -- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 msgr2=0x564bd40d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 -- 172.21.15.86:0/4242546777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd3da63c0 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0007590 tx=0x7ff6a009b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e952acb0c22d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff68effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0018cb0 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7ff6a00a4480 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff698011920 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6a000fb00 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6b8334740 10 client.18570 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bd40e8c20 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.294+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff698011920 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7ff6a00977f0 con 0x564bd40cf4a0 2024-09-05T06:54:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 10 client.18570 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 20 client.18570 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd40ea0e0 0x564bd40ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 4 client.18570 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bd4042850 con 0x564bd40ea0e0 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd40ea0e0 0x564bd40ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff68effd640 10 client.18570 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.295+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd40ea0e0 0x564bd40ec530 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432202fecf5e6486 server_cookie=498e72de27f62038 in_seq=0 out_seq=0 2024-09-05T06:54:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff68effd640 1 -- 172.21.15.86:0/4242546777 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bd4042850 con 0x564bd40ea0e0 2024-09-05T06:54:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff68effd640 10 client.18570 handle_command_reply: tid=0 2024-09-05T06:54:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 client.18570 shutdown 2024-09-05T06:54:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 msgr2=0x564bd40d9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0007590 tx=0x7ff6a009b870 comp rx=0 tx=0).stop 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 shutdown_connections 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd40ea0e0 0x564bd40ec530 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40da680 0x564bd40d7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cfdc0 0x564bd40d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 --2- 172.21.15.86:0/4242546777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cf4a0 0x564bd40d9fa0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 >> 172.21.15.86:0/4242546777 conn(0x564bd40cba30 msgr2=0x564bd40d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff68effd640 0 client.18570 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 shutdown_connections 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.296+0000 7ff6b8334740 1 -- 172.21.15.86:0/4242546777 wait complete. 2024-09-05T06:54:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.297+0000 7ff6b8334740 20 client.18570 trim_cache size 0 max 16384 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:20 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.313+0000 7ff6b8334740 1 Processor -- start 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.313+0000 7ff6b8334740 1 -- start start 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 0x564bd40e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cfdc0 0x564bd40d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01af090 con 0x564bd40da680 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01b0c30 con 0x564bd40cf4a0 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6b8334740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01af760 con 0x564bd40cfdc0 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36482/0 (socket says 172.21.15.86:36482) 2024-09-05T06:54:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 -- 172.21.15.86:0/730495548 learned_addr learned my addr 172.21.15.86:0/730495548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cfdc0 0x564bd40d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 0x564bd40e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 -- 172.21.15.86:0/730495548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cfdc0 msgr2=0x564bd40d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cfdc0 0x564bd40d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 -- 172.21.15.86:0/730495548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 msgr2=0x564bd40e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 0x564bd40e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 -- 172.21.15.86:0/730495548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd40d6250 con 0x564bd40da680 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 0x564bd40e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.314+0000 7ff6affff640 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8035000 tx=0x7ff6a8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ae836cb3b5eb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6a57fa640 1 -- 172.21.15.86:0/730495548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8056070 con 0x564bd40da680 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6a57fa640 1 -- 172.21.15.86:0/730495548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6a8004050 con 0x564bd40da680 2024-09-05T06:54:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6a57fa640 1 -- 172.21.15.86:0/730495548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8034040 con 0x564bd40da680 2024-09-05T06:54:20.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- 172.21.15.86:0/730495548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 msgr2=0x564bd40f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8035000 tx=0x7ff6a8044000 comp rx=0 tx=0).stop 2024-09-05T06:54:20.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- 172.21.15.86:0/730495548 shutdown_connections 2024-09-05T06:54:20.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40da680 0x564bd40f1e00 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cfdc0 0x564bd40d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 --2- 172.21.15.86:0/730495548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40cf4a0 0x564bd40e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- 172.21.15.86:0/730495548 >> 172.21.15.86:0/730495548 conn(0x564bd40cba30 msgr2=0x564bd40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- 172.21.15.86:0/730495548 shutdown_connections 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- 172.21.15.86:0/730495548 wait complete. 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.315+0000 7ff6b8334740 1 -- start start 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 Processor -- start 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 0x564bd4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01b0530 con 0x564bd40cfdc0 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01afe30 con 0x564bd40da680 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b01ad090 con 0x564bd40cf4a0 2024-09-05T06:54:20.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6b8334740 10 client.0 _ll_register_callbacks cb 0x7ff6a000fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36490/0 (socket says 172.21.15.86:36490) 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2952672826 learned_addr learned my addr 172.21.15.86:0/2952672826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6affff640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 0x564bd4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2952672826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 msgr2=0x564bd40de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40de790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 msgr2=0x564bd4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 0x564bd4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6a6ffd640 1 -- 172.21.15.86:0/2952672826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd40cfbf0 con 0x564bd40cfdc0 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.316+0000 7ff6affff640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 0x564bd4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:20.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6a6ffd640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff698009be0 tx=0x7ff69800efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a746ce2795f886b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69802c070 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7ff698005450 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a8004050 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff698016040 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 10 client.18576 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bd40d7b70 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6a8004050 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7ff69800f040 con 0x564bd40cfdc0 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 10 client.18576 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 20 client.18576 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:20.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd41556b0 0x564bd4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 4 client.18576 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.317+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bd403cd60 con 0x564bd41556b0 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd41556b0 0x564bd4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff68effd640 10 client.18576 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6af7fe640 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd41556b0 0x564bd4157ad0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612042389537b9b4 server_cookie=64a243019f32d267 in_seq=0 out_seq=0 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff68effd640 1 -- 172.21.15.86:0/2952672826 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564bd403cd60 con 0x564bd41556b0 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff68effd640 10 client.18576 handle_command_reply: tid=0 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 client.18576 shutdown 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 msgr2=0x564bd40decd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff698009be0 tx=0x7ff69800efa0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 shutdown_connections 2024-09-05T06:54:20.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564bd41556b0 0x564bd4157ad0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bd40da680 0x564bd4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd40cfdc0 0x564bd40decd0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 --2- 172.21.15.86:0/2952672826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564bd40cf4a0 0x564bd40de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 >> 172.21.15.86:0/2952672826 conn(0x564bd40cba30 msgr2=0x564bd40d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.318+0000 7ff68effd640 0 client.18576 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 shutdown_connections 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b8334740 1 -- 172.21.15.86:0/2952672826 wait complete. 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b8334740 20 client.18576 trim_cache size 0 max 16384 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:20.325 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: "99f98ccd-ac9b-474d-a173-f23488918247": { 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "99f98ccd-ac9b-474d-a173-f23488918247", 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:20.326 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 msgr2=0x7ff69c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 0x7ff69c0a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005fe0 tx=0x7ff6a803c040 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 msgr2=0x7ff6b01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c920 tx=0x7ff6a000cde0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 shutdown_connections 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff69c0a2cf0 0x7ff69c0a51b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6b010c170 0x7ff6b019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b0103660 0x7ff6b0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 --2- 172.21.15.86:0/3406589905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b0102d40 0x7ff6b01982f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 >> 172.21.15.86:0/3406589905 conn(0x7ff6b00fc7a0 msgr2=0x7ff6b00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:20.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 shutdown_connections 2024-09-05T06:54:20.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:20.319+0000 7ff6b5f5b640 1 -- 172.21.15.86:0/3406589905 wait complete. 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:20 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:54:20.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.54 MB/sec execute 580 sec latency 16567.015 ms 2024-09-05T06:54:20.980 INFO:tasks.cephfs.filesystem:scrub status for tag:99f98ccd-ac9b-474d-a173-f23488918247 - {'path': '/', 'tag': '99f98ccd-ac9b-474d-a173-f23488918247', 'options': 'recursive,force'} 2024-09-05T06:54:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:21 smithi086 ceph-mon[30943]: pgmap v409: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:21.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:21 smithi096 ceph-mon[34468]: pgmap v409: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:21 smithi115 ceph-mon[34791]: pgmap v409: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2024-09-05T06:54:21.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.54 MB/sec execute 581 sec latency 17567.092 ms 2024-09-05T06:54:22.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.54 MB/sec execute 582 sec latency 18567.180 ms 2024-09-05T06:54:23.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:23 smithi096 ceph-mon[34468]: pgmap v410: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-05T06:54:23.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:23 smithi115 ceph-mon[34791]: pgmap v410: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-05T06:54:23.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:23 smithi086 ceph-mon[30943]: pgmap v410: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-05T06:54:23.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.53 MB/sec execute 583 sec latency 19567.231 ms 2024-09-05T06:54:24.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.53 MB/sec execute 584 sec latency 20567.317 ms 2024-09-05T06:54:25.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:25 smithi096 ceph-mon[34468]: pgmap v411: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-05T06:54:25.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:25 smithi115 ceph-mon[34791]: pgmap v411: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-05T06:54:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:25 smithi086 ceph-mon[30943]: pgmap v411: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-05T06:54:25.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.53 MB/sec execute 585 sec latency 21567.368 ms 2024-09-05T06:54:26.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:54:26.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:54:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:54:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:54:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:54:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:54:26.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.53 MB/sec execute 586 sec latency 22567.450 ms 2024-09-05T06:54:27.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:27 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:54:27.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:27 smithi096 ceph-mon[34468]: pgmap v412: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:54:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:27 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:54:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:27 smithi115 ceph-mon[34791]: pgmap v412: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:54:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:27 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:54:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:27 smithi086 ceph-mon[30943]: pgmap v412: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:54:27.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.52 MB/sec execute 587 sec latency 23567.484 ms 2024-09-05T06:54:27.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:54:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 -- 172.21.15.86:0/398753560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008102570 msgr2=0x7ff0081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 --2- 172.21.15.86:0/398753560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008102570 0x7ff0081029b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7feff0009900 tx=0x7feff002a480 comp rx=0 tx=0).stop 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 -- 172.21.15.86:0/398753560 shutdown_connections 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 --2- 172.21.15.86:0/398753560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 0x7ff00810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 --2- 172.21.15.86:0/398753560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008102570 0x7ff0081029b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 --2- 172.21.15.86:0/398753560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008100e20 0x7ff008101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 -- 172.21.15.86:0/398753560 >> 172.21.15.86:0/398753560 conn(0x7ff0080fc7d0 msgr2=0x7ff0080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 -- 172.21.15.86:0/398753560 shutdown_connections 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.351+0000 7ff00cfad640 1 -- 172.21.15.86:0/398753560 wait complete. 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 Processor -- start 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 -- start start 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008102570 0x7ff008197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 0x7ff00819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff008110e00 con 0x7ff008100e20 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff008110c80 con 0x7ff008102ef0 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00cfad640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff008110f80 con 0x7ff008102570 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40748/0 (socket says 172.21.15.86:40748) 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 -- 172.21.15.86:0/1082188289 learned_addr learned my addr 172.21.15.86:0/1082188289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff006d9d640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 0x7ff00819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7feffffff640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008102570 0x7ff008197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 -- 172.21.15.86:0/1082188289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008102570 msgr2=0x7ff008197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008102570 0x7ff008197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 -- 172.21.15.86:0/1082188289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 msgr2=0x7ff00819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 0x7ff00819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 -- 172.21.15.86:0/1082188289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00819c2b0 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.352+0000 7ff00659c640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7feff800c920 tx=0x7feff800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.353+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8007c70 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.353+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feff8007e10 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.353+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00819c5a0 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.353+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8018820 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.353+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a3e80 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.354+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008101200 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.355+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feff80189c0 con 0x7ff008100e20 2024-09-05T06:54:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.355+0000 7fefff7fe640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 0x7feff40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:28.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.356+0000 7feffffff640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 0x7feff40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:28.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.356+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7feff80cf960 con 0x7ff008100e20 2024-09-05T06:54:28.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.356+0000 7feffffff640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 0x7feff40a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ffe30 tx=0x7feff00095f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:28.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.360+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feff8014030 con 0x7ff008100e20 2024-09-05T06:54:28.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.52 MB/sec execute 588 sec latency 24567.552 ms 2024-09-05T06:54:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.558+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff008109090 con 0x7ff008100e20 2024-09-05T06:54:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.559+0000 7fefff7fe640 1 -- 172.21.15.86:0/1082188289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11976 (secure 0 0 0) 0x7feff809c140 con 0x7ff008100e20 2024-09-05T06:54:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:54:28.560 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"btime":"2024-09-05T06:53:11:090172+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:53:10.401210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:54:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-09-05T06:54:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 msgr2=0x7feff40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 0x7feff40a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ffe30 tx=0x7feff00095f0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 msgr2=0x7ff008197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7feff800c920 tx=0x7feff800cde0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 shutdown_connections 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feff40a2d70 0x7feff40a5230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff008102ef0 0x7ff00819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff008102570 0x7ff008197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 --2- 172.21.15.86:0/1082188289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff008100e20 0x7ff008197300 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 >> 172.21.15.86:0/1082188289 conn(0x7ff0080fc7d0 msgr2=0x7ff0080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 shutdown_connections 2024-09-05T06:54:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:28.561+0000 7ff00cfad640 1 -- 172.21.15.86:0/1082188289 wait complete. 2024-09-05T06:54:29.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:29 smithi086 ceph-mon[30943]: pgmap v413: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-05T06:54:29.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1082188289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:29.382 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id b43bdd40 --release --await' 2024-09-05T06:54:29.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id b43bdd40 --release --await 2024-09-05T06:54:29.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:29 smithi115 ceph-mon[34791]: pgmap v413: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-05T06:54:29.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1082188289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:29.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220934 1.52 MB/sec execute 589 sec latency 25567.585 ms 2024-09-05T06:54:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:29 smithi096 ceph-mon[34468]: pgmap v413: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-05T06:54:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1082188289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 -- 172.21.15.86:0/4278856915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 msgr2=0x7f490c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 --2- 172.21.15.86:0/4278856915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c100ee0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009930 tx=0x7f48f402a420 comp rx=0 tx=0).stop 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 -- 172.21.15.86:0/4278856915 shutdown_connections 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 --2- 172.21.15.86:0/4278856915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c101420 0x7f490c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 --2- 172.21.15.86:0/4278856915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c100ee0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 --2- 172.21.15.86:0/4278856915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c106a80 0x7f490c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 -- 172.21.15.86:0/4278856915 >> 172.21.15.86:0/4278856915 conn(0x7f490c0fc7a0 msgr2=0x7f490c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 -- 172.21.15.86:0/4278856915 shutdown_connections 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 -- 172.21.15.86:0/4278856915 wait complete. 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.118+0000 7f49108ed640 1 Processor -- start 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 -- start start 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c101420 0x7f490c0fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c106a80 0x7f490c0fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110d30 con 0x7f490c100a80 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110bb0 con 0x7f490c101420 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49108ed640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110eb0 con 0x7f490c106a80 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40776/0 (socket says 172.21.15.86:40776) 2024-09-05T06:54:30.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 -- 172.21.15.86:0/1231286303 learned_addr learned my addr 172.21.15.86:0/1231286303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f4903fff640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c101420 0x7f490c0fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490ad9d640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c106a80 0x7f490c0fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 -- 172.21.15.86:0/1231286303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c106a80 msgr2=0x7f490c0fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c106a80 0x7f490c0fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 -- 172.21.15.86:0/1231286303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c101420 msgr2=0x7f490c0fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c101420 0x7f490c0fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 -- 172.21.15.86:0/1231286303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a67e0 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f490a59c640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d950 tx=0x7f48fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.119+0000 7f49037fe640 1 -- 172.21.15.86:0/1231286303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc017070 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.120+0000 7f49037fe640 1 -- 172.21.15.86:0/1231286303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48fc009c30 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.120+0000 7f49037fe640 1 -- 172.21.15.86:0/1231286303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00f620 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.120+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a6ad0 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.120+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a7030 con 0x7f490c100a80 2024-09-05T06:54:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.122+0000 7f49037fe640 1 -- 172.21.15.86:0/1231286303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f48fc0109a0 con 0x7f490c100a80 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 Processor -- start 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 -- start start 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7041d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f7044630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631f6f9c620 con 0x5631f7037060 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631f6f43e50 con 0x5631f7037980 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f4912cc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631f6f985e0 con 0x5631f7042240 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f49037fe640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 0x7f48f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40784/0 (socket says 172.21.15.86:40784) 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.123+0000 7f490ad9d640 1 -- 172.21.15.86:0/494902901 learned_addr learned my addr 172.21.15.86:0/494902901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 0x7f48f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f4903fff640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f7044630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f49037fe640 1 -- 172.21.15.86:0/1231286303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f48fc0cf3c0 con 0x7f490c100a80 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490a59c640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 -- 172.21.15.86:0/494902901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 msgr2=0x5631f7044630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f7044630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 -- 172.21.15.86:0/494902901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 msgr2=0x5631f7041d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7041d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 -- 172.21.15.86:0/494902901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f6f34d40 con 0x5631f7037060 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490a59c640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7041d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f49040138c0 tx=0x7f4904013d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470602f39b40c456 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f4900ff9640 1 -- 172.21.15.86:0/494902901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904013070 con 0x5631f7037060 2024-09-05T06:54:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f490ad9d640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 0x7f48f80a5230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4904020000 tx=0x7f490401f000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f4900ff9640 1 -- 172.21.15.86:0/494902901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4904009e40 con 0x5631f7037060 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.124+0000 7f4900ff9640 1 -- 172.21.15.86:0/494902901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004680 con 0x5631f7037060 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- 172.21.15.86:0/494902901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 msgr2=0x5631f7037440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f49040138c0 tx=0x7f4904013d80 comp rx=0 tx=0).stop 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- 172.21.15.86:0/494902901 shutdown_connections 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f7044630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7041d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 --2- 172.21.15.86:0/494902901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7037440 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- 172.21.15.86:0/494902901 >> 172.21.15.86:0/494902901 conn(0x5631f70335f0 msgr2=0x5631f70339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- 172.21.15.86:0/494902901 shutdown_connections 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- 172.21.15.86:0/494902901 wait complete. 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 -- start start 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.125+0000 7f4912cc6740 1 Processor -- start 2024-09-05T06:54:30.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 0x5631f703f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f703fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631f7044e10 con 0x5631f7037980 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631f6f24f10 con 0x5631f7037060 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631f7044f90 con 0x5631f7042240 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 10 client.0 _ll_register_callbacks cb 0x5631f6f48db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40800/0 (socket says 172.21.15.86:40800) 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 -- 172.21.15.86:0/3503150997 learned_addr learned my addr 172.21.15.86:0/3503150997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f490ad9d640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f703fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f490a59c640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 0x5631f703f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 -- 172.21.15.86:0/3503150997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 msgr2=0x5631f703fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f703fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 -- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 msgr2=0x5631f703f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 0x5631f703f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 -- 172.21.15.86:0/3503150997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f6cab580 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f490ad9d640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f703fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f490a59c640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 0x5631f703f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4903fff640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f48f402a940 tx=0x7f48f4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1f99eba0e80bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f48f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4035040 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f48f4031860 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631f6fa9b90 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f48f4004040 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.126+0000 7f4912cc6740 10 client.18594 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5631f6d75960 con 0x5631f7037980 2024-09-05T06:54:30.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5631f6fa9b90 con 0x5631f7037980 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f48f4045470 con 0x5631f7037980 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 10 client.18594 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 10 client.18594 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 20 client.18594 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f7051d80 0x5631f70541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 4 client.18594 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631f6fa40a0 con 0x5631f7051d80 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.127+0000 7f490a59c640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f7051d80 0x5631f70541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f490a59c640 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f7051d80 0x5631f70541d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621f185dc676906b server_cookie=46779885919fb5f8 in_seq=0 out_seq=0 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f48f1ffb640 10 client.18594 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3503150997 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631f6fa40a0 con 0x5631f7051d80 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f48f1ffb640 10 client.18594 handle_command_reply: tid=0 2024-09-05T06:54:30.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f4912cc6740 1 client.18594 shutdown 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 msgr2=0x5631f703f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f48f402a940 tx=0x7f48f4009670 comp rx=0 tx=0).stop 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 shutdown_connections 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.128+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f7051d80 0x5631f70541d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f703fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037980 0x5631f703f6c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f48f1ffb640 0 client.18594 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3503150997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037060 0x5631f703f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 >> 172.21.15.86:0/3503150997 conn(0x5631f70335f0 msgr2=0x5631f7032750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 shutdown_connections 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 1 -- 172.21.15.86:0/3503150997 wait complete. 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.129+0000 7f4912cc6740 20 client.18594 trim_cache size 0 max 16384 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 Processor -- start 2024-09-05T06:54:30.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 -- start start 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 0x5631f7039b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f703a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1b1340 con 0x5631f7042240 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1b2ee0 con 0x5631f7037980 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.145+0000 7f4912cc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1b1a10 con 0x5631f7037060 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40816/0 (socket says 172.21.15.86:40816) 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 -- 172.21.15.86:0/493886155 learned_addr learned my addr 172.21.15.86:0/493886155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490a59c640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 0x5631f7039b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f4903fff640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f703a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 -- 172.21.15.86:0/493886155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 msgr2=0x5631f7039b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 0x5631f7039b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 -- 172.21.15.86:0/493886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 msgr2=0x5631f703a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f703a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 -- 172.21.15.86:0/493886155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631f70410c0 con 0x5631f7042240 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490a59c640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 0x5631f7039b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f4903fff640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f703a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f490ad9d640 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4904014010 tx=0x7f4904005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16857d1e519d1f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f48f3fff640 1 -- 172.21.15.86:0/493886155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904013050 con 0x5631f7042240 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f48f3fff640 1 -- 172.21.15.86:0/493886155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49040131f0 con 0x5631f7042240 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.146+0000 7f48f3fff640 1 -- 172.21.15.86:0/493886155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401d440 con 0x5631f7042240 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- 172.21.15.86:0/493886155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 msgr2=0x5631f7032640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4904014010 tx=0x7f4904005a10 comp rx=0 tx=0).stop 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- 172.21.15.86:0/493886155 shutdown_connections 2024-09-05T06:54:30.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7042240 0x5631f7032640 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f703a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- 172.21.15.86:0/493886155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7037060 0x5631f7039b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- 172.21.15.86:0/493886155 >> 172.21.15.86:0/493886155 conn(0x5631f70335f0 msgr2=0x5631f7031f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- 172.21.15.86:0/493886155 shutdown_connections 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- 172.21.15.86:0/493886155 wait complete. 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- start start 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 Processor -- start 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7046c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f70bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1b27e0 con 0x5631f7037060 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1b20e0 con 0x5631f7037980 2024-09-05T06:54:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1af340 con 0x5631f7042240 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.147+0000 7f4912cc6740 10 client.0 _ll_register_callbacks cb 0x7f48f4031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40822/0 (socket says 172.21.15.86:40822) 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 -- 172.21.15.86:0/3646355534 learned_addr learned my addr 172.21.15.86:0/3646355534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f4903fff640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7046c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490ad9d640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f70bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 -- 172.21.15.86:0/3646355534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 msgr2=0x5631f70bbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f70bbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 -- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 msgr2=0x5631f7046c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7046c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 -- 172.21.15.86:0/3646355534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631f70bc0a0 con 0x5631f7037060 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490ad9d640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f70bbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f490a59c640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0cccd0 tx=0x7f48fc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac8d979e55babc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f48f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc082030 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f48fc0cf120 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49040131f0 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f48fc020040 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.148+0000 7f4912cc6740 10 client.18600 fetch_fsmap learned FSMap version 36 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5631f70bd330 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49040131f0 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f48fc097b10 con 0x5631f7037060 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 10 client.18600 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 10 client.18600 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 20 client.18600 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f70bda30 0x5631f70bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 4 client.18600 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b43bdd40", "release": true, "await": true}] 2024-09-05T06:54:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b43bdd40", "release": true, "await": true}) -- 0x5631f6f205a0 con 0x5631f70bda30 2024-09-05T06:54:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4903fff640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f70bda30 0x5631f70bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f4903fff640 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f70bda30 0x5631f70bfe50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66bb86eea842be10 server_cookie=e3f6a55e9beee4c9 in_seq=0 out_seq=0 2024-09-05T06:54:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.149+0000 7f48f1ffb640 10 client.18600 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f48f1ffb640 1 -- 172.21.15.86:0/3646355534 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5631f6f205a0 con 0x5631f70bda30 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f48f1ffb640 10 client.18600 handle_command_reply: tid=0 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 client.18600 shutdown 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 msgr2=0x5631f7046740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0cccd0 tx=0x7f48fc005d30 comp rx=0 tx=0).stop 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 shutdown_connections 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631f70bda30 0x5631f70bfe50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f48f1ffb640 0 client.18600 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631f7042240 0x5631f70bbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631f7037980 0x5631f7046c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 --2- 172.21.15.86:0/3646355534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631f7037060 0x5631f7046740 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 >> 172.21.15.86:0/3646355534 conn(0x5631f70335f0 msgr2=0x5631f7032750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 shutdown_connections 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 1 -- 172.21.15.86:0/3646355534 wait complete. 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.155+0000 7f4912cc6740 20 client.18600 trim_cache size 0 max 16384 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 36, 2024-09-05T06:54:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 48, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "b43bdd40": { 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 48, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 146.4, 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:54:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:54:30.169 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 msgr2=0x7f48f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 0x7f48f80a5230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4904020000 tx=0x7f490401f000 comp rx=0 tx=0).stop 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 msgr2=0x7f490c0fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d950 tx=0x7f48fc00de10 comp rx=0 tx=0).stop 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 shutdown_connections 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48f80a2d70 0x7f48f80a5230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f490c106a80 0x7f490c0fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f490c101420 0x7f490c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 --2- 172.21.15.86:0/1231286303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490c100a80 0x7f490c0fced0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:30.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 >> 172.21.15.86:0/1231286303 conn(0x7f490c0fc7a0 msgr2=0x7f490c06ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:30.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 shutdown_connections 2024-09-05T06:54:30.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:30.156+0000 7f49108ed640 1 -- 172.21.15.86:0/1231286303 wait complete. 2024-09-05T06:54:30.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 221142 1.52 MB/sec execute 590 sec latency 26185.677 ms 2024-09-05T06:54:31.044 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b43bdd40', total seconds elapsed: 29.8 2024-09-05T06:54:31.358 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:31 smithi096 ceph-mon[34468]: pgmap v414: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-05T06:54:31.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:31 smithi115 ceph-mon[34791]: pgmap v414: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-05T06:54:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:31 smithi086 ceph-mon[30943]: pgmap v414: 129 pgs: 129 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-05T06:54:31.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 221700 1.52 MB/sec execute 591 sec latency 20.296 ms 2024-09-05T06:54:32.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 222351 1.52 MB/sec execute 592 sec latency 22.049 ms 2024-09-05T06:54:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:33 smithi115 ceph-mon[34791]: pgmap v415: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 202 KiB/s wr, 27 op/s 2024-09-05T06:54:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:33 smithi086 ceph-mon[30943]: pgmap v415: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 202 KiB/s wr, 27 op/s 2024-09-05T06:54:33.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 222987 1.53 MB/sec execute 593 sec latency 17.856 ms 2024-09-05T06:54:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:33 smithi096 ceph-mon[34468]: pgmap v415: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 202 KiB/s wr, 27 op/s 2024-09-05T06:54:34.539 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223582 1.53 MB/sec execute 594 sec latency 32.040 ms 2024-09-05T06:54:35.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:35 smithi115 ceph-mon[34791]: pgmap v416: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:35.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:35 smithi086 ceph-mon[30943]: pgmap v416: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:35.541 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223582 1.53 MB/sec execute 595 sec latency 1032.090 ms 2024-09-05T06:54:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:35 smithi096 ceph-mon[34468]: pgmap v416: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:36.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223582 1.53 MB/sec execute 596 sec latency 2032.209 ms 2024-09-05T06:54:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:37 smithi115 ceph-mon[34791]: pgmap v417: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:37 smithi086 ceph-mon[30943]: pgmap v417: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:37.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223649 1.53 MB/sec execute 597 sec latency 2237.434 ms 2024-09-05T06:54:37.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:37 smithi096 ceph-mon[34468]: pgmap v417: 129 pgs: 129 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 195 KiB/s wr, 22 op/s 2024-09-05T06:54:38.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:54:38.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:54:38.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:54:38.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:54:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:54:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:54:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:54:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:54:38.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223649 1.52 MB/sec execute 598 sec latency 1573.988 ms 2024-09-05T06:54:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:54:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:54:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:54:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:54:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:39 smithi115 ceph-mon[34791]: pgmap v418: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-05T06:54:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:39 smithi086 ceph-mon[30943]: pgmap v418: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-05T06:54:39.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 223649 1.52 MB/sec execute 599 sec latency 2574.063 ms 2024-09-05T06:54:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:39 smithi096 ceph-mon[34468]: pgmap v418: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-05T06:54:40.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 cleanup 600 sec 2024-09-05T06:54:41.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:41 smithi115 ceph-mon[34791]: pgmap v419: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 779 KiB/s wr, 55 op/s 2024-09-05T06:54:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:41 smithi086 ceph-mon[30943]: pgmap v419: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 779 KiB/s wr, 55 op/s 2024-09-05T06:54:41.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 cleanup 601 sec 2024-09-05T06:54:41.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:41 smithi096 ceph-mon[34468]: pgmap v419: 129 pgs: 129 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 779 KiB/s wr, 55 op/s 2024-09-05T06:54:42.540 INFO:tasks.workunit.client.0.smithi086.stdout: 1 cleanup 602 sec 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: 0 cleanup 602 sec 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: Operation Count AvgLat MaxLat 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: ---------------------------------------- 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: NTCreateX 29034 4.993 26185.673 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: Close 21352 0.046 1.853 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: Rename 1236 87.803 23849.516 2024-09-05T06:54:42.643 INFO:tasks.workunit.client.0.smithi086.stdout: Unlink 5860 15.672 31678.262 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Qpathinfo 26356 3.071 1612.998 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Qfileinfo 4611 0.068 0.383 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Qfsinfo 4851 0.293 4.647 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Sfileinfo 2366 1.268 31.602 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Find 10216 9.428 77.670 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: WriteX 14384 4.563 32823.302 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: ReadX 45861 0.004 1.285 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: LockX 96 0.568 1.317 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: UnlockX 96 0.463 0.820 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: Flush 2034 3.239 116.949 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T06:54:42.644 INFO:tasks.workunit.client.0.smithi086.stdout:Throughput 1.52072 MB/sec 1 clients 1 procs max_latency=32823.654 ms 2024-09-05T06:54:42.645 INFO:tasks.workunit.client.0.smithi086.stderr:+ dbench 10 2024-09-05T06:54:42.645 INFO:tasks.workunit.client.0.smithi086.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-05T06:54:42.645 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T06:54:42.645 INFO:tasks.workunit.client.0.smithi086.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-05T06:54:42.645 INFO:tasks.workunit.client.0.smithi086.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-05T06:54:42.745 INFO:tasks.workunit.client.0.smithi086.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-05T06:54:42.745 INFO:tasks.workunit.client.0.smithi086.stdout:releasing clients 2024-09-05T06:54:43.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:43 smithi115 ceph-mon[34791]: pgmap v420: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 841 KiB/s wr, 67 op/s 2024-09-05T06:54:43.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:43 smithi086 ceph-mon[30943]: pgmap v420: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 841 KiB/s wr, 67 op/s 2024-09-05T06:54:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:43 smithi096 ceph-mon[34468]: pgmap v420: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 841 KiB/s wr, 67 op/s 2024-09-05T06:54:43.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 282 105.22 MB/sec warmup 1 sec latency 59.473 ms 2024-09-05T06:54:44.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 516 90.39 MB/sec warmup 2 sec latency 96.471 ms 2024-09-05T06:54:45.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:45 smithi115 ceph-mon[34791]: pgmap v421: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:45 smithi086 ceph-mon[30943]: pgmap v421: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:45 smithi096 ceph-mon[34468]: pgmap v421: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:45.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 708 70.74 MB/sec warmup 3 sec latency 100.864 ms 2024-09-05T06:54:46.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:46 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:46.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:46 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:46.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:46 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:46.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 859 53.99 MB/sec warmup 4 sec latency 89.342 ms 2024-09-05T06:54:47.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:47 smithi115 ceph-mon[34791]: pgmap v422: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:47 smithi086 ceph-mon[30943]: pgmap v422: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:47 smithi096 ceph-mon[34468]: pgmap v422: 129 pgs: 129 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 646 KiB/s wr, 48 op/s 2024-09-05T06:54:47.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1016 45.53 MB/sec warmup 5 sec latency 84.818 ms 2024-09-05T06:54:48.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1151 38.34 MB/sec warmup 6 sec latency 99.088 ms 2024-09-05T06:54:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:49 smithi115 ceph-mon[34791]: pgmap v423: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 18 MiB/s wr, 177 op/s 2024-09-05T06:54:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:49 smithi086 ceph-mon[30943]: pgmap v423: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 18 MiB/s wr, 177 op/s 2024-09-05T06:54:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:49 smithi096 ceph-mon[34468]: pgmap v423: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 18 MiB/s wr, 177 op/s 2024-09-05T06:54:49.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1267 33.04 MB/sec warmup 7 sec latency 187.941 ms 2024-09-05T06:54:50.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1415 32.16 MB/sec warmup 8 sec latency 219.108 ms 2024-09-05T06:54:50.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:54:51.333 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:51.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:51 smithi115 ceph-mon[34791]: pgmap v424: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 17 MiB/s wr, 140 op/s 2024-09-05T06:54:51.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:51 smithi086 ceph-mon[30943]: pgmap v424: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 17 MiB/s wr, 140 op/s 2024-09-05T06:54:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:51 smithi096 ceph-mon[34468]: pgmap v424: 129 pgs: 129 active+clean; 474 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 17 MiB/s wr, 140 op/s 2024-09-05T06:54:51.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1577 29.04 MB/sec warmup 9 sec latency 82.878 ms 2024-09-05T06:54:51.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 -- 172.21.15.86:0/2464074755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444100e20 msgr2=0x7fc444101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 --2- 172.21.15.86:0/2464074755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444100e20 0x7fc444101200 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc440009900 tx=0x7fc44002a480 comp rx=0 tx=0).stop 2024-09-05T06:54:51.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 -- 172.21.15.86:0/2464074755 shutdown_connections 2024-09-05T06:54:51.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 --2- 172.21.15.86:0/2464074755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 0x7fc44410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 --2- 172.21.15.86:0/2464074755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444102570 0x7fc4441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 --2- 172.21.15.86:0/2464074755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444100e20 0x7fc444101200 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 -- 172.21.15.86:0/2464074755 >> 172.21.15.86:0/2464074755 conn(0x7fc4440fc7d0 msgr2=0x7fc4440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 -- 172.21.15.86:0/2464074755 shutdown_connections 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.882+0000 7fc44c760640 1 -- 172.21.15.86:0/2464074755 wait complete. 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 Processor -- start 2024-09-05T06:54:51.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 -- start start 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 0x7fc444198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 0x7fc44419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110e30 con 0x7fc444102570 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110cb0 con 0x7fc444100e20 2024-09-05T06:54:51.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.883+0000 7fc44c760640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110fb0 con 0x7fc444102ef0 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38010/0 (socket says 172.21.15.86:38010) 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 -- 172.21.15.86:0/3930631081 learned_addr learned my addr 172.21.15.86:0/3930631081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44acfd640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 0x7fc44419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc449cfb640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 0x7fc444198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 -- 172.21.15.86:0/3930631081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 msgr2=0x7fc44419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 0x7fc44419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 -- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 msgr2=0x7fc444198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 0x7fc444198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc44a4fc640 1 -- 172.21.15.86:0/3930631081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44419d3d0 con 0x7fc444100e20 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.884+0000 7fc449cfb640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 0x7fc444198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc440006010 tx=0x7fc4400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc43b7fe640 1 -- 172.21.15.86:0/3930631081 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44003f030 con 0x7fc444100e20 2024-09-05T06:54:51.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44419d660 con 0x7fc444100e20 2024-09-05T06:54:51.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc43b7fe640 1 -- 172.21.15.86:0/3930631081 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400044c0 con 0x7fc444100e20 2024-09-05T06:54:51.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc43b7fe640 1 -- 172.21.15.86:0/3930631081 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440035070 con 0x7fc444100e20 2024-09-05T06:54:51.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.885+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441a4fa0 con 0x7fc444100e20 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 Processor -- start 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 -- start start 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc43b7fe640 1 -- 172.21.15.86:0/3930631081 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc440042030 con 0x7fc444100e20 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbf99dd2e0 con 0x55cbf9a7b8b0 2024-09-05T06:54:51.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cbf9984ce0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.888+0000 7fc44eb39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cbf99d92a0 con 0x55cbf9a86a90 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44acfd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38022/0 (socket says 172.21.15.86:38022) 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 -- 172.21.15.86:0/14865489 learned_addr learned my addr 172.21.15.86:0/14865489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc449cfb640 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 -- 172.21.15.86:0/14865489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 msgr2=0x55cbf9a88e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a88e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 -- 172.21.15.86:0/14865489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 msgr2=0x55cbf9a7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 -- 172.21.15.86:0/14865489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf9969440 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc449cfb640 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.889+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ec990 tx=0x7fc4400f5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635349b854d0f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.890+0000 7fc438ff9640 1 -- 172.21.15.86:0/14865489 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400fe070 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.890+0000 7fc438ff9640 1 -- 172.21.15.86:0/14865489 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400f7d30 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.890+0000 7fc438ff9640 1 -- 172.21.15.86:0/14865489 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400fa420 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.890+0000 7fc43b7fe640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 0x7fc4300a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.890+0000 7fc44acfd640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 0x7fc4300a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc43b7fe640 1 -- 172.21.15.86:0/3930631081 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc4400ef7a0 con 0x7fc444100e20 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 -- 172.21.15.86:0/14865489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 msgr2=0x55cbf9a86550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ec990 tx=0x7fc4400f5cb0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44acfd640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 0x7fc4300a52b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005fd0 tx=0x7fc43c009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 -- 172.21.15.86:0/14865489 shutdown_connections 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a86550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 --2- 172.21.15.86:0/14865489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.891+0000 7fc44eb39740 1 -- 172.21.15.86:0/14865489 >> 172.21.15.86:0/14865489 conn(0x55cbf9a77e40 msgr2=0x55cbf9a78220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.892+0000 7fc44eb39740 1 -- 172.21.15.86:0/14865489 shutdown_connections 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.892+0000 7fc44eb39740 1 -- 172.21.15.86:0/14865489 wait complete. 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.892+0000 7fc44eb39740 1 -- start start 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.892+0000 7fc44eb39740 1 Processor -- start 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a83600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbf9a89420 con 0x55cbf9a7b8b0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cbf9998050 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cbf9a895a0 con 0x55cbf9a86a90 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44eb39740 10 client.0 _ll_register_callbacks cb 0x55cbf99d8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a83600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a860e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47954/0 (socket says 172.21.15.86:47954) 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc44a4fc640 1 -- 172.21.15.86:0/154188883 learned_addr learned my addr 172.21.15.86:0/154188883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 -- 172.21.15.86:0/154188883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 msgr2=0x55cbf9a83b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 -- 172.21.15.86:0/154188883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 msgr2=0x55cbf9a860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 -- 172.21.15.86:0/154188883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf974e3c0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc449cfb640 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a83600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc444100080 tx=0x7fc43400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c4e591f8656eba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.893+0000 7fc42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340120d0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc434004020 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4400f7d30 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc434012270 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc44eb39740 10 client.27590 fetch_fsmap learned FSMap version 37 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbf9a99160 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400f7d30 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.894+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fc43401c500 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 10 client.27590 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 20 client.27590 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a9a6c0 0x55cbf9a9cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 4 client.27590 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf99ea850 con 0x55cbf9a9a6c0 2024-09-05T06:54:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a9a6c0 0x55cbf9a9cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc42dffb640 10 client.27590 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.895+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a9a6c0 0x55cbf9a9cb00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36689c67427bfcd server_cookie=99ed7c42053e1a83 in_seq=0 out_seq=0 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc42dffb640 1 -- 172.21.15.86:0/154188883 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbf99ea850 con 0x55cbf9a9a6c0 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc42dffb640 10 client.27590 handle_command_reply: tid=0 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 client.27590 shutdown 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 msgr2=0x55cbf9a83600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a83600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc444100080 tx=0x7fc43400f790 comp rx=0 tx=0).stop 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 shutdown_connections 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a9a6c0 0x55cbf9a9cb00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a86a90 0x55cbf9a83b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a83600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 --2- 172.21.15.86:0/154188883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 >> 172.21.15.86:0/154188883 conn(0x55cbf9a77e40 msgr2=0x55cbf9a72b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc42dffb640 0 client.27590 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 shutdown_connections 2024-09-05T06:54:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 1 -- 172.21.15.86:0/154188883 wait complete. 2024-09-05T06:54:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.897+0000 7fc44eb39740 20 client.27590 trim_cache size 0 max 16384 2024-09-05T06:54:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 Processor -- start 2024-09-05T06:54:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 -- start start 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9aa2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a80860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441acf70 con 0x55cbf9a77e40 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aeb40 con 0x55cbf9a7b8b0 2024-09-05T06:54:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44eb39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441ad640 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44a4fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47966/0 (socket says 172.21.15.86:47966) 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc44a4fc640 1 -- 172.21.15.86:0/4097196534 learned_addr learned my addr 172.21.15.86:0/4097196534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.923+0000 7fc449cfb640 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9aa2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44acfd640 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a80860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 -- 172.21.15.86:0/4097196534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a7c1d0 msgr2=0x55cbf9a80860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a80860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 -- 172.21.15.86:0/4097196534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7b8b0 msgr2=0x55cbf9aa2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9aa2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 -- 172.21.15.86:0/4097196534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf9a77c70 con 0x55cbf9a77e40 2024-09-05T06:54:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.924+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc4400a6010 tx=0x7fc4400f1540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a100d6e6783882d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc42ffff640 1 -- 172.21.15.86:0/4097196534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440049070 con 0x55cbf9a77e40 2024-09-05T06:54:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc42ffff640 1 -- 172.21.15.86:0/4097196534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400bb8d0 con 0x55cbf9a77e40 2024-09-05T06:54:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc42ffff640 1 -- 172.21.15.86:0/4097196534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400fe040 con 0x55cbf9a77e40 2024-09-05T06:54:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc44eb39740 1 -- 172.21.15.86:0/4097196534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 msgr2=0x55cbf9aa23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc44eb39740 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc4400a6010 tx=0x7fc4400f1540 comp rx=0 tx=0).stop 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc44eb39740 1 -- 172.21.15.86:0/4097196534 shutdown_connections 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.925+0000 7fc44eb39740 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a80860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9aa2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 --2- 172.21.15.86:0/4097196534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a77e40 0x55cbf9aa23d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 -- 172.21.15.86:0/4097196534 >> 172.21.15.86:0/4097196534 conn(0x55cbf9a787c0 msgr2=0x55cbf9a85680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 -- 172.21.15.86:0/4097196534 shutdown_connections 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 -- 172.21.15.86:0/4097196534 wait complete. 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 -- start start 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.926+0000 7fc44eb39740 1 Processor -- start 2024-09-05T06:54:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 0x55cbf9a978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441ae440 con 0x55cbf9a7b8b0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441add40 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aaf70 con 0x55cbf9a77e40 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44eb39740 10 client.0 _ll_register_callbacks cb 0x7fc434009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38066/0 (socket says 172.21.15.86:38066) 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 -- 172.21.15.86:0/1317245401 learned_addr learned my addr 172.21.15.86:0/1317245401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc449cfb640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 0x55cbf9a978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 -- 172.21.15.86:0/1317245401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 msgr2=0x55cbf9a978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 0x55cbf9a978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 -- 172.21.15.86:0/1317245401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 msgr2=0x55cbf9a95c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a95c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44acfd640 1 -- 172.21.15.86:0/1317245401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf9a969c0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.927+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 0x55cbf9a978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc44acfd640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004270 tx=0x7fc43c004670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a62df2778f2510 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004860 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fc43c017070 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4400bb8d0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc43c033050 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc44eb39740 10 client.27596 fetch_fsmap learned FSMap version 37 2024-09-05T06:54:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbf9b06470 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.928+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400bb8d0 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fc43c01e400 con 0x55cbf9a7c1d0 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 10 client.27596 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 20 client.27596 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a86a90 0x55cbf9b08ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 4 client.27596 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbf99e4d60 con 0x55cbf9a86a90 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a86a90 0x55cbf9b08ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc42dffb640 10 client.27596 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.929+0000 7fc44a4fc640 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a86a90 0x55cbf9b08ce0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5090e22d6dc73f2 server_cookie=23ca8780fbe2d64d in_seq=0 out_seq=0 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc42dffb640 1 -- 172.21.15.86:0/1317245401 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbf99e4d60 con 0x55cbf9a86a90 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc42dffb640 10 client.27596 handle_command_reply: tid=0 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 client.27596 shutdown 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 msgr2=0x55cbf9a961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004270 tx=0x7fc43c004670 comp rx=0 tx=0).stop 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 shutdown_connections 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cbf9a86a90 0x55cbf9b08ce0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cbf9a7c1d0 0x55cbf9a961b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf9a7b8b0 0x55cbf9a95c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 --2- 172.21.15.86:0/1317245401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cbf9a77e40 0x55cbf9a978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 >> 172.21.15.86:0/1317245401 conn(0x55cbf9a787c0 msgr2=0x55cbf9a7d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc42dffb640 0 client.27596 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 shutdown_connections 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.931+0000 7fc44eb39740 1 -- 172.21.15.86:0/1317245401 wait complete. 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44eb39740 20 client.27596 trim_cache size 0 max 16384 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:54:51.935 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:54:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 msgr2=0x7fc4300a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 0x7fc4300a52b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005fd0 tx=0x7fc43c009500 comp rx=0 tx=0).stop 2024-09-05T06:54:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 msgr2=0x7fc444198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc440006010 tx=0x7fc4400040c0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 shutdown_connections 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc4300a2df0 0x7fc4300a52b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc444102ef0 0x7fc44419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc444102570 0x7fc444198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 --2- 172.21.15.86:0/3930631081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc444100e20 0x7fc444198420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 >> 172.21.15.86:0/3930631081 conn(0x7fc4440fc7d0 msgr2=0x7fc44406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 shutdown_connections 2024-09-05T06:54:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:51.932+0000 7fc44c760640 1 -- 172.21.15.86:0/3930631081 wait complete. 2024-09-05T06:54:52.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1750 26.28 MB/sec warmup 10 sec latency 79.087 ms 2024-09-05T06:54:52.943 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:54:52.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:54:53.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:53 smithi086 ceph-mon[30943]: pgmap v425: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 217 op/s 2024-09-05T06:54:53.286 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:53.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:53 smithi115 ceph-mon[34791]: pgmap v425: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 217 op/s 2024-09-05T06:54:53.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:53 smithi096 ceph-mon[34468]: pgmap v425: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 217 op/s 2024-09-05T06:54:53.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1930 24.05 MB/sec warmup 11 sec latency 72.969 ms 2024-09-05T06:54:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 -- 172.21.15.86:0/674131104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 msgr2=0x7f51cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/674131104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc10da40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009ca0 tx=0x7f51c402a940 comp rx=0 tx=0).stop 2024-09-05T06:54:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 -- 172.21.15.86:0/674131104 shutdown_connections 2024-09-05T06:54:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/674131104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc10da40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/674131104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc102570 0x7f51cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/674131104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc100e20 0x7f51cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 -- 172.21.15.86:0/674131104 >> 172.21.15.86:0/674131104 conn(0x7f51cc0fc7d0 msgr2=0x7f51cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 -- 172.21.15.86:0/674131104 shutdown_connections 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.144+0000 7f51d3fdb640 1 -- 172.21.15.86:0/674131104 wait complete. 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 Processor -- start 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 -- start start 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc100e20 0x7f51cc196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc10fd30 con 0x7f51cc102ef0 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc10fbb0 con 0x7f51cc100e20 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d3fdb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc10feb0 con 0x7f51cc102570 2024-09-05T06:54:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d1576640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d1576640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55794/0 (socket says 172.21.15.86:55794) 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d1576640 1 -- 172.21.15.86:0/3282220049 learned_addr learned my addr 172.21.15.86:0/3282220049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d2578640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.145+0000 7f51d1d77640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc100e20 0x7f51cc196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 -- 172.21.15.86:0/3282220049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc100e20 msgr2=0x7f51cc196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc100e20 0x7f51cc196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 -- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 msgr2=0x7f51cc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 -- 172.21.15.86:0/3282220049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc19b120 con 0x7f51cc102570 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.146+0000 7f51d1576640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9f0 tx=0x7f51bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.147+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00cd80 con 0x7f51cc102570 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.147+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc19b410 con 0x7f51cc102570 2024-09-05T06:54:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.147+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51bc0108e0 con 0x7f51cc102570 2024-09-05T06:54:54.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.148+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc018850 con 0x7f51cc102570 2024-09-05T06:54:54.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.148+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc1a2cf0 con 0x7f51cc102570 2024-09-05T06:54:54.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.149+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc101200 con 0x7f51cc102570 2024-09-05T06:54:54.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.150+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51bc010a90 con 0x7f51cc102570 2024-09-05T06:54:54.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.151+0000 7f51c2ffd640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 0x7f51b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:54.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.151+0000 7f51d1d77640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 0x7f51b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:54.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.151+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f51bc0cfc20 con 0x7f51cc102570 2024-09-05T06:54:54.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.151+0000 7f51d1d77640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 0x7f51b80a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009bd0 tx=0x7f51c8005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:54.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.156+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51bc0751b0 con 0x7f51cc102570 2024-09-05T06:54:54.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.481+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51cc109090 con 0x7f51cc102570 2024-09-05T06:54:54.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.482+0000 7f51c2ffd640 1 -- 172.21.15.86:0/3282220049 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11977 (secure 0 0 0) 0x7f51bc09c510 con 0x7f51cc102570 2024-09-05T06:54:54.483 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:54:54.483 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"btime":"2024-09-05T06:54:45:087985+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:44.172062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:54:54.483 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-09-05T06:54:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 msgr2=0x7f51b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 0x7f51b80a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009bd0 tx=0x7f51c8005f90 comp rx=0 tx=0).stop 2024-09-05T06:54:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 msgr2=0x7f51cc1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9f0 tx=0x7f51bc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 shutdown_connections 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51b80a2d70 0x7f51b80a5230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51cc102ef0 0x7f51cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51cc102570 0x7f51cc1966b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 --2- 172.21.15.86:0/3282220049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51cc100e20 0x7f51cc196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 >> 172.21.15.86:0/3282220049 conn(0x7f51cc0fc7d0 msgr2=0x7f51cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 shutdown_connections 2024-09-05T06:54:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:54.485+0000 7f51d3fdb640 1 -- 172.21.15.86:0/3282220049 wait complete. 2024-09-05T06:54:54.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2115 22.32 MB/sec warmup 12 sec latency 80.044 ms 2024-09-05T06:54:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:55 smithi115 ceph-mon[34791]: pgmap v426: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:55 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3282220049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:55.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:55 smithi086 ceph-mon[30943]: pgmap v426: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:55.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:55 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3282220049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:55.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:55 smithi096 ceph-mon[34468]: pgmap v426: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:55.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:55 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3282220049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:54:55.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:54:55.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2297 21.00 MB/sec warmup 13 sec latency 85.830 ms 2024-09-05T06:54:55.954 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:54:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1902024377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 msgr2=0x7f89d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1902024377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d010da40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009cd0 tx=0x7f89c802a970 comp rx=0 tx=0).stop 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1902024377 shutdown_connections 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1902024377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d010da40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1902024377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1902024377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.612+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1902024377 >> 172.21.15.86:0/1902024377 conn(0x7f89d00fc7d0 msgr2=0x7f89d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.613+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1902024377 shutdown_connections 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.613+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1902024377 wait complete. 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.613+0000 7f89d6b0d640 1 Processor -- start 2024-09-05T06:54:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.613+0000 7f89d6b0d640 1 -- start start 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010fd70 con 0x7f89d0102570 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010fbf0 con 0x7f89d0100e20 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d6b0d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010fef0 con 0x7f89d0102ef0 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48006/0 (socket says 172.21.15.86:48006) 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d48a9640 1 -- 172.21.15.86:0/1872203126 learned_addr learned my addr 172.21.15.86:0/1872203126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89d50aa640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 -- 172.21.15.86:0/1872203126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 msgr2=0x7f89d019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 -- 172.21.15.86:0/1872203126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 msgr2=0x7f89d019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.614+0000 7f89cffff640 1 -- 172.21.15.86:0/1872203126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d019f590 con 0x7f89d0102570 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.615+0000 7f89cffff640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d019ab20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00eab0 tx=0x7f89bc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.615+0000 7f89cdffb640 1 -- 172.21.15.86:0/1872203126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00cc90 con 0x7f89d0102570 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.615+0000 7f89cdffb640 1 -- 172.21.15.86:0/1872203126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89bc00ce30 con 0x7f89d0102570 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.615+0000 7f89cdffb640 1 -- 172.21.15.86:0/1872203126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0188e0 con 0x7f89d0102570 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.615+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d019f880 con 0x7f89d0102570 2024-09-05T06:54:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.616+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d019fcb0 con 0x7f89d0102570 2024-09-05T06:54:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.618+0000 7f89cdffb640 1 -- 172.21.15.86:0/1872203126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f89bc018a80 con 0x7f89d0102570 2024-09-05T06:54:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.619+0000 7f89cdffb640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 0x7f89c00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.619+0000 7f89cdffb640 1 -- 172.21.15.86:0/1872203126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f89bc0cfc20 con 0x7f89d0102570 2024-09-05T06:54:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.619+0000 7f89d48a9640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 0x7f89c00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 Processor -- start 2024-09-05T06:54:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 -- start start 2024-09-05T06:54:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00fed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00feda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b00f532f0 con 0x563b00fed5d0 2024-09-05T06:54:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b00efacf0 con 0x563b00ff8370 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563b00f4f2b0 con 0x563b00feeb40 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d50aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d50aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48008/0 (socket says 172.21.15.86:48008) 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d48a9640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 0x7f89c00a5300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b800 tx=0x7f89c400d040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d50aa640 1 -- 172.21.15.86:0/2628311423 learned_addr learned my addr 172.21.15.86:0/2628311423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.620+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00fed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89cffff640 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00feda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 -- 172.21.15.86:0/2628311423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 msgr2=0x563b00ffa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 -- 172.21.15.86:0/2628311423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 msgr2=0x563b00feda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00feda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 -- 172.21.15.86:0/2628311423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b00edf3a0 con 0x563b00feeb40 2024-09-05T06:54:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00fed090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009aa0 tx=0x7f89c4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c447b44d6e00df14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89baffd640 1 -- 172.21.15.86:0/2628311423 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004220 con 0x563b00feeb40 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.621+0000 7f89baffd640 1 -- 172.21.15.86:0/2628311423 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c401f040 con 0x563b00feeb40 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.622+0000 7f89baffd640 1 -- 172.21.15.86:0/2628311423 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4023690 con 0x563b00feeb40 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.622+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2628311423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 msgr2=0x563b00fed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.622+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00fed090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009aa0 tx=0x7f89c4014000 comp rx=0 tx=0).stop 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2628311423 shutdown_connections 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00feda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2628311423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00fed090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2628311423 >> 172.21.15.86:0/2628311423 conn(0x563b00fb6310 msgr2=0x563b00fe9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2628311423 shutdown_connections 2024-09-05T06:54:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2628311423 wait complete. 2024-09-05T06:54:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.623+0000 7f89d8ee6740 1 -- start start 2024-09-05T06:54:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 Processor -- start 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00ff7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00ff4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b00ffb110 con 0x563b00fed5d0 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b00f0e060 con 0x563b00ff8370 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563b00ffb290 con 0x563b00feeb40 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d8ee6740 10 client.0 _ll_register_callbacks cb 0x563b00c618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00ff7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00ff4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00ff7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43012/0 (socket says 172.21.15.86:43012) 2024-09-05T06:54:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d48a9640 1 -- 172.21.15.86:0/1106203431 learned_addr learned my addr 172.21.15.86:0/1106203431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89cffff640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00ff4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55828/0 (socket says 172.21.15.86:55828) 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.624+0000 7f89d50aa640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 -- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 msgr2=0x563b00ff4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00ff4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 -- 172.21.15.86:0/1106203431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 msgr2=0x563b00ff7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00ff7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 -- 172.21.15.86:0/1106203431 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b00cc43c0 con 0x563b00ff8370 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d50aa640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff5420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c8004af0 tx=0x7f89c802fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb97a185517eceb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89affff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8049060 con 0x563b00ff8370 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c401f040 con 0x563b00ff8370 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.625+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f89c8003b70 con 0x563b00ff8370 2024-09-05T06:54:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f89c8050050 con 0x563b00ff8370 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 10 client.27614 fetch_fsmap learned FSMap version 38 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563b01006b40 con 0x563b00ff8370 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c401f040 con 0x563b00ff8370 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f89c8004680 con 0x563b00ff8370 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 10 client.27614 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 10 client.27614 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 20 client.27614 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01008330 0x563b0100a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 4 client.27614 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b00f60860 con 0x563b01008330 2024-09-05T06:54:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.626+0000 7f89d48a9640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01008330 0x563b0100a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.627+0000 7f89affff640 10 client.27614 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.627+0000 7f89d48a9640 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01008330 0x563b0100a760 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=260b0c6aa68f0f8 server_cookie=e7771c19dbd2eb6e in_seq=0 out_seq=0 2024-09-05T06:54:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.628+0000 7f89affff640 1 -- 172.21.15.86:0/1106203431 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b00f60860 con 0x563b01008330 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.628+0000 7f89affff640 10 client.27614 handle_command_reply: tid=0 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.628+0000 7f89d8ee6740 1 client.27614 shutdown 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 msgr2=0x563b00ff5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff5420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c8004af0 tx=0x7f89c802fce0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 shutdown_connections 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01008330 0x563b0100a760 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff5420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b00ff4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/1106203431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00fed5d0 0x563b00ff7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 >> 172.21.15.86:0/1106203431 conn(0x563b00fb6310 msgr2=0x563b00fef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89affff640 0 client.27614 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 shutdown_connections 2024-09-05T06:54:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 1 -- 172.21.15.86:0/1106203431 wait complete. 2024-09-05T06:54:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.629+0000 7f89d8ee6740 20 client.27614 trim_cache size 0 max 16384 2024-09-05T06:54:56.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.648+0000 7f89d8ee6740 1 Processor -- start 2024-09-05T06:54:56.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 -- start start 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 0x563b00ff0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01acf20 con 0x563b00feeb40 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01aeb80 con 0x563b00ff8370 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01ad5f0 con 0x563b00fed5d0 2024-09-05T06:54:56.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55834/0 (socket says 172.21.15.86:55834) 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 -- 172.21.15.86:0/3099122217 learned_addr learned my addr 172.21.15.86:0/3099122217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89cffff640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 0x563b00ff0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d50aa640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 -- 172.21.15.86:0/3099122217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 msgr2=0x563b00ff0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 -- 172.21.15.86:0/3099122217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 msgr2=0x563b00ff0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 0x563b00ff0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.649+0000 7f89d48a9640 1 -- 172.21.15.86:0/3099122217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b00ffaad0 con 0x563b00fed5d0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.650+0000 7f89cffff640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 0x563b00ff0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.650+0000 7f89d48a9640 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b3c0 tx=0x7f89c4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b42d8bb163af651 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.650+0000 7f89ba7fc640 1 -- 172.21.15.86:0/3099122217 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c403b070 con 0x563b00fed5d0 2024-09-05T06:54:56.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.650+0000 7f89ba7fc640 1 -- 172.21.15.86:0/3099122217 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c401f070 con 0x563b00fed5d0 2024-09-05T06:54:56.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.650+0000 7f89ba7fc640 1 -- 172.21.15.86:0/3099122217 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c40352f0 con 0x563b00fed5d0 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 -- 172.21.15.86:0/3099122217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 msgr2=0x563b00ff6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b3c0 tx=0x7f89c4014000 comp rx=0 tx=0).stop 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 -- 172.21.15.86:0/3099122217 shutdown_connections 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ff0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00feeb40 0x563b00ff0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/3099122217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00fed5d0 0x563b00ff6e50 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b3c0 tx=0x7f89c4014000 comp rx=0 tx=0).stop 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 -- 172.21.15.86:0/3099122217 >> 172.21.15.86:0/3099122217 conn(0x563b00fb6310 msgr2=0x563b00ff6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 -- 172.21.15.86:0/3099122217 shutdown_connections 2024-09-05T06:54:56.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.651+0000 7f89d8ee6740 1 -- 172.21.15.86:0/3099122217 wait complete. 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.652+0000 7f89d8ee6740 1 -- start start 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.652+0000 7f89d8ee6740 1 Processor -- start 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.652+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b01000c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.652+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.652+0000 7f89d8ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 0x563b00ffc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01ae3f0 con 0x563b00ffc3e0 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01adcf0 con 0x563b00ff8370 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d8ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01aaf40 con 0x563b00feeb40 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48036/0 (socket says 172.21.15.86:48036) 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 -- 172.21.15.86:0/2166453300 learned_addr learned my addr 172.21.15.86:0/2166453300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d50aa640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 0x563b00ffc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b01000c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d8ee6740 10 client.0 _ll_register_callbacks cb 0x7f89c8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 -- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 msgr2=0x563b01000c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b01000c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 -- 172.21.15.86:0/2166453300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 msgr2=0x563b00ffc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 0x563b00ffc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 -- 172.21.15.86:0/2166453300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b01072090 con 0x563b00ff8370 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89d50aa640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 0x563b00ffc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89cffff640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00ea80 tx=0x7f89bc09c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58bb291eec262193 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.653+0000 7f89affff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc09b350 con 0x563b00ff8370 2024-09-05T06:54:56.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7f89bc097a50 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c401f070 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f89bc0a4970 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 10 client.27617 fetch_fsmap learned FSMap version 38 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563b01073350 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c401f070 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f89bc0a1240 con 0x563b00ff8370 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 10 client.27617 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 10 client.27617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 20 client.27617 populate_metadata read hostname 'smithi086' 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01073af0 0x563b01075f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 4 client.27617 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:54:56.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.654+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b00f5ad70 con 0x563b01073af0 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01073af0 0x563b01075f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89d48a9640 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01073af0 0x563b01075f10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=946934963c549ac8 server_cookie=88d69f1ae17726b6 in_seq=0 out_seq=0 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89affff640 10 client.27617 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89affff640 1 -- 172.21.15.86:0/2166453300 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b00f5ad70 con 0x563b01073af0 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89affff640 10 client.27617 handle_command_reply: tid=0 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.655+0000 7f89d8ee6740 1 client.27617 shutdown 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 msgr2=0x563b00ffbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00ea80 tx=0x7f89bc09c4c0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 shutdown_connections 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563b01073af0 0x563b01075f10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b00ffc3e0 0x563b00ffc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b00ff8370 0x563b00ffbea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 --2- 172.21.15.86:0/2166453300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b00feeb40 0x563b01000c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 >> 172.21.15.86:0/2166453300 conn(0x563b00fb6310 msgr2=0x563b00ff6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89affff640 0 client.27617 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 shutdown_connections 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 1 -- 172.21.15.86:0/2166453300 wait complete. 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.656+0000 7f89d8ee6740 20 client.27617 trim_cache size 0 max 16384 2024-09-05T06:54:56.660 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 msgr2=0x7f89c00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 0x7f89c00a5300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b800 tx=0x7f89c400d040 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 msgr2=0x7f89d019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d019ab20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00eab0 tx=0x7f89bc00ef70 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 shutdown_connections 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89c00a2e40 0x7f89c00a5300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89d0102ef0 0x7f89d019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d0102570 0x7f89d019ab20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 --2- 172.21.15.86:0/1872203126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d0100e20 0x7f89d019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:54:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 >> 172.21.15.86:0/1872203126 conn(0x7f89d00fc7d0 msgr2=0x7f89d0104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:54:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 shutdown_connections 2024-09-05T06:54:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:54:56.657+0000 7f89d6b0d640 1 -- 172.21.15.86:0/1872203126 wait complete. 2024-09-05T06:54:56.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2477 20.47 MB/sec warmup 14 sec latency 148.402 ms 2024-09-05T06:54:57.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:57 smithi115 ceph-mon[34791]: pgmap v427: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:57.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:57 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:57 smithi086 ceph-mon[30943]: pgmap v427: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:57 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:57 smithi096 ceph-mon[34468]: pgmap v427: 129 pgs: 129 active+clean; 480 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-09-05T06:54:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:57 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:54:57.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2660 19.95 MB/sec warmup 15 sec latency 83.417 ms 2024-09-05T06:54:58.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2914 20.25 MB/sec warmup 16 sec latency 101.042 ms 2024-09-05T06:54:59.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:54:59 smithi115 ceph-mon[34791]: pgmap v428: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 299 op/s 2024-09-05T06:54:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:54:59 smithi086 ceph-mon[30943]: pgmap v428: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 299 op/s 2024-09-05T06:54:59.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:54:59 smithi096 ceph-mon[34468]: pgmap v428: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 20 MiB/s wr, 299 op/s 2024-09-05T06:54:59.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3121 20.18 MB/sec warmup 17 sec latency 208.698 ms 2024-09-05T06:54:59.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:55:00.283 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:00.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3330 19.77 MB/sec warmup 18 sec latency 79.179 ms 2024-09-05T06:55:01.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:01 smithi115 ceph-mon[34791]: pgmap v429: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.0 MiB/s wr, 170 op/s 2024-09-05T06:55:01.436 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:01 smithi086 ceph-mon[30943]: pgmap v429: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.0 MiB/s wr, 170 op/s 2024-09-05T06:55:01.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:01 smithi096 ceph-mon[34468]: pgmap v429: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.0 MiB/s wr, 170 op/s 2024-09-05T06:55:01.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3423 19.18 MB/sec warmup 19 sec latency 129.888 ms 2024-09-05T06:55:02.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3527 18.58 MB/sec warmup 20 sec latency 154.593 ms 2024-09-05T06:55:03.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:03 smithi115 ceph-mon[34791]: pgmap v430: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 274 op/s 2024-09-05T06:55:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:03 smithi086 ceph-mon[30943]: pgmap v430: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 274 op/s 2024-09-05T06:55:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:03 smithi096 ceph-mon[34468]: pgmap v430: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 274 op/s 2024-09-05T06:55:03.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3674 18.22 MB/sec warmup 21 sec latency 112.333 ms 2024-09-05T06:55:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.600+0000 7fb3dd906640 1 -- 172.21.15.86:0/1838006453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 msgr2=0x7fb3d81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.600+0000 7fb3dd906640 1 --2- 172.21.15.86:0/1838006453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d81031b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009900 tx=0x7fb3c802a480 comp rx=0 tx=0).stop 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 -- 172.21.15.86:0/1838006453 shutdown_connections 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 --2- 172.21.15.86:0/1838006453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d810c170 0x7fb3d810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 --2- 172.21.15.86:0/1838006453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d81036f0 0x7fb3d810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 --2- 172.21.15.86:0/1838006453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d81031b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 -- 172.21.15.86:0/1838006453 >> 172.21.15.86:0/1838006453 conn(0x7fb3d80fc790 msgr2=0x7fb3d80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 -- 172.21.15.86:0/1838006453 shutdown_connections 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 -- 172.21.15.86:0/1838006453 wait complete. 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.601+0000 7fb3dd906640 1 Processor -- start 2024-09-05T06:55:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 -- start start 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 0x7fb3d8100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d810c170 0x7fb3d81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111e10 con 0x7fb3d8102dd0 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111c90 con 0x7fb3d81036f0 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3dd906640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111f90 con 0x7fb3d810c170 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47892/0 (socket says 172.21.15.86:47892) 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/3922116775 learned_addr learned my addr 172.21.15.86:0/3922116775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 0x7fb3d8100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.602+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d810c170 0x7fb3d81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 -- 172.21.15.86:0/3922116775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d810c170 msgr2=0x7fb3d81014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d810c170 0x7fb3d81014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 -- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 msgr2=0x7fb3d8100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 -- 172.21.15.86:0/3922116775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81a6890 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 0x7fb3d8100f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e9f0 tx=0x7fb3cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3dc904640 1 -- 172.21.15.86:0/3922116775 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00ccc0 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d81a6b80 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3dc904640 1 -- 172.21.15.86:0/3922116775 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3cc005020 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a70c0 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.603+0000 7fb3dc904640 1 -- 172.21.15.86:0/3922116775 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004a00 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 Processor -- start 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dc904640 1 -- 172.21.15.86:0/3922116775 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb3cc004ba0 con 0x7fb3d81036f0 2024-09-05T06:55:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 -- start start 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d110a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d11d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d0d0762f0 con 0x559d0d1105d0 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d0d01dcf0 con 0x559d0d111b40 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.606+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559d0d0722b0 con 0x559d0d11b370 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d110a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46928/0 (socket says 172.21.15.86:46928) 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 -- 172.21.15.86:0/2309365397 learned_addr learned my addr 172.21.15.86:0/2309365397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d11d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 -- 172.21.15.86:0/2309365397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 msgr2=0x559d0d11d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d11d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 -- 172.21.15.86:0/2309365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 msgr2=0x559d0d110a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d110a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 -- 172.21.15.86:0/2309365397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0d0023a0 con 0x559d0d111b40 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009f50 tx=0x7fb3d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b816d83ce74482a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3bdffb640 1 -- 172.21.15.86:0/2309365397 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0004d20 con 0x559d0d111b40 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3bdffb640 1 -- 172.21.15.86:0/2309365397 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d0004050 con 0x559d0d111b40 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3bdffb640 1 -- 172.21.15.86:0/2309365397 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0005280 con 0x559d0d111b40 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.607+0000 7fb3dc904640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 0x7fb3c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 0x7fb3c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dc904640 1 -- 172.21.15.86:0/3922116775 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb3cc0cf870 con 0x7fb3d81036f0 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/2309365397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 msgr2=0x559d0d110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009f50 tx=0x7fb3d00118d0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 0x7fb3c00a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8031440 tx=0x7fb3c80058d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/2309365397 shutdown_connections 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d11d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/2309365397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d110090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.608+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/2309365397 >> 172.21.15.86:0/2309365397 conn(0x559d0d0d9310 msgr2=0x559d0d10cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.609+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/2309365397 shutdown_connections 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.609+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/2309365397 wait complete. 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.609+0000 7fb3dfcdf740 1 -- start start 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.609+0000 7fb3dfcdf740 1 Processor -- start 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d1185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d0d11e070 con 0x559d0d1105d0 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d0d031060 con 0x559d0d111b40 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559d0d11e1f0 con 0x559d0d11b370 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47916/0 (socket says 172.21.15.86:47916) 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/3022100161 learned_addr learned my addr 172.21.15.86:0/3022100161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 10 client.0 _ll_register_callbacks cb 0x559d0cd848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d1185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/3022100161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 msgr2=0x559d0d1185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d1185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 msgr2=0x559d0d117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d117ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/3022100161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0cde73c0 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d0004050 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009a20 tx=0x7fb3c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e986fe5d979a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.610+0000 7fb3b6ffd640 10 client.18654 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/3022100161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c804d070 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/3022100161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb3c8003b80 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/3022100161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3c8043070 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 10 client.18654 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=38}) -- 0x559d0d129a30 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/3022100161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb3c803d450 con 0x559d0d1105d0 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 10 client.18654 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 10 client.18654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 20 client.18654 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:04.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d12b210 0x559d0d12d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 4 client.18654 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.611+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0d083860 con 0x559d0d12b210 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.612+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d12b210 0x559d0d12d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.612+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d12b210 0x559d0d12d660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d1f04fd0a85aa20 server_cookie=125e00086930b073 in_seq=0 out_seq=0 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.612+0000 7fb3b6ffd640 10 client.18654 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/3022100161 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d0d083860 con 0x559d0d12b210 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3b6ffd640 10 client.18654 handle_command_reply: tid=0 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 client.18654 shutdown 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 msgr2=0x559d0d11ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009a20 tx=0x7fb3c8009260 comp rx=0 tx=0).stop 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 shutdown_connections 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d12b210 0x559d0d12d660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d11b370 0x559d0d1185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3022100161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d11ad40 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3b6ffd640 0 client.18654 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 >> 172.21.15.86:0/3022100161 conn(0x559d0d0d9310 msgr2=0x559d0d112960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.613+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 shutdown_connections 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.614+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3022100161 wait complete. 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.614+0000 7fb3dfcdf740 20 client.18654 trim_cache size 0 max 16384 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.626+0000 7fb3dfcdf740 1 Processor -- start 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.626+0000 7fb3dfcdf740 1 -- start start 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.626+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d1105d0 0x559d0d10cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.626+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d10bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.626+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b12c0 con 0x559d0d11b370 2024-09-05T06:55:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b2f20 con 0x559d0d111b40 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b1990 con 0x559d0d1105d0 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47920/0 (socket says 172.21.15.86:47920) 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 -- 172.21.15.86:0/3417185135 learned_addr learned my addr 172.21.15.86:0/3417185135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d1105d0 0x559d0d10cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d10bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 -- 172.21.15.86:0/3417185135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d1105d0 msgr2=0x559d0d10cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d1105d0 0x559d0d10cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 -- 172.21.15.86:0/3417185135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 msgr2=0x559d0d10bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d10bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 -- 172.21.15.86:0/3417185135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0d118950 con 0x559d0d11b370 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006010 tx=0x7fb3d0003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb263cbd0a2608f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3bd7fa640 1 -- 172.21.15.86:0/3417185135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0005260 con 0x559d0d11b370 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3bd7fa640 1 -- 172.21.15.86:0/3417185135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d00058b0 con 0x559d0d11b370 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.627+0000 7fb3bd7fa640 1 -- 172.21.15.86:0/3417185135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000fe90 con 0x559d0d11b370 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3417185135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 msgr2=0x559d0d10c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006010 tx=0x7fb3d0003c70 comp rx=0 tx=0).stop 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3417185135 shutdown_connections 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d11b370 0x559d0d10c130 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d111b40 0x559d0d10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/3417185135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d1105d0 0x559d0d10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3417185135 >> 172.21.15.86:0/3417185135 conn(0x559d0d0d9310 msgr2=0x559d0d10b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3417185135 shutdown_connections 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/3417185135 wait complete. 2024-09-05T06:55:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.628+0000 7fb3dfcdf740 1 -- start start 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 Processor -- start 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 0x559d0d114a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d11b370 0x559d0d114f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b2790 con 0x559d0d1105d0 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b2090 con 0x559d0d11b370 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81af2c0 con 0x559d0d111b40 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3dfcdf740 10 client.0 _ll_register_callbacks cb 0x7fb3c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47936/0 (socket says 172.21.15.86:47936) 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/1283493614 learned_addr learned my addr 172.21.15.86:0/1283493614 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d77fe640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d11b370 0x559d0d114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 0x559d0d114a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/1283493614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 msgr2=0x559d0d114a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 0x559d0d114a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d11b370 msgr2=0x559d0d114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d11b370 0x559d0d114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 -- 172.21.15.86:0/1283493614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0d1154b0 con 0x559d0d1105d0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 0x559d0d114a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3d6ffd640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80057c0 tx=0x7fb3c8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87c9e78f92073fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.629+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c804d070 con 0x559d0d1105d0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9605+0+0 (secure 0 0 0) 0x7fb3c8003980 con 0x559d0d1105d0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d00058b0 con 0x559d0d1105d0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3c8070050 con 0x559d0d1105d0 2024-09-05T06:55:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 10 client.18660 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559d0d195c20 con 0x559d0d1105d0 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d00058b0 con 0x559d0d1105d0 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb3c80052b0 con 0x559d0d1105d0 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 10 client.18660 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 10 client.18660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 20 client.18660 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d1963d0 0x559d0d1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 4 client.18660 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d0d07dd70 con 0x559d0d1963d0 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d1963d0 0x559d0d1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3b6ffd640 10 client.18660 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.630+0000 7fb3d67fc640 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d1963d0 0x559d0d1987f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a666dbea3c470784 server_cookie=ee03eed1d05c44fc in_seq=0 out_seq=0 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3b6ffd640 1 -- 172.21.15.86:0/1283493614 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d0d07dd70 con 0x559d0d1963d0 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3b6ffd640 10 client.18660 handle_command_reply: tid=0 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 client.18660 shutdown 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 msgr2=0x559d0d123900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80057c0 tx=0x7fb3c8041000 comp rx=0 tx=0).stop 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 shutdown_connections 2024-09-05T06:55:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x559d0d1963d0 0x559d0d1987f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3b6ffd640 0 client.18660 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d0d11b370 0x559d0d114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559d0d111b40 0x559d0d114a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 --2- 172.21.15.86:0/1283493614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0d1105d0 0x559d0d123900 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 >> 172.21.15.86:0/1283493614 conn(0x559d0d0d9310 msgr2=0x559d0d10bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 shutdown_connections 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 1 -- 172.21.15.86:0/1283493614 wait complete. 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.631+0000 7fb3dfcdf740 20 client.18660 trim_cache size 0 max 16384 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 msgr2=0x7fb3c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 0x7fb3c00a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8031440 tx=0x7fb3c80058d0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 msgr2=0x7fb3d8100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 0x7fb3d8100f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e9f0 tx=0x7fb3cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 shutdown_connections 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb3c00a2cf0 0x7fb3c00a51b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3d810c170 0x7fb3d81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d81036f0 0x7fb3d8100f70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 --2- 172.21.15.86:0/3922116775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102dd0 0x7fb3d8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 >> 172.21.15.86:0/3922116775 conn(0x7fb3d80fc790 msgr2=0x7fb3d80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 shutdown_connections 2024-09-05T06:55:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:03.632+0000 7fb3dd906640 1 -- 172.21.15.86:0/3922116775 wait complete. 2024-09-05T06:55:04.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3827 17.95 MB/sec warmup 22 sec latency 143.302 ms 2024-09-05T06:55:05.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:05 smithi115 ceph-mon[34791]: pgmap v431: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:05 smithi086 ceph-mon[30943]: pgmap v431: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:05.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:05 smithi096 ceph-mon[34468]: pgmap v431: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:05.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3970 17.33 MB/sec warmup 23 sec latency 379.793 ms 2024-09-05T06:55:05.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:55:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:06.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.744+0000 7fd425e26640 1 -- 172.21.15.86:0/627171787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420101de0 msgr2=0x7fd4201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.744+0000 7fd425e26640 1 --2- 172.21.15.86:0/627171787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420101de0 0x7fd4201021c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd41000dc90 tx=0x7fd41002e370 comp rx=0 tx=0).stop 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 -- 172.21.15.86:0/627171787 shutdown_connections 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 --2- 172.21.15.86:0/627171787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 --2- 172.21.15.86:0/627171787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420102790 0x7fd4201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 --2- 172.21.15.86:0/627171787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420101de0 0x7fd4201021c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 -- 172.21.15.86:0/627171787 >> 172.21.15.86:0/627171787 conn(0x7fd4200fd8e0 msgr2=0x7fd4200ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 -- 172.21.15.86:0/627171787 shutdown_connections 2024-09-05T06:55:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 -- 172.21.15.86:0/627171787 wait complete. 2024-09-05T06:55:06.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4097 16.73 MB/sec warmup 24 sec latency 1379.826 ms 2024-09-05T06:55:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.745+0000 7fd425e26640 1 Processor -- start 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 -- start start 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 0x7fd420196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd42010fc30 con 0x7fd420106f10 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd42010fab0 con 0x7fd420102790 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd425e26640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd42010fdb0 con 0x7fd420101de0 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 0x7fd420196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46976/0 (socket says 172.21.15.86:46976) 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd417fff640 1 -- 172.21.15.86:0/2990655477 learned_addr learned my addr 172.21.15.86:0/2990655477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 -- 172.21.15.86:0/2990655477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 msgr2=0x7fd420196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 -- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 msgr2=0x7fd42019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd41f7fe640 1 -- 172.21.15.86:0/2990655477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42019b200 con 0x7fd420101de0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.746+0000 7fd417fff640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 0x7fd420196230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd410005ab0 tx=0x7fd41000d3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd41d7fa640 1 -- 172.21.15.86:0/2990655477 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000bad0 con 0x7fd420101de0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42019b490 con 0x7fd420101de0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd41d7fa640 1 -- 172.21.15.86:0/2990655477 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd41000bc70 con 0x7fd420101de0 2024-09-05T06:55:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd41d7fa640 1 -- 172.21.15.86:0/2990655477 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41003f740 con 0x7fd420101de0 2024-09-05T06:55:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.747+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42006bc10 con 0x7fd420101de0 2024-09-05T06:55:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.750+0000 7fd41d7fa640 1 -- 172.21.15.86:0/2990655477 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd410046020 con 0x7fd420101de0 2024-09-05T06:55:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.750+0000 7fd4281ff740 1 Processor -- start 2024-09-05T06:55:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.750+0000 7fd4281ff740 1 -- start start 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f50fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e76f4752f0 con 0x55e76f510b40 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e76f41ccf0 con 0x55e76f51a370 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd4281ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e76f4712b0 con 0x55e76f50f5d0 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46986/0 (socket says 172.21.15.86:46986) 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 -- 172.21.15.86:0/2832737820 learned_addr learned my addr 172.21.15.86:0/2832737820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47960/0 (socket says 172.21.15.86:47960) 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41d7fa640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 0x7fd40c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f50fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 -- 172.21.15.86:0/2832737820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 msgr2=0x55e76f50fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f50fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 -- 172.21.15.86:0/2832737820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 msgr2=0x55e76f50f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd417fff640 1 -- 172.21.15.86:0/2832737820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76f4013a0 con 0x55e76f51a370 2024-09-05T06:55:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.751+0000 7fd41d7fa640 1 -- 172.21.15.86:0/2990655477 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7fd4100f3130 con 0x7fd420101de0 2024-09-05T06:55:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.752+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 0x7fd40c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.752+0000 7fd417fff640 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4080115f0 tx=0x7fd408011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c72587b3c573238 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.752+0000 7fd415ffb640 1 -- 172.21.15.86:0/2832737820 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40801e070 con 0x55e76f51a370 2024-09-05T06:55:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.752+0000 7fd415ffb640 1 -- 172.21.15.86:0/2832737820 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd40800f660 con 0x55e76f51a370 2024-09-05T06:55:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.752+0000 7fd415ffb640 1 -- 172.21.15.86:0/2832737820 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408003850 con 0x55e76f51a370 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 -- 172.21.15.86:0/2832737820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 msgr2=0x55e76f51c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 0x7fd40c0a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd418005fd0 tx=0x7fd418004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4080115f0 tx=0x7fd408011ab0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 -- 172.21.15.86:0/2832737820 shutdown_connections 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51a370 0x55e76f51c760 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f50fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2832737820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f510b40 0x55e76f50f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 -- 172.21.15.86:0/2832737820 >> 172.21.15.86:0/2832737820 conn(0x55e76f4d8310 msgr2=0x55e76f50bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 -- 172.21.15.86:0/2832737820 shutdown_connections 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.753+0000 7fd4281ff740 1 -- 172.21.15.86:0/2832737820 wait complete. 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 -- start start 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 Processor -- start 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f50f5d0 0x55e76f519d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f517600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e76f51d070 con 0x55e76f51a370 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e76f430060 con 0x55e76f50f5d0 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd417fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f50f5d0 0x55e76f519d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd417fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44312/0 (socket says 172.21.15.86:44312) 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd417fff640 1 -- 172.21.15.86:0/327995380 learned_addr learned my addr 172.21.15.86:0/327995380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.754+0000 7fd4281ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e76f51d1f0 con 0x55e76f510b40 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd41ffff640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f517600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd4281ff740 10 client.0 _ll_register_callbacks cb 0x55e76f1838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 -- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f50f5d0 msgr2=0x55e76f519d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f50f5d0 0x55e76f519d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 -- 172.21.15.86:0/327995380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 msgr2=0x55e76f517600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f517600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 -- 172.21.15.86:0/327995380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76f1e63c0 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd41ffff640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f517600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd417fff640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd40801c040 tx=0x7fd408005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ecad500b1dd36a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd3feffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40801e040 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40800f660 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.755+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7fd408005460 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.756+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd408030050 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.756+0000 7fd4281ff740 10 client.27622 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.756+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e76f528af0 con 0x55e76f510b40 2024-09-05T06:55:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.756+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408027070 con 0x55e76f510b40 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd408021070 con 0x55e76f510b40 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 10 client.27622 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 10 client.27622 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 20 client.27622 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f529ef0 0x55e76f52c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 4 client.27622 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd40800f660 con 0x55e76f529ef0 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f529ef0 0x55e76f52c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f529ef0 0x55e76f52c2e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1aa2fe199b251e7e server_cookie=fb518a25df4734e9 in_seq=0 out_seq=0 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.757+0000 7fd3feffd640 10 client.27622 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.758+0000 7fd3feffd640 1 -- 172.21.15.86:0/327995380 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd40800f660 con 0x55e76f529ef0 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.758+0000 7fd3feffd640 10 client.27622 handle_command_reply: tid=0 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.758+0000 7fd4281ff740 1 client.27622 shutdown 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.758+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 msgr2=0x55e76f517010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.758+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd40801c040 tx=0x7fd408005270 comp rx=0 tx=0).stop 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 shutdown_connections 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f529ef0 0x55e76f52c2e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd3feffd640 0 client.27622 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f517600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f510b40 0x55e76f517010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 --2- 172.21.15.86:0/327995380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f50f5d0 0x55e76f519d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 >> 172.21.15.86:0/327995380 conn(0x55e76f4d8310 msgr2=0x55e76f511960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 shutdown_connections 2024-09-05T06:55:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 1 -- 172.21.15.86:0/327995380 wait complete. 2024-09-05T06:55:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.759+0000 7fd4281ff740 20 client.27622 trim_cache size 0 max 16384 2024-09-05T06:55:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 Processor -- start 2024-09-05T06:55:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 -- start start 2024-09-05T06:55:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 0x55e76f526310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f51a370 0x55e76f531f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b5990 con 0x55e76f50f5d0 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b75f0 con 0x55e76f510b40 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd4281ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b6060 con 0x55e76f51a370 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 0x55e76f526310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47014/0 (socket says 172.21.15.86:47014) 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 -- 172.21.15.86:0/2731967965 learned_addr learned my addr 172.21.15.86:0/2731967965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 0x55e76f526310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47978/0 (socket says 172.21.15.86:47978) 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f51a370 0x55e76f531f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 -- 172.21.15.86:0/2731967965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f51a370 msgr2=0x55e76f531f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f51a370 0x55e76f531f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 -- 172.21.15.86:0/2731967965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 msgr2=0x55e76f526310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 0x55e76f526310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.775+0000 7fd417fff640 1 -- 172.21.15.86:0/2731967965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76f515560 con 0x55e76f510b40 2024-09-05T06:55:06.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.776+0000 7fd417fff640 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd408005e00 tx=0x7fd408005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a3651a3df8d58b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.776+0000 7fd4157fa640 1 -- 172.21.15.86:0/2731967965 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40801e070 con 0x55e76f510b40 2024-09-05T06:55:06.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.776+0000 7fd4157fa640 1 -- 172.21.15.86:0/2731967965 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408027040 con 0x55e76f510b40 2024-09-05T06:55:06.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.776+0000 7fd4157fa640 1 -- 172.21.15.86:0/2731967965 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800fea0 con 0x55e76f510b40 2024-09-05T06:55:06.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- 172.21.15.86:0/2731967965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 msgr2=0x55e76f531a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd408005e00 tx=0x7fd408005970 comp rx=0 tx=0).stop 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- 172.21.15.86:0/2731967965 shutdown_connections 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f51a370 0x55e76f531f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f510b40 0x55e76f531a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2731967965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f50f5d0 0x55e76f526310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- 172.21.15.86:0/2731967965 >> 172.21.15.86:0/2731967965 conn(0x55e76f4d8310 msgr2=0x55e76f516100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- 172.21.15.86:0/2731967965 shutdown_connections 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- 172.21.15.86:0/2731967965 wait complete. 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.777+0000 7fd4281ff740 1 -- start start 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 Processor -- start 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f51e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f51e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b6e60 con 0x55e76f51a370 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201b6760 con 0x55e76f51ef60 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd420073ff0 con 0x55e76f50f5d0 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd4281ff740 10 client.0 _ll_register_callbacks cb 0x7fd40801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd417fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f51e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd417fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f51e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47980/0 (socket says 172.21.15.86:47980) 2024-09-05T06:55:06.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47020/0 (socket says 172.21.15.86:47020) 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd417fff640 1 -- 172.21.15.86:0/2332713622 learned_addr learned my addr 172.21.15.86:0/2332713622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f51e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 -- 172.21.15.86:0/2332713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 msgr2=0x55e76f51e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f51e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 -- 172.21.15.86:0/2332713622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 msgr2=0x55e76f51e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:06.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f51e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.778+0000 7fd41ffff640 1 -- 172.21.15.86:0/2332713622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76f59c940 con 0x55e76f51ef60 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f51e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd41ffff640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd418009fd0 tx=0x7fd418004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=367cb2ed8f9bfc57 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd3feffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418012660 con 0x55e76f51ef60 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd408027040 con 0x55e76f51ef60 2024-09-05T06:55:06.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7fd418012800 con 0x55e76f51ef60 2024-09-05T06:55:06.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd418012a00 con 0x55e76f51ef60 2024-09-05T06:55:06.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd4281ff740 10 client.27650 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:06.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.779+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e76f59dca0 con 0x55e76f51ef60 2024-09-05T06:55:06.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408027040 con 0x55e76f51ef60 2024-09-05T06:55:06.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd41801fce0 con 0x55e76f51ef60 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 10 client.27650 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 10 client.27650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 20 client.27650 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f59e470 0x55e76f5a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 4 client.27650 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e76f482860 con 0x55e76f59e470 2024-09-05T06:55:06.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.780+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f59e470 0x55e76f5a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd3feffd640 10 client.27650 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd41f7fe640 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f59e470 0x55e76f5a0890 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d18efc66be811238 server_cookie=e3f1a26d89fab81a in_seq=0 out_seq=0 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd3feffd640 1 -- 172.21.15.86:0/2332713622 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e76f482860 con 0x55e76f59e470 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd3feffd640 10 client.27650 handle_command_reply: tid=0 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 client.27650 shutdown 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 msgr2=0x55e76f59c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd418009fd0 tx=0x7fd418004ab0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 shutdown_connections 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e76f59e470 0x55e76f5a0890 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd3feffd640 0 client.27650 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f51ef60 0x55e76f59c230 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e76f51a370 0x55e76f51e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 --2- 172.21.15.86:0/2332713622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e76f50f5d0 0x55e76f51e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.781+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 >> 172.21.15.86:0/2332713622 conn(0x55e76f4d8310 msgr2=0x55e76f50a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 shutdown_connections 2024-09-05T06:55:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd4281ff740 1 -- 172.21.15.86:0/2332713622 wait complete. 2024-09-05T06:55:06.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd4281ff740 20 client.27650 trim_cache size 0 max 16384 2024-09-05T06:55:06.791 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 msgr2=0x7fd40c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 0x7fd40c0a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd418005fd0 tx=0x7fd418004a40 comp rx=0 tx=0).stop 2024-09-05T06:55:06.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 msgr2=0x7fd420196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 0x7fd420196230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd410005ab0 tx=0x7fd41000d3d0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 shutdown_connections 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd40c0a2cf0 0x7fd40c0a51b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420106f10 0x7fd42019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd420102790 0x7fd420196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 --2- 172.21.15.86:0/2990655477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd420101de0 0x7fd420196230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.782+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 >> 172.21.15.86:0/2990655477 conn(0x7fd4200fd8e0 msgr2=0x7fd4200feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.783+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 shutdown_connections 2024-09-05T06:55:06.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:06.783+0000 7fd425e26640 1 -- 172.21.15.86:0/2990655477 wait complete. 2024-09-05T06:55:07.338 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:07 smithi086 ceph-mon[30943]: pgmap v432: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:07.338 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:07 smithi086 ceph-mon[30943]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T06:55:07.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:07 smithi115 ceph-mon[34791]: pgmap v432: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:07 smithi115 ceph-mon[34791]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T06:55:07.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:55:07.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:07 smithi096 ceph-mon[34468]: pgmap v432: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-09-05T06:55:07.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:07 smithi096 ceph-mon[34468]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T06:55:07.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4255 16.26 MB/sec warmup 25 sec latency 2379.861 ms 2024-09-05T06:55:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.466+0000 7f956359e640 1 -- 172.21.15.86:0/1354275172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108020 msgr2=0x7f9564108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.466+0000 7f956359e640 1 --2- 172.21.15.86:0/1354275172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108020 0x7f9564108440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f955c009be0 tx=0x7f955c02a6d0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 -- 172.21.15.86:0/1354275172 shutdown_connections 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 --2- 172.21.15.86:0/1354275172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9564108980 0x7f9564114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 --2- 172.21.15.86:0/1354275172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108020 0x7f9564108440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 --2- 172.21.15.86:0/1354275172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f956410e060 0x7f956410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 -- 172.21.15.86:0/1354275172 >> 172.21.15.86:0/1354275172 conn(0x7f9564077980 msgr2=0x7f9564079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 -- 172.21.15.86:0/1354275172 shutdown_connections 2024-09-05T06:55:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.467+0000 7f956359e640 1 -- 172.21.15.86:0/1354275172 wait complete. 2024-09-05T06:55:08.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.469+0000 7f956359e640 1 Processor -- start 2024-09-05T06:55:08.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.469+0000 7f956359e640 1 -- start start 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.469+0000 7f956359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 0x7f95640829a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 0x7f956407ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956410e060 0x7f956407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 0x7f95640829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 0x7f95640829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44334/0 (socket says 172.21.15.86:44334) 2024-09-05T06:55:08.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 0x7f956407ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956259c640 1 -- 172.21.15.86:0/265731400 learned_addr learned my addr 172.21.15.86:0/265731400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f956407b340 con 0x7f956410e060 2024-09-05T06:55:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f956407b1c0 con 0x7f9564108980 2024-09-05T06:55:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f956407b4c0 con 0x7f9564108020 2024-09-05T06:55:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.470+0000 7f9562d9d640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956410e060 0x7f956407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 -- 172.21.15.86:0/265731400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 msgr2=0x7f95640829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 0x7f95640829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 -- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956410e060 msgr2=0x7f956407cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956410e060 0x7f956407cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956407d690 con 0x7f9564108980 2024-09-05T06:55:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.471+0000 7f9561d9b640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 0x7f956407ca20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f955c02acc0 tx=0x7f955c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.472+0000 7f95537fe640 1 -- 172.21.15.86:0/265731400 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c041020 con 0x7f9564108980 2024-09-05T06:55:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.472+0000 7f95537fe640 1 -- 172.21.15.86:0/265731400 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955c037040 con 0x7f9564108980 2024-09-05T06:55:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.472+0000 7f95537fe640 1 -- 172.21.15.86:0/265731400 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c036070 con 0x7f9564108980 2024-09-05T06:55:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.473+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95641b36b0 con 0x7f9564108980 2024-09-05T06:55:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.473+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641b3a50 con 0x7f9564108980 2024-09-05T06:55:08.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.476+0000 7f95537fe640 1 -- 172.21.15.86:0/265731400 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f955c044020 con 0x7f9564108980 2024-09-05T06:55:08.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.479+0000 7f956b3c7740 1 Processor -- start 2024-09-05T06:55:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.479+0000 7f956b3c7740 1 -- start start 2024-09-05T06:55:08.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cb8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee76c1e2f0 con 0x55ee76cb9cf0 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee76bc5cf0 con 0x55ee76cc3520 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956b3c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ee76c1a2b0 con 0x55ee76cb8780 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44346/0 (socket says 172.21.15.86:44346) 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.480+0000 7f956259c640 1 -- 172.21.15.86:0/1431333386 learned_addr learned my addr 172.21.15.86:0/1431333386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 -- 172.21.15.86:0/1431333386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 msgr2=0x55ee76cc5910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc5910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 -- 172.21.15.86:0/1431333386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 msgr2=0x55ee76cb8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cb8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 -- 172.21.15.86:0/1431333386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee76baa3a0 con 0x55ee76cb8780 2024-09-05T06:55:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f95537fe640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 0x7f954c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f956259c640 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9558009d00 tx=0x7f9558011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d41c88d9ea502b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.481+0000 7f95537fe640 1 -- 172.21.15.86:0/265731400 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f955c0effb0 con 0x7f9564108980 2024-09-05T06:55:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.482+0000 7f9550ff9640 1 -- 172.21.15.86:0/1431333386 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955801e070 con 0x55ee76cb8780 2024-09-05T06:55:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.482+0000 7f9550ff9640 1 -- 172.21.15.86:0/1431333386 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9558019ae0 con 0x55ee76cb8780 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.483+0000 7f9550ff9640 1 -- 172.21.15.86:0/1431333386 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955800ea80 con 0x55ee76cb8780 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.483+0000 7f956b3c7740 1 -- 172.21.15.86:0/1431333386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 msgr2=0x55ee76cb8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.483+0000 7f956b3c7740 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9558009d00 tx=0x7f9558011810 comp rx=0 tx=0).stop 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f956b3c7740 1 -- 172.21.15.86:0/1431333386 shutdown_connections 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f956b3c7740 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f956b3c7740 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb8780 0x55ee76cb8bc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f956b3c7740 1 --2- 172.21.15.86:0/1431333386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cb8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f956b3c7740 1 -- 172.21.15.86:0/1431333386 >> 172.21.15.86:0/1431333386 conn(0x55ee76c81310 msgr2=0x55ee76cb4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.485+0000 7f9562d9d640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 0x7f954c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.486+0000 7f9562d9d640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 0x7f954c0a53c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9554005fd0 tx=0x7f95540046c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.490+0000 7f956b3c7740 1 -- 172.21.15.86:0/1431333386 shutdown_connections 2024-09-05T06:55:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.490+0000 7f956b3c7740 1 -- 172.21.15.86:0/1431333386 wait complete. 2024-09-05T06:55:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.490+0000 7f956b3c7740 1 -- start start 2024-09-05T06:55:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.490+0000 7f956b3c7740 1 Processor -- start 2024-09-05T06:55:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee76cc62c0 con 0x55ee76cb8780 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee76bd9060 con 0x55ee76cc3520 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ee76cc6440 con 0x55ee76cb9cf0 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.491+0000 7f956b3c7740 10 client.0 _ll_register_callbacks cb 0x55ee7692c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47060/0 (socket says 172.21.15.86:47060) 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 -- 172.21.15.86:0/2164235323 learned_addr learned my addr 172.21.15.86:0/2164235323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9561d9b640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 -- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb9cf0 msgr2=0x55ee76cc00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 -- 172.21.15.86:0/2164235323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 msgr2=0x55ee76cc2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.492+0000 7f9562d9d640 1 -- 172.21.15.86:0/2164235323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7698f3c0 con 0x55ee76cc3520 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.493+0000 7f9562d9d640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9554003ce0 tx=0x7f955400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8e3d2bb46c4739 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.493+0000 7f9549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.493+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955401e620 con 0x55ee76cc3520 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.493+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9558019ae0 con 0x55ee76cc3520 2024-09-05T06:55:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f955401e7c0 con 0x55ee76cc3520 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f955401e9c0 con 0x55ee76cc3520 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f956b3c7740 10 client.27659 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ee76ce2df0 con 0x55ee76cc3520 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9558019ae0 con 0x55ee76cc3520 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.494+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f9554026b80 con 0x55ee76cc3520 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 10 client.27659 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 10 client.27659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 20 client.27659 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76ce46b0 0x55ee76ce6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 4 client.27659 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.495+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee76c2b860 con 0x55ee76ce46b0 2024-09-05T06:55:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.496+0000 7f956259c640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76ce46b0 0x55ee76ce6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.497+0000 7f956259c640 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76ce46b0 0x55ee76ce6b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=237b90c2fccc4910 server_cookie=dcf80fdd4fd661ad in_seq=0 out_seq=0 2024-09-05T06:55:08.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.497+0000 7f9549ffb640 10 client.27659 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:55:08.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f9549ffb640 1 -- 172.21.15.86:0/2164235323 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee76c2b860 con 0x55ee76ce46b0 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f9549ffb640 10 client.27659 handle_command_reply: tid=0 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 client.27659 shutdown 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 msgr2=0x55ee76cc0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9554003ce0 tx=0x7f955400b260 comp rx=0 tx=0).stop 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 shutdown_connections 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76ce46b0 0x55ee76ce6b50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cc3520 0x55ee76cc0600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 --2- 172.21.15.86:0/2164235323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.499+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 >> 172.21.15.86:0/2164235323 conn(0x55ee76c81310 msgr2=0x55ee76cbaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.500+0000 7f9549ffb640 0 client.27659 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:55:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.500+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 shutdown_connections 2024-09-05T06:55:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.500+0000 7f956b3c7740 1 -- 172.21.15.86:0/2164235323 wait complete. 2024-09-05T06:55:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.500+0000 7f956b3c7740 20 client.27659 trim_cache size 0 max 16384 2024-09-05T06:55:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 Processor -- start 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 -- start start 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 0x55ee76cb4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cafb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b57f0 con 0x55ee76cb9cf0 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b7450 con 0x55ee76cb8780 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956b3c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b5ec0 con 0x55ee76cc3520 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 0x55ee76cb4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 0x55ee76cb4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47076/0 (socket says 172.21.15.86:47076) 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f956259c640 1 -- 172.21.15.86:0/64588070 learned_addr learned my addr 172.21.15.86:0/64588070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44374/0 (socket says 172.21.15.86:44374) 2024-09-05T06:55:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.517+0000 7f9561d9b640 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cafb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 -- 172.21.15.86:0/64588070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 msgr2=0x55ee76cb4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 0x55ee76cb4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 -- 172.21.15.86:0/64588070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 msgr2=0x55ee76cafb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cafb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 -- 172.21.15.86:0/64588070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee76cbf070 con 0x55ee76cc3520 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f9562d9d640 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f955400b480 tx=0x7f9554003fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3144cf26f3e58c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f954bfff640 1 -- 172.21.15.86:0/64588070 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540038c0 con 0x55ee76cc3520 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f954bfff640 1 -- 172.21.15.86:0/64588070 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955401ed60 con 0x55ee76cc3520 2024-09-05T06:55:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.518+0000 7f954bfff640 1 -- 172.21.15.86:0/64588070 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554026a70 con 0x55ee76cc3520 2024-09-05T06:55:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 -- 172.21.15.86:0/64588070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 msgr2=0x55ee76cbb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f955400b480 tx=0x7f9554003fd0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 -- 172.21.15.86:0/64588070 shutdown_connections 2024-09-05T06:55:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cbb2d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb9cf0 0x55ee76cafb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 --2- 172.21.15.86:0/64588070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb8780 0x55ee76cb4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 -- 172.21.15.86:0/64588070 >> 172.21.15.86:0/64588070 conn(0x55ee76c81310 msgr2=0x55ee76cb48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 -- 172.21.15.86:0/64588070 shutdown_connections 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.519+0000 7f956b3c7740 1 -- 172.21.15.86:0/64588070 wait complete. 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.520+0000 7f956b3c7740 1 -- start start 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.520+0000 7f956b3c7740 1 Processor -- start 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.520+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.520+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.520+0000 7f956b3c7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f956b3c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b6cc0 con 0x55ee76cb8780 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f956b3c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b65c0 con 0x55ee76cb9cf0 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f956b3c7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9564073ea0 con 0x55ee76cc3520 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f956b3c7740 10 client.0 _ll_register_callbacks cb 0x7f95540042e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44390/0 (socket says 172.21.15.86:44390) 2024-09-05T06:55:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47082/0 (socket says 172.21.15.86:47082) 2024-09-05T06:55:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 -- 172.21.15.86:0/4098844974 learned_addr learned my addr 172.21.15.86:0/4098844974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:08.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f956259c640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 -- 172.21.15.86:0/4098844974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 msgr2=0x55ee76cc6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 -- 172.21.15.86:0/4098844974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 msgr2=0x55ee76cc6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.521+0000 7f9562d9d640 1 -- 172.21.15.86:0/4098844974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee76d4d4d0 con 0x55ee76cc3520 2024-09-05T06:55:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f9562d9d640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9554026630 tx=0x7f9554005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be397fe06924e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f9549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540038c0 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f95540267f0 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955401ed60 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.522+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f95540269f0 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 10 client.27646 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ee76d4e7b0 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955401ed60 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f955401cc60 con 0x55ee76cc3520 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 10 client.27646 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 10 client.27646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:55:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 20 client.27646 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76d4ecb0 0x55ee76d510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 4 client.27646 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.523+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee76c25d70 con 0x55ee76d4ecb0 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f956259c640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76d4ecb0 0x55ee76d510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f956259c640 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76d4ecb0 0x55ee76d510d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce2cef351768a84b server_cookie=2ad245f2fa944077 in_seq=0 out_seq=0 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f9549ffb640 10 client.27646 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:55:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f9549ffb640 1 -- 172.21.15.86:0/4098844974 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee76c25d70 con 0x55ee76d4ecb0 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f9549ffb640 10 client.27646 handle_command_reply: tid=0 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.524+0000 7f956b3c7740 1 client.27646 shutdown 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 msgr2=0x55ee76cc71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9554026630 tx=0x7f9554005970 comp rx=0 tx=0).stop 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 shutdown_connections 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ee76d4ecb0 0x55ee76d510d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee76cc3520 0x55ee76cc71d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f9549ffb640 0 client.27646 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee76cb9cf0 0x55ee76cc6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 --2- 172.21.15.86:0/4098844974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee76cb8780 0x55ee76cc6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 >> 172.21.15.86:0/4098844974 conn(0x55ee76c81310 msgr2=0x55ee76cc2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 shutdown_connections 2024-09-05T06:55:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 1 -- 172.21.15.86:0/4098844974 wait complete. 2024-09-05T06:55:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.525+0000 7f956b3c7740 20 client.27646 trim_cache size 0 max 16384 2024-09-05T06:55:08.528 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 msgr2=0x7f954c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 0x7f954c0a53c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9554005fd0 tx=0x7f95540046c0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 msgr2=0x7f956407ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 0x7f956407ca20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f955c02acc0 tx=0x7f955c003f10 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 shutdown_connections 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f954c0a2f00 0x7f954c0a53c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956410e060 0x7f956407cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9564108980 0x7f956407ca20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 --2- 172.21.15.86:0/265731400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9564108020 0x7f95640829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 >> 172.21.15.86:0/265731400 conn(0x7f9564077980 msgr2=0x7f9564079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 shutdown_connections 2024-09-05T06:55:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:08.526+0000 7f956359e640 1 -- 172.21.15.86:0/265731400 wait complete. 2024-09-05T06:55:08.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4400 15.78 MB/sec warmup 26 sec latency 3379.895 ms 2024-09-05T06:55:09.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:09 smithi086 ceph-mon[30943]: pgmap v434: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:09.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:55:09.539 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:09 smithi115 ceph-mon[34791]: pgmap v434: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:09.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:09 smithi096 ceph-mon[34468]: pgmap v434: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:09.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4553 15.48 MB/sec warmup 27 sec latency 4379.928 ms 2024-09-05T06:55:09.927 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:10.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4674 15.05 MB/sec warmup 28 sec latency 4885.091 ms 2024-09-05T06:55:11.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:11 smithi086 ceph-mon[30943]: pgmap v435: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:11.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:11 smithi115 ceph-mon[34791]: pgmap v435: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 -- 172.21.15.86:0/2191661458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984100aa0 msgr2=0x7f398410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 --2- 172.21.15.86:0/2191661458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984100aa0 0x7f398410ea20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f397c009ca0 tx=0x7f397c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 -- 172.21.15.86:0/2191661458 shutdown_connections 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 --2- 172.21.15.86:0/2191661458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984100aa0 0x7f398410ea20 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 --2- 172.21.15.86:0/2191661458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 0x7f3984100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 --2- 172.21.15.86:0/2191661458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984106b80 0x7f3984106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 -- 172.21.15.86:0/2191661458 >> 172.21.15.86:0/2191661458 conn(0x7f3984077230 msgr2=0x7f39840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 -- 172.21.15.86:0/2191661458 shutdown_connections 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.108+0000 7f3988fe8640 1 -- 172.21.15.86:0/2191661458 wait complete. 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.109+0000 7f3988fe8640 1 Processor -- start 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.109+0000 7f3988fe8640 1 -- start start 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 0x7f3984198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984106b80 0x7f398419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3984111f00 con 0x7f3984106b80 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3984111d80 con 0x7f3984100100 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3988fe8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3984112080 con 0x7f3984100aa0 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3982d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984106b80 0x7f398419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f398259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 0x7f3984198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44402/0 (socket says 172.21.15.86:44402) 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 -- 172.21.15.86:0/1247461123 learned_addr learned my addr 172.21.15.86:0/1247461123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 -- 172.21.15.86:0/1247461123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 msgr2=0x7f3984198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 0x7f3984198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 -- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984106b80 msgr2=0x7f398419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984106b80 0x7f398419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 -- 172.21.15.86:0/1247461123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398419d330 con 0x7f3984100aa0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f397800d950 tx=0x7f397800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.110+0000 7f39737fe640 1 -- 172.21.15.86:0/1247461123 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978017070 con 0x7f3984100aa0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.111+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398419d620 con 0x7f3984100aa0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.111+0000 7f39737fe640 1 -- 172.21.15.86:0/1247461123 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978009c30 con 0x7f3984100aa0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.111+0000 7f39737fe640 1 -- 172.21.15.86:0/1247461123 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397800f620 con 0x7f3984100aa0 2024-09-05T06:55:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.111+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39841a4f00 con 0x7f3984100aa0 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.114+0000 7f39737fe640 1 -- 172.21.15.86:0/1247461123 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3978010460 con 0x7f3984100aa0 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.114+0000 7f398b3c1740 1 Processor -- start 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 -- start start 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 0x557f2b1c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 0x557f2b1d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f2b12f2f0 con 0x557f2b1d4370 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f2b0d6cf0 con 0x557f2b1cab40 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398b3c1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557f2b12b2b0 con 0x557f2b1c95d0 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f3982d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f3982d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47128/0 (socket says 172.21.15.86:47128) 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f3982d9d640 1 -- 172.21.15.86:0/872919604 learned_addr learned my addr 172.21.15.86:0/872919604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f3981d9b640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 0x557f2b1d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.115+0000 7f398259c640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 0x557f2b1c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3982d9d640 1 -- 172.21.15.86:0/872919604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 msgr2=0x557f2b1c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3982d9d640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 0x557f2b1c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3982d9d640 1 -- 172.21.15.86:0/872919604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 msgr2=0x557f2b1d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3982d9d640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 0x557f2b1d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3982d9d640 1 -- 172.21.15.86:0/872919604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2b0bb3a0 con 0x557f2b1cab40 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f39737fe640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 0x7f396c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f398259c640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 0x557f2b1c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.116+0000 7f3981d9b640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 0x557f2b1d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f398259c640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 0x7f396c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f39737fe640 1 -- 172.21.15.86:0/1247461123 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f39780cf4f0 con 0x7f3984100aa0 2024-09-05T06:55:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f3982d9d640 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f397c009ca0 tx=0x7f397c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777d363b1cf3a0dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f398259c640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 0x7f396c0a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x557f2b1c6ac0 tx=0x7f3974005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f3970ff9640 1 -- 172.21.15.86:0/872919604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c031e00 con 0x557f2b1cab40 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f3970ff9640 1 -- 172.21.15.86:0/872919604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c036030 con 0x557f2b1cab40 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.117+0000 7f3970ff9640 1 -- 172.21.15.86:0/872919604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c004370 con 0x557f2b1cab40 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 -- 172.21.15.86:0/872919604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 msgr2=0x557f2b1c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f397c009ca0 tx=0x7f397c009a20 comp rx=0 tx=0).stop 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 -- 172.21.15.86:0/872919604 shutdown_connections 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1d4370 0x557f2b1d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1c95d0 0x557f2b1c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 --2- 172.21.15.86:0/872919604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1c9090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 -- 172.21.15.86:0/872919604 >> 172.21.15.86:0/872919604 conn(0x557f2b192310 msgr2=0x557f2b1c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 -- 172.21.15.86:0/872919604 shutdown_connections 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.118+0000 7f398b3c1740 1 -- 172.21.15.86:0/872919604 wait complete. 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 -- start start 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 Processor -- start 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f2b1db2a0 con 0x557f2b1c95d0 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f2b0ea060 con 0x557f2b1cab40 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557f2b1db420 con 0x557f2b1d4370 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398b3c1740 10 client.0 _ll_register_callbacks cb 0x557f2ae3d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47130/0 (socket says 172.21.15.86:47130) 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f3981d9b640 1 -- 172.21.15.86:0/1523427727 learned_addr learned my addr 172.21.15.86:0/1523427727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f398259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48060/0 (socket says 172.21.15.86:48060) 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.119+0000 7f3982d9d640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 -- 172.21.15.86:0/1523427727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 msgr2=0x557f2b1d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 -- 172.21.15.86:0/1523427727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 msgr2=0x557f2b1d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 -- 172.21.15.86:0/1523427727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2aea03c0 con 0x557f2b1cab40 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3981d9b640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f39780cc7b0 tx=0x7f3978005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e712949a4d6d81ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978020070 con 0x557f2b1cab40 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.120+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f39780109a0 con 0x557f2b1cab40 2024-09-05T06:55:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c036030 con 0x557f2b1cab40 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f397809b110 con 0x557f2b1cab40 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 10 client.27677 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f2b1eb1c0 con 0x557f2b1cab40 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c036030 con 0x557f2b1cab40 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f39780cf100 con 0x557f2b1cab40 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 10 client.27677 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 10 client.27677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 20 client.27677 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b1ec670 0x557f2b1eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 4 client.27677 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.121+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f2b13c860 con 0x557f2b1ec670 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.122+0000 7f398259c640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b1ec670 0x557f2b1eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.122+0000 7f3969ffb640 10 client.27677 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.122+0000 7f398259c640 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b1ec670 0x557f2b1eead0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f47c8bd467b6e34 server_cookie=feaa3cd13a58a06b in_seq=0 out_seq=0 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f3969ffb640 1 -- 172.21.15.86:0/1523427727 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f2b13c860 con 0x557f2b1ec670 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f3969ffb640 10 client.27677 handle_command_reply: tid=0 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 client.27677 shutdown 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 msgr2=0x557f2b1d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f39780cc7b0 tx=0x7f3978005bf0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 shutdown_connections 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b1ec670 0x557f2b1eead0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1d1000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 --2- 172.21.15.86:0/1523427727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 >> 172.21.15.86:0/1523427727 conn(0x557f2b192310 msgr2=0x557f2b1cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.123+0000 7f3969ffb640 0 client.27677 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.124+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 shutdown_connections 2024-09-05T06:55:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.124+0000 7f398b3c1740 1 -- 172.21.15.86:0/1523427727 wait complete. 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.124+0000 7f398b3c1740 20 client.27677 trim_cache size 0 max 16384 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 Processor -- start 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 -- start start 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1c95d0 0x557f2b1d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3984073a50 con 0x557f2b1cab40 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39841a63d0 con 0x557f2b1c95d0 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f398b3c1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3984074120 con 0x557f2b1d4370 2024-09-05T06:55:11.514 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.141+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48066/0 (socket says 172.21.15.86:48066) 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f398259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1c95d0 0x557f2b1d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 -- 172.21.15.86:0/3747523902 learned_addr learned my addr 172.21.15.86:0/3747523902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3982d9d640 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 -- 172.21.15.86:0/3747523902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 msgr2=0x557f2b1cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 -- 172.21.15.86:0/3747523902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1c95d0 msgr2=0x557f2b1d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1c95d0 0x557f2b1d9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 -- 172.21.15.86:0/3747523902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2b1cc660 con 0x557f2b1cab40 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.142+0000 7f3981d9b640 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3978097fa0 tx=0x7f39780971b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9695feb848f0e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.209+0000 7f396bfff640 1 -- 172.21.15.86:0/3747523902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978020040 con 0x557f2b1cab40 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.209+0000 7f396bfff640 1 -- 172.21.15.86:0/3747523902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978009de0 con 0x557f2b1cab40 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.209+0000 7f396bfff640 1 -- 172.21.15.86:0/3747523902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397809af30 con 0x557f2b1cab40 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 -- 172.21.15.86:0/3747523902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 msgr2=0x557f2b1d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3978097fa0 tx=0x7f39780971b0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 -- 172.21.15.86:0/3747523902 shutdown_connections 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1cab40 0x557f2b1d2860 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 --2- 172.21.15.86:0/3747523902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1c95d0 0x557f2b1d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 -- 172.21.15.86:0/3747523902 >> 172.21.15.86:0/3747523902 conn(0x557f2b192310 msgr2=0x557f2b1c4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 -- 172.21.15.86:0/3747523902 shutdown_connections 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.210+0000 7f398b3c1740 1 -- 172.21.15.86:0/3747523902 wait complete. 2024-09-05T06:55:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 -- start start 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 Processor -- start 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39841a5c40 con 0x557f2b1c95d0 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39841a5540 con 0x557f2b1cab40 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3984071a50 con 0x557f2b1d4370 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398b3c1740 10 client.0 _ll_register_callbacks cb 0x7f397800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48074/0 (socket says 172.21.15.86:48074) 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f398259c640 1 -- 172.21.15.86:0/2086258164 learned_addr learned my addr 172.21.15.86:0/2086258164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f3981d9b640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.211+0000 7f3982d9d640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 -- 172.21.15.86:0/2086258164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 msgr2=0x557f2b1e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 -- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 msgr2=0x557f2b1e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 -- 172.21.15.86:0/2086258164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2b1e52a0 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398259c640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f397401d5b0 tx=0x7f3974017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07d63a40e3f71d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f3969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397401dda0 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9693+0+0 (secure 0 0 0) 0x7f3974004020 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.212+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978009de0 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3974004220 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 10 client.18705 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f2b257050 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3978009de0 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f397402b4c0 con 0x557f2b1c95d0 2024-09-05T06:55:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 10 client.18705 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 10 client.18705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 20 client.18705 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b257890 0x557f2b259cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 4 client.18705 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f2b136d70 con 0x557f2b257890 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.213+0000 7f3981d9b640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b257890 0x557f2b259cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.214+0000 7f3969ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.214+0000 7f3981d9b640 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b257890 0x557f2b259cb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a848967dca007295 server_cookie=ccc4f0bff8233a08 in_seq=0 out_seq=0 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f3969ffb640 1 -- 172.21.15.86:0/2086258164 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f2b136d70 con 0x557f2b257890 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f3969ffb640 10 client.18705 handle_command_reply: tid=0 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 client.18705 shutdown 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 msgr2=0x557f2b1e9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f397401d5b0 tx=0x7f3974017000 comp rx=0 tx=0).stop 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 shutdown_connections 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557f2b257890 0x557f2b259cb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557f2b1d4370 0x557f2b1e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f2b1cab40 0x557f2b1e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 --2- 172.21.15.86:0/2086258164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f2b1c95d0 0x557f2b1e9480 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 >> 172.21.15.86:0/2086258164 conn(0x557f2b192310 msgr2=0x557f2b1c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f3969ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 shutdown_connections 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 1 -- 172.21.15.86:0/2086258164 wait complete. 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.216+0000 7f398b3c1740 20 client.18705 trim_cache size 0 max 16384 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 msgr2=0x7f396c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 0x7f396c0a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x557f2b1c6ac0 tx=0x7f3974005f90 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 msgr2=0x7f39841988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f397800d950 tx=0x7f397800de10 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 shutdown_connections 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f396c0a2cf0 0x7f396c0a51b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3984106b80 0x7f398419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3984100aa0 0x7f39841988c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 --2- 172.21.15.86:0/1247461123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3984100100 0x7f3984198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 >> 172.21.15.86:0/1247461123 conn(0x7f3984077230 msgr2=0x7f39840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 shutdown_connections 2024-09-05T06:55:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:11.217+0000 7f3988fe8640 1 -- 172.21.15.86:0/1247461123 wait complete. 2024-09-05T06:55:11.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:11 smithi096 ceph-mon[34468]: pgmap v435: 129 pgs: 129 active+clean; 532 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 231 op/s 2024-09-05T06:55:11.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4844 14.92 MB/sec warmup 29 sec latency 144.825 ms 2024-09-05T06:55:12.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5023 14.76 MB/sec warmup 30 sec latency 120.468 ms 2024-09-05T06:55:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:13 smithi096 ceph-mon[34468]: pgmap v436: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-05T06:55:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:13 smithi096 ceph-mon[34468]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T06:55:13.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:55:13.330 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:55:13.358 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:13 smithi086 ceph-mon[30943]: pgmap v436: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-05T06:55:13.358 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:13 smithi086 ceph-mon[30943]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T06:55:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:13 smithi115 ceph-mon[34791]: pgmap v436: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-05T06:55:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:13 smithi115 ceph-mon[34791]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T06:55:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:13.745 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5192 14.43 MB/sec warmup 31 sec latency 66.086 ms 2024-09-05T06:55:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 -- 172.21.15.86:0/23055705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4101dd0 msgr2=0x7fcbe41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 --2- 172.21.15.86:0/23055705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4101dd0 0x7fcbe41021b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009900 tx=0x7fcbd402a410 comp rx=0 tx=0).stop 2024-09-05T06:55:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 -- 172.21.15.86:0/23055705 shutdown_connections 2024-09-05T06:55:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 --2- 172.21.15.86:0/23055705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 0x7fcbe410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 --2- 172.21.15.86:0/23055705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4102780 0x7fcbe41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 --2- 172.21.15.86:0/23055705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4101dd0 0x7fcbe41021b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.231+0000 7fcbea53d640 1 -- 172.21.15.86:0/23055705 >> 172.21.15.86:0/23055705 conn(0x7fcbe40fd8b0 msgr2=0x7fcbe40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.232+0000 7fcbea53d640 1 -- 172.21.15.86:0/23055705 shutdown_connections 2024-09-05T06:55:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.232+0000 7fcbea53d640 1 -- 172.21.15.86:0/23055705 wait complete. 2024-09-05T06:55:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.232+0000 7fcbea53d640 1 Processor -- start 2024-09-05T06:55:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.232+0000 7fcbea53d640 1 -- start start 2024-09-05T06:55:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 0x7fcbe419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 0x7fcbe419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4113fd0 con 0x7fcbe4102780 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4113e50 con 0x7fcbe4101dd0 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbea53d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4114150 con 0x7fcbe4106f00 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39650/0 (socket says 172.21.15.86:39650) 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 -- 172.21.15.86:0/3445660576 learned_addr learned my addr 172.21.15.86:0/3445660576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 0x7fcbe419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 -- 172.21.15.86:0/3445660576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 msgr2=0x7fcbe419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 0x7fcbe419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 -- 172.21.15.86:0/3445660576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 msgr2=0x7fcbe419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 0x7fcbe419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 0x7fcbe419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 -- 172.21.15.86:0/3445660576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe419f500 con 0x7fcbe4102780 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 0x7fcbe419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.233+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00e9f0 tx=0x7fcbcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.234+0000 7fcbe1ffb640 1 -- 172.21.15.86:0/3445660576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00cd80 con 0x7fcbe4102780 2024-09-05T06:55:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.234+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe419f7f0 con 0x7fcbe4102780 2024-09-05T06:55:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.234+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe41a70d0 con 0x7fcbe4102780 2024-09-05T06:55:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.234+0000 7fcbe1ffb640 1 -- 172.21.15.86:0/3445660576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbcc0108e0 con 0x7fcbe4102780 2024-09-05T06:55:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.234+0000 7fcbe1ffb640 1 -- 172.21.15.86:0/3445660576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc018920 con 0x7fcbe4102780 2024-09-05T06:55:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 Processor -- start 2024-09-05T06:55:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 -- start start 2024-09-05T06:55:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 0x5558692c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555869221280 con 0x5558692bbb60 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558691c8b10 con 0x5558692c6d40 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbec916740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55586921d240 con 0x5558692bc480 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbe1ffb640 1 -- 172.21.15.86:0/3445660576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcbcc018ac0 con 0x7fcbe4102780 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.238+0000 7fcbe8ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbe8ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39654/0 (socket says 172.21.15.86:39654) 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbe8ada640 1 -- 172.21.15.86:0/4277869319 learned_addr learned my addr 172.21.15.86:0/4277869319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 0x5558692c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbe8ada640 1 -- 172.21.15.86:0/4277869319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 msgr2=0x5558692c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.239+0000 7fcbe1ffb640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 0x7fcbd00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 -- 172.21.15.86:0/4277869319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 msgr2=0x5558692c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 0x5558692c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 -- 172.21.15.86:0/4277869319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558691d51f0 con 0x5558692bbb60 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 0x5558692c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe1ffb640 1 -- 172.21.15.86:0/3445660576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fcbcc0cfac0 con 0x7fcbe4102780 2024-09-05T06:55:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 0x7fcbd00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009f50 tx=0x7fcbdc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0f73638284ff6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.240+0000 7fcbda7fc640 1 -- 172.21.15.86:0/4277869319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00f680 con 0x5558692bbb60 2024-09-05T06:55:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.241+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 0x7fcbd00a52b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc012fd0 tx=0x7fcbdc021000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.241+0000 7fcbda7fc640 1 -- 172.21.15.86:0/4277869319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbdc003cb0 con 0x5558692bbb60 2024-09-05T06:55:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.241+0000 7fcbda7fc640 1 -- 172.21.15.86:0/4277869319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01bb60 con 0x5558692bbb60 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- 172.21.15.86:0/4277869319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 msgr2=0x5558692bbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009f50 tx=0x7fcbdc0118d0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- 172.21.15.86:0/4277869319 shutdown_connections 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692c6d40 0x5558692c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 --2- 172.21.15.86:0/4277869319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692bbf40 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- 172.21.15.86:0/4277869319 >> 172.21.15.86:0/4277869319 conn(0x5558692b80f0 msgr2=0x5558692b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- 172.21.15.86:0/4277869319 shutdown_connections 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- 172.21.15.86:0/4277869319 wait complete. 2024-09-05T06:55:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 -- start start 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.242+0000 7fcbec916740 1 Processor -- start 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bc480 0x5558692c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 0x5558692c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558692c9910 con 0x5558692bbb60 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558691dbf10 con 0x5558692bc480 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5558692c9a90 con 0x5558692c6d40 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbec916740 10 client.0 _ll_register_callbacks cb 0x5558691a4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bc480 0x5558692c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692c3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39662/0 (socket says 172.21.15.86:39662) 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1674143546 learned_addr learned my addr 172.21.15.86:0/1674143546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 0x5558692c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bc480 msgr2=0x5558692c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bc480 0x5558692c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1674143546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 msgr2=0x5558692c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.243+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1674143546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555868f926a0 con 0x5558692c6d40 2024-09-05T06:55:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.244+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 0x5558692c4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009230 tx=0x7fcbdc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c606310668e3dafe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.244+0000 7fcbc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:55:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.244+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc009be0 con 0x5558692c6d40 2024-09-05T06:55:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.244+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fcbdc00fb50 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.244+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55586922e7f0 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.245+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcbdc00fd70 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.245+0000 7fcbec916740 10 client.27676 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.245+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5558692d5510 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.245+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55586922e7f0 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.245+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fcbdc025780 con 0x5558692c6d40 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.246+0000 7fcbec916740 10 client.27676 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.246+0000 7fcbec916740 10 client.27676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.246+0000 7fcbec916740 20 client.27676 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5558692d68c0 0x5558692d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbec916740 4 client.27676 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555869228d00 con 0x5558692d68c0 2024-09-05T06:55:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5558692d68c0 0x5558692d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5558692d68c0 0x5558692d8cb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83689829021b4930 server_cookie=7c40fda22c95402c in_seq=0 out_seq=0 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.247+0000 7fcbc37fe640 10 client.27676 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.248+0000 7fcbc37fe640 1 -- 172.21.15.86:0/1674143546 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555869228d00 con 0x5558692d68c0 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.248+0000 7fcbc37fe640 10 client.27676 handle_command_reply: tid=0 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.248+0000 7fcbec916740 1 client.27676 shutdown 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 msgr2=0x5558692c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 0x5558692c4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009230 tx=0x7fcbdc005bd0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 shutdown_connections 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5558692d68c0 0x5558692d8cb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692c6d40 0x5558692c4600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bc480 0x5558692c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 --2- 172.21.15.86:0/1674143546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692bbb60 0x5558692c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbc37fe640 0 client.27676 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 >> 172.21.15.86:0/1674143546 conn(0x5558692b80f0 msgr2=0x5558692b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 shutdown_connections 2024-09-05T06:55:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 1 -- 172.21.15.86:0/1674143546 wait complete. 2024-09-05T06:55:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.249+0000 7fcbec916740 20 client.27676 trim_cache size 0 max 16384 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.280+0000 7fcbec916740 1 Processor -- start 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.280+0000 7fcbec916740 1 -- start start 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39666/0 (socket says 172.21.15.86:39666) 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1927561825 learned_addr learned my addr 172.21.15.86:0/1927561825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc40101b0 con 0x5558692c6d40 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4011e10 con 0x5558692bbb60 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbdbfff640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4010880 con 0x5558692bc480 2024-09-05T06:55:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1927561825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 msgr2=0x5558692c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1927561825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 msgr2=0x5558692be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe8ada640 1 -- 172.21.15.86:0/1927561825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558692bc2b0 con 0x5558692c6d40 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.281+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.282+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc025b50 tx=0x7fcbdc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3b117657040699 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.282+0000 7fcbd9ffb640 1 -- 172.21.15.86:0/1927561825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc009b30 con 0x5558692c6d40 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.282+0000 7fcbd9ffb640 1 -- 172.21.15.86:0/1927561825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbdc04b070 con 0x5558692c6d40 2024-09-05T06:55:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.282+0000 7fcbd9ffb640 1 -- 172.21.15.86:0/1927561825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01b550 con 0x5558692c6d40 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 msgr2=0x5558692de4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc025b50 tx=0x7fcbdc009500 comp rx=0 tx=0).stop 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 shutdown_connections 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692de4e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 --2- 172.21.15.86:0/1927561825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 >> 172.21.15.86:0/1927561825 conn(0x5558692b80f0 msgr2=0x5558692c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 shutdown_connections 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.283+0000 7fcbec916740 1 -- 172.21.15.86:0/1927561825 wait complete. 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbec916740 1 -- start start 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbec916740 1 Processor -- start 2024-09-05T06:55:14.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbec916740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbec916740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbec916740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.284+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33486/0 (socket says 172.21.15.86:33486) 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:46864/0 (socket says 172.21.15.86:46864) 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbec916740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4011680 con 0x5558692c6d40 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4010f80 con 0x5558692bbb60 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3356850420 learned_addr learned my addr 172.21.15.86:0/3356850420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc400e1b0 con 0x5558692bc480 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbec916740 10 client.0 _ll_register_callbacks cb 0x7fcbcc0d3370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 -- 172.21.15.86:0/3356850420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 msgr2=0x5558692ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 -- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 msgr2=0x5558692cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.285+0000 7fcbe8ada640 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558693404b0 con 0x5558692c6d40 2024-09-05T06:55:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbe8ada640 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692cb070 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009230 tx=0x7fcbdc025d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71db4bc9212d8022 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00fab0 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fcbdc032a10 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbdc04b070 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcbdc05c050 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbec916740 10 client.18723 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.286+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5558693416a0 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbdc029050 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fcbdc032c10 con 0x5558692c6d40 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 10 client.18723 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 20 client.18723 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555869341c60 0x555869344080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 4 client.18723 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fcbdc04b070 con 0x555869341c60 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.287+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555869341c60 0x555869344080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.288+0000 7fcbe3fff640 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555869341c60 0x555869344080 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2d034c83b4cfcb server_cookie=8c57e7c98d07783f in_seq=0 out_seq=0 2024-09-05T06:55:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.288+0000 7fcbc37fe640 10 client.18723 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.289+0000 7fcbc37fe640 1 -- 172.21.15.86:0/3356850420 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcbdc04b070 con 0x555869341c60 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.289+0000 7fcbc37fe640 10 client.18723 handle_command_reply: tid=0 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.289+0000 7fcbec916740 1 client.18723 shutdown 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 msgr2=0x5558692cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692cb070 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009230 tx=0x7fcbdc025d40 comp rx=0 tx=0).stop 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 shutdown_connections 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555869341c60 0x555869344080 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558692c6d40 0x5558692cb070 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558692bc480 0x5558692ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbc37fe640 0 client.18723 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 --2- 172.21.15.86:0/3356850420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558692bbb60 0x5558692cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.290+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 >> 172.21.15.86:0/3356850420 conn(0x5558692b80f0 msgr2=0x5558692c42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.291+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 shutdown_connections 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.291+0000 7fcbec916740 1 -- 172.21.15.86:0/3356850420 wait complete. 2024-09-05T06:55:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.291+0000 7fcbec916740 20 client.18723 trim_cache size 0 max 16384 2024-09-05T06:55:14.293 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:55:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6448d741-085a-4d0f-b9cb-04dd128b8eca", 2024-09-05T06:55:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:55:14.293 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 msgr2=0x7fcbd00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 0x7fcbd00a52b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc012fd0 tx=0x7fcbdc021000 comp rx=0 tx=0).stop 2024-09-05T06:55:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 msgr2=0x7fcbe419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00e9f0 tx=0x7fcbcc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 shutdown_connections 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcbd00a2df0 0x7fcbd00a52b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbe4106f00 0x7fcbe419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4102780 0x7fcbe419aa90 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 --2- 172.21.15.86:0/3445660576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbe4101dd0 0x7fcbe419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 >> 172.21.15.86:0/3445660576 conn(0x7fcbe40fd8b0 msgr2=0x7fcbe40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 shutdown_connections 2024-09-05T06:55:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:14.292+0000 7fcbea53d640 1 -- 172.21.15.86:0/3445660576 wait complete. 2024-09-05T06:55:14.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5365 14.13 MB/sec warmup 32 sec latency 84.159 ms 2024-09-05T06:55:15.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:55:15.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: pgmap v438: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-05T06:55:15.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:55:15.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: pgmap v438: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:15.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.308 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: pgmap v438: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:15.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5552 13.93 MB/sec warmup 33 sec latency 81.340 ms 2024-09-05T06:55:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:16 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:16 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:55:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:16 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:16 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:55:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:16 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:16 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:55:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:55:16.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5724 13.70 MB/sec warmup 34 sec latency 119.430 ms 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.821+0000 7f3ea3d53640 1 -- 172.21.15.86:0/1220765142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 msgr2=0x7f3e9c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.821+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/1220765142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c10da40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009ca0 tx=0x7f3e9402a950 comp rx=0 tx=0).stop 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 -- 172.21.15.86:0/1220765142 shutdown_connections 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/1220765142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c10da40 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/1220765142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c102570 0x7f3e9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/1220765142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c100e20 0x7f3e9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 -- 172.21.15.86:0/1220765142 >> 172.21.15.86:0/1220765142 conn(0x7f3e9c0fc7d0 msgr2=0x7f3e9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 -- 172.21.15.86:0/1220765142 shutdown_connections 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.822+0000 7f3ea3d53640 1 -- 172.21.15.86:0/1220765142 wait complete. 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 Processor -- start 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 -- start start 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c102570 0x7f3e9c107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33498/0 (socket says 172.21.15.86:33498) 2024-09-05T06:55:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea1aef640 1 -- 172.21.15.86:0/2530937309 learned_addr learned my addr 172.21.15.86:0/2530937309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c1084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110e30 con 0x7f3e9c102ef0 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110cb0 con 0x7f3e9c100e20 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c110fb0 con 0x7f3e9c102570 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.823+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c102570 0x7f3e9c107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c1084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 -- 172.21.15.86:0/2530937309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c102570 msgr2=0x7f3e9c107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c102570 0x7f3e9c107f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 -- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 msgr2=0x7f3e9c1084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c1084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c1aac00 con 0x7f3e9c100e20 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ea50 tx=0x7f3e9800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3e92ffd640 1 -- 172.21.15.86:0/2530937309 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800cd60 con 0x7f3e9c100e20 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3e92ffd640 1 -- 172.21.15.86:0/2530937309 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e980054d0 con 0x7f3e9c100e20 2024-09-05T06:55:16.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c1aaef0 con 0x7f3e9c100e20 2024-09-05T06:55:16.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.824+0000 7f3e92ffd640 1 -- 172.21.15.86:0/2530937309 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98010430 con 0x7f3e9c100e20 2024-09-05T06:55:16.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.825+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c1ab3a0 con 0x7f3e9c100e20 2024-09-05T06:55:16.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.827+0000 7f3e92ffd640 1 -- 172.21.15.86:0/2530937309 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3e98005020 con 0x7f3e9c100e20 2024-09-05T06:55:16.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 Processor -- start 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 -- start start 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cbf6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 0x55e83cc01140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e83cb5c2e0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e83cb03ce0 con 0x55e83cc01680 2024-09-05T06:55:16.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.828+0000 7f3ea612c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e83cb582a0 con 0x55e83cbf64a0 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea22f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cbf6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33510/0 (socket says 172.21.15.86:33510) 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea22f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cbf6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:46900/0 (socket says 172.21.15.86:46900) 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 -- 172.21.15.86:0/728376414 learned_addr learned my addr 172.21.15.86:0/728376414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 0x55e83cc01140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3e92ffd640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 0x7f3e880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 -- 172.21.15.86:0/728376414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 msgr2=0x55e83cbf6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cbf6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 -- 172.21.15.86:0/728376414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 msgr2=0x55e83cc01140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 0x7f3e880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 0x55e83cc01140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 -- 172.21.15.86:0/728376414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e83cae8440 con 0x55e83cc01680 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 0x55e83cc01140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3e92ffd640 1 -- 172.21.15.86:0/2530937309 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f3e980cfc70 con 0x7f3e9c100e20 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.829+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009c60 tx=0x7f3e8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b192adede369d422 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.830+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 0x7f3e880a5230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009a60 tx=0x7f3e9402a950 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.830+0000 7f3e87fff640 1 -- 172.21.15.86:0/728376414 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c004230 con 0x55e83cc01680 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.830+0000 7f3e87fff640 1 -- 172.21.15.86:0/728376414 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c009e10 con 0x55e83cc01680 2024-09-05T06:55:16.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.830+0000 7f3e87fff640 1 -- 172.21.15.86:0/728376414 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0197d0 con 0x55e83cc01680 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- 172.21.15.86:0/728376414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 msgr2=0x55e83cc03a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009c60 tx=0x7f3e8c011810 comp rx=0 tx=0).stop 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- 172.21.15.86:0/728376414 shutdown_connections 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cc03a70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf6dc0 0x55e83cc01140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 --2- 172.21.15.86:0/728376414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cbf6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- 172.21.15.86:0/728376414 >> 172.21.15.86:0/728376414 conn(0x55e83cbf2a30 msgr2=0x55e83cbf2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- 172.21.15.86:0/728376414 shutdown_connections 2024-09-05T06:55:16.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- 172.21.15.86:0/728376414 wait complete. 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 -- start start 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.831+0000 7f3ea612c740 1 Processor -- start 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cc00f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cc01680 0x55e83cbfe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e83cc04250 con 0x55e83cc01680 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e83cb17050 con 0x55e83cbf6dc0 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e83cc043d0 con 0x55e83cbf64a0 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea612c740 10 client.0 _ll_register_callbacks cb 0x55e83cb57160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33514/0 (socket says 172.21.15.86:33514) 2024-09-05T06:55:16.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 -- 172.21.15.86:0/2580644634 learned_addr learned my addr 172.21.15.86:0/2580644634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cc00f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cc01680 0x55e83cbfe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 -- 172.21.15.86:0/2580644634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 msgr2=0x55e83cc00f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cc00f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 -- 172.21.15.86:0/2580644634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cc01680 msgr2=0x55e83cbfe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cc01680 0x55e83cbfe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.832+0000 7f3ea12ee640 1 -- 172.21.15.86:0/2580644634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83c8cd3c0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c003850 tx=0x7f3e8c003880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ac6341baa0316f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3e857fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c028020 con 0x55e83cbf6dc0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f3e8c005040 con 0x55e83cbf6dc0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e8c009e10 con 0x55e83cbf6dc0 2024-09-05T06:55:16.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e8c033050 con 0x55e83cbf6dc0 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3ea612c740 10 client.27710 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e83cc0fae0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.833+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c009e10 con 0x55e83cbf6dc0 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f3e8c018070 con 0x55e83cbf6dc0 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 10 client.27710 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 10 client.27710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 20 client.27710 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 4 client.27710 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e83cb69850 con 0x55e83cc11440 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3e857fa640 10 client.27710 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:16.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.834+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc138a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45e8180bf99dc9e server_cookie=c3d789cae7d3cd68 in_seq=0 out_seq=0 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.836+0000 7f3e857fa640 1 -- 172.21.15.86:0/2580644634 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e83cb69850 con 0x55e83cc11440 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.836+0000 7f3e857fa640 10 client.27710 handle_command_reply: tid=0 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.836+0000 7f3ea612c740 1 client.27710 shutdown 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.836+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 msgr2=0x55e83cbfe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.836+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c003850 tx=0x7f3e8c003880 comp rx=0 tx=0).stop 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 shutdown_connections 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc138a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cc01680 0x55e83cbfe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfe1e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 --2- 172.21.15.86:0/2580644634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf64a0 0x55e83cc00f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3e857fa640 0 client.27710 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 >> 172.21.15.86:0/2580644634 conn(0x55e83cbf2a30 msgr2=0x55e83cbf8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 shutdown_connections 2024-09-05T06:55:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 1 -- 172.21.15.86:0/2580644634 wait complete. 2024-09-05T06:55:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.837+0000 7f3ea612c740 20 client.27710 trim_cache size 0 max 16384 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 Processor -- start 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 -- start start 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cbf97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b56f0 con 0x55e83cbf64a0 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b7350 con 0x55e83cc01680 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea612c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b5dc0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.866+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39722/0 (socket says 172.21.15.86:39722) 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 -- 172.21.15.86:0/1836563120 learned_addr learned my addr 172.21.15.86:0/1836563120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cbf97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 -- 172.21.15.86:0/1836563120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 msgr2=0x55e83cbfdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 -- 172.21.15.86:0/1836563120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 msgr2=0x55e83cbf97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cbf97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 -- 172.21.15.86:0/1836563120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e83cbfd190 con 0x55e83cbf64a0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ea20 tx=0x7f3e9800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795bb99f84820b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3e877fe640 1 -- 172.21.15.86:0/1836563120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980b9070 con 0x55e83cbf64a0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3e877fe640 1 -- 172.21.15.86:0/1836563120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e9809bcf0 con 0x55e83cbf64a0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.867+0000 7f3e877fe640 1 -- 172.21.15.86:0/1836563120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980cd1a0 con 0x55e83cbf64a0 2024-09-05T06:55:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.868+0000 7f3ea612c740 1 -- 172.21.15.86:0/1836563120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 msgr2=0x55e83cc0d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.868+0000 7f3ea612c740 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ea20 tx=0x7f3e9800cf00 comp rx=0 tx=0).stop 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 -- 172.21.15.86:0/1836563120 shutdown_connections 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cc01680 0x55e83cbf97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cbf6dc0 0x55e83cbfdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 --2- 172.21.15.86:0/1836563120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc0d4b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 -- 172.21.15.86:0/1836563120 >> 172.21.15.86:0/1836563120 conn(0x55e83cbf2a30 msgr2=0x55e83cbf90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 -- 172.21.15.86:0/1836563120 shutdown_connections 2024-09-05T06:55:16.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 -- 172.21.15.86:0/1836563120 wait complete. 2024-09-05T06:55:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 -- start start 2024-09-05T06:55:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.869+0000 7f3ea612c740 1 Processor -- start 2024-09-05T06:55:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cc01680 0x55e83cc7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea12ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33542/0 (socket says 172.21.15.86:33542) 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea1aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc04b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39738/0 (socket says 172.21.15.86:39738) 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea12ee640 1 -- 172.21.15.86:0/3714885186 learned_addr learned my addr 172.21.15.86:0/3714885186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b6bc0 con 0x55e83cbf64a0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea22f0640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cc01680 0x55e83cc7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b64c0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.870+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1b36f0 con 0x55e83cc01680 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea612c740 10 client.0 _ll_register_callbacks cb 0x7f3e8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 -- 172.21.15.86:0/3714885186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cc01680 msgr2=0x55e83cc7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cc01680 0x55e83cc7a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 -- 172.21.15.86:0/3714885186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 msgr2=0x55e83cc04b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc04b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83cc7a670 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9809bcf0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3ea12ee640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c004940 tx=0x7f3e8c009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1166134f436396fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3e857fa640 10 client.27716 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3e857fa640 1 -- 172.21.15.86:0/3714885186 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c03c050 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.871+0000 7f3e857fa640 1 -- 172.21.15.86:0/3714885186 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f3e8c004c00 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3e857fa640 1 -- 172.21.15.86:0/3714885186 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e8c028040 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 10 client.27716 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55e83cc7b7b0 con 0x55e83cbf6dc0 2024-09-05T06:55:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3e857fa640 1 -- 172.21.15.86:0/3714885186 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f3e8c03f020 con 0x55e83cbf6dc0 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 10 client.27716 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 10 client.27716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 20 client.27716 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc7de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 4 client.27716 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e83cb63d60 con 0x55e83cc11440 2024-09-05T06:55:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.872+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc7de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:16.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea1aef640 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc7de60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181b8ae59009f138 server_cookie=2e05625c8ee476be in_seq=0 out_seq=0 2024-09-05T06:55:16.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3e857fa640 10 client.27716 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3e857fa640 1 -- 172.21.15.86:0/3714885186 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e83cb63d60 con 0x55e83cc11440 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3e857fa640 10 client.27716 handle_command_reply: tid=0 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 client.27716 shutdown 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 msgr2=0x55e83cc050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c004940 tx=0x7f3e8c009960 comp rx=0 tx=0).stop 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 shutdown_connections 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e83cc11440 0x55e83cc7de60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e83cc01680 0x55e83cc7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3e857fa640 0 client.27716 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e83cbf6dc0 0x55e83cc050c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 --2- 172.21.15.86:0/3714885186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e83cbf64a0 0x55e83cc04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.873+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 >> 172.21.15.86:0/3714885186 conn(0x55e83cbf2a30 msgr2=0x55e83cbf1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.874+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 shutdown_connections 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.874+0000 7f3ea612c740 1 -- 172.21.15.86:0/3714885186 wait complete. 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.874+0000 7f3ea612c740 20 client.27716 trim_cache size 0 max 16384 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout: "6448d741-085a-4d0f-b9cb-04dd128b8eca": { 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:55:16.878 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6448d741-085a-4d0f-b9cb-04dd128b8eca", 2024-09-05T06:55:16.879 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:55:16.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:16.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:16.879 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 msgr2=0x7f3e880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 0x7f3e880a5230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009a60 tx=0x7f3e9402a950 comp rx=0 tx=0).stop 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 msgr2=0x7f3e9c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ea50 tx=0x7f3e9800ee50 comp rx=0 tx=0).stop 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 shutdown_connections 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3e880a2d70 0x7f3e880a5230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9c102ef0 0x7f3e9c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e9c102570 0x7f3e9c107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:16.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 --2- 172.21.15.86:0/2530937309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e9c100e20 0x7f3e9c107a50 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800ea50 tx=0x7f3e9800ee50 comp rx=0 tx=0).stop 2024-09-05T06:55:16.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 >> 172.21.15.86:0/2530937309 conn(0x7f3e9c0fc7d0 msgr2=0x7f3e9c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:16.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 shutdown_connections 2024-09-05T06:55:16.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:16.875+0000 7f3ea3d53640 1 -- 172.21.15.86:0/2530937309 wait complete. 2024-09-05T06:55:17.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:17 smithi086 ceph-mon[30943]: pgmap v439: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-05T06:55:17.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:17 smithi096 ceph-mon[34468]: pgmap v439: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-05T06:55:17.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:17 smithi115 ceph-mon[34791]: pgmap v439: 129 pgs: 129 active+clean; 538 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-05T06:55:17.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5896 13.59 MB/sec warmup 35 sec latency 65.406 ms 2024-09-05T06:55:17.748 INFO:tasks.cephfs.filesystem:scrub status for tag:6448d741-085a-4d0f-b9cb-04dd128b8eca - {'path': '/', 'tag': '6448d741-085a-4d0f-b9cb-04dd128b8eca', 'options': 'recursive,force'} 2024-09-05T06:55:18.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6107 13.64 MB/sec warmup 36 sec latency 53.322 ms 2024-09-05T06:55:19.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:19 smithi096 ceph-mon[34468]: pgmap v440: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:19.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:19 smithi115 ceph-mon[34791]: pgmap v440: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:19 smithi086 ceph-mon[30943]: pgmap v440: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:19.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6354 13.81 MB/sec warmup 37 sec latency 83.809 ms 2024-09-05T06:55:20.074 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T06:55:20.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:55:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:20.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6579 13.78 MB/sec warmup 38 sec latency 76.497 ms 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.011+0000 7f8ccfb96640 1 -- 172.21.15.86:0/509895218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc8109120 msgr2=0x7f8cc8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.011+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/509895218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc8109120 0x7f8cc8110ca0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009ca0 tx=0x7f8cc002a8c0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 -- 172.21.15.86:0/509895218 shutdown_connections 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/509895218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc8109120 0x7f8cc8110ca0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/509895218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc81049a0 0x7f8cc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/509895218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8103ff0 0x7f8cc81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 -- 172.21.15.86:0/509895218 >> 172.21.15.86:0/509895218 conn(0x7f8cc80ffab0 msgr2=0x7f8cc8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 -- 172.21.15.86:0/509895218 shutdown_connections 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 -- 172.21.15.86:0/509895218 wait complete. 2024-09-05T06:55:21.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.012+0000 7f8ccfb96640 1 Processor -- start 2024-09-05T06:55:21.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 -- start start 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 0x7f8cc819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccd932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccd131640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 0x7f8cc819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccd932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33556/0 (socket says 172.21.15.86:33556) 2024-09-05T06:55:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccd932640 1 -- 172.21.15.86:0/1941587639 learned_addr learned my addr 172.21.15.86:0/1941587639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8109120 0x7f8cc819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8114000 con 0x7f8cc81049a0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8113e80 con 0x7f8cc8103ff0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8114180 con 0x7f8cc8109120 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.013+0000 7f8cce133640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8109120 0x7f8cc819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 -- 172.21.15.86:0/1941587639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8109120 msgr2=0x7f8cc819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8109120 0x7f8cc819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 -- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 msgr2=0x7f8cc819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 0x7f8cc819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc819f4a0 con 0x7f8cc8103ff0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd131640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 0x7f8cc819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccd932640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cd80 con 0x7f8cc8103ff0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc819f790 con 0x7f8cc8103ff0 2024-09-05T06:55:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc40054d0 con 0x7f8cc8103ff0 2024-09-05T06:55:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4010430 con 0x7f8cc8103ff0 2024-09-05T06:55:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.014+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81a7070 con 0x7f8cc8103ff0 2024-09-05T06:55:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.016+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc81043d0 con 0x7f8cc8103ff0 2024-09-05T06:55:21.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.017+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8cc4005020 con 0x7f8cc8103ff0 2024-09-05T06:55:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.018+0000 7f8cbeffd640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 0x7f8cb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.019+0000 7f8ccd131640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 0x7f8cb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.019+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f8cc40cfc80 con 0x7f8cc8103ff0 2024-09-05T06:55:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.019+0000 7f8ccd131640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 0x7f8cb40a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ffed0 tx=0x7f8cb8005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:21.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.024+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cc409c460 con 0x7f8cc8103ff0 2024-09-05T06:55:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:21 smithi086 ceph-mon[30943]: pgmap v441: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:21.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:21 smithi096 ceph-mon[34468]: pgmap v441: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.370+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cc810c390 con 0x7f8cc8103ff0 2024-09-05T06:55:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.371+0000 7f8cbeffd640 1 -- 172.21.15.86:0/1941587639 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7f8cc409bd60 con 0x7f8cc8103ff0 2024-09-05T06:55:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:55:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:55:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 msgr2=0x7f8cb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 0x7f8cb40a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ffed0 tx=0x7f8cb8005f90 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 msgr2=0x7f8cc819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400edf0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 shutdown_connections 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8cb40a2cf0 0x7f8cb40a51b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cc8109120 0x7f8cc819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc81049a0 0x7f8cc819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 --2- 172.21.15.86:0/1941587639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cc8103ff0 0x7f8cc819a4f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.373+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 >> 172.21.15.86:0/1941587639 conn(0x7f8cc80ffab0 msgr2=0x7f8cc8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.374+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 shutdown_connections 2024-09-05T06:55:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:21.374+0000 7f8ccfb96640 1 -- 172.21.15.86:0/1941587639 wait complete. 2024-09-05T06:55:21.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:21 smithi115 ceph-mon[34791]: pgmap v441: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-05T06:55:21.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6733 13.78 MB/sec warmup 39 sec latency 65.766 ms 2024-09-05T06:55:22.153 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T06:55:22.153 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T06:55:22.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:22 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1941587639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:22.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:22 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1941587639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:22.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:22 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1941587639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:22.502 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:22.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6903 13.74 MB/sec warmup 40 sec latency 127.458 ms 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 -- 172.21.15.86:0/2036310360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc1036b0 msgr2=0x7f95bc10bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 --2- 172.21.15.86:0/2036310360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc1036b0 0x7f95bc10bba0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009900 tx=0x7f95b002a480 comp rx=0 tx=0).stop 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 -- 172.21.15.86:0/2036310360 shutdown_connections 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 --2- 172.21.15.86:0/2036310360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc10c170 0x7f95bc10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 --2- 172.21.15.86:0/2036310360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc1036b0 0x7f95bc10bba0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 --2- 172.21.15.86:0/2036310360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc102d90 0x7f95bc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 -- 172.21.15.86:0/2036310360 >> 172.21.15.86:0/2036310360 conn(0x7f95bc0fc790 msgr2=0x7f95bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 -- 172.21.15.86:0/2036310360 shutdown_connections 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 -- 172.21.15.86:0/2036310360 wait complete. 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.064+0000 7f95c250f640 1 Processor -- start 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 -- start start 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc1036b0 0x7f95bc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 0x7f95bc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc111e50 con 0x7f95bc10c170 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc111cd0 con 0x7f95bc102d90 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c250f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc111fd0 con 0x7f95bc1036b0 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc1036b0 0x7f95bc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95c0aac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 0x7f95bc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:36716/0 (socket says 172.21.15.86:36716) 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.065+0000 7f95bbfff640 1 -- 172.21.15.86:0/2840700550 learned_addr learned my addr 172.21.15.86:0/2840700550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 -- 172.21.15.86:0/2840700550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc1036b0 msgr2=0x7f95bc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc1036b0 0x7f95bc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 -- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 msgr2=0x7f95bc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 0x7f95bc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 -- 172.21.15.86:0/2840700550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc19d320 con 0x7f95bc102d90 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95c0aac640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 0x7f95bc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95bbfff640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ea60 tx=0x7f95ac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95b97fa640 1 -- 172.21.15.86:0/2840700550 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00cde0 con 0x7f95bc102d90 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc19d610 con 0x7f95bc102d90 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95b97fa640 1 -- 172.21.15.86:0/2840700550 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95ac0035f0 con 0x7f95bc102d90 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95b97fa640 1 -- 172.21.15.86:0/2840700550 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac010690 con 0x7f95bc102d90 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.066+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1a4ef0 con 0x7f95bc102d90 2024-09-05T06:55:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.069+0000 7f95b97fa640 1 -- 172.21.15.86:0/2840700550 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95ac0037a0 con 0x7f95bc102d90 2024-09-05T06:55:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95b97fa640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 0x7f95a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95c48e8740 1 Processor -- start 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95b97fa640 1 -- 172.21.15.86:0/2840700550 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f95ac0cfc20 con 0x7f95bc102d90 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 0x7f95a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95c48e8740 1 -- start start 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.070+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 0x7f95a40a51b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005ab0 tx=0x7f95b0005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065380b70 0x55f065380f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f06538e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0652e22a0 con 0x55f06538bd50 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f065289b10 con 0x55f065380b70 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c48e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f0652de260 con 0x55f065381490 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60408/0 (socket says 172.21.15.86:60408) 2024-09-05T06:55:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 -- 172.21.15.86:0/75382564 learned_addr learned my addr 172.21.15.86:0/75382564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95c0aac640 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f06538e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bbfff640 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065380b70 0x55f065380f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 -- 172.21.15.86:0/75382564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065380b70 msgr2=0x55f065380f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065380b70 0x55f065380f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 -- 172.21.15.86:0/75382564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 msgr2=0x55f06538e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f06538e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 -- 172.21.15.86:0/75382564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f065053b90 con 0x55f065381490 2024-09-05T06:55:23.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.071+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f95b00096c0 tx=0x7f95b003d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbd49724b4f4f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.072+0000 7f95a27fc640 1 -- 172.21.15.86:0/75382564 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0035040 con 0x55f065381490 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.072+0000 7f95a27fc640 1 -- 172.21.15.86:0/75382564 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95b004e070 con 0x55f065381490 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.072+0000 7f95a27fc640 1 -- 172.21.15.86:0/75382564 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0003d50 con 0x55f065381490 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- 172.21.15.86:0/75382564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 msgr2=0x55f06538b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f95b00096c0 tx=0x7f95b003d000 comp rx=0 tx=0).stop 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- 172.21.15.86:0/75382564 shutdown_connections 2024-09-05T06:55:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f06538e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06538b810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 --2- 172.21.15.86:0/75382564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065380b70 0x55f065380f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- 172.21.15.86:0/75382564 >> 172.21.15.86:0/75382564 conn(0x55f06537d100 msgr2=0x55f06537d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- 172.21.15.86:0/75382564 shutdown_connections 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- 172.21.15.86:0/75382564 wait complete. 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.073+0000 7f95c48e8740 1 -- start start 2024-09-05T06:55:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 Processor -- start 2024-09-05T06:55:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065381490 0x55f0653891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f06538bd50 0x55f065389730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f06538e970 con 0x55f065380b70 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f06526a6e0 con 0x55f065381490 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f06538eaf0 con 0x55f06538bd50 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c48e8740 10 client.0 _ll_register_callbacks cb 0x55f06528ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45246/0 (socket says 172.21.15.86:45246) 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95bbfff640 1 -- 172.21.15.86:0/4253377427 learned_addr learned my addr 172.21.15.86:0/4253377427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065381490 0x55f0653891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.074+0000 7f95c0aac640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f06538bd50 0x55f065389730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 -- 172.21.15.86:0/4253377427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f06538bd50 msgr2=0x55f065389730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f06538bd50 0x55f065389730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 -- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065381490 msgr2=0x55f0653891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065381490 0x55f0653891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 -- 172.21.15.86:0/4253377427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f065397ee0 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95bbfff640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ea30 tx=0x7f95ac09b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8d35e1c1675d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f959b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0a0050 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f95ac098310 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.075+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b004e070 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f95ac0a4910 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 10 client.18753 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f0653786c0 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95b004e070 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f95ac065050 con 0x55f065380b70 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 10 client.18753 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 10 client.18753 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 20 client.18753 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065379b20 0x55f06537bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 4 client.18753 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.076+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0652ef9d0 con 0x55f065379b20 2024-09-05T06:55:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.077+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065379b20 0x55f06537bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.077+0000 7f959b7fe640 10 client.18753 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.077+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065379b20 0x55f06537bf80 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b068860c7ad7cd3 server_cookie=a443c6daaa5c0058 in_seq=0 out_seq=0 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f959b7fe640 1 -- 172.21.15.86:0/4253377427 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0652ef9d0 con 0x55f065379b20 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f959b7fe640 10 client.18753 handle_command_reply: tid=0 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 client.18753 shutdown 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 msgr2=0x55f065388cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ea30 tx=0x7f95ac09b6f0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 shutdown_connections 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065379b20 0x55f06537bf80 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f06538bd50 0x55f065389730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065381490 0x55f0653891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 --2- 172.21.15.86:0/4253377427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065380b70 0x55f065388cb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 >> 172.21.15.86:0/4253377427 conn(0x55f06537d100 msgr2=0x55f065377e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.078+0000 7f959b7fe640 0 client.18753 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.079+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 shutdown_connections 2024-09-05T06:55:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.079+0000 7f95c48e8740 1 -- 172.21.15.86:0/4253377427 wait complete. 2024-09-05T06:55:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.079+0000 7f95c48e8740 20 client.18753 trim_cache size 0 max 16384 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 Processor -- start 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 -- start start 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f065387bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f0653835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065385f10 0x55f065386370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1acff0 con 0x55f06538bd50 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1aec50 con 0x55f065385f10 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.103+0000 7f95c48e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1ad6c0 con 0x55f065381490 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f065387bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f0653835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f0653835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45250/0 (socket says 172.21.15.86:45250) 2024-09-05T06:55:23.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bb7fe640 1 -- 172.21.15.86:0/1742332060 learned_addr learned my addr 172.21.15.86:0/1742332060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:23.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95c0aac640 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065385f10 0x55f065386370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 -- 172.21.15.86:0/1742332060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065385f10 msgr2=0x55f065386370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065385f10 0x55f065386370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 -- 172.21.15.86:0/1742332060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 msgr2=0x55f0653835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f0653835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 -- 172.21.15.86:0/1742332060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06537dd90 con 0x55f065381490 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95bbfff640 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f065387bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95ac09cfd0 tx=0x7f95ac097e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d817583a68e8de8e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.104+0000 7f95a1ffb640 1 -- 172.21.15.86:0/1742332060 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0a0050 con 0x55f065381490 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.105+0000 7f95a1ffb640 1 -- 172.21.15.86:0/1742332060 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95ac09c3a0 con 0x55f065381490 2024-09-05T06:55:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.105+0000 7f95a1ffb640 1 -- 172.21.15.86:0/1742332060 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac09b350 con 0x55f065381490 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 -- 172.21.15.86:0/1742332060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 msgr2=0x55f065387bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f065387bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95ac09cfd0 tx=0x7f95ac097e30 comp rx=0 tx=0).stop 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 -- 172.21.15.86:0/1742332060 shutdown_connections 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f065385f10 0x55f065386370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f06538bd50 0x55f0653835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 --2- 172.21.15.86:0/1742332060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f065387bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 -- 172.21.15.86:0/1742332060 >> 172.21.15.86:0/1742332060 conn(0x55f06537d100 msgr2=0x55f06538ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 -- 172.21.15.86:0/1742332060 shutdown_connections 2024-09-05T06:55:23.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.106+0000 7f95c48e8740 1 -- 172.21.15.86:0/1742332060 wait complete. 2024-09-05T06:55:23.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.107+0000 7f95c48e8740 1 -- start start 2024-09-05T06:55:23.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.107+0000 7f95c48e8740 1 Processor -- start 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.107+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06539de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c48e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 0x55f065401880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06539de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06539de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60446/0 (socket says 172.21.15.86:60446) 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45262/0 (socket says 172.21.15.86:45262) 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c0aac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 0x55f065401880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c0aac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 0x55f065401880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:36766/0 (socket says 172.21.15.86:36766) 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bbfff640 1 -- 172.21.15.86:0/596568195 learned_addr learned my addr 172.21.15.86:0/596568195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1ae4c0 con 0x55f065385f10 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1addc0 con 0x55f06538bd50 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1aaff0 con 0x55f065381490 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 -- 172.21.15.86:0/596568195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 msgr2=0x55f06539de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06539de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 -- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 msgr2=0x55f065401880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 0x55f065401880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.108+0000 7f95bb7fe640 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f065401ec0 con 0x55f065385f10 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f95c48e8740 10 client.0 _ll_register_callbacks cb 0x7f95ac09b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f95bb7fe640 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004650 tx=0x7f95b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88639efa6729820 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f959b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f959b7fe640 1 -- 172.21.15.86:0/596568195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0035040 con 0x55f065385f10 2024-09-05T06:55:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f959b7fe640 1 -- 172.21.15.86:0/596568195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f95b00048c0 con 0x55f065385f10 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ac09c3a0 con 0x55f065385f10 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f959b7fe640 1 -- 172.21.15.86:0/596568195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f95b0004ac0 con 0x55f065385f10 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f95c48e8740 10 client.18759 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55f065402fc0 con 0x55f065385f10 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.109+0000 7f959b7fe640 1 -- 172.21.15.86:0/596568195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f95b0004c60 con 0x55f065385f10 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 10 client.18759 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 10 client.18759 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 20 client.18759 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065403290 0x55f0654056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 4 client.18759 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55f0652e9d20 con 0x55f065403290 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95bbfff640 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065403290 0x55f0654056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f959b7fe640 10 client.18759 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.110+0000 7f95bbfff640 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065403290 0x55f0654056b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a630b2567558882a server_cookie=dfae598ee03d8644 in_seq=0 out_seq=0 2024-09-05T06:55:23.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:23 smithi086 ceph-mon[30943]: pgmap v442: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-05T06:55:23.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f959b7fe640 1 -- 172.21.15.86:0/596568195 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f0652e9d20 con 0x55f065403290 2024-09-05T06:55:23.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f959b7fe640 10 client.18759 handle_command_reply: tid=0 2024-09-05T06:55:23.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 client.18759 shutdown 2024-09-05T06:55:23.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 msgr2=0x55f06539e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004650 tx=0x7f95b0005fd0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 shutdown_connections 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f065403290 0x55f0654056b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06538bd50 0x55f065401880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f065385f10 0x55f06539e340 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 --2- 172.21.15.86:0/596568195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f065381490 0x55f06539de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f959b7fe640 0 client.18759 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 >> 172.21.15.86:0/596568195 conn(0x55f06537d100 msgr2=0x55f065385720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 shutdown_connections 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 1 -- 172.21.15.86:0/596568195 wait complete. 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.376+0000 7f95c48e8740 20 client.18759 trim_cache size 0 max 16384 2024-09-05T06:55:23.378 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 38, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 50, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "d634bbb4": { 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 50, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:55:23.379 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:23.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:23.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:23.381 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:23.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 msgr2=0x7f95a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 0x7f95a40a51b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005ab0 tx=0x7f95b0005a20 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 msgr2=0x7f95bc198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ea60 tx=0x7f95ac00ef20 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 shutdown_connections 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f95a40a2cf0 0x7f95a40a51b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95bc10c170 0x7f95bc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95bc1036b0 0x7f95bc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 --2- 172.21.15.86:0/2840700550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95bc102d90 0x7f95bc198370 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 >> 172.21.15.86:0/2840700550 conn(0x7f95bc0fc790 msgr2=0x7f95bc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 shutdown_connections 2024-09-05T06:55:23.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:23.377+0000 7f95c250f640 1 -- 172.21.15.86:0/2840700550 wait complete. 2024-09-05T06:55:23.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:23 smithi115 ceph-mon[34791]: pgmap v442: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-05T06:55:23.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:23 smithi096 ceph-mon[34468]: pgmap v442: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-05T06:55:23.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 13.63 MB/sec warmup 41 sec latency 632.869 ms 2024-09-05T06:55:24.044 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd634bbb4', quiesce took 4.0 seconds. Will release after: 6.0 2024-09-05T06:55:24.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 13.31 MB/sec warmup 42 sec latency 1632.985 ms 2024-09-05T06:55:25.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:25 smithi115 ceph-mon[34791]: pgmap v443: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 222 op/s 2024-09-05T06:55:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:25 smithi086 ceph-mon[30943]: pgmap v443: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 222 op/s 2024-09-05T06:55:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:25 smithi096 ceph-mon[34468]: pgmap v443: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 222 op/s 2024-09-05T06:55:25.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 13.00 MB/sec warmup 43 sec latency 2633.033 ms 2024-09-05T06:55:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:55:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:55:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:55:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:55:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:55:26.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:55:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:55:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:55:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:55:26.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 12.70 MB/sec warmup 44 sec latency 3633.074 ms 2024-09-05T06:55:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:27 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:55:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:27 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:55:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:27 smithi115 ceph-mon[34791]: pgmap v444: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-05T06:55:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:27 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:55:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:27 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:55:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:27 smithi086 ceph-mon[30943]: pgmap v444: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-05T06:55:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:27 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:55:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:27 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:55:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:27 smithi096 ceph-mon[34468]: pgmap v444: 129 pgs: 129 active+clean; 568 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-05T06:55:27.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 12.42 MB/sec warmup 45 sec latency 4633.122 ms 2024-09-05T06:55:28.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 12.15 MB/sec warmup 46 sec latency 5633.180 ms 2024-09-05T06:55:29.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:29 smithi115 ceph-mon[34791]: pgmap v445: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-09-05T06:55:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:29 smithi086 ceph-mon[30943]: pgmap v445: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-09-05T06:55:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:29 smithi096 ceph-mon[34468]: pgmap v445: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 284 op/s 2024-09-05T06:55:29.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 11.89 MB/sec warmup 47 sec latency 6633.276 ms 2024-09-05T06:55:30.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:55:30.301 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:30.746 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 11.65 MB/sec warmup 48 sec latency 7633.354 ms 2024-09-05T06:55:31.359 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:31 smithi096 ceph-mon[34468]: pgmap v446: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 4.0 MiB/s wr, 171 op/s 2024-09-05T06:55:31.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:31 smithi086 ceph-mon[30943]: pgmap v446: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 4.0 MiB/s wr, 171 op/s 2024-09-05T06:55:31.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:31 smithi115 ceph-mon[34791]: pgmap v446: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 4.0 MiB/s wr, 171 op/s 2024-09-05T06:55:31.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 11.41 MB/sec warmup 49 sec latency 8633.450 ms 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 -- 172.21.15.86:0/3529172919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 msgr2=0x7f90c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/3529172919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c81067e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a410 comp rx=0 tx=0).stop 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 -- 172.21.15.86:0/3529172919 shutdown_connections 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/3529172919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8106f10 0x7f90c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/3529172919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c81067e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/3529172919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8101de0 0x7f90c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 -- 172.21.15.86:0/3529172919 >> 172.21.15.86:0/3529172919 conn(0x7f90c80fd8c0 msgr2=0x7f90c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 -- 172.21.15.86:0/3529172919 shutdown_connections 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.486+0000 7f90cd3cf640 1 -- 172.21.15.86:0/3529172919 wait complete. 2024-09-05T06:55:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 Processor -- start 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 -- start start 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8101de0 0x7f90c81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8106f10 0x7f90c819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90c8111de0 con 0x7f90c8102790 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90c8111c60 con 0x7f90c8106f10 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.487+0000 7f90cd3cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90c8111f60 con 0x7f90c8101de0 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35400/0 (socket says 172.21.15.86:35400) 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 -- 172.21.15.86:0/1117893265 learned_addr learned my addr 172.21.15.86:0/1117893265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c6ffd640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8101de0 0x7f90c81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c77fe640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8106f10 0x7f90c819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 -- 172.21.15.86:0/1117893265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8101de0 msgr2=0x7f90c81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8101de0 0x7f90c81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 -- 172.21.15.86:0/1117893265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8106f10 msgr2=0x7f90c819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8106f10 0x7f90c819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 -- 172.21.15.86:0/1117893265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c819d390 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90c67fc640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005940 tx=0x7f90bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc03f030 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90bc003aa0 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c819d620 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc035070 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.488+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c81a4f60 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.490+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c81032c0 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.491+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f90bc042030 con 0x7f90c8102790 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.491+0000 7f90affff640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 0x7f90b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.491+0000 7f90c6ffd640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 0x7f90b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:32.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.491+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f90bc0f0040 con 0x7f90c8102790 2024-09-05T06:55:32.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.492+0000 7f90c6ffd640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 0x7f90b00a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005d00 tx=0x7f90b800b410 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:32.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.496+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90bc0bc820 con 0x7f90c8102790 2024-09-05T06:55:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.697+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90c810a0f0 con 0x7f90c8102790 2024-09-05T06:55:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.698+0000 7f90affff640 1 -- 172.21.15.86:0/1117893265 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7f90bc0bc120 con 0x7f90c8102790 2024-09-05T06:55:32.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:55:32.700 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:55:32.700 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 msgr2=0x7f90b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 0x7f90b00a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005d00 tx=0x7f90b800b410 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 msgr2=0x7f90c8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005940 tx=0x7f90bc004270 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 shutdown_connections 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90b00a2cf0 0x7f90b00a51b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90c8106f10 0x7f90c819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.700+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c8102790 0x7f90c8198920 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.701+0000 7f90cd3cf640 1 --2- 172.21.15.86:0/1117893265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c8101de0 0x7f90c81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.701+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 >> 172.21.15.86:0/1117893265 conn(0x7f90c80fd8c0 msgr2=0x7f90c80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.701+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 shutdown_connections 2024-09-05T06:55:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:32.701+0000 7f90cd3cf640 1 -- 172.21.15.86:0/1117893265 wait complete. 2024-09-05T06:55:32.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 11.18 MB/sec warmup 50 sec latency 9633.603 ms 2024-09-05T06:55:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:33 smithi115 ceph-mon[34791]: pgmap v447: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.1 MiB/s wr, 180 op/s 2024-09-05T06:55:33.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:33 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1117893265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:33.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:33 smithi086 ceph-mon[30943]: pgmap v447: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.1 MiB/s wr, 180 op/s 2024-09-05T06:55:33.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:33 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1117893265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:33.504 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id d634bbb4 --release --await' 2024-09-05T06:55:33.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id d634bbb4 --release --await 2024-09-05T06:55:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:33 smithi096 ceph-mon[34468]: pgmap v447: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.1 MiB/s wr, 180 op/s 2024-09-05T06:55:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:33 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1117893265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:33.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7007 10.96 MB/sec warmup 51 sec latency 10633.631 ms 2024-09-05T06:55:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 -- 172.21.15.86:0/3467248725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00100e20 msgr2=0x7f0a00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 --2- 172.21.15.86:0/3467248725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00100e20 0x7f0a00101200 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009900 tx=0x7f09f002a480 comp rx=0 tx=0).stop 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 -- 172.21.15.86:0/3467248725 shutdown_connections 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 --2- 172.21.15.86:0/3467248725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00102ef0 0x7f0a0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 --2- 172.21.15.86:0/3467248725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 --2- 172.21.15.86:0/3467248725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00100e20 0x7f0a00101200 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 -- 172.21.15.86:0/3467248725 >> 172.21.15.86:0/3467248725 conn(0x7f0a000fc7d0 msgr2=0x7f0a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 -- 172.21.15.86:0/3467248725 shutdown_connections 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.292+0000 7f0a05240640 1 -- 172.21.15.86:0/3467248725 wait complete. 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 Processor -- start 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 -- start start 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 0x7f0a000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00110e50 con 0x7f0a00102ef0 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00110cd0 con 0x7f0a00102570 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f0a05240640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00110fd0 con 0x7f0a00100e20 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35412/0 (socket says 172.21.15.86:35412) 2024-09-05T06:55:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 -- 172.21.15.86:0/528085958 learned_addr learned my addr 172.21.15.86:0/528085958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09fed9d640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 0x7f0a000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09fe59c640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 -- 172.21.15.86:0/528085958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 msgr2=0x7f0a000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 0x7f0a000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 -- 172.21.15.86:0/528085958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 msgr2=0x7f0a000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09ff59e640 1 -- 172.21.15.86:0/528085958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a001a68b0 con 0x7f0a00102ef0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09fed9d640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 0x7f0a000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.293+0000 7f09fe59c640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f09ff59e640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b970 tx=0x7f09f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f09e7fff640 1 -- 172.21.15.86:0/528085958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8019070 con 0x7f0a00102ef0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a001a6ba0 con 0x7f0a00102ef0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f09e7fff640 1 -- 172.21.15.86:0/528085958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f8014440 con 0x7f0a00102ef0 2024-09-05T06:55:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a001a70e0 con 0x7f0a00102ef0 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.294+0000 7f09e7fff640 1 -- 172.21.15.86:0/528085958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f801d5b0 con 0x7f0a00102ef0 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.296+0000 7f09e7fff640 1 -- 172.21.15.86:0/528085958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09f801c070 con 0x7f0a00102ef0 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.296+0000 7f0a07619740 1 Processor -- start 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.296+0000 7f0a07619740 1 -- start start 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09e7fff640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 0x7f09e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900143440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900150630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609000a8620 con 0x560900143980 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56090004fe50 con 0x56090014e240 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f0a07619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5609000a45e0 con 0x560900143060 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 0x7f09e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09e7fff640 1 -- 172.21.15.86:0/528085958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f09f80ced10 con 0x7f0a00102ef0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35418/0 (socket says 172.21.15.86:35418) 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 -- 172.21.15.86:0/3497643042 learned_addr learned my addr 172.21.15.86:0/3497643042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fed9d640 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09ff59e640 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 -- 172.21.15.86:0/3497643042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 msgr2=0x560900143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 -- 172.21.15.86:0/3497643042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 msgr2=0x560900150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 -- 172.21.15.86:0/3497643042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560900040d40 con 0x560900143980 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 0x7f09e80a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09f4013a30 tx=0x7f09f400bc00 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.297+0000 7f09fe59c640 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005e10 tx=0x7f09f401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b81fadbf9c084 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f09e57fa640 1 -- 172.21.15.86:0/3497643042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4021070 con 0x560900143980 2024-09-05T06:55:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f09e57fa640 1 -- 172.21.15.86:0/3497643042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f400ee20 con 0x560900143980 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f09e57fa640 1 -- 172.21.15.86:0/3497643042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40047a0 con 0x560900143980 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 -- 172.21.15.86:0/3497643042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 msgr2=0x56090014dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005e10 tx=0x7f09f401a000 comp rx=0 tx=0).stop 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 -- 172.21.15.86:0/3497643042 shutdown_connections 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x56090014dd00 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 --2- 172.21.15.86:0/3497643042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 -- 172.21.15.86:0/3497643042 >> 172.21.15.86:0/3497643042 conn(0x56090013f5f0 msgr2=0x56090013f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 -- 172.21.15.86:0/3497643042 shutdown_connections 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.298+0000 7f0a07619740 1 -- 172.21.15.86:0/3497643042 wait complete. 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 -- start start 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 Processor -- start 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x56090014b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143980 0x56090014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560900150e10 con 0x56090014e240 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560900030f10 con 0x560900143980 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560900150f90 con 0x560900143060 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f0a07619740 10 client.0 _ll_register_callbacks cb 0x560900054db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35422/0 (socket says 172.21.15.86:35422) 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 -- 172.21.15.86:0/348784622 learned_addr learned my addr 172.21.15.86:0/348784622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09fe59c640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143980 0x56090014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09fed9d640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x56090014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 -- 172.21.15.86:0/348784622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 msgr2=0x56090014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x56090014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 -- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143980 msgr2=0x56090014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143980 0x56090014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.299+0000 7f09ff59e640 1 -- 172.21.15.86:0/348784622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ffdb7580 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09ff59e640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f09f80978d0 tx=0x7f09f809a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e811978e16425f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f809af00 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f09f80a36e0 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f400ee20 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f09f809f2b0 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 10 client.18777 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5608ffe81960 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f400ee20 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f09f809a350 con 0x56090014e240 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 10 client.18777 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 10 client.18777 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:34.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 20 client.18777 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56090015de50 0x560900160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 4 client.18777 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.300+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609000b5b90 con 0x56090015de50 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.301+0000 7f09fed9d640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56090015de50 0x560900160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.301+0000 7f09e2ff5640 10 client.18777 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.301+0000 7f09fed9d640 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56090015de50 0x560900160290 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad81c4360eed18c6 server_cookie=70c91af282ed9d88 in_seq=0 out_seq=0 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f09e2ff5640 1 -- 172.21.15.86:0/348784622 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609000b5b90 con 0x56090015de50 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f09e2ff5640 10 client.18777 handle_command_reply: tid=0 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 client.18777 shutdown 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 msgr2=0x56090014bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f09f80978d0 tx=0x7f09f809a920 comp rx=0 tx=0).stop 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 shutdown_connections 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56090015de50 0x560900160290 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x56090014bbf0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f09e2ff5640 0 client.18777 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143980 0x56090014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 --2- 172.21.15.86:0/348784622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x56090014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 >> 172.21.15.86:0/348784622 conn(0x56090013f5f0 msgr2=0x560900145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 shutdown_connections 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 1 -- 172.21.15.86:0/348784622 wait complete. 2024-09-05T06:55:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.302+0000 7f0a07619740 20 client.18777 trim_cache size 0 max 16384 2024-09-05T06:55:34.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 Processor -- start 2024-09-05T06:55:34.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 -- start start 2024-09-05T06:55:34.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143060 0x560900145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143980 0x56090013e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001b1300 con 0x56090014e240 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001b2f60 con 0x560900143060 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f0a07619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001b19d0 con 0x560900143980 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35438/0 (socket says 172.21.15.86:35438) 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 -- 172.21.15.86:0/224430015 learned_addr learned my addr 172.21.15.86:0/224430015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09fed9d640 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143060 0x560900145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09fe59c640 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143980 0x56090013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 -- 172.21.15.86:0/224430015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143980 msgr2=0x56090013e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143980 0x56090013e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 -- 172.21.15.86:0/224430015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143060 msgr2=0x560900145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.319+0000 7f09ff59e640 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143060 0x560900145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f09ff59e640 1 -- 172.21.15.86:0/224430015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560900140280 con 0x56090014e240 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f09ff59e640 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f09f809ac00 tx=0x7f09f8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4930ce9723cf8b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f09e4ff9640 1 -- 172.21.15.86:0/224430015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f809af00 con 0x56090014e240 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f09e4ff9640 1 -- 172.21.15.86:0/224430015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f80a3b70 con 0x56090014e240 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f09e4ff9640 1 -- 172.21.15.86:0/224430015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80a36e0 con 0x56090014e240 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 -- 172.21.15.86:0/224430015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 msgr2=0x560900165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f09f809ac00 tx=0x7f09f8000f40 comp rx=0 tx=0).stop 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 -- 172.21.15.86:0/224430015 shutdown_connections 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56090014e240 0x560900165b60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143980 0x56090013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 --2- 172.21.15.86:0/224430015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560900143060 0x560900145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.320+0000 7f0a07619740 1 -- 172.21.15.86:0/224430015 >> 172.21.15.86:0/224430015 conn(0x56090013f5f0 msgr2=0x56090013df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- 172.21.15.86:0/224430015 shutdown_connections 2024-09-05T06:55:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- 172.21.15.86:0/224430015 wait complete. 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- start start 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 Processor -- start 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900156230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900153200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001b27d0 con 0x560900143980 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001b20d0 con 0x56090014e240 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001af300 con 0x560900143060 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f0a07619740 10 client.0 _ll_register_callbacks cb 0x7f09f809acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f09fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35446/0 (socket says 172.21.15.86:35446) 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f09fe59c640 1 -- 172.21.15.86:0/2086155707 learned_addr learned my addr 172.21.15.86:0/2086155707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:34.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f09ff59e640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.321+0000 7f09fed9d640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 -- 172.21.15.86:0/2086155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 msgr2=0x560900156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 -- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 msgr2=0x560900153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 -- 172.21.15.86:0/2086155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609001490c0 con 0x560900143980 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fed9d640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09fe59c640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005390 tx=0x7f09f400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d025df280c275f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4021040 con 0x560900143980 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f80a3b70 con 0x560900143980 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f09f40132b0 con 0x560900143980 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f09f400ee30 con 0x560900143980 2024-09-05T06:55:34.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f0a07619740 10 client.18783 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5609001c90a0 con 0x560900143980 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.322+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f80a3b70 con 0x560900143980 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f09f4025400 con 0x560900143980 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 10 client.18783 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 10 client.18783 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 20 client.18783 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5609001c9520 0x5609001cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 4 client.18783 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d634bbb4", "release": true, "await": true}] 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d634bbb4", "release": true, "await": true}) -- 0x5609000b00a0 con 0x5609001c9520 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f09fed9d640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5609001c9520 0x5609001cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f09e2ff5640 10 client.18783 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:34.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.323+0000 7f09fed9d640 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5609001c9520 0x5609001cb940 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46de5f28472e993 server_cookie=b2f021b21aaa9830 in_seq=0 out_seq=0 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f09e2ff5640 1 -- 172.21.15.86:0/2086155707 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5609000b00a0 con 0x5609001c9520 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f09e2ff5640 10 client.18783 handle_command_reply: tid=0 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 client.18783 shutdown 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 msgr2=0x560900156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005390 tx=0x7f09f400bfa0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 shutdown_connections 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5609001c9520 0x5609001cb940 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090014e240 0x560900153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560900143980 0x560900156770 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 --2- 172.21.15.86:0/2086155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560900143060 0x560900156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 >> 172.21.15.86:0/2086155707 conn(0x56090013f5f0 msgr2=0x56090014b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f09e2ff5640 0 client.18783 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.353+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 shutdown_connections 2024-09-05T06:55:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.354+0000 7f0a07619740 1 -- 172.21.15.86:0/2086155707 wait complete. 2024-09-05T06:55:34.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.354+0000 7f0a07619740 20 client.18783 trim_cache size 0 max 16384 2024-09-05T06:55:34.356 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 38, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 52, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "d634bbb4": { 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 52, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:55:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:34.358 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 msgr2=0x7f09e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 0x7f09e80a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09f4013a30 tx=0x7f09f400bc00 comp rx=0 tx=0).stop 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 msgr2=0x7f0a000fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b970 tx=0x7f09f800be30 comp rx=0 tx=0).stop 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 shutdown_connections 2024-09-05T06:55:34.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09e80a2cf0 0x7f09e80a51b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102ef0 0x7f0a000fd700 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00102570 0x7f0a000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 --2- 172.21.15.86:0/528085958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00100e20 0x7f0a000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 >> 172.21.15.86:0/528085958 conn(0x7f0a000fc7d0 msgr2=0x7f0a00069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 shutdown_connections 2024-09-05T06:55:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:34.355+0000 7f0a05240640 1 -- 172.21.15.86:0/528085958 wait complete. 2024-09-05T06:55:34.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7066 10.77 MB/sec warmup 52 sec latency 11237.583 ms 2024-09-05T06:55:35.267 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd634bbb4', total seconds elapsed: 15.2 2024-09-05T06:55:35.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:35 smithi115 ceph-mon[34791]: pgmap v448: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:35 smithi086 ceph-mon[30943]: pgmap v448: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:35.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:35 smithi096 ceph-mon[34468]: pgmap v448: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:35.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7226 10.74 MB/sec warmup 53 sec latency 60.453 ms 2024-09-05T06:55:36.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7416 10.81 MB/sec warmup 54 sec latency 69.480 ms 2024-09-05T06:55:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:37 smithi115 ceph-mon[34791]: pgmap v449: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:37 smithi086 ceph-mon[30943]: pgmap v449: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:37.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:37 smithi096 ceph-mon[34468]: pgmap v449: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-05T06:55:37.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7584 10.72 MB/sec warmup 55 sec latency 63.171 ms 2024-09-05T06:55:38.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:38 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:55:38.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:38 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:55:38.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:38 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:55:38.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7711 10.61 MB/sec warmup 56 sec latency 134.487 ms 2024-09-05T06:55:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:39 smithi115 ceph-mon[34791]: pgmap v450: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.3 MiB/s wr, 123 op/s 2024-09-05T06:55:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:39 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:55:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:39 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:55:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:39 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:55:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:39 smithi086 ceph-mon[30943]: pgmap v450: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.3 MiB/s wr, 123 op/s 2024-09-05T06:55:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:39 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:55:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:39 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:55:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:39 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:55:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:39 smithi096 ceph-mon[34468]: pgmap v450: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.3 MiB/s wr, 123 op/s 2024-09-05T06:55:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:39 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:55:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:39 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:55:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:39 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:55:39.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7858 10.48 MB/sec warmup 57 sec latency 87.463 ms 2024-09-05T06:55:40.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8034 10.47 MB/sec warmup 58 sec latency 80.102 ms 2024-09-05T06:55:41.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:41 smithi115 ceph-mon[34791]: pgmap v451: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-05T06:55:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:41 smithi086 ceph-mon[30943]: pgmap v451: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-05T06:55:41.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:41 smithi096 ceph-mon[34468]: pgmap v451: 129 pgs: 129 active+clean; 581 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-05T06:55:41.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8212 10.37 MB/sec warmup 59 sec latency 69.252 ms 2024-09-05T06:55:42.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8399 10.44 MB/sec warmup 60 sec latency 144.036 ms 2024-09-05T06:55:43.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:43 smithi115 ceph-mon[34791]: pgmap v452: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.1 MiB/s wr, 178 op/s 2024-09-05T06:55:43.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:43 smithi086 ceph-mon[30943]: pgmap v452: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.1 MiB/s wr, 178 op/s 2024-09-05T06:55:43.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:43 smithi096 ceph-mon[34468]: pgmap v452: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 3.1 MiB/s wr, 178 op/s 2024-09-05T06:55:43.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8557 10.35 MB/sec warmup 61 sec latency 72.632 ms 2024-09-05T06:55:44.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8746 10.26 MB/sec warmup 62 sec latency 68.820 ms 2024-09-05T06:55:45.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:45 smithi115 ceph-mon[34791]: pgmap v453: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:45 smithi086 ceph-mon[30943]: pgmap v453: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:45.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:45 smithi096 ceph-mon[34468]: pgmap v453: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:45.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8955 10.30 MB/sec warmup 63 sec latency 58.260 ms 2024-09-05T06:55:46.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9147 10.23 MB/sec warmup 64 sec latency 67.026 ms 2024-09-05T06:55:47.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:47 smithi115 ceph-mon[34791]: pgmap v454: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:47 smithi086 ceph-mon[30943]: pgmap v454: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:47 smithi096 ceph-mon[34468]: pgmap v454: 129 pgs: 129 active+clean; 573 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 170 op/s 2024-09-05T06:55:47.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9329 10.21 MB/sec warmup 65 sec latency 61.164 ms 2024-09-05T06:55:47.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:55:48.105 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:48.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9499 10.18 MB/sec warmup 66 sec latency 62.199 ms 2024-09-05T06:55:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 -- 172.21.15.86:0/3131061920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102570 msgr2=0x7efee41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 --2- 172.21.15.86:0/3131061920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102570 0x7efee41029b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efed4009890 tx=0x7efed402a410 comp rx=0 tx=0).stop 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 -- 172.21.15.86:0/3131061920 shutdown_connections 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 --2- 172.21.15.86:0/3131061920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4102ef0 0x7efee410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 --2- 172.21.15.86:0/3131061920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102570 0x7efee41029b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 --2- 172.21.15.86:0/3131061920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4100e20 0x7efee4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.976+0000 7efeec2b0640 1 -- 172.21.15.86:0/3131061920 >> 172.21.15.86:0/3131061920 conn(0x7efee40fc7d0 msgr2=0x7efee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.977+0000 7efeec2b0640 1 -- 172.21.15.86:0/3131061920 shutdown_connections 2024-09-05T06:55:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.977+0000 7efeec2b0640 1 -- 172.21.15.86:0/3131061920 wait complete. 2024-09-05T06:55:48.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.977+0000 7efeec2b0640 1 Processor -- start 2024-09-05T06:55:48.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.977+0000 7efeec2b0640 1 -- start start 2024-09-05T06:55:48.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 0x7efee419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 0x7efee419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110e80 con 0x7efee4102570 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110d00 con 0x7efee4100e20 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeec2b0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efee4111000 con 0x7efee4102ef0 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40200/0 (socket says 172.21.15.86:40200) 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 -- 172.21.15.86:0/470960174 learned_addr learned my addr 172.21.15.86:0/470960174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeea84d640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 0x7efee419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeea04c640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 0x7efee419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 -- 172.21.15.86:0/470960174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 msgr2=0x7efee419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 0x7efee419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 -- 172.21.15.86:0/470960174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 msgr2=0x7efee419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 0x7efee419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efee984b640 1 -- 172.21.15.86:0/470960174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419f5b0 con 0x7efee4102570 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeea84d640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 0x7efee419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.978+0000 7efeea04c640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 0x7efee419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efee984b640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efedb7fe640 1 -- 172.21.15.86:0/470960174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4040070 con 0x7efee4102570 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419f840 con 0x7efee4102570 2024-09-05T06:55:48.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a7180 con 0x7efee4102570 2024-09-05T06:55:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efedb7fe640 1 -- 172.21.15.86:0/470960174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed4005570 con 0x7efee4102570 2024-09-05T06:55:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.979+0000 7efedb7fe640 1 -- 172.21.15.86:0/470960174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40046c0 con 0x7efee4102570 2024-09-05T06:55:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.982+0000 7efedb7fe640 1 -- 172.21.15.86:0/470960174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efed4043070 con 0x7efee4102570 2024-09-05T06:55:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.983+0000 7efedb7fe640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 0x7efed00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.983+0000 7efedb7fe640 1 -- 172.21.15.86:0/470960174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7efed40f0700 con 0x7efee4102570 2024-09-05T06:55:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.983+0000 7efeea04c640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 0x7efed00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 Processor -- start 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeea04c640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 0x7efed00a5230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee0009c20 tx=0x7efee0005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 -- start start 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 0x556a778067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a7776c2e0 con 0x556a77806ce0 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556a77713ce0 con 0x556a778063c0 2024-09-05T06:55:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.984+0000 7efeee689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a777682a0 con 0x556a77819e90 2024-09-05T06:55:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efee984b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42954/0 (socket says 172.21.15.86:42954) 2024-09-05T06:55:48.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 -- 172.21.15.86:0/1003520374 learned_addr learned my addr 172.21.15.86:0/1003520374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:48.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efee984b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40204/0 (socket says 172.21.15.86:40204) 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea04c640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 0x556a778067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 -- 172.21.15.86:0/1003520374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 msgr2=0x556a778067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 0x556a778067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 -- 172.21.15.86:0/1003520374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 msgr2=0x556a77819950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 -- 172.21.15.86:0/1003520374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a776f8440 con 0x556a77819e90 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea04c640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 0x556a778067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efee984b640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.985+0000 7efeea84d640 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efedc010460 tx=0x7efedc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbe8751ee54d216 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.986+0000 7efed8dea640 1 -- 172.21.15.86:0/1003520374 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d070 con 0x556a77819e90 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.986+0000 7efed8dea640 1 -- 172.21.15.86:0/1003520374 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc00eb10 con 0x556a77819e90 2024-09-05T06:55:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.986+0000 7efed8dea640 1 -- 172.21.15.86:0/1003520374 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc018ba0 con 0x556a77819e90 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- 172.21.15.86:0/1003520374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 msgr2=0x556a7781c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efedc010460 tx=0x7efedc010860 comp rx=0 tx=0).stop 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- 172.21.15.86:0/1003520374 shutdown_connections 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a7781c280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a77806ce0 0x556a77819950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 --2- 172.21.15.86:0/1003520374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a778063c0 0x556a778067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- 172.21.15.86:0/1003520374 >> 172.21.15.86:0/1003520374 conn(0x556a77802a30 msgr2=0x556a77802e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- 172.21.15.86:0/1003520374 shutdown_connections 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- 172.21.15.86:0/1003520374 wait complete. 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 -- start start 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.987+0000 7efeee689740 1 Processor -- start 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77819790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 0x556a77816a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea04c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77819790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea04c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77819790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40218/0 (socket says 172.21.15.86:40218) 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea04c640 1 -- 172.21.15.86:0/3788501220 learned_addr learned my addr 172.21.15.86:0/3788501220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a77816ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a7781ca00 con 0x556a778063c0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556a77727050 con 0x556a77806ce0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a7781cb80 con 0x556a77819e90 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efee984b640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 0x556a77816a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea84d640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a77816ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeee689740 10 client.0 _ll_register_callbacks cb 0x556a77767160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea84d640 1 -- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 msgr2=0x556a77816a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.988+0000 7efeea84d640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 0x556a77816a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeea84d640 1 -- 172.21.15.86:0/3788501220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 msgr2=0x556a77819790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeea84d640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77819790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeea84d640 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a774dd3c0 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efee984b640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 0x556a77816a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeea84d640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a77816ff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efedc018780 tx=0x7efedc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2defe4fa89851254 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d040 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7efedc015040 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efedc00eb10 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efedc00ecc0 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeee689740 10 client.27748 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.989+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556a77828460 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc00eb10 con 0x556a77819e90 2024-09-05T06:55:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7efedc00b040 con 0x556a77819e90 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 10 client.27748 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 10 client.27748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 20 client.27748 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a778297b0 0x556a7782bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 4 client.27748 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a77779850 con 0x556a778297b0 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.990+0000 7efeea04c640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a778297b0 0x556a7782bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.991+0000 7efecdffb640 10 client.27748 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.991+0000 7efeea04c640 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a778297b0 0x556a7782bba0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d0d771389dc1b3 server_cookie=2481b9806f6d0964 in_seq=0 out_seq=0 2024-09-05T06:55:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.991+0000 7efecdffb640 1 -- 172.21.15.86:0/3788501220 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a77779850 con 0x556a778297b0 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.991+0000 7efecdffb640 10 client.27748 handle_command_reply: tid=0 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.991+0000 7efeee689740 1 client.27748 shutdown 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 msgr2=0x556a77816ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a77816ff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efedc018780 tx=0x7efedc009e30 comp rx=0 tx=0).stop 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 shutdown_connections 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a778297b0 0x556a7782bba0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77819e90 0x556a77816ff0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77806ce0 0x556a77816a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 --2- 172.21.15.86:0/3788501220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77819790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 >> 172.21.15.86:0/3788501220 conn(0x556a77802a30 msgr2=0x556a77801b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efecdffb640 0 client.27748 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 shutdown_connections 2024-09-05T06:55:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 1 -- 172.21.15.86:0/3788501220 wait complete. 2024-09-05T06:55:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:48.992+0000 7efeee689740 20 client.27748 trim_cache size 0 max 16384 2024-09-05T06:55:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 Processor -- start 2024-09-05T06:55:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 -- start start 2024-09-05T06:55:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a77815360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a77818a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efec8010090 con 0x556a778063c0 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efec8011cf0 con 0x556a77819e90 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeee689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efec8010760 con 0x556a77806ce0 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeea04c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efee984b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a77815360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeea04c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40234/0 (socket says 172.21.15.86:40234) 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.018+0000 7efeea04c640 1 -- 172.21.15.86:0/3522510193 learned_addr learned my addr 172.21.15.86:0/3522510193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea84d640 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a77818a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 -- 172.21.15.86:0/3522510193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 msgr2=0x556a77815360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a77815360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 -- 172.21.15.86:0/3522510193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 msgr2=0x556a77818a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a77818a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 -- 172.21.15.86:0/3522510193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a77815a40 con 0x556a778063c0 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efee984b640 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a77815360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efeea04c640 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efee0012010 tx=0x7efee0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f22ff2fe8b4bc0f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efecffff640 1 -- 172.21.15.86:0/3522510193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee002f070 con 0x556a778063c0 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efecffff640 1 -- 172.21.15.86:0/3522510193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee0016070 con 0x556a778063c0 2024-09-05T06:55:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.019+0000 7efecffff640 1 -- 172.21.15.86:0/3522510193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00124b0 con 0x556a778063c0 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 -- 172.21.15.86:0/3522510193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 msgr2=0x556a77825d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efee0012010 tx=0x7efee0014000 comp rx=0 tx=0).stop 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 -- 172.21.15.86:0/3522510193 shutdown_connections 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a77818a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a77815360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 --2- 172.21.15.86:0/3522510193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a77825d30 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.020+0000 7efeee689740 1 -- 172.21.15.86:0/3522510193 >> 172.21.15.86:0/3522510193 conn(0x556a77802a30 msgr2=0x556a778182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.021+0000 7efeee689740 1 -- 172.21.15.86:0/3522510193 shutdown_connections 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.021+0000 7efeee689740 1 -- 172.21.15.86:0/3522510193 wait complete. 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.021+0000 7efeee689740 1 -- start start 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.021+0000 7efeee689740 1 Processor -- start 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a778226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a7781da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efec8011560 con 0x556a778063c0 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efec8010e60 con 0x556a77819e90 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efec800e090 con 0x556a77806ce0 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeee689740 10 client.0 _ll_register_callbacks cb 0x7efedc01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:35800/0 (socket says 172.21.15.86:35800) 2024-09-05T06:55:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 -- 172.21.15.86:0/2683903194 learned_addr learned my addr 172.21.15.86:0/2683903194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea04c640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a778226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efee984b640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a7781da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 -- 172.21.15.86:0/2683903194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 msgr2=0x556a7781da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a7781da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.022+0000 7efeea84d640 1 -- 172.21.15.86:0/2683903194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 msgr2=0x556a778226d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeea84d640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a778226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeea84d640 1 -- 172.21.15.86:0/2683903194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7781e870 con 0x556a77819e90 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efee984b640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a7781da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeea04c640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a778226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeea84d640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efedc0187b0 tx=0x7efedc0097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=165cff558d8ae7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc034020 con 0x556a77819e90 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee0016070 con 0x556a77819e90 2024-09-05T06:55:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7efedc005330 con 0x556a77819e90 2024-09-05T06:55:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efedc03c050 con 0x556a77819e90 2024-09-05T06:55:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeee689740 10 client.27776 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.023+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556a778944e0 con 0x556a77819e90 2024-09-05T06:55:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee0016070 con 0x556a77819e90 2024-09-05T06:55:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7efedc00b050 con 0x556a77819e90 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 10 client.27776 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 10 client.27776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 20 client.27776 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a77894cb0 0x556a778970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 4 client.27776 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:55:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a77773d60 con 0x556a77894cb0 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.024+0000 7efeea04c640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a77894cb0 0x556a778970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeea04c640 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a77894cb0 0x556a778970d0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e773e0ba3381e350 server_cookie=e1c16ea33d8b1dd2 in_seq=0 out_seq=0 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efecdffb640 10 client.27776 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efecdffb640 1 -- 172.21.15.86:0/2683903194 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a77773d60 con 0x556a77894cb0 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efecdffb640 10 client.27776 handle_command_reply: tid=0 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 client.27776 shutdown 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 msgr2=0x556a7781e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efedc0187b0 tx=0x7efedc0097e0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 shutdown_connections 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556a77894cb0 0x556a778970d0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a77819e90 0x556a7781e040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efecdffb640 0 client.27776 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a77806ce0 0x556a7781da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 --2- 172.21.15.86:0/2683903194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a778063c0 0x556a778226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 >> 172.21.15.86:0/2683903194 conn(0x556a77802a30 msgr2=0x556a778182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 shutdown_connections 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 1 -- 172.21.15.86:0/2683903194 wait complete. 2024-09-05T06:55:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.025+0000 7efeee689740 20 client.27776 trim_cache size 0 max 16384 2024-09-05T06:55:49.032 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:49.032 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:55:49.032 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:55:49.032 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 msgr2=0x7efed00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 0x7efed00a5230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee0009c20 tx=0x7efee0005f90 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 msgr2=0x7efee419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4030f90 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 shutdown_connections 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efed00a2d70 0x7efed00a5230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efee4102ef0 0x7efee419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee4102570 0x7efee419ab40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 --2- 172.21.15.86:0/470960174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee4100e20 0x7efee419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 >> 172.21.15.86:0/470960174 conn(0x7efee40fc7d0 msgr2=0x7efee406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.026+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 shutdown_connections 2024-09-05T06:55:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:49.027+0000 7efeec2b0640 1 -- 172.21.15.86:0/470960174 wait complete. 2024-09-05T06:55:49.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:49 smithi115 ceph-mon[34791]: pgmap v455: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 296 op/s 2024-09-05T06:55:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:49 smithi086 ceph-mon[30943]: pgmap v455: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 296 op/s 2024-09-05T06:55:49.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:49 smithi096 ceph-mon[34468]: pgmap v455: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 296 op/s 2024-09-05T06:55:49.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:55:49.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:55:49.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9713 10.32 MB/sec warmup 67 sec latency 98.972 ms 2024-09-05T06:55:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:50.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 -- 172.21.15.86:0/3392546951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f4108e00 msgr2=0x7f57f410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:50.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 --2- 172.21.15.86:0/3392546951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f4108e00 0x7f57f410ce00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009900 tx=0x7f57e802a410 comp rx=0 tx=0).stop 2024-09-05T06:55:50.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 -- 172.21.15.86:0/3392546951 shutdown_connections 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 --2- 172.21.15.86:0/3392546951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f410d530 0x7f57f41150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 --2- 172.21.15.86:0/3392546951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f4108e00 0x7f57f410ce00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 --2- 172.21.15.86:0/3392546951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108450 0x7f57f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 -- 172.21.15.86:0/3392546951 >> 172.21.15.86:0/3392546951 conn(0x7f57f4077ba0 msgr2=0x7f57f4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.668+0000 7f57f9d15640 1 -- 172.21.15.86:0/3392546951 shutdown_connections 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.669+0000 7f57f9d15640 1 -- 172.21.15.86:0/3392546951 wait complete. 2024-09-05T06:55:50.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.669+0000 7f57f9d15640 1 Processor -- start 2024-09-05T06:55:50.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.669+0000 7f57f9d15640 1 -- start start 2024-09-05T06:55:50.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f4108450 0x7f57f41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 0x7f57f41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4118430 con 0x7f57f410d530 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41182b0 con 0x7f57f4108e00 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f9d15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41185b0 con 0x7f57f4108450 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 0x7f57f41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40264/0 (socket says 172.21.15.86:40264) 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f3fff640 1 -- 172.21.15.86:0/32015146 learned_addr learned my addr 172.21.15.86:0/32015146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f4108450 0x7f57f41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 -- 172.21.15.86:0/32015146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f4108450 msgr2=0x7f57f41a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:50.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f4108450 0x7f57f41a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 -- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 msgr2=0x7f57f419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.670+0000 7f57f2ffd640 1 -- 172.21.15.86:0/32015146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f419ef20 con 0x7f57f4108e00 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f3fff640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f2ffd640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 0x7f57f41a4b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57e802aa00 tx=0x7f57e8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e803f070 con 0x7f57f4108e00 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57e8031b50 con 0x7f57f4108e00 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f419f1b0 con 0x7f57f4108e00 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8035030 con 0x7f57f4108e00 2024-09-05T06:55:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.671+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f41ab460 con 0x7f57f4108e00 2024-09-05T06:55:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.672+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f41098e0 con 0x7f57f4108e00 2024-09-05T06:55:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.674+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f57e8042070 con 0x7f57f4108e00 2024-09-05T06:55:50.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.675+0000 7f57f0ff9640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:50.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.675+0000 7f57f37fe640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:50.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.675+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f57e80eff20 con 0x7f57f4108e00 2024-09-05T06:55:50.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.675+0000 7f57f37fe640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 0x7f57dc0a5230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009bd0 tx=0x7f57e4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:50.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:50.678+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57e80bc0b0 con 0x7f57f4108e00 2024-09-05T06:55:50.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9936 10.32 MB/sec warmup 68 sec latency 80.139 ms 2024-09-05T06:55:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.023+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57f4108830 con 0x7f57f4108e00 2024-09-05T06:55:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.024+0000 7f57f0ff9640 1 -- 172.21.15.86:0/32015146 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7f57f4108830 con 0x7f57f4108e00 2024-09-05T06:55:51.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:55:51.025 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:55:51.025 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 msgr2=0x7f57dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 0x7f57dc0a5230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009bd0 tx=0x7f57e4005f90 comp rx=0 tx=0).stop 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 msgr2=0x7f57f41a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 0x7f57f41a4b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57e802aa00 tx=0x7f57e8003b90 comp rx=0 tx=0).stop 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 shutdown_connections 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.027+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410d530 0x7f57f419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f4108e00 0x7f57f41a4b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 --2- 172.21.15.86:0/32015146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57f4108450 0x7f57f41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 >> 172.21.15.86:0/32015146 conn(0x7f57f4077ba0 msgr2=0x7f57f4078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 shutdown_connections 2024-09-05T06:55:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:51.028+0000 7f57f9d15640 1 -- 172.21.15.86:0/32015146 wait complete. 2024-09-05T06:55:51.411 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:51 smithi086 ceph-mon[30943]: pgmap v456: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-05T06:55:51.411 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/32015146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:51.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:51 smithi115 ceph-mon[34791]: pgmap v456: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-05T06:55:51.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/32015146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:51.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:51 smithi096 ceph-mon[34468]: pgmap v456: 129 pgs: 129 active+clean; 576 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-05T06:55:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/32015146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:51.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10124 10.36 MB/sec warmup 69 sec latency 63.193 ms 2024-09-05T06:55:51.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:55:52.161 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:52.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10257 10.35 MB/sec warmup 70 sec latency 89.351 ms 2024-09-05T06:55:53.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:53 smithi086 ceph-mon[30943]: pgmap v457: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 358 op/s 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 -- 172.21.15.86:0/443879152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4100e20 msgr2=0x7fd0d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 --2- 172.21.15.86:0/443879152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4100e20 0x7fd0d4101200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009900 tx=0x7fd0d002a480 comp rx=0 tx=0).stop 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 -- 172.21.15.86:0/443879152 shutdown_connections 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 --2- 172.21.15.86:0/443879152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4102ef0 0x7fd0d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 --2- 172.21.15.86:0/443879152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 --2- 172.21.15.86:0/443879152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4100e20 0x7fd0d4101200 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 -- 172.21.15.86:0/443879152 >> 172.21.15.86:0/443879152 conn(0x7fd0d40fc7d0 msgr2=0x7fd0d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 -- 172.21.15.86:0/443879152 shutdown_connections 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.037+0000 7fd0db43d640 1 -- 172.21.15.86:0/443879152 wait complete. 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.038+0000 7fd0db43d640 1 Processor -- start 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.038+0000 7fd0db43d640 1 -- start start 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.038+0000 7fd0db43d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4100e20 0x7fd0d419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.038+0000 7fd0db43d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0db43d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4102ef0 0x7fd0d419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0db43d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410fd30 con 0x7fd0d4102ef0 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0db43d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410fbb0 con 0x7fd0d4102570 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0db43d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410feb0 con 0x7fd0d4100e20 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44534/0 (socket says 172.21.15.86:44534) 2024-09-05T06:55:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 -- 172.21.15.86:0/4047129474 learned_addr learned my addr 172.21.15.86:0/4047129474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4100e20 0x7fd0d419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d99da640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4102ef0 0x7fd0d419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 -- 172.21.15.86:0/4047129474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4100e20 msgr2=0x7fd0d419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4100e20 0x7fd0d419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 -- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4102ef0 msgr2=0x7fd0d419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4102ef0 0x7fd0d419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.039+0000 7fd0d89d8640 1 -- 172.21.15.86:0/4047129474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d419f5a0 con 0x7fd0d4102570 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0ca7fc640 1 -- 172.21.15.86:0/4047129474 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400cd80 con 0x7fd0d4102570 2024-09-05T06:55:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d419f890 con 0x7fd0d4102570 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0ca7fc640 1 -- 172.21.15.86:0/4047129474 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c40108e0 con 0x7fd0d4102570 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d41a7170 con 0x7fd0d4102570 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.040+0000 7fd0ca7fc640 1 -- 172.21.15.86:0/4047129474 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4018920 con 0x7fd0d4102570 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.042+0000 7fd0ca7fc640 1 -- 172.21.15.86:0/4047129474 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd0c4010430 con 0x7fd0d4102570 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.043+0000 7fd0dd816740 1 Processor -- start 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.043+0000 7fd0dd816740 1 -- start start 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b0ffc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b1009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f622f0 con 0x5621b0ffc5d0 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f09cf0 con 0x5621b1007370 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0dd816740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f5e2b0 con 0x5621b0ffdb40 2024-09-05T06:55:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33350/0 (socket says 172.21.15.86:33350) 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3642904175 learned_addr learned my addr 172.21.15.86:0/3642904175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b0ffc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b1009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 -- 172.21.15.86:0/3642904175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 msgr2=0x5621b1009760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b1009760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 -- 172.21.15.86:0/3642904175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 msgr2=0x5621b0ffca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d99da640 1 -- 172.21.15.86:0/3642904175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b0eee3a0 con 0x5621b0ffdb40 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.044+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b0ffc090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009f00 tx=0x7fd0cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c522a77d37bcbbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0ca7fc640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 0x7fd0c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0bf7fe640 1 -- 172.21.15.86:0/3642904175 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc004d20 con 0x5621b0ffdb40 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0bf7fe640 1 -- 172.21.15.86:0/3642904175 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc004030 con 0x5621b0ffdb40 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0bf7fe640 1 -- 172.21.15.86:0/3642904175 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc005240 con 0x5621b0ffdb40 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 0x7fd0c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.045+0000 7fd0ca7fc640 1 -- 172.21.15.86:0/4047129474 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fd0c40cfa20 con 0x7fd0d4102570 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 0x7fd0c00a5230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5621b0ff9ac0 tx=0x7fd0d0005e80 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- 172.21.15.86:0/3642904175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 msgr2=0x5621b0ffc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b0ffc090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009f00 tx=0x7fd0cc011810 comp rx=0 tx=0).stop 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- 172.21.15.86:0/3642904175 shutdown_connections 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b1009760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b0ffca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3642904175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b0ffc090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- 172.21.15.86:0/3642904175 >> 172.21.15.86:0/3642904175 conn(0x5621b0fc5310 msgr2=0x5621b0ff8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- 172.21.15.86:0/3642904175 shutdown_connections 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- 172.21.15.86:0/3642904175 wait complete. 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.046+0000 7fd0dd816740 1 -- start start 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 Processor -- start 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 0x5621b1003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b10044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621b100a070 con 0x5621b0ffdb40 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621b0f1d060 con 0x5621b0ffc5d0 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5621b100a1f0 con 0x5621b1007370 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0dd816740 10 client.0 _ll_register_callbacks cb 0x5621b0c708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44554/0 (socket says 172.21.15.86:44554) 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0d91d9640 1 -- 172.21.15.86:0/1261151556 learned_addr learned my addr 172.21.15.86:0/1261151556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 0x5621b1003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.047+0000 7fd0d99da640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b10044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 -- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 msgr2=0x5621b10044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b10044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 -- 172.21.15.86:0/1261151556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 msgr2=0x5621b1006ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 -- 172.21.15.86:0/1261151556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b0cd33c0 con 0x5621b0ffdb40 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 0x5621b1003f90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40cccd0 tx=0x7fd0c409b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebe943908f3ca81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40cd160 con 0x5621b0ffdb40 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fd0c409b1a0 con 0x5621b0ffdb40 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.048+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0cc004030 con 0x5621b0ffdb40 2024-09-05T06:55:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd0c409fe50 con 0x5621b0ffdb40 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 10 client.18816 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5621b1015b70 con 0x5621b0ffdb40 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc004030 con 0x5621b0ffdb40 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd0c409aa10 con 0x5621b0ffdb40 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 10 client.18816 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 10 client.18816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 20 client.18816 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1016fd0 0x5621b10193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 4 client.18816 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.049+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621b0f6f860 con 0x5621b1016fd0 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.050+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1016fd0 0x5621b10193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.050+0000 7fd0bcff9640 10 client.18816 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.050+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1016fd0 0x5621b10193f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bca4df48e5f0455 server_cookie=c70831cbb9dce3f8 in_seq=0 out_seq=0 2024-09-05T06:55:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0bcff9640 1 -- 172.21.15.86:0/1261151556 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621b0f6f860 con 0x5621b1016fd0 2024-09-05T06:55:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0bcff9640 10 client.18816 handle_command_reply: tid=0 2024-09-05T06:55:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 client.18816 shutdown 2024-09-05T06:55:53.228 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 msgr2=0x5621b1003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 0x5621b1003f90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40cccd0 tx=0x7fd0c409b4b0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 shutdown_connections 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1016fd0 0x5621b10193f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b10044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffdb40 0x5621b1003f90 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 --2- 172.21.15.86:0/1261151556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffc5d0 0x5621b1006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0bcff9640 0 client.18816 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.051+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 >> 172.21.15.86:0/1261151556 conn(0x5621b0fc5310 msgr2=0x5621b0ffe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.052+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 shutdown_connections 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.052+0000 7fd0dd816740 1 -- 172.21.15.86:0/1261151556 wait complete. 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.052+0000 7fd0dd816740 20 client.18816 trim_cache size 0 max 16384 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 Processor -- start 2024-09-05T06:55:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 -- start start 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b1002f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b101eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80100e0 con 0x5621b0ffc5d0 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8011d40 con 0x5621b1007370 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.065+0000 7fd0dd816740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80107b0 con 0x5621b0ffdb40 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33378/0 (socket says 172.21.15.86:33378) 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3213685551 learned_addr learned my addr 172.21.15.86:0/3213685551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b1002f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b101eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3213685551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 msgr2=0x5621b1002f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b1002f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3213685551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 msgr2=0x5621b101eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b101eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3213685551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b1005310 con 0x5621b0ffc5d0 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b1002f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0004650 tx=0x7fd0d003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fef74c36405939 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0beffd640 1 -- 172.21.15.86:0/3213685551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0035040 con 0x5621b0ffc5d0 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0beffd640 1 -- 172.21.15.86:0/3213685551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d005a070 con 0x5621b0ffc5d0 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.066+0000 7fd0beffd640 1 -- 172.21.15.86:0/3213685551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00543a0 con 0x5621b0ffc5d0 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- 172.21.15.86:0/3213685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 msgr2=0x5621b10133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0004650 tx=0x7fd0d003f000 comp rx=0 tx=0).stop 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- 172.21.15.86:0/3213685551 shutdown_connections 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b1007370 0x5621b101eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b0ffdb40 0x5621b1002f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3213685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b10133a0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- 172.21.15.86:0/3213685551 >> 172.21.15.86:0/3213685551 conn(0x5621b0fc5310 msgr2=0x5621b1002860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- 172.21.15.86:0/3213685551 shutdown_connections 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- 172.21.15.86:0/3213685551 wait complete. 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.067+0000 7fd0dd816740 1 -- start start 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 Processor -- start 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 0x5621b1000a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b1000fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80115b0 con 0x5621b0ffc5d0 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8010eb0 con 0x5621b0ffdb40 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b800e0e0 con 0x5621b1007370 2024-09-05T06:55:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0dd816740 10 client.0 _ll_register_callbacks cb 0x7fd0c400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 0x5621b1000a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d91d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33386/0 (socket says 172.21.15.86:33386) 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d91d9640 1 -- 172.21.15.86:0/3273770123 learned_addr learned my addr 172.21.15.86:0/3273770123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d99da640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b1000fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 -- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 msgr2=0x5621b1000fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b1000fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 -- 172.21.15.86:0/3273770123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 msgr2=0x5621b100fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d89d8640 1 -- 172.21.15.86:0/3273770123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b1001530 con 0x5621b0ffdb40 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.068+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0d89d8640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 0x5621b1000a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b850 tx=0x7fd0c4097ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2de833bd17d0c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40cd160 con 0x5621b0ffdb40 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0d005a070 con 0x5621b0ffdb40 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fd0c4010ae0 con 0x5621b0ffdb40 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd0c409b920 con 0x5621b0ffdb40 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 10 client.27800 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5621b10819a0 con 0x5621b0ffdb40 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d005a070 con 0x5621b0ffdb40 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd0c40ba2e0 con 0x5621b0ffdb40 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 10 client.27800 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 10 client.27800 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 20 client.27800 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1081f10 0x5621b1084330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 4 client.27800 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.069+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621b0f69d70 con 0x5621b1081f10 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1081f10 0x5621b1084330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0bcff9640 10 client.27800 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0d91d9640 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1081f10 0x5621b1084330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6c4b4bc1d5bceb8 server_cookie=19d98f94cd5d1c3d in_seq=0 out_seq=0 2024-09-05T06:55:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0bcff9640 1 -- 172.21.15.86:0/3273770123 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621b0f69d70 con 0x5621b1081f10 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0bcff9640 10 client.27800 handle_command_reply: tid=0 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 client.27800 shutdown 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 msgr2=0x5621b1000a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 0x5621b1000a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b850 tx=0x7fd0c4097ad0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 shutdown_connections 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5621b1081f10 0x5621b1084330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5621b1007370 0x5621b1000fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0bcff9640 0 client.27800 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621b0ffdb40 0x5621b1000a80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 --2- 172.21.15.86:0/3273770123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621b0ffc5d0 0x5621b100fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 >> 172.21.15.86:0/3273770123 conn(0x5621b0fc5310 msgr2=0x5621b1003080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 shutdown_connections 2024-09-05T06:55:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 1 -- 172.21.15.86:0/3273770123 wait complete. 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.070+0000 7fd0dd816740 20 client.27800 trim_cache size 0 max 16384 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 msgr2=0x7fd0c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 0x7fd0c00a5230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5621b0ff9ac0 tx=0x7fd0d0005e80 comp rx=0 tx=0).stop 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 msgr2=0x7fd0d419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400eeb0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 shutdown_connections 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd0c00a2d70 0x7fd0c00a5230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0d4102ef0 0x7fd0d419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d4102570 0x7fd0d419ab30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 --2- 172.21.15.86:0/4047129474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0d4100e20 0x7fd0d419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 >> 172.21.15.86:0/4047129474 conn(0x7fd0d40fc7d0 msgr2=0x7fd0d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 shutdown_connections 2024-09-05T06:55:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:53.071+0000 7fd0db43d640 1 -- 172.21.15.86:0/4047129474 wait complete. 2024-09-05T06:55:53.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:53 smithi115 ceph-mon[34791]: pgmap v457: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 358 op/s 2024-09-05T06:55:53.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:53 smithi096 ceph-mon[34468]: pgmap v457: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 358 op/s 2024-09-05T06:55:53.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10426 10.37 MB/sec warmup 71 sec latency 128.481 ms 2024-09-05T06:55:53.839 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.8 2024-09-05T06:55:53.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:55:54.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10626 10.46 MB/sec warmup 72 sec latency 117.157 ms 2024-09-05T06:55:54.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:55:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:55.363 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:55 smithi086 ceph-mon[30943]: pgmap v458: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:55 smithi115 ceph-mon[34791]: pgmap v458: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 -- 172.21.15.86:0/1337562210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 msgr2=0x7f3224107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 --2- 172.21.15.86:0/1337562210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f3224107b90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3210009930 tx=0x7f321002a440 comp rx=0 tx=0).stop 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 -- 172.21.15.86:0/1337562210 shutdown_connections 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 --2- 172.21.15.86:0/1337562210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 --2- 172.21.15.86:0/1337562210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f3224107b90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 --2- 172.21.15.86:0/1337562210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 0x7f3224103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 -- 172.21.15.86:0/1337562210 >> 172.21.15.86:0/1337562210 conn(0x7f32240fc7d0 msgr2=0x7f32240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 -- 172.21.15.86:0/1337562210 shutdown_connections 2024-09-05T06:55:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.482+0000 7f322afc3640 1 -- 172.21.15.86:0/1337562210 wait complete. 2024-09-05T06:55:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 Processor -- start 2024-09-05T06:55:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 -- start start 2024-09-05T06:55:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 0x7f32241a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3224111fc0 con 0x7f32241036e0 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3224111e40 con 0x7f32241082a0 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.483+0000 7f322afc3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3224112140 con 0x7f3224102d30 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33394/0 (socket says 172.21.15.86:33394) 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 -- 172.21.15.86:0/4004893975 learned_addr learned my addr 172.21.15.86:0/4004893975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3228d5f640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 0x7f32241a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3229560640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 -- 172.21.15.86:0/4004893975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 msgr2=0x7f32241a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 0x7f32241a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 -- 172.21.15.86:0/4004893975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 msgr2=0x7f322410deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3223fff640 1 -- 172.21.15.86:0/4004893975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322410e6e0 con 0x7f32241036e0 2024-09-05T06:55:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.484+0000 7f3229560640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f3223fff640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3210031440 tx=0x7f3210003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321003f070 con 0x7f32241036e0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3210031b50 con 0x7f32241036e0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210035030 con 0x7f32241036e0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322410e970 con 0x7f32241036e0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.485+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32241ab660 con 0x7f32241036e0 2024-09-05T06:55:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.487+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322406a850 con 0x7f32241036e0 2024-09-05T06:55:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.488+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3210042070 con 0x7f32241036e0 2024-09-05T06:55:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.489+0000 7f3221ffb640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 0x7f32140a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.490+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f32100f0000 con 0x7f32241036e0 2024-09-05T06:55:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.490+0000 7f3228d5f640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 0x7f32140a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.491+0000 7f3228d5f640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 0x7f32140a5280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bb0 tx=0x7f3218005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.495+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32100bc7e0 con 0x7f32241036e0 2024-09-05T06:55:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:55.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:55 smithi096 ceph-mon[34468]: pgmap v458: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:55.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10769 10.42 MB/sec warmup 73 sec latency 104.940 ms 2024-09-05T06:55:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.870+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3224107270 con 0x7f32241036e0 2024-09-05T06:55:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.871+0000 7f3221ffb640 1 -- 172.21.15.86:0/4004893975 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7f32100bc0e0 con 0x7f32241036e0 2024-09-05T06:55:55.879 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:55:55.879 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:55:55.879 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:55:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 msgr2=0x7f32140a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 0x7f32140a5280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bb0 tx=0x7f3218005f90 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 msgr2=0x7f32241a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3210031440 tx=0x7f3210003c00 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 shutdown_connections 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f32140a2dc0 0x7f32140a5280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32241082a0 0x7f322410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32241036e0 0x7f32241a4db0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 --2- 172.21.15.86:0/4004893975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3224102d30 0x7f32241a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 >> 172.21.15.86:0/4004893975 conn(0x7f32240fc7d0 msgr2=0x7f3224100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.874+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 shutdown_connections 2024-09-05T06:55:55.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:55.875+0000 7f322afc3640 1 -- 172.21.15.86:0/4004893975 wait complete. 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 -- 172.21.15.86:0/3053521127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 msgr2=0x7fe4341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3053521127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341067e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe424009860 tx=0x7fe42402a340 comp rx=0 tx=0).stop 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 -- 172.21.15.86:0/3053521127 shutdown_connections 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3053521127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434106f10 0x7fe43410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3053521127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341067e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3053521127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434101de0 0x7fe4341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 -- 172.21.15.86:0/3053521127 >> 172.21.15.86:0/3053521127 conn(0x7fe4340fd8e0 msgr2=0x7fe4340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 -- 172.21.15.86:0/3053521127 shutdown_connections 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.057+0000 7fe43c06c640 1 -- 172.21.15.86:0/3053521127 wait complete. 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.058+0000 7fe43c06c640 1 Processor -- start 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.058+0000 7fe43c06c640 1 -- start start 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.058+0000 7fe43c06c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.058+0000 7fe43c06c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.058+0000 7fe43c06c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43c06c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe434111de0 con 0x7fe434101de0 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43c06c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe434111c60 con 0x7fe434102790 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43c06c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe434111f60 con 0x7fe434106f10 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe439e08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:39790/0 (socket says 172.21.15.86:39790) 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe439e08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33412/0 (socket says 172.21.15.86:33412) 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 -- 172.21.15.86:0/3617473645 learned_addr learned my addr 172.21.15.86:0/3617473645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe439607640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 -- 172.21.15.86:0/3617473645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 msgr2=0x7fe4341988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 -- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 msgr2=0x7fe4341983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 -- 172.21.15.86:0/3617473645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43419d350 con 0x7fe434106f10 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe439e08640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe439607640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe43a609640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00b330 tx=0x7fe42c00b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.059+0000 7fe42affd640 1 -- 172.21.15.86:0/3617473645 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c003970 con 0x7fe434106f10 2024-09-05T06:55:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.060+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43419d640 con 0x7fe434106f10 2024-09-05T06:55:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.060+0000 7fe42affd640 1 -- 172.21.15.86:0/3617473645 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe42c004020 con 0x7fe434106f10 2024-09-05T06:55:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.060+0000 7fe42affd640 1 -- 172.21.15.86:0/3617473645 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0178c0 con 0x7fe434106f10 2024-09-05T06:55:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.060+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe434074420 con 0x7fe434106f10 2024-09-05T06:55:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.062+0000 7fe42affd640 1 -- 172.21.15.86:0/3617473645 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe42c0041d0 con 0x7fe434106f10 2024-09-05T06:55:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.063+0000 7fe43e445740 1 Processor -- start 2024-09-05T06:55:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 -- start start 2024-09-05T06:55:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1de6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1df1370 0x556cd1df3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556cd1d4c2f0 con 0x556cd1de65d0 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556cd1cf3cf0 con 0x556cd1de7b40 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe43e445740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556cd1d482b0 con 0x556cd1df1370 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.064+0000 7fe42affd640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 0x7fe4200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33420/0 (socket says 172.21.15.86:33420) 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 -- 172.21.15.86:0/652950837 learned_addr learned my addr 172.21.15.86:0/652950837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe43a609640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 0x7fe4200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439607640 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1de6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 -- 172.21.15.86:0/652950837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1df1370 msgr2=0x556cd1df3760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1df1370 0x556cd1df3760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 -- 172.21.15.86:0/652950837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 msgr2=0x556cd1de6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1de6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 -- 172.21.15.86:0/652950837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd1cd83a0 con 0x556cd1de65d0 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439607640 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1de6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe439e08640 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe430010520 tx=0x7fe4300109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358cf2cc1afd4d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe43a609640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 0x7fe4200a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe42c09c230 tx=0x7fe42c0ca3f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.065+0000 7fe42affd640 1 -- 172.21.15.86:0/3617473645 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fe42c0d3780 con 0x7fe434106f10 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.066+0000 7fe41ffff640 1 -- 172.21.15.86:0/652950837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000e770 con 0x556cd1de65d0 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.066+0000 7fe41ffff640 1 -- 172.21.15.86:0/652950837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43000edc0 con 0x556cd1de65d0 2024-09-05T06:55:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.066+0000 7fe41ffff640 1 -- 172.21.15.86:0/652950837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43001a380 con 0x556cd1de65d0 2024-09-05T06:55:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 -- 172.21.15.86:0/652950837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 msgr2=0x556cd1de6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe430010520 tx=0x7fe4300109e0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 -- 172.21.15.86:0/652950837 shutdown_connections 2024-09-05T06:55:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1df1370 0x556cd1df3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de65d0 0x556cd1de6a10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 --2- 172.21.15.86:0/652950837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1de6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 -- 172.21.15.86:0/652950837 >> 172.21.15.86:0/652950837 conn(0x556cd1daf310 msgr2=0x556cd1de2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 -- 172.21.15.86:0/652950837 shutdown_connections 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.067+0000 7fe43e445740 1 -- 172.21.15.86:0/652950837 wait complete. 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 -- start start 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 Processor -- start 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de7b40 0x556cd1dee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1df1370 0x556cd1dee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556cd1df40c0 con 0x556cd1de7b40 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556cd1d07060 con 0x556cd1df1370 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556cd1df4240 con 0x556cd1de65d0 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.068+0000 7fe43e445740 10 client.0 _ll_register_callbacks cb 0x556cd1a5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:39796/0 (socket says 172.21.15.86:39796) 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 -- 172.21.15.86:0/4274409689 learned_addr learned my addr 172.21.15.86:0/4274409689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1df1370 0x556cd1dee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439e08640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de7b40 0x556cd1dee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 -- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1df1370 msgr2=0x556cd1dee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1df1370 0x556cd1dee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 -- 172.21.15.86:0/4274409689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de7b40 msgr2=0x556cd1dee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de7b40 0x556cd1dee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 -- 172.21.15.86:0/4274409689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd1abd3c0 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe439607640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe424009250 tx=0x7fe424030380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1432489c91b1691b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe41d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424046060 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43000edc0 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.069+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fe424005230 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe424037030 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 10 client.27790 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556cd1e08400 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43000edc0 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe4240035f0 con 0x556cd1de65d0 2024-09-05T06:55:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 10 client.27790 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 10 client.27790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 20 client.27790 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e098c0 0x556cd1e0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 4 client.27790 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd1d59860 con 0x556cd1e098c0 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.070+0000 7fe439e08640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e098c0 0x556cd1e0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.071+0000 7fe439e08640 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e098c0 0x556cd1e0bcb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca1da1da1666b6d3 server_cookie=a1c95f8a501bef08 in_seq=0 out_seq=0 2024-09-05T06:55:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.071+0000 7fe41d7fa640 10 client.27790 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe41d7fa640 1 -- 172.21.15.86:0/4274409689 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cd1d59860 con 0x556cd1e098c0 2024-09-05T06:55:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe41d7fa640 10 client.27790 handle_command_reply: tid=0 2024-09-05T06:55:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 client.27790 shutdown 2024-09-05T06:55:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 msgr2=0x556cd1df0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe424009250 tx=0x7fe424030380 comp rx=0 tx=0).stop 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 shutdown_connections 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e098c0 0x556cd1e0bcb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1df1370 0x556cd1dee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1de7b40 0x556cd1dee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe41d7fa640 0 client.27790 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 --2- 172.21.15.86:0/4274409689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1df0da0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 >> 172.21.15.86:0/4274409689 conn(0x556cd1daf310 msgr2=0x556cd1de8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 shutdown_connections 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.072+0000 7fe43e445740 1 -- 172.21.15.86:0/4274409689 wait complete. 2024-09-05T06:55:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.073+0000 7fe43e445740 20 client.27790 trim_cache size 0 max 16384 2024-09-05T06:55:56.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.097+0000 7fe43e445740 1 Processor -- start 2024-09-05T06:55:56.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.097+0000 7fe43e445740 1 -- start start 2024-09-05T06:55:56.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1ded050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1e11370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341ad050 con 0x556cd1df1370 2024-09-05T06:55:56.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341aecb0 con 0x556cd1de7b40 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe43e445740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341ad720 con 0x556cd1de65d0 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:39798/0 (socket says 172.21.15.86:39798) 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 -- 172.21.15.86:0/4115302251 learned_addr learned my addr 172.21.15.86:0/4115302251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439e08640 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1ded050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 -- 172.21.15.86:0/4115302251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 msgr2=0x556cd1ded050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1ded050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 -- 172.21.15.86:0/4115302251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 msgr2=0x556cd1e11370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1e11370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439607640 1 -- 172.21.15.86:0/4115302251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd1dedff0 con 0x556cd1de65d0 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.098+0000 7fe439e08640 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1ded050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:55:56.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.099+0000 7fe439607640 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe424003990 tx=0x7fe424009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3881671c8f271c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.099+0000 7fe41f7fe640 1 -- 172.21.15.86:0/4115302251 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424046070 con 0x556cd1de65d0 2024-09-05T06:55:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.099+0000 7fe41f7fe640 1 -- 172.21.15.86:0/4115302251 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe424038040 con 0x556cd1de65d0 2024-09-05T06:55:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.099+0000 7fe41f7fe640 1 -- 172.21.15.86:0/4115302251 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240055d0 con 0x556cd1de65d0 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- 172.21.15.86:0/4115302251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 msgr2=0x556cd1de2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe424003990 tx=0x7fe424009570 comp rx=0 tx=0).stop 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- 172.21.15.86:0/4115302251 shutdown_connections 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1e11370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de7b40 0x556cd1ded050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 --2- 172.21.15.86:0/4115302251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de65d0 0x556cd1de2da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- 172.21.15.86:0/4115302251 >> 172.21.15.86:0/4115302251 conn(0x556cd1daf310 msgr2=0x556cd1dec940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- 172.21.15.86:0/4115302251 shutdown_connections 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- 172.21.15.86:0/4115302251 wait complete. 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.100+0000 7fe43e445740 1 -- start start 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 Processor -- start 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 0x556cd1deb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de7b40 0x556cd1deb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1deba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341ae520 con 0x556cd1df1370 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341ade20 con 0x556cd1de65d0 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.101+0000 7fe43e445740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341ab050 con 0x556cd1de7b40 2024-09-05T06:55:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe43e445740 10 client.0 _ll_register_callbacks cb 0x7fe424035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:56.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 0x556cd1deb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1deba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1deba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33446/0 (socket says 172.21.15.86:33446) 2024-09-05T06:55:56.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 -- 172.21.15.86:0/3497282188 learned_addr learned my addr 172.21.15.86:0/3497282188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439e08640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de7b40 0x556cd1deb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 -- 172.21.15.86:0/3497282188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de7b40 msgr2=0x556cd1deb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de7b40 0x556cd1deb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 -- 172.21.15.86:0/3497282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 msgr2=0x556cd1deba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1deba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 -- 172.21.15.86:0/3497282188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd1deeb50 con 0x556cd1de65d0 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe439607640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 0x556cd1deb000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe42402a900 tx=0x7fe42402aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3a332ca50656dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.102+0000 7fe41d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:56.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424055030 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7fe424056070 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe424038040 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe424064050 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe43e445740 10 client.27812 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556cd1e742f0 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.103+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe424005560 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe424032440 con 0x556cd1de65d0 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 10 client.27812 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 10 client.27812 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 20 client.27812 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e74950 0x556cd1e76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 4 client.27812 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe424038040 con 0x556cd1e74950 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe439e08640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e74950 0x556cd1e76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe41d7fa640 10 client.27812 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.104+0000 7fe439e08640 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e74950 0x556cd1e76d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93c803cb11c45ccf server_cookie=38a697671fa277d4 in_seq=0 out_seq=0 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe41d7fa640 1 -- 172.21.15.86:0/3497282188 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe424038040 con 0x556cd1e74950 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe41d7fa640 10 client.27812 handle_command_reply: tid=0 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 client.27812 shutdown 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 msgr2=0x556cd1deb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 0x556cd1deb000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe42402a900 tx=0x7fe42402aac0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 shutdown_connections 2024-09-05T06:55:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556cd1e74950 0x556cd1e76d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cd1df1370 0x556cd1deba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cd1de7b40 0x556cd1deb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe41d7fa640 0 client.27812 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 --2- 172.21.15.86:0/3497282188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556cd1de65d0 0x556cd1deb000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 >> 172.21.15.86:0/3497282188 conn(0x556cd1daf310 msgr2=0x556cd1de2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 shutdown_connections 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 1 -- 172.21.15.86:0/3497282188 wait complete. 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.105+0000 7fe43e445740 20 client.27812 trim_cache size 0 max 16384 2024-09-05T06:55:56.109 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 msgr2=0x7fe4200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 0x7fe4200a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe42c09c230 tx=0x7fe42c0ca3f0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 msgr2=0x7fe43419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00b330 tx=0x7fe42c00b7f0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 shutdown_connections 2024-09-05T06:55:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe4200a2cf0 0x7fe4200a51b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe434106f10 0x7fe43419cc70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe434102790 0x7fe4341988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 --2- 172.21.15.86:0/3617473645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe434101de0 0x7fe4341983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 >> 172.21.15.86:0/3617473645 conn(0x7fe4340fd8e0 msgr2=0x7fe4340fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.106+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 shutdown_connections 2024-09-05T06:55:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:56.107+0000 7fe43c06c640 1 -- 172.21.15.86:0/3617473645 wait complete. 2024-09-05T06:55:56.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:56 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4004893975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:56.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:56 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4004893975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:56 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4004893975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:55:56.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10924 10.39 MB/sec warmup 74 sec latency 87.596 ms 2024-09-05T06:55:56.779 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 143.8 --await-for 45.0' 2024-09-05T06:55:56.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 143.8 --await-for 45.0 2024-09-05T06:55:57.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:57 smithi115 ceph-mon[34791]: pgmap v459: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:57 smithi086 ceph-mon[30943]: pgmap v459: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:57.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:57 smithi096 ceph-mon[34468]: pgmap v459: 129 pgs: 129 active+clean; 592 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-05T06:55:57.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:55:57.685 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:55:57.747 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11065 10.36 MB/sec warmup 75 sec latency 82.739 ms 2024-09-05T06:55:58.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 10.30 MB/sec warmup 76 sec latency 222.713 ms 2024-09-05T06:55:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 -- 172.21.15.86:0/3944312883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608106a80 msgr2=0x7f2608106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 --2- 172.21.15.86:0/3944312883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608106a80 0x7f2608106e60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009930 tx=0x7f25f802a420 comp rx=0 tx=0).stop 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 -- 172.21.15.86:0/3944312883 shutdown_connections 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 --2- 172.21.15.86:0/3944312883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608101420 0x7f260810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 --2- 172.21.15.86:0/3944312883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608100a80 0x7f2608100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 --2- 172.21.15.86:0/3944312883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608106a80 0x7f2608106e60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 -- 172.21.15.86:0/3944312883 >> 172.21.15.86:0/3944312883 conn(0x7f26080fc7a0 msgr2=0x7f26080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 -- 172.21.15.86:0/3944312883 shutdown_connections 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.358+0000 7f260dcaf640 1 -- 172.21.15.86:0/3944312883 wait complete. 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 Processor -- start 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 -- start start 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 0x7f2608196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 0x7f2608197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2608110d40 con 0x7f2608101420 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2608110bc0 con 0x7f2608100a80 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.359+0000 7f260dcaf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2608110ec0 con 0x7f2608106a80 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33458/0 (socket says 172.21.15.86:33458) 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 -- 172.21.15.86:0/4086662973 learned_addr learned my addr 172.21.15.86:0/4086662973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f26077fe640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 0x7f2608196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2607fff640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 0x7f2608197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 -- 172.21.15.86:0/4086662973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 msgr2=0x7f2608197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 0x7f2608197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 -- 172.21.15.86:0/4086662973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 msgr2=0x7f2608196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 0x7f2608196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.231 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:55:59 smithi086 ceph-mon[30943]: pgmap v460: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 -- 172.21.15.86:0/4086662973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a6850 con 0x7f2608101420 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f26077fe640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 0x7f2608196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2607fff640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 0x7f2608197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2606ffd640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9f0 tx=0x7f25fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2604ff9640 1 -- 172.21.15.86:0/4086662973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00cd80 con 0x7f2608101420 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2604ff9640 1 -- 172.21.15.86:0/4086662973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc0108e0 con 0x7f2608101420 2024-09-05T06:55:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081a6b40 con 0x7f2608101420 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.360+0000 7f2604ff9640 1 -- 172.21.15.86:0/4086662973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc018920 con 0x7f2608101420 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.361+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a70d0 con 0x7f2608101420 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.363+0000 7f2610088740 1 Processor -- start 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 -- start start 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad688400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad68ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aad5e32a0 con 0x556aad688940 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556aad58ab10 con 0x556aad67d760 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2610088740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556aad5df260 con 0x556aad67e080 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2607fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2607fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44668/0 (socket says 172.21.15.86:44668) 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2607fff640 1 -- 172.21.15.86:0/3514950271 learned_addr learned my addr 172.21.15.86:0/3514950271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f26077fe640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad688400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2604ff9640 1 -- 172.21.15.86:0/4086662973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f25fc010a90 con 0x7f2608101420 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.364+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad68ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 -- 172.21.15.86:0/3514950271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 msgr2=0x556aad688400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad688400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 -- 172.21.15.86:0/3514950271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 msgr2=0x556aad68ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad68ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 -- 172.21.15.86:0/3514950271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aad354b90 con 0x556aad67d760 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad68ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2607fff640 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2600011790 tx=0x7f2600011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61eaf1bd87dc0b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f25edffb640 1 -- 172.21.15.86:0/3514950271 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600003c30 con 0x556aad67d760 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f25edffb640 1 -- 172.21.15.86:0/3514950271 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260000fe50 con 0x556aad67d760 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f25edffb640 1 -- 172.21.15.86:0/3514950271 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000193e0 con 0x556aad67d760 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2604ff9640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 0x7f25f00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f2604ff9640 1 -- 172.21.15.86:0/4086662973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f25fc0cfb20 con 0x7f2608101420 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.365+0000 7f26077fe640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 0x7f25f00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f26077fe640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 0x7f25f00a5300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25f8031440 tx=0x7f25f8005fd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 -- 172.21.15.86:0/3514950271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 msgr2=0x556aad67db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2600011790 tx=0x7f2600011c50 comp rx=0 tx=0).stop 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 -- 172.21.15.86:0/3514950271 shutdown_connections 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad68ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad688400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 --2- 172.21.15.86:0/3514950271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad67db40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 -- 172.21.15.86:0/3514950271 >> 172.21.15.86:0/3514950271 conn(0x556aad679cf0 msgr2=0x556aad67a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 -- 172.21.15.86:0/3514950271 shutdown_connections 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.366+0000 7f2610088740 1 -- 172.21.15.86:0/3514950271 wait complete. 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2610088740 1 -- start start 2024-09-05T06:55:59.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2610088740 1 Processor -- start 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2610088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2610088740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 0x556aad685d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2610088740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad686260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f26077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f26077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44684/0 (socket says 172.21.15.86:44684) 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.367+0000 7f2606ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 0x556aad685d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f26077fe640 1 -- 172.21.15.86:0/1469726057 learned_addr learned my addr 172.21.15.86:0/1469726057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aad68b510 con 0x556aad67e080 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556aad56b6e0 con 0x556aad67d760 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2607fff640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad686260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556aad68b690 con 0x556aad688940 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2610088740 10 client.0 _ll_register_callbacks cb 0x556aad58fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 -- 172.21.15.86:0/1469726057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 msgr2=0x556aad686260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad686260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 -- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 msgr2=0x556aad6857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aad69d210 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2606ffd640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 0x556aad685d20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0ccdd0 tx=0x7f25fc09bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e6cd87dde60f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f25e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc010630 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f25fc09b0a0 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.368+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260000fe50 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f25fc0b3070 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 10 client.18855 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556aad69fb40 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260000fe50 con 0x556aad67e080 2024-09-05T06:55:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f25fc0ba2d0 con 0x556aad67e080 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 10 client.18855 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 10 client.18855 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 20 client.18855 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad6a1000 0x556aad6a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 4 client.18855 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aad5f09d0 con 0x556aad6a1000 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.369+0000 7f26077fe640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad6a1000 0x556aad6a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.370+0000 7f25e6ffd640 10 client.18855 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.370+0000 7f26077fe640 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad6a1000 0x556aad6a3470 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253800dbeaa13b6c server_cookie=858893ab7d0e49f in_seq=0 out_seq=0 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f25e6ffd640 1 -- 172.21.15.86:0/1469726057 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556aad5f09d0 con 0x556aad6a1000 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f25e6ffd640 10 client.18855 handle_command_reply: tid=0 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 client.18855 shutdown 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 msgr2=0x556aad685d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 0x556aad685d20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0ccdd0 tx=0x7f25fc09bb10 comp rx=0 tx=0).stop 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 shutdown_connections 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad6a1000 0x556aad6a3470 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad686260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67e080 0x556aad685d20 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f25e6ffd640 0 client.18855 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 --2- 172.21.15.86:0/1469726057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 >> 172.21.15.86:0/1469726057 conn(0x556aad679cf0 msgr2=0x556aad678e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.371+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 shutdown_connections 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.372+0000 7f2610088740 1 -- 172.21.15.86:0/1469726057 wait complete. 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.372+0000 7f2610088740 20 client.18855 trim_cache size 0 max 16384 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 Processor -- start 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 -- start start 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad6840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ad050 con 0x556aad688940 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26081aecb0 con 0x556aad67d760 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.405+0000 7f2610088740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ad720 con 0x556aad67e080 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:39832/0 (socket says 172.21.15.86:39832) 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 -- 172.21.15.86:0/2227956026 learned_addr learned my addr 172.21.15.86:0/2227956026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2607fff640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad6840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f26077fe640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 -- 172.21.15.86:0/2227956026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 msgr2=0x556aad6801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 -- 172.21.15.86:0/2227956026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 msgr2=0x556aad6840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad6840b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 -- 172.21.15.86:0/2227956026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aad6a3af0 con 0x556aad67e080 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f26077fe640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f2606ffd640 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0a49d0 tx=0x7f25fc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69128bfc5d006083 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f25ed7fa640 1 -- 172.21.15.86:0/2227956026 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc09b5b0 con 0x556aad67e080 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.406+0000 7f25ed7fa640 1 -- 172.21.15.86:0/2227956026 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09bc50 con 0x556aad67e080 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f25ed7fa640 1 -- 172.21.15.86:0/2227956026 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc018ac0 con 0x556aad67e080 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 -- 172.21.15.86:0/2227956026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 msgr2=0x556aad6876b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0a49d0 tx=0x7f25fc000bc0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 -- 172.21.15.86:0/2227956026 shutdown_connections 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad688940 0x556aad6840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad67e080 0x556aad6876b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 --2- 172.21.15.86:0/2227956026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67d760 0x556aad6801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 -- 172.21.15.86:0/2227956026 >> 172.21.15.86:0/2227956026 conn(0x556aad679cf0 msgr2=0x556aad678e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 -- 172.21.15.86:0/2227956026 shutdown_connections 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.407+0000 7f2610088740 1 -- 172.21.15.86:0/2227956026 wait complete. 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 -- start start 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 Processor -- start 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 0x556aad695820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 0x556aad695d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ae520 con 0x556aad67d760 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ade20 con 0x556aad67e080 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ab050 con 0x556aad688940 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.408+0000 7f2610088740 10 client.0 _ll_register_callbacks cb 0x7f25fc097ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f26077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 0x556aad695820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2607fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2607fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:39840/0 (socket says 172.21.15.86:39840) 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f26077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 0x556aad695820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33478/0 (socket says 172.21.15.86:33478) 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2607fff640 1 -- 172.21.15.86:0/3425837418 learned_addr learned my addr 172.21.15.86:0/3425837418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 0x556aad695d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 -- 172.21.15.86:0/3425837418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 msgr2=0x556aad704d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 -- 172.21.15.86:0/3425837418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 msgr2=0x556aad695820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 0x556aad695820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 -- 172.21.15.86:0/3425837418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aad67a980 con 0x556aad67e080 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2607fff640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2606ffd640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 0x556aad695d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x556aad67ed10 tx=0x7f25fc09b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4b0344945b32e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f25e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc09b5b0 con 0x556aad67e080 2024-09-05T06:55:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9605+0+0 (secure 0 0 0) 0x7f25fc0ba2a0 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.409+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc09bc50 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f25fc0ab050 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 10 client.27830 fetch_fsmap learned FSMap version 38 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556aad675260 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09bc50 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f25fc09ab10 con 0x556aad67e080 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 10 client.27830 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 10 client.27830 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 20 client.27830 populate_metadata read hostname 'smithi086' 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad71d8b0 0x556aad71fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 4 client.27830 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.8, "await_for": 45.0}] 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.8, "await_for": 45.0}) -- 0x556aad5ead20 con 0x556aad71d8b0 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.410+0000 7f26077fe640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad71d8b0 0x556aad71fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:55:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.411+0000 7f26077fe640 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad71d8b0 0x556aad71fcd0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71b54edaa5fc3ef server_cookie=4ebb57c620130897 in_seq=0 out_seq=0 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.411+0000 7f25e6ffd640 10 client.27830 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f25e6ffd640 1 -- 172.21.15.86:0/3425837418 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556aad5ead20 con 0x556aad71d8b0 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f25e6ffd640 10 client.27830 handle_command_reply: tid=0 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 client.27830 shutdown 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 msgr2=0x556aad695d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 0x556aad695d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x556aad67ed10 tx=0x7f25fc09b350 comp rx=0 tx=0).stop 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 shutdown_connections 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556aad71d8b0 0x556aad71fcd0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556aad688940 0x556aad704d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f25e6ffd640 0 client.27830 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556aad67e080 0x556aad695d60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.649+0000 7f2610088740 1 --2- 172.21.15.86:0/3425837418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aad67d760 0x556aad695820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 38, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 54, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "95100888": { 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 54, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:55:59.247 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 143.8, 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:55:59.248 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.650+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 >> 172.21.15.86:0/3425837418 conn(0x556aad679cf0 msgr2=0x556aad6857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.650+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 shutdown_connections 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.650+0000 7f2610088740 1 -- 172.21.15.86:0/3425837418 wait complete. 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.650+0000 7f2610088740 20 client.27830 trim_cache size 0 max 16384 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 msgr2=0x7f25f00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 0x7f25f00a5300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25f8031440 tx=0x7f25f8005fd0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 msgr2=0x7f2608197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9f0 tx=0x7f25fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 shutdown_connections 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f25f00a2e40 0x7f25f00a5300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2608106a80 0x7f2608197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2608101420 0x7f2608197520 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 --2- 172.21.15.86:0/4086662973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2608100a80 0x7f2608196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 >> 172.21.15.86:0/4086662973 conn(0x7f26080fc7a0 msgr2=0x7f26080fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 shutdown_connections 2024-09-05T06:55:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:55:58.651+0000 7f260dcaf640 1 -- 172.21.15.86:0/4086662973 wait complete. 2024-09-05T06:55:59.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:55:59 smithi115 ceph-mon[34791]: pgmap v460: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-09-05T06:55:59.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:55:59 smithi096 ceph-mon[34468]: pgmap v460: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-09-05T06:55:59.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 10.16 MB/sec warmup 77 sec latency 1222.767 ms 2024-09-05T06:56:00.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 10.03 MB/sec warmup 78 sec latency 2222.880 ms 2024-09-05T06:56:01.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:01 smithi115 ceph-mon[34791]: pgmap v461: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 201 op/s 2024-09-05T06:56:01.478 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:01.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:01 smithi086 ceph-mon[30943]: pgmap v461: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 201 op/s 2024-09-05T06:56:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:01 smithi096 ceph-mon[34468]: pgmap v461: 129 pgs: 129 active+clean; 606 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 201 op/s 2024-09-05T06:56:01.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.90 MB/sec warmup 79 sec latency 3222.932 ms 2024-09-05T06:56:02.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.78 MB/sec warmup 80 sec latency 4223.004 ms 2024-09-05T06:56:03.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:03 smithi115 ceph-mon[34791]: pgmap v462: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-09-05T06:56:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:03 smithi086 ceph-mon[30943]: pgmap v462: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-09-05T06:56:03.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:03 smithi096 ceph-mon[34468]: pgmap v462: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-09-05T06:56:03.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.66 MB/sec warmup 81 sec latency 5223.081 ms 2024-09-05T06:56:04.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.54 MB/sec warmup 82 sec latency 6223.178 ms 2024-09-05T06:56:05.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:05 smithi115 ceph-mon[34791]: pgmap v463: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:05.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:05 smithi086 ceph-mon[30943]: pgmap v463: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:05.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:05 smithi096 ceph-mon[34468]: pgmap v463: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:05.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.43 MB/sec warmup 83 sec latency 7223.268 ms 2024-09-05T06:56:06.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.31 MB/sec warmup 84 sec latency 8223.342 ms 2024-09-05T06:56:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:07 smithi115 ceph-mon[34791]: pgmap v464: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:07 smithi115 ceph-mon[34791]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T06:56:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:07 smithi086 ceph-mon[30943]: pgmap v464: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:07 smithi086 ceph-mon[30943]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T06:56:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:07 smithi096 ceph-mon[34468]: pgmap v464: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-09-05T06:56:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:07 smithi096 ceph-mon[34468]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T06:56:07.624 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '95100888', quiesce took 13.8 seconds. Will release after: 10.0 2024-09-05T06:56:07.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.21 MB/sec warmup 85 sec latency 9223.401 ms 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/2960307871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 msgr2=0x7fe8c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/2960307871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8101200 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402a480 comp rx=0 tx=0).stop 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/2960307871 shutdown_connections 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/2960307871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 0x7fe8c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/2960307871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/2960307871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8101200 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/2960307871 >> 172.21.15.86:0/2960307871 conn(0x7fe8c80fc7d0 msgr2=0x7fe8c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/2960307871 shutdown_connections 2024-09-05T06:56:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.885+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/2960307871 wait complete. 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.886+0000 7fe8cf9b6640 1 Processor -- start 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.886+0000 7fe8cf9b6640 1 -- start start 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 0x7fe8c819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8110e50 con 0x7fe8c8100e20 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8110cd0 con 0x7fe8c8102ef0 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cf9b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8110fd0 con 0x7fe8c8102570 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:46770/0 (socket says 172.21.15.86:46770) 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 -- 172.21.15.86:0/851760514 learned_addr learned my addr 172.21.15.86:0/851760514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cd752640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 0x7fe8c819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 -- 172.21.15.86:0/851760514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 msgr2=0x7fe8c819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 0x7fe8c819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 -- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 msgr2=0x7fe8c8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.887+0000 7fe8ccf51640 1 -- 172.21.15.86:0/851760514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c819d3c0 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c920 tx=0x7fe8b800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8be7fc640 1 -- 172.21.15.86:0/851760514 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8007c70 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c819d6b0 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8be7fc640 1 -- 172.21.15.86:0/851760514 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b8007e10 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8be7fc640 1 -- 172.21.15.86:0/851760514 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8018820 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.888+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a4f90 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.890+0000 7fe8be7fc640 1 -- 172.21.15.86:0/851760514 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe8b80189c0 con 0x7fe8c8102570 2024-09-05T06:56:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 Processor -- start 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 -- start start 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efa9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea0ef0f2f0 con 0x55ea0efa95d0 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ea0eeb6cf0 con 0x55ea0efb4370 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ea0ef0b2b0 con 0x55ea0efaab40 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59368/0 (socket says 172.21.15.86:59368) 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 -- 172.21.15.86:0/1311732706 learned_addr learned my addr 172.21.15.86:0/1311732706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efa9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8be7fc640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 0x7fe8b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 -- 172.21.15.86:0/1311732706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 msgr2=0x55ea0efa9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efa9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 -- 172.21.15.86:0/1311732706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 msgr2=0x55ea0efb6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efb6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.891+0000 7fe8cd752640 1 -- 172.21.15.86:0/1311732706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0ee9b3a0 con 0x55ea0efa95d0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efa9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 0x7fe8b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8be7fc640 1 -- 172.21.15.86:0/851760514 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fe8b80cf960 con 0x7fe8c8102570 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8cd752640 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c4032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead13e2c7350815d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8b37fe640 1 -- 172.21.15.86:0/1311732706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035070 con 0x55ea0efa95d0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8b37fe640 1 -- 172.21.15.86:0/1311732706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c40316f0 con 0x55ea0efa95d0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.892+0000 7fe8b37fe640 1 -- 172.21.15.86:0/1311732706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4003d70 con 0x55ea0efa95d0 2024-09-05T06:56:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/1311732706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 msgr2=0x55ea0efa9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c4032e20 comp rx=0 tx=0).stop 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 0x7fe8b40a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000a5a0 tx=0x7fe8c0004b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/1311732706 shutdown_connections 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efb6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efa9a10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/1311732706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efa9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/1311732706 >> 172.21.15.86:0/1311732706 conn(0x55ea0ef72310 msgr2=0x55ea0efa5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/1311732706 shutdown_connections 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/1311732706 wait complete. 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 -- start start 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.893+0000 7fe8d1d8f740 1 Processor -- start 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 0x55ea0efb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efb4370 0x55ea0efb15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea0efb7010 con 0x55ea0efaab40 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ea0eeca060 con 0x55ea0efa95d0 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ea0efb7190 con 0x55ea0efb4370 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8d1d8f740 10 client.0 _ll_register_callbacks cb 0x55ea0ec1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59378/0 (socket says 172.21.15.86:59378) 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 -- 172.21.15.86:0/3472079398 learned_addr learned my addr 172.21.15.86:0/3472079398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 0x55ea0efb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efb4370 0x55ea0efb15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 -- 172.21.15.86:0/3472079398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efb4370 msgr2=0x55ea0efb15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efb4370 0x55ea0efb15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 -- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 msgr2=0x55ea0efb3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 0x55ea0efb3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 -- 172.21.15.86:0/3472079398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0ec803c0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 0x55ea0efb3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80ccc80 tx=0x7fe8b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caac05b61ca928fd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.894+0000 7fe8b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8018cb0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fe8b80a41d0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c40316f0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe8b80cd0a0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 10 client.18873 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ea0efc2aa0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c40316f0 con 0x55ea0efaab40 2024-09-05T06:56:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe8b80860b0 con 0x55ea0efaab40 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 10 client.18873 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 10 client.18873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 20 client.18873 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0efc3f60 0x55ea0efc63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 4 client.18873 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0ef1c860 con 0x55ea0efc3f60 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.895+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0efc3f60 0x55ea0efc63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.896+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0efc3f60 0x55ea0efc63c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e63b12b0c7c42f7 server_cookie=fd6948902094c158 in_seq=0 out_seq=0 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.896+0000 7fe8b0ff9640 10 client.18873 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.896+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/3472079398 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0ef1c860 con 0x55ea0efc3f60 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8b0ff9640 10 client.18873 handle_command_reply: tid=0 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 client.18873 shutdown 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 msgr2=0x55ea0efb0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80ccc80 tx=0x7fe8b800a620 comp rx=0 tx=0).stop 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 shutdown_connections 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0efc3f60 0x55ea0efc63c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efb4370 0x55ea0efb15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efaab40 0x55ea0efb0fe0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8b0ff9640 0 client.18873 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3472079398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efa95d0 0x55ea0efb3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 >> 172.21.15.86:0/3472079398 conn(0x55ea0ef72310 msgr2=0x55ea0efab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 shutdown_connections 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3472079398 wait complete. 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.897+0000 7fe8d1d8f740 20 client.18873 trim_cache size 0 max 16384 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 Processor -- start 2024-09-05T06:56:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 -- start start 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81af0b0 con 0x55ea0efa95d0 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b0d10 con 0x55ea0efb4370 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.910+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81af780 con 0x55ea0efaab40 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59384/0 (socket says 172.21.15.86:59384) 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 -- 172.21.15.86:0/3653549080 learned_addr learned my addr 172.21.15.86:0/3653549080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cdf53640 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 -- 172.21.15.86:0/3653549080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 msgr2=0x55ea0efac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 -- 172.21.15.86:0/3653549080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 msgr2=0x55ea0efac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 -- 172.21.15.86:0/3653549080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0efb40a0 con 0x55ea0efa95d0 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8cd752640 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031a20 tx=0x7fe8c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d842b624a69aa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8b2ffd640 1 -- 172.21.15.86:0/3653549080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035040 con 0x55ea0efa95d0 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8b2ffd640 1 -- 172.21.15.86:0/3653549080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c4003780 con 0x55ea0efa95d0 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.911+0000 7fe8b2ffd640 1 -- 172.21.15.86:0/3653549080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40456d0 con 0x55ea0efa95d0 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3653549080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 msgr2=0x55ea0efc02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031a20 tx=0x7fe8c4005fd0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3653549080 shutdown_connections 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/3653549080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efc02e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3653549080 >> 172.21.15.86:0/3653549080 conn(0x55ea0ef72310 msgr2=0x55ea0efb00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3653549080 shutdown_connections 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.912+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/3653549080 wait complete. 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 -- start start 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 Processor -- start 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efadfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b0580 con 0x55ea0efa95d0 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81afe80 con 0x55ea0efb4370 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ad0b0 con 0x55ea0efaab40 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8d1d8f740 10 client.0 _ll_register_callbacks cb 0x7fe8b800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.913+0000 7fe8cd752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59386/0 (socket says 172.21.15.86:59386) 2024-09-05T06:56:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 -- 172.21.15.86:0/2383414534 learned_addr learned my addr 172.21.15.86:0/2383414534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 -- 172.21.15.86:0/2383414534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 msgr2=0x55ea0efada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 -- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 msgr2=0x55ea0efadfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efadfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 -- 172.21.15.86:0/2383414534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0efae530 con 0x55ea0efa95d0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8cd752640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe8c402a9a0 tx=0x7fe8c4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677fd6bc55d7f8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4065060 con 0x55ea0efa95d0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fe8c40316f0 con 0x55ea0efa95d0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.914+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c4003780 con 0x55ea0efa95d0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe8c4035040 con 0x55ea0efa95d0 2024-09-05T06:56:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 10 client.18879 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ea0f02e990 con 0x55ea0efa95d0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c4003780 con 0x55ea0efa95d0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe8c4045a20 con 0x55ea0efa95d0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 10 client.18879 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 10 client.18879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 20 client.18879 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0f02ebb0 0x55ea0f030fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 4 client.18879 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0ef16d70 con 0x55ea0f02ebb0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.915+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0f02ebb0 0x55ea0f030fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8ccf51640 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0f02ebb0 0x55ea0f030fa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac40762ec71eb2d1 server_cookie=6db7e84d530a87b1 in_seq=0 out_seq=0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8b0ff9640 10 client.18879 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8b0ff9640 1 -- 172.21.15.86:0/2383414534 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0ef16d70 con 0x55ea0f02ebb0 2024-09-05T06:56:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8b0ff9640 10 client.18879 handle_command_reply: tid=0 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 client.18879 shutdown 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 msgr2=0x55ea0efbc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe8c402a9a0 tx=0x7fe8c4005e00 comp rx=0 tx=0).stop 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 shutdown_connections 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ea0f02ebb0 0x55ea0f030fa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ea0efb4370 0x55ea0efadfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea0efaab40 0x55ea0efada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 --2- 172.21.15.86:0/2383414534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea0efa95d0 0x55ea0efbc960 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 >> 172.21.15.86:0/2383414534 conn(0x55ea0ef72310 msgr2=0x55ea0efb2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8b0ff9640 0 client.18879 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 shutdown_connections 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 1 -- 172.21.15.86:0/2383414534 wait complete. 2024-09-05T06:56:07.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.916+0000 7fe8d1d8f740 20 client.18879 trim_cache size 0 max 16384 2024-09-05T06:56:07.947 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:07.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 msgr2=0x7fe8b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 0x7fe8b40a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000a5a0 tx=0x7fe8c0004b50 comp rx=0 tx=0).stop 2024-09-05T06:56:07.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 msgr2=0x7fe8c8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c920 tx=0x7fe8b800cde0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 shutdown_connections 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe8b40a2cf0 0x7fe8b40a51b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c8102ef0 0x7fe8c819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8c8102570 0x7fe8c8198950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 --2- 172.21.15.86:0/851760514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8100e20 0x7fe8c8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 >> 172.21.15.86:0/851760514 conn(0x7fe8c80fc7d0 msgr2=0x7fe8c806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 shutdown_connections 2024-09-05T06:56:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:07.917+0000 7fe8cf9b6640 1 -- 172.21.15.86:0/851760514 wait complete. 2024-09-05T06:56:08.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 9.10 MB/sec warmup 86 sec latency 10223.428 ms 2024-09-05T06:56:08.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:56:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:09.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:09 smithi115 ceph-mon[34791]: pgmap v466: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:09.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:09 smithi086 ceph-mon[30943]: pgmap v466: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:09.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:09 smithi096 ceph-mon[34468]: pgmap v466: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:09.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.99 MB/sec warmup 87 sec latency 11223.480 ms 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 -- 172.21.15.86:0/2214347110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080101de0 msgr2=0x7fc0801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 --2- 172.21.15.86:0/2214347110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080101de0 0x7fc0801021c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc070009900 tx=0x7fc07002a410 comp rx=0 tx=0).stop 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 -- 172.21.15.86:0/2214347110 shutdown_connections 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 --2- 172.21.15.86:0/2214347110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080106f10 0x7fc08010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 --2- 172.21.15.86:0/2214347110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080102790 0x7fc0801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 --2- 172.21.15.86:0/2214347110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080101de0 0x7fc0801021c0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 -- 172.21.15.86:0/2214347110 >> 172.21.15.86:0/2214347110 conn(0x7fc0800fd8c0 msgr2=0x7fc0800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 -- 172.21.15.86:0/2214347110 shutdown_connections 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 -- 172.21.15.86:0/2214347110 wait complete. 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.804+0000 7fc08555c640 1 Processor -- start 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 -- start start 2024-09-05T06:56:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080101de0 0x7fc0801982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102790 0x7fc0801987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc080111de0 con 0x7fc080106f10 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc080111c60 con 0x7fc080102790 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc08555c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc080111f60 con 0x7fc080101de0 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59408/0 (socket says 172.21.15.86:59408) 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4277744561 learned_addr learned my addr 172.21.15.86:0/4277744561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc077fff640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102790 0x7fc0801987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07effd640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080101de0 0x7fc0801982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4277744561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080101de0 msgr2=0x7fc0801982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080101de0 0x7fc0801982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4277744561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102790 msgr2=0x7fc0801987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102790 0x7fc0801987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4277744561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08019d260 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fc07800cad0 tx=0x7fc07800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.805+0000 7fc07cff9640 1 -- 172.21.15.86:0/4277744561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800ad60 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.806+0000 7fc07cff9640 1 -- 172.21.15.86:0/4277744561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc078005020 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.806+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08019d550 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.806+0000 7fc07cff9640 1 -- 172.21.15.86:0/4277744561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f6b0 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.806+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a4e30 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc07cff9640 1 -- 172.21.15.86:0/4277744561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc0780051d0 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc087935740 1 Processor -- start 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc087935740 1 -- start start 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc07cff9640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 0x7fc06c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc07cff9640 1 -- 172.21.15.86:0/4277744561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fc0780cf890 con 0x7fc080106f10 2024-09-05T06:56:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.808+0000 7fc077fff640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 0x7fc06c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f168090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f168a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c6f0ce2f0 con 0x563c6f177780 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563c6f075cf0 con 0x563c6f169b40 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc087935740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563c6f0ca2b0 con 0x563c6f1685d0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59412/0 (socket says 172.21.15.86:59412) 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4226956173 learned_addr learned my addr 172.21.15.86:0/4226956173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07effd640 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc077fff640 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc077fff640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 0x7fc06c0a51b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0800fdce0 tx=0x7fc06800f3c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4226956173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 msgr2=0x563c6f168a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f168a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4226956173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 msgr2=0x563c6f168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f168090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 -- 172.21.15.86:0/4226956173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6f05a3a0 con 0x563c6f177780 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc0780a4000 tx=0x7fc07809b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d570404d235d68ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc0757fa640 1 -- 172.21.15.86:0/4226956173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07801d040 con 0x563c6f177780 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc0757fa640 1 -- 172.21.15.86:0/4226956173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc078099bc0 con 0x563c6f177780 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.809+0000 7fc0757fa640 1 -- 172.21.15.86:0/4226956173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078086070 con 0x563c6f177780 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 -- 172.21.15.86:0/4226956173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 msgr2=0x563c6f179b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc0780a4000 tx=0x7fc07809b8c0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 -- 172.21.15.86:0/4226956173 shutdown_connections 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f177780 0x563c6f179b70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 --2- 172.21.15.86:0/4226956173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f168090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 -- 172.21.15.86:0/4226956173 >> 172.21.15.86:0/4226956173 conn(0x563c6f131310 msgr2=0x563c6f164c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 -- 172.21.15.86:0/4226956173 shutdown_connections 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.810+0000 7fc087935740 1 -- 172.21.15.86:0/4226956173 wait complete. 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 -- start start 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 Processor -- start 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f177200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f174bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c6f17a480 con 0x563c6f169b40 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563c6f089060 con 0x563c6f177780 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563c6f17a600 con 0x563c6f1685d0 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc087935740 10 client.0 _ll_register_callbacks cb 0x563c6eddc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59418/0 (socket says 172.21.15.86:59418) 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f174bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 -- 172.21.15.86:0/3634167397 learned_addr learned my addr 172.21.15.86:0/3634167397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc07effd640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f177200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 -- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 msgr2=0x563c6f177200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f177200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 -- 172.21.15.86:0/3634167397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 msgr2=0x563c6f174bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f174bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc077fff640 1 -- 172.21.15.86:0/3634167397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6ee3f3c0 con 0x563c6f169b40 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.811+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f174bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc077fff640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc068010de0 tx=0x7fc068004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e70e3f6b14ac248 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06802a060 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fc0680255f0 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc078099bc0 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc068031050 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 10 client.18891 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563c6f186010 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc078099bc0 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc068024030 con 0x563c6f169b40 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 10 client.18891 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 10 client.18891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 20 client.18891 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f187520 0x563c6f189940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 4 client.18891 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.812+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6f0db860 con 0x563c6f187520 2024-09-05T06:56:09.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.813+0000 7fc07effd640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f187520 0x563c6f189940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.813+0000 7fc07effd640 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f187520 0x563c6f189940 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13f9d56d1e6ecea6 server_cookie=d94d48eddcac4e2f in_seq=0 out_seq=0 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.813+0000 7fc0667fc640 10 client.18891 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc0667fc640 1 -- 172.21.15.86:0/3634167397 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c6f0db860 con 0x563c6f187520 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc0667fc640 10 client.18891 handle_command_reply: tid=0 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 client.18891 shutdown 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 msgr2=0x563c6f1744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc068010de0 tx=0x7fc068004730 comp rx=0 tx=0).stop 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 shutdown_connections 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f187520 0x563c6f189940 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f174bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1744b0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc0667fc640 0 client.18891 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 --2- 172.21.15.86:0/3634167397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f177200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 >> 172.21.15.86:0/3634167397 conn(0x563c6f131310 msgr2=0x563c6f16a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 shutdown_connections 2024-09-05T06:56:09.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 1 -- 172.21.15.86:0/3634167397 wait complete. 2024-09-05T06:56:09.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.814+0000 7fc087935740 20 client.18891 trim_cache size 0 max 16384 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 Processor -- start 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 -- start start 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f177780 0x563c6f18f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801acfd0 con 0x563c6f1685d0 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801aec30 con 0x563c6f169b40 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc087935740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801ad6a0 con 0x563c6f177780 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc07effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc07effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59424/0 (socket says 172.21.15.86:59424) 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc07effd640 1 -- 172.21.15.86:0/4028738711 learned_addr learned my addr 172.21.15.86:0/4028738711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc077fff640 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc07effd640 1 -- 172.21.15.86:0/4028738711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f177780 msgr2=0x563c6f18f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.827+0000 7fc07effd640 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f177780 0x563c6f18f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc07effd640 1 -- 172.21.15.86:0/4028738711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 msgr2=0x563c6f163bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc07effd640 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f163bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc07effd640 1 -- 172.21.15.86:0/4028738711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6f183c60 con 0x563c6f1685d0 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc077fff640 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc07effd640 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fc07002a930 tx=0x7fc070005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe5b60a9c91b23b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc074ff9640 1 -- 172.21.15.86:0/4028738711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07004a030 con 0x563c6f1685d0 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc074ff9640 1 -- 172.21.15.86:0/4028738711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0700319d0 con 0x563c6f1685d0 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.828+0000 7fc074ff9640 1 -- 172.21.15.86:0/4028738711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070035040 con 0x563c6f1685d0 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- 172.21.15.86:0/4028738711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 msgr2=0x563c6f183720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fc07002a930 tx=0x7fc070005e80 comp rx=0 tx=0).stop 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- 172.21.15.86:0/4028738711 shutdown_connections 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f177780 0x563c6f18f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f169b40 0x563c6f163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- 172.21.15.86:0/4028738711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f1685d0 0x563c6f183720 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- 172.21.15.86:0/4028738711 >> 172.21.15.86:0/4028738711 conn(0x563c6f131310 msgr2=0x563c6f1634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- 172.21.15.86:0/4028738711 shutdown_connections 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- 172.21.15.86:0/4028738711 wait complete. 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- start start 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 Processor -- start 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f180030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f171720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801ae4a0 con 0x563c6f169b40 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801adda0 con 0x563c6f177780 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801aafd0 con 0x563c6f1685d0 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc087935740 10 client.0 _ll_register_callbacks cb 0x7fc06800fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59438/0 (socket says 172.21.15.86:59438) 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.829+0000 7fc077fff640 1 -- 172.21.15.86:0/3168656365 learned_addr learned my addr 172.21.15.86:0/3168656365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc07effd640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f180030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc07f7fe640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f171720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 -- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 msgr2=0x563c6f180030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f180030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 -- 172.21.15.86:0/3168656365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 msgr2=0x563c6f171720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f171720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 -- 172.21.15.86:0/3168656365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6f171f50 con 0x563c6f169b40 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc077fff640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc06800f8c0 tx=0x7fc06801bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e374a356701f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc0667fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068018070 con 0x563c6f169b40 2024-09-05T06:56:09.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0700319d0 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7fc06801bda0 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc06802a030 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc087935740 10 client.18897 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563c6f1f2120 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.830+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0700319d0 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc068024040 con 0x563c6f169b40 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 10 client.18897 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 10 client.18897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 20 client.18897 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f1f2770 0x563c6f1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 4 client.18897 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c6f0d5d70 con 0x563c6f1f2770 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc07effd640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f1f2770 0x563c6f1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc0667fc640 10 client.18897 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.831+0000 7fc07effd640 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f1f2770 0x563c6f1f4b90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd05e2ec66d61111 server_cookie=896ce29c7171ef3f in_seq=0 out_seq=0 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc0667fc640 1 -- 172.21.15.86:0/3168656365 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c6f0d5d70 con 0x563c6f1f2770 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc0667fc640 10 client.18897 handle_command_reply: tid=0 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 client.18897 shutdown 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 msgr2=0x563c6f1711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc06800f8c0 tx=0x7fc06801bd30 comp rx=0 tx=0).stop 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 shutdown_connections 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x563c6f1f2770 0x563c6f1f4b90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c6f177780 0x563c6f171720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c6f169b40 0x563c6f1711e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 --2- 172.21.15.86:0/3168656365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c6f1685d0 0x563c6f180030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc0667fc640 0 client.18897 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 >> 172.21.15.86:0/3168656365 conn(0x563c6f131310 msgr2=0x563c6f176530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 shutdown_connections 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 1 -- 172.21.15.86:0/3168656365 wait complete. 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc087935740 20 client.18897 trim_cache size 0 max 16384 2024-09-05T06:56:09.841 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.832+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 msgr2=0x7fc06c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 0x7fc06c0a51b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0800fdce0 tx=0x7fc06800f3c0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 msgr2=0x7fc08019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fc07800cad0 tx=0x7fc07800ced0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 shutdown_connections 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc06c0a2cf0 0x7fc06c0a51b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080106f10 0x7fc08019cb80 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102790 0x7fc0801987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 --2- 172.21.15.86:0/4277744561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc080101de0 0x7fc0801982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 >> 172.21.15.86:0/4277744561 conn(0x7fc0800fd8c0 msgr2=0x7fc0800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 shutdown_connections 2024-09-05T06:56:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:09.833+0000 7fc08555c640 1 -- 172.21.15.86:0/4277744561 wait complete. 2024-09-05T06:56:10.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.89 MB/sec warmup 88 sec latency 12223.582 ms 2024-09-05T06:56:10.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:56:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:11.415 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:11 smithi086 ceph-mon[30943]: pgmap v467: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:11.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:11 smithi115 ceph-mon[34791]: pgmap v467: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:11 smithi096 ceph-mon[34468]: pgmap v467: 129 pgs: 129 active+clean; 595 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 -- 172.21.15.86:0/2228705661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 msgr2=0x7f4ce0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 --2- 172.21.15.86:0/2228705661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce0100ee0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009930 tx=0x7f4cc802a420 comp rx=0 tx=0).stop 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 -- 172.21.15.86:0/2228705661 shutdown_connections 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 --2- 172.21.15.86:0/2228705661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 0x7f4ce010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 --2- 172.21.15.86:0/2228705661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce0100ee0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 --2- 172.21.15.86:0/2228705661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 0x7f4ce0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 -- 172.21.15.86:0/2228705661 >> 172.21.15.86:0/2228705661 conn(0x7f4ce00fc7a0 msgr2=0x7f4ce00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 -- 172.21.15.86:0/2228705661 shutdown_connections 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 -- 172.21.15.86:0/2228705661 wait complete. 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.614+0000 7f4ce531a640 1 Processor -- start 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 -- start start 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 0x7f4ce0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 0x7f4ce019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0110d80 con 0x7f4ce0100a80 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0110c00 con 0x7f4ce0106a80 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4ce531a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0110f00 con 0x7f4ce0101420 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59464/0 (socket says 172.21.15.86:59464) 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1602325503 learned_addr learned my addr 172.21.15.86:0/1602325503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:11.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 0x7f4ce019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 0x7f4ce0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1602325503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 msgr2=0x7f4ce0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 0x7f4ce0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1602325503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 msgr2=0x7f4ce019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 0x7f4ce019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.615+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1602325503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce019d290 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000d950 tx=0x7f4cd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4cdcff9640 1 -- 172.21.15.86:0/1602325503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0017070 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce019d580 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4cdcff9640 1 -- 172.21.15.86:0/1602325503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cd0009c30 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4cdcff9640 1 -- 172.21.15.86:0/1602325503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000f620 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.616+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce01a4e60 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.618+0000 7f4cdcff9640 1 -- 172.21.15.86:0/1602325503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4cd00109a0 con 0x7f4ce0100a80 2024-09-05T06:56:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 Processor -- start 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 -- start start 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434f0b40 0x55da434ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da434552f0 con 0x55da434fa370 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da433fccf0 con 0x55da434ef5d0 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da434512b0 con 0x55da434f0b40 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4cdcff9640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 0x7f4ccc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59472/0 (socket says 172.21.15.86:59472) 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4cd67fc640 1 -- 172.21.15.86:0/222712271 learned_addr learned my addr 172.21.15.86:0/222712271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.619+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 0x7f4ccc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cdcff9640 1 -- 172.21.15.86:0/1602325503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7f4cd00cf460 con 0x7f4ce0100a80 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434f0b40 0x55da434ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 -- 172.21.15.86:0/222712271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434f0b40 msgr2=0x55da434ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434f0b40 0x55da434ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 -- 172.21.15.86:0/222712271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 msgr2=0x55da434efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 -- 172.21.15.86:0/222712271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da433e13a0 con 0x55da434fa370 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 0x7f4ccc0a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80077f0 tx=0x7f4cd8007780 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802aa10 tx=0x7f4cc8004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b9b87f2918e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd57fa640 1 -- 172.21.15.86:0/222712271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8035040 con 0x55da434fa370 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd57fa640 1 -- 172.21.15.86:0/222712271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc8004750 con 0x55da434fa370 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.620+0000 7f4cd57fa640 1 -- 172.21.15.86:0/222712271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8003850 con 0x55da434fa370 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- 172.21.15.86:0/222712271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 msgr2=0x55da434fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802aa10 tx=0x7f4cc8004310 comp rx=0 tx=0).stop 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- 172.21.15.86:0/222712271 shutdown_connections 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434fa370 0x55da434fc760 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/222712271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434f0b40 0x55da434ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- 172.21.15.86:0/222712271 >> 172.21.15.86:0/222712271 conn(0x55da434b8310 msgr2=0x55da434ebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- 172.21.15.86:0/222712271 shutdown_connections 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- 172.21.15.86:0/222712271 wait complete. 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 -- start start 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.621+0000 7f4ce76f3740 1 Processor -- start 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da434fd010 con 0x55da434f0b40 2024-09-05T06:56:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da43410060 con 0x55da434ef5d0 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da434fd190 con 0x55da434fa370 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 10 client.0 _ll_register_callbacks cb 0x55da431638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59486/0 (socket says 172.21.15.86:59486) 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 -- 172.21.15.86:0/3189645994 learned_addr learned my addr 172.21.15.86:0/3189645994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 -- 172.21.15.86:0/3189645994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 msgr2=0x55da434f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 -- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 msgr2=0x55da434f9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434f9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 -- 172.21.15.86:0/3189645994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da431c63c0 con 0x55da434f0b40 2024-09-05T06:56:11.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000c00 tx=0x7f4cc8031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5716b6edcc656ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8035030 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7f4cc8031c10 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.622+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc8004750 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4cc8057050 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 10 client.18909 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55da43508a60 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc8004750 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f4cc804e4b0 con 0x55da434f0b40 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 10 client.18909 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 20 client.18909 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43509db0 0x55da4350c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 4 client.18909 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da43462860 con 0x55da43509db0 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.623+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43509db0 0x55da4350c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43509db0 0x55da4350c1a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecfd6fea319b881 server_cookie=35f340b3126f14a in_seq=0 out_seq=0 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4cc67fc640 10 client.18909 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4cc67fc640 1 -- 172.21.15.86:0/3189645994 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da43462860 con 0x55da43509db0 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4cc67fc640 10 client.18909 handle_command_reply: tid=0 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4ce76f3740 1 client.18909 shutdown 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 msgr2=0x55da434f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.624+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000c00 tx=0x7f4cc8031810 comp rx=0 tx=0).stop 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 shutdown_connections 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43509db0 0x55da4350c1a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4cc67fc640 0 client.18909 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da434f6fd0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/3189645994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da434f9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 >> 172.21.15.86:0/3189645994 conn(0x55da434b8310 msgr2=0x55da434f1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 shutdown_connections 2024-09-05T06:56:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 1 -- 172.21.15.86:0/3189645994 wait complete. 2024-09-05T06:56:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.625+0000 7f4ce76f3740 20 client.18909 trim_cache size 0 max 16384 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 Processor -- start 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 -- start start 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434f0b40 0x55da434f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01acf50 con 0x55da434ef5d0 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01aebb0 con 0x55da434f0b40 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01ad620 con 0x55da434fa370 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59494/0 (socket says 172.21.15.86:59494) 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1472600138 learned_addr learned my addr 172.21.15.86:0/1472600138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434f0b40 0x55da434f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.638+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1472600138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 msgr2=0x55da434f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1472600138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434f0b40 msgr2=0x55da434f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434f0b40 0x55da434f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 -- 172.21.15.86:0/1472600138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da434f5f80 con 0x55da434ef5d0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4cd009b240 tx=0x7f4cd006d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bb14dda1c25faa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cd4ff9640 1 -- 172.21.15.86:0/1472600138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0020040 con 0x55da434ef5d0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cd4ff9640 1 -- 172.21.15.86:0/1472600138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cd009f840 con 0x55da434ef5d0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.639+0000 7f4cd4ff9640 1 -- 172.21.15.86:0/1472600138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00cf240 con 0x55da434ef5d0 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- 172.21.15.86:0/1472600138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 msgr2=0x55da434f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4cd009b240 tx=0x7f4cd006d500 comp rx=0 tx=0).stop 2024-09-05T06:56:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- 172.21.15.86:0/1472600138 shutdown_connections 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434f0b40 0x55da434f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/1472600138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434ef5d0 0x55da434f6890 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- 172.21.15.86:0/1472600138 >> 172.21.15.86:0/1472600138 conn(0x55da434b8310 msgr2=0x55da434f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- 172.21.15.86:0/1472600138 shutdown_connections 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- 172.21.15.86:0/1472600138 wait complete. 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- start start 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 Processor -- start 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da43502420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01ae420 con 0x55da434f0b40 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01add20 con 0x55da434ef5d0 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01aaf50 con 0x55da434fa370 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.640+0000 7f4ce76f3740 10 client.0 _ll_register_callbacks cb 0x7f4cc8004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59502/0 (socket says 172.21.15.86:59502) 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 -- 172.21.15.86:0/2489934029 learned_addr learned my addr 172.21.15.86:0/2489934029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da43502420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cdf7fe640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 -- 172.21.15.86:0/2489934029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 msgr2=0x55da434ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 -- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 msgr2=0x55da43502420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da43502420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 -- 172.21.15.86:0/2489934029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da434ff930 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cd67fc640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802a940 tx=0x7f4cc804ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72618995799b0349 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8035040 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9693+0+0 (secure 0 0 0) 0x7f4cc8004480 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd009f840 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4cc804e630 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4ce76f3740 10 client.18915 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.641+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55da43574f50 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cd009f840 con 0x55da434f0b40 2024-09-05T06:56:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f4cc8004950 con 0x55da434f0b40 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 10 client.18915 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 10 client.18915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 20 client.18915 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43575710 0x55da43577b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 4 client.18915 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da4345cd70 con 0x55da43575710 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43575710 0x55da43577b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4cc67fc640 10 client.18915 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.642+0000 7f4cdeffd640 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43575710 0x55da43577b30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797b6211c0db540f server_cookie=ca8c093e3538c62c in_seq=0 out_seq=0 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4cc67fc640 1 -- 172.21.15.86:0/2489934029 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da4345cd70 con 0x55da43575710 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4cc67fc640 10 client.18915 handle_command_reply: tid=0 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 client.18915 shutdown 2024-09-05T06:56:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 msgr2=0x55da43502960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802a940 tx=0x7f4cc804ecb0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 shutdown_connections 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55da43575710 0x55da43577b30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da434fa370 0x55da434ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4cc67fc640 0 client.18915 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da434f0b40 0x55da43502960 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 --2- 172.21.15.86:0/2489934029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da434ef5d0 0x55da43502420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 >> 172.21.15.86:0/2489934029 conn(0x55da434b8310 msgr2=0x55da434ea4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 shutdown_connections 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 1 -- 172.21.15.86:0/2489934029 wait complete. 2024-09-05T06:56:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce76f3740 20 client.18915 trim_cache size 0 max 16384 2024-09-05T06:56:11.654 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 msgr2=0x7f4ccc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 0x7f4ccc0a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80077f0 tx=0x7f4cd8007780 comp rx=0 tx=0).stop 2024-09-05T06:56:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 msgr2=0x7f4ce01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000d950 tx=0x7f4cd000de10 comp rx=0 tx=0).stop 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 shutdown_connections 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4ccc0a2cf0 0x7f4ccc0a51b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ce0106a80 0x7f4ce019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ce0101420 0x7f4ce0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 --2- 172.21.15.86:0/1602325503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ce0100a80 0x7f4ce01982e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.643+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 >> 172.21.15.86:0/1602325503 conn(0x7f4ce00fc7a0 msgr2=0x7f4ce006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.644+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 shutdown_connections 2024-09-05T06:56:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:11.644+0000 7f4ce531a640 1 -- 172.21.15.86:0/1602325503 wait complete. 2024-09-05T06:56:11.748 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.79 MB/sec warmup 89 sec latency 13223.689 ms 2024-09-05T06:56:12.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.69 MB/sec warmup 90 sec latency 14223.789 ms 2024-09-05T06:56:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:13 smithi096 ceph-mon[34468]: pgmap v468: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 233 KiB/s wr, 17 op/s 2024-09-05T06:56:13.307 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:13 smithi096 ceph-mon[34468]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T06:56:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:13 smithi115 ceph-mon[34791]: pgmap v468: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 233 KiB/s wr, 17 op/s 2024-09-05T06:56:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:13 smithi115 ceph-mon[34791]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T06:56:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:13 smithi086 ceph-mon[30943]: pgmap v468: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 233 KiB/s wr, 17 op/s 2024-09-05T06:56:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:13 smithi086 ceph-mon[30943]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T06:56:13.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:56:13.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:56:13.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.60 MB/sec warmup 91 sec latency 15223.876 ms 2024-09-05T06:56:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 -- 172.21.15.86:0/1263533595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76841027a0 msgr2=0x7f76841067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 --2- 172.21.15.86:0/1263533595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76841027a0 0x7f76841067f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402a410 comp rx=0 tx=0).stop 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 -- 172.21.15.86:0/1263533595 shutdown_connections 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 --2- 172.21.15.86:0/1263533595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 0x7f768410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 --2- 172.21.15.86:0/1263533595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76841027a0 0x7f76841067f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 --2- 172.21.15.86:0/1263533595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7684101df0 0x7f76841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.259+0000 7f768b4cf640 1 -- 172.21.15.86:0/1263533595 >> 172.21.15.86:0/1263533595 conn(0x7f76840fd8f0 msgr2=0x7f76840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- 172.21.15.86:0/1263533595 shutdown_connections 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- 172.21.15.86:0/1263533595 wait complete. 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 Processor -- start 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- start start 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76841027a0 0x7f7684198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 0x7f768419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7684111e70 con 0x7f7684101df0 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7684111cf0 con 0x7f7684106f20 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.260+0000 7f768b4cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7684111ff0 con 0x7f76841027a0 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47736/0 (socket says 172.21.15.86:47736) 2024-09-05T06:56:14.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 -- 172.21.15.86:0/701959475 learned_addr learned my addr 172.21.15.86:0/701959475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f7688a6a640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76841027a0 0x7f7684198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f7689a6c640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 0x7f768419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 -- 172.21.15.86:0/701959475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76841027a0 msgr2=0x7f7684198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76841027a0 0x7f7684198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 -- 172.21.15.86:0/701959475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 msgr2=0x7f768419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 0x7f768419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 -- 172.21.15.86:0/701959475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768419d3a0 con 0x7f7684101df0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768926b640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f768000eb90 tx=0x7f768000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f767a7fc640 1 -- 172.21.15.86:0/701959475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680004d20 con 0x7f7684101df0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f767a7fc640 1 -- 172.21.15.86:0/701959475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f768000ee90 con 0x7f7684101df0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768419d690 con 0x7f7684101df0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f767a7fc640 1 -- 172.21.15.86:0/701959475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680010670 con 0x7f7684101df0 2024-09-05T06:56:14.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.261+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a4f70 con 0x7f7684101df0 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f767a7fc640 1 -- 172.21.15.86:0/701959475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7680005020 con 0x7f7684101df0 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 Processor -- start 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 -- start start 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 0x55882afee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 0x55882aff1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55882af49280 con 0x55882afeed40 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55882aef0b10 con 0x55882afe3b60 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.264+0000 7f768d8a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55882af45240 con 0x55882afe4480 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 0x55882aff1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f767a7fc640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 0x7f76700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 0x55882aff1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47750/0 (socket says 172.21.15.86:47750) 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7688a6a640 1 -- 172.21.15.86:0/358899934 learned_addr learned my addr 172.21.15.86:0/358899934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 0x55882afee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7689a6c640 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7689a6c640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 0x7f76700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f767a7fc640 1 -- 172.21.15.86:0/701959475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f76800cfee0 con 0x7f7684101df0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 -- 172.21.15.86:0/358899934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 msgr2=0x55882afe3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 -- 172.21.15.86:0/358899934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 msgr2=0x55882aff1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 0x55882aff1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 -- 172.21.15.86:0/358899934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55882aefd1f0 con 0x55882afe4480 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f7689a6c640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 0x7f76700a5230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f767c0176b0 tx=0x7f767c009520 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f768926b640 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 0x55882afee800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76800cd190 tx=0x7f768000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79476b349a829f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.265+0000 7f766f7fe640 1 -- 172.21.15.86:0/358899934 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800a0310 con 0x55882afe4480 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f766f7fe640 1 -- 172.21.15.86:0/358899934 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76800a4bd0 con 0x55882afe4480 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f766f7fe640 1 -- 172.21.15.86:0/358899934 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800cd410 con 0x55882afe4480 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 -- 172.21.15.86:0/358899934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 msgr2=0x55882afee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 0x55882afee800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76800cd190 tx=0x7f768000cf90 comp rx=0 tx=0).stop 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 -- 172.21.15.86:0/358899934 shutdown_connections 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afeed40 0x55882aff1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe4480 0x55882afee800 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 --2- 172.21.15.86:0/358899934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 -- 172.21.15.86:0/358899934 >> 172.21.15.86:0/358899934 conn(0x55882afe00f0 msgr2=0x55882afe04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 -- 172.21.15.86:0/358899934 shutdown_connections 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.266+0000 7f768d8a8740 1 -- 172.21.15.86:0/358899934 wait complete. 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 -- start start 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 Processor -- start 2024-09-05T06:56:14.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe4480 0x55882afec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882afec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55882aff1910 con 0x55882afe3b60 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55882af03f10 con 0x55882afe4480 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55882aff1a90 con 0x55882afeed40 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768d8a8740 10 client.0 _ll_register_callbacks cb 0x55882aecca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768926b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768926b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47754/0 (socket says 172.21.15.86:47754) 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.267+0000 7f768926b640 1 -- 172.21.15.86:0/4196585952 learned_addr learned my addr 172.21.15.86:0/4196585952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f7689a6c640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882afec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f7688a6a640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe4480 0x55882afec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 -- 172.21.15.86:0/4196585952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 msgr2=0x55882afec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882afec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 -- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe4480 msgr2=0x55882afec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe4480 0x55882afec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 -- 172.21.15.86:0/4196585952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55882acba6a0 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f7689a6c640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882afec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768926b640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f768009bc60 tx=0x7f7680098260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bfe8c7059a7df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800a0310 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f7680098640 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76800a4bd0 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76800a4740 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768d8a8740 10 client.18927 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55882affd590 con 0x55882afe3b60 2024-09-05T06:56:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76800a4bd0 con 0x55882afe3b60 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.268+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f7680097cb0 con 0x55882afe3b60 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 10 client.18927 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 20 client.18927 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882affe8e0 0x55882b000cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 4 client.18927 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55882af567f0 con 0x55882affe8e0 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f7688a6a640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882affe8e0 0x55882b000cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f7688a6a640 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882affe8e0 0x55882b000cd0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5a7d84966469b1 server_cookie=dedd8a7049e50b1d in_seq=0 out_seq=0 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.269+0000 7f766cff9640 10 client.18927 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f766cff9640 1 -- 172.21.15.86:0/4196585952 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55882af567f0 con 0x55882affe8e0 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f766cff9640 10 client.18927 handle_command_reply: tid=0 2024-09-05T06:56:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 client.18927 shutdown 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 msgr2=0x55882afebc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f768009bc60 tx=0x7f7680098260 comp rx=0 tx=0).stop 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 shutdown_connections 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882affe8e0 0x55882b000cd0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882afec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe4480 0x55882afec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 --2- 172.21.15.86:0/4196585952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe3b60 0x55882afebc90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f766cff9640 0 client.18927 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 >> 172.21.15.86:0/4196585952 conn(0x55882afe00f0 msgr2=0x55882afe5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 shutdown_connections 2024-09-05T06:56:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 1 -- 172.21.15.86:0/4196585952 wait complete. 2024-09-05T06:56:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.270+0000 7f768d8a8740 20 client.18927 trim_cache size 0 max 16384 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.287+0000 7f768d8a8740 1 Processor -- start 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 -- start start 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882b0067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76841af210 con 0x55882afe4480 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76841b0e70 con 0x55882afe3b60 2024-09-05T06:56:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768d8a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76841af8e0 con 0x55882afeed40 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47768/0 (socket says 172.21.15.86:47768) 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 -- 172.21.15.86:0/243297613 learned_addr learned my addr 172.21.15.86:0/243297613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768926b640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 -- 172.21.15.86:0/243297613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 msgr2=0x55882b0067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7689a6c640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882b0067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882b0067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 -- 172.21.15.86:0/243297613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 msgr2=0x55882afe65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 -- 172.21.15.86:0/243297613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55882afdfb90 con 0x55882afe4480 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f768926b640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f7688a6a640 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7674005c00 tx=0x7f7674000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53d6e8e335755bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.288+0000 7f766effd640 1 -- 172.21.15.86:0/243297613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674035040 con 0x55882afe4480 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f766effd640 1 -- 172.21.15.86:0/243297613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7674049070 con 0x55882afe4480 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f766effd640 1 -- 172.21.15.86:0/243297613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674004de0 con 0x55882afe4480 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 -- 172.21.15.86:0/243297613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 msgr2=0x55882b0062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7674005c00 tx=0x7f7674000bc0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 -- 172.21.15.86:0/243297613 shutdown_connections 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afeed40 0x55882b0067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882b0062a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 --2- 172.21.15.86:0/243297613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afe3b60 0x55882afe65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 -- 172.21.15.86:0/243297613 >> 172.21.15.86:0/243297613 conn(0x55882afe00f0 msgr2=0x55882afedc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 -- 172.21.15.86:0/243297613 shutdown_connections 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.289+0000 7f768d8a8740 1 -- 172.21.15.86:0/243297613 wait complete. 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 -- start start 2024-09-05T06:56:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 Processor -- start 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 0x55882aff2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afeed40 0x55882aff3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76841b06e0 con 0x55882afe4480 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76841affe0 con 0x55882afeed40 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ad210 con 0x55882afe3b60 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f768d8a8740 10 client.0 _ll_register_callbacks cb 0x7f7674009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f7688a6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47770/0 (socket says 172.21.15.86:47770) 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f7688a6a640 1 -- 172.21.15.86:0/3755813318 learned_addr learned my addr 172.21.15.86:0/3755813318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.290+0000 7f7689a6c640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afeed40 0x55882aff3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f768926b640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 0x55882aff2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 -- 172.21.15.86:0/3755813318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 msgr2=0x55882aff2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 0x55882aff2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 -- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afeed40 msgr2=0x55882aff3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afeed40 0x55882aff3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 -- 172.21.15.86:0/3755813318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55882b068270 con 0x55882afe4480 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f768926b640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 0x55882aff2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f7688a6a640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7674009640 tx=0x7f7674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc0ff8b4890ac22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f766cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674035040 con 0x55882afe4480 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f7674032260 con 0x55882afe4480 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7674049070 con 0x55882afe4480 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f767405b050 con 0x55882afe4480 2024-09-05T06:56:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f768d8a8740 10 client.18933 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55882b069520 con 0x55882afe4480 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.291+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7674049070 con 0x55882afe4480 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f7674031b20 con 0x55882afe4480 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 10 client.18933 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 10 client.18933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 20 client.18933 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882b069d70 0x55882b06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 4 client.18933 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55882af50d00 con 0x55882b069d70 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768926b640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882b069d70 0x55882b06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f768926b640 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882b069d70 0x55882b06c190 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65ad44f30e8d5b9 server_cookie=b7b41006be4300a6 in_seq=0 out_seq=0 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.292+0000 7f766cff9640 10 client.18933 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.293+0000 7f766cff9640 1 -- 172.21.15.86:0/3755813318 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55882af50d00 con 0x55882b069d70 2024-09-05T06:56:14.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.293+0000 7f766cff9640 10 client.18933 handle_command_reply: tid=0 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.293+0000 7f768d8a8740 1 client.18933 shutdown 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.293+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 msgr2=0x55882aff2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.293+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7674009640 tx=0x7f7674004750 comp rx=0 tx=0).stop 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 shutdown_connections 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55882b069d70 0x55882b06c190 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882afeed40 0x55882aff3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55882afe4480 0x55882aff2ec0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f766cff9640 0 client.18933 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 --2- 172.21.15.86:0/3755813318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55882afe3b60 0x55882aff2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 >> 172.21.15.86:0/3755813318 conn(0x55882afe00f0 msgr2=0x55882afed170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 shutdown_connections 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 1 -- 172.21.15.86:0/3755813318 wait complete. 2024-09-05T06:56:14.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768d8a8740 20 client.18933 trim_cache size 0 max 16384 2024-09-05T06:56:14.308 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:56:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:56:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "47121f98-ffdb-4953-9109-b0f7a04a2eac", 2024-09-05T06:56:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:56:14.308 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:56:14.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 msgr2=0x7f76700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 0x7f76700a5230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f767c0176b0 tx=0x7f767c009520 comp rx=0 tx=0).stop 2024-09-05T06:56:14.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 msgr2=0x7f76841983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:14.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f768000eb90 tx=0x7f768000c6a0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 shutdown_connections 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f76700a2d70 0x7f76700a5230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7684106f20 0x7f768419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76841027a0 0x7f7684198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 --2- 172.21.15.86:0/701959475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684101df0 0x7f76841983f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.294+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 >> 172.21.15.86:0/701959475 conn(0x7f76840fd8f0 msgr2=0x7f76840feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.295+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 shutdown_connections 2024-09-05T06:56:14.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:14.295+0000 7f768b4cf640 1 -- 172.21.15.86:0/701959475 wait complete. 2024-09-05T06:56:14.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.50 MB/sec warmup 92 sec latency 16223.905 ms 2024-09-05T06:56:15.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: pgmap v470: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 291 KiB/s wr, 21 op/s 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:56:15.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.238 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:15 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.364 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: pgmap v470: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 291 KiB/s wr, 21 op/s 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:15 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: pgmap v470: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 291 KiB/s wr, 21 op/s 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:56:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:15 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:15.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.41 MB/sec warmup 93 sec latency 17223.929 ms 2024-09-05T06:56:15.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 -- 172.21.15.86:0/3776556079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4102ef0 msgr2=0x7fe5b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 --2- 172.21.15.86:0/3776556079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4102ef0 0x7fe5b410da40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009ca0 tx=0x7fe5ac02a950 comp rx=0 tx=0).stop 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 -- 172.21.15.86:0/3776556079 shutdown_connections 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 --2- 172.21.15.86:0/3776556079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4102ef0 0x7fe5b410da40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 --2- 172.21.15.86:0/3776556079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 0x7fe5b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 --2- 172.21.15.86:0/3776556079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4100e20 0x7fe5b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 -- 172.21.15.86:0/3776556079 >> 172.21.15.86:0/3776556079 conn(0x7fe5b40fc7d0 msgr2=0x7fe5b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 -- 172.21.15.86:0/3776556079 shutdown_connections 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.934+0000 7fe5b9320640 1 -- 172.21.15.86:0/3776556079 wait complete. 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 Processor -- start 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 -- start start 2024-09-05T06:56:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 0x7fe5b41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4102ef0 0x7fe5b41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4110ed0 con 0x7fe5b4100e20 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4110d50 con 0x7fe5b4102ef0 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b9320640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4111050 con 0x7fe5b4102570 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47784/0 (socket says 172.21.15.86:47784) 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/2468436554 learned_addr learned my addr 172.21.15.86:0/2468436554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4102ef0 0x7fe5b41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 0x7fe5b41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/2468436554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 msgr2=0x7fe5b41998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 0x7fe5b41998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/2468436554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4102ef0 msgr2=0x7fe5b41a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4102ef0 0x7fe5b41a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.935+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/2468436554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b41a4960 con 0x7fe5b4100e20 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b4a0 tx=0x7fe5a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe59bfff640 1 -- 172.21.15.86:0/2468436554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4014070 con 0x7fe5b4100e20 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b41a4b30 con 0x7fe5b4100e20 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe59bfff640 1 -- 172.21.15.86:0/2468436554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a4010ac0 con 0x7fe5b4100e20 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe59bfff640 1 -- 172.21.15.86:0/2468436554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4017c30 con 0x7fe5b4100e20 2024-09-05T06:56:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.936+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a5070 con 0x7fe5b4100e20 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.938+0000 7fe59bfff640 1 -- 172.21.15.86:0/2468436554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5a4017070 con 0x7fe5b4100e20 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 Processor -- start 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 -- start start 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 0x561b2e859140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e859680 0x561b2e85ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b2e7b42e0 con 0x561b2e84edc0 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b2e75bce0 con 0x561b2e84e4a0 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b2e7b02a0 con 0x561b2e859680 2024-09-05T06:56:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe59bfff640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 0x561b2e859140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:39900/0 (socket says 172.21.15.86:39900) 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 -- 172.21.15.86:0/1606771165 learned_addr learned my addr 172.21.15.86:0/1606771165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e859680 0x561b2e85ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe59bfff640 1 -- 172.21.15.86:0/2468436554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fe5a40cee40 con 0x7fe5b4100e20 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 -- 172.21.15.86:0/1606771165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e859680 msgr2=0x561b2e85ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e859680 0x561b2e85ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 -- 172.21.15.86:0/1606771165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 msgr2=0x561b2e859140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.939+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 0x561b2e859140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5b37fe640 1 -- 172.21.15.86:0/1606771165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2e740440 con 0x561b2e84e4a0 2024-09-05T06:56:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 0x561b2e859140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac030040 tx=0x7fe5ac043000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a60 tx=0x7fe5ac02f700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476780b39ac85e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5997fa640 1 -- 172.21.15.86:0/1606771165 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac034040 con 0x561b2e84e4a0 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5997fa640 1 -- 172.21.15.86:0/1606771165 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ac04e070 con 0x561b2e84e4a0 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.940+0000 7fe5997fa640 1 -- 172.21.15.86:0/1606771165 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac005050 con 0x561b2e84e4a0 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1606771165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 msgr2=0x561b2e84e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a60 tx=0x7fe5ac02f700 comp rx=0 tx=0).stop 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1606771165 shutdown_connections 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e859680 0x561b2e85ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84edc0 0x561b2e859140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1606771165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84e4a0 0x561b2e84e880 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1606771165 >> 172.21.15.86:0/1606771165 conn(0x561b2e84aa30 msgr2=0x561b2e84ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1606771165 shutdown_connections 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1606771165 wait complete. 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- start start 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 Processor -- start 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84edc0 0x561b2e856210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e859680 0x561b2e856750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b2e85c250 con 0x561b2e84e4a0 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b2e76f050 con 0x561b2e859680 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b2e85c3d0 con 0x561b2e84edc0 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5bb6f9740 10 client.0 _ll_register_callbacks cb 0x561b2e7af160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.941+0000 7fe5b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47810/0 (socket says 172.21.15.86:47810) 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/3246684656 learned_addr learned my addr 172.21.15.86:0/3246684656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e859680 0x561b2e856750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84edc0 0x561b2e856210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/3246684656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84edc0 msgr2=0x561b2e856210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84edc0 0x561b2e856210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e859680 msgr2=0x561b2e856750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e859680 0x561b2e856750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 -- 172.21.15.86:0/3246684656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2e5253c0 con 0x561b2e84e4a0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400be60 tx=0x7fe5a406ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6449cba5c08c3d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a401e050 con 0x561b2e84e4a0 2024-09-05T06:56:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fe5a40cc3d0 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ac04e070 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe5a409b730 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5bb6f9740 10 client.18945 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561b2e867c20 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.942+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5ac04e070 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe5a40976a0 con 0x561b2e84e4a0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 10 client.18945 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 10 client.18945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 20 client.18945 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e869160 0x561b2e86b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 4 client.18945 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2e7c1850 con 0x561b2e869160 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e869160 0x561b2e86b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e869160 0x561b2e86b600 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677368f2f612619a server_cookie=d514a4d6285492ea in_seq=0 out_seq=0 2024-09-05T06:56:15.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.943+0000 7fe596ff5640 10 client.18945 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe596ff5640 1 -- 172.21.15.86:0/3246684656 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b2e7c1850 con 0x561b2e869160 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe596ff5640 10 client.18945 handle_command_reply: tid=0 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 client.18945 shutdown 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 msgr2=0x561b2e858f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400be60 tx=0x7fe5a406ae90 comp rx=0 tx=0).stop 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 shutdown_connections 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e869160 0x561b2e86b600 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e859680 0x561b2e856750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84edc0 0x561b2e856210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3246684656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e858f60 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 >> 172.21.15.86:0/3246684656 conn(0x561b2e84aa30 msgr2=0x561b2e849b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe596ff5640 0 client.18945 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:15.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 shutdown_connections 2024-09-05T06:56:15.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.944+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3246684656 wait complete. 2024-09-05T06:56:15.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.945+0000 7fe5bb6f9740 20 client.18945 trim_cache size 0 max 16384 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 Processor -- start 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 -- start start 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e8654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e871410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41af1f0 con 0x561b2e84e4a0 2024-09-05T06:56:15.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b0e50 con 0x561b2e84edc0 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41af8c0 con 0x561b2e84aa30 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47816/0 (socket says 172.21.15.86:47816) 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 -- 172.21.15.86:0/3290787006 learned_addr learned my addr 172.21.15.86:0/3290787006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e8654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e871410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 -- 172.21.15.86:0/3290787006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 msgr2=0x561b2e8654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e8654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 -- 172.21.15.86:0/3290787006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 msgr2=0x561b2e871410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e871410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 -- 172.21.15.86:0/3290787006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2e84bba0 con 0x561b2e84e4a0 2024-09-05T06:56:15.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e8654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e871410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.962+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x561b2e84bdb0 tx=0x7fe5a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff2d7fb40b8e618 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe598ff9640 1 -- 172.21.15.86:0/3290787006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8016bb0 con 0x561b2e84e4a0 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe598ff9640 1 -- 172.21.15.86:0/3290787006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a8020030 con 0x561b2e84e4a0 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe598ff9640 1 -- 172.21.15.86:0/3290787006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a801ccd0 con 0x561b2e84e4a0 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3290787006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 msgr2=0x561b2e870ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x561b2e84bdb0 tx=0x7fe5a8008040 comp rx=0 tx=0).stop 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3290787006 shutdown_connections 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e871410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e870ed0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/3290787006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e8654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3290787006 >> 172.21.15.86:0/3290787006 conn(0x561b2e8568b0 msgr2=0x561b2e854ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3290787006 shutdown_connections 2024-09-05T06:56:15.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.963+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/3290787006 wait complete. 2024-09-05T06:56:15.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 -- start start 2024-09-05T06:56:15.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 Processor -- start 2024-09-05T06:56:15.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e861b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e85d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b06c0 con 0x561b2e84e4a0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41affc0 con 0x561b2e84edc0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41ad1f0 con 0x561b2e84aa30 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5bb6f9740 10 client.0 _ll_register_callbacks cb 0x7fe5a400b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47818/0 (socket says 172.21.15.86:47818) 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 -- 172.21.15.86:0/1309490249 learned_addr learned my addr 172.21.15.86:0/1309490249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e861b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b37fe640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e85d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 -- 172.21.15.86:0/1309490249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 msgr2=0x561b2e861b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e861b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 -- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 msgr2=0x561b2e85d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:15.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e85d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.964+0000 7fe5b27fc640 1 -- 172.21.15.86:0/1309490249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2e85da30 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e861b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5b27fc640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800ba80 tx=0x7fe5a801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd914e2d2704b14b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a801c400 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fe5a801c5a0 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a8020030 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe5a8033050 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 10 client.18951 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561b2e8d3cc0 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a8020030 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fe5a802e3b0 con 0x561b2e84e4a0 2024-09-05T06:56:15.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 10 client.18951 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 20 client.18951 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e859680 0x561b2e8d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 4 client.18951 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.965+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b2e7bbd60 con 0x561b2e859680 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e859680 0x561b2e8d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe5b2ffd640 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e859680 0x561b2e8d6500 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f841c30d69d61b30 server_cookie=96d120849c199a0b in_seq=0 out_seq=0 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe596ff5640 10 client.18951 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe596ff5640 1 -- 172.21.15.86:0/1309490249 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b2e7bbd60 con 0x561b2e859680 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe596ff5640 10 client.18951 handle_command_reply: tid=0 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe5bb6f9740 1 client.18951 shutdown 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 msgr2=0x561b2e85ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.966+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800ba80 tx=0x7fe5a801c210 comp rx=0 tx=0).stop 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 shutdown_connections 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b2e859680 0x561b2e8d6500 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b2e84edc0 0x561b2e85d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b2e84e4a0 0x561b2e85ceb0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 --2- 172.21.15.86:0/1309490249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b2e84aa30 0x561b2e861b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe596ff5640 0 client.18951 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 >> 172.21.15.86:0/1309490249 conn(0x561b2e8568b0 msgr2=0x561b2e849370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 shutdown_connections 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 1 -- 172.21.15.86:0/1309490249 wait complete. 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5bb6f9740 20 client.18951 trim_cache size 0 max 16384 2024-09-05T06:56:15.974 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "47121f98-ffdb-4953-9109-b0f7a04a2eac": { 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "47121f98-ffdb-4953-9109-b0f7a04a2eac", 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:15.975 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 msgr2=0x7fe59c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac030040 tx=0x7fe5ac043000 comp rx=0 tx=0).stop 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 msgr2=0x7fe5b41993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b4a0 tx=0x7fe5a400b960 comp rx=0 tx=0).stop 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 shutdown_connections 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b4102ef0 0x7fe5b41a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5b4102570 0x7fe5b41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 --2- 172.21.15.86:0/2468436554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b4100e20 0x7fe5b41993a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:15.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.967+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 >> 172.21.15.86:0/2468436554 conn(0x7fe5b40fc7d0 msgr2=0x7fe5b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:15.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.968+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 shutdown_connections 2024-09-05T06:56:15.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:15.968+0000 7fe5b9320640 1 -- 172.21.15.86:0/2468436554 wait complete. 2024-09-05T06:56:16.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:16 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:56:16.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:16 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:56:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:16 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:56:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:56:16.713 INFO:tasks.cephfs.filesystem:scrub status for tag:47121f98-ffdb-4953-9109-b0f7a04a2eac - {'path': '/', 'tag': '47121f98-ffdb-4953-9109-b0f7a04a2eac', 'options': 'recursive,force'} 2024-09-05T06:56:16.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.32 MB/sec warmup 94 sec latency 18223.953 ms 2024-09-05T06:56:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:17 smithi115 ceph-mon[34791]: pgmap v471: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 247 KiB/s wr, 18 op/s 2024-09-05T06:56:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:17 smithi086 ceph-mon[30943]: pgmap v471: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 247 KiB/s wr, 18 op/s 2024-09-05T06:56:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:17 smithi096 ceph-mon[34468]: pgmap v471: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 247 KiB/s wr, 18 op/s 2024-09-05T06:56:17.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:56:17.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.24 MB/sec warmup 95 sec latency 19224.005 ms 2024-09-05T06:56:17.885 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.367+0000 7fef9c94a640 1 -- 172.21.15.86:0/3874045746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 msgr2=0x7fef981067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.367+0000 7fef9c94a640 1 --2- 172.21.15.86:0/3874045746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef981067f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef8002a410 comp rx=0 tx=0).stop 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 -- 172.21.15.86:0/3874045746 shutdown_connections 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 --2- 172.21.15.86:0/3874045746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 0x7fef9810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 --2- 172.21.15.86:0/3874045746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef981067f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 --2- 172.21.15.86:0/3874045746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 0x7fef981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 -- 172.21.15.86:0/3874045746 >> 172.21.15.86:0/3874045746 conn(0x7fef980fd8f0 msgr2=0x7fef980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:18.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 -- 172.21.15.86:0/3874045746 shutdown_connections 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 -- 172.21.15.86:0/3874045746 wait complete. 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.368+0000 7fef9c94a640 1 Processor -- start 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 -- start start 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 0x7fef98198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 0x7fef9819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef98111e70 con 0x7fef981027a0 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef98111cf0 con 0x7fef98106f20 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9c94a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fef98111ff0 con 0x7fef98101df0 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef9659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 0x7fef98198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47842/0 (socket says 172.21.15.86:47842) 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef8dd9b640 1 -- 172.21.15.86:0/2275871561 learned_addr learned my addr 172.21.15.86:0/2275871561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef96d9d640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 0x7fef9819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef8dd9b640 1 -- 172.21.15.86:0/2275871561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 msgr2=0x7fef98198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.369+0000 7fef8dd9b640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 0x7fef98198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8dd9b640 1 -- 172.21.15.86:0/2275871561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 msgr2=0x7fef9819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8dd9b640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 0x7fef9819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8dd9b640 1 -- 172.21.15.86:0/2275871561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9819d400 con 0x7fef981027a0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8dd9b640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fef80005940 tx=0x7fef80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8003f030 con 0x7fef981027a0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9819d690 con 0x7fef981027a0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef80003aa0 con 0x7fef981027a0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80035070 con 0x7fef981027a0 2024-09-05T06:56:18.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.370+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981a4fd0 con 0x7fef981027a0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.371+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef981021d0 con 0x7fef981027a0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.372+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef80042030 con 0x7fef981027a0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.373+0000 7fef8ffff640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 0x7fef840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.373+0000 7fef9659c640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 0x7fef840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.373+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fef800eff90 con 0x7fef981027a0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.373+0000 7fef9659c640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 0x7fef840a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef8800b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.377+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef800bc120 con 0x7fef981027a0 2024-09-05T06:56:18.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.578+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef9810a160 con 0x7fef981027a0 2024-09-05T06:56:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.579+0000 7fef8ffff640 1 -- 172.21.15.86:0/2275871561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7fef800c0420 con 0x7fef981027a0 2024-09-05T06:56:18.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:56:18.581 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:56:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:56:18.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 msgr2=0x7fef840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 0x7fef840a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef8800b390 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 msgr2=0x7fef98198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fef80005940 tx=0x7fef80004270 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 shutdown_connections 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fef840a2cf0 0x7fef840a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef98106f20 0x7fef9819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef981027a0 0x7fef98198990 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 --2- 172.21.15.86:0/2275871561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef98101df0 0x7fef98198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 >> 172.21.15.86:0/2275871561 conn(0x7fef980fd8f0 msgr2=0x7fef980fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 shutdown_connections 2024-09-05T06:56:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:18.581+0000 7fef9c94a640 1 -- 172.21.15.86:0/2275871561 wait complete. 2024-09-05T06:56:18.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.15 MB/sec warmup 96 sec latency 20224.077 ms 2024-09-05T06:56:19.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:19 smithi086 ceph-mon[30943]: pgmap v472: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:19.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2275871561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:19.336 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 95100888 --release --await' 2024-09-05T06:56:19.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 95100888 --release --await 2024-09-05T06:56:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:19 smithi115 ceph-mon[34791]: pgmap v472: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2275871561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:19 smithi096 ceph-mon[34468]: pgmap v472: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2275871561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:19.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11178 8.07 MB/sec warmup 97 sec latency 21224.149 ms 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 -- 172.21.15.86:0/546829116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828106f50 msgr2=0x7fd82810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/546829116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828106f50 0x7fd82810eaf0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd820009cd0 tx=0x7fd82002a8e0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 -- 172.21.15.86:0/546829116 shutdown_connections 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/546829116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828106f50 0x7fd82810eaf0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/546829116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd828100ca0 0x7fd828106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/546829116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8281002f0 0x7fd8281006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 -- 172.21.15.86:0/546829116 >> 172.21.15.86:0/546829116 conn(0x7fd828077210 msgr2=0x7fd8280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 -- 172.21.15.86:0/546829116 shutdown_connections 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.105+0000 7fd82d4cd640 1 -- 172.21.15.86:0/546829116 wait complete. 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 Processor -- start 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 -- start start 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8281002f0 0x7fd8281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd828106f50 0x7fd82819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd828111eb0 con 0x7fd828100ca0 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd828111d30 con 0x7fd8281002f0 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd82d4cd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd828112030 con 0x7fd828106f50 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47858/0 (socket says 172.21.15.86:47858) 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 -- 172.21.15.86:0/1867440088 learned_addr learned my addr 172.21.15.86:0/1867440088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd826ffd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8281002f0 0x7fd8281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd8277fe640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd828106f50 0x7fd82819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 -- 172.21.15.86:0/1867440088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd828106f50 msgr2=0x7fd82819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd828106f50 0x7fd82819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 -- 172.21.15.86:0/1867440088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8281002f0 msgr2=0x7fd8281983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8281002f0 0x7fd8281983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.106+0000 7fd81ffff640 1 -- 172.21.15.86:0/1867440088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82819d380 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd81ffff640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c970 tx=0x7fd81000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd824ff9640 1 -- 172.21.15.86:0/1867440088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810014070 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd824ff9640 1 -- 172.21.15.86:0/1867440088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd810004440 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82819d670 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd824ff9640 1 -- 172.21.15.86:0/1867440088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810003d30 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.107+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281a4f50 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd824ff9640 1 -- 172.21.15.86:0/1867440088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd810017070 con 0x7fd828100ca0 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 Processor -- start 2024-09-05T06:56:20.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 -- start start 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a06a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a079650 0x55797a07ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557979fcf620 con 0x55797a06a860 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557979f76e50 con 0x55797a079650 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557979fcb5e0 con 0x55797a069f40 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd826ffd640 1 -- 172.21.15.86:0/458967664 learned_addr learned my addr 172.21.15.86:0/458967664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.110+0000 7fd824ff9640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 0x7fd8140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd81ffff640 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a079650 0x55797a07ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd8277fe640 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd8277fe640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 0x7fd8140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 -- 172.21.15.86:0/458967664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 msgr2=0x55797a06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a06a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 -- 172.21.15.86:0/458967664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a079650 msgr2=0x55797a07ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a079650 0x55797a07ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 -- 172.21.15.86:0/458967664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557979f67d40 con 0x55797a06a860 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd824ff9640 1 -- 172.21.15.86:0/1867440088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fd8100cf710 con 0x7fd828100ca0 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd826ffd640 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd818009a70 tx=0x7fd818009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c09f67a543b8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd8277fe640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 0x7fd8140a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd820033040 tx=0x7fd8200095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd81d7fa640 1 -- 172.21.15.86:0/458967664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818017070 con 0x55797a06a860 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd81d7fa640 1 -- 172.21.15.86:0/458967664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd818013470 con 0x55797a06a860 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.111+0000 7fd81d7fa640 1 -- 172.21.15.86:0/458967664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818012630 con 0x55797a06a860 2024-09-05T06:56:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- 172.21.15.86:0/458967664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 msgr2=0x55797a079110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd818009a70 tx=0x7fd818009f30 comp rx=0 tx=0).stop 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- 172.21.15.86:0/458967664 shutdown_connections 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a079650 0x55797a07ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a06a860 0x55797a079110 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/458967664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- 172.21.15.86:0/458967664 >> 172.21.15.86:0/458967664 conn(0x55797a0665f0 msgr2=0x55797a0669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- 172.21.15.86:0/458967664 shutdown_connections 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- 172.21.15.86:0/458967664 wait complete. 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 -- start start 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.112+0000 7fd82f8a6740 1 Processor -- start 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a076580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55797a07c220 con 0x55797a079650 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557979f57f10 con 0x55797a06a860 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55797a07c3a0 con 0x55797a069f40 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 10 client.0 _ll_register_callbacks cb 0x557979f7bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47882/0 (socket says 172.21.15.86:47882) 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 -- 172.21.15.86:0/3937140384 learned_addr learned my addr 172.21.15.86:0/3937140384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd826ffd640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a076580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd81ffff640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 -- 172.21.15.86:0/3937140384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 msgr2=0x55797a076580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a076580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 -- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 msgr2=0x55797a076ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a076ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 -- 172.21.15.86:0/3937140384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557979cde580 con 0x55797a079650 2024-09-05T06:56:20.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd8277fe640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd820003bf0 tx=0x7fd82002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5960422397acab09 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd80e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82004d070 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fd8200046c0 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.113+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd818013470 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd820043070 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 10 client.18969 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557979da8960 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd818013470 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd820030070 con 0x55797a079650 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 10 client.18969 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 10 client.18969 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 20 client.18969 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a08b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 4 client.18969 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557979fdcb90 con 0x55797a089320 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.114+0000 7fd826ffd640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a08b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.115+0000 7fd80e7fc640 10 client.18969 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.115+0000 7fd826ffd640 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a08b7a0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d30217e914a0d1 server_cookie=8c4d364f911ebcf0 in_seq=0 out_seq=0 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.115+0000 7fd80e7fc640 1 -- 172.21.15.86:0/3937140384 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557979fdcb90 con 0x55797a089320 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.115+0000 7fd80e7fc640 10 client.18969 handle_command_reply: tid=0 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.115+0000 7fd82f8a6740 1 client.18969 shutdown 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 msgr2=0x55797a077000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd820003bf0 tx=0x7fd82002a8e0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 shutdown_connections 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a08b7a0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd80e7fc640 0 client.18969 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a077000 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a076ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/3937140384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 >> 172.21.15.86:0/3937140384 conn(0x55797a0665f0 msgr2=0x55797a06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 shutdown_connections 2024-09-05T06:56:20.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 1 -- 172.21.15.86:0/3937140384 wait complete. 2024-09-05T06:56:20.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.116+0000 7fd82f8a6740 20 client.18969 trim_cache size 0 max 16384 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.133+0000 7fd82f8a6740 1 Processor -- start 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.133+0000 7fd82f8a6740 1 -- start start 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.133+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 0x55797a078450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.133+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a06a860 0x55797a0910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.133+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281acff0 con 0x55797a079650 2024-09-05T06:56:20.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281aec50 con 0x55797a069f40 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281ad6c0 con 0x55797a06a860 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47884/0 (socket says 172.21.15.86:47884) 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 -- 172.21.15.86:0/2769634762 learned_addr learned my addr 172.21.15.86:0/2769634762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd826ffd640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 0x55797a078450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd81ffff640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a06a860 0x55797a0910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 -- 172.21.15.86:0/2769634762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a06a860 msgr2=0x55797a0910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a06a860 0x55797a0910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 -- 172.21.15.86:0/2769634762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 msgr2=0x55797a078450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 0x55797a078450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 -- 172.21.15.86:0/2769634762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797a067710 con 0x55797a079650 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd826ffd640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 0x55797a078450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd8277fe640 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd820031c60 tx=0x7fd820003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa740ae3c73251 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd81cff9640 1 -- 172.21.15.86:0/2769634762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820064070 con 0x55797a079650 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd81cff9640 1 -- 172.21.15.86:0/2769634762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd820036040 con 0x55797a079650 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.134+0000 7fd81cff9640 1 -- 172.21.15.86:0/2769634762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82004d070 con 0x55797a079650 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 -- 172.21.15.86:0/2769634762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 msgr2=0x55797a0915f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd820031c60 tx=0x7fd820003ef0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 -- 172.21.15.86:0/2769634762 shutdown_connections 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0915f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a06a860 0x55797a0910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/2769634762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a069f40 0x55797a078450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 -- 172.21.15.86:0/2769634762 >> 172.21.15.86:0/2769634762 conn(0x55797a0665f0 msgr2=0x55797a065750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 -- 172.21.15.86:0/2769634762 shutdown_connections 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.135+0000 7fd82f8a6740 1 -- 172.21.15.86:0/2769634762 wait complete. 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 -- start start 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 Processor -- start 2024-09-05T06:56:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a07dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a07e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281ae4c0 con 0x55797a079650 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281addc0 con 0x55797a06a860 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281aaff0 con 0x55797a069f40 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd82f8a6740 10 client.0 _ll_register_callbacks cb 0x7fd820031cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47896/0 (socket says 172.21.15.86:47896) 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 -- 172.21.15.86:0/937931106 learned_addr learned my addr 172.21.15.86:0/937931106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd826ffd640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a07dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd81ffff640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a07e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 -- 172.21.15.86:0/937931106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 msgr2=0x55797a07dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a07dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 -- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 msgr2=0x55797a07e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a07e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd8277fe640 1 -- 172.21.15.86:0/937931106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797a0ed770 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd81ffff640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a07e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.136+0000 7fd826ffd640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a07dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd8277fe640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd820052500 tx=0x7fd820004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc79441f092692b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820064020 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fd820052b10 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd820036040 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd82006f050 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 10 client.18975 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55797a0f4b40 con 0x55797a079650 2024-09-05T06:56:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd820036040 con 0x55797a079650 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fd820030040 con 0x55797a079650 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 10 client.18975 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 10 client.18975 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 20 client.18975 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a0f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 4 client.18975 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "95100888", "release": true, "await": true}] 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.137+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "95100888", "release": true, "await": true}) -- 0x557979fd70a0 con 0x55797a089320 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.138+0000 7fd826ffd640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a0f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.138+0000 7fd826ffd640 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a0f72c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9607289a87db3198 server_cookie=adad6eb0f3c95181 in_seq=0 out_seq=0 2024-09-05T06:56:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.138+0000 7fd80e7fc640 10 client.18975 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.170+0000 7fd80e7fc640 1 -- 172.21.15.86:0/937931106 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557979fd70a0 con 0x55797a089320 2024-09-05T06:56:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.170+0000 7fd80e7fc640 10 client.18975 handle_command_reply: tid=0 2024-09-05T06:56:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.170+0000 7fd82f8a6740 1 client.18975 shutdown 2024-09-05T06:56:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.170+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 msgr2=0x55797a0ed130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.170+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd820052500 tx=0x7fd820004640 comp rx=0 tx=0).stop 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 shutdown_connections 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55797a089320 0x55797a0f72c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797a079650 0x55797a0ed130 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55797a06a860 0x55797a07e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd80e7fc640 0 client.18975 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 --2- 172.21.15.86:0/937931106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55797a069f40 0x55797a07dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 >> 172.21.15.86:0/937931106 conn(0x55797a0665f0 msgr2=0x55797a06ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 shutdown_connections 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 1 -- 172.21.15.86:0/937931106 wait complete. 2024-09-05T06:56:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.171+0000 7fd82f8a6740 20 client.18975 trim_cache size 0 max 16384 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 38, 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 56, 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "95100888": { 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 56, 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:56:20.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 143.8, 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:20.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:20.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:20.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:20.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:56:20.176 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 msgr2=0x7fd8140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 0x7fd8140a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd820033040 tx=0x7fd8200095c0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 msgr2=0x7fd828198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c970 tx=0x7fd81000cd70 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 shutdown_connections 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd8140a2cf0 0x7fd8140a51b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd828106f50 0x7fd82819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd828100ca0 0x7fd828198910 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 --2- 172.21.15.86:0/1867440088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8281002f0 0x7fd8281983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 >> 172.21.15.86:0/1867440088 conn(0x7fd828077210 msgr2=0x7fd828105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 shutdown_connections 2024-09-05T06:56:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:20.172+0000 7fd82d4cd640 1 -- 172.21.15.86:0/1867440088 wait complete. 2024-09-05T06:56:20.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11277 8.01 MB/sec warmup 98 sec latency 21640.946 ms 2024-09-05T06:56:20.998 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '95100888', total seconds elapsed: 27.2 2024-09-05T06:56:21.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:21 smithi086 ceph-mon[30943]: pgmap v473: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:21.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:21 smithi115 ceph-mon[34791]: pgmap v473: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:21.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:21 smithi096 ceph-mon[34468]: pgmap v473: 129 pgs: 129 active+clean; 596 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T06:56:21.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11464 8.03 MB/sec warmup 99 sec latency 59.907 ms 2024-09-05T06:56:22.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11629 8.02 MB/sec warmup 100 sec latency 55.152 ms 2024-09-05T06:56:23.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:23 smithi115 ceph-mon[34791]: pgmap v474: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 697 KiB/s wr, 54 op/s 2024-09-05T06:56:23.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:23 smithi086 ceph-mon[30943]: pgmap v474: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 697 KiB/s wr, 54 op/s 2024-09-05T06:56:23.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:23 smithi096 ceph-mon[34468]: pgmap v474: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 697 KiB/s wr, 54 op/s 2024-09-05T06:56:23.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11812 8.02 MB/sec warmup 101 sec latency 63.907 ms 2024-09-05T06:56:24.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11987 8.01 MB/sec warmup 102 sec latency 63.255 ms 2024-09-05T06:56:25.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:25 smithi115 ceph-mon[34791]: pgmap v475: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 606 KiB/s wr, 47 op/s 2024-09-05T06:56:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:25 smithi086 ceph-mon[30943]: pgmap v475: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 606 KiB/s wr, 47 op/s 2024-09-05T06:56:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:25 smithi096 ceph-mon[34468]: pgmap v475: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 606 KiB/s wr, 47 op/s 2024-09-05T06:56:25.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12165 7.98 MB/sec warmup 103 sec latency 58.854 ms 2024-09-05T06:56:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:56:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:56:26.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:26 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:26 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:56:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:56:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:56:26.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:26 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:56:26.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12366 7.98 MB/sec warmup 104 sec latency 69.862 ms 2024-09-05T06:56:27.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:27 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:56:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:27 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:56:27.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:27 smithi115 ceph-mon[34791]: pgmap v476: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 581 KiB/s wr, 45 op/s 2024-09-05T06:56:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:27 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:56:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:27 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:56:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:27 smithi086 ceph-mon[30943]: pgmap v476: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 581 KiB/s wr, 45 op/s 2024-09-05T06:56:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:27 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:56:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:27 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:56:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:27 smithi096 ceph-mon[34468]: pgmap v476: 129 pgs: 129 active+clean; 602 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 581 KiB/s wr, 45 op/s 2024-09-05T06:56:27.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12571 8.02 MB/sec warmup 105 sec latency 65.266 ms 2024-09-05T06:56:28.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12752 8.02 MB/sec warmup 106 sec latency 57.411 ms 2024-09-05T06:56:29.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:29 smithi115 ceph-mon[34791]: pgmap v477: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-05T06:56:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:29 smithi086 ceph-mon[30943]: pgmap v477: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-05T06:56:29.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:29 smithi096 ceph-mon[34468]: pgmap v477: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-05T06:56:29.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12948 8.08 MB/sec warmup 107 sec latency 61.197 ms 2024-09-05T06:56:30.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13146 8.11 MB/sec warmup 108 sec latency 62.940 ms 2024-09-05T06:56:31.358 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:31 smithi096 ceph-mon[34468]: pgmap v478: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-05T06:56:31.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:31 smithi115 ceph-mon[34791]: pgmap v478: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-05T06:56:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:31 smithi086 ceph-mon[30943]: pgmap v478: 129 pgs: 129 active+clean; 619 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-09-05T06:56:31.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13387 8.17 MB/sec warmup 109 sec latency 66.389 ms 2024-09-05T06:56:32.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13608 8.21 MB/sec warmup 110 sec latency 60.109 ms 2024-09-05T06:56:33.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:33 smithi115 ceph-mon[34791]: pgmap v479: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-09-05T06:56:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:33 smithi086 ceph-mon[30943]: pgmap v479: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-09-05T06:56:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:33 smithi096 ceph-mon[34468]: pgmap v479: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-09-05T06:56:33.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13728 8.17 MB/sec warmup 111 sec latency 65.261 ms 2024-09-05T06:56:34.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13909 8.24 MB/sec warmup 112 sec latency 62.146 ms 2024-09-05T06:56:35.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:35 smithi115 ceph-mon[34791]: pgmap v480: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:35 smithi086 ceph-mon[30943]: pgmap v480: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:35.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:35 smithi096 ceph-mon[34468]: pgmap v480: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:35.749 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14122 8.31 MB/sec warmup 113 sec latency 66.733 ms 2024-09-05T06:56:36.750 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14306 8.32 MB/sec warmup 114 sec latency 87.847 ms 2024-09-05T06:56:37.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:37 smithi115 ceph-mon[34791]: pgmap v481: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:37.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:37 smithi086 ceph-mon[30943]: pgmap v481: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:37 smithi096 ceph-mon[34468]: pgmap v481: 129 pgs: 129 active+clean; 634 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 239 op/s 2024-09-05T06:56:37.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14463 8.34 MB/sec warmup 115 sec latency 421.812 ms 2024-09-05T06:56:38.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14629 8.35 MB/sec warmup 116 sec latency 388.621 ms 2024-09-05T06:56:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:39 smithi115 ceph-mon[34791]: pgmap v482: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 349 op/s 2024-09-05T06:56:39.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:39 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:56:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:39 smithi086 ceph-mon[30943]: pgmap v482: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 349 op/s 2024-09-05T06:56:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:39 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:56:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:39 smithi096 ceph-mon[34468]: pgmap v482: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 349 op/s 2024-09-05T06:56:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:39 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:56:39.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14773 8.32 MB/sec warmup 117 sec latency 137.435 ms 2024-09-05T06:56:40.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14944 8.31 MB/sec warmup 118 sec latency 90.826 ms 2024-09-05T06:56:41.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:41 smithi115 ceph-mon[34791]: pgmap v483: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 227 op/s 2024-09-05T06:56:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:41 smithi086 ceph-mon[30943]: pgmap v483: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 227 op/s 2024-09-05T06:56:41.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:41 smithi096 ceph-mon[34468]: pgmap v483: 129 pgs: 129 active+clean; 652 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 227 op/s 2024-09-05T06:56:41.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15105 8.31 MB/sec warmup 119 sec latency 273.236 ms 2024-09-05T06:56:43.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:43 smithi115 ceph-mon[34791]: pgmap v484: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 344 op/s 2024-09-05T06:56:43.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:43 smithi086 ceph-mon[30943]: pgmap v484: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 344 op/s 2024-09-05T06:56:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:43 smithi096 ceph-mon[34468]: pgmap v484: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.2 MiB/s wr, 344 op/s 2024-09-05T06:56:43.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15448 10.73 MB/sec execute 1 sec latency 189.691 ms 2024-09-05T06:56:44.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15617 8.08 MB/sec execute 2 sec latency 592.923 ms 2024-09-05T06:56:45.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:45 smithi115 ceph-mon[34791]: pgmap v485: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:45.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:45 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:56:45.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:45 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:56:45.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:45 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:56:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:45 smithi086 ceph-mon[30943]: pgmap v485: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:45 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:56:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:45 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:56:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:45 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:56:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:45 smithi096 ceph-mon[34468]: pgmap v485: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:45 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:56:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:45 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:56:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:45 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:56:45.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15802 7.82 MB/sec execute 3 sec latency 134.684 ms 2024-09-05T06:56:46.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:56:46.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15985 7.63 MB/sec execute 4 sec latency 76.669 ms 2024-09-05T06:56:47.083 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:47.105 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:47 smithi086 ceph-mon[30943]: pgmap v486: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:47.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:47 smithi115 ceph-mon[34791]: pgmap v486: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:47.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:47 smithi096 ceph-mon[34468]: pgmap v486: 129 pgs: 129 active+clean; 656 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 227 op/s 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.667+0000 7fc939add640 1 -- 172.21.15.86:0/2954949699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102ef0 msgr2=0x7fc93410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.667+0000 7fc939add640 1 --2- 172.21.15.86:0/2954949699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102ef0 0x7fc93410da40 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009ca0 tx=0x7fc92c02a950 comp rx=0 tx=0).stop 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 -- 172.21.15.86:0/2954949699 shutdown_connections 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 --2- 172.21.15.86:0/2954949699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102ef0 0x7fc93410da40 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 --2- 172.21.15.86:0/2954949699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102570 0x7fc9341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 --2- 172.21.15.86:0/2954949699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 -- 172.21.15.86:0/2954949699 >> 172.21.15.86:0/2954949699 conn(0x7fc9340fc7d0 msgr2=0x7fc9340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 -- 172.21.15.86:0/2954949699 shutdown_connections 2024-09-05T06:56:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.668+0000 7fc939add640 1 -- 172.21.15.86:0/2954949699 wait complete. 2024-09-05T06:56:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 Processor -- start 2024-09-05T06:56:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 -- start start 2024-09-05T06:56:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102ef0 0x7fc93419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410fd30 con 0x7fc934102570 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410fbb0 con 0x7fc934102ef0 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.669+0000 7fc939add640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410feb0 con 0x7fc934100e20 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55414/0 (socket says 172.21.15.86:55414) 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc9337fe640 1 -- 172.21.15.86:0/201378563 learned_addr learned my addr 172.21.15.86:0/201378563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42392/0 (socket says 172.21.15.86:42392) 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc933fff640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102ef0 0x7fc93419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 -- 172.21.15.86:0/201378563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 msgr2=0x7fc934196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 -- 172.21.15.86:0/201378563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102ef0 msgr2=0x7fc93419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102ef0 0x7fc93419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc932ffd640 1 -- 172.21.15.86:0/201378563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93419b1e0 con 0x7fc934102570 2024-09-05T06:56:47.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.670+0000 7fc9337fe640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:47.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc932ffd640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e9f0 tx=0x7fc92800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc930ff9640 1 -- 172.21.15.86:0/201378563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92800cc40 con 0x7fc934102570 2024-09-05T06:56:47.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93419b4d0 con 0x7fc934102570 2024-09-05T06:56:47.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc930ff9640 1 -- 172.21.15.86:0/201378563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92800cde0 con 0x7fc934102570 2024-09-05T06:56:47.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a2db0 con 0x7fc934102570 2024-09-05T06:56:47.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.671+0000 7fc930ff9640 1 -- 172.21.15.86:0/201378563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9280187a0 con 0x7fc934102570 2024-09-05T06:56:47.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.674+0000 7fc930ff9640 1 -- 172.21.15.86:0/201378563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc928010430 con 0x7fc934102570 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.674+0000 7fc93beb6740 1 Processor -- start 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 -- start start 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e63880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e70a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a78dc92e0 con 0x563a78e6e680 2024-09-05T06:56:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a78d70ce0 con 0x563a78e63dc0 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc93beb6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a78dc52a0 con 0x563a78e634a0 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:41122/0 (socket says 172.21.15.86:41122) 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 -- 172.21.15.86:0/1791694569 learned_addr learned my addr 172.21.15.86:0/1791694569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc932ffd640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e70a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc930ff9640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 0x7fc91c0a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc933fff640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e63880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 -- 172.21.15.86:0/1791694569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 msgr2=0x563a78e63880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e63880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 -- 172.21.15.86:0/1791694569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 msgr2=0x563a78e70a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e70a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc9337fe640 1 -- 172.21.15.86:0/1791694569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a78d55440 con 0x563a78e63dc0 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc933fff640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e63880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.675+0000 7fc932ffd640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e70a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc930ff9640 1 -- 172.21.15.86:0/201378563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fc928018540 con 0x7fc934102570 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc933fff640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 0x7fc91c0a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc9337fe640 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc924011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77caff370148ae05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc919ffb640 1 -- 172.21.15.86:0/1791694569 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92401e070 con 0x563a78e63dc0 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc919ffb640 1 -- 172.21.15.86:0/1791694569 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92400fba0 con 0x563a78e63dc0 2024-09-05T06:56:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.676+0000 7fc919ffb640 1 -- 172.21.15.86:0/1791694569 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924019cc0 con 0x563a78e63dc0 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 -- 172.21.15.86:0/1791694569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 msgr2=0x563a78e6e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fd0 tx=0x7fc924011ae0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc933fff640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 0x7fc91c0a9800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005e30 tx=0x7fc92c03c040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 -- 172.21.15.86:0/1791694569 shutdown_connections 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e70a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e6e140 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 --2- 172.21.15.86:0/1791694569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e63880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 -- 172.21.15.86:0/1791694569 >> 172.21.15.86:0/1791694569 conn(0x563a78e5fa30 msgr2=0x563a78e5fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 -- 172.21.15.86:0/1791694569 shutdown_connections 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.677+0000 7fc93beb6740 1 -- 172.21.15.86:0/1791694569 wait complete. 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 -- start start 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 Processor -- start 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e6df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e6e680 0x563a78e6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a78e71250 con 0x563a78e63dc0 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a78d84050 con 0x563a78e6e680 2024-09-05T06:56:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a78e713d0 con 0x563a78e634a0 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc93beb6740 10 client.0 _ll_register_callbacks cb 0x563a78dc4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e6df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42412/0 (socket says 172.21.15.86:42412) 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc932ffd640 1 -- 172.21.15.86:0/3188679999 learned_addr learned my addr 172.21.15.86:0/3188679999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.678+0000 7fc933fff640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e6e680 0x563a78e6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 -- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e6e680 msgr2=0x563a78e6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e6e680 0x563a78e6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 -- 172.21.15.86:0/3188679999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 msgr2=0x563a78e6b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 -- 172.21.15.86:0/3188679999 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a78b3a3c0 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc9337fe640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e6df40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc92401c040 tx=0x7fc92400ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53822e02c12e3220 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.679+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924027060 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92400fba0 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fc9240199c0 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc92402e050 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 10 client.27913 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563a78e7cb70 con 0x563a78e634a0 2024-09-05T06:56:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92400fba0 con 0x563a78e634a0 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc924021030 con 0x563a78e634a0 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 10 client.27913 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 10 client.27913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 20 client.27913 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78e7df80 0x563a78e80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 4 client.27913 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.680+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a78dd6850 con 0x563a78e7df80 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.681+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78e7df80 0x563a78e80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.681+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78e7df80 0x563a78e80370 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73321df371d2014b server_cookie=ed727a7937c717ac in_seq=0 out_seq=0 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.681+0000 7fc912ffd640 10 client.27913 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc912ffd640 1 -- 172.21.15.86:0/3188679999 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a78dd6850 con 0x563a78e7df80 2024-09-05T06:56:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc912ffd640 10 client.27913 handle_command_reply: tid=0 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 client.27913 shutdown 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 msgr2=0x563a78e6df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e6df40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc92401c040 tx=0x7fc92400ff60 comp rx=0 tx=0).stop 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 shutdown_connections 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78e7df80 0x563a78e80370 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e6e680 0x563a78e6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e63dc0 0x563a78e6b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3188679999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e6df40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc912ffd640 0 client.27913 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.682+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 >> 172.21.15.86:0/3188679999 conn(0x563a78e5fa30 msgr2=0x563a78e65710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.683+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 shutdown_connections 2024-09-05T06:56:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.683+0000 7fc93beb6740 1 -- 172.21.15.86:0/3188679999 wait complete. 2024-09-05T06:56:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.683+0000 7fc93beb6740 20 client.27913 trim_cache size 0 max 16384 2024-09-05T06:56:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.710+0000 7fc93beb6740 1 Processor -- start 2024-09-05T06:56:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.710+0000 7fc93beb6740 1 -- start start 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.710+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e7a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e66900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e6d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc93beb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ad050 con 0x563a78e6e680 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc93beb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341aecb0 con 0x563a78e63dc0 2024-09-05T06:56:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc93beb6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ad720 con 0x563a78e634a0 2024-09-05T06:56:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e7a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e7a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55426/0 (socket says 172.21.15.86:55426) 2024-09-05T06:56:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc9337fe640 1 -- 172.21.15.86:0/4280222756 learned_addr learned my addr 172.21.15.86:0/4280222756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc932ffd640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e66900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.711+0000 7fc933fff640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e6d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc932ffd640 1 -- 172.21.15.86:0/4280222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 msgr2=0x563a78e7a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc932ffd640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e7a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc932ffd640 1 -- 172.21.15.86:0/4280222756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 msgr2=0x563a78e6d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc932ffd640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e6d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc932ffd640 1 -- 172.21.15.86:0/4280222756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a78e71130 con 0x563a78e63dc0 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.712+0000 7fc933fff640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e6d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.713+0000 7fc932ffd640 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e66900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9280cd1a0 tx=0x7fc928086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a805f7b5a2130d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.713+0000 7fc9197fa640 1 -- 172.21.15.86:0/4280222756 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9280a45e0 con 0x563a78e63dc0 2024-09-05T06:56:47.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.713+0000 7fc9197fa640 1 -- 172.21.15.86:0/4280222756 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc928099d20 con 0x563a78e63dc0 2024-09-05T06:56:47.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.713+0000 7fc9197fa640 1 -- 172.21.15.86:0/4280222756 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92809ae70 con 0x563a78e63dc0 2024-09-05T06:56:47.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 -- 172.21.15.86:0/4280222756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 msgr2=0x563a78e66900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e66900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9280cd1a0 tx=0x7fc928086040 comp rx=0 tx=0).stop 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 -- 172.21.15.86:0/4280222756 shutdown_connections 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e6d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e66900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 --2- 172.21.15.86:0/4280222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e7a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 -- 172.21.15.86:0/4280222756 >> 172.21.15.86:0/4280222756 conn(0x563a78e5fa30 msgr2=0x563a78e6ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 -- 172.21.15.86:0/4280222756 shutdown_connections 2024-09-05T06:56:47.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.714+0000 7fc93beb6740 1 -- 172.21.15.86:0/4280222756 wait complete. 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 -- start start 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 Processor -- start 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ae520 con 0x563a78e6e680 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ade20 con 0x563a78e63dc0 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341ab050 con 0x563a78e634a0 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.715+0000 7fc93beb6740 10 client.0 _ll_register_callbacks cb 0x7fc92401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:41162/0 (socket says 172.21.15.86:41162) 2024-09-05T06:56:47.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 -- 172.21.15.86:0/3488629148 learned_addr learned my addr 172.21.15.86:0/3488629148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:47.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc933fff640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 -- 172.21.15.86:0/3488629148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 msgr2=0x563a78e767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 -- 172.21.15.86:0/3488629148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 msgr2=0x563a78e71be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e71be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 -- 172.21.15.86:0/3488629148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a78e723e0 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc933fff640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc932ffd640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c6a0 tx=0x7fc92809fde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a36e433aaccaf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9280d5070 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fc9280cce20 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.716+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc928099d20 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc928098090 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 10 client.27929 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563a78ef0e50 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc928010720 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fc9280ce040 con 0x563a78e63dc0 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 10 client.27929 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 10 client.27929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:56:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 20 client.27929 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78ef1590 0x563a78ef39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 4 client.27929 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc928099d20 con 0x563a78ef1590 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.717+0000 7fc9337fe640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78ef1590 0x563a78ef39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc9337fe640 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78ef1590 0x563a78ef39b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a12de82d151c7ad server_cookie=a42ee964f700dbcc in_seq=0 out_seq=0 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc912ffd640 10 client.27929 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc912ffd640 1 -- 172.21.15.86:0/3488629148 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc928099d20 con 0x563a78ef1590 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc912ffd640 10 client.27929 handle_command_reply: tid=0 2024-09-05T06:56:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 client.27929 shutdown 2024-09-05T06:56:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 msgr2=0x563a78e76d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c6a0 tx=0x7fc92809fde0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 shutdown_connections 2024-09-05T06:56:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563a78ef1590 0x563a78ef39b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a78e6e680 0x563a78e71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a78e63dc0 0x563a78e76d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 --2- 172.21.15.86:0/3488629148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a78e634a0 0x563a78e767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 >> 172.21.15.86:0/3488629148 conn(0x563a78e5fa30 msgr2=0x563a78e69990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.718+0000 7fc912ffd640 0 client.27929 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 shutdown_connections 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc93beb6740 1 -- 172.21.15.86:0/3488629148 wait complete. 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc93beb6740 20 client.27929 trim_cache size 0 max 16384 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:56:47.722 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 msgr2=0x7fc91c0a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 0x7fc91c0a9800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005e30 tx=0x7fc92c03c040 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 msgr2=0x7fc934196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.719+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e9f0 tx=0x7fc92800eeb0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 shutdown_connections 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc91c0a7340 0x7fc91c0a9800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc934102ef0 0x7fc93419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934102570 0x7fc934196770 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 --2- 172.21.15.86:0/201378563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc934100e20 0x7fc934196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 >> 172.21.15.86:0/201378563 conn(0x7fc9340fc7d0 msgr2=0x7fc9340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:47.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 shutdown_connections 2024-09-05T06:56:47.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:47.720+0000 7fc939add640 1 -- 172.21.15.86:0/201378563 wait complete. 2024-09-05T06:56:47.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16158 7.70 MB/sec execute 5 sec latency 104.099 ms 2024-09-05T06:56:48.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16340 8.55 MB/sec execute 6 sec latency 66.997 ms 2024-09-05T06:56:48.968 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:56:48.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:56:49.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:49 smithi086 ceph-mon[30943]: pgmap v487: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 350 op/s 2024-09-05T06:56:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:49.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:49 smithi115 ceph-mon[34791]: pgmap v487: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 350 op/s 2024-09-05T06:56:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:49 smithi096 ceph-mon[34468]: pgmap v487: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 350 op/s 2024-09-05T06:56:49.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16502 9.01 MB/sec execute 7 sec latency 96.659 ms 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 -- 172.21.15.86:0/3729206078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 msgr2=0x7fbb40100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3729206078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb40100ed0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009930 tx=0x7fbb3402a420 comp rx=0 tx=0).stop 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 -- 172.21.15.86:0/3729206078 shutdown_connections 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3729206078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40101410 0x7fbb4010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3729206078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb40100ed0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3729206078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40106a70 0x7fbb40106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 -- 172.21.15.86:0/3729206078 >> 172.21.15.86:0/3729206078 conn(0x7fbb400fc790 msgr2=0x7fbb400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 -- 172.21.15.86:0/3729206078 shutdown_connections 2024-09-05T06:56:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.941+0000 7fbb461e6640 1 -- 172.21.15.86:0/3729206078 wait complete. 2024-09-05T06:56:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 Processor -- start 2024-09-05T06:56:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 -- start start 2024-09-05T06:56:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40101410 0x7fbb4010c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40106a70 0x7fbb4010c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40110d20 con 0x7fbb40100a70 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40110ba0 con 0x7fbb40101410 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb461e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40110ea0 con 0x7fbb40106a70 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.942+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42428/0 (socket says 172.21.15.86:42428) 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 -- 172.21.15.86:0/3131007806 learned_addr learned my addr 172.21.15.86:0/3131007806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3effd640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40101410 0x7fbb4010c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3ffff640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40106a70 0x7fbb4010c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 -- 172.21.15.86:0/3131007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40106a70 msgr2=0x7fbb4010c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40106a70 0x7fbb4010c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 -- 172.21.15.86:0/3131007806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40101410 msgr2=0x7fbb4010c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40101410 0x7fbb4010c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 -- 172.21.15.86:0/3131007806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb401aacc0 con 0x7fbb40100a70 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3f7fe640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000e9f0 tx=0x7fbb3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000cd80 con 0x7fbb40100a70 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb300108e0 con 0x7fbb40100a70 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30018920 con 0x7fbb40100a70 2024-09-05T06:56:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.943+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb401aafb0 con 0x7fbb40100a70 2024-09-05T06:56:49.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.944+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401ab460 con 0x7fbb40100a70 2024-09-05T06:56:49.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.945+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb40106e50 con 0x7fbb40100a70 2024-09-05T06:56:49.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.946+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbb30010a90 con 0x7fbb40100a70 2024-09-05T06:56:49.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.947+0000 7fbb3cff9640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 0x7fbb280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.947+0000 7fbb3effd640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 0x7fbb280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.947+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fbb300cfa60 con 0x7fbb40100a70 2024-09-05T06:56:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.948+0000 7fbb3effd640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 0x7fbb280a5230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009900 tx=0x7fbb34000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:49.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:49.953+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb3009bbf0 con 0x7fbb40100a70 2024-09-05T06:56:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.296+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb401ab790 con 0x7fbb40100a70 2024-09-05T06:56:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.297+0000 7fbb3cff9640 1 -- 172.21.15.86:0/3131007806 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7fbb3009fef0 con 0x7fbb40100a70 2024-09-05T06:56:50.298 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:56:50.298 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:56:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 msgr2=0x7fbb280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 0x7fbb280a5230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009900 tx=0x7fbb34000bc0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 msgr2=0x7fbb4010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000e9f0 tx=0x7fbb3000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 shutdown_connections 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbb280a2d70 0x7fbb280a5230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb40106a70 0x7fbb4010c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40101410 0x7fbb4010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 --2- 172.21.15.86:0/3131007806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb40100a70 0x7fbb4010bac0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.300+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 >> 172.21.15.86:0/3131007806 conn(0x7fbb400fc790 msgr2=0x7fbb401126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.301+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 shutdown_connections 2024-09-05T06:56:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:50.301+0000 7fbb461e6640 1 -- 172.21.15.86:0/3131007806 wait complete. 2024-09-05T06:56:50.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16706 9.74 MB/sec execute 8 sec latency 71.546 ms 2024-09-05T06:56:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:51 smithi086 ceph-mon[30943]: pgmap v488: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 240 op/s 2024-09-05T06:56:51.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:51 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3131007806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:51.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:51 smithi115 ceph-mon[34791]: pgmap v488: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 240 op/s 2024-09-05T06:56:51.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:51 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3131007806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:51 smithi096 ceph-mon[34468]: pgmap v488: 129 pgs: 129 active+clean; 655 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 240 op/s 2024-09-05T06:56:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:51 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3131007806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:56:51.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16909 9.92 MB/sec execute 9 sec latency 129.127 ms 2024-09-05T06:56:51.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:56:52.158 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:52.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1487562368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 msgr2=0x7fb0c4100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1487562368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c4100560 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a410 comp rx=0 tx=0).stop 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1487562368 shutdown_connections 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1487562368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4100aa0 0x7fb0c410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1487562368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c4100560 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1487562368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4106b80 0x7fb0c4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.715+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1487562368 >> 172.21.15.86:0/1487562368 conn(0x7fb0c4077230 msgr2=0x7fb0c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.716+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1487562368 shutdown_connections 2024-09-05T06:56:52.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.716+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1487562368 wait complete. 2024-09-05T06:56:52.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 Processor -- start 2024-09-05T06:56:52.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 -- start start 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4106b80 0x7fb0c4101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4111f50 con 0x7fb0c4100100 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4111dd0 con 0x7fb0c4106b80 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c98f5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41120d0 con 0x7fb0c4100aa0 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.717+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36830/0 (socket says 172.21.15.86:36830) 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c41012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55710/0 (socket says 172.21.15.86:55710) 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1148855212 learned_addr learned my addr 172.21.15.86:0/1148855212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4106b80 0x7fb0c4101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1148855212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4106b80 msgr2=0x7fb0c4101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4106b80 0x7fb0c4101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 msgr2=0x7fb0c41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c41012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1148855212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41a6960 con 0x7fb0c4100aa0 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005ab0 tx=0x7fb0b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.718+0000 7fb0c88f3640 1 -- 172.21.15.86:0/1148855212 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803f070 con 0x7fb0c4100aa0 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.719+0000 7fb0c88f3640 1 -- 172.21.15.86:0/1148855212 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b8031b60 con 0x7fb0c4100aa0 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.719+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41a6bf0 con 0x7fb0c4100aa0 2024-09-05T06:56:52.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.719+0000 7fb0c88f3640 1 -- 172.21.15.86:0/1148855212 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8035030 con 0x7fb0c4100aa0 2024-09-05T06:56:52.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.719+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c41a7120 con 0x7fb0c4100aa0 2024-09-05T06:56:52.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.722+0000 7fb0c88f3640 1 -- 172.21.15.86:0/1148855212 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb0b8042070 con 0x7fb0c4100aa0 2024-09-05T06:56:52.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.723+0000 7fb0cbcce740 1 Processor -- start 2024-09-05T06:56:52.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.723+0000 7fb0cbcce740 1 -- start start 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.723+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 0x5616b37dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.723+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.723+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b37432f0 con 0x5616b37deb40 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616b36eacf0 con 0x5616b37dd5d0 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616b373f2b0 con 0x5616b37e8370 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46374/0 (socket says 172.21.15.86:46374) 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/1477936629 learned_addr learned my addr 172.21.15.86:0/1477936629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 0x5616b37dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.724+0000 7fb0c88f3640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 0x7fb0ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1477936629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 msgr2=0x5616b37dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1477936629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 msgr2=0x5616b37dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 0x5616b37dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c27fc640 1 -- 172.21.15.86:0/1477936629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b36cf3a0 con 0x5616b37e8370 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c88f3640 1 -- 172.21.15.86:0/1148855212 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb0b80efeb0 con 0x7fb0c4100aa0 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.725+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 0x5616b37dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.726+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37ea760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80ed1c0 tx=0x7fb0b8086c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f2c1f8ff70ccca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.726+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 0x7fb0ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.726+0000 7fb0a9ffb640 1 -- 172.21.15.86:0/1477936629 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80bbb80 con 0x5616b37e8370 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.726+0000 7fb0a9ffb640 1 -- 172.21.15.86:0/1477936629 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b80f3e20 con 0x5616b37e8370 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.726+0000 7fb0a9ffb640 1 -- 172.21.15.86:0/1477936629 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b808f800 con 0x5616b37e8370 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- 172.21.15.86:0/1477936629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 msgr2=0x5616b37ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37ea760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80ed1c0 tx=0x7fb0b8086c70 comp rx=0 tx=0).stop 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 0x7fb0ac0a5230 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0c41077c0 tx=0x7fb0bc004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- 172.21.15.86:0/1477936629 shutdown_connections 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37ea760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37dd5d0 0x5616b37dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/1477936629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37deb40 0x5616b37dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- 172.21.15.86:0/1477936629 >> 172.21.15.86:0/1477936629 conn(0x5616b37a6310 msgr2=0x5616b37d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- 172.21.15.86:0/1477936629 shutdown_connections 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- 172.21.15.86:0/1477936629 wait complete. 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.727+0000 7fb0cbcce740 1 -- start start 2024-09-05T06:56:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 Processor -- start 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37e8370 0x5616b37e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b37f3640 con 0x5616b37dd5d0 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616b36fe060 con 0x5616b37e8370 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616b37f37c0 con 0x5616b37deb40 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0cbcce740 10 client.0 _ll_register_callbacks cb 0x5616b34518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0c27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0c27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36844/0 (socket says 172.21.15.86:36844) 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0c27fc640 1 -- 172.21.15.86:0/490003470 learned_addr learned my addr 172.21.15.86:0/490003470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.728+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37e8370 0x5616b37e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 -- 172.21.15.86:0/490003470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37e8370 msgr2=0x5616b37e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37e8370 0x5616b37e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 -- 172.21.15.86:0/490003470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 msgr2=0x5616b37e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 -- 172.21.15.86:0/490003470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b34b43c0 con 0x5616b37deb40 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80b84a0 tx=0x7fb0b80bbf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb16afb6107cafe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b808a030 con 0x5616b37deb40 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb0b80ba340 con 0x5616b37deb40 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b80f3e20 con 0x5616b37deb40 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb0b8049070 con 0x5616b37deb40 2024-09-05T06:56:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0cbcce740 10 client.27937 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.729+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616b37f3640 con 0x5616b37deb40 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b80f3e20 con 0x5616b37deb40 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb0b80ed440 con 0x5616b37deb40 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 10 client.27937 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 10 client.27937 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 20 client.27937 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b3800240 0x5616b3802630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 4 client.27937 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b3750860 con 0x5616b3800240 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.730+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b3800240 0x5616b3802630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0a2ffd640 10 client.27937 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b3800240 0x5616b3802630 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=941540602e181f30 server_cookie=1d49eb5c4acebdef in_seq=0 out_seq=0 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/490003470 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616b3750860 con 0x5616b3800240 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0a2ffd640 10 client.27937 handle_command_reply: tid=0 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0cbcce740 1 client.27937 shutdown 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 msgr2=0x5616b37e5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.731+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80b84a0 tx=0x7fb0b80bbf10 comp rx=0 tx=0).stop 2024-09-05T06:56:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 shutdown_connections 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b3800240 0x5616b3802630 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37e8370 0x5616b37e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37deb40 0x5616b37e5000 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/490003470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 >> 172.21.15.86:0/490003470 conn(0x5616b37a6310 msgr2=0x5616b37df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0a2ffd640 0 client.27937 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:56:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 shutdown_connections 2024-09-05T06:56:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 1 -- 172.21.15.86:0/490003470 wait complete. 2024-09-05T06:56:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.732+0000 7fb0cbcce740 20 client.27937 trim_cache size 0 max 16384 2024-09-05T06:56:52.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17077 9.62 MB/sec execute 10 sec latency 59.927 ms 2024-09-05T06:56:52.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.754+0000 7fb0cbcce740 1 Processor -- start 2024-09-05T06:56:52.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 -- start start 2024-09-05T06:56:52.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b3807f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41acf90 con 0x5616b37dd5d0 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41aebf0 con 0x5616b37deb40 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41ad660 con 0x5616b37e8370 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55726/0 (socket says 172.21.15.86:55726) 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/2775412959 learned_addr learned my addr 172.21.15.86:0/2775412959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:52.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c37fe640 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b3807f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c27fc640 1 -- 172.21.15.86:0/2775412959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 msgr2=0x5616b3807f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.755+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b3807f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0c27fc640 1 -- 172.21.15.86:0/2775412959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 msgr2=0x5616b37e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0c27fc640 1 -- 172.21.15.86:0/2775412959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b37e0120 con 0x5616b37deb40 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37fc810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80bbf80 tx=0x7fb0b80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d82cee1b80e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0a97fa640 1 -- 172.21.15.86:0/2775412959 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b808a070 con 0x5616b37deb40 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0a97fa640 1 -- 172.21.15.86:0/2775412959 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b80ba0e0 con 0x5616b37deb40 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.756+0000 7fb0a97fa640 1 -- 172.21.15.86:0/2775412959 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b808ae60 con 0x5616b37deb40 2024-09-05T06:56:52.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 -- 172.21.15.86:0/2775412959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 msgr2=0x5616b37fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37fc810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80bbf80 tx=0x7fb0b80045e0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 -- 172.21.15.86:0/2775412959 shutdown_connections 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b3807f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37fc810 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/2775412959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.757+0000 7fb0cbcce740 1 -- 172.21.15.86:0/2775412959 >> 172.21.15.86:0/2775412959 conn(0x5616b37a6310 msgr2=0x5616b37e48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.758+0000 7fb0cbcce740 1 -- 172.21.15.86:0/2775412959 shutdown_connections 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.758+0000 7fb0cbcce740 1 -- 172.21.15.86:0/2775412959 wait complete. 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.758+0000 7fb0cbcce740 1 -- start start 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.758+0000 7fb0cbcce740 1 Processor -- start 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41ae460 con 0x5616b37dd5d0 2024-09-05T06:56:52.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0c2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55738/0 (socket says 172.21.15.86:55738) 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/3812611797 learned_addr learned my addr 172.21.15.86:0/3812611797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41add60 con 0x5616b37deb40 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41aaf90 con 0x5616b37e8370 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0cbcce740 10 client.0 _ll_register_callbacks cb 0x7fb0bc009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.759+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 msgr2=0x5616b37e1970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37e1970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/3812611797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 msgr2=0x5616b37f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b37e21a0 con 0x5616b37dd5d0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0c2ffd640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009c20 tx=0x7fb0b4004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b095e3b5c26da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b402b060 con 0x5616b37dd5d0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b80ba0e0 con 0x5616b37dd5d0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb0b4003be0 con 0x5616b37dd5d0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb0b4032050 con 0x5616b37dd5d0 2024-09-05T06:56:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0cbcce740 10 client.19017 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.760+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616b386aa90 con 0x5616b37dd5d0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b80ba0e0 con 0x5616b37dd5d0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb0b4025030 con 0x5616b37dd5d0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 10 client.19017 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 10 client.19017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 20 client.19017 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b386b1c0 0x5616b386d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 4 client.19017 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b374ad70 con 0x5616b386b1c0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.761+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b386b1c0 0x5616b386d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.762+0000 7fb0a2ffd640 10 client.19017 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.762+0000 7fb0c27fc640 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b386b1c0 0x5616b386d5e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fa62aa38ddc5ca7 server_cookie=b3677590cd13f421 in_seq=0 out_seq=0 2024-09-05T06:56:52.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.762+0000 7fb0a2ffd640 1 -- 172.21.15.86:0/3812611797 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b374ad70 con 0x5616b386b1c0 2024-09-05T06:56:52.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.762+0000 7fb0a2ffd640 10 client.19017 handle_command_reply: tid=0 2024-09-05T06:56:52.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.762+0000 7fb0cbcce740 1 client.19017 shutdown 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 msgr2=0x5616b37f8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009c20 tx=0x7fb0b4004100 comp rx=0 tx=0).stop 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 shutdown_connections 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5616b386b1c0 0x5616b386d5e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616b37e8370 0x5616b37e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616b37deb40 0x5616b37f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 --2- 172.21.15.86:0/3812611797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b37dd5d0 0x5616b37f8c60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 >> 172.21.15.86:0/3812611797 conn(0x5616b37a6310 msgr2=0x5616b37e7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0a2ffd640 0 client.19017 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:56:52.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 shutdown_connections 2024-09-05T06:56:52.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.763+0000 7fb0cbcce740 1 -- 172.21.15.86:0/3812611797 wait complete. 2024-09-05T06:56:52.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0cbcce740 20 client.19017 trim_cache size 0 max 16384 2024-09-05T06:56:52.769 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:52.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 msgr2=0x7fb0ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 0x7fb0ac0a5230 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0c41077c0 tx=0x7fb0bc004a40 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 msgr2=0x7fb0c4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005ab0 tx=0x7fb0b8003c10 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 shutdown_connections 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0ac0a2d70 0x7fb0ac0a5230 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0c4106b80 0x7fb0c4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0c4100aa0 0x7fb0c4101820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 --2- 172.21.15.86:0/1148855212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c4100100 0x7fb0c41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.764+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 >> 172.21.15.86:0/1148855212 conn(0x7fb0c4077230 msgr2=0x7fb0c40fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.765+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 shutdown_connections 2024-09-05T06:56:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:52.765+0000 7fb0c98f5640 1 -- 172.21.15.86:0/1148855212 wait complete. 2024-09-05T06:56:53.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:53 smithi115 ceph-mon[34791]: pgmap v489: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 355 op/s 2024-09-05T06:56:53.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:53 smithi086 ceph-mon[30943]: pgmap v489: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 355 op/s 2024-09-05T06:56:53.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:56:53.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:53 smithi096 ceph-mon[34468]: pgmap v489: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 355 op/s 2024-09-05T06:56:53.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17216 9.15 MB/sec execute 11 sec latency 77.782 ms 2024-09-05T06:56:53.895 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:54.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 -- 172.21.15.86:0/1957861239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0408104010 msgr2=0x7f04081043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 --2- 172.21.15.86:0/1957861239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0408104010 0x7f04081043f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009900 tx=0x7f03f802a410 comp rx=0 tx=0).stop 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 -- 172.21.15.86:0/1957861239 shutdown_connections 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 --2- 172.21.15.86:0/1957861239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 0x7f0408110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 --2- 172.21.15.86:0/1957861239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04081049c0 0x7f0408108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 --2- 172.21.15.86:0/1957861239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0408104010 0x7f04081043f0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 -- 172.21.15.86:0/1957861239 >> 172.21.15.86:0/1957861239 conn(0x7f04080ffa50 msgr2=0x7f0408101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 -- 172.21.15.86:0/1957861239 shutdown_connections 2024-09-05T06:56:54.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.449+0000 7f040df67640 1 -- 172.21.15.86:0/1957861239 wait complete. 2024-09-05T06:56:54.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 Processor -- start 2024-09-05T06:56:54.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 -- start start 2024-09-05T06:56:54.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 0x7f0408076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 0x7f0408077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0408114010 con 0x7f04081049c0 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0408113e90 con 0x7f0408109140 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.450+0000 7f040df67640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0408114190 con 0x7f0408104010 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36870/0 (socket says 172.21.15.86:36870) 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 -- 172.21.15.86:0/2187265136 learned_addr learned my addr 172.21.15.86:0/2187265136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f0406ffd640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 0x7f0408076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f0407fff640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 0x7f0408077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 -- 172.21.15.86:0/2187265136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 msgr2=0x7f0408077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 0x7f0408077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 -- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 msgr2=0x7f0408076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 0x7f0408076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 -- 172.21.15.86:0/2187265136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04081aac80 con 0x7f0408104010 2024-09-05T06:56:54.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f0406ffd640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 0x7f0408076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:54.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.451+0000 7f04077fe640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f8006010 tx=0x7f03f80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.452+0000 7f0404ff9640 1 -- 172.21.15.86:0/2187265136 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f803f030 con 0x7f0408104010 2024-09-05T06:56:54.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.452+0000 7f0404ff9640 1 -- 172.21.15.86:0/2187265136 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f80044c0 con 0x7f0408104010 2024-09-05T06:56:54.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.452+0000 7f0404ff9640 1 -- 172.21.15.86:0/2187265136 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8035070 con 0x7f0408104010 2024-09-05T06:56:54.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.452+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04081aaf10 con 0x7f0408104010 2024-09-05T06:56:54.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.452+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081ab2b0 con 0x7f0408104010 2024-09-05T06:56:54.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.455+0000 7f0404ff9640 1 -- 172.21.15.86:0/2187265136 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03f8042030 con 0x7f0408104010 2024-09-05T06:56:54.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.455+0000 7f0410340740 1 Processor -- start 2024-09-05T06:56:54.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.455+0000 7f0410340740 1 -- start start 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c24c892f0 con 0x556c24d235d0 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556c24c30cf0 con 0x556c24d24b40 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0410340740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c24c852b0 con 0x556c24d2e370 2024-09-05T06:56:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46406/0 (socket says 172.21.15.86:46406) 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 -- 172.21.15.86:0/1464335918 learned_addr learned my addr 172.21.15.86:0/1464335918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f04077fe640 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 -- 172.21.15.86:0/1464335918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 msgr2=0x556c24d30760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d30760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 -- 172.21.15.86:0/1464335918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 msgr2=0x556c24d23a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d23a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 -- 172.21.15.86:0/1464335918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c24c153a0 con 0x556c24d24b40 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0404ff9640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 0x7f03f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f04077fe640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 0x7f03f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.456+0000 7f0407fff640 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0400009f40 tx=0x7f0400010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec844d8091eaed63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.457+0000 7f03edffb640 1 -- 172.21.15.86:0/1464335918 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400003b70 con 0x556c24d24b40 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.457+0000 7f03edffb640 1 -- 172.21.15.86:0/1464335918 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0400003d10 con 0x556c24d24b40 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.457+0000 7f03edffb640 1 -- 172.21.15.86:0/1464335918 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000119c0 con 0x556c24d24b40 2024-09-05T06:56:54.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.457+0000 7f0404ff9640 1 -- 172.21.15.86:0/2187265136 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f03f80efc60 con 0x7f0408104010 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- 172.21.15.86:0/1464335918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 msgr2=0x556c24d23090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0400009f40 tx=0x7f0400010860 comp rx=0 tx=0).stop 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f04077fe640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 0x7f03f00a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ed020 tx=0x7f03f80f32c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- 172.21.15.86:0/1464335918 shutdown_connections 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d30760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d23a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 --2- 172.21.15.86:0/1464335918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d23090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- 172.21.15.86:0/1464335918 >> 172.21.15.86:0/1464335918 conn(0x556c24cec310 msgr2=0x556c24d1fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- 172.21.15.86:0/1464335918 shutdown_connections 2024-09-05T06:56:54.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- 172.21.15.86:0/1464335918 wait complete. 2024-09-05T06:56:54.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.460+0000 7f0410340740 1 -- start start 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.461+0000 7f0410340740 1 Processor -- start 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.461+0000 7f0410340740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d235d0 0x556c24d2dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.461+0000 7f0410340740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0410340740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 0x556c24d2b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0410340740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c24d31070 con 0x556c24d2e370 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46420/0 (socket says 172.21.15.86:46420) 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 -- 172.21.15.86:0/1215683051 learned_addr learned my addr 172.21.15.86:0/1215683051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0407fff640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d235d0 0x556c24d2dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556c24c44060 con 0x556c24d24b40 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c24d311f0 con 0x556c24d235d0 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0410340740 10 client.0 _ll_register_callbacks cb 0x556c249978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0407fff640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 0x556c24d2b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 -- 172.21.15.86:0/1215683051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d235d0 msgr2=0x556c24d2dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d235d0 0x556c24d2dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 -- 172.21.15.86:0/1215683051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 msgr2=0x556c24d2b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 0x556c24d2b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0406ffd640 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c249fa3c0 con 0x556c24d24b40 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.462+0000 7f0407fff640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 0x556c24d2b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0129e0 tx=0x7f03fc012ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9363c464aa4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f03e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:56:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004030 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0400003d10 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f03fc0041d0 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f03fc025050 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f0410340740 10 client.27965 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556c24d3cb50 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.463+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03fc0043d0 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f03fc003bf0 con 0x556c24d24b40 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 10 client.27965 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 10 client.27965 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 20 client.27965 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24d3e060 0x556c24d40480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 4 client.27965 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0400003d10 con 0x556c24d3e060 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0407fff640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24d3e060 0x556c24d40480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f0407fff640 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24d3e060 0x556c24d40480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e4e481751874b38 server_cookie=f60397f6f3d120e2 in_seq=0 out_seq=0 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.464+0000 7f03e6ffd640 10 client.27965 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.465+0000 7f03e6ffd640 1 -- 172.21.15.86:0/1215683051 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0400003d10 con 0x556c24d3e060 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.465+0000 7f03e6ffd640 10 client.27965 handle_command_reply: tid=0 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.465+0000 7f0410340740 1 client.27965 shutdown 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 msgr2=0x556c24d2b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0129e0 tx=0x7f03fc012ea0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 shutdown_connections 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24d3e060 0x556c24d40480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d2e370 0x556c24d2b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d2b020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 --2- 172.21.15.86:0/1215683051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d235d0 0x556c24d2dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 >> 172.21.15.86:0/1215683051 conn(0x556c24cec310 msgr2=0x556c24d25930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f03e6ffd640 0 client.27965 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:56:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 shutdown_connections 2024-09-05T06:56:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.466+0000 7f0410340740 1 -- 172.21.15.86:0/1215683051 wait complete. 2024-09-05T06:56:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.467+0000 7f0410340740 20 client.27965 trim_cache size 0 max 16384 2024-09-05T06:56:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.486+0000 7f0410340740 1 Processor -- start 2024-09-05T06:56:54.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.486+0000 7f0410340740 1 -- start start 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d1eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d1f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b57a0 con 0x556c24d235d0 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b7400 con 0x556c24d24b40 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0410340740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b5e70 con 0x556c24d2e370 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55788/0 (socket says 172.21.15.86:55788) 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 -- 172.21.15.86:0/1990041675 learned_addr learned my addr 172.21.15.86:0/1990041675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d1eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d1f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 -- 172.21.15.86:0/1990041675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 msgr2=0x556c24d1f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d1f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 -- 172.21.15.86:0/1990041675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 msgr2=0x556c24d1eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d1eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 -- 172.21.15.86:0/1990041675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c24d29fd0 con 0x556c24d235d0 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0406ffd640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d1eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.487+0000 7f0407fff640 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0400011570 tx=0x7f040001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddaed17289f035a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.488+0000 7f03ed7fa640 1 -- 172.21.15.86:0/1990041675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400004bc0 con 0x556c24d235d0 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.488+0000 7f03ed7fa640 1 -- 172.21.15.86:0/1990041675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0400030040 con 0x556c24d235d0 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.488+0000 7f03ed7fa640 1 -- 172.21.15.86:0/1990041675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040002b740 con 0x556c24d235d0 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 -- 172.21.15.86:0/1990041675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 msgr2=0x556c24d1ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0400011570 tx=0x7f040001e040 comp rx=0 tx=0).stop 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 -- 172.21.15.86:0/1990041675 shutdown_connections 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d2e370 0x556c24d1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d24b40 0x556c24d1eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 --2- 172.21.15.86:0/1990041675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d1ac60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 -- 172.21.15.86:0/1990041675 >> 172.21.15.86:0/1990041675 conn(0x556c24cec310 msgr2=0x556c24d1e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 -- 172.21.15.86:0/1990041675 shutdown_connections 2024-09-05T06:56:54.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.489+0000 7f0410340740 1 -- 172.21.15.86:0/1990041675 wait complete. 2024-09-05T06:56:54.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.490+0000 7f0410340740 1 -- start start 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.490+0000 7f0410340740 1 Processor -- start 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d2e370 0x556c24da7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b6c70 con 0x556c24d235d0 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b6570 con 0x556c24d2e370 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b37a0 con 0x556c24d24b40 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0410340740 10 client.0 _ll_register_callbacks cb 0x7f03fc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:54.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36906/0 (socket says 172.21.15.86:36906) 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 -- 172.21.15.86:0/2666942754 learned_addr learned my addr 172.21.15.86:0/2666942754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0407fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0407fff640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d2e370 0x556c24da7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 -- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d2e370 msgr2=0x556c24da7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d2e370 0x556c24da7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 -- 172.21.15.86:0/2666942754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 msgr2=0x556c24d32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 -- 172.21.15.86:0/2666942754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c24d33340 con 0x556c24d24b40 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0407fff640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f0406ffd640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03fc012fd0 tx=0x7f03fc011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c885490c2af664e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:54.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.491+0000 7f03e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:56:54.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc02d020 con 0x556c24d24b40 2024-09-05T06:56:54.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f03fc00fbc0 con 0x556c24d24b40 2024-09-05T06:56:54.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0400030040 con 0x556c24d24b40 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f03fc038050 con 0x556c24d24b40 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f0410340740 10 client.27958 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556c24da9320 con 0x556c24d24b40 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0400030040 con 0x556c24d24b40 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.492+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f03fc0035f0 con 0x556c24d24b40 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 10 client.27958 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 10 client.27958 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 20 client.27958 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:54.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24da9a70 0x556c24dabe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 4 client.27958 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c24c96860 con 0x556c24da9a70 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0407fff640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24da9a70 0x556c24dabe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f03e6ffd640 10 client.27958 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.493+0000 7f0407fff640 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24da9a70 0x556c24dabe90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d9660cac101a4bf server_cookie=828cb25c3bf2f40c in_seq=0 out_seq=0 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f03e6ffd640 1 -- 172.21.15.86:0/2666942754 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c24c96860 con 0x556c24da9a70 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f03e6ffd640 10 client.27958 handle_command_reply: tid=0 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 client.27958 shutdown 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 msgr2=0x556c24d32a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03fc012fd0 tx=0x7f03fc011d50 comp rx=0 tx=0).stop 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 shutdown_connections 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x556c24da9a70 0x556c24dabe90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f03e6ffd640 0 client.27958 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:56:54.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c24d2e370 0x556c24da7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c24d24b40 0x556c24d32a80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 --2- 172.21.15.86:0/2666942754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c24d235d0 0x556c24d32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.494+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 >> 172.21.15.86:0/2666942754 conn(0x556c24cec310 msgr2=0x556c24d2d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.496+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 shutdown_connections 2024-09-05T06:56:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.496+0000 7f0410340740 1 -- 172.21.15.86:0/2666942754 wait complete. 2024-09-05T06:56:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.496+0000 7f0410340740 20 client.27958 trim_cache size 0 max 16384 2024-09-05T06:56:54.498 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 msgr2=0x7f03f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 0x7f03f00a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ed020 tx=0x7f03f80f32c0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 msgr2=0x7f0408076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f8006010 tx=0x7f03f80040c0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 shutdown_connections 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03f00a2cf0 0x7f03f00a51b0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ed020 tx=0x7f03f80f32c0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0408109140 0x7f0408077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04081049c0 0x7f0408076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 --2- 172.21.15.86:0/2187265136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0408104010 0x7f0408076700 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f8006010 tx=0x7f03f80040c0 comp rx=0 tx=0).stop 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 >> 172.21.15.86:0/2187265136 conn(0x7f04080ffa50 msgr2=0x7f0408100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 shutdown_connections 2024-09-05T06:56:54.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:54.498+0000 7f040df67640 1 -- 172.21.15.86:0/2187265136 wait complete. 2024-09-05T06:56:54.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17377 9.24 MB/sec execute 12 sec latency 78.660 ms 2024-09-05T06:56:55.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:55 smithi086 ceph-mon[30943]: pgmap v490: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:55.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:56:55.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:55 smithi115 ceph-mon[34791]: pgmap v490: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:55.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:55 smithi096 ceph-mon[34468]: pgmap v490: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:55.754 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17562 9.60 MB/sec execute 13 sec latency 97.072 ms 2024-09-05T06:56:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 -- 172.21.15.86:0/1633379436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504102570 msgr2=0x7f55041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 --2- 172.21.15.86:0/1633379436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504102570 0x7f55041029b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009900 tx=0x7f54f402a480 comp rx=0 tx=0).stop 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 -- 172.21.15.86:0/1633379436 shutdown_connections 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 --2- 172.21.15.86:0/1633379436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 --2- 172.21.15.86:0/1633379436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504102570 0x7f55041029b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 --2- 172.21.15.86:0/1633379436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504100e20 0x7f5504101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 -- 172.21.15.86:0/1633379436 >> 172.21.15.86:0/1633379436 conn(0x7f55040fc7d0 msgr2=0x7f55040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 -- 172.21.15.86:0/1633379436 shutdown_connections 2024-09-05T06:56:56.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.339+0000 7f550b557640 1 -- 172.21.15.86:0/1633379436 wait complete. 2024-09-05T06:56:56.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 Processor -- start 2024-09-05T06:56:56.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 -- start start 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f55092f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f55092f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55808/0 (socket says 172.21.15.86:55808) 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f5508af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f5508af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36922/0 (socket says 172.21.15.86:36922) 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111f80 con 0x7f5504100e20 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111e00 con 0x7f5504102ef0 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f55092f3640 1 -- 172.21.15.86:0/1490059689 learned_addr learned my addr 172.21.15.86:0/1490059689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.340+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5504112100 con 0x7f5504102570 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f5509af4640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 -- 172.21.15.86:0/1490059689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 msgr2=0x7f550410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 -- 172.21.15.86:0/1490059689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 msgr2=0x7f550410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 -- 172.21.15.86:0/1490059689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55041a6930 con 0x7f5504100e20 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f5508af2640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f5509af4640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f55092f3640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f550000ea60 tx=0x7f550000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f54fa7fc640 1 -- 172.21.15.86:0/1490059689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550000cde0 con 0x7f5504100e20 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f54fa7fc640 1 -- 172.21.15.86:0/1490059689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55000035f0 con 0x7f5504100e20 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.341+0000 7f54fa7fc640 1 -- 172.21.15.86:0/1490059689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500010690 con 0x7f5504100e20 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.342+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55041a6c20 con 0x7f5504100e20 2024-09-05T06:56:56.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.342+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55041a70e0 con 0x7f5504100e20 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 Processor -- start 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 -- start start 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f54fa7fc640 1 -- 172.21.15.86:0/1490059689 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55000037a0 con 0x7f5504100e20 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631e84372f0 con 0x5631e84db350 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631e83decf0 con 0x5631e84e4b80 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.346+0000 7f550d930740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631e84332b0 con 0x5631e84d9de0 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f5509af4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f55092f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f5508af2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f5508af2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46462/0 (socket says 172.21.15.86:46462) 2024-09-05T06:56:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f5508af2640 1 -- 172.21.15.86:0/3405046312 learned_addr learned my addr 172.21.15.86:0/3405046312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.347+0000 7f54fa7fc640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 0x7f54f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5508af2640 1 -- 172.21.15.86:0/3405046312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 msgr2=0x5631e84da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f54fa7fc640 1 -- 172.21.15.86:0/1490059689 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f55000cfab0 con 0x7f5504100e20 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5508af2640 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5508af2640 1 -- 172.21.15.86:0/3405046312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 msgr2=0x5631e84d98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5508af2640 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84d98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5508af2640 1 -- 172.21.15.86:0/3405046312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e83c33a0 con 0x5631e84e4b80 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5509af4640 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.348+0000 7f5509af4640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 0x7f54f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.349+0000 7f5508af2640 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009900 tx=0x7f54f4004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8272e81901fe3394 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.350+0000 7f54ef7fe640 1 -- 172.21.15.86:0/3405046312 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4035040 con 0x5631e84e4b80 2024-09-05T06:56:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.350+0000 7f54ef7fe640 1 -- 172.21.15.86:0/3405046312 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4004760 con 0x5631e84e4b80 2024-09-05T06:56:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.350+0000 7f54ef7fe640 1 -- 172.21.15.86:0/3405046312 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40039e0 con 0x5631e84e4b80 2024-09-05T06:56:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 -- 172.21.15.86:0/3405046312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 msgr2=0x5631e84e6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009900 tx=0x7f54f4004480 comp rx=0 tx=0).stop 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f5509af4640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 0x7f54f00a52b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0094d0 tx=0x7f54fc014040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 -- 172.21.15.86:0/3405046312 shutdown_connections 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e84e6f70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 --2- 172.21.15.86:0/3405046312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 -- 172.21.15.86:0/3405046312 >> 172.21.15.86:0/3405046312 conn(0x5631e849a310 msgr2=0x5631e84d6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.351+0000 7f550d930740 1 -- 172.21.15.86:0/3405046312 shutdown_connections 2024-09-05T06:56:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.352+0000 7f550d930740 1 -- 172.21.15.86:0/3405046312 wait complete. 2024-09-05T06:56:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.352+0000 7f550d930740 1 -- start start 2024-09-05T06:56:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.352+0000 7f550d930740 1 Processor -- start 2024-09-05T06:56:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.353+0000 7f550d930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.353+0000 7f550d930740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.353+0000 7f55092f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.353+0000 7f55092f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46470/0 (socket says 172.21.15.86:46470) 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.353+0000 7f550d930740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e84e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 -- 172.21.15.86:0/658060979 learned_addr learned my addr 172.21.15.86:0/658060979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631e84cda60 con 0x5631e84db350 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f5508af2640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631e83f2060 con 0x5631e84d9de0 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631e84cdbe0 con 0x5631e84e4b80 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f5509af4640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e84e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f550d930740 10 client.0 _ll_register_callbacks cb 0x5631e81458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 -- 172.21.15.86:0/658060979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 msgr2=0x5631e84e4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e84e4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 -- 172.21.15.86:0/658060979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 msgr2=0x5631e84e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f55092f3640 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e81a83c0 con 0x5631e84d9de0 2024-09-05T06:56:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.354+0000 7f5508af2640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.355+0000 7f55092f3640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f550000ea30 tx=0x7f5500097e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4e2627761d9db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.355+0000 7f54ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.355+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550009ce60 con 0x5631e84d9de0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.355+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4004760 con 0x5631e84d9de0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.355+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f55000ccfe0 con 0x5631e84d9de0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f55000cfe60 con 0x5631e84d9de0 2024-09-05T06:56:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 10 client.27983 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5631e84fbb20 con 0x5631e84d9de0 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4004760 con 0x5631e84d9de0 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f55000981a0 con 0x5631e84d9de0 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 10 client.27983 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 10 client.27983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 20 client.27983 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.356+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e84fd390 0x5631e84ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.357+0000 7f550d930740 4 client.27983 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.357+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631e8444860 con 0x5631e84fd390 2024-09-05T06:56:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.357+0000 7f5508af2640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e84fd390 0x5631e84ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.358+0000 7f5508af2640 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e84fd390 0x5631e84ff820 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=673a74474067a3a7 server_cookie=e03979b739e98c08 in_seq=0 out_seq=0 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.358+0000 7f54ecff9640 10 client.27983 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.358+0000 7f54ecff9640 1 -- 172.21.15.86:0/658060979 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631e8444860 con 0x5631e84fd390 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.358+0000 7f54ecff9640 10 client.27983 handle_command_reply: tid=0 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 client.27983 shutdown 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 msgr2=0x5631e84d5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f550000ea30 tx=0x7f5500097e90 comp rx=0 tx=0).stop 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 shutdown_connections 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e84fd390 0x5631e84ff820 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e84e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 --2- 172.21.15.86:0/658060979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84d9de0 0x5631e84d5910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 >> 172.21.15.86:0/658060979 conn(0x5631e849a310 msgr2=0x5631e84dc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f54ecff9640 0 client.27983 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 shutdown_connections 2024-09-05T06:56:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.359+0000 7f550d930740 1 -- 172.21.15.86:0/658060979 wait complete. 2024-09-05T06:56:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.360+0000 7f550d930740 20 client.27983 trim_cache size 0 max 16384 2024-09-05T06:56:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 Processor -- start 2024-09-05T06:56:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 -- start start 2024-09-05T06:56:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 0x5631e8505130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e8505670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ad050 con 0x5631e84d9de0 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55041aecb0 con 0x5631e84db350 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f550d930740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ad720 con 0x5631e84e4b80 2024-09-05T06:56:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f55092f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.378+0000 7f5508af2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 0x5631e8505130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f55092f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55830/0 (socket says 172.21.15.86:55830) 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f55092f3640 1 -- 172.21.15.86:0/3184287650 learned_addr learned my addr 172.21.15.86:0/3184287650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5509af4640 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e8505670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5508af2640 1 -- 172.21.15.86:0/3184287650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 msgr2=0x5631e8505670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5508af2640 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e8505670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5508af2640 1 -- 172.21.15.86:0/3184287650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 msgr2=0x5631e84d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5508af2640 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f5508af2640 1 -- 172.21.15.86:0/3184287650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e84e44f0 con 0x5631e84db350 2024-09-05T06:56:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.379+0000 7f55092f3640 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.380+0000 7f5508af2640 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 0x5631e8505130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54f402aa70 tx=0x7f54f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e97330556dd925 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.380+0000 7f54eeffd640 1 -- 172.21.15.86:0/3184287650 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4035040 con 0x5631e84db350 2024-09-05T06:56:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.380+0000 7f54eeffd640 1 -- 172.21.15.86:0/3184287650 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f4005570 con 0x5631e84db350 2024-09-05T06:56:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.380+0000 7f54eeffd640 1 -- 172.21.15.86:0/3184287650 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4003cb0 con 0x5631e84db350 2024-09-05T06:56:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- 172.21.15.86:0/3184287650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 msgr2=0x5631e8505130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 0x5631e8505130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54f402aa70 tx=0x7f54f4000bc0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- 172.21.15.86:0/3184287650 shutdown_connections 2024-09-05T06:56:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84e4b80 0x5631e8505670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84db350 0x5631e8505130 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 --2- 172.21.15.86:0/3184287650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84d9de0 0x5631e84d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- 172.21.15.86:0/3184287650 >> 172.21.15.86:0/3184287650 conn(0x5631e849a310 msgr2=0x5631e84d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- 172.21.15.86:0/3184287650 shutdown_connections 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- 172.21.15.86:0/3184287650 wait complete. 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.382+0000 7f550d930740 1 -- start start 2024-09-05T06:56:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.383+0000 7f550d930740 1 Processor -- start 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.383+0000 7f550d930740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.383+0000 7f550d930740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.383+0000 7f55092f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f5508af2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f5508af2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84de400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55836/0 (socket says 172.21.15.86:55836) 2024-09-05T06:56:56.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f5508af2640 1 -- 172.21.15.86:0/3115307171 learned_addr learned my addr 172.21.15.86:0/3115307171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e8566100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ae520 con 0x5631e84db350 2024-09-05T06:56:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ade20 con 0x5631e84e4b80 2024-09-05T06:56:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ab050 con 0x5631e84d9de0 2024-09-05T06:56:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f550d930740 10 client.0 _ll_register_callbacks cb 0x7f55000976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.384+0000 7f5509af4640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e8566100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 -- 172.21.15.86:0/3115307171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 msgr2=0x5631e84ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 -- 172.21.15.86:0/3115307171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 msgr2=0x5631e84de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84de400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e8566740 con 0x5631e84e4b80 2024-09-05T06:56:56.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f5509af4640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e8566100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005fd0 tx=0x7f54fc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee912f8f417b7fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:56.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.385+0000 7f54ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:56:56.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.386+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc010b80 con 0x5631e84e4b80 2024-09-05T06:56:56.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.386+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f54fc010d20 con 0x5631e84e4b80 2024-09-05T06:56:56.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.386+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4005570 con 0x5631e84e4b80 2024-09-05T06:56:56.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.388+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f54fc029410 con 0x5631e84e4b80 2024-09-05T06:56:56.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f550d930740 10 client.27989 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:56.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5631e8567a30 con 0x5631e84e4b80 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54fc029a60 con 0x5631e84e4b80 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f54fc01d720 con 0x5631e84e4b80 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f550d930740 10 client.27989 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.389+0000 7f550d930740 10 client.27989 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.390+0000 7f550d930740 20 client.27989 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.390+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e8568240 0x5631e856a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.390+0000 7f550d930740 4 client.27989 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.390+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f54f4005570 con 0x5631e8568240 2024-09-05T06:56:56.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.390+0000 7f55092f3640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e8568240 0x5631e856a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:56.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f55092f3640 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e8568240 0x5631e856a660 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bce6891092ec7dd server_cookie=daf330c1bc43528 in_seq=0 out_seq=0 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f54ecff9640 10 client.27989 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f54ecff9640 1 -- 172.21.15.86:0/3115307171 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f54f4005570 con 0x5631e8568240 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f54ecff9640 10 client.27989 handle_command_reply: tid=0 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 client.27989 shutdown 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 msgr2=0x5631e8566100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e8566100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005fd0 tx=0x7f54fc009500 comp rx=0 tx=0).stop 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 shutdown_connections 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5631e8568240 0x5631e856a660 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631e84e4b80 0x5631e8566100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631e84db350 0x5631e84de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 --2- 172.21.15.86:0/3115307171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631e84d9de0 0x5631e84ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 >> 172.21.15.86:0/3115307171 conn(0x5631e849a310 msgr2=0x5631e84cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.391+0000 7f54ecff9640 0 client.27989 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:56:56.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 shutdown_connections 2024-09-05T06:56:56.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550d930740 1 -- 172.21.15.86:0/3115307171 wait complete. 2024-09-05T06:56:56.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550d930740 20 client.27989 trim_cache size 0 max 16384 2024-09-05T06:56:56.394 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:56.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 msgr2=0x7f54f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 0x7f54f00a52b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0094d0 tx=0x7f54fc014040 comp rx=0 tx=0).stop 2024-09-05T06:56:56.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 msgr2=0x7f550410e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:56.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.392+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f550000ea60 tx=0x7f550000ef20 comp rx=0 tx=0).stop 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 shutdown_connections 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f54f00a2df0 0x7f54f00a52b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5504102ef0 0x7f550410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5504102570 0x7f550410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 --2- 172.21.15.86:0/1490059689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5504100e20 0x7f550410e4d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 >> 172.21.15.86:0/1490059689 conn(0x7f55040fc7d0 msgr2=0x7f55040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 shutdown_connections 2024-09-05T06:56:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:56.393+0000 7f550b557640 1 -- 172.21.15.86:0/1490059689 wait complete. 2024-09-05T06:56:56.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17762 9.78 MB/sec execute 14 sec latency 66.708 ms 2024-09-05T06:56:57.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:57 smithi086 ceph-mon[30943]: pgmap v491: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:57.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:57 smithi115 ceph-mon[34791]: pgmap v491: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:57.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:57 smithi096 ceph-mon[34468]: pgmap v491: 129 pgs: 129 active+clean; 667 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-09-05T06:56:57.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17928 9.89 MB/sec execute 15 sec latency 99.694 ms 2024-09-05T06:56:57.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:56:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:56:58.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18068 9.49 MB/sec execute 16 sec latency 80.831 ms 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 -- 172.21.15.86:0/2834587005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 msgr2=0x7f35c01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 --2- 172.21.15.86:0/2834587005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c01067f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35b400dbf0 tx=0x7f35b402e2a0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 -- 172.21.15.86:0/2834587005 shutdown_connections 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 --2- 172.21.15.86:0/2834587005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 0x7f35c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 --2- 172.21.15.86:0/2834587005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c01067f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 --2- 172.21.15.86:0/2834587005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 0x7f35c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 -- 172.21.15.86:0/2834587005 >> 172.21.15.86:0/2834587005 conn(0x7f35c00fd8f0 msgr2=0x7f35c00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 -- 172.21.15.86:0/2834587005 shutdown_connections 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.823+0000 7f35c512a640 1 -- 172.21.15.86:0/2834587005 wait complete. 2024-09-05T06:56:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 Processor -- start 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 -- start start 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 0x7f35c01960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 0x7f35c019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010fbc0 con 0x7f35c0101df0 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010fa40 con 0x7f35c0106f20 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.824+0000 7f35c512a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010fd40 con 0x7f35c01027a0 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36988/0 (socket says 172.21.15.86:36988) 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 -- 172.21.15.86:0/4109514929 learned_addr learned my addr 172.21.15.86:0/4109514929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35bed9d640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 0x7f35c01960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35bf59e640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 0x7f35c019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 -- 172.21.15.86:0/4109514929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 msgr2=0x7f35c019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 0x7f35c019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 -- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 msgr2=0x7f35c01960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 0x7f35c01960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 -- 172.21.15.86:0/4109514929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c019b0a0 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35be59c640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005940 tx=0x7f35b4036af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35a7fff640 1 -- 172.21.15.86:0/4109514929 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4044030 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c019b330 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35a7fff640 1 -- 172.21.15.86:0/4109514929 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b40046c0 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35a7fff640 1 -- 172.21.15.86:0/4109514929 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b400b5d0 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.825+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c01a2c70 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.828+0000 7f35a7fff640 1 -- 172.21.15.86:0/4109514929 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f35b4047070 con 0x7f35c01027a0 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 Processor -- start 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 -- start start 2024-09-05T06:56:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 0x562df4d1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562df4c762f0 con 0x562df4d105d0 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562df4c1dcf0 con 0x562df4d1b370 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35c7503740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562df4c722b0 con 0x562df4d11b40 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.829+0000 7f35a7fff640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 0x7f35a80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35be59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 0x562df4d1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55866/0 (socket says 172.21.15.86:55866) 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 -- 172.21.15.86:0/4186390294 learned_addr learned my addr 172.21.15.86:0/4186390294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bf59e640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 0x7f35a80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 -- 172.21.15.86:0/4186390294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 msgr2=0x562df4d10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 -- 172.21.15.86:0/4186390294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 msgr2=0x562df4d1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 0x562df4d1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 -- 172.21.15.86:0/4186390294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df4c023a0 con 0x562df4d105d0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35be59c640 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 0x562df4d1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bed9d640 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009d10 tx=0x7f35b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7940a60e21ab3050 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35bf59e640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 0x7f35a80a52b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009270 tx=0x7f35b80091e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.830+0000 7f35a57fa640 1 -- 172.21.15.86:0/4186390294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0005220 con 0x562df4d105d0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35a57fa640 1 -- 172.21.15.86:0/4186390294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b0004c50 con 0x562df4d105d0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35a57fa640 1 -- 172.21.15.86:0/4186390294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00198a0 con 0x562df4d105d0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35a7fff640 1 -- 172.21.15.86:0/4109514929 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f35b40f3720 con 0x7f35c01027a0 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 -- 172.21.15.86:0/4186390294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 msgr2=0x562df4d10a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009d10 tx=0x7f35b0011810 comp rx=0 tx=0).stop 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 -- 172.21.15.86:0/4186390294 shutdown_connections 2024-09-05T06:56:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d1b370 0x562df4d1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d10a10 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 --2- 172.21.15.86:0/4186390294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.831+0000 7f35c7503740 1 -- 172.21.15.86:0/4186390294 >> 172.21.15.86:0/4186390294 conn(0x562df4cd9310 msgr2=0x562df4d0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- 172.21.15.86:0/4186390294 shutdown_connections 2024-09-05T06:56:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- 172.21.15.86:0/4186390294 wait complete. 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- start start 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 Processor -- start 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d1ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d18020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 0x562df4d18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562df4d1e070 con 0x562df4d1b370 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562df4c31060 con 0x562df4d105d0 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562df4d1e1f0 con 0x562df4d11b40 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.832+0000 7f35c7503740 10 client.0 _ll_register_callbacks cb 0x562df49848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d18020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 0x562df4d18630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d18020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:37004/0 (socket says 172.21.15.86:37004) 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35be59c640 1 -- 172.21.15.86:0/2446462989 learned_addr learned my addr 172.21.15.86:0/2446462989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bed9d640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d1ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 -- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 msgr2=0x562df4d18020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d18020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 -- 172.21.15.86:0/2446462989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 msgr2=0x562df4d1ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d1ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 -- 172.21.15.86:0/2446462989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df49e73c0 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35bf59e640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 0x562df4d18630 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f35b8012d20 tx=0x7f35b8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf3ac4a059c795f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8029060 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f35b8017070 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.833+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b0004c50 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f35b8030050 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 10 client.19062 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562df4d29b60 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b0004c50 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f35b8023070 con 0x562df4d1b370 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 10 client.19062 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 10 client.19062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 20 client.19062 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4d2d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 4 client.19062 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.834+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df4c83860 con 0x562df4d2b020 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.835+0000 7f35bed9d640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4d2d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.835+0000 7f35bed9d640 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4d2d450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6980c48ee3cfb2cd server_cookie=4dc676d58a2ef097 in_seq=0 out_seq=0 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.835+0000 7f35a2ff5640 10 client.19062 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.836+0000 7f35a2ff5640 1 -- 172.21.15.86:0/2446462989 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562df4c83860 con 0x562df4d2b020 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.836+0000 7f35a2ff5640 10 client.19062 handle_command_reply: tid=0 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.836+0000 7f35c7503740 1 client.19062 shutdown 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.836+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 msgr2=0x562df4d18630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 0x562df4d18630 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f35b8012d20 tx=0x7f35b8009500 comp rx=0 tx=0).stop 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 shutdown_connections 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4d2d450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d1b370 0x562df4d18630 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35a2ff5640 0 client.19062 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d11b40 0x562df4d18020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 --2- 172.21.15.86:0/2446462989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 >> 172.21.15.86:0/2446462989 conn(0x562df4cd9310 msgr2=0x562df4d12960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 shutdown_connections 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 1 -- 172.21.15.86:0/2446462989 wait complete. 2024-09-05T06:56:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.837+0000 7f35c7503740 20 client.19062 trim_cache size 0 max 16384 2024-09-05T06:56:58.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 Processor -- start 2024-09-05T06:56:58.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 -- start start 2024-09-05T06:56:58.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 0x562df4d13120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 0x562df4d13120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01acf40 con 0x562df4d11b40 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01aeba0 con 0x562df4d105d0 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35c7503740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01ad610 con 0x562df4d1b370 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35bed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35bed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46536/0 (socket says 172.21.15.86:46536) 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 0x562df4d13120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55888/0 (socket says 172.21.15.86:55888) 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.862+0000 7f35bed9d640 1 -- 172.21.15.86:0/3811253193 learned_addr learned my addr 172.21.15.86:0/3811253193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bf59e640 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 -- 172.21.15.86:0/3811253193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 msgr2=0x562df4d13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d13660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 -- 172.21.15.86:0/3811253193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 msgr2=0x562df4d13120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 0x562df4d13120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 -- 172.21.15.86:0/3811253193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df4d0c4f0 con 0x562df4d105d0 2024-09-05T06:56:58.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562df4d0dac0 tx=0x7f35b00054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d7d382b9a2a504 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35a4ff9640 1 -- 172.21.15.86:0/3811253193 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0019910 con 0x562df4d105d0 2024-09-05T06:56:58.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35a4ff9640 1 -- 172.21.15.86:0/3811253193 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b002e2d0 con 0x562df4d105d0 2024-09-05T06:56:58.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.863+0000 7f35a4ff9640 1 -- 172.21.15.86:0/3811253193 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0024670 con 0x562df4d105d0 2024-09-05T06:56:58.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 -- 172.21.15.86:0/3811253193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 msgr2=0x562df4d19f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562df4d0dac0 tx=0x7f35b00054d0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 -- 172.21.15.86:0/3811253193 shutdown_connections 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d11b40 0x562df4d13120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 --2- 172.21.15.86:0/3811253193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d105d0 0x562df4d19f90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 -- 172.21.15.86:0/3811253193 >> 172.21.15.86:0/3811253193 conn(0x562df4cd9310 msgr2=0x562df4d19880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.864+0000 7f35c7503740 1 -- 172.21.15.86:0/3811253193 shutdown_connections 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 -- 172.21.15.86:0/3811253193 wait complete. 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 -- start start 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 Processor -- start 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d15a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.865+0000 7f35c7503740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01ae410 con 0x562df4d105d0 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46550/0 (socket says 172.21.15.86:46550) 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 -- 172.21.15.86:0/3857632878 learned_addr learned my addr 172.21.15.86:0/3857632878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35bf59e640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d15a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01add10 con 0x562df4d11b40 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01aaf40 con 0x562df4d1b370 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35c7503740 10 client.0 _ll_register_callbacks cb 0x7f35b80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 -- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 msgr2=0x562df4d15a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d15a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 -- 172.21.15.86:0/3857632878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 msgr2=0x562df4d14f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:56:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d14f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df4d9d500 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35be59c640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005fd0 tx=0x7f35b40bf740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc27aa0b2f7ede9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b40bf8b0 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.866+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7f35b40f0c50 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b002e2d0 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f35b40c3c60 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 10 client.28007 fetch_fsmap learned FSMap version 38 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562df4d9e760 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b002e2d0 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f35b4004870 con 0x562df4d11b40 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 10 client.28007 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 10 client.28007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 20 client.28007 populate_metadata read hostname 'smithi086' 2024-09-05T06:56:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4da0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:56:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 4 client.28007 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:56:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.867+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562df4c7dd70 con 0x562df4d2b020 2024-09-05T06:56:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4da0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35bed9d640 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4da0f50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac8cc8cceb75b071 server_cookie=3e47a918ac42834 in_seq=0 out_seq=0 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35a2ff5640 10 client.28007 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35a2ff5640 1 -- 172.21.15.86:0/3857632878 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562df4c7dd70 con 0x562df4d2b020 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35a2ff5640 10 client.28007 handle_command_reply: tid=0 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.868+0000 7f35c7503740 1 client.28007 shutdown 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 msgr2=0x562df4d154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005fd0 tx=0x7f35b40bf740 comp rx=0 tx=0).stop 2024-09-05T06:56:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 shutdown_connections 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562df4d2b020 0x562df4da0f50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562df4d1b370 0x562df4d15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562df4d11b40 0x562df4d154c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 --2- 172.21.15.86:0/3857632878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562df4d105d0 0x562df4d14f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 >> 172.21.15.86:0/3857632878 conn(0x562df4cd9310 msgr2=0x562df4d1a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35a2ff5640 0 client.28007 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 shutdown_connections 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.869+0000 7f35c7503740 1 -- 172.21.15.86:0/3857632878 wait complete. 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c7503740 20 client.28007 trim_cache size 0 max 16384 2024-09-05T06:56:58.873 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:56:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 msgr2=0x7f35a80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 0x7f35a80a52b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009270 tx=0x7f35b80091e0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 msgr2=0x7f35c0196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:56:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005940 tx=0x7f35b4036af0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 shutdown_connections 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f35a80a2df0 0x7f35a80a52b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35c0106f20 0x7f35c019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35c01027a0 0x7f35c0196630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 --2- 172.21.15.86:0/4109514929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c0101df0 0x7f35c01960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 >> 172.21.15.86:0/4109514929 conn(0x7f35c00fd8f0 msgr2=0x7f35c00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.870+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 shutdown_connections 2024-09-05T06:56:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:56:58.871+0000 7f35c512a640 1 -- 172.21.15.86:0/4109514929 wait complete. 2024-09-05T06:56:59.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:56:59 smithi115 ceph-mon[34791]: pgmap v492: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 336 op/s 2024-09-05T06:56:59.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:56:59 smithi086 ceph-mon[30943]: pgmap v492: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 336 op/s 2024-09-05T06:56:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:56:59 smithi096 ceph-mon[34468]: pgmap v492: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 336 op/s 2024-09-05T06:56:59.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18253 9.69 MB/sec execute 17 sec latency 110.637 ms 2024-09-05T06:56:59.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:57:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:00.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.723+0000 7fb4cb31f640 1 -- 172.21.15.86:0/567209987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 msgr2=0x7fb4c410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.723+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/567209987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c410ec30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009cd0 tx=0x7fb4bc02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 -- 172.21.15.86:0/567209987 shutdown_connections 2024-09-05T06:57:00.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/567209987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c410ec30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/567209987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4103740 0x7fb4c4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/567209987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4102d90 0x7fb4c4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 -- 172.21.15.86:0/567209987 >> 172.21.15.86:0/567209987 conn(0x7fb4c40fc7d0 msgr2=0x7fb4c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 -- 172.21.15.86:0/567209987 shutdown_connections 2024-09-05T06:57:00.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 -- 172.21.15.86:0/567209987 wait complete. 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.724+0000 7fb4cb31f640 1 Processor -- start 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 -- start start 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4102d90 0x7fb4c41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 0x7fb4c4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4111f50 con 0x7fb4c4102d90 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4111dd0 con 0x7fb4c4108290 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4cb31f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41120d0 con 0x7fb4c4103740 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4c98bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4102d90 0x7fb4c41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4c98bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46574/0 (socket says 172.21.15.86:46574) 2024-09-05T06:57:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4c98bc640 1 -- 172.21.15.86:0/860928553 learned_addr learned my addr 172.21.15.86:0/860928553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.725+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 0x7fb4c4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 -- 172.21.15.86:0/860928553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 msgr2=0x7fb4c4101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 -- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4102d90 msgr2=0x7fb4c41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4102d90 0x7fb4c41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 -- 172.21.15.86:0/860928553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c41aae20 con 0x7fb4c4103740 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 0x7fb4c4100f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9c0 tx=0x7fb4b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4ba7fc640 1 -- 172.21.15.86:0/860928553 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400ccc0 con 0x7fb4c4103740 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.726+0000 7fb4ba7fc640 1 -- 172.21.15.86:0/860928553 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b40108e0 con 0x7fb4c4103740 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.727+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c41aaff0 con 0x7fb4c4103740 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.727+0000 7fb4ba7fc640 1 -- 172.21.15.86:0/860928553 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40187e0 con 0x7fb4c4103740 2024-09-05T06:57:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.727+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41ab520 con 0x7fb4c4103740 2024-09-05T06:57:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.729+0000 7fb4ba7fc640 1 -- 172.21.15.86:0/860928553 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4b4010a90 con 0x7fb4c4103740 2024-09-05T06:57:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 Processor -- start 2024-09-05T06:57:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 -- start start 2024-09-05T06:57:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24ebb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 0x55bc24ec8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc24e212f0 con 0x55bc24ebb5d0 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc24dc8cf0 con 0x55bc24ec6370 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc24e1d2b0 con 0x55bc24ebcb40 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4c88ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 0x55bc24ec8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55930/0 (socket says 172.21.15.86:55930) 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.730+0000 7fb4c90bb640 1 -- 172.21.15.86:0/1813632169 learned_addr learned my addr 172.21.15.86:0/1813632169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4ba7fc640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 0x7fb4b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24ebb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4ba7fc640 1 -- 172.21.15.86:0/860928553 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb4b40cfab0 con 0x7fb4c4103740 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 -- 172.21.15.86:0/1813632169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 msgr2=0x55bc24ebb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24ebb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 -- 172.21.15.86:0/1813632169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 msgr2=0x55bc24ebba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 -- 172.21.15.86:0/1813632169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc24dad3a0 con 0x55bc24ec6370 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 0x7fb4b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 0x55bc24ec8760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ccdc0 tx=0x7fb4b400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec9cbb649970d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4af7fe640 1 -- 172.21.15.86:0/1813632169 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b409bc40 con 0x55bc24ec6370 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4af7fe640 1 -- 172.21.15.86:0/1813632169 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b4086070 con 0x55bc24ec6370 2024-09-05T06:57:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.731+0000 7fb4af7fe640 1 -- 172.21.15.86:0/1813632169 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40cc980 con 0x55bc24ec6370 2024-09-05T06:57:00.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.732+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1813632169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 msgr2=0x55bc24ec8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.732+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 0x55bc24ec8760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ccdc0 tx=0x7fb4b400efb0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.732+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 0x7fb4b00a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009a60 tx=0x7fb4bc009360 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1813632169 shutdown_connections 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ec6370 0x55bc24ec8760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ebba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1813632169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24ebb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1813632169 >> 172.21.15.86:0/1813632169 conn(0x55bc24e84310 msgr2=0x55bc24eb7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1813632169 shutdown_connections 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1813632169 wait complete. 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 -- start start 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.733+0000 7fb4cd6f8740 1 Processor -- start 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ec5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ec3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ec5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ec5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55936/0 (socket says 172.21.15.86:55936) 2024-09-05T06:57:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 0x55bc24ec3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c88ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ec3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c88ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ec3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46596/0 (socket says 172.21.15.86:46596) 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc24ec9010 con 0x55bc24ebb5d0 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c90bb640 1 -- 172.21.15.86:0/350827932 learned_addr learned my addr 172.21.15.86:0/350827932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc24ddc060 con 0x55bc24ebcb40 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc24ec9190 con 0x55bc24ec6370 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 0x55bc24ec3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.734+0000 7fb4cd6f8740 10 client.0 _ll_register_callbacks cb 0x55bc24b2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 -- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 msgr2=0x55bc24ec3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ec3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 -- 172.21.15.86:0/350827932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 msgr2=0x55bc24ec5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ec5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 -- 172.21.15.86:0/350827932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc24b923c0 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 0x55bc24ec3810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d924b9808fa5a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4acff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0305a0 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb4bc030740 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.735+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b4086070 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.736+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4bc030940 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.736+0000 7fb4cd6f8740 10 client.28000 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.736+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc24ed4b10 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.736+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4bc036040 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb4bc030070 con 0x55bc24ec6370 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 10 client.28000 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 10 client.28000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 20 client.28000 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24ed6490 0x55bc24ed8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 4 client.28000 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb4b4086070 con 0x55bc24ed6490 2024-09-05T06:57:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24ed6490 0x55bc24ed8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.737+0000 7fb4acff9640 10 client.28000 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.738+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24ed6490 0x55bc24ed8910 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2325791655fda99 server_cookie=59d3829432f71f2a in_seq=0 out_seq=0 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.738+0000 7fb4acff9640 1 -- 172.21.15.86:0/350827932 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb4b4086070 con 0x55bc24ed6490 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.738+0000 7fb4acff9640 10 client.28000 handle_command_reply: tid=0 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.738+0000 7fb4cd6f8740 1 client.28000 shutdown 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 msgr2=0x55bc24ec3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 0x55bc24ec3810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc042000 comp rx=0 tx=0).stop 2024-09-05T06:57:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 shutdown_connections 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24ed6490 0x55bc24ed8910 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ec6370 0x55bc24ec3810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ec3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/350827932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ebb5d0 0x55bc24ec5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 >> 172.21.15.86:0/350827932 conn(0x55bc24e84310 msgr2=0x55bc24ebd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4acff9640 0 client.28000 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 shutdown_connections 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/350827932 wait complete. 2024-09-05T06:57:00.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.739+0000 7fb4cd6f8740 20 client.28000 trim_cache size 0 max 16384 2024-09-05T06:57:00.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18421 9.51 MB/sec execute 18 sec latency 104.999 ms 2024-09-05T06:57:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 Processor -- start 2024-09-05T06:57:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 -- start start 2024-09-05T06:57:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebb5d0 0x55bc24eb7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24eb7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b5830 con 0x55bc24ec6370 2024-09-05T06:57:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b7490 con 0x55bc24ebb5d0 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.761+0000 7fb4cd6f8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b5f00 con 0x55bc24ebcb40 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55942/0 (socket says 172.21.15.86:55942) 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 -- 172.21.15.86:0/1800175173 learned_addr learned my addr 172.21.15.86:0/1800175173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24eb7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebb5d0 0x55bc24eb7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 -- 172.21.15.86:0/1800175173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 msgr2=0x55bc24eb7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24eb7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 -- 172.21.15.86:0/1800175173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebb5d0 msgr2=0x55bc24eb7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebb5d0 0x55bc24eb7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 -- 172.21.15.86:0/1800175173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc24eb8180 con 0x55bc24ec6370 2024-09-05T06:57:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009ca0 tx=0x7fb4bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b18a020d2adfe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4aeffd640 1 -- 172.21.15.86:0/1800175173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc030ae0 con 0x55bc24ec6370 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4aeffd640 1 -- 172.21.15.86:0/1800175173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4bc036070 con 0x55bc24ec6370 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.762+0000 7fb4aeffd640 1 -- 172.21.15.86:0/1800175173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0466c0 con 0x55bc24ec6370 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1800175173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 msgr2=0x55bc24ede220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009ca0 tx=0x7fb4bc009b20 comp rx=0 tx=0).stop 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1800175173 shutdown_connections 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ede220 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebcb40 0x55bc24eb7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/1800175173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebb5d0 0x55bc24eb7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.763+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1800175173 >> 172.21.15.86:0/1800175173 conn(0x55bc24e84310 msgr2=0x55bc24ebe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.764+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1800175173 shutdown_connections 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.764+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/1800175173 wait complete. 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.764+0000 7fb4cd6f8740 1 -- start start 2024-09-05T06:57:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.764+0000 7fb4cd6f8740 1 Processor -- start 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ebfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ec0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:37072/0 (socket says 172.21.15.86:37072) 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 -- 172.21.15.86:0/2819490179 learned_addr learned my addr 172.21.15.86:0/2819490179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b6d00 con 0x55bc24ec6370 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b6600 con 0x55bc24ebcb40 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b3830 con 0x55bc24ebb5d0 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ebfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4cd6f8740 10 client.0 _ll_register_callbacks cb 0x7fb4bc031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ec0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 -- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 msgr2=0x55bc24ebfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ebfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 -- 172.21.15.86:0/2819490179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 msgr2=0x55bc24ec0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ec0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc24ec05e0 con 0x55bc24ebb5d0 2024-09-05T06:57:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c98bc640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ec0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4c90bb640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005eb0 tx=0x7fb4c0012900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d5747f4856157 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.765+0000 7fb4acff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0004230 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9605+0+0 (secure 0 0 0) 0x7fb4c00043d0 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4bc036070 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4c0005020 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4cd6f8740 10 client.28006 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc24f40f90 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.766+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4bc036070 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fb4c0004cb0 con 0x55bc24ebb5d0 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 10 client.28006 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 10 client.28006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 20 client.28006 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24f41880 0x55bc24f43ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 4 client.28006 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc24e2e860 con 0x55bc24f41880 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.767+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24f41880 0x55bc24f43ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4acff9640 10 client.28006 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4c88ba640 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24f41880 0x55bc24f43ca0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d8867cf16ca77f server_cookie=4d7bd732c4ec622c in_seq=0 out_seq=0 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4acff9640 1 -- 172.21.15.86:0/2819490179 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc24e2e860 con 0x55bc24f41880 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4acff9640 10 client.28006 handle_command_reply: tid=0 2024-09-05T06:57:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 client.28006 shutdown 2024-09-05T06:57:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 msgr2=0x55bc24ece8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005eb0 tx=0x7fb4c0012900 comp rx=0 tx=0).stop 2024-09-05T06:57:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 shutdown_connections 2024-09-05T06:57:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc24f41880 0x55bc24f43ca0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc24ec6370 0x55bc24ec0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc24ebcb40 0x55bc24ebfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 --2- 172.21.15.86:0/2819490179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc24ebb5d0 0x55bc24ece8a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4acff9640 0 client.28006 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.768+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 >> 172.21.15.86:0/2819490179 conn(0x55bc24e84310 msgr2=0x55bc24ebe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.769+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 shutdown_connections 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.769+0000 7fb4cd6f8740 1 -- 172.21.15.86:0/2819490179 wait complete. 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.769+0000 7fb4cd6f8740 20 client.28006 trim_cache size 0 max 16384 2024-09-05T06:57:00.773 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:57:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 msgr2=0x7fb4b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 0x7fb4b00a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009a60 tx=0x7fb4bc009360 comp rx=0 tx=0).stop 2024-09-05T06:57:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 msgr2=0x7fb4c4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 0x7fb4c4100f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9c0 tx=0x7fb4b400ee80 comp rx=0 tx=0).stop 2024-09-05T06:57:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 shutdown_connections 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb4b00a2d70 0x7fb4b00a5230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c4108290 0x7fb4c4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4c4103740 0x7fb4c4100f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 --2- 172.21.15.86:0/860928553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c4102d90 0x7fb4c41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 >> 172.21.15.86:0/860928553 conn(0x7fb4c40fc7d0 msgr2=0x7fb4c4105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 shutdown_connections 2024-09-05T06:57:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:00.770+0000 7fb4cb31f640 1 -- 172.21.15.86:0/860928553 wait complete. 2024-09-05T06:57:01.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:01 smithi115 ceph-mon[34791]: pgmap v493: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-09-05T06:57:01.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:01 smithi086 ceph-mon[30943]: pgmap v493: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-09-05T06:57:01.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:01 smithi096 ceph-mon[34468]: pgmap v493: 129 pgs: 129 active+clean; 683 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-09-05T06:57:01.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18579 9.50 MB/sec execute 19 sec latency 103.149 ms 2024-09-05T06:57:02.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:02 smithi115 ceph-mon[34791]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T06:57:02.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:02 smithi086 ceph-mon[30943]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T06:57:02.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:02 smithi096 ceph-mon[34468]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T06:57:02.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18715 9.39 MB/sec execute 20 sec latency 143.881 ms 2024-09-05T06:57:03.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:03 smithi115 ceph-mon[34791]: pgmap v495: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:03 smithi086 ceph-mon[30943]: pgmap v495: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:03.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:03 smithi096 ceph-mon[34468]: pgmap v495: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:03.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18843 9.20 MB/sec execute 21 sec latency 122.923 ms 2024-09-05T06:57:04.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19001 9.26 MB/sec execute 22 sec latency 127.752 ms 2024-09-05T06:57:05.199 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.6 2024-09-05T06:57:05.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:57:05.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:05 smithi115 ceph-mon[34791]: pgmap v496: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:05.486 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:05 smithi086 ceph-mon[30943]: pgmap v496: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:05.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:05 smithi096 ceph-mon[34468]: pgmap v496: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:05.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19142 9.13 MB/sec execute 23 sec latency 151.246 ms 2024-09-05T06:57:05.896 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:06.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19274 8.88 MB/sec execute 24 sec latency 130.660 ms 2024-09-05T06:57:06.843 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:57:06.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:57:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:07 smithi115 ceph-mon[34791]: pgmap v497: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:07.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:07 smithi115 ceph-mon[34791]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T06:57:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:07 smithi086 ceph-mon[30943]: pgmap v497: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:07 smithi086 ceph-mon[30943]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T06:57:07.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:07 smithi096 ceph-mon[34468]: pgmap v497: 129 pgs: 129 active+clean; 683 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 281 op/s 2024-09-05T06:57:07.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:07 smithi096 ceph-mon[34468]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T06:57:07.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19415 8.71 MB/sec execute 25 sec latency 131.219 ms 2024-09-05T06:57:08.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19536 8.58 MB/sec execute 26 sec latency 184.063 ms 2024-09-05T06:57:09.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:09 smithi086 ceph-mon[30943]: pgmap v499: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 355 op/s 2024-09-05T06:57:09.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:09 smithi115 ceph-mon[34791]: pgmap v499: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 355 op/s 2024-09-05T06:57:09.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:09 smithi096 ceph-mon[34468]: pgmap v499: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 355 op/s 2024-09-05T06:57:09.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19657 8.44 MB/sec execute 27 sec latency 205.895 ms 2024-09-05T06:57:10.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2223053252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:10.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2223053252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:10.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2223053252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:10.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:57:10.578 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"btime":"2024-09-05T06:54:56:110779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:54:55.403358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:57:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- 172.21.15.86:0/3925031222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f899810c160 msgr2=0x7f899810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- 172.21.15.86:0/3925031222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f899810c160 0x7f899810e5e0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f8988009900 tx=0x7f898802a480 comp rx=0 tx=0).stop 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- 172.21.15.86:0/3925031222 shutdown_connections 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- 172.21.15.86:0/3925031222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f899810c160 0x7f899810e5e0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- 172.21.15.86:0/3925031222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89981036a0 0x7f899810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- 172.21.15.86:0/3925031222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8998102d80 0x7f8998103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- 172.21.15.86:0/3925031222 >> 172.21.15.86:0/3925031222 conn(0x7f89980fc780 msgr2=0x7f89980feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- 172.21.15.86:0/3925031222 shutdown_connections 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- 172.21.15.86:0/3925031222 wait complete. 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 Processor -- start 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- start start 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 0x7f8998198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f899810c160 0x7f899819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8998111db0 con 0x7f89981036a0 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8998111c30 con 0x7f8998102d80 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.589+0000 7f899eb2d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8998111f30 con 0x7f899810c160 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38492/0 (socket says 172.21.15.86:38492) 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 -- 172.21.15.86:0/2223053252 learned_addr learned my addr 172.21.15.86:0/2223053252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f899c8c9640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 0x7f8998198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f899d0ca640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f899810c160 0x7f899819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 -- 172.21.15.86:0/2223053252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f899810c160 msgr2=0x7f899819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f899810c160 0x7f899819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 -- 172.21.15.86:0/2223053252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 msgr2=0x7f8998198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 0x7f8998198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 -- 172.21.15.86:0/2223053252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819d2b0 con 0x7f89981036a0 2024-09-05T06:57:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f899c8c9640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 0x7f8998198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8997fff640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00cd80 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f898c0108e0 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819d5a0 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c018920 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.590+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a4e80 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.592+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8998103160 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.593+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f898c010a90 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.593+0000 7f8995ffb640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 0x7f89800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.593+0000 7f899c8c9640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 0x7f89800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.593+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f898c0cfa70 con 0x7f89981036a0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.594+0000 7f899c8c9640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 0x7f89800a5230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8990009fd0 tx=0x7f8990009760 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.598+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f898c09bc00 con 0x7f89981036a0 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.894+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f899802d7c0 con 0x7f89981036a0 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.895+0000 7f8995ffb640 1 -- 172.21.15.86:0/2223053252 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11983 (secure 0 0 0) 0x7f898c09ff00 con 0x7f89981036a0 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 38 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 msgr2=0x7f89800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 0x7f89800a5230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8990009fd0 tx=0x7f8990009760 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 msgr2=0x7f8998198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 shutdown_connections 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f89800a2d70 0x7f89800a5230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f899810c160 0x7f899819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89981036a0 0x7f8998198840 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 --2- 172.21.15.86:0/2223053252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8998102d80 0x7f8998198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.900+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 >> 172.21.15.86:0/2223053252 conn(0x7f89980fc780 msgr2=0x7f89980fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.901+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 shutdown_connections 2024-09-05T06:57:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:09.901+0000 7f899eb2d640 1 -- 172.21.15.86:0/2223053252 wait complete. 2024-09-05T06:57:10.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19826 8.55 MB/sec execute 28 sec latency 117.916 ms 2024-09-05T06:57:11.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:11 smithi115 ceph-mon[34791]: pgmap v500: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-05T06:57:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:11 smithi086 ceph-mon[30943]: pgmap v500: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-05T06:57:11.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:11 smithi096 ceph-mon[34468]: pgmap v500: 129 pgs: 129 active+clean; 698 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 336 op/s 2024-09-05T06:57:11.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19968 8.60 MB/sec execute 29 sec latency 168.689 ms 2024-09-05T06:57:12.660 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:12.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20132 8.58 MB/sec execute 30 sec latency 114.234 ms 2024-09-05T06:57:13.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:13 smithi115 ceph-mon[34791]: pgmap v501: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:13 smithi086 ceph-mon[30943]: pgmap v501: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:13 smithi096 ceph-mon[34468]: pgmap v501: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:13.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20354 8.92 MB/sec execute 31 sec latency 138.233 ms 2024-09-05T06:57:14.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20503 8.82 MB/sec execute 32 sec latency 133.456 ms 2024-09-05T06:57:15.232 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 131.6 --await-for 45.0' 2024-09-05T06:57:15.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 131.6 --await-for 45.0 2024-09-05T06:57:15.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:15 smithi115 ceph-mon[34791]: pgmap v502: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:15.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:15 smithi086 ceph-mon[30943]: pgmap v502: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:15 smithi096 ceph-mon[34468]: pgmap v502: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 -- 172.21.15.86:0/2768752054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830101430 msgr2=0x7f383010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2768752054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830101430 0x7f383010d960 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3828009cd0 tx=0x7f382802a8f0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 -- 172.21.15.86:0/2768752054 shutdown_connections 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2768752054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830101430 0x7f383010d960 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2768752054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 0x7f3830100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2768752054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830106a90 0x7f3830106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.716+0000 7f3837fb9640 1 -- 172.21.15.86:0/2768752054 >> 172.21.15.86:0/2768752054 conn(0x7f38300fc7d0 msgr2=0x7f38300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.717+0000 7f3837fb9640 1 -- 172.21.15.86:0/2768752054 shutdown_connections 2024-09-05T06:57:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.717+0000 7f3837fb9640 1 -- 172.21.15.86:0/2768752054 wait complete. 2024-09-05T06:57:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 Processor -- start 2024-09-05T06:57:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 -- start start 2024-09-05T06:57:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 0x7f38301a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830101430 0x7f38301a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3830111ea0 con 0x7f3830106a90 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3830111d20 con 0x7f3830100a90 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.718+0000 7f3837fb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3830112020 con 0x7f3830101430 2024-09-05T06:57:15.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3835554640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830101430 0x7f38301a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3835d55640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 0x7f38301a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49720/0 (socket says 172.21.15.86:49720) 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 -- 172.21.15.86:0/2208492724 learned_addr learned my addr 172.21.15.86:0/2208492724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 -- 172.21.15.86:0/2208492724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830101430 msgr2=0x7f38301a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830101430 0x7f38301a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 -- 172.21.15.86:0/2208492724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 msgr2=0x7f38301a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 0x7f38301a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 -- 172.21.15.86:0/2208492724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383010c240 con 0x7f3830106a90 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.719+0000 7f3836556640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f3828030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.720+0000 7f3826ffd640 1 -- 172.21.15.86:0/2208492724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828004040 con 0x7f3830106a90 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.720+0000 7f3826ffd640 1 -- 172.21.15.86:0/2208492724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3828040070 con 0x7f3830106a90 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.720+0000 7f3826ffd640 1 -- 172.21.15.86:0/2208492724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828045af0 con 0x7f3830106a90 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.720+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383010c4d0 con 0x7f3830106a90 2024-09-05T06:57:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.720+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301ab410 con 0x7f3830106a90 2024-09-05T06:57:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.722+0000 7f3826ffd640 1 -- 172.21.15.86:0/2208492724 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3828045c90 con 0x7f3830106a90 2024-09-05T06:57:15.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.723+0000 7f383a392740 1 Processor -- start 2024-09-05T06:57:15.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.723+0000 7f383a392740 1 -- start start 2024-09-05T06:57:15.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fbec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fbef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a2fb47280 con 0x557a2fbe1b60 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557a2faeeb10 con 0x557a2fbe2480 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f383a392740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a2fb43240 con 0x557a2fbecd40 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3826ffd640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 0x7f381c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49722/0 (socket says 172.21.15.86:49722) 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 -- 172.21.15.86:0/1444449140 learned_addr learned my addr 172.21.15.86:0/1444449140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fbef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3826ffd640 1 -- 172.21.15.86:0/2208492724 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f38280f0680 con 0x7f3830106a90 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3836556640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 0x7f381c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 -- 172.21.15.86:0/1444449140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 msgr2=0x557a2fbef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fbef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 -- 172.21.15.86:0/1444449140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 msgr2=0x557a2fbec800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fbec800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.724+0000 7f3835554640 1 -- 172.21.15.86:0/1444449140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2fafb1f0 con 0x557a2fbe1b60 2024-09-05T06:57:15.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.725+0000 7f3835554640 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f38200058c0 tx=0x7f3820005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b6926f3db724bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.725+0000 7f381bfff640 1 -- 172.21.15.86:0/1444449140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820013d30 con 0x557a2fbe1b60 2024-09-05T06:57:15.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.725+0000 7f3836556640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 0x7f381c0a5230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f38280bc800 tx=0x7f38280095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.725+0000 7f381bfff640 1 -- 172.21.15.86:0/1444449140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38200054d0 con 0x557a2fbe1b60 2024-09-05T06:57:15.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.725+0000 7f381bfff640 1 -- 172.21.15.86:0/1444449140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820004c60 con 0x557a2fbe1b60 2024-09-05T06:57:15.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- 172.21.15.86:0/1444449140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 msgr2=0x557a2fbe1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f38200058c0 tx=0x7f3820005850 comp rx=0 tx=0).stop 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- 172.21.15.86:0/1444449140 shutdown_connections 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fbef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fbec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 --2- 172.21.15.86:0/1444449140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe1f40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- 172.21.15.86:0/1444449140 >> 172.21.15.86:0/1444449140 conn(0x557a2fbde0f0 msgr2=0x557a2fbde4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- 172.21.15.86:0/1444449140 shutdown_connections 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- 172.21.15.86:0/1444449140 wait complete. 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.726+0000 7f383a392740 1 -- start start 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 Processor -- start 2024-09-05T06:57:15.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbecd40 0x557a2fbea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a2fbef910 con 0x557a2fbe1b60 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557a2fb01f10 con 0x557a2fbecd40 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a2fbefa90 con 0x557a2fbe2480 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f383a392740 10 client.0 _ll_register_callbacks cb 0x557a2facaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49736/0 (socket says 172.21.15.86:49736) 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f3835554640 1 -- 172.21.15.86:0/3344110266 learned_addr learned my addr 172.21.15.86:0/3344110266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f3835d55640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.727+0000 7f3835554640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbecd40 0x557a2fbea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 -- 172.21.15.86:0/3344110266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 msgr2=0x557a2fbe9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 -- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbecd40 msgr2=0x557a2fbea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbecd40 0x557a2fbea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 -- 172.21.15.86:0/3344110266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2f8b86a0 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835d55640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f3835554640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3820006010 tx=0x7f38200048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ff275904e4a14f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f38197fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820004310 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38200054d0 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.728+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f382000f790 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3820031050 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 10 client.19101 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a2fbfb450 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38200054d0 con 0x557a2fbe1b60 2024-09-05T06:57:15.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f3820003a00 con 0x557a2fbe1b60 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 10 client.19101 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 20 client.19101 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fbfebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 4 client.19101 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.729+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2fb547f0 con 0x557a2fbfc800 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.730+0000 7f3835d55640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fbfebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.730+0000 7f38197fa640 10 client.19101 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.730+0000 7f3835d55640 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fbfebf0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd37f670d553d3 server_cookie=44885eae97dff6d6 in_seq=0 out_seq=0 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f38197fa640 1 -- 172.21.15.86:0/3344110266 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a2fb547f0 con 0x557a2fbfc800 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f38197fa640 10 client.19101 handle_command_reply: tid=0 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f383a392740 1 client.19101 shutdown 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 msgr2=0x557a2fbec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3820006010 tx=0x7f38200048e0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 shutdown_connections 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.731+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fbfebf0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbecd40 0x557a2fbea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f38197fa640 0 client.19101 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 --2- 172.21.15.86:0/3344110266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbec7e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 >> 172.21.15.86:0/3344110266 conn(0x557a2fbde0f0 msgr2=0x557a2fbe3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 shutdown_connections 2024-09-05T06:57:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 1 -- 172.21.15.86:0/3344110266 wait complete. 2024-09-05T06:57:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.732+0000 7f383a392740 20 client.19101 trim_cache size 0 max 16384 2024-09-05T06:57:15.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20648 8.77 MB/sec execute 33 sec latency 100.445 ms 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.765+0000 7f383a392740 1 Processor -- start 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.765+0000 7f383a392740 1 -- start start 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fc043c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fc04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b56e0 con 0x557a2fbe1b60 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b7340 con 0x557a2fbe2480 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f383a392740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b5db0 con 0x557a2fbecd40 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49752/0 (socket says 172.21.15.86:49752) 2024-09-05T06:57:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 -- 172.21.15.86:0/1567302950 learned_addr learned my addr 172.21.15.86:0/1567302950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fc04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835d55640 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fc043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 -- 172.21.15.86:0/1567302950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 msgr2=0x557a2fc04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fc04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 -- 172.21.15.86:0/1567302950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 msgr2=0x557a2fc043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.766+0000 7f3835554640 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fc043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.767+0000 7f3835554640 1 -- 172.21.15.86:0/1567302950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2fbddf20 con 0x557a2fbe1b60 2024-09-05T06:57:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.767+0000 7f3835554640 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3820016040 tx=0x7f3820003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6653dd3e7ea1327 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.768+0000 7f381b7fe640 1 -- 172.21.15.86:0/1567302950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38200056e0 con 0x557a2fbe1b60 2024-09-05T06:57:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.768+0000 7f381b7fe640 1 -- 172.21.15.86:0/1567302950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3820017040 con 0x557a2fbe1b60 2024-09-05T06:57:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.768+0000 7f381b7fe640 1 -- 172.21.15.86:0/1567302950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820003b30 con 0x557a2fbe1b60 2024-09-05T06:57:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.769+0000 7f383a392740 1 -- 172.21.15.86:0/1567302950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 msgr2=0x557a2fbe8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.769+0000 7f383a392740 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3820016040 tx=0x7f3820003fc0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 -- 172.21.15.86:0/1567302950 shutdown_connections 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbecd40 0x557a2fc04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe2480 0x557a2fc043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 --2- 172.21.15.86:0/1567302950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbe1b60 0x557a2fbe8b10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 -- 172.21.15.86:0/1567302950 >> 172.21.15.86:0/1567302950 conn(0x557a2fbde0f0 msgr2=0x557a2fbe8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.770+0000 7f383a392740 1 -- 172.21.15.86:0/1567302950 shutdown_connections 2024-09-05T06:57:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.771+0000 7f383a392740 1 -- 172.21.15.86:0/1567302950 wait complete. 2024-09-05T06:57:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.771+0000 7f383a392740 1 -- start start 2024-09-05T06:57:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.771+0000 7f383a392740 1 Processor -- start 2024-09-05T06:57:15.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 0x557a2fbf55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbecd40 0x557a2fbe6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b6bb0 con 0x557a2fbecd40 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b64b0 con 0x557a2fbe1b60 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38301b36e0 con 0x557a2fbe2480 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f383a392740 10 client.0 _ll_register_callbacks cb 0x7f3820009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835554640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 0x557a2fbf55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835d55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835554640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 0x557a2fbf55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43658/0 (socket says 172.21.15.86:43658) 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835554640 1 -- 172.21.15.86:0/1863875418 learned_addr learned my addr 172.21.15.86:0/1863875418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835554640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbecd40 0x557a2fbe6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835d55640 1 -- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 msgr2=0x557a2fbf55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835d55640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 0x557a2fbf55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.772+0000 7f3835d55640 1 -- 172.21.15.86:0/1863875418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbecd40 msgr2=0x557a2fbe6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.773+0000 7f3835d55640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbecd40 0x557a2fbe6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.773+0000 7f3835d55640 1 -- 172.21.15.86:0/1863875418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2fbe77d0 con 0x557a2fbe2480 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.773+0000 7f3835d55640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe6990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x557a2fbff300 tx=0x7f382c013890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e680cdb08a97163 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.773+0000 7f38197fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c023070 con 0x557a2fbe2480 2024-09-05T06:57:15.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f382c004460 con 0x557a2fbe2480 2024-09-05T06:57:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3820017040 con 0x557a2fbe2480 2024-09-05T06:57:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f382c0035f0 con 0x557a2fbe2480 2024-09-05T06:57:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f383a392740 10 client.28030 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a2fc67560 con 0x557a2fbe2480 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.774+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3820017040 con 0x557a2fbe2480 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f382c01f8c0 con 0x557a2fbe2480 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 10 client.28030 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 10 client.28030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 20 client.28030 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fc69dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 4 client.28030 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a2fb4ed00 con 0x557a2fbfc800 2024-09-05T06:57:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.775+0000 7f3835554640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fc69dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.776+0000 7f3835554640 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fc69dc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed414f678086afeb server_cookie=8b2af81fb76631e6 in_seq=0 out_seq=0 2024-09-05T06:57:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.776+0000 7f38197fa640 10 client.28030 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f38197fa640 1 -- 172.21.15.86:0/1863875418 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a2fb4ed00 con 0x557a2fbfc800 2024-09-05T06:57:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f38197fa640 10 client.28030 handle_command_reply: tid=0 2024-09-05T06:57:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 client.28030 shutdown 2024-09-05T06:57:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 msgr2=0x557a2fbe6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe6990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x557a2fbff300 tx=0x7f382c013890 comp rx=0 tx=0).stop 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 shutdown_connections 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557a2fbfc800 0x557a2fc69dc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a2fbecd40 0x557a2fbe6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f38197fa640 0 client.28030 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a2fbe2480 0x557a2fbe6990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 --2- 172.21.15.86:0/1863875418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a2fbe1b60 0x557a2fbf55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.777+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 >> 172.21.15.86:0/1863875418 conn(0x557a2fbde0f0 msgr2=0x557a2fbe8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.778+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 shutdown_connections 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.778+0000 7f383a392740 1 -- 172.21.15.86:0/1863875418 wait complete. 2024-09-05T06:57:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.778+0000 7f383a392740 20 client.28030 trim_cache size 0 max 16384 2024-09-05T06:57:15.780 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:15.780 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:57:15.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "0a225a34-4421-4af7-8b34-7debeab6d8c8", 2024-09-05T06:57:15.780 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:57:15.780 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 msgr2=0x7f381c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 0x7f381c0a5230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f38280bc800 tx=0x7f38280095c0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 msgr2=0x7f383010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f3828030e00 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 shutdown_connections 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f381c0a2d70 0x7f381c0a5230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3830106a90 0x7f383010b980 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3830101430 0x7f38301a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 --2- 172.21.15.86:0/2208492724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3830100a90 0x7f38301a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 >> 172.21.15.86:0/2208492724 conn(0x7f38300fc7d0 msgr2=0x7f383010e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 shutdown_connections 2024-09-05T06:57:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:15.779+0000 7f3837fb9640 1 -- 172.21.15.86:0/2208492724 wait complete. 2024-09-05T06:57:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:16 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:16 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:16.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:16 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:16 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:16 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:16 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:16.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:16 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:16 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:16 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:16.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:57:16.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20787 8.68 MB/sec execute 34 sec latency 111.435 ms 2024-09-05T06:57:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.139+0000 7f197c01c640 1 -- 172.21.15.86:0/791683265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 msgr2=0x7f197410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.139+0000 7f197c01c640 1 --2- 172.21.15.86:0/791683265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f197410e610 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02a950 comp rx=0 tx=0).stop 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 -- 172.21.15.86:0/791683265 shutdown_connections 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 --2- 172.21.15.86:0/791683265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f197410e610 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 --2- 172.21.15.86:0/791683265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974103660 0x7f197410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 --2- 172.21.15.86:0/791683265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974102d40 0x7f1974103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 -- 172.21.15.86:0/791683265 >> 172.21.15.86:0/791683265 conn(0x7f19740fc7c0 msgr2=0x7f19740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 -- 172.21.15.86:0/791683265 shutdown_connections 2024-09-05T06:57:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.140+0000 7f197c01c640 1 -- 172.21.15.86:0/791683265 wait complete. 2024-09-05T06:57:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 Processor -- start 2024-09-05T06:57:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 -- start start 2024-09-05T06:57:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974103660 0x7f197410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1974111e70 con 0x7f197410c170 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1974111cf0 con 0x7f1974102d40 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.141+0000 7f197c01c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1974111ff0 con 0x7f1974103660 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f19795b7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974103660 0x7f197410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f197a5b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f197a5b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49772/0 (socket says 172.21.15.86:49772) 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:43676/0 (socket says 172.21.15.86:43676) 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f197a5b9640 1 -- 172.21.15.86:0/4280168868 learned_addr learned my addr 172.21.15.86:0/4280168868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 -- 172.21.15.86:0/4280168868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974103660 msgr2=0x7f197410b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974103660 0x7f197410b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 -- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 msgr2=0x7f19741a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f1979db8640 1 -- 172.21.15.86:0/4280168868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741a39f0 con 0x7f1974102d40 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.142+0000 7f197a5b9640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f1979db8640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9f0 tx=0x7f197000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f196affd640 1 -- 172.21.15.86:0/4280168868 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000ccc0 con 0x7f1974102d40 2024-09-05T06:57:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f196affd640 1 -- 172.21.15.86:0/4280168868 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1970005020 con 0x7f1974102d40 2024-09-05T06:57:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f196affd640 1 -- 172.21.15.86:0/4280168868 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970004a00 con 0x7f1974102d40 2024-09-05T06:57:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741a3ce0 con 0x7f1974102d40 2024-09-05T06:57:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.143+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741ab570 con 0x7f1974102d40 2024-09-05T06:57:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.146+0000 7f196affd640 1 -- 172.21.15.86:0/4280168868 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1970004ba0 con 0x7f1974102d40 2024-09-05T06:57:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.148+0000 7f197e3f5740 1 Processor -- start 2024-09-05T06:57:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.148+0000 7f197e3f5740 1 -- start start 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.148+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.148+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af70400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.148+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f197e3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56320aecb2a0 con 0x56320af70940 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f197e3f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56320ae72b10 con 0x56320af66080 2024-09-05T06:57:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f197e3f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56320aec7260 con 0x56320af65760 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f196affd640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 0x7f19600a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49786/0 (socket says 172.21.15.86:49786) 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 -- 172.21.15.86:0/3510173437 learned_addr learned my addr 172.21.15.86:0/3510173437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f196affd640 1 -- 172.21.15.86:0/4280168868 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f19700cf800 con 0x7f1974102d40 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f1979db8640 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af70400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f197a5b9640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 0x7f19600a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 -- 172.21.15.86:0/3510173437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 msgr2=0x56320af65b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af65b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 -- 172.21.15.86:0/3510173437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 msgr2=0x56320af70400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af70400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.149+0000 7f19795b7640 1 -- 172.21.15.86:0/3510173437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320ac3cb90 con 0x56320af70940 2024-09-05T06:57:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.150+0000 7f19795b7640 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1964009b90 tx=0x7f1964011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca7653c313407b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.150+0000 7f197a5b9640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 0x7f19600a52b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f196c009a60 tx=0x7f196c005f30 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.150+0000 7f195ffff640 1 -- 172.21.15.86:0/3510173437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964004020 con 0x56320af70940 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.150+0000 7f195ffff640 1 -- 172.21.15.86:0/3510173437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19640054d0 con 0x56320af70940 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.150+0000 7f195ffff640 1 -- 172.21.15.86:0/3510173437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964004c60 con 0x56320af70940 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 -- 172.21.15.86:0/3510173437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 msgr2=0x56320af72d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1964009b90 tx=0x7f1964011810 comp rx=0 tx=0).stop 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 -- 172.21.15.86:0/3510173437 shutdown_connections 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af72d30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af70400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3510173437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af65b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 -- 172.21.15.86:0/3510173437 >> 172.21.15.86:0/3510173437 conn(0x56320af61cf0 msgr2=0x56320af620d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.151+0000 7f197e3f5740 1 -- 172.21.15.86:0/3510173437 shutdown_connections 2024-09-05T06:57:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.152+0000 7f197e3f5740 1 -- 172.21.15.86:0/3510173437 wait complete. 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.152+0000 7f197e3f5740 1 -- start start 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.152+0000 7f197e3f5740 1 Processor -- start 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af6dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af6e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56320af73510 con 0x56320af65760 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56320ae536e0 con 0x56320af66080 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56320af73690 con 0x56320af70940 2024-09-05T06:57:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f1979db8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af6e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f1979db8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49800/0 (socket says 172.21.15.86:49800) 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f19795b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af6dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f1979db8640 1 -- 172.21.15.86:0/3249515622 learned_addr learned my addr 172.21.15.86:0/3249515622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197e3f5740 10 client.0 _ll_register_callbacks cb 0x56320ae77a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 -- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 msgr2=0x56320af6dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af6dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 -- 172.21.15.86:0/3249515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 msgr2=0x56320af6d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f197a5b9640 1 -- 172.21.15.86:0/3249515622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320af7ca30 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.153+0000 7f1979db8640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f197a5b9640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af6e290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098d0 tx=0x7f196c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd133bb41f1f3da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f195d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c034040 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f196c030910 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19640054d0 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f196c030b10 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f197e3f5740 10 client.28039 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.154+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56320af7f3a0 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19640054d0 con 0x56320af70940 2024-09-05T06:57:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f196c04f020 con 0x56320af70940 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 10 client.28039 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 10 client.28039 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 20 client.28039 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320af808b0 0x56320af82cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 4 client.28039 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320aed89d0 con 0x56320af808b0 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f1979db8640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320af808b0 0x56320af82cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f195d7fa640 10 client.28039 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.155+0000 7f1979db8640 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320af808b0 0x56320af82cf0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559132e9090750b8 server_cookie=c050c71c6d4037d9 in_seq=0 out_seq=0 2024-09-05T06:57:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.156+0000 7f195d7fa640 1 -- 172.21.15.86:0/3249515622 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56320aed89d0 con 0x56320af808b0 2024-09-05T06:57:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.156+0000 7f195d7fa640 10 client.28039 handle_command_reply: tid=0 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.156+0000 7f197e3f5740 1 client.28039 shutdown 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 msgr2=0x56320af6e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af6e290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098d0 tx=0x7f196c0095c0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 shutdown_connections 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320af808b0 0x56320af82cf0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af6e290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af6dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 --2- 172.21.15.86:0/3249515622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af65760 0x56320af6d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 >> 172.21.15.86:0/3249515622 conn(0x56320af61cf0 msgr2=0x56320af60e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f195d7fa640 0 client.28039 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 shutdown_connections 2024-09-05T06:57:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 1 -- 172.21.15.86:0/3249515622 wait complete. 2024-09-05T06:57:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.157+0000 7f197e3f5740 20 client.28039 trim_cache size 0 max 16384 2024-09-05T06:57:17.174 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.184+0000 7f197e3f5740 1 Processor -- start 2024-09-05T06:57:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.184+0000 7f197e3f5740 1 -- start start 2024-09-05T06:57:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af88b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b5730 con 0x56320af70940 2024-09-05T06:57:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b7390 con 0x56320af66080 2024-09-05T06:57:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197e3f5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b5e00 con 0x56320af65760 2024-09-05T06:57:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36732/0 (socket says 172.21.15.86:36732) 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 -- 172.21.15.86:0/2635918627 learned_addr learned my addr 172.21.15.86:0/2635918627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f19795b7640 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f197a5b9640 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 -- 172.21.15.86:0/2635918627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 msgr2=0x56320af885c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af885c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 -- 172.21.15.86:0/2635918627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 msgr2=0x56320af88b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af88b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 -- 172.21.15.86:0/2635918627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320af691c0 con 0x56320af65760 2024-09-05T06:57:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.185+0000 7f1979db8640 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1970098250 tx=0x7f1970016fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad74b8a8de56b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.186+0000 7f195f7fe640 1 -- 172.21.15.86:0/2635918627 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700b4070 con 0x56320af65760 2024-09-05T06:57:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.186+0000 7f195f7fe640 1 -- 172.21.15.86:0/2635918627 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19700a44f0 con 0x56320af65760 2024-09-05T06:57:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.186+0000 7f195f7fe640 1 -- 172.21.15.86:0/2635918627 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197001c040 con 0x56320af65760 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.186+0000 7f197e3f5740 1 -- 172.21.15.86:0/2635918627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 msgr2=0x56320af6f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.186+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1970098250 tx=0x7f1970016fb0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 -- 172.21.15.86:0/2635918627 shutdown_connections 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af70940 0x56320af88b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af66080 0x56320af885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2635918627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af65760 0x56320af6f780 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 -- 172.21.15.86:0/2635918627 >> 172.21.15.86:0/2635918627 conn(0x56320af61cf0 msgr2=0x56320af6f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 -- 172.21.15.86:0/2635918627 shutdown_connections 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 -- 172.21.15.86:0/2635918627 wait complete. 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 -- start start 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.187+0000 7f197e3f5740 1 Processor -- start 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 0x56320af749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f19795b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f19795b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49818/0 (socket says 172.21.15.86:49818) 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f19795b7640 1 -- 172.21.15.86:0/2458841631 learned_addr learned my addr 172.21.15.86:0/2458841631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f1979db8640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 0x56320af749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b6c00 con 0x56320af66080 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b6500 con 0x56320af65760 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b3730 con 0x56320af70940 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.188+0000 7f197e3f5740 10 client.0 _ll_register_callbacks cb 0x7f196c02fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f197a5b9640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 -- 172.21.15.86:0/2458841631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 msgr2=0x56320af75430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af75430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 -- 172.21.15.86:0/2458841631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 msgr2=0x56320af74ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320af65eb0 con 0x56320af65760 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f197a5b9640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:57:17.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f19795b7640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f1979db8640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 0x56320af749b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1970005480 tx=0x7f1970005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173be9635fbfbabf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f195d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700b4070 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f1970099b80 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19700a44f0 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f197001c040 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f197e3f5740 10 client.28064 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.189+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56320afeb920 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19700a44f0 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7f197000b7f0 con 0x56320af65760 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 10 client.28064 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:17.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 10 client.28064 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 20 client.28064 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320afec140 0x56320afee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 4 client.28064 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.6, "await_for": 45.0}] 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.6, "await_for": 45.0}) -- 0x56320aed2d20 con 0x56320afec140 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f19795b7640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320afec140 0x56320afee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f195d7fa640 10 client.28064 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.190+0000 7f19795b7640 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320afec140 0x56320afee560 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f213cea41c85f57 server_cookie=ef245228368dfeff in_seq=0 out_seq=0 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: pgmap v503: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.434 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:17 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: pgmap v503: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:57:17.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.448 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:17 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.469 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:17.469 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 38, 2024-09-05T06:57:17.469 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:57:17.469 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 58, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "7bd3f587": { 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 58, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 131.6, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:57:17.470 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.471 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f195d7fa640 1 -- 172.21.15.86:0/2458841631 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56320aed2d20 con 0x56320afec140 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f195d7fa640 10 client.28064 handle_command_reply: tid=0 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f197e3f5740 1 client.28064 shutdown 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 msgr2=0x56320af749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 0x56320af749b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1970005480 tx=0x7f1970005c50 comp rx=0 tx=0).stop 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 shutdown_connections 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.466+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56320afec140 0x56320afee560 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56320af70940 0x56320af75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56320af66080 0x56320af74ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 --2- 172.21.15.86:0/2458841631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56320af65760 0x56320af749b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 >> 172.21.15.86:0/2458841631 conn(0x56320af61cf0 msgr2=0x56320af6f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f195d7fa640 0 client.28064 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 shutdown_connections 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 1 -- 172.21.15.86:0/2458841631 wait complete. 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.467+0000 7f197e3f5740 20 client.28064 trim_cache size 0 max 16384 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.468+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 msgr2=0x7f19600a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.468+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 0x7f19600a52b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f196c009a60 tx=0x7f196c005f30 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 msgr2=0x7f197410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9f0 tx=0x7f197000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 shutdown_connections 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f19600a2df0 0x7f19600a52b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197410c170 0x7f19741a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1974103660 0x7f197410b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 --2- 172.21.15.86:0/4280168868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1974102d40 0x7f197410af10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.469+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 >> 172.21.15.86:0/4280168868 conn(0x7f19740fc7c0 msgr2=0x7f19740fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.470+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 shutdown_connections 2024-09-05T06:57:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.470+0000 7f197c01c640 1 -- 172.21.15.86:0/4280168868 wait complete. 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: pgmap v503: 129 pgs: 129 active+clean; 708 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:17 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:57:17.755 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20879 8.57 MB/sec execute 35 sec latency 567.610 ms 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 -- 172.21.15.86:0/14147693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106f10 msgr2=0x7fbeb010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/14147693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106f10 0x7fbeb010ea90 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009ca0 tx=0x7fbea802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 -- 172.21.15.86:0/14147693 shutdown_connections 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/14147693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106f10 0x7fbeb010ea90 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/14147693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0102790 0x7fbeb01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/14147693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0101de0 0x7fbeb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 -- 172.21.15.86:0/14147693 >> 172.21.15.86:0/14147693 conn(0x7fbeb00fd8c0 msgr2=0x7fbeb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 -- 172.21.15.86:0/14147693 shutdown_connections 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.892+0000 7fbeb72a9640 1 -- 172.21.15.86:0/14147693 wait complete. 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 Processor -- start 2024-09-05T06:57:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 -- start start 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 0x7fbeb0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0106f10 0x7fbeb019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0111e20 con 0x7fbeb0101de0 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0111ca0 con 0x7fbeb0102790 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb72a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0111fa0 con 0x7fbeb0106f10 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49844/0 (socket says 172.21.15.86:49844) 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 -- 172.21.15.86:0/2849997531 learned_addr learned my addr 172.21.15.86:0/2849997531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb4844640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 0x7fbeb0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5846640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0106f10 0x7fbeb019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 -- 172.21.15.86:0/2849997531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0106f10 msgr2=0x7fbeb019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0106f10 0x7fbeb019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 -- 172.21.15.86:0/2849997531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 msgr2=0x7fbeb0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 0x7fbeb0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 -- 172.21.15.86:0/2849997531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb019d2f0 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb4844640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 0x7fbeb0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.893+0000 7fbeb5045640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00d950 tx=0x7fbeac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.894+0000 7fbea67fc640 1 -- 172.21.15.86:0/2849997531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac017030 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.894+0000 7fbea67fc640 1 -- 172.21.15.86:0/2849997531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeac004020 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.894+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb019d5e0 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.894+0000 7fbea67fc640 1 -- 172.21.15.86:0/2849997531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0047a0 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.894+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01a4ec0 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.896+0000 7fbea67fc640 1 -- 172.21.15.86:0/2849997531 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbeac0041d0 con 0x7fbeb0101de0 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.896+0000 7fbeb9682740 1 Processor -- start 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.896+0000 7fbeb9682740 1 -- start start 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595e23242e0 con 0x5595e23bec80 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595e22cbce0 con 0x5595e23be360 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb9682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595e23202a0 con 0x5595e23cda90 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbea67fc640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 0x7fbe9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5846640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 0x7fbe9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49858/0 (socket says 172.21.15.86:49858) 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36766/0 (socket says 172.21.15.86:36766) 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 -- 172.21.15.86:0/3248393454 learned_addr learned my addr 172.21.15.86:0/3248393454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5846640 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbea67fc640 1 -- 172.21.15.86:0/2849997531 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fbeac0cf3d0 con 0x7fbeb0101de0 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 -- 172.21.15.86:0/3248393454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 msgr2=0x5595e23cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 -- 172.21.15.86:0/3248393454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 msgr2=0x5595e23be740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23be740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 -- 172.21.15.86:0/3248393454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e22b0440 con 0x5595e23bec80 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb4844640 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5846640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 0x7fbe9c0a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbea8031040 tx=0x7fbea8009820 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbeb5045640 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbeac097910 tx=0x7fbeac09b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9883cdb388969fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbe9b7fe640 1 -- 172.21.15.86:0/3248393454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac09bbb0 con 0x5595e23bec80 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.897+0000 7fbe9b7fe640 1 -- 172.21.15.86:0/3248393454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeac09b4b0 con 0x5595e23bec80 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbe9b7fe640 1 -- 172.21.15.86:0/3248393454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0a3c90 con 0x5595e23bec80 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- 172.21.15.86:0/3248393454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 msgr2=0x5595e23cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbeac097910 tx=0x7fbeac09b3b0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- 172.21.15.86:0/3248393454 shutdown_connections 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23cd550 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 --2- 172.21.15.86:0/3248393454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23be740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- 172.21.15.86:0/3248393454 >> 172.21.15.86:0/3248393454 conn(0x5595e23baa30 msgr2=0x5595e23bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- 172.21.15.86:0/3248393454 shutdown_connections 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- 172.21.15.86:0/3248393454 wait complete. 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.898+0000 7fbeb9682740 1 -- start start 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 Processor -- start 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595e23d0600 con 0x5595e23bec80 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595e22df050 con 0x5595e23be360 2024-09-05T06:57:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595e23d0780 con 0x5595e23cda90 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb9682740 10 client.0 _ll_register_callbacks cb 0x5595e231f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49866/0 (socket says 172.21.15.86:49866) 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 -- 172.21.15.86:0/262309875 learned_addr learned my addr 172.21.15.86:0/262309875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb5846640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb5045640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 -- 172.21.15.86:0/262309875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 msgr2=0x5595e23cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 -- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 msgr2=0x5595e23cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb4844640 1 -- 172.21.15.86:0/262309875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e20953c0 con 0x5595e23bec80 2024-09-05T06:57:17.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.899+0000 7fbeb5045640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb4844640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e70 tx=0x7fbea00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1365279bde829ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0012860 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fbea0012a00 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeac09b4b0 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbea0025050 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 10 client.19137 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5595e23dc130 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeac09b4b0 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fbea000f040 con 0x5595e23bec80 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 10 client.19137 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 10 client.19137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 20 client.19137 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e23dd640 0x5595e23dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 4 client.19137 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e2331850 con 0x5595e23dd640 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.900+0000 7fbeb5045640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e23dd640 0x5595e23dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.901+0000 7fbe98ff9640 10 client.19137 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.901+0000 7fbeb5045640 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e23dd640 0x5595e23dfaa0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22533a1702c93628 server_cookie=2039210f8c6f2155 in_seq=0 out_seq=0 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.901+0000 7fbe98ff9640 1 -- 172.21.15.86:0/262309875 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595e2331850 con 0x5595e23dd640 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.901+0000 7fbe98ff9640 10 client.19137 handle_command_reply: tid=0 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.901+0000 7fbeb9682740 1 client.19137 shutdown 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 msgr2=0x5595e23ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e70 tx=0x7fbea00117b0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 shutdown_connections 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e23dd640 0x5595e23dfaa0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23ca750 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbe98ff9640 0 client.19137 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 --2- 172.21.15.86:0/262309875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23be360 0x5595e23cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 >> 172.21.15.86:0/262309875 conn(0x5595e23baa30 msgr2=0x5595e23c0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 shutdown_connections 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 1 -- 172.21.15.86:0/262309875 wait complete. 2024-09-05T06:57:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.902+0000 7fbeb9682740 20 client.19137 trim_cache size 0 max 16384 2024-09-05T06:57:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 Processor -- start 2024-09-05T06:57:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 -- start start 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23be360 0x5595e23d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 0x5595e23e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ad030 con 0x5595e23bec80 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01aec90 con 0x5595e23cda90 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb9682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ad700 con 0x5595e23be360 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49870/0 (socket says 172.21.15.86:49870) 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 -- 172.21.15.86:0/601798059 learned_addr learned my addr 172.21.15.86:0/601798059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb5846640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 0x5595e23e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb5045640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23be360 0x5595e23d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 -- 172.21.15.86:0/601798059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23be360 msgr2=0x5595e23d98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23be360 0x5595e23d98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 -- 172.21.15.86:0/601798059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 msgr2=0x5595e23e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 0x5595e23e58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.919+0000 7fbeb4844640 1 -- 172.21.15.86:0/601798059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e23cd810 con 0x5595e23bec80 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.920+0000 7fbeb5846640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 0x5595e23e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.920+0000 7fbeb4844640 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aac61cdebe319bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.920+0000 7fbe9affd640 1 -- 172.21.15.86:0/601798059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea002d070 con 0x5595e23bec80 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.920+0000 7fbe9affd640 1 -- 172.21.15.86:0/601798059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea0009700 con 0x5595e23bec80 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.920+0000 7fbe9affd640 1 -- 172.21.15.86:0/601798059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea001ab30 con 0x5595e23bec80 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- 172.21.15.86:0/601798059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 msgr2=0x5595e23e53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0009ae0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- 172.21.15.86:0/601798059 shutdown_connections 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23cda90 0x5595e23e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23bec80 0x5595e23e53b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- 172.21.15.86:0/601798059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23be360 0x5595e23d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- 172.21.15.86:0/601798059 >> 172.21.15.86:0/601798059 conn(0x5595e23baa30 msgr2=0x5595e23cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- 172.21.15.86:0/601798059 shutdown_connections 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- 172.21.15.86:0/601798059 wait complete. 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- start start 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 Processor -- start 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 0x5595e23d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ae500 con 0x5595e23be360 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ade00 con 0x5595e23bec80 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.921+0000 7fbeb9682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ab030 con 0x5595e23cda90 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb9682740 10 client.0 _ll_register_callbacks cb 0x7fbea0009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49872/0 (socket says 172.21.15.86:49872) 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 -- 172.21.15.86:0/1805321618 learned_addr learned my addr 172.21.15.86:0/1805321618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5846640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb4844640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 0x5595e23d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 -- 172.21.15.86:0/1805321618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 msgr2=0x5595e23d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 -- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 msgr2=0x5595e23d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 0x5595e23d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 -- 172.21.15.86:0/1805321618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e23beab0 con 0x5595e23be360 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb4844640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 0x5595e23d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb5045640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbeac09b1b0 tx=0x7fbeac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a63d26935883250 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbe98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac09f860 con 0x5595e23be360 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fbeac0cc960 con 0x5595e23be360 2024-09-05T06:57:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0009700 con 0x5595e23be360 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbeac09bbb0 con 0x5595e23be360 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb9682740 10 client.19143 fetch_fsmap learned FSMap version 38 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5595e24484f0 con 0x5595e23be360 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.922+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea0009700 con 0x5595e23be360 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8394+0+0 (secure 0 0 0) 0x7fbeac0a3c90 con 0x5595e23be360 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 10 client.19143 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 10 client.19143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 20 client.19143 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e2448880 0x5595e244ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 4 client.19143 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595e232bd60 con 0x5595e2448880 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb4844640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e2448880 0x5595e244ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbe98ff9640 10 client.19143 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb4844640 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e2448880 0x5595e244ac70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f79694039f6dd45 server_cookie=baf8f37d709267c1 in_seq=0 out_seq=0 2024-09-05T06:57:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbe98ff9640 1 -- 172.21.15.86:0/1805321618 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fbea0037110 con 0x5595e2448880 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbe98ff9640 10 client.19143 handle_command_reply: tid=0 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.923+0000 7fbeb9682740 1 client.19143 shutdown 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 msgr2=0x5595e23d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbeac09b1b0 tx=0x7fbeac000bc0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 shutdown_connections 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5595e2448880 0x5595e244ac70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595e23cda90 0x5595e23d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e23bec80 0x5595e23d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbe98ff9640 0 client.19143 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 --2- 172.21.15.86:0/1805321618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e23be360 0x5595e23d17c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 >> 172.21.15.86:0/1805321618 conn(0x5595e23baa30 msgr2=0x5595e23c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 shutdown_connections 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 1 -- 172.21.15.86:0/1805321618 wait complete. 2024-09-05T06:57:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb9682740 20 client.19143 trim_cache size 0 max 16384 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "0a225a34-4421-4af7-8b34-7debeab6d8c8": { 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "0a225a34-4421-4af7-8b34-7debeab6d8c8", 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.937 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:17.938 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:17.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 msgr2=0x7fbe9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 0x7fbe9c0a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbea8031040 tx=0x7fbea8009820 comp rx=0 tx=0).stop 2024-09-05T06:57:17.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 msgr2=0x7fbeb0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:17.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.924+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00d950 tx=0x7fbeac00de10 comp rx=0 tx=0).stop 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 shutdown_connections 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbe9c0a2cf0 0x7fbe9c0a51b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb0106f10 0x7fbeb019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0102790 0x7fbeb0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 --2- 172.21.15.86:0/2849997531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0101de0 0x7fbeb0198340 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 >> 172.21.15.86:0/2849997531 conn(0x7fbeb00fd8c0 msgr2=0x7fbeb00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 shutdown_connections 2024-09-05T06:57:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:17.925+0000 7fbeb72a9640 1 -- 172.21.15.86:0/2849997531 wait complete. 2024-09-05T06:57:18.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20879 8.34 MB/sec execute 36 sec latency 1567.686 ms 2024-09-05T06:57:18.853 INFO:tasks.cephfs.filesystem:scrub status for tag:0a225a34-4421-4af7-8b34-7debeab6d8c8 - {'path': '/', 'tag': '0a225a34-4421-4af7-8b34-7debeab6d8c8', 'options': 'recursive,force'} 2024-09-05T06:57:19.303 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7bd3f587', quiesce took 14.1 seconds. Will release after: -2.5 2024-09-05T06:57:19.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:57:19.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:19 smithi115 ceph-mon[34791]: pgmap v504: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-09-05T06:57:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:19 smithi086 ceph-mon[30943]: pgmap v504: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-09-05T06:57:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:19 smithi096 ceph-mon[34468]: pgmap v504: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 230 op/s 2024-09-05T06:57:19.561 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:19.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20879 8.11 MB/sec execute 37 sec latency 2567.808 ms 2024-09-05T06:57:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 -- 172.21.15.86:0/1047538207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50106f10 msgr2=0x7f5f5010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/1047538207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50106f10 0x7f5f5010ea90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009ca0 tx=0x7f5f4802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 -- 172.21.15.86:0/1047538207 shutdown_connections 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/1047538207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50106f10 0x7f5f5010ea90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/1047538207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50102790 0x7f5f501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/1047538207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 0x7f5f501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 -- 172.21.15.86:0/1047538207 >> 172.21.15.86:0/1047538207 conn(0x7f5f500fd8c0 msgr2=0x7f5f500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 -- 172.21.15.86:0/1047538207 shutdown_connections 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 -- 172.21.15.86:0/1047538207 wait complete. 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.288+0000 7f5f56ff5640 1 Processor -- start 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 -- start start 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 0x7f5f50198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50106f10 0x7f5f5019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111e40 con 0x7f5f50102790 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111cc0 con 0x7f5f50106f10 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f56ff5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111fc0 con 0x7f5f50101de0 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49894/0 (socket says 172.21.15.86:49894) 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 -- 172.21.15.86:0/2458925580 learned_addr learned my addr 172.21.15.86:0/2458925580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f55592640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50106f10 0x7f5f5019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f54d91640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 0x7f5f50198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 -- 172.21.15.86:0/2458925580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 msgr2=0x7f5f50198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 0x7f5f50198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 -- 172.21.15.86:0/2458925580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50106f10 msgr2=0x7f5f5019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50106f10 0x7f5f5019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 -- 172.21.15.86:0/2458925580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5019d3b0 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4ffff640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00d950 tx=0x7f5f3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.289+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c017070 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.290+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f3c009c30 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.290+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5019d6a0 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.290+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00f620 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.290+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a4f80 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.291+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f501021c0 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.292+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f3c0109a0 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.292+0000 7f5f4dffb640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 0x7f5f400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.292+0000 7f5f54d91640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 0x7f5f400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.292+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f5f3c0cf4e0 con 0x7f5f50102790 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.293+0000 7f5f54d91640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 0x7f5f400a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009890 tx=0x7f5f44008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:20.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.297+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f3c009de0 con 0x7f5f50102790 2024-09-05T06:57:20.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.502+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f5010a0f0 con 0x7f5f50102790 2024-09-05T06:57:20.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.503+0000 7f5f4dffb640 1 -- 172.21.15.86:0/2458925580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11985 (secure 0 0 0) 0x7f5f3c09bcc0 con 0x7f5f50102790 2024-09-05T06:57:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:57:20.504 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":39,"btime":"2024-09-05T06:57:20:169069+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:19.179433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:57:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 39 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 msgr2=0x7f5f400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 0x7f5f400a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009890 tx=0x7f5f44008040 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 msgr2=0x7f5f50198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00d950 tx=0x7f5f3c00de10 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 shutdown_connections 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f400a2cf0 0x7f5f400a51b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f50106f10 0x7f5f5019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50102790 0x7f5f50198940 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 --2- 172.21.15.86:0/2458925580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f50101de0 0x7f5f50198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 >> 172.21.15.86:0/2458925580 conn(0x7f5f500fd8c0 msgr2=0x7f5f500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 shutdown_connections 2024-09-05T06:57:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:20.505+0000 7f5f56ff5640 1 -- 172.21.15.86:0/2458925580 wait complete. 2024-09-05T06:57:20.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20879 7.90 MB/sec execute 38 sec latency 3567.851 ms 2024-09-05T06:57:21.225 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 7bd3f587 --release --await' 2024-09-05T06:57:21.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 7bd3f587 --release --await 2024-09-05T06:57:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:21 smithi115 ceph-mon[34791]: pgmap v505: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-09-05T06:57:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:21 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:21.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:21 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2458925580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:21 smithi086 ceph-mon[30943]: pgmap v505: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-09-05T06:57:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:21 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:21 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2458925580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:21 smithi096 ceph-mon[34468]: pgmap v505: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2024-09-05T06:57:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:21 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:21 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2458925580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:21.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20879 7.69 MB/sec execute 39 sec latency 4567.881 ms 2024-09-05T06:57:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- 172.21.15.86:0/2457791546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c106b70 msgr2=0x7fd63c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 --2- 172.21.15.86:0/2457791546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c106b70 0x7fd63c106f50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd638009ca0 tx=0x7fd63802a8c0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- 172.21.15.86:0/2457791546 shutdown_connections 2024-09-05T06:57:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 --2- 172.21.15.86:0/2457791546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100a50 0x7fd63c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 --2- 172.21.15.86:0/2457791546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c100090 0x7fd63c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 --2- 172.21.15.86:0/2457791546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c106b70 0x7fd63c106f50 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- 172.21.15.86:0/2457791546 >> 172.21.15.86:0/2457791546 conn(0x7fd63c077210 msgr2=0x7fd63c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- 172.21.15.86:0/2457791546 shutdown_connections 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- 172.21.15.86:0/2457791546 wait complete. 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 Processor -- start 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.988+0000 7fd6434fb640 1 -- start start 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100090 0x7fd63c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c106b70 0x7fd63c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c118470 con 0x7fd63c100a50 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1182f0 con 0x7fd63c106b70 2024-09-05T06:57:22.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd6434fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1185f0 con 0x7fd63c100090 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49918/0 (socket says 172.21.15.86:49918) 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 -- 172.21.15.86:0/3633328210 learned_addr learned my addr 172.21.15.86:0/3633328210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd641297640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100090 0x7fd63c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd641a98640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c106b70 0x7fd63c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 -- 172.21.15.86:0/3633328210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100090 msgr2=0x7fd63c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100090 0x7fd63c1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 -- 172.21.15.86:0/3633328210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c106b70 msgr2=0x7fd63c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c106b70 0x7fd63c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 -- 172.21.15.86:0/3633328210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c19ef80 con 0x7fd63c100a50 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.989+0000 7fd640a96640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00d950 tx=0x7fd62c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.990+0000 7fd6327fc640 1 -- 172.21.15.86:0/3633328210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c017030 con 0x7fd63c100a50 2024-09-05T06:57:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.990+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c19f270 con 0x7fd63c100a50 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.990+0000 7fd6327fc640 1 -- 172.21.15.86:0/3633328210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62c004020 con 0x7fd63c100a50 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.990+0000 7fd6327fc640 1 -- 172.21.15.86:0/3633328210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0047a0 con 0x7fd63c100a50 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.990+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1ab4c0 con 0x7fd63c100a50 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.992+0000 7fd6327fc640 1 -- 172.21.15.86:0/3633328210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd62c0041d0 con 0x7fd63c100a50 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.992+0000 7fd6458d4740 1 Processor -- start 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.992+0000 7fd6458d4740 1 -- start start 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x565267982440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 0x56526798cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652678e7620 con 0x56526798d240 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56526788ee50 con 0x565267982060 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6458d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5652678e35e0 con 0x565267982980 2024-09-05T06:57:22.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6327fc640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 0x7fd6280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49932/0 (socket says 172.21.15.86:49932) 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd640a96640 1 -- 172.21.15.86:0/3845087802 learned_addr learned my addr 172.21.15.86:0/3845087802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 0x56526798cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641a98640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 0x7fd6280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641a98640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x565267982440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd6327fc640 1 -- 172.21.15.86:0/3633328210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fd62c0cf2c0 con 0x7fd63c100a50 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 -- 172.21.15.86:0/3845087802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 msgr2=0x565267982440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x565267982440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 -- 172.21.15.86:0/3845087802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 msgr2=0x56526798f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641297640 1 -- 172.21.15.86:0/3845087802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526787fd40 con 0x565267982980 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd640a96640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.993+0000 7fd641a98640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 0x7fd6280a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd634007d10 tx=0x7fd634007c60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd641297640 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 0x56526798cd00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd638000f80 tx=0x7fd638009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3073db314ea4c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd6277fe640 1 -- 172.21.15.86:0/3845087802 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638031450 con 0x565267982980 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd6277fe640 1 -- 172.21.15.86:0/3845087802 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd638036030 con 0x565267982980 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd6277fe640 1 -- 172.21.15.86:0/3845087802 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638004500 con 0x565267982980 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd6458d4740 1 -- 172.21.15.86:0/3845087802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 msgr2=0x56526798cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.994+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 0x56526798cd00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd638000f80 tx=0x7fd638009a20 comp rx=0 tx=0).stop 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- 172.21.15.86:0/3845087802 shutdown_connections 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x56526798f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982980 0x56526798cd00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3845087802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x565267982440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- 172.21.15.86:0/3845087802 >> 172.21.15.86:0/3845087802 conn(0x56526797e5f0 msgr2=0x56526797e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- 172.21.15.86:0/3845087802 shutdown_connections 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- 172.21.15.86:0/3845087802 wait complete. 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- start start 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 Processor -- start 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x56526798a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56526798d240 0x56526798ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56526798fc90 con 0x565267982980 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56526786ff10 con 0x565267982060 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56526798fe10 con 0x56526798d240 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd6458d4740 10 client.0 _ll_register_callbacks cb 0x565267893db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.995+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49940/0 (socket says 172.21.15.86:49940) 2024-09-05T06:57:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 -- 172.21.15.86:0/497327506 learned_addr learned my addr 172.21.15.86:0/497327506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd641a98640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56526798d240 0x56526798ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd641297640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x56526798a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 -- 172.21.15.86:0/497327506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56526798d240 msgr2=0x56526798ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56526798d240 0x56526798ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 -- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 msgr2=0x56526798a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x56526798a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 -- 172.21.15.86:0/497327506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652675f6580 con 0x565267982980 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd640a96640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009610 tx=0x7fd62c09b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807bfbf515c78328 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c09baa0 con 0x565267982980 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fd62c09a7f0 con 0x565267982980 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd638036030 con 0x565267982980 2024-09-05T06:57:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd62c0b1e60 con 0x565267982980 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd6458d4740 10 client.19161 fetch_fsmap learned FSMap version 39 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.996+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5652676c0960 con 0x565267982980 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd638036030 con 0x565267982980 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8398+0+0 (secure 0 0 0) 0x7fd62c0b4140 con 0x565267982980 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 10 client.19161 fetch_fsmap finished waiting for FSMap version 39 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 10 client.19161 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 20 client.19161 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56526799cc40 0x56526799f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 4 client.19161 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652678f4b90 con 0x56526799cc40 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd641297640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56526799cc40 0x56526799f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd624ff9640 10 client.19161 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.997+0000 7fd641297640 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56526799cc40 0x56526799f060 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3063c6d30bca774 server_cookie=ef7aedc82139db98 in_seq=0 out_seq=0 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd624ff9640 1 -- 172.21.15.86:0/497327506 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652678f4b90 con 0x56526799cc40 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd624ff9640 10 client.19161 handle_command_reply: tid=0 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 client.19161 shutdown 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 msgr2=0x56526798a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009610 tx=0x7fd62c09b270 comp rx=0 tx=0).stop 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 shutdown_connections 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56526799cc40 0x56526799f060 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56526798d240 0x56526798ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x56526798a610 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd624ff9640 0 client.19161 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 --2- 172.21.15.86:0/497327506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982060 0x56526798a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 >> 172.21.15.86:0/497327506 conn(0x56526797e5f0 msgr2=0x5652679842d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 shutdown_connections 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.998+0000 7fd6458d4740 1 -- 172.21.15.86:0/497327506 wait complete. 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:21.999+0000 7fd6458d4740 20 client.19161 trim_cache size 0 max 16384 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 Processor -- start 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 -- start start 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x56526798bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982980 0x5652679a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b5720 con 0x56526798d240 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b73b0 con 0x565267982980 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd6458d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b5df0 con 0x565267982060 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd641a98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd641a98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49950/0 (socket says 172.21.15.86:49950) 2024-09-05T06:57:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd641a98640 1 -- 172.21.15.86:0/3927979118 learned_addr learned my addr 172.21.15.86:0/3927979118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.015+0000 7fd640a96640 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982980 0x5652679a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641297640 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x56526798bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 -- 172.21.15.86:0/3927979118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 msgr2=0x56526798bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x56526798bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 -- 172.21.15.86:0/3927979118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982980 msgr2=0x5652679a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982980 0x5652679a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 -- 172.21.15.86:0/3927979118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526797e420 con 0x56526798d240 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd641a98640 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd634001be0 tx=0x7fd634011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6430bf5814934 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd626ffd640 1 -- 172.21.15.86:0/3927979118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63402a070 con 0x56526798d240 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd626ffd640 1 -- 172.21.15.86:0/3927979118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd63401ae50 con 0x56526798d240 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.016+0000 7fd626ffd640 1 -- 172.21.15.86:0/3927979118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634010040 con 0x56526798d240 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- 172.21.15.86:0/3927979118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 msgr2=0x5652679a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd634001be0 tx=0x7fd634011000 comp rx=0 tx=0).stop 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- 172.21.15.86:0/3927979118 shutdown_connections 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526798d240 0x5652679a4eb0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565267982980 0x5652679a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 --2- 172.21.15.86:0/3927979118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x56526798bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- 172.21.15.86:0/3927979118 >> 172.21.15.86:0/3927979118 conn(0x56526797e5f0 msgr2=0x56526797d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- 172.21.15.86:0/3927979118 shutdown_connections 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- 172.21.15.86:0/3927979118 wait complete. 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 -- start start 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.017+0000 7fd6458d4740 1 Processor -- start 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x5652679958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56526798d240 0x5652679921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b6c20 con 0x565267982980 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b6520 con 0x56526798d240 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1b3720 con 0x565267982060 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd6458d4740 10 client.0 _ll_register_callbacks cb 0x7fd62c0050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49964/0 (socket says 172.21.15.86:49964) 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 -- 172.21.15.86:0/4266856452 learned_addr learned my addr 172.21.15.86:0/4266856452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd641a98640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56526798d240 0x5652679921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd641297640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x5652679958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 -- 172.21.15.86:0/4266856452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 msgr2=0x5652679958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x5652679958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 -- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56526798d240 msgr2=0x5652679921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56526798d240 0x5652679921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 -- 172.21.15.86:0/4266856452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565267984ed0 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd641297640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x5652679958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd640a96640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005d80 tx=0x7fd62c00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62de79a36c8f3fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.018+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c09baa0 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fd62c097190 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63401ae50 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd62c09b3a0 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 10 client.19167 fetch_fsmap learned FSMap version 39 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565267a08880 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd63401ae50 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8398+0+0 (secure 0 0 0) 0x7fd62c09f1b0 con 0x565267982980 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 10 client.19167 fetch_fsmap finished waiting for FSMap version 39 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 10 client.19167 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:22.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 20 client.19167 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565267a08ef0 0x565267a0b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 4 client.19167 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7bd3f587", "release": true, "await": true}] 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7bd3f587", "release": true, "await": true}) -- 0x5652678ef0a0 con 0x565267a08ef0 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.019+0000 7fd641297640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565267a08ef0 0x565267a0b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.020+0000 7fd624ff9640 10 client.19167 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:22.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.020+0000 7fd641297640 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565267a08ef0 0x565267a0b310 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d751d61f1fda91 server_cookie=59d3fcad71953564 in_seq=0 out_seq=0 2024-09-05T06:57:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.055+0000 7fd624ff9640 1 -- 172.21.15.86:0/4266856452 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5652678ef0a0 con 0x565267a08ef0 2024-09-05T06:57:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.055+0000 7fd624ff9640 10 client.19167 handle_command_reply: tid=0 2024-09-05T06:57:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.055+0000 7fd6458d4740 1 client.19167 shutdown 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.055+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 msgr2=0x565267991c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.055+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005d80 tx=0x7fd62c00df40 comp rx=0 tx=0).stop 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 shutdown_connections 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x565267a08ef0 0x565267a0b310 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56526798d240 0x5652679921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267982980 0x565267991c40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 --2- 172.21.15.86:0/4266856452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565267982060 0x5652679958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 >> 172.21.15.86:0/4266856452 conn(0x56526797e5f0 msgr2=0x56526797cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd624ff9640 0 client.19167 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 shutdown_connections 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 1 -- 172.21.15.86:0/4266856452 wait complete. 2024-09-05T06:57:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.056+0000 7fd6458d4740 20 client.19167 trim_cache size 0 max 16384 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 39, 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 60, 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "7bd3f587": { 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 60, 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:57:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 131.6, 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:22.059 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 msgr2=0x7fd6280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 0x7fd6280a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd634007d10 tx=0x7fd634007c60 comp rx=0 tx=0).stop 2024-09-05T06:57:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 msgr2=0x7fd63c1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00d950 tx=0x7fd62c00de10 comp rx=0 tx=0).stop 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 shutdown_connections 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6280a2cf0 0x7fd6280a51b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd63c106b70 0x7fd63c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c100a50 0x7fd63c1a4ba0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 --2- 172.21.15.86:0/3633328210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd63c100090 0x7fd63c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 >> 172.21.15.86:0/3633328210 conn(0x7fd63c077210 msgr2=0x7fd63c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 shutdown_connections 2024-09-05T06:57:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:22.057+0000 7fd6434fb640 1 -- 172.21.15.86:0/3633328210 wait complete. 2024-09-05T06:57:22.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21025 7.80 MB/sec execute 40 sec latency 4862.156 ms 2024-09-05T06:57:22.784 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7bd3f587', total seconds elapsed: 17.6 2024-09-05T06:57:23.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:23 smithi115 ceph-mon[34791]: pgmap v506: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-09-05T06:57:23.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:23 smithi086 ceph-mon[30943]: pgmap v506: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-09-05T06:57:23.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:23 smithi096 ceph-mon[34468]: pgmap v506: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-09-05T06:57:23.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21226 7.89 MB/sec execute 41 sec latency 100.898 ms 2024-09-05T06:57:24.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21390 7.87 MB/sec execute 42 sec latency 229.084 ms 2024-09-05T06:57:25.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:25 smithi115 ceph-mon[34791]: pgmap v507: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:25 smithi086 ceph-mon[30943]: pgmap v507: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:25.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:25 smithi096 ceph-mon[34468]: pgmap v507: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:25.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21561 7.82 MB/sec execute 43 sec latency 61.564 ms 2024-09-05T06:57:26.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21752 7.93 MB/sec execute 44 sec latency 68.244 ms 2024-09-05T06:57:27.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:27 smithi115 ceph-mon[34791]: pgmap v508: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:27 smithi086 ceph-mon[30943]: pgmap v508: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:27 smithi096 ceph-mon[34468]: pgmap v508: 129 pgs: 129 active+clean; 723 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-09-05T06:57:27.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21931 7.89 MB/sec execute 45 sec latency 72.429 ms 2024-09-05T06:57:28.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22108 7.97 MB/sec execute 46 sec latency 72.414 ms 2024-09-05T06:57:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:29 smithi086 ceph-mon[30943]: pgmap v509: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-05T06:57:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:29 smithi096 ceph-mon[34468]: pgmap v509: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-05T06:57:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:29 smithi115 ceph-mon[34791]: pgmap v509: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-05T06:57:29.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22276 7.99 MB/sec execute 47 sec latency 137.107 ms 2024-09-05T06:57:30.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22487 8.19 MB/sec execute 48 sec latency 65.065 ms 2024-09-05T06:57:31.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:31 smithi115 ceph-mon[34791]: pgmap v510: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-09-05T06:57:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:57:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:57:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:31 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:57:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:31 smithi086 ceph-mon[30943]: pgmap v510: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-09-05T06:57:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:57:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:57:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:31 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:57:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:31 smithi096 ceph-mon[34468]: pgmap v510: 129 pgs: 129 active+clean; 736 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 154 op/s 2024-09-05T06:57:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:57:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:57:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:31 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:57:31.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22672 8.14 MB/sec execute 49 sec latency 77.844 ms 2024-09-05T06:57:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:32 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:57:32.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:32 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:57:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:32 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:57:32.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:32 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:57:32.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:32 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:57:32.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:32 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:57:32.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22834 8.06 MB/sec execute 50 sec latency 235.576 ms 2024-09-05T06:57:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:33 smithi086 ceph-mon[30943]: pgmap v511: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-09-05T06:57:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:33 smithi096 ceph-mon[34468]: pgmap v511: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-09-05T06:57:33.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:33 smithi115 ceph-mon[34791]: pgmap v511: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 287 op/s 2024-09-05T06:57:33.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23011 8.07 MB/sec execute 51 sec latency 249.805 ms 2024-09-05T06:57:34.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23198 8.09 MB/sec execute 52 sec latency 367.083 ms 2024-09-05T06:57:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:35 smithi086 ceph-mon[30943]: pgmap v512: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:35 smithi096 ceph-mon[34468]: pgmap v512: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:35 smithi115 ceph-mon[34791]: pgmap v512: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:35.756 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23379 8.16 MB/sec execute 53 sec latency 142.394 ms 2024-09-05T06:57:36.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23546 8.19 MB/sec execute 54 sec latency 152.453 ms 2024-09-05T06:57:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:37 smithi086 ceph-mon[30943]: pgmap v513: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:37 smithi096 ceph-mon[34468]: pgmap v513: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:37.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:37 smithi115 ceph-mon[34791]: pgmap v513: 129 pgs: 129 active+clean; 741 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-05T06:57:37.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23783 8.38 MB/sec execute 55 sec latency 73.480 ms 2024-09-05T06:57:38.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:38 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:38.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:38 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:38.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:38 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:38.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23979 8.38 MB/sec execute 56 sec latency 72.702 ms 2024-09-05T06:57:39.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:39 smithi086 ceph-mon[30943]: pgmap v514: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 351 op/s 2024-09-05T06:57:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:39 smithi096 ceph-mon[34468]: pgmap v514: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 351 op/s 2024-09-05T06:57:39.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:39 smithi115 ceph-mon[34791]: pgmap v514: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.8 MiB/s wr, 351 op/s 2024-09-05T06:57:39.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24103 8.30 MB/sec execute 57 sec latency 72.035 ms 2024-09-05T06:57:40.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24258 8.38 MB/sec execute 58 sec latency 66.370 ms 2024-09-05T06:57:41.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:41 smithi086 ceph-mon[30943]: pgmap v515: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-05T06:57:41.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:41 smithi096 ceph-mon[34468]: pgmap v515: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-05T06:57:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:41 smithi115 ceph-mon[34791]: pgmap v515: 129 pgs: 129 active+clean; 751 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-05T06:57:41.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24443 8.42 MB/sec execute 59 sec latency 57.080 ms 2024-09-05T06:57:42.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:42 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:42.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:42 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:42.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:42 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:42.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24625 8.50 MB/sec execute 60 sec latency 57.063 ms 2024-09-05T06:57:43.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:43 smithi086 ceph-mon[30943]: pgmap v516: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 376 op/s 2024-09-05T06:57:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:43 smithi096 ceph-mon[34468]: pgmap v516: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 376 op/s 2024-09-05T06:57:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:43 smithi115 ceph-mon[34791]: pgmap v516: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.0 MiB/s wr, 376 op/s 2024-09-05T06:57:43.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24792 8.44 MB/sec execute 61 sec latency 58.440 ms 2024-09-05T06:57:44.328 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T06:57:44.328 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T06:57:44.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:57:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:44.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24914 8.36 MB/sec execute 62 sec latency 213.649 ms 2024-09-05T06:57:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:45 smithi086 ceph-mon[30943]: pgmap v517: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:45.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:45 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:57:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:45 smithi096 ceph-mon[34468]: pgmap v517: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:45 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:57:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:45 smithi115 ceph-mon[34791]: pgmap v517: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:45 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:57:45.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25062 8.32 MB/sec execute 63 sec latency 143.367 ms 2024-09-05T06:57:46.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25237 8.30 MB/sec execute 64 sec latency 111.553 ms 2024-09-05T06:57:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:47 smithi086 ceph-mon[30943]: pgmap v518: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:47.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:47 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:47 smithi096 ceph-mon[34468]: pgmap v518: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:47 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:47.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:47 smithi115 ceph-mon[34791]: pgmap v518: 129 pgs: 129 active+clean; 759 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-05T06:57:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:47 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:47.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25401 8.30 MB/sec execute 65 sec latency 122.699 ms 2024-09-05T06:57:48.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25523 8.29 MB/sec execute 66 sec latency 155.917 ms 2024-09-05T06:57:48.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:57:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 -- 172.21.15.86:0/3857161062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 msgr2=0x7fa354101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 --2- 172.21.15.86:0/3857161062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa354101200 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa35002a480 comp rx=0 tx=0).stop 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 -- 172.21.15.86:0/3857161062 shutdown_connections 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 --2- 172.21.15.86:0/3857161062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 0x7fa35410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 --2- 172.21.15.86:0/3857161062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 --2- 172.21.15.86:0/3857161062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa354101200 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 -- 172.21.15.86:0/3857161062 >> 172.21.15.86:0/3857161062 conn(0x7fa3540fc7d0 msgr2=0x7fa3540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.114+0000 7fa35c677640 1 -- 172.21.15.86:0/3857161062 shutdown_connections 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.115+0000 7fa35c677640 1 -- 172.21.15.86:0/3857161062 wait complete. 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.115+0000 7fa35c677640 1 Processor -- start 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.115+0000 7fa35c677640 1 -- start start 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa3541a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 0x7fa35410cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa354111f50 con 0x7fa354100e20 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa354111dd0 con 0x7fa354102ef0 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa359c12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa359c12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:40608/0 (socket says 172.21.15.86:40608) 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa359c12640 1 -- 172.21.15.86:0/1024381060 learned_addr learned my addr 172.21.15.86:0/1024381060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35a413640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa3541a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35ac14640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 0x7fa35410cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.116+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541120d0 con 0x7fa354102570 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 -- 172.21.15.86:0/1024381060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 msgr2=0x7fa3541a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 -- 172.21.15.86:0/1024381060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 msgr2=0x7fa35410cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 0x7fa35410cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"btime":"2024-09-05T06:57:46:228427+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:45.409020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35410d2f0 con 0x7fa354100e20 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35a413640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa3541a3640 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa35002aa70 tx=0x7fa350003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35003f070 con 0x7fa354100e20 2024-09-05T06:57:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa35003bb10 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35410d580 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350035030 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.117+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541ab570 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.119+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354101200 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.120+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa350042070 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.121+0000 7fa34b7fe640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 0x7fa3400a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.121+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fa3500f02c0 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.121+0000 7fa359c12640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 0x7fa3400a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.122+0000 7fa359c12640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 0x7fa3400a5280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3440095f0 tx=0x7fa344005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.125+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa350049030 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.390+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3540fe2c0 con 0x7fa354100e20 2024-09-05T06:57:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.390+0000 7fa34b7fe640 1 -- 172.21.15.86:0/1024381060 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11978 (secure 0 0 0) 0x7fa3500bc450 con 0x7fa354100e20 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 42 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 msgr2=0x7fa3400a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 0x7fa3400a5280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3440095f0 tx=0x7fa344005f90 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 msgr2=0x7fa3541a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa3541a3640 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa35002aa70 tx=0x7fa350003b90 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 shutdown_connections 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa3400a2dc0 0x7fa3400a5280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa354102ef0 0x7fa35410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa354102570 0x7fa3541a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 --2- 172.21.15.86:0/1024381060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100e20 0x7fa3541a3640 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.393+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 >> 172.21.15.86:0/1024381060 conn(0x7fa3540fc7d0 msgr2=0x7fa354105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.394+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 shutdown_connections 2024-09-05T06:57:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:48.394+0000 7fa35c677640 1 -- 172.21.15.86:0/1024381060 wait complete. 2024-09-05T06:57:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:49 smithi096 ceph-mon[34468]: pgmap v519: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-05T06:57:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:49 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1024381060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:49 smithi115 ceph-mon[34791]: pgmap v519: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-05T06:57:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:49 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1024381060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:49 smithi086 ceph-mon[30943]: pgmap v519: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-05T06:57:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:49 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1024381060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:49.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25653 8.24 MB/sec execute 67 sec latency 163.047 ms 2024-09-05T06:57:50.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25848 8.38 MB/sec execute 68 sec latency 89.299 ms 2024-09-05T06:57:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:51 smithi096 ceph-mon[34468]: pgmap v520: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-05T06:57:51.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:51 smithi115 ceph-mon[34791]: pgmap v520: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-05T06:57:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:51 smithi086 ceph-mon[30943]: pgmap v520: 129 pgs: 129 active+clean; 769 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 231 op/s 2024-09-05T06:57:51.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25979 8.36 MB/sec execute 69 sec latency 181.513 ms 2024-09-05T06:57:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:52 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:52 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:57:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:52 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:57:52.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:52 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:57:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:52 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:52.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:52 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:57:52.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:52 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:57:52.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:52 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:57:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:52 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:57:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:52 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:57:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:52 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:57:52.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:52 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:57:52.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26124 8.33 MB/sec execute 70 sec latency 214.413 ms 2024-09-05T06:57:53.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:53 smithi115 ceph-mon[34791]: pgmap v521: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 344 op/s 2024-09-05T06:57:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:53 smithi086 ceph-mon[30943]: pgmap v521: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 344 op/s 2024-09-05T06:57:53.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26254 8.26 MB/sec execute 71 sec latency 139.930 ms 2024-09-05T06:57:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:53 smithi096 ceph-mon[34468]: pgmap v521: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 344 op/s 2024-09-05T06:57:54.704 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T06:57:54.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T06:57:54.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26425 8.24 MB/sec execute 72 sec latency 104.382 ms 2024-09-05T06:57:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:55.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2892065051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 msgr2=0x7fc9d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2892065051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d01067e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009900 tx=0x7fc9c002a410 comp rx=0 tx=0).stop 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2892065051 shutdown_connections 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2892065051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2892065051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d01067e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2892065051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2892065051 >> 172.21.15.86:0/2892065051 conn(0x7fc9d00fd880 msgr2=0x7fc9d00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2892065051 shutdown_connections 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.322+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2892065051 wait complete. 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 Processor -- start 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 -- start start 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111e20 con 0x7fc9d0102790 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111ca0 con 0x7fc9d0101de0 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d7c11640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111fa0 con 0x7fc9d0106f10 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d51ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45516/0 (socket says 172.21.15.86:45516) 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d59ad640 1 -- 172.21.15.86:0/2868545039 learned_addr learned my addr 172.21.15.86:0/2868545039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.323+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 -- 172.21.15.86:0/2868545039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 msgr2=0x7fc9d019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 -- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 msgr2=0x7fc9d0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 -- 172.21.15.86:0/2868545039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d019d2d0 con 0x7fc9d0101de0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00e9f0 tx=0x7fc9cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9c6ffd640 1 -- 172.21.15.86:0/2868545039 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00cd80 con 0x7fc9d0101de0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9c6ffd640 1 -- 172.21.15.86:0/2868545039 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9cc0108e0 con 0x7fc9d0101de0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d019d5c0 con 0x7fc9d0101de0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d0074420 con 0x7fc9d0101de0 2024-09-05T06:57:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.324+0000 7fc9c6ffd640 1 -- 172.21.15.86:0/2868545039 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc018920 con 0x7fc9d0101de0 2024-09-05T06:57:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.327+0000 7fc9c6ffd640 1 -- 172.21.15.86:0/2868545039 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc9cc010a90 con 0x7fc9d0101de0 2024-09-05T06:57:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.327+0000 7fc9d9fea740 1 Processor -- start 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.327+0000 7fc9d9fea740 1 -- start start 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 0x55bc22eaf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc22e0a2e0 con 0x55bc22eaf680 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc22db1ce0 con 0x55bc22ea44a0 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc22e062a0 con 0x55bc22ea4dc0 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 0x55bc22eaf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45528/0 (socket says 172.21.15.86:45528) 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 -- 172.21.15.86:0/3882959556 learned_addr learned my addr 172.21.15.86:0/3882959556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9c6ffd640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 0x7fc9bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 -- 172.21.15.86:0/3882959556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 msgr2=0x55bc22eaf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 0x55bc22eaf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 -- 172.21.15.86:0/3882959556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 msgr2=0x55bc22eb1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d61ae640 1 -- 172.21.15.86:0/3882959556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc22d96440 con 0x55bc22ea44a0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.328+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 0x55bc22eaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 0x7fc9bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9c6ffd640 1 -- 172.21.15.86:0/2868545039 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fc9cc0cfbc0 con 0x7fc9d0101de0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009fd0 tx=0x7fc9c8010a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2ecc7276292604 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9bbfff640 1 -- 172.21.15.86:0/3882959556 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c802a070 con 0x55bc22ea44a0 2024-09-05T06:57:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9bbfff640 1 -- 172.21.15.86:0/3882959556 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c8033070 con 0x55bc22ea44a0 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.329+0000 7fc9bbfff640 1 -- 172.21.15.86:0/3882959556 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8037b70 con 0x55bc22ea44a0 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 -- 172.21.15.86:0/3882959556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 msgr2=0x55bc22ea4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009fd0 tx=0x7fc9c8010a00 comp rx=0 tx=0).stop 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 0x7fc9bc0a5230 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8032000 tx=0x7fc9c8030000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 -- 172.21.15.86:0/3882959556 shutdown_connections 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea4dc0 0x55bc22eaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/3882959556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea44a0 0x55bc22ea4880 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 -- 172.21.15.86:0/3882959556 >> 172.21.15.86:0/3882959556 conn(0x55bc22ea0a30 msgr2=0x55bc22ea0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 -- 172.21.15.86:0/3882959556 shutdown_connections 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.330+0000 7fc9d9fea740 1 -- 172.21.15.86:0/3882959556 wait complete. 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 -- start start 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 Processor -- start 2024-09-05T06:57:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eaeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 0x55bc22eac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eaeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d51ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc22eb2250 con 0x55bc22ea4dc0 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eaeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52684/0 (socket says 172.21.15.86:52684) 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.331+0000 7fc9d51ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40630/0 (socket says 172.21.15.86:40630) 2024-09-05T06:57:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d59ad640 1 -- 172.21.15.86:0/2297817437 learned_addr learned my addr 172.21.15.86:0/2297817437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc22dc5050 con 0x55bc22eaf680 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc22eb23d0 con 0x55bc22ea44a0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d9fea740 10 client.0 _ll_register_callbacks cb 0x55bc22e05160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 0x55bc22eac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 -- 172.21.15.86:0/2297817437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 msgr2=0x55bc22eaeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eaeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 -- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 msgr2=0x55bc22eac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 0x55bc22eac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc22b7b3c0 con 0x55bc22ea4dc0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 0x55bc22eac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009230 tx=0x7fc9c0031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e0a13b9370c322 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:55.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035040 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fc9c0004240 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.332+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c8033070 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc9c004f050 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 10 client.19185 fetch_fsmap learned FSMap version 43 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bc22ebdc00 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c8033070 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc9c0004900 con 0x55bc22ea4dc0 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 10 client.19185 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 10 client.19185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 20 client.19185 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22ebf0c0 0x55bc22ec1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 4 client.19185 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc22e17850 con 0x55bc22ebf0c0 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.333+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22ebf0c0 0x55bc22ec1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.334+0000 7fc9b97fa640 10 client.19185 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.334+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22ebf0c0 0x55bc22ec1510 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b275b51a51b5ad68 server_cookie=47cb7ea562a29c95 in_seq=0 out_seq=0 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9b97fa640 1 -- 172.21.15.86:0/2297817437 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc22e17850 con 0x55bc22ebf0c0 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9b97fa640 10 client.19185 handle_command_reply: tid=0 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 client.19185 shutdown 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 msgr2=0x55bc22eac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009230 tx=0x7fc9c0031740 comp rx=0 tx=0).stop 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 shutdown_connections 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22ebf0c0 0x55bc22ec1510 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22eaf680 0x55bc22eac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9b97fa640 0 client.19185 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22ea4dc0 0x55bc22eac2a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/2297817437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eaeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 >> 172.21.15.86:0/2297817437 conn(0x55bc22ea0a30 msgr2=0x55bc22ea6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 shutdown_connections 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 1 -- 172.21.15.86:0/2297817437 wait complete. 2024-09-05T06:57:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.335+0000 7fc9d9fea740 20 client.19185 trim_cache size 0 max 16384 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.358+0000 7fc9d9fea740 1 Processor -- start 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 -- start start 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22ebb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22ec6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aceb0 con 0x55bc22eaf680 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aea50 con 0x55bc22ea4dc0 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ad580 con 0x55bc22ea44a0 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40638/0 (socket says 172.21.15.86:40638) 2024-09-05T06:57:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1112593929 learned_addr learned my addr 172.21.15.86:0/1112593929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22ec6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22ebb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1112593929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 msgr2=0x55bc22ebb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22ebb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1112593929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 msgr2=0x55bc22ec6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22ec6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.359+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1112593929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc22ea04d0 con 0x55bc22eaf680 2024-09-05T06:57:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.360+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8012000 tx=0x7fc9c8039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9075e92c1895a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.360+0000 7fc9bb7fe640 1 -- 172.21.15.86:0/1112593929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8045070 con 0x55bc22eaf680 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.360+0000 7fc9bb7fe640 1 -- 172.21.15.86:0/1112593929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c8033040 con 0x55bc22eaf680 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.360+0000 7fc9bb7fe640 1 -- 172.21.15.86:0/1112593929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c802a050 con 0x55bc22eaf680 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1112593929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 msgr2=0x55bc22ec7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8012000 tx=0x7fc9c8039000 comp rx=0 tx=0).stop 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1112593929 shutdown_connections 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22ec7320 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22ec6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1112593929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22ebb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1112593929 >> 172.21.15.86:0/1112593929 conn(0x55bc22ea0a30 msgr2=0x55bc22eab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1112593929 shutdown_connections 2024-09-05T06:57:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.361+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1112593929 wait complete. 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.362+0000 7fc9d9fea740 1 -- start start 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.362+0000 7fc9d9fea740 1 Processor -- start 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.362+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eb7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.362+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d9fea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eb7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d59ad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eb7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52694/0 (socket says 172.21.15.86:52694) 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d51ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ae380 con 0x55bc22eaf680 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01adcb0 con 0x55bc22ea4dc0 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d9fea740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aaeb0 con 0x55bc22ea44a0 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d51ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45562/0 (socket says 172.21.15.86:45562) 2024-09-05T06:57:55.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d51ac640 1 -- 172.21.15.86:0/1991720136 learned_addr learned my addr 172.21.15.86:0/1991720136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d9fea740 10 client.0 _ll_register_callbacks cb 0x7fc9c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1991720136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 msgr2=0x55bc22eb7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eb7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 msgr2=0x55bc22eb2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 -- 172.21.15.86:0/1991720136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc22eb3940 con 0x55bc22eaf680 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d51ac640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9d61ae640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb3380 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8010f00 tx=0x7fc9c8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5231fbdf5924cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.363+0000 7fc9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8050060 con 0x55bc22eaf680 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c8033040 con 0x55bc22eaf680 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fc9c8024840 con 0x55bc22eaf680 2024-09-05T06:57:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc9c802a020 con 0x55bc22eaf680 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9d9fea740 10 client.19191 fetch_fsmap learned FSMap version 43 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bc22f29b30 con 0x55bc22eaf680 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c8033040 con 0x55bc22eaf680 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.364+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc9c80375b0 con 0x55bc22eaf680 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 10 client.19191 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 20 client.19191 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22f2a010 0x55bc22f2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 4 client.19191 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc22ec18a0 con 0x55bc22f2a010 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22f2a010 0x55bc22f2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9b97fa640 10 client.19191 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.365+0000 7fc9d59ad640 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22f2a010 0x55bc22f2c400 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fbc6f7bd9be608 server_cookie=64930ad8d10f9b7d in_seq=0 out_seq=0 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9b97fa640 1 -- 172.21.15.86:0/1991720136 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc22ec18a0 con 0x55bc22f2a010 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9b97fa640 10 client.19191 handle_command_reply: tid=0 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 client.19191 shutdown 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 msgr2=0x55bc22eb3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb3380 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8010f00 tx=0x7fc9c8009f00 comp rx=0 tx=0).stop 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 shutdown_connections 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55bc22f2a010 0x55bc22f2c400 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc22eaf680 0x55bc22eb3380 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc22ea4dc0 0x55bc22eb2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 --2- 172.21.15.86:0/1991720136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc22ea44a0 0x55bc22eb7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 >> 172.21.15.86:0/1991720136 conn(0x55bc22ea0a30 msgr2=0x55bc22eadb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9b97fa640 0 client.19191 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 shutdown_connections 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 1 -- 172.21.15.86:0/1991720136 wait complete. 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.366+0000 7fc9d9fea740 20 client.19191 trim_cache size 0 max 16384 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:57:55.373 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:57:55.374 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 msgr2=0x7fc9bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 0x7fc9bc0a5230 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8032000 tx=0x7fc9c8030000 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 msgr2=0x7fc9d0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00e9f0 tx=0x7fc9cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 shutdown_connections 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc9bc0a2d70 0x7fc9bc0a5230 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9d0106f10 0x7fc9d019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d0102790 0x7fc9d0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 --2- 172.21.15.86:0/2868545039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9d0101de0 0x7fc9d0198320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 >> 172.21.15.86:0/2868545039 conn(0x7fc9d00fd880 msgr2=0x7fc9d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 shutdown_connections 2024-09-05T06:57:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:55.367+0000 7fc9d7c11640 1 -- 172.21.15.86:0/2868545039 wait complete. 2024-09-05T06:57:55.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:55 smithi086 ceph-mon[30943]: pgmap v522: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:55 smithi115 ceph-mon[34791]: pgmap v522: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:55.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26606 8.30 MB/sec execute 73 sec latency 92.592 ms 2024-09-05T06:57:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:55 smithi096 ceph-mon[34468]: pgmap v522: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:56.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:57:56.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:57:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 -- 172.21.15.86:0/1981949748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590100e20 msgr2=0x7f3590101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 --2- 172.21.15.86:0/1981949748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590100e20 0x7f3590101200 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f358c009900 tx=0x7f358c02a490 comp rx=0 tx=0).stop 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 -- 172.21.15.86:0/1981949748 shutdown_connections 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 --2- 172.21.15.86:0/1981949748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590102ef0 0x7f359010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 --2- 172.21.15.86:0/1981949748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 --2- 172.21.15.86:0/1981949748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590100e20 0x7f3590101200 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 -- 172.21.15.86:0/1981949748 >> 172.21.15.86:0/1981949748 conn(0x7f35900fc7d0 msgr2=0x7f35900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 -- 172.21.15.86:0/1981949748 shutdown_connections 2024-09-05T06:57:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.752+0000 7f3597ad3640 1 -- 172.21.15.86:0/1981949748 wait complete. 2024-09-05T06:57:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.754+0000 7f3597ad3640 1 Processor -- start 2024-09-05T06:57:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.754+0000 7f3597ad3640 1 -- start start 2024-09-05T06:57:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 0x7f35901a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590102ef0 0x7f359010c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f359586f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 0x7f35901a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f359586f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 0x7f35901a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45590/0 (socket says 172.21.15.86:45590) 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f359586f640 1 -- 172.21.15.86:0/764185685 learned_addr learned my addr 172.21.15.86:0/764185685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f359010fb30 con 0x7f3590102ef0 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f359506e640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3596070640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590102ef0 0x7f359010c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f359010f9b0 con 0x7f3590100e20 2024-09-05T06:57:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.755+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f359010fcb0 con 0x7f3590102570 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 -- 172.21.15.86:0/764185685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 msgr2=0x7f35901a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 0x7f35901a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 -- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590102ef0 msgr2=0x7f359010c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590102ef0 0x7f359010c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359010cbd0 con 0x7f3590102570 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f359506e640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901a4a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f358000e9f0 tx=0x7f358000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f3586ffd640 1 -- 172.21.15.86:0/764185685 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358000ccc0 con 0x7f3590102570 2024-09-05T06:57:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f3586ffd640 1 -- 172.21.15.86:0/764185685 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35800108e0 con 0x7f3590102570 2024-09-05T06:57:56.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26761 8.32 MB/sec execute 74 sec latency 107.776 ms 2024-09-05T06:57:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.756+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359010cec0 con 0x7f3590102570 2024-09-05T06:57:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.757+0000 7f3586ffd640 1 -- 172.21.15.86:0/764185685 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580018880 con 0x7f3590102570 2024-09-05T06:57:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.757+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901ab340 con 0x7f3590102570 2024-09-05T06:57:56.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.761+0000 7f3599eac740 1 Processor -- start 2024-09-05T06:57:56.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 -- start start 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 0x55fe0f74a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f74cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f6a52a0 con 0x55fe0f740080 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f64cb10 con 0x55fe0f74a940 2024-09-05T06:57:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.762+0000 7f3599eac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f6a1260 con 0x55fe0f73f760 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f3586ffd640 1 -- 172.21.15.86:0/764185685 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3580010a90 con 0x7f3590102570 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f3596070640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f3596070640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52712/0 (socket says 172.21.15.86:52712) 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f3596070640 1 -- 172.21.15.86:0/1053595919 learned_addr learned my addr 172.21.15.86:0/1053595919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f359586f640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 0x55fe0f74a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.763+0000 7f359506e640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f74cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 -- 172.21.15.86:0/1053595919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 msgr2=0x55fe0f73fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 -- 172.21.15.86:0/1053595919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 msgr2=0x55fe0f74cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f74cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 -- 172.21.15.86:0/1053595919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0f416b90 con 0x55fe0f740080 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f3596070640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:57:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.764+0000 7f359586f640 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 0x55fe0f74a400 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f358c02a9b0 tx=0x7f358c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4790b5d5bd18bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.765+0000 7f3586ffd640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 0x7f357c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.765+0000 7f357bfff640 1 -- 172.21.15.86:0/1053595919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c035040 con 0x55fe0f740080 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.765+0000 7f3586ffd640 1 -- 172.21.15.86:0/764185685 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f35800cfb20 con 0x7f3590102570 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.765+0000 7f3596070640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 0x7f357c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.765+0000 7f357bfff640 1 -- 172.21.15.86:0/1053595919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c0054d0 con 0x55fe0f740080 2024-09-05T06:57:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f357bfff640 1 -- 172.21.15.86:0/1053595919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c003850 con 0x55fe0f740080 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 -- 172.21.15.86:0/1053595919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 msgr2=0x55fe0f74a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 0x55fe0f74a400 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f358c02a9b0 tx=0x7f358c000bc0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3596070640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 0x7f357c0a5230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f358800bfd0 tx=0x7f358800d040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 -- 172.21.15.86:0/1053595919 shutdown_connections 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f74cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f740080 0x55fe0f74a400 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 --2- 172.21.15.86:0/1053595919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f73fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.766+0000 7f3599eac740 1 -- 172.21.15.86:0/1053595919 >> 172.21.15.86:0/1053595919 conn(0x55fe0f73bcf0 msgr2=0x55fe0f73c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.767+0000 7f3599eac740 1 -- 172.21.15.86:0/1053595919 shutdown_connections 2024-09-05T06:57:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.767+0000 7f3599eac740 1 -- 172.21.15.86:0/1053595919 wait complete. 2024-09-05T06:57:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.767+0000 7f3599eac740 1 -- start start 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.767+0000 7f3599eac740 1 Processor -- start 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f73f760 0x55fe0f747730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f747c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f75e5c0 con 0x55fe0f74a940 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f62d6e0 con 0x55fe0f73f760 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0f75e740 con 0x55fe0f740080 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3599eac740 10 client.0 _ll_register_callbacks cb 0x55fe0f651a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3596070640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3596070640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40688/0 (socket says 172.21.15.86:40688) 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f3596070640 1 -- 172.21.15.86:0/2852194350 learned_addr learned my addr 172.21.15.86:0/2852194350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.768+0000 7f359506e640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f747c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 -- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f73f760 msgr2=0x55fe0f747730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:57:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f73f760 0x55fe0f747730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 -- 172.21.15.86:0/2852194350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 msgr2=0x55fe0f7481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 -- 172.21.15.86:0/2852194350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0f767a80 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f359506e640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f747c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35800cce30 tx=0x7f358009b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f9896f97787e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f35797fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.769+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580010630 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c0054d0 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f358009b100 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f358009ffb0 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f3599eac740 10 client.28111 fetch_fsmap learned FSMap version 43 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe0f76a390 con 0x55fe0f740080 2024-09-05T06:57:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.770+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c0054d0 con 0x55fe0f740080 2024-09-05T06:57:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f358009ab70 con 0x55fe0f740080 2024-09-05T06:57:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 10 client.28111 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:57:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 10 client.28111 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 20 client.28111 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f76b7a0 0x55fe0f76dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 4 client.28111 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe0f6b29d0 con 0x55fe0f76b7a0 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.771+0000 7f359586f640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f76b7a0 0x55fe0f76dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.772+0000 7f35797fa640 10 client.28111 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.772+0000 7f359586f640 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f76b7a0 0x55fe0f76dc10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a88683bf8d287f server_cookie=7a7af1cdd148156d in_seq=0 out_seq=0 2024-09-05T06:57:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.773+0000 7f35797fa640 1 -- 172.21.15.86:0/2852194350 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe0f6b29d0 con 0x55fe0f76b7a0 2024-09-05T06:57:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.773+0000 7f35797fa640 10 client.28111 handle_command_reply: tid=0 2024-09-05T06:57:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.773+0000 7f3599eac740 1 client.28111 shutdown 2024-09-05T06:57:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.773+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 msgr2=0x55fe0f747c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.773+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f747c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35800cce30 tx=0x7f358009b610 comp rx=0 tx=0).stop 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 shutdown_connections 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f76b7a0 0x55fe0f76dc10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f747c70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 --2- 172.21.15.86:0/2852194350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f73f760 0x55fe0f747730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 >> 172.21.15.86:0/2852194350 conn(0x55fe0f73bcf0 msgr2=0x55fe0f7419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f35797fa640 0 client.28111 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 shutdown_connections 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.774+0000 7f3599eac740 1 -- 172.21.15.86:0/2852194350 wait complete. 2024-09-05T06:57:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.775+0000 7f3599eac740 20 client.28111 trim_cache size 0 max 16384 2024-09-05T06:57:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.808+0000 7f3599eac740 1 Processor -- start 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 -- start start 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f740080 0x55fe0f7419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ad170 con 0x55fe0f74a940 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f359586f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f359586f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52736/0 (socket says 172.21.15.86:52736) 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f359586f640 1 -- 172.21.15.86:0/1906582965 learned_addr learned my addr 172.21.15.86:0/1906582965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35901aed10 con 0x55fe0f740080 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ad840 con 0x55fe0f73f760 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f359506e640 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f740080 0x55fe0f7419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.809+0000 7f3596070640 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 -- 172.21.15.86:0/1906582965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f740080 msgr2=0x55fe0f7419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f740080 0x55fe0f7419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 -- 172.21.15.86:0/1906582965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 msgr2=0x55fe0f7734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 -- 172.21.15.86:0/1906582965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0f73bb20 con 0x55fe0f73f760 2024-09-05T06:57:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f359586f640 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f358c000c00 tx=0x7f358c004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1156630837a60bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f357b7fe640 1 -- 172.21.15.86:0/1906582965 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c035040 con 0x55fe0f73f760 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f357b7fe640 1 -- 172.21.15.86:0/1906582965 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c004d40 con 0x55fe0f73f760 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.810+0000 7f357b7fe640 1 -- 172.21.15.86:0/1906582965 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c003810 con 0x55fe0f73f760 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 msgr2=0x55fe0f748000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f358c000c00 tx=0x7f358c004c10 comp rx=0 tx=0).stop 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 shutdown_connections 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f74a940 0x55fe0f7734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f740080 0x55fe0f7419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 --2- 172.21.15.86:0/1906582965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f73f760 0x55fe0f748000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.811+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 >> 172.21.15.86:0/1906582965 conn(0x55fe0f73bcf0 msgr2=0x55fe0f749830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.812+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 shutdown_connections 2024-09-05T06:57:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.812+0000 7f3599eac740 1 -- 172.21.15.86:0/1906582965 wait complete. 2024-09-05T06:57:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.812+0000 7f3599eac740 1 -- start start 2024-09-05T06:57:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.812+0000 7f3599eac740 1 Processor -- start 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 0x55fe0f75fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f75ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f7d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ae640 con 0x55fe0f73f760 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35901adf70 con 0x55fe0f74a940 2024-09-05T06:57:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35901021f0 con 0x55fe0f740080 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3599eac740 10 client.0 _ll_register_callbacks cb 0x7f358009ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f359586f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 0x55fe0f75fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f359586f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 0x55fe0f75fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40704/0 (socket says 172.21.15.86:40704) 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f359586f640 1 -- 172.21.15.86:0/9306205 learned_addr learned my addr 172.21.15.86:0/9306205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f359506e640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f75ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.813+0000 7f3596070640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f7d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 -- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 msgr2=0x55fe0f7d54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f7d54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 -- 172.21.15.86:0/9306205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 msgr2=0x55fe0f75fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 0x55fe0f75fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 -- 172.21.15.86:0/9306205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0f760830 con 0x55fe0f740080 2024-09-05T06:57:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f359506e640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f75ffa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55fe0f76a2f0 tx=0x7f35800a1490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80e01a93ee9b604 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f35797fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.814+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800a4870 con 0x55fe0f740080 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f3580010640 con 0x55fe0f740080 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c004d40 con 0x55fe0f740080 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f358009b100 con 0x55fe0f740080 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 10 client.28117 fetch_fsmap learned FSMap version 43 2024-09-05T06:57:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe0f7d6aa0 con 0x55fe0f740080 2024-09-05T06:57:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35800b2070 con 0x55fe0f740080 2024-09-05T06:57:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f358007aad0 con 0x55fe0f740080 2024-09-05T06:57:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 10 client.28117 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:57:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 10 client.28117 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 20 client.28117 populate_metadata read hostname 'smithi086' 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.815+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f7d7260 0x55fe0f7d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.816+0000 7f3599eac740 4 client.28117 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.816+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f358c004d40 con 0x55fe0f7d7260 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.816+0000 7f359586f640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f7d7260 0x55fe0f7d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.816+0000 7f35797fa640 10 client.28117 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:56.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:56.816+0000 7f359586f640 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f7d7260 0x55fe0f7d9680 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa911f58b1016de2 server_cookie=9546863f3d17b185 in_seq=0 out_seq=0 2024-09-05T06:57:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f35797fa640 1 -- 172.21.15.86:0/9306205 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f358c004d40 con 0x55fe0f7d7260 2024-09-05T06:57:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f35797fa640 10 client.28117 handle_command_reply: tid=0 2024-09-05T06:57:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 client.28117 shutdown 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 msgr2=0x55fe0f75ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f75ffa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55fe0f76a2f0 tx=0x7f35800a1490 comp rx=0 tx=0).stop 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 shutdown_connections 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fe0f7d7260 0x55fe0f7d9680 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fe0f74a940 0x55fe0f7d54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0f740080 0x55fe0f75ffa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 --2- 172.21.15.86:0/9306205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe0f73f760 0x55fe0f75fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f35797fa640 0 client.28117 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 >> 172.21.15.86:0/9306205 conn(0x55fe0f73bcf0 msgr2=0x55fe0f7466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.075+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 shutdown_connections 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3599eac740 1 -- 172.21.15.86:0/9306205 wait complete. 2024-09-05T06:57:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3599eac740 20 client.28117 trim_cache size 0 max 16384 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 43, 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 62, 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "96a4659f": { 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 62, 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:57.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:57.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 msgr2=0x7f357c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 0x7f357c0a5230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f358800bfd0 tx=0x7f358800d040 comp rx=0 tx=0).stop 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 msgr2=0x7f35901a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:57.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901a4a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f358000e9f0 tx=0x7f358000eeb0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 shutdown_connections 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f357c0a2d70 0x7f357c0a5230 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590102ef0 0x7f359010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3590102570 0x7f35901a4a80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 --2- 172.21.15.86:0/764185685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3590100e20 0x7f35901a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.076+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 >> 172.21.15.86:0/764185685 conn(0x7f35900fc7d0 msgr2=0x7f3590111ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.077+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 shutdown_connections 2024-09-05T06:57:57.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:57.077+0000 7f3597ad3640 1 -- 172.21.15.86:0/764185685 wait complete. 2024-09-05T06:57:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:57 smithi115 ceph-mon[34791]: pgmap v523: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:57 smithi086 ceph-mon[30943]: pgmap v523: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:57.757 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 8.22 MB/sec execute 75 sec latency 939.120 ms 2024-09-05T06:57:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:57 smithi096 ceph-mon[34468]: pgmap v523: 129 pgs: 129 active+clean; 772 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 222 op/s 2024-09-05T06:57:57.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '96a4659f', quiesce took 13.5 seconds. Will release after: 16.5 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 -- 172.21.15.86:0/3648601307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 msgr2=0x7fc074100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 --2- 172.21.15.86:0/3648601307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc074100ee0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009930 tx=0x7fc05c02a420 comp rx=0 tx=0).stop 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 -- 172.21.15.86:0/3648601307 shutdown_connections 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 --2- 172.21.15.86:0/3648601307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101420 0x7fc07410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 --2- 172.21.15.86:0/3648601307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc074100ee0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 --2- 172.21.15.86:0/3648601307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074106a80 0x7fc074106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 -- 172.21.15.86:0/3648601307 >> 172.21.15.86:0/3648601307 conn(0x7fc0740fc7a0 msgr2=0x7fc0740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 -- 172.21.15.86:0/3648601307 shutdown_connections 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 -- 172.21.15.86:0/3648601307 wait complete. 2024-09-05T06:57:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.160+0000 7fc07992c640 1 Processor -- start 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 -- start start 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 0x7fc0740fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 0x7fc0740fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc074110d30 con 0x7fc074100a80 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc074110bb0 con 0x7fc074106a80 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc074110eb0 con 0x7fc074101420 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40732/0 (socket says 172.21.15.86:40732) 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 -- 172.21.15.86:0/3491258008 learned_addr learned my addr 172.21.15.86:0/3491258008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc06bfff640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 0x7fc0740fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc0737fe640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 0x7fc0740fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 -- 172.21.15.86:0/3491258008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 msgr2=0x7fc0740fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 0x7fc0740fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 -- 172.21.15.86:0/3491258008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 msgr2=0x7fc0740fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 0x7fc0740fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 -- 172.21.15.86:0/3491258008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0741a6890 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc06bfff640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 0x7fc0740fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc0737fe640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 0x7fc0740fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc072ffd640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc06400d770 tx=0x7fc06400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064017070 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.161+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741a6b80 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.162+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0640054d0 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.162+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004cf0 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.162+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a7090 con 0x7fc074100a80 2024-09-05T06:57:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.163+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074100ee0 con 0x7fc074100a80 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.164+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc064004e90 con 0x7fc074100a80 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.164+0000 7fc07892a640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 0x7fc0600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.165+0000 7fc06bfff640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 0x7fc0600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.165+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fc0640cf6d0 con 0x7fc074100a80 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.165+0000 7fc06bfff640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 0x7fc0600a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c009610 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:57:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.168+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc06409beb0 con 0x7fc074100a80 2024-09-05T06:57:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.366+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc07402d7c0 con 0x7fc074100a80 2024-09-05T06:57:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.367+0000 7fc07892a640 1 -- 172.21.15.86:0/3491258008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7fc06409b7b0 con 0x7fc074100a80 2024-09-05T06:57:58.368 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:57:58.368 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:57:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 msgr2=0x7fc0600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 0x7fc0600a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c009610 comp rx=0 tx=0).stop 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 msgr2=0x7fc0740fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc06400d770 tx=0x7fc06400dc30 comp rx=0 tx=0).stop 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 shutdown_connections 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc0600a2cf0 0x7fc0600a51b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106a80 0x7fc0740fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074101420 0x7fc0740fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 --2- 172.21.15.86:0/3491258008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc074100a80 0x7fc0740fce50 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:57:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 >> 172.21.15.86:0/3491258008 conn(0x7fc0740fc7a0 msgr2=0x7fc07406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:57:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 shutdown_connections 2024-09-05T06:57:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:57:58.369+0000 7fc07992c640 1 -- 172.21.15.86:0/3491258008 wait complete. 2024-09-05T06:57:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:58 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3491258008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:58 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3491258008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:58.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 8.11 MB/sec execute 76 sec latency 1939.203 ms 2024-09-05T06:57:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:58 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3491258008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:57:59.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:57:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:57:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:57:59 smithi115 ceph-mon[34791]: pgmap v524: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-09-05T06:57:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:57:59 smithi086 ceph-mon[30943]: pgmap v524: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-09-05T06:57:59.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 8.01 MB/sec execute 77 sec latency 2939.341 ms 2024-09-05T06:57:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:57:59 smithi096 ceph-mon[34468]: pgmap v524: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 319 op/s 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 -- 172.21.15.86:0/3698851787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 msgr2=0x7f096010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 --2- 172.21.15.86:0/3698851787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f096010d940 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0958009cd0 tx=0x7f095802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 -- 172.21.15.86:0/3698851787 shutdown_connections 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 --2- 172.21.15.86:0/3698851787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f096010d940 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 --2- 172.21.15.86:0/3698851787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960100a70 0x7f0960100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 --2- 172.21.15.86:0/3698851787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960106a70 0x7f0960106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 -- 172.21.15.86:0/3698851787 >> 172.21.15.86:0/3698851787 conn(0x7f09600fc790 msgr2=0x7f09600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 -- 172.21.15.86:0/3698851787 shutdown_connections 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.158+0000 7f096556f640 1 -- 172.21.15.86:0/3698851787 wait complete. 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 Processor -- start 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 -- start start 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 0x7f09600fce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960106a70 0x7f09600fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0960110d20 con 0x7f0960101410 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0960110ba0 con 0x7f0960100a70 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.159+0000 7f096556f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0960110ea0 con 0x7f0960106a70 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40758/0 (socket says 172.21.15.86:40758) 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960106a70 0x7f09600fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 -- 172.21.15.86:0/3724117790 learned_addr learned my addr 172.21.15.86:0/3724117790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095effd640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 0x7f09600fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 -- 172.21.15.86:0/3724117790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960106a70 msgr2=0x7f09600fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960106a70 0x7f09600fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 -- 172.21.15.86:0/3724117790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 msgr2=0x7f09600fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 0x7f09600fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 -- 172.21.15.86:0/3724117790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09601a67d0 con 0x7f0960101410 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095effd640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 0x7f09600fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f09567fc640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f094800e9f0 tx=0x7f094800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095cff9640 1 -- 172.21.15.86:0/3724117790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094800cd80 con 0x7f0960101410 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095cff9640 1 -- 172.21.15.86:0/3724117790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09480054d0 con 0x7f0960101410 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f095cff9640 1 -- 172.21.15.86:0/3724117790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948010490 con 0x7f0960101410 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09601a6ac0 con 0x7f0960101410 2024-09-05T06:58:00.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.160+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601a6fb0 con 0x7f0960101410 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.163+0000 7f095cff9640 1 -- 172.21.15.86:0/3724117790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0948005020 con 0x7f0960101410 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.163+0000 7f0967948740 1 Processor -- start 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 -- start start 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c22f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559795b732f0 con 0x559795c0d5d0 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559795b1acf0 con 0x559795c20b80 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f0967948740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559795b6f2b0 con 0x559795c0eb40 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095cff9640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 0x7f094c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40772/0 (socket says 172.21.15.86:40772) 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095effd640 1 -- 172.21.15.86:0/3069942729 learned_addr learned my addr 172.21.15.86:0/3069942729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:00.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095f7fe640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f095f7fe640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 0x7f094c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.164+0000 7f09567fc640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c22f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 -- 172.21.15.86:0/3069942729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 msgr2=0x559795c0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 -- 172.21.15.86:0/3069942729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 msgr2=0x559795c22f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c22f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 -- 172.21.15.86:0/3069942729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559795aff3a0 con 0x559795c0d5d0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f09567fc640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c22f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095cff9640 1 -- 172.21.15.86:0/3724117790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f09480cfc50 con 0x7f0960101410 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095effd640 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0950009c80 tx=0x7f0950010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736723e5a3308f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f095f7fe640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 0x7f094c0a5230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0958004af0 tx=0x7f0958005da0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f09557fa640 1 -- 172.21.15.86:0/3069942729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950004030 con 0x559795c0d5d0 2024-09-05T06:58:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f09557fa640 1 -- 172.21.15.86:0/3069942729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09500035f0 con 0x559795c0d5d0 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.165+0000 7f09557fa640 1 -- 172.21.15.86:0/3069942729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950012670 con 0x559795c0d5d0 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- 172.21.15.86:0/3069942729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 msgr2=0x559795c0da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0950009c80 tx=0x7f0950010740 comp rx=0 tx=0).stop 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- 172.21.15.86:0/3069942729 shutdown_connections 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c22f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c0da10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 --2- 172.21.15.86:0/3069942729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- 172.21.15.86:0/3069942729 >> 172.21.15.86:0/3069942729 conn(0x559795bd6310 msgr2=0x559795c09c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- 172.21.15.86:0/3069942729 shutdown_connections 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- 172.21.15.86:0/3069942729 wait complete. 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 -- start start 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.166+0000 7f0967948740 1 Processor -- start 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c1d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559795c23880 con 0x559795c0d5d0 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559795b2e060 con 0x559795c20b80 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559795c23a00 con 0x559795c0eb40 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f0967948740 10 client.0 _ll_register_callbacks cb 0x5597958818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40784/0 (socket says 172.21.15.86:40784) 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 -- 172.21.15.86:0/3386812183 learned_addr learned my addr 172.21.15.86:0/3386812183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095f7fe640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f09567fc640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c1d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 -- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 msgr2=0x559795c1d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c1d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 -- 172.21.15.86:0/3386812183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 msgr2=0x559795c1dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095effd640 1 -- 172.21.15.86:0/3386812183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597958e43c0 con 0x559795c0d5d0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f09567fc640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c1d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.167+0000 7f095f7fe640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f095effd640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f09500099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddc5b2e445c3bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950005050 con 0x559795c0d5d0 2024-09-05T06:58:00.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f09500051f0 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09500035f0 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0950005430 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f0967948740 10 client.19227 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559795c2f280 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09500035f0 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.168+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f095000f040 con 0x559795c0d5d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 10 client.19227 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 10 client.19227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 20 client.19227 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c305d0 0x559795c329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 4 client.19227 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559795b80860 con 0x559795c305d0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f09567fc640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c305d0 0x559795c329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f09467fc640 10 client.19227 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.169+0000 7f09567fc640 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c305d0 0x559795c329c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11b8f7dd464e525c server_cookie=ac4cfaad328937b in_seq=0 out_seq=0 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f09467fc640 1 -- 172.21.15.86:0/3386812183 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559795b80860 con 0x559795c305d0 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f09467fc640 10 client.19227 handle_command_reply: tid=0 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 client.19227 shutdown 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 msgr2=0x559795c20540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f09500099a0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 shutdown_connections 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c305d0 0x559795c329c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f09467fc640 0 client.19227 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0eb40 0x559795c1d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 --2- 172.21.15.86:0/3386812183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0d5d0 0x559795c20540 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 >> 172.21.15.86:0/3386812183 conn(0x559795bd6310 msgr2=0x559795c0f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 shutdown_connections 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 1 -- 172.21.15.86:0/3386812183 wait complete. 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.170+0000 7f0967948740 20 client.19227 trim_cache size 0 max 16384 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 Processor -- start 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 -- start start 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c09d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09601acf20 con 0x559795c0eb40 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09601aeac0 con 0x559795c20b80 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f0967948740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ad5f0 con 0x559795c0d5d0 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f09567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f095effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c09d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f09567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40790/0 (socket says 172.21.15.86:40790) 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.188+0000 7f09567fc640 1 -- 172.21.15.86:0/1037154875 learned_addr learned my addr 172.21.15.86:0/1037154875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 -- 172.21.15.86:0/1037154875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 msgr2=0x559795c09d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f095f7fe640 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c09d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 -- 172.21.15.86:0/1037154875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 msgr2=0x559795c1d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 -- 172.21.15.86:0/1037154875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559795c1bd40 con 0x559795c0eb40 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f095effd640 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c09d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f09567fc640 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f09480ccfc0 tx=0x7f094805d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6aa1a9758cb0572 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f0954ff9640 1 -- 172.21.15.86:0/1037154875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094809c430 con 0x559795c0eb40 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f0954ff9640 1 -- 172.21.15.86:0/1037154875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f094809bd30 con 0x559795c0eb40 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.189+0000 7f0954ff9640 1 -- 172.21.15.86:0/1037154875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09480a4510 con 0x559795c0eb40 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- 172.21.15.86:0/1037154875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 msgr2=0x559795c2cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f09480ccfc0 tx=0x7f094805d040 comp rx=0 tx=0).stop 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- 172.21.15.86:0/1037154875 shutdown_connections 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c20b80 0x559795c1d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c0eb40 0x559795c2cb50 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 --2- 172.21.15.86:0/1037154875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c09d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- 172.21.15.86:0/1037154875 >> 172.21.15.86:0/1037154875 conn(0x559795bd6310 msgr2=0x559795c08d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- 172.21.15.86:0/1037154875 shutdown_connections 2024-09-05T06:58:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- 172.21.15.86:0/1037154875 wait complete. 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 -- start start 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.190+0000 7f0967948740 1 Processor -- start 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c0eb40 0x559795c293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 0x559795c1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ae3f0 con 0x559795c20b80 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09601add20 con 0x559795c0eb40 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09601aaf20 con 0x559795c0d5d0 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 0x559795c1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 0x559795c1a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40800/0 (socket says 172.21.15.86:40800) 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095f7fe640 1 -- 172.21.15.86:0/2531595188 learned_addr learned my addr 172.21.15.86:0/2531595188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:00.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f0967948740 10 client.0 _ll_register_callbacks cb 0x7f0948005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f09567fc640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c0eb40 0x559795c293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 -- 172.21.15.86:0/2531595188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c0eb40 msgr2=0x559795c293e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c0eb40 0x559795c293e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 -- 172.21.15.86:0/2531595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 msgr2=0x559795c1a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 0x559795c1a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.191+0000 7f095effd640 1 -- 172.21.15.86:0/2531595188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559795c1a6f0 con 0x559795c0d5d0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f095effd640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c28ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0950010c40 tx=0x7f0950003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f6be15129a77e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500035f0 con 0x559795c0d5d0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f09500128a0 con 0x559795c0d5d0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094809bd30 con 0x559795c0d5d0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f09500055d0 con 0x559795c0d5d0 2024-09-05T06:58:00.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f0967948740 10 client.28141 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559795c9aa60 con 0x559795c0d5d0 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0950009ad0 con 0x559795c0d5d0 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.192+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f0950033030 con 0x559795c0d5d0 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 10 client.28141 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 10 client.28141 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 20 client.28141 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c9b180 0x559795c9d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 4 client.28141 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f094809bd30 con 0x559795c9b180 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f09567fc640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c9b180 0x559795c9d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f09567fc640 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c9b180 0x559795c9d570 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d13ee808aceed314 server_cookie=37bafb9d4acf107e in_seq=0 out_seq=0 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.193+0000 7f09467fc640 10 client.28141 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:00.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f09467fc640 1 -- 172.21.15.86:0/2531595188 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f094809bd30 con 0x559795c9b180 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f09467fc640 10 client.28141 handle_command_reply: tid=0 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 client.28141 shutdown 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 msgr2=0x559795c28ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c28ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0950010c40 tx=0x7f0950003b90 comp rx=0 tx=0).stop 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 shutdown_connections 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x559795c9b180 0x559795c9d570 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559795c20b80 0x559795c1a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559795c0eb40 0x559795c293e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 --2- 172.21.15.86:0/2531595188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559795c0d5d0 0x559795c28ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f09467fc640 0 client.28141 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 >> 172.21.15.86:0/2531595188 conn(0x559795bd6310 msgr2=0x559795c0f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 shutdown_connections 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 1 -- 172.21.15.86:0/2531595188 wait complete. 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.194+0000 7f0967948740 20 client.28141 trim_cache size 0 max 16384 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 msgr2=0x7f094c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 0x7f094c0a5230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0958004af0 tx=0x7f0958005da0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 msgr2=0x7f09600fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f094800e9f0 tx=0x7f094800edf0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 shutdown_connections 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f094c0a2d70 0x7f094c0a5230 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0960106a70 0x7f09600fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960101410 0x7f09600fd360 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 --2- 172.21.15.86:0/3724117790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960100a70 0x7f09600fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 >> 172.21.15.86:0/3724117790 conn(0x7f09600fc790 msgr2=0x7f096006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 shutdown_connections 2024-09-05T06:58:00.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:00.195+0000 7f096556f640 1 -- 172.21.15.86:0/3724117790 wait complete. 2024-09-05T06:58:00.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.90 MB/sec execute 78 sec latency 3939.431 ms 2024-09-05T06:58:00.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:58:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:01.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:01 smithi086 ceph-mon[30943]: pgmap v525: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-05T06:58:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:01 smithi115 ceph-mon[34791]: pgmap v525: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-05T06:58:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 -- 172.21.15.86:0/916295062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6100108b50 msgr2=0x7f6100115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 --2- 172.21.15.86:0/916295062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6100108b50 0x7f6100115070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009be0 tx=0x7f60f802a6c0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 -- 172.21.15.86:0/916295062 shutdown_connections 2024-09-05T06:58:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 --2- 172.21.15.86:0/916295062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6100108b50 0x7f6100115070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 --2- 172.21.15.86:0/916295062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 0x7f6100108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 --2- 172.21.15.86:0/916295062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f610010e230 0x7f610010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 -- 172.21.15.86:0/916295062 >> 172.21.15.86:0/916295062 conn(0x7f6100077bb0 msgr2=0x7f6100079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 -- 172.21.15.86:0/916295062 shutdown_connections 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.703+0000 7f6104cc0640 1 -- 172.21.15.86:0/916295062 wait complete. 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.704+0000 7f6104cc0640 1 Processor -- start 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.704+0000 7f6104cc0640 1 -- start start 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 0x7f61001a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610010e230 0x7f610019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61001183f0 con 0x7f6100108b50 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6100118270 con 0x7f610010e230 2024-09-05T06:58:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6100118570 con 0x7f61001081f0 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40828/0 (socket says 172.21.15.86:40828) 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1605845426 learned_addr learned my addr 172.21.15.86:0/1605845426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fe59c640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 0x7f61001a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610010e230 0x7f610019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1605845426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 msgr2=0x7f61001a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 0x7f61001a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1605845426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610010e230 msgr2=0x7f610019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610010e230 0x7f610019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1605845426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019ef20 con 0x7f6100108b50 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60f77fe640 1 -- 172.21.15.86:0/1605845426 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00cd80 con 0x7f6100108b50 2024-09-05T06:58:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60f77fe640 1 -- 172.21.15.86:0/1605845426 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60ec0108e0 con 0x7f6100108b50 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019f210 con 0x7f6100108b50 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f60f77fe640 1 -- 172.21.15.86:0/1605845426 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec018920 con 0x7f6100108b50 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.705+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001ab460 con 0x7f6100108b50 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f60f77fe640 1 -- 172.21.15.86:0/1605845426 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60ec010a90 con 0x7f6100108b50 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 Processor -- start 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 -- start start 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef14da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef158370 0x55a3ef15a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0b32f0 con 0x55a3ef14eb40 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef05acf0 con 0x55a3ef14d5d0 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.708+0000 7f6107099740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef0af2b0 con 0x55a3ef158370 2024-09-05T06:58:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40836/0 (socket says 172.21.15.86:40836) 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60f77fe640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 0x7f60e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 -- 172.21.15.86:0/291302747 learned_addr learned my addr 172.21.15.86:0/291302747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef158370 0x55a3ef15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 0x7f60e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fe59c640 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 -- 172.21.15.86:0/291302747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef158370 msgr2=0x55a3ef15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef158370 0x55a3ef15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 -- 172.21.15.86:0/291302747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 msgr2=0x55a3ef14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60f77fe640 1 -- 172.21.15.86:0/1605845426 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f60ec0cfc10 con 0x7f6100108b50 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 -- 172.21.15.86:0/291302747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ef03f3a0 con 0x55a3ef14eb40 2024-09-05T06:58:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005f00 tx=0x7f60f8003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6761740cc0c1634 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60f4ff9640 1 -- 172.21.15.86:0/291302747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f804e070 con 0x55a3ef14eb40 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60f4ff9640 1 -- 172.21.15.86:0/291302747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60f8037040 con 0x55a3ef14eb40 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60f4ff9640 1 -- 172.21.15.86:0/291302747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8036070 con 0x55a3ef14eb40 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.709+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 0x7f60e40a51b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60f804b000 tx=0x7f60f8047000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 -- 172.21.15.86:0/291302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 msgr2=0x55a3ef14d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005f00 tx=0x7f60f8003c90 comp rx=0 tx=0).stop 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 -- 172.21.15.86:0/291302747 shutdown_connections 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef158370 0x55a3ef15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 --2- 172.21.15.86:0/291302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef14d090 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 -- 172.21.15.86:0/291302747 >> 172.21.15.86:0/291302747 conn(0x55a3ef116310 msgr2=0x55a3ef149c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 -- 172.21.15.86:0/291302747 shutdown_connections 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.710+0000 7f6107099740 1 -- 172.21.15.86:0/291302747 wait complete. 2024-09-05T06:58:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 -- start start 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 Processor -- start 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef157d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef158370 0x55a3ef155650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef15b070 con 0x55a3ef14eb40 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef06e060 con 0x55a3ef158370 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a3ef15b1f0 con 0x55a3ef14d5d0 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f6107099740 10 client.0 _ll_register_callbacks cb 0x55a3eedc18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40850/0 (socket says 172.21.15.86:40850) 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1512325839 learned_addr learned my addr 172.21.15.86:0/1512325839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fe59c640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef158370 0x55a3ef155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1512325839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 msgr2=0x55a3ef157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef158370 msgr2=0x55a3ef155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef158370 0x55a3ef155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.711+0000 7f60fdd9b640 1 -- 172.21.15.86:0/1512325839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3eee243c0 con 0x55a3ef14eb40 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00cfd0 tx=0x7f60ec06ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61bf1eb50e1e681 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec026040 con 0x55a3ef14eb40 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f60ec010630 con 0x55a3ef14eb40 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f8037040 con 0x55a3ef14eb40 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60ec09bcf0 con 0x55a3ef14eb40 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 10 client.19245 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a3ef166b90 con 0x55a3ef14eb40 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60f8037040 con 0x55a3ef14eb40 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f60ec097bd0 con 0x55a3ef14eb40 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 10 client.19245 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 10 client.19245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 20 client.19245 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef168050 0x55a3ef16a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 4 client.19245 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.712+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3ef0c0860 con 0x55a3ef168050 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.713+0000 7f60fe59c640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef168050 0x55a3ef16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.713+0000 7f60fe59c640 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef168050 0x55a3ef16a440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ad344937d29fc5d server_cookie=e604b826679c6b0f in_seq=0 out_seq=0 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.713+0000 7f60e1ffb640 10 client.19245 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f60e1ffb640 1 -- 172.21.15.86:0/1512325839 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3ef0c0860 con 0x55a3ef168050 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f60e1ffb640 10 client.19245 handle_command_reply: tid=0 2024-09-05T06:58:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 client.19245 shutdown 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 msgr2=0x55a3ef155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00cfd0 tx=0x7f60ec06ace0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 shutdown_connections 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef168050 0x55a3ef16a440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef158370 0x55a3ef155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef14eb40 0x55a3ef155040 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 --2- 172.21.15.86:0/1512325839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f60e1ffb640 0 client.19245 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 >> 172.21.15.86:0/1512325839 conn(0x55a3ef116310 msgr2=0x55a3ef14f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 shutdown_connections 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 1 -- 172.21.15.86:0/1512325839 wait complete. 2024-09-05T06:58:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.714+0000 7f6107099740 20 client.19245 trim_cache size 0 max 16384 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 Processor -- start 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 -- start start 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14eb40 0x55a3ef150120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b57a0 con 0x55a3ef158370 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b7340 con 0x55a3ef14d5d0 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f6107099740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b5e70 con 0x55a3ef14eb40 2024-09-05T06:58:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40864/0 (socket says 172.21.15.86:40864) 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 -- 172.21.15.86:0/1089609831 learned_addr learned my addr 172.21.15.86:0/1089609831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14eb40 0x55a3ef150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fe59c640 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 -- 172.21.15.86:0/1089609831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14eb40 msgr2=0x55a3ef150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14eb40 0x55a3ef150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 -- 172.21.15.86:0/1089609831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 msgr2=0x55a3ef164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.727+0000 7f60fed9d640 1 -- 172.21.15.86:0/1089609831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ef149ed0 con 0x55a3ef158370 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f60fed9d640 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009be0 tx=0x7f60f8005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e27eda2863822b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f60e3fff640 1 -- 172.21.15.86:0/1089609831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8056020 con 0x55a3ef158370 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f60e3fff640 1 -- 172.21.15.86:0/1089609831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60f8037040 con 0x55a3ef158370 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f60e3fff640 1 -- 172.21.15.86:0/1089609831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8036070 con 0x55a3ef158370 2024-09-05T06:58:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 -- 172.21.15.86:0/1089609831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 msgr2=0x55a3ef150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009be0 tx=0x7f60f8005e30 comp rx=0 tx=0).stop 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 -- 172.21.15.86:0/1089609831 shutdown_connections 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef150660 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14eb40 0x55a3ef150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 --2- 172.21.15.86:0/1089609831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14d5d0 0x55a3ef164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 -- 172.21.15.86:0/1089609831 >> 172.21.15.86:0/1089609831 conn(0x55a3ef116310 msgr2=0x55a3ef154130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 -- 172.21.15.86:0/1089609831 shutdown_connections 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.728+0000 7f6107099740 1 -- 172.21.15.86:0/1089609831 wait complete. 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 -- start start 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 Processor -- start 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14eb40 0x55a3ef15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b6c70 con 0x55a3ef158370 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b65a0 con 0x55a3ef14eb40 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b37a0 con 0x55a3ef14d5d0 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f6107099740 10 client.0 _ll_register_callbacks cb 0x7f60ec09bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40874/0 (socket says 172.21.15.86:40874) 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 -- 172.21.15.86:0/4050151521 learned_addr learned my addr 172.21.15.86:0/4050151521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fdd9b640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14eb40 0x55a3ef15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 -- 172.21.15.86:0/4050151521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 msgr2=0x55a3ef160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 -- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14eb40 msgr2=0x55a3ef15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14eb40 0x55a3ef15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.729+0000 7f60fed9d640 1 -- 172.21.15.86:0/4050151521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ef158100 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60fed9d640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f60f802abe0 tx=0x7f60f8003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607e1ca86d28e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8056030 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f8037040 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f60f8004450 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60f8036040 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 10 client.19251 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a3ef1d2c20 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60f8037040 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f60f8048030 con 0x55a3ef158370 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 10 client.19251 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 10 client.19251 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 20 client.19251 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef1d2e50 0x55a3ef1d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 4 client.19251 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.730+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3ef0bad70 con 0x55a3ef1d2e50 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f60fe59c640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef1d2e50 0x55a3ef1d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f60fe59c640 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef1d2e50 0x55a3ef1d5240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c04f8b184b2ed2b0 server_cookie=caa37256b6c4b0d9 in_seq=0 out_seq=0 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f60e1ffb640 10 client.19251 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f60e1ffb640 1 -- 172.21.15.86:0/4050151521 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3ef0bad70 con 0x55a3ef1d2e50 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f60e1ffb640 10 client.19251 handle_command_reply: tid=0 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f6107099740 1 client.19251 shutdown 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 msgr2=0x55a3ef15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.731+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f60f802abe0 tx=0x7f60f8003c90 comp rx=0 tx=0).stop 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 shutdown_connections 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55a3ef1d2e50 0x55a3ef1d5240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3ef158370 0x55a3ef15c800 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f60e1ffb640 0 client.19251 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3ef14eb40 0x55a3ef15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 --2- 172.21.15.86:0/4050151521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a3ef14d5d0 0x55a3ef160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 >> 172.21.15.86:0/4050151521 conn(0x55a3ef116310 msgr2=0x55a3ef1484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 shutdown_connections 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 1 -- 172.21.15.86:0/4050151521 wait complete. 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6107099740 20 client.19251 trim_cache size 0 max 16384 2024-09-05T06:58:01.739 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 msgr2=0x7f60e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 0x7f60e40a51b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60f804b000 tx=0x7f60f8047000 comp rx=0 tx=0).stop 2024-09-05T06:58:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 msgr2=0x7f61001a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 shutdown_connections 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f60e40a2cf0 0x7f60e40a51b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610010e230 0x7f610019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6100108b50 0x7f61001a4b40 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 --2- 172.21.15.86:0/1605845426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001081f0 0x7f61001a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.732+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 >> 172.21.15.86:0/1605845426 conn(0x7f6100077bb0 msgr2=0x7f6100079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.733+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 shutdown_connections 2024-09-05T06:58:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:01.733+0000 7f6104cc0640 1 -- 172.21.15.86:0/1605845426 wait complete. 2024-09-05T06:58:01.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.80 MB/sec execute 79 sec latency 4939.475 ms 2024-09-05T06:58:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:01 smithi096 ceph-mon[34468]: pgmap v525: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-05T06:58:02.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:58:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:02.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.71 MB/sec execute 80 sec latency 5939.507 ms 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 -- 172.21.15.86:0/189058289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874101420 msgr2=0x7f787410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 --2- 172.21.15.86:0/189058289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874101420 0x7f787410d950 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f786c009cd0 tx=0x7f786c02a8e0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 -- 172.21.15.86:0/189058289 shutdown_connections 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 --2- 172.21.15.86:0/189058289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874101420 0x7f787410d950 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 --2- 172.21.15.86:0/189058289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 0x7f7874100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 --2- 172.21.15.86:0/189058289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874106a80 0x7f7874106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 -- 172.21.15.86:0/189058289 >> 172.21.15.86:0/189058289 conn(0x7f78740fc7a0 msgr2=0x7f78740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 -- 172.21.15.86:0/189058289 shutdown_connections 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.243+0000 7f787a724640 1 -- 172.21.15.86:0/189058289 wait complete. 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 Processor -- start 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 -- start start 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 0x7f7874199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874101420 0x7f78741998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7874110dd0 con 0x7f7874106a80 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7874110c50 con 0x7f7874100a80 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f787a724640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7874110f50 con 0x7f7874101420 2024-09-05T06:58:03.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49906/0 (socket says 172.21.15.86:49906) 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 -- 172.21.15.86:0/1326890182 learned_addr learned my addr 172.21.15.86:0/1326890182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7873fff640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 0x7f7874199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f786bfff640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874101420 0x7f78741998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 -- 172.21.15.86:0/1326890182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874101420 msgr2=0x7f78741998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874101420 0x7f78741998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 -- 172.21.15.86:0/1326890182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 msgr2=0x7f7874199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 0x7f7874199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 -- 172.21.15.86:0/1326890182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741a4920 con 0x7f7874106a80 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f786c005cc0 tx=0x7f786c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.244+0000 7f7871ffb640 1 -- 172.21.15.86:0/1326890182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c004240 con 0x7f7874106a80 2024-09-05T06:58:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.245+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741a4af0 con 0x7f7874106a80 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.245+0000 7f7871ffb640 1 -- 172.21.15.86:0/1326890182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f786c0305a0 con 0x7f7874106a80 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.245+0000 7f7871ffb640 1 -- 172.21.15.86:0/1326890182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0466c0 con 0x7f7874106a80 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.245+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741a4fd0 con 0x7f7874106a80 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f7871ffb640 1 -- 172.21.15.86:0/1326890182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f786c030ac0 con 0x7f7874106a80 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 Processor -- start 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 -- start start 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad07d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad07da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8acfe32f0 con 0x55f8ad088370 2024-09-05T06:58:03.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8acf8acf0 con 0x55f8ad07eb40 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f787cafd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8acfdf2b0 con 0x55f8ad07d5d0 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f786bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f786bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49910/0 (socket says 172.21.15.86:49910) 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f786bfff640 1 -- 172.21.15.86:0/2045461644 learned_addr learned my addr 172.21.15.86:0/2045461644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f786bfff640 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad07d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.247+0000 7f7873fff640 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad07da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 -- 172.21.15.86:0/2045461644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 msgr2=0x55f8ad07da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad07da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 -- 172.21.15.86:0/2045461644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 msgr2=0x55f8ad07d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad07d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 -- 172.21.15.86:0/2045461644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8acf6f3a0 con 0x55f8ad088370 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f7871ffb640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 0x7f78600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786bfff640 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f785c016530 tx=0x7f785c00a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b189f206e8d04ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f7873fff640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 0x7f78600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f7871ffb640 1 -- 172.21.15.86:0/1326890182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f786c02e030 con 0x7f7874106a80 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786a7fc640 1 -- 172.21.15.86:0/2045461644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c018430 con 0x55f8ad088370 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786a7fc640 1 -- 172.21.15.86:0/2045461644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f785c018a80 con 0x55f8ad088370 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.248+0000 7f786a7fc640 1 -- 172.21.15.86:0/2045461644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0179a0 con 0x55f8ad088370 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- 172.21.15.86:0/2045461644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 msgr2=0x55f8ad08a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f7873fff640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 0x7f78600a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55f8ad07aac0 tx=0x7f7864008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f785c016530 tx=0x7f785c00a2b0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- 172.21.15.86:0/2045461644 shutdown_connections 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad08a760 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad07da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 --2- 172.21.15.86:0/2045461644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad07d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- 172.21.15.86:0/2045461644 >> 172.21.15.86:0/2045461644 conn(0x55f8ad046310 msgr2=0x55f8ad079c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- 172.21.15.86:0/2045461644 shutdown_connections 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- 172.21.15.86:0/2045461644 wait complete. 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 -- start start 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.249+0000 7f787cafd740 1 Processor -- start 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad085010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad088370 0x55f8ad085600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad08afb0 con 0x55f8ad07d5d0 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8acf9e060 con 0x55f8ad07eb40 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f8ad08b130 con 0x55f8ad088370 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 10 client.0 _ll_register_callbacks cb 0x55f8accf18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49916/0 (socket says 172.21.15.86:49916) 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 -- 172.21.15.86:0/2760346260 learned_addr learned my addr 172.21.15.86:0/2760346260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7878cc1640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad088370 0x55f8ad085600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f786bfff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad085010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 -- 172.21.15.86:0/2760346260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad088370 msgr2=0x55f8ad085600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad088370 0x55f8ad085600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 -- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 msgr2=0x55f8ad085010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad085010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 -- 172.21.15.86:0/2760346260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8acd543c0 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f7873fff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7864014710 tx=0x7f78640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9ce4ff2e8a665d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f78537fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786401b050 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f7864015820 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c018a80 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.250+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f786402e050 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 10 client.19263 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f8ad096b60 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f785c018a80 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f7864020ba0 con 0x55f8ad07d5d0 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 10 client.19263 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 10 client.19263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:58:03.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 20 client.19263 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad0980e0 0x55f8ad09a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 4 client.19263 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8acff0860 con 0x55f8ad0980e0 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.251+0000 7f786bfff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad0980e0 0x55f8ad09a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f786bfff640 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad0980e0 0x55f8ad09a530 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7948c1cc4a356f44 server_cookie=81534d7302b9517 in_seq=0 out_seq=0 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f78537fe640 10 client.19263 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f78537fe640 1 -- 172.21.15.86:0/2760346260 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8acff0860 con 0x55f8ad0980e0 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f78537fe640 10 client.19263 handle_command_reply: tid=0 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f787cafd740 1 client.19263 shutdown 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 msgr2=0x55f8ad087d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.252+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7864014710 tx=0x7f78640046c0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 shutdown_connections 2024-09-05T06:58:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad0980e0 0x55f8ad09a530 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad088370 0x55f8ad085600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07eb40 0x55f8ad085010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f78537fe640 0 client.19263 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 --2- 172.21.15.86:0/2760346260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07d5d0 0x55f8ad087d60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 >> 172.21.15.86:0/2760346260 conn(0x55f8ad046310 msgr2=0x55f8ad07f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 shutdown_connections 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 1 -- 172.21.15.86:0/2760346260 wait complete. 2024-09-05T06:58:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.253+0000 7f787cafd740 20 client.19263 trim_cache size 0 max 16384 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 Processor -- start 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 -- start start 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07d5d0 0x55f8ad0942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07eb40 0x55f8ad09fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78741af170 con 0x55f8ad088370 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78741b0d10 con 0x55f8ad07d5d0 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f787cafd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78741af840 con 0x55f8ad07eb40 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49920/0 (socket says 172.21.15.86:49920) 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 -- 172.21.15.86:0/1242706803 learned_addr learned my addr 172.21.15.86:0/1242706803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7873fff640 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07d5d0 0x55f8ad0942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f786bfff640 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07eb40 0x55f8ad09fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 -- 172.21.15.86:0/1242706803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07eb40 msgr2=0x55f8ad09fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07eb40 0x55f8ad09fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 -- 172.21.15.86:0/1242706803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07d5d0 msgr2=0x55f8ad0942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07d5d0 0x55f8ad0942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 -- 172.21.15.86:0/1242706803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ad079d70 con 0x55f8ad088370 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.266+0000 7f7878cc1640 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f786c0bcce0 tx=0x7f786c0bc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60a025f9e1f4c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f7869ffb640 1 -- 172.21.15.86:0/1242706803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0d5020 con 0x55f8ad088370 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f7869ffb640 1 -- 172.21.15.86:0/1242706803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f786c036040 con 0x55f8ad088370 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f7869ffb640 1 -- 172.21.15.86:0/1242706803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0b9650 con 0x55f8ad088370 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 -- 172.21.15.86:0/1242706803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 msgr2=0x55f8ad0a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f786c0bcce0 tx=0x7f786c0bc940 comp rx=0 tx=0).stop 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 -- 172.21.15.86:0/1242706803 shutdown_connections 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad088370 0x55f8ad0a0380 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07eb40 0x55f8ad09fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 --2- 172.21.15.86:0/1242706803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad07d5d0 0x55f8ad0942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 -- 172.21.15.86:0/1242706803 >> 172.21.15.86:0/1242706803 conn(0x55f8ad046310 msgr2=0x55f8ad084160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 -- 172.21.15.86:0/1242706803 shutdown_connections 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.267+0000 7f787cafd740 1 -- 172.21.15.86:0/1242706803 wait complete. 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 -- start start 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 Processor -- start 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad090420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 0x55f8ad08d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78741b0640 con 0x55f8ad07eb40 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78741aff70 con 0x55f8ad088370 2024-09-05T06:58:03.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f78741ad170 con 0x55f8ad07d5d0 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f787cafd740 10 client.0 _ll_register_callbacks cb 0x7f786400b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f786bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f786bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49924/0 (socket says 172.21.15.86:49924) 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f786bfff640 1 -- 172.21.15.86:0/17091226 learned_addr learned my addr 172.21.15.86:0/17091226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.268+0000 7f7878cc1640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 0x55f8ad08d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f7873fff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad090420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 -- 172.21.15.86:0/17091226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 msgr2=0x55f8ad090420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad090420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 -- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 msgr2=0x55f8ad08d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 0x55f8ad08d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 -- 172.21.15.86:0/17091226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ad083000 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f7878cc1640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 0x55f8ad08d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f7873fff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad090420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f786bfff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f785c01a040 tx=0x7f785c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4a18cc361fa89d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c032060 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c036040 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f785c018d30 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f785c039050 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f787cafd740 10 client.19269 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f8ad1077a0 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f786c036040 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.269+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f785c012050 con 0x55f8ad07eb40 2024-09-05T06:58:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 10 client.19269 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 10 client.19269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 20 client.19269 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad107c30 0x55f8ad10a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 4 client.19269 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8acfead70 con 0x55f8ad107c30 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f7873fff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad107c30 0x55f8ad10a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f7873fff640 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad107c30 0x55f8ad10a050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=156753438f79f39e server_cookie=75a2958533844a00 in_seq=0 out_seq=0 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f78537fe640 10 client.19269 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f78537fe640 1 -- 172.21.15.86:0/17091226 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8acfead70 con 0x55f8ad107c30 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f78537fe640 10 client.19269 handle_command_reply: tid=0 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.270+0000 7f787cafd740 1 client.19269 shutdown 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 msgr2=0x55f8ad090960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f785c01a040 tx=0x7f785c0046c0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 shutdown_connections 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55f8ad107c30 0x55f8ad10a050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8ad088370 0x55f8ad08d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8ad07eb40 0x55f8ad090960 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 --2- 172.21.15.86:0/17091226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8ad07d5d0 0x55f8ad090420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 >> 172.21.15.86:0/17091226 conn(0x55f8ad046310 msgr2=0x55f8ad07f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f78537fe640 0 client.19269 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 shutdown_connections 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 1 -- 172.21.15.86:0/17091226 wait complete. 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787cafd740 20 client.19269 trim_cache size 0 max 16384 2024-09-05T06:58:03.285 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 msgr2=0x7f78600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 0x7f78600a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55f8ad07aac0 tx=0x7f7864008040 comp rx=0 tx=0).stop 2024-09-05T06:58:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 msgr2=0x7f78741a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f786c005cc0 tx=0x7f786c003ba0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 shutdown_connections 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f78600a2cf0 0x7f78600a51b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7874106a80 0x7f78741a43e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7874101420 0x7f78741998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.271+0000 7f787a724640 1 --2- 172.21.15.86:0/1326890182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7874100a80 0x7f7874199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.272+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 >> 172.21.15.86:0/1326890182 conn(0x7f78740fc7a0 msgr2=0x7f78740fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.272+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 shutdown_connections 2024-09-05T06:58:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:03.272+0000 7f787a724640 1 -- 172.21.15.86:0/1326890182 wait complete. 2024-09-05T06:58:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:03 smithi086 ceph-mon[30943]: pgmap v526: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-05T06:58:03.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:03 smithi115 ceph-mon[34791]: pgmap v526: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-05T06:58:03.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.61 MB/sec execute 81 sec latency 6939.585 ms 2024-09-05T06:58:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:03 smithi096 ceph-mon[34468]: pgmap v526: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-09-05T06:58:04.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:58:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:04.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.52 MB/sec execute 82 sec latency 7939.623 ms 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 -- 172.21.15.86:0/1986936809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 msgr2=0x7f5200100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 --2- 172.21.15.86:0/1986936809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200100ee0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009930 tx=0x7f51e802a420 comp rx=0 tx=0).stop 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 -- 172.21.15.86:0/1986936809 shutdown_connections 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 --2- 172.21.15.86:0/1986936809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 0x7f520010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 --2- 172.21.15.86:0/1986936809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200100ee0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 --2- 172.21.15.86:0/1986936809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 0x7f5200106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 -- 172.21.15.86:0/1986936809 >> 172.21.15.86:0/1986936809 conn(0x7f52000fc7a0 msgr2=0x7f52000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 -- 172.21.15.86:0/1986936809 shutdown_connections 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.745+0000 7f520481b640 1 -- 172.21.15.86:0/1986936809 wait complete. 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 Processor -- start 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 -- start start 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 0x7f52001988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 0x7f520019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110d30 con 0x7f5200100a80 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110bb0 con 0x7f5200106a80 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f520481b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110eb0 con 0x7f5200101420 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49936/0 (socket says 172.21.15.86:49936) 2024-09-05T06:58:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 -- 172.21.15.86:0/678193718 learned_addr learned my addr 172.21.15.86:0/678193718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fed9d640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 0x7f520019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51f7fff640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 0x7f52001988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 -- 172.21.15.86:0/678193718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 msgr2=0x7f52001988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 0x7f52001988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 -- 172.21.15.86:0/678193718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 msgr2=0x7f520019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 0x7f520019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 -- 172.21.15.86:0/678193718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520019d320 con 0x7f5200100a80 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.746+0000 7f51fe59c640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c890 tx=0x7f51f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.747+0000 7f51f77fe640 1 -- 172.21.15.86:0/678193718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004220 con 0x7f5200100a80 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.747+0000 7f51f77fe640 1 -- 172.21.15.86:0/678193718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51f00043c0 con 0x7f5200100a80 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.747+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520019d610 con 0x7f5200100a80 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.747+0000 7f51f77fe640 1 -- 172.21.15.86:0/678193718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004a00 con 0x7f5200100a80 2024-09-05T06:58:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.747+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a4ef0 con 0x7f5200100a80 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f51f77fe640 1 -- 172.21.15.86:0/678193718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51f0004ba0 con 0x7f5200100a80 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f5206bf4740 1 Processor -- start 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f5206bf4740 1 -- start start 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e2515d0 0x55c21e251a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.749+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e25c370 0x55c21e25e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f5206bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c21e1b72f0 con 0x55c21e252b40 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f5206bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c21e15ecf0 con 0x55c21e2515d0 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f5206bf4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c21e1b32b0 con 0x55c21e25c370 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51f77fe640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 0x7f51ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49938/0 (socket says 172.21.15.86:49938) 2024-09-05T06:58:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 -- 172.21.15.86:0/3032202037 learned_addr learned my addr 172.21.15.86:0/3032202037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fe59c640 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e2515d0 0x55c21e251a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51f7fff640 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e25c370 0x55c21e25e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 0x7f51ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51f77fe640 1 -- 172.21.15.86:0/678193718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f51f00cf4d0 con 0x7f5200100a80 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 -- 172.21.15.86:0/3032202037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e25c370 msgr2=0x55c21e25e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e25c370 0x55c21e25e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 -- 172.21.15.86:0/3032202037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e2515d0 msgr2=0x55c21e251a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e2515d0 0x55c21e251a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 -- 172.21.15.86:0/3032202037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e1433a0 con 0x55c21e252b40 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51fed9d640 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f51f80139c0 tx=0x7f51f8013e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70a3d2826443027 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.750+0000 7f51f4ff9640 1 -- 172.21.15.86:0/3032202037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8021030 con 0x55c21e252b40 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f51f4ff9640 1 -- 172.21.15.86:0/3032202037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51f8011c80 con 0x55c21e252b40 2024-09-05T06:58:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f51f4ff9640 1 -- 172.21.15.86:0/3032202037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8012450 con 0x55c21e252b40 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f51fed9d640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 0x7f51ec0a51b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51f8021000 tx=0x7f51f8020000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 -- 172.21.15.86:0/3032202037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 msgr2=0x55c21e251090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f51f80139c0 tx=0x7f51f8013e80 comp rx=0 tx=0).stop 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 -- 172.21.15.86:0/3032202037 shutdown_connections 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e25c370 0x55c21e25e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e2515d0 0x55c21e251a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3032202037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e251090 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 -- 172.21.15.86:0/3032202037 >> 172.21.15.86:0/3032202037 conn(0x55c21e21a310 msgr2=0x55c21e24dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 -- 172.21.15.86:0/3032202037 shutdown_connections 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.751+0000 7f5206bf4740 1 -- 172.21.15.86:0/3032202037 wait complete. 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 -- start start 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 Processor -- start 2024-09-05T06:58:04.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e25bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e252b40 0x55c21e259040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c21e263480 con 0x55c21e25c370 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c21e172060 con 0x55c21e252b40 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c21e263600 con 0x55c21e2515d0 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f5206bf4740 10 client.0 _ll_register_callbacks cb 0x55c21dec58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49948/0 (socket says 172.21.15.86:49948) 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 -- 172.21.15.86:0/81896247 learned_addr learned my addr 172.21.15.86:0/81896247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fe59c640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e25bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51f7fff640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e252b40 0x55c21e259040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 -- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 msgr2=0x55c21e25bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e25bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 -- 172.21.15.86:0/81896247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e252b40 msgr2=0x55c21e259040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e252b40 0x55c21e259040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.752+0000 7f51fed9d640 1 -- 172.21.15.86:0/81896247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21df283c0 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51fed9d640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004570 tx=0x7f51f800a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed36ba75133f8a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8013260 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f51f8013400 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f8011c80 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f51f8012770 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 10 client.19281 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c21e26ef50 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51f8011c80 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f51f8031540 con 0x55c21e25c370 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 10 client.19281 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 10 client.19281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:58:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 20 client.19281 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2703b0 0x55c21e2727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 4 client.19281 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.753+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21e1c4860 con 0x55c21e2703b0 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.754+0000 7f51fe59c640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2703b0 0x55c21e2727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.754+0000 7f51fe59c640 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2703b0 0x55c21e2727a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2285a64cdebbde52 server_cookie=3054ef96ef283b86 in_seq=0 out_seq=0 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.754+0000 7f51e5ffb640 10 client.19281 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f51e5ffb640 1 -- 172.21.15.86:0/81896247 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c21e1c4860 con 0x55c21e2703b0 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f51e5ffb640 10 client.19281 handle_command_reply: tid=0 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 client.19281 shutdown 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 msgr2=0x55c21e259650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004570 tx=0x7f51f800a670 comp rx=0 tx=0).stop 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 shutdown_connections 2024-09-05T06:58:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2703b0 0x55c21e2727a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e25c370 0x55c21e259650 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e252b40 0x55c21e259040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 --2- 172.21.15.86:0/81896247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e25bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 >> 172.21.15.86:0/81896247 conn(0x55c21e21a310 msgr2=0x55c21e253960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f51e5ffb640 0 client.19281 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 shutdown_connections 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 1 -- 172.21.15.86:0/81896247 wait complete. 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.755+0000 7f5206bf4740 20 client.19281 trim_cache size 0 max 16384 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 Processor -- start 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 -- start start 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e26c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e254120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52001acf80 con 0x55c21e252b40 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52001aeb20 con 0x55c21e25c370 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f5206bf4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52001ad650 con 0x55c21e2515d0 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49952/0 (socket says 172.21.15.86:49952) 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 -- 172.21.15.86:0/645428636 learned_addr learned my addr 172.21.15.86:0/645428636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51fe59c640 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e26c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 -- 172.21.15.86:0/645428636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 msgr2=0x55c21e26c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e26c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 -- 172.21.15.86:0/645428636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 msgr2=0x55c21e254120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e254120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51f7fff640 1 -- 172.21.15.86:0/645428636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e24e960 con 0x55c21e252b40 2024-09-05T06:58:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.768+0000 7f51fe59c640 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e26c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f51f7fff640 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009930 tx=0x7f51e80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d158036c8eef46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f51e7fff640 1 -- 172.21.15.86:0/645428636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8035040 con 0x55c21e252b40 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f51e7fff640 1 -- 172.21.15.86:0/645428636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51e8004030 con 0x55c21e252b40 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f51e7fff640 1 -- 172.21.15.86:0/645428636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80047a0 con 0x55c21e252b40 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 -- 172.21.15.86:0/645428636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 msgr2=0x55c21e257910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009930 tx=0x7f51e80314a0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 -- 172.21.15.86:0/645428636 shutdown_connections 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e254120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e252b40 0x55c21e257910 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 --2- 172.21.15.86:0/645428636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e2515d0 0x55c21e26c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 -- 172.21.15.86:0/645428636 >> 172.21.15.86:0/645428636 conn(0x55c21e21a310 msgr2=0x55c21e25b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 -- 172.21.15.86:0/645428636 shutdown_connections 2024-09-05T06:58:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.769+0000 7f5206bf4740 1 -- 172.21.15.86:0/645428636 wait complete. 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 -- start start 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 Processor -- start 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 0x55c21e263d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e2642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52001ae450 con 0x55c21e2515d0 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52001add80 con 0x55c21e25c370 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52001aaf80 con 0x55c21e252b40 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f5206bf4740 10 client.0 _ll_register_callbacks cb 0x7f51f00058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49962/0 (socket says 172.21.15.86:49962) 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f51fe59c640 1 -- 172.21.15.86:0/3512878736 learned_addr learned my addr 172.21.15.86:0/3512878736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f51fed9d640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e2642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.770+0000 7f51f7fff640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 0x55c21e263d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 -- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 msgr2=0x55c21e263d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 0x55c21e263d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 -- 172.21.15.86:0/3512878736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 msgr2=0x55c21e2642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e2642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 -- 172.21.15.86:0/3512878736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21e2d90a0 con 0x55c21e2515d0 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fed9d640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e2642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51f7fff640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 0x55c21e263d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51fe59c640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cc7e0 tx=0x7f51f00c60e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f68012eecad1f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f001d040 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7f51f00050b0 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e8004030 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f51f0097040 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 10 client.19287 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c21e2da2f0 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51e8004030 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f51f00cf120 con 0x55c21e2515d0 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 10 client.19287 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 10 client.19287 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:58:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 20 client.19287 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2da9d0 0x55c21e2dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 4 client.19287 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.771+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c21e1bed70 con 0x55c21e2da9d0 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51f7fff640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2da9d0 0x55c21e2dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51e5ffb640 10 client.19287 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51f7fff640 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2da9d0 0x55c21e2dcdf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc150fec253f6dfb server_cookie=732646395f01777b in_seq=0 out_seq=0 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51e5ffb640 1 -- 172.21.15.86:0/3512878736 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c21e1bed70 con 0x55c21e2da9d0 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51e5ffb640 10 client.19287 handle_command_reply: tid=0 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 client.19287 shutdown 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 msgr2=0x55c21e263850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cc7e0 tx=0x7f51f00c60e0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 shutdown_connections 2024-09-05T06:58:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c21e2da9d0 0x55c21e2dcdf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c21e25c370 0x55c21e2642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c21e252b40 0x55c21e263d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 --2- 172.21.15.86:0/3512878736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c21e2515d0 0x55c21e263850 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f51e5ffb640 0 client.19287 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 >> 172.21.15.86:0/3512878736 conn(0x55c21e21a310 msgr2=0x55c21e24c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 shutdown_connections 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.772+0000 7f5206bf4740 1 -- 172.21.15.86:0/3512878736 wait complete. 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f5206bf4740 20 client.19287 trim_cache size 0 max 16384 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 msgr2=0x7f51ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 0x7f51ec0a51b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51f8021000 tx=0x7f51f8020000 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 msgr2=0x7f5200198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c890 tx=0x7f51f000cd50 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 shutdown_connections 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51ec0a2cf0 0x7f51ec0a51b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5200106a80 0x7f520019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5200101420 0x7f52001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 --2- 172.21.15.86:0/678193718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200100a80 0x7f5200198370 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 >> 172.21.15.86:0/678193718 conn(0x7f52000fc7a0 msgr2=0x7f5200104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 shutdown_connections 2024-09-05T06:58:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:04.773+0000 7f520481b640 1 -- 172.21.15.86:0/678193718 wait complete. 2024-09-05T06:58:05.577 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:05 smithi086 ceph-mon[30943]: pgmap v527: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:05.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:58:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:05 smithi115 ceph-mon[34791]: pgmap v527: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:05.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.43 MB/sec execute 83 sec latency 8939.658 ms 2024-09-05T06:58:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:05 smithi096 ceph-mon[34468]: pgmap v527: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 -- 172.21.15.86:0/4131394920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 msgr2=0x7fd9c810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/4131394920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c810d960 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009cd0 tx=0x7fd9c002a8e0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 -- 172.21.15.86:0/4131394920 shutdown_connections 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/4131394920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c810d960 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/4131394920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/4131394920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 -- 172.21.15.86:0/4131394920 >> 172.21.15.86:0/4131394920 conn(0x7fd9c80fc7d0 msgr2=0x7fd9c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 -- 172.21.15.86:0/4131394920 shutdown_connections 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.379+0000 7fd9cf48c640 1 -- 172.21.15.86:0/4131394920 wait complete. 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 Processor -- start 2024-09-05T06:58:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 -- start start 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8110d60 con 0x7fd9c8101430 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8110be0 con 0x7fd9c8106a90 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cf48c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8110ee0 con 0x7fd9c8100a90 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49986/0 (socket says 172.21.15.86:49986) 2024-09-05T06:58:06.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 -- 172.21.15.86:0/3411853637 learned_addr learned my addr 172.21.15.86:0/3411853637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cda29640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cd228640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 -- 172.21.15.86:0/3411853637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 msgr2=0x7fd9c8197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 -- 172.21.15.86:0/3411853637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 msgr2=0x7fd9c819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cca27640 1 -- 172.21.15.86:0/3411853637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c819c230 con 0x7fd9c8101430 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cd228640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.380+0000 7fd9cda29640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9cca27640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400b4a0 tx=0x7fd9c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9ba7fc640 1 -- 172.21.15.86:0/3411853637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4014070 con 0x7fd9c8101430 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9ba7fc640 1 -- 172.21.15.86:0/3411853637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c4010ac0 con 0x7fd9c8101430 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c819c520 con 0x7fd9c8101430 2024-09-05T06:58:06.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9ba7fc640 1 -- 172.21.15.86:0/3411853637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4017c30 con 0x7fd9c8101430 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.381+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a3e00 con 0x7fd9c8101430 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9ba7fc640 1 -- 172.21.15.86:0/3411853637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9c4017070 con 0x7fd9c8101430 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 Processor -- start 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 -- start start 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efdbb40 0x55fa7efda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efda5d0 0x55fa7efdaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ef402f0 con 0x55fa7efe7580 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa7eee7cf0 con 0x55fa7efdbb40 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9d1865740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ef3c2b0 con 0x55fa7efda5d0 2024-09-05T06:58:06.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50000/0 (socket says 172.21.15.86:50000) 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9cca27640 1 -- 172.21.15.86:0/630934560 learned_addr learned my addr 172.21.15.86:0/630934560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.384+0000 7fd9cda29640 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efdbb40 0x55fa7efda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cd228640 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efda5d0 0x55fa7efdaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 -- 172.21.15.86:0/630934560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efda5d0 msgr2=0x55fa7efdaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efda5d0 0x55fa7efdaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 -- 172.21.15.86:0/630934560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efdbb40 msgr2=0x55fa7efda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efdbb40 0x55fa7efda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 -- 172.21.15.86:0/630934560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7eecc3a0 con 0x55fa7efe7580 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9ba7fc640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 0x7fd9b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cca27640 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cc290 tx=0x7fd9c40d7220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e222e0e6b72eb039 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9cda29640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 0x7fd9b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9ba7fc640 1 -- 172.21.15.86:0/3411853637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fd9c40cef00 con 0x7fd9c8101430 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9b37fe640 1 -- 172.21.15.86:0/630934560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4097390 con 0x55fa7efe7580 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9b37fe640 1 -- 172.21.15.86:0/630934560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c409b7f0 con 0x55fa7efe7580 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.385+0000 7fd9b37fe640 1 -- 172.21.15.86:0/630934560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c409aa50 con 0x55fa7efe7580 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- 172.21.15.86:0/630934560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 msgr2=0x55fa7efe9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9cda29640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 0x7fd9b40a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005fe0 tx=0x7fd9c0005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cc290 tx=0x7fd9c40d7220 comp rx=0 tx=0).stop 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- 172.21.15.86:0/630934560 shutdown_connections 2024-09-05T06:58:06.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efe9970 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efda5d0 0x55fa7efdaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 --2- 172.21.15.86:0/630934560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efdbb40 0x55fa7efda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- 172.21.15.86:0/630934560 >> 172.21.15.86:0/630934560 conn(0x55fa7efa3310 msgr2=0x55fa7efd6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- 172.21.15.86:0/630934560 shutdown_connections 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- 172.21.15.86:0/630934560 wait complete. 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 -- start start 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.386+0000 7fd9d1865740 1 Processor -- start 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7efe6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7efe47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa7efea220 con 0x55fa7efdbb40 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa7eefb060 con 0x55fa7efda5d0 2024-09-05T06:58:06.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fa7efea3a0 con 0x55fa7efe7580 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9d1865740 10 client.0 _ll_register_callbacks cb 0x55fa7ec4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50010/0 (socket says 172.21.15.86:50010) 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 -- 172.21.15.86:0/2107907413 learned_addr learned my addr 172.21.15.86:0/2107907413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cd228640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7efe6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cda29640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7efe47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 -- 172.21.15.86:0/2107907413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 msgr2=0x55fa7efe47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7efe47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 -- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 msgr2=0x55fa7efe6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7efe6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 -- 172.21.15.86:0/2107907413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7ecb13c0 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cda29640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7efe47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9cca27640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cc810 tx=0x7fd9c40d2610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2389503b8c5fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.387+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401e050 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9605+0+0 (secure 0 0 0) 0x7fd9c40d2b80 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c409b7f0 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9c409ac30 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 10 client.19299 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa7efea220 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c409b7f0 con 0x55fa7efdbb40 2024-09-05T06:58:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fd9c40b5080 con 0x55fa7efdbb40 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 10 client.19299 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 10 client.19299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 20 client.19299 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7eff6e20 0x55fa7eff9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 4 client.19299 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7ef4d860 con 0x55fa7eff6e20 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.388+0000 7fd9cd228640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7eff6e20 0x55fa7eff9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.389+0000 7fd9b0ff9640 10 client.19299 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.389+0000 7fd9cd228640 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7eff6e20 0x55fa7eff9210 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f280c393fd510faa server_cookie=35acebe480b2d7e8 in_seq=0 out_seq=0 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.389+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2107907413 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa7ef4d860 con 0x55fa7eff6e20 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.389+0000 7fd9b0ff9640 10 client.19299 handle_command_reply: tid=0 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.389+0000 7fd9d1865740 1 client.19299 shutdown 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 msgr2=0x55fa7efe4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cc810 tx=0x7fd9c40d2610 comp rx=0 tx=0).stop 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 shutdown_connections 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7eff6e20 0x55fa7eff9210 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7efe47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9b0ff9640 0 client.19299 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe4210 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2107907413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7efe6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 >> 172.21.15.86:0/2107907413 conn(0x55fa7efa3310 msgr2=0x55fa7efdc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 shutdown_connections 2024-09-05T06:58:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 1 -- 172.21.15.86:0/2107907413 wait complete. 2024-09-05T06:58:06.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.390+0000 7fd9d1865740 20 client.19299 trim_cache size 0 max 16384 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 Processor -- start 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 -- start start 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7eff34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7effe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81acf90 con 0x55fa7efdbb40 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81aeb30 con 0x55fa7efda5d0 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.409+0000 7fd9d1865740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81ad660 con 0x55fa7efe7580 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50020/0 (socket says 172.21.15.86:50020) 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 -- 172.21.15.86:0/169883063 learned_addr learned my addr 172.21.15.86:0/169883063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cd228640 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7eff34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 -- 172.21.15.86:0/169883063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 msgr2=0x55fa7effe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7effe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 -- 172.21.15.86:0/169883063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 msgr2=0x55fa7eff34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7eff34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 -- 172.21.15.86:0/169883063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7efe41b0 con 0x55fa7efdbb40 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.410+0000 7fd9cca27640 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd9c409adf0 tx=0x7fd9c4017660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1495dd43a38bd6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.411+0000 7fd9b2ffd640 1 -- 172.21.15.86:0/169883063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401e070 con 0x55fa7efdbb40 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.411+0000 7fd9b2ffd640 1 -- 172.21.15.86:0/169883063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c40b40c0 con 0x55fa7efdbb40 2024-09-05T06:58:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.411+0000 7fd9b2ffd640 1 -- 172.21.15.86:0/169883063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c409aec0 con 0x55fa7efdbb40 2024-09-05T06:58:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.412+0000 7fd9d1865740 1 -- 172.21.15.86:0/169883063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 msgr2=0x55fa7efe31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.412+0000 7fd9d1865740 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd9c409adf0 tx=0x7fd9c4017660 comp rx=0 tx=0).stop 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 -- 172.21.15.86:0/169883063 shutdown_connections 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efe7580 0x55fa7effe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efdbb40 0x55fa7efe31f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 --2- 172.21.15.86:0/169883063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7eff34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 -- 172.21.15.86:0/169883063 >> 172.21.15.86:0/169883063 conn(0x55fa7efa3310 msgr2=0x55fa7efe2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 -- 172.21.15.86:0/169883063 shutdown_connections 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.413+0000 7fd9d1865740 1 -- 172.21.15.86:0/169883063 wait complete. 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 -- start start 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 Processor -- start 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7f05d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efdbb40 0x55fa7efef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81ae460 con 0x55fa7efe7580 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81add90 con 0x55fa7efda5d0 2024-09-05T06:58:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81aaf90 con 0x55fa7efdbb40 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.414+0000 7fd9d1865740 10 client.0 _ll_register_callbacks cb 0x7fd9c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50022/0 (socket says 172.21.15.86:50022) 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 -- 172.21.15.86:0/2136203140 learned_addr learned my addr 172.21.15.86:0/2136203140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cca27640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efdbb40 0x55fa7efef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 -- 172.21.15.86:0/2136203140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efdbb40 msgr2=0x55fa7efef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efdbb40 0x55fa7efef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 -- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 msgr2=0x55fa7f05d430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7f05d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.415+0000 7fd9cda29640 1 -- 172.21.15.86:0/2136203140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7eff0200 con 0x55fa7efe7580 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.416+0000 7fd9cda29640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009cd0 tx=0x7fd9c00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23122ced703eeac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.416+0000 7fd9b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.416+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0056060 con 0x55fa7efe7580 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.416+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c40b40c0 con 0x55fa7efe7580 2024-09-05T06:58:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.416+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7fd9c0041030 con 0x55fa7efe7580 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.417+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9c005d050 con 0x55fa7efe7580 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.417+0000 7fd9d1865740 10 client.19305 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.417+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa7f079500 con 0x55fa7efe7580 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.418+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c40b40c0 con 0x55fa7efe7580 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.418+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fd9c0030070 con 0x55fa7efe7580 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 10 client.19305 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 10 client.19305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 20 client.19305 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7f079c60 0x55fa7f07c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 4 client.19305 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.419+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa7ef47d70 con 0x55fa7f079c60 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.421+0000 7fd9cd228640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7f079c60 0x55fa7f07c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.424+0000 7fd9cd228640 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7f079c60 0x55fa7f07c0c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec87fd5a75f8506b server_cookie=7af5e29f5e0ffad in_seq=0 out_seq=0 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.424+0000 7fd9b0ff9640 10 client.19305 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.424+0000 7fd9b0ff9640 1 -- 172.21.15.86:0/2136203140 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa7ef47d70 con 0x55fa7f079c60 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.424+0000 7fd9b0ff9640 10 client.19305 handle_command_reply: tid=0 2024-09-05T06:58:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.424+0000 7fd9d1865740 1 client.19305 shutdown 2024-09-05T06:58:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.426+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 msgr2=0x55fa7efefb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.426+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009cd0 tx=0x7fd9c00039c0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 shutdown_connections 2024-09-05T06:58:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fa7f079c60 0x55fa7f07c0c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa7efe7580 0x55fa7efefb60 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa7efdbb40 0x55fa7efef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 --2- 172.21.15.86:0/2136203140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa7efda5d0 0x55fa7f05d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 >> 172.21.15.86:0/2136203140 conn(0x55fa7efa3310 msgr2=0x55fa7efd6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9b0ff9640 0 client.19305 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 shutdown_connections 2024-09-05T06:58:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.432+0000 7fd9d1865740 1 -- 172.21.15.86:0/2136203140 wait complete. 2024-09-05T06:58:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.433+0000 7fd9d1865740 20 client.19305 trim_cache size 0 max 16384 2024-09-05T06:58:06.435 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.434+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 msgr2=0x7fd9b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.434+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 0x7fd9b40a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005fe0 tx=0x7fd9c0005c50 comp rx=0 tx=0).stop 2024-09-05T06:58:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.434+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 msgr2=0x7fd9c81977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.434+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400b4a0 tx=0x7fd9c400b960 comp rx=0 tx=0).stop 2024-09-05T06:58:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 shutdown_connections 2024-09-05T06:58:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd9b40a2cf0 0x7fd9b40a51b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8106a90 0x7fd9c819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8101430 0x7fd9c81977c0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 --2- 172.21.15.86:0/3411853637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8100a90 0x7fd9c8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 >> 172.21.15.86:0/3411853637 conn(0x7fd9c80fc7d0 msgr2=0x7fd9c80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 shutdown_connections 2024-09-05T06:58:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:06.435+0000 7fd9cf48c640 1 -- 172.21.15.86:0/3411853637 wait complete. 2024-09-05T06:58:06.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.34 MB/sec execute 84 sec latency 9939.702 ms 2024-09-05T06:58:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:07 smithi086 ceph-mon[30943]: pgmap v528: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:07 smithi086 ceph-mon[30943]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T06:58:07.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:07 smithi115 ceph-mon[34791]: pgmap v528: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:07 smithi115 ceph-mon[34791]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T06:58:07.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.25 MB/sec execute 85 sec latency 10939.780 ms 2024-09-05T06:58:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:07 smithi096 ceph-mon[34468]: pgmap v528: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-09-05T06:58:07.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:07 smithi096 ceph-mon[34468]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T06:58:08.290 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:58:08.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:58:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:08.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.17 MB/sec execute 86 sec latency 11939.812 ms 2024-09-05T06:58:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.117+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/2189902123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581036e0 msgr2=0x7f1f58107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/2189902123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581036e0 0x7f1f58107b90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009930 tx=0x7f1f4802a420 comp rx=0 tx=0).stop 2024-09-05T06:58:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/2189902123 shutdown_connections 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/2189902123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f581082a0 0x7f1f5810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/2189902123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581036e0 0x7f1f58107b90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/2189902123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f58102d30 0x7f1f58103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/2189902123 >> 172.21.15.86:0/2189902123 conn(0x7f1f580fc7d0 msgr2=0x7f1f580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/2189902123 shutdown_connections 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/2189902123 wait complete. 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.118+0000 7f1f5d2b1640 1 Processor -- start 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 -- start start 2024-09-05T06:58:09.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 0x7f1f58198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581036e0 0x7f1f58198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58111f70 con 0x7f1f581082a0 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58111df0 con 0x7f1f581036e0 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f5d2b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581120f0 con 0x7f1f58102d30 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50034/0 (socket says 172.21.15.86:50034) 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 -- 172.21.15.86:0/1913241398 learned_addr learned my addr 172.21.15.86:0/1913241398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f567fc640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581036e0 0x7f1f58198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 0x7f1f58198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 -- 172.21.15.86:0/1913241398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 msgr2=0x7f1f58198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 0x7f1f58198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 -- 172.21.15.86:0/1913241398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581036e0 msgr2=0x7f1f58198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.119+0000 7f1f577fe640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581036e0 0x7f1f58198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f577fe640 1 -- 172.21.15.86:0/1913241398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5819d4e0 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 0x7f1f58198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f577fe640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b970 tx=0x7f1f5000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f4ffff640 1 -- 172.21.15.86:0/1913241398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50019070 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f4ffff640 1 -- 172.21.15.86:0/1913241398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f50014440 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f4ffff640 1 -- 172.21.15.86:0/1913241398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500075d0 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5819d7d0 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.120+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a50b0 con 0x7f1f581082a0 2024-09-05T06:58:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f4ffff640 1 -- 172.21.15.86:0/1913241398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1f5001c070 con 0x7f1f581082a0 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 Processor -- start 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 -- start start 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eaef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ebc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f05e14280 con 0x555f05eb9d40 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f05dbbb10 con 0x555f05eaf480 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.123+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f05e10240 con 0x555f05eaeb60 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ebc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:53282/0 (socket says 172.21.15.86:53282) 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 -- 172.21.15.86:0/411200181 learned_addr learned my addr 172.21.15.86:0/411200181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ebc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50044/0 (socket says 172.21.15.86:50044) 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f4ffff640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 0x7f1f440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f567fc640 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eaef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f4ffff640 1 -- 172.21.15.86:0/1913241398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f1f500ceda0 con 0x7f1f581082a0 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f577fe640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 0x7f1f440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 -- 172.21.15.86:0/411200181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 msgr2=0x555f05eaef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eaef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 -- 172.21.15.86:0/411200181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 msgr2=0x555f05ebc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ebc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 -- 172.21.15.86:0/411200181 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f05dc81f0 con 0x555f05eaf480 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f40008c60 tx=0x7f1f4000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9499e13db2f4cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f577fe640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 0x7f1f440a51b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f50097450 tx=0x7f1f50014b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.124+0000 7f1f4d7fa640 1 -- 172.21.15.86:0/411200181 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40014070 con 0x555f05eaf480 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f4d7fa640 1 -- 172.21.15.86:0/411200181 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f400047a0 con 0x555f05eaf480 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f4d7fa640 1 -- 172.21.15.86:0/411200181 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40010670 con 0x555f05eaf480 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 -- 172.21.15.86:0/411200181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 msgr2=0x555f05eb9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f40008c60 tx=0x7f1f4000e720 comp rx=0 tx=0).stop 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 -- 172.21.15.86:0/411200181 shutdown_connections 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ebc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb9800 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/411200181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eaef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 -- 172.21.15.86:0/411200181 >> 172.21.15.86:0/411200181 conn(0x555f05eab0f0 msgr2=0x555f05eab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.125+0000 7f1f5f68a740 1 -- 172.21.15.86:0/411200181 shutdown_connections 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 -- 172.21.15.86:0/411200181 wait complete. 2024-09-05T06:58:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 -- start start 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 Processor -- start 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eb6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f05ebc910 con 0x555f05eb9d40 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f05dcef10 con 0x555f05eaf480 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f05ebca90 con 0x555f05eaeb60 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.126+0000 7f1f5f68a740 10 client.0 _ll_register_callbacks cb 0x555f05d97a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50048/0 (socket says 172.21.15.86:50048) 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f567fc640 1 -- 172.21.15.86:0/3688365553 learned_addr learned my addr 172.21.15.86:0/3688365553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f567fc640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eb6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 -- 172.21.15.86:0/3688365553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 msgr2=0x555f05eb6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eb6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 -- 172.21.15.86:0/3688365553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 msgr2=0x555f05eb7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 -- 172.21.15.86:0/3688365553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f05b856a0 con 0x555f05eaf480 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f567fc640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb7100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f40008c30 tx=0x7f1f40005160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89c616e43122df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40014030 con 0x555f05eaf480 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f400047a0 con 0x555f05eaf480 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.127+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f1f40005560 con 0x555f05eaf480 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1f40004050 con 0x555f05eaf480 2024-09-05T06:58:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 10 client.28211 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f05ec84c0 con 0x555f05eaf480 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f400047a0 con 0x555f05eaf480 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1f40003a20 con 0x555f05eaf480 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 10 client.28211 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 10 client.28211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 20 client.28211 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05ec98d0 0x555f05ecbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 4 client.28211 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f05e217f0 con 0x555f05ec98d0 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.128+0000 7f1f567fc640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05ec98d0 0x555f05ecbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.129+0000 7f1f3e7fc640 10 client.28211 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.129+0000 7f1f567fc640 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05ec98d0 0x555f05ecbcc0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b0268260a36136 server_cookie=5d1b6dbcf316d5bc in_seq=0 out_seq=0 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.129+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/3688365553 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f05e217f0 con 0x555f05ec98d0 2024-09-05T06:58:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.129+0000 7f1f3e7fc640 10 client.28211 handle_command_reply: tid=0 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.129+0000 7f1f5f68a740 1 client.28211 shutdown 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 msgr2=0x555f05eb7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb7100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f40008c30 tx=0x7f1f40005160 comp rx=0 tx=0).stop 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 shutdown_connections 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05ec98d0 0x555f05ecbcc0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05eb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaf480 0x555f05eb7100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f3e7fc640 0 client.28211 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/3688365553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaeb60 0x555f05eb6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 >> 172.21.15.86:0/3688365553 conn(0x555f05eab0f0 msgr2=0x555f05eaa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 shutdown_connections 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 1 -- 172.21.15.86:0/3688365553 wait complete. 2024-09-05T06:58:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.130+0000 7f1f5f68a740 20 client.28211 trim_cache size 0 max 16384 2024-09-05T06:58:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 Processor -- start 2024-09-05T06:58:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 -- start start 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaeb60 0x555f05eb15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05ed1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ad0c0 con 0x555f05eb9d40 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581aec60 con 0x555f05eaeb60 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ad790 con 0x555f05eaf480 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50056/0 (socket says 172.21.15.86:50056) 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.147+0000 7f1f567fc640 1 -- 172.21.15.86:0/2958326675 learned_addr learned my addr 172.21.15.86:0/2958326675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaeb60 0x555f05eb15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05ed1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 -- 172.21.15.86:0/2958326675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 msgr2=0x555f05ed1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05ed1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 -- 172.21.15.86:0/2958326675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaeb60 msgr2=0x555f05eb15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaeb60 0x555f05eb15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 -- 172.21.15.86:0/2958326675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f05eb91c0 con 0x555f05eb9d40 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05ed1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1f48031440 tx=0x7f1f48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad299a82d81a71ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f4cff9640 1 -- 172.21.15.86:0/2958326675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48035040 con 0x555f05eb9d40 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f4cff9640 1 -- 172.21.15.86:0/2958326675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4803dc10 con 0x555f05eb9d40 2024-09-05T06:58:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.148+0000 7f1f4cff9640 1 -- 172.21.15.86:0/2958326675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48003970 con 0x555f05eb9d40 2024-09-05T06:58:09.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2958326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 msgr2=0x555f05ed1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1f48031440 tx=0x7f1f48009260 comp rx=0 tx=0).stop 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2958326675 shutdown_connections 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eb9d40 0x555f05ed1990 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05ed1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2958326675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eaeb60 0x555f05eb15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2958326675 >> 172.21.15.86:0/2958326675 conn(0x555f05eab0f0 msgr2=0x555f05eb5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2958326675 shutdown_connections 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2958326675 wait complete. 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 -- start start 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.149+0000 7f1f5f68a740 1 Processor -- start 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05eb39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eb9d40 0x555f05eb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ae590 con 0x555f05eaeb60 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581adec0 con 0x555f05eb9d40 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ab0c0 con 0x555f05eaf480 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50062/0 (socket says 172.21.15.86:50062) 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 -- 172.21.15.86:0/2956427366 learned_addr learned my addr 172.21.15.86:0/2956427366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f5f68a740 10 client.0 _ll_register_callbacks cb 0x7f1f48032ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eb9d40 0x555f05eb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05eb39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 -- 172.21.15.86:0/2956427366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 msgr2=0x555f05eb39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05eb39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 -- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eb9d40 msgr2=0x555f05eb3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:09.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eb9d40 0x555f05eb3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 -- 172.21.15.86:0/2956427366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f05eb4750 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f567fc640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009230 tx=0x7f1f48004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9006deb425f4db4f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.150+0000 7f1f3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48035070 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f1f480042c0 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4803dc10 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1f4803d760 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 10 client.19323 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f05f344f0 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4803dc10 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1f48062020 con 0x555f05eaeb60 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 10 client.19323 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 10 client.19323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:09.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 20 client.19323 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05f34b00 0x555f05f36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 4 client.19323 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f05e1bd00 con 0x555f05f34b00 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.151+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05f34b00 0x555f05f36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.152+0000 7f1f3e7fc640 10 client.19323 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.152+0000 7f1f56ffd640 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05f34b00 0x555f05f36ef0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ec6df8729fa420 server_cookie=73df8824e14a08cc in_seq=0 out_seq=0 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f3e7fc640 1 -- 172.21.15.86:0/2956427366 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f05e1bd00 con 0x555f05f34b00 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f3e7fc640 10 client.19323 handle_command_reply: tid=0 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 client.19323 shutdown 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 msgr2=0x555f05ec24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009230 tx=0x7f1f48004580 comp rx=0 tx=0).stop 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 shutdown_connections 2024-09-05T06:58:09.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f05f34b00 0x555f05f36ef0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f3e7fc640 0 client.19323 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f05eb9d40 0x555f05eb3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f05eaf480 0x555f05eb39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 --2- 172.21.15.86:0/2956427366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f05eaeb60 0x555f05ec24c0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 >> 172.21.15.86:0/2956427366 conn(0x555f05eab0f0 msgr2=0x555f05eb5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 shutdown_connections 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 1 -- 172.21.15.86:0/2956427366 wait complete. 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.153+0000 7f1f5f68a740 20 client.19323 trim_cache size 0 max 16384 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:09.158 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "d16fa05a-01b5-4df2-84aa-31ed02ddd1b7", 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 msgr2=0x7f1f440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 0x7f1f440a51b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f50097450 tx=0x7f1f50014b50 comp rx=0 tx=0).stop 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 msgr2=0x7f1f5819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b970 tx=0x7f1f5000be30 comp rx=0 tx=0).stop 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 shutdown_connections 2024-09-05T06:58:09.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1f440a2cf0 0x7f1f440a51b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f581082a0 0x7f1f5819ce00 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581036e0 0x7f1f58198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 --2- 172.21.15.86:0/1913241398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58102d30 0x7f1f58198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 >> 172.21.15.86:0/1913241398 conn(0x7f1f580fc7d0 msgr2=0x7f1f58106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 shutdown_connections 2024-09-05T06:58:09.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:09.154+0000 7f1f5d2b1640 1 -- 172.21.15.86:0/1913241398 wait complete. 2024-09-05T06:58:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:09 smithi115 ceph-mon[34791]: pgmap v530: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:09 smithi086 ceph-mon[30943]: pgmap v530: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:09.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.09 MB/sec execute 87 sec latency 12939.891 ms 2024-09-05T06:58:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:09 smithi096 ceph-mon[34468]: pgmap v530: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:10.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:58:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:10 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:10 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 7.01 MB/sec execute 88 sec latency 13939.981 ms 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:10 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 -- 172.21.15.86:0/34813722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 msgr2=0x7efd681021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 --2- 172.21.15.86:0/34813722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd681021b0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7efd58009900 tx=0x7efd5802a410 comp rx=0 tx=0).stop 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 -- 172.21.15.86:0/34813722 shutdown_connections 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 --2- 172.21.15.86:0/34813722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd6810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 --2- 172.21.15.86:0/34813722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd681067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 --2- 172.21.15.86:0/34813722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd681021b0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 -- 172.21.15.86:0/34813722 >> 172.21.15.86:0/34813722 conn(0x7efd680fd8b0 msgr2=0x7efd680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 -- 172.21.15.86:0/34813722 shutdown_connections 2024-09-05T06:58:10.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.858+0000 7efd6e490640 1 -- 172.21.15.86:0/34813722 wait complete. 2024-09-05T06:58:10.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 Processor -- start 2024-09-05T06:58:10.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 -- start start 2024-09-05T06:58:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd68111e30 con 0x7efd68101dd0 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd68111cb0 con 0x7efd68106f00 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd6e490640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd68111fb0 con 0x7efd68102780 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50080/0 (socket says 172.21.15.86:50080) 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.859+0000 7efd67fff640 1 -- 172.21.15.86:0/3289651150 learned_addr learned my addr 172.21.15.86:0/3289651150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd677fe640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 -- 172.21.15.86:0/3289651150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 msgr2=0x7efd68074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd68074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 -- 172.21.15.86:0/3289651150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 msgr2=0x7efd68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd68074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 -- 172.21.15.86:0/3289651150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd681a68d0 con 0x7efd68101dd0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd677fe640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd67fff640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efd58005940 tx=0x7efd58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd657fa640 1 -- 172.21.15.86:0/3289651150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5803f030 con 0x7efd68101dd0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd657fa640 1 -- 172.21.15.86:0/3289651150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd58003aa0 con 0x7efd68101dd0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd657fa640 1 -- 172.21.15.86:0/3289651150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58035070 con 0x7efd68101dd0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a6b60 con 0x7efd68101dd0 2024-09-05T06:58:10.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.860+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a7090 con 0x7efd68101dd0 2024-09-05T06:58:10.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.862+0000 7efd657fa640 1 -- 172.21.15.86:0/3289651150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd58042030 con 0x7efd68101dd0 2024-09-05T06:58:10.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 Processor -- start 2024-09-05T06:58:10.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 -- start start 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c06140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd657fa640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 0x7efd500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c08a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561144b612e0 con 0x561144bfb4a0 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561144b08ce0 con 0x561144bfbdc0 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd70869740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561144b5d2a0 con 0x561144c06680 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd6ca2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd677fe640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 0x7efd500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd6ca2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50092/0 (socket says 172.21.15.86:50092) 2024-09-05T06:58:10.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3112293208 learned_addr learned my addr 172.21.15.86:0/3112293208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd657fa640 1 -- 172.21.15.86:0/3289651150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7efd580f00c0 con 0x7efd68101dd0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.863+0000 7efd67fff640 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c06140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3112293208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 msgr2=0x561144c08a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c08a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3112293208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 msgr2=0x561144c06140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c06140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3112293208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561144aed440 con 0x561144bfb4a0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd677fe640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 0x7efd500a5230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006fd0 tx=0x7efd5c008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efd60010490 tx=0x7efd60010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e476e1cedff5e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd4e7fc640 1 -- 172.21.15.86:0/3112293208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60016070 con 0x561144bfb4a0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd4e7fc640 1 -- 172.21.15.86:0/3112293208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd60012bc0 con 0x561144bfb4a0 2024-09-05T06:58:10.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.864+0000 7efd4e7fc640 1 -- 172.21.15.86:0/3112293208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60011c20 con 0x561144bfb4a0 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- 172.21.15.86:0/3112293208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 msgr2=0x561144bfb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efd60010490 tx=0x7efd60010950 comp rx=0 tx=0).stop 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- 172.21.15.86:0/3112293208 shutdown_connections 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c08a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c06140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 --2- 172.21.15.86:0/3112293208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144bfb880 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- 172.21.15.86:0/3112293208 >> 172.21.15.86:0/3112293208 conn(0x561144bf7a30 msgr2=0x561144bf7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- 172.21.15.86:0/3112293208 shutdown_connections 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- 172.21.15.86:0/3112293208 wait complete. 2024-09-05T06:58:10.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 -- start start 2024-09-05T06:58:10.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.865+0000 7efd70869740 1 Processor -- start 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfb4a0 0x561144c06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561144c09250 con 0x561144bfbdc0 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561144b1c050 con 0x561144bfb4a0 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561144c093d0 con 0x561144c06680 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 10 client.0 _ll_register_callbacks cb 0x561144b5c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50096/0 (socket says 172.21.15.86:50096) 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 -- 172.21.15.86:0/1813584420 learned_addr learned my addr 172.21.15.86:0/1813584420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd67fff640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfb4a0 0x561144c06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 -- 172.21.15.86:0/1813584420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 msgr2=0x561144c03880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c03880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 -- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfb4a0 msgr2=0x561144c06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfb4a0 0x561144c06030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 -- 172.21.15.86:0/1813584420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611448d23c0 con 0x561144bfbdc0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd677fe640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00cea0 tx=0x7efd5c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f96022cc4a27e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd477fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c023060 con 0x561144bfbdc0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd60012bc0 con 0x561144bfbdc0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.866+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7efd5c01e600 con 0x561144bfbdc0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efd5c02a050 con 0x561144bfbdc0 2024-09-05T06:58:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 10 client.19335 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561144c14d10 con 0x561144bfbdc0 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd60012bc0 con 0x561144bfbdc0 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7efd5c01d430 con 0x561144bfbdc0 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 10 client.19335 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 20 client.19335 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c16120 0x561144c18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 4 client.19335 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561144b6e850 con 0x561144c16120 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.867+0000 7efd67fff640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c16120 0x561144c18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.868+0000 7efd67fff640 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c16120 0x561144c18580 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8afbbb150021cb server_cookie=63bf5fa443443ff7 in_seq=0 out_seq=0 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.868+0000 7efd477fe640 10 client.19335 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.868+0000 7efd477fe640 1 -- 172.21.15.86:0/1813584420 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561144b6e850 con 0x561144c16120 2024-09-05T06:58:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.868+0000 7efd477fe640 10 client.19335 handle_command_reply: tid=0 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.868+0000 7efd70869740 1 client.19335 shutdown 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 msgr2=0x561144c032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00cea0 tx=0x7efd5c0047d0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 shutdown_connections 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c16120 0x561144c18580 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfbdc0 0x561144c032e0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 --2- 172.21.15.86:0/1813584420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfb4a0 0x561144c06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 >> 172.21.15.86:0/1813584420 conn(0x561144bf7a30 msgr2=0x561144bfd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd477fe640 0 client.19335 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 shutdown_connections 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 1 -- 172.21.15.86:0/1813584420 wait complete. 2024-09-05T06:58:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.869+0000 7efd70869740 20 client.19335 trim_cache size 0 max 16384 2024-09-05T06:58:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 Processor -- start 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 -- start start 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144bfb4a0 0x561144c18130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c1de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b1500 con 0x561144c06680 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b30a0 con 0x561144bfbdc0 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd70869740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b1bd0 con 0x561144bfb4a0 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50104/0 (socket says 172.21.15.86:50104) 2024-09-05T06:58:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3056981538 learned_addr learned my addr 172.21.15.86:0/3056981538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd67fff640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144bfb4a0 0x561144c18130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd677fe640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c1de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3056981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144bfb4a0 msgr2=0x561144c18130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144bfb4a0 0x561144c18130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3056981538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 msgr2=0x561144c1de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c1de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 -- 172.21.15.86:0/3056981538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561144c02290 con 0x561144c06680 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd677fe640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.886+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efd60010460 tx=0x7efd60009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1eb562e3b5d9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd4dffb640 1 -- 172.21.15.86:0/3056981538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60029070 con 0x561144c06680 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd4dffb640 1 -- 172.21.15.86:0/3056981538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd600116e0 con 0x561144c06680 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd4dffb640 1 -- 172.21.15.86:0/3056981538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60016040 con 0x561144c06680 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 -- 172.21.15.86:0/3056981538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 msgr2=0x561144c1e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efd60010460 tx=0x7efd60009d50 comp rx=0 tx=0).stop 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 -- 172.21.15.86:0/3056981538 shutdown_connections 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144c06680 0x561144c1e390 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 --2- 172.21.15.86:0/3056981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144bfb4a0 0x561144c18130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 -- 172.21.15.86:0/3056981538 >> 172.21.15.86:0/3056981538 conn(0x561144bf7a30 msgr2=0x561144c05360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 -- 172.21.15.86:0/3056981538 shutdown_connections 2024-09-05T06:58:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.887+0000 7efd70869740 1 -- 172.21.15.86:0/3056981538 wait complete. 2024-09-05T06:58:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 -- start start 2024-09-05T06:58:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 Processor -- start 2024-09-05T06:58:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b29d0 con 0x561144bfb4a0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd681b2300 con 0x561144bfbdc0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd681af500 con 0x561144c06680 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd70869740 10 client.0 _ll_register_callbacks cb 0x7efd5c00ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.888+0000 7efd67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50118/0 (socket says 172.21.15.86:50118) 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 -- 172.21.15.86:0/4026688732 learned_addr learned my addr 172.21.15.86:0/4026688732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd6ca2d640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd677fe640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 -- 172.21.15.86:0/4026688732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 msgr2=0x561144c0ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c0ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 -- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 msgr2=0x561144c0c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c0c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 -- 172.21.15.86:0/4026688732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561144c7fe30 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd677fe640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c0c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd67fff640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efd58006010 tx=0x7efd58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0f8b163aabcc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd477fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58049040 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7efd580c01b0 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561144b68d60 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efd580c4dc0 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd70869740 10 client.19341 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561144c810f0 con 0x561144bfb4a0 2024-09-05T06:58:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.889+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561144b68d60 con 0x561144bfb4a0 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7efd580b87c0 con 0x561144bfb4a0 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 10 client.19341 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 20 client.19341 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c81810 0x561144c83c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 4 client.19341 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561144ae5210 con 0x561144c81810 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd677fe640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c81810 0x561144c83c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd677fe640 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c81810 0x561144c83c30 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc29a207d1f65039 server_cookie=fce01c25473328dd in_seq=0 out_seq=0 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.890+0000 7efd477fe640 10 client.19341 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd477fe640 1 -- 172.21.15.86:0/4026688732 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561144ae5210 con 0x561144c81810 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd477fe640 10 client.19341 handle_command_reply: tid=0 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 client.19341 shutdown 2024-09-05T06:58:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 msgr2=0x561144c0c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efd58006010 tx=0x7efd58009250 comp rx=0 tx=0).stop 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 shutdown_connections 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561144c81810 0x561144c83c30 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561144c06680 0x561144c0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561144bfbdc0 0x561144c0c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 --2- 172.21.15.86:0/4026688732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561144bfb4a0 0x561144c0c3e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 >> 172.21.15.86:0/4026688732 conn(0x561144bf7a30 msgr2=0x561144c04b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd477fe640 0 client.19341 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 shutdown_connections 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 1 -- 172.21.15.86:0/4026688732 wait complete. 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.891+0000 7efd70869740 20 client.19341 trim_cache size 0 max 16384 2024-09-05T06:58:10.898 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "d16fa05a-01b5-4df2-84aa-31ed02ddd1b7": { 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "d16fa05a-01b5-4df2-84aa-31ed02ddd1b7", 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:10.899 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 msgr2=0x7efd500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 0x7efd500a5230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006fd0 tx=0x7efd5c008040 comp rx=0 tx=0).stop 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 msgr2=0x7efd680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efd58005940 tx=0x7efd58004270 comp rx=0 tx=0).stop 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 shutdown_connections 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd500a2d70 0x7efd500a5230 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd68106f00 0x7efd68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102780 0x7efd68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 --2- 172.21.15.86:0/3289651150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68101dd0 0x7efd680744f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 >> 172.21.15.86:0/3289651150 conn(0x7efd680fd8b0 msgr2=0x7efd680fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 shutdown_connections 2024-09-05T06:58:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:10.892+0000 7efd6e490640 1 -- 172.21.15.86:0/3289651150 wait complete. 2024-09-05T06:58:11.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:11 smithi086 ceph-mon[30943]: pgmap v531: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:11.667 INFO:tasks.cephfs.filesystem:scrub status for tag:d16fa05a-01b5-4df2-84aa-31ed02ddd1b7 - {'path': '/', 'tag': 'd16fa05a-01b5-4df2-84aa-31ed02ddd1b7', 'options': 'recursive,force'} 2024-09-05T06:58:11.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:11 smithi115 ceph-mon[34791]: pgmap v531: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:11.758 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.93 MB/sec execute 89 sec latency 14940.072 ms 2024-09-05T06:58:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:11 smithi096 ceph-mon[34468]: pgmap v531: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 819 KiB/s wr, 89 op/s 2024-09-05T06:58:12.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.85 MB/sec execute 90 sec latency 15940.172 ms 2024-09-05T06:58:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:13 smithi115 ceph-mon[34791]: pgmap v532: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:13 smithi086 ceph-mon[30943]: pgmap v532: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:13.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.78 MB/sec execute 91 sec latency 16940.272 ms 2024-09-05T06:58:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:13 smithi096 ceph-mon[34468]: pgmap v532: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:14.332 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:58:14.591 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:14.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.70 MB/sec execute 92 sec latency 17940.378 ms 2024-09-05T06:58:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 -- 172.21.15.86:0/488786419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164106f10 msgr2=0x7f516410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 --2- 172.21.15.86:0/488786419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164106f10 0x7f516410ea90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f515c009ca0 tx=0x7f515c02a8c0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 -- 172.21.15.86:0/488786419 shutdown_connections 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 --2- 172.21.15.86:0/488786419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164106f10 0x7f516410ea90 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 --2- 172.21.15.86:0/488786419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164102790 0x7f51641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 --2- 172.21.15.86:0/488786419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164101de0 0x7f51641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 -- 172.21.15.86:0/488786419 >> 172.21.15.86:0/488786419 conn(0x7f51640fd8c0 msgr2=0x7f51640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 -- 172.21.15.86:0/488786419 shutdown_connections 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 -- 172.21.15.86:0/488786419 wait complete. 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.115+0000 7f5169b88640 1 Processor -- start 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 -- start start 2024-09-05T06:58:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164102790 0x7f5164074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 0x7f5164074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5164111e60 con 0x7f5164101de0 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5164111ce0 con 0x7f5164106f10 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5169b88640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5164111fe0 con 0x7f5164102790 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34848/0 (socket says 172.21.15.86:34848) 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 -- 172.21.15.86:0/170552246 learned_addr learned my addr 172.21.15.86:0/170552246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5162ffd640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164102790 0x7f5164074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5163fff640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 0x7f5164074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 -- 172.21.15.86:0/170552246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164102790 msgr2=0x7f5164074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164102790 0x7f5164074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 -- 172.21.15.86:0/170552246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 msgr2=0x7f5164074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 0x7f5164074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f51637fe640 1 -- 172.21.15.86:0/170552246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641a6880 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.116+0000 7f5163fff640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 0x7f5164074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f51637fe640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f515400d770 tx=0x7f515400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154017070 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5154009820 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641a6b70 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154004ba0 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.117+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a7060 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.118+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51641021c0 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.119+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51540099d0 con 0x7f5164101de0 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.120+0000 7f5160ff9640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 0x7f51500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:15.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.120+0000 7f5162ffd640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 0x7f51500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:15.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.120+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f51540cf310 con 0x7f5164101de0 2024-09-05T06:58:15.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.120+0000 7f5162ffd640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 0x7f51500a51b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5164075f80 tx=0x7f514c009110 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:15.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.124+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f515409baa0 con 0x7f5164101de0 2024-09-05T06:58:15.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.321+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f516410a150 con 0x7f5164101de0 2024-09-05T06:58:15.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.322+0000 7f5160ff9640 1 -- 172.21.15.86:0/170552246 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7f515409b3a0 con 0x7f5164101de0 2024-09-05T06:58:15.323 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:58:15.323 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:58:15.323 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 msgr2=0x7f51500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 0x7f51500a51b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5164075f80 tx=0x7f514c009110 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 msgr2=0x7f51640744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f515400d770 tx=0x7f515400db70 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 shutdown_connections 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f51500a2cf0 0x7f51500a51b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5164106f10 0x7f5164074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5164102790 0x7f5164074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 --2- 172.21.15.86:0/170552246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164101de0 0x7f51640744f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.324+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 >> 172.21.15.86:0/170552246 conn(0x7f51640fd8c0 msgr2=0x7f51640fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:15.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.325+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 shutdown_connections 2024-09-05T06:58:15.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:15.325+0000 7f5169b88640 1 -- 172.21.15.86:0/170552246 wait complete. 2024-09-05T06:58:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:15 smithi115 ceph-mon[34791]: pgmap v533: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:15 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/170552246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:15 smithi086 ceph-mon[30943]: pgmap v533: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:15 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/170552246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:15.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.63 MB/sec execute 93 sec latency 18940.432 ms 2024-09-05T06:58:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:15 smithi096 ceph-mon[34468]: pgmap v533: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:15 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/170552246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:16.068 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 96a4659f --release --await' 2024-09-05T06:58:16.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 96a4659f --release --await 2024-09-05T06:58:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:16.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26776 6.56 MB/sec execute 94 sec latency 19940.461 ms 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 -- 172.21.15.86:0/2013099204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101de0 msgr2=0x7f75001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 --2- 172.21.15.86:0/2013099204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101de0 0x7f75001021c0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009900 tx=0x7f74fc02a410 comp rx=0 tx=0).stop 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 -- 172.21.15.86:0/2013099204 shutdown_connections 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 --2- 172.21.15.86:0/2013099204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500106f10 0x7f750010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 --2- 172.21.15.86:0/2013099204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f75001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 --2- 172.21.15.86:0/2013099204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101de0 0x7f75001021c0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 -- 172.21.15.86:0/2013099204 >> 172.21.15.86:0/2013099204 conn(0x7f75000fd8c0 msgr2=0x7f75000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 -- 172.21.15.86:0/2013099204 shutdown_connections 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.839+0000 7f75078a8640 1 -- 172.21.15.86:0/2013099204 wait complete. 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 Processor -- start 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 -- start start 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 0x7f75001984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f7500198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7500111ec0 con 0x7f7500106f10 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7500111d40 con 0x7f7500102790 2024-09-05T06:58:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f75078a8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7500112040 con 0x7f7500101de0 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f7505e45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f7505e45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34866/0 (socket says 172.21.15.86:34866) 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f7505e45640 1 -- 172.21.15.86:0/820701590 learned_addr learned my addr 172.21.15.86:0/820701590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f7505644640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 0x7f75001984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.840+0000 7f7504e43640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f7500198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 -- 172.21.15.86:0/820701590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 msgr2=0x7f75001984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 0x7f75001984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 -- 172.21.15.86:0/820701590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 msgr2=0x7f7500198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f7500198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 -- 172.21.15.86:0/820701590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750019d490 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505644640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 0x7f75001984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7504e43640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f7500198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f7505e45640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b970 tx=0x7f74f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f74f67fc640 1 -- 172.21.15.86:0/820701590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8019070 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f74f67fc640 1 -- 172.21.15.86:0/820701590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f8014440 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f74f67fc640 1 -- 172.21.15.86:0/820701590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80075d0 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750019d780 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.841+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75001a5060 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.843+0000 7f74f67fc640 1 -- 172.21.15.86:0/820701590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74f801c070 con 0x7f7500106f10 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.843+0000 7f7509c81740 1 Processor -- start 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 -- start start 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b060 0x558f7040b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b980 0x558f70415d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f74f67fc640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 0x7f74ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558f70370620 con 0x558f70416240 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f70317e50 con 0x558f7040b060 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7509c81740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f7036c5e0 con 0x558f7040b980 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7505e45640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 0x7f74ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34876/0 (socket says 172.21.15.86:34876) 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 -- 172.21.15.86:0/544557424 learned_addr learned my addr 172.21.15.86:0/544557424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f74f67fc640 1 -- 172.21.15.86:0/820701590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f74f80cee50 con 0x7f7500106f10 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7505644640 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b980 0x558f70415d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b060 0x558f7040b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 -- 172.21.15.86:0/544557424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b980 msgr2=0x558f70415d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b980 0x558f70415d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 -- 172.21.15.86:0/544557424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b060 msgr2=0x558f7040b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b060 0x558f7040b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7504e43640 1 -- 172.21.15.86:0/544557424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f70308d40 con 0x558f70416240 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.844+0000 7f7505e45640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 0x7f74ec0a51b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74f80cc2d0 tx=0x7f74f806cfb0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f7504e43640 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f000c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baaca3ba4dbe9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f74eb7fe640 1 -- 172.21.15.86:0/544557424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f001c070 con 0x558f70416240 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f74eb7fe640 1 -- 172.21.15.86:0/544557424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f0017ac0 con 0x558f70416240 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f74eb7fe640 1 -- 172.21.15.86:0/544557424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f000f040 con 0x558f70416240 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f7509c81740 1 -- 172.21.15.86:0/544557424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 msgr2=0x558f70418630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f7509c81740 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f000c230 comp rx=0 tx=0).stop 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f7509c81740 1 -- 172.21.15.86:0/544557424 shutdown_connections 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.845+0000 7f7509c81740 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f70418630 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b980 0x558f70415d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 --2- 172.21.15.86:0/544557424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b060 0x558f7040b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- 172.21.15.86:0/544557424 >> 172.21.15.86:0/544557424 conn(0x558f704075f0 msgr2=0x558f704079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- 172.21.15.86:0/544557424 shutdown_connections 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- 172.21.15.86:0/544557424 wait complete. 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- start start 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 Processor -- start 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f70413740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f70413c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558f70418e10 con 0x558f7040b060 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f702f8f10 con 0x558f7040b980 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f70418f90 con 0x558f70416240 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7509c81740 10 client.0 _ll_register_callbacks cb 0x558f7031cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34890/0 (socket says 172.21.15.86:34890) 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.846+0000 7f7504e43640 1 -- 172.21.15.86:0/2704034321 learned_addr learned my addr 172.21.15.86:0/2704034321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 -- 172.21.15.86:0/2704034321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 msgr2=0x558f70413c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f70413c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 -- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 msgr2=0x558f70413740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f70413740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 -- 172.21.15.86:0/2704034321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7007f580 con 0x558f7040b060 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7504e43640 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f74f0001eb0 tx=0x7f74f000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4247fdcdffe24eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f74e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:16.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f001c040 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f74f0017620 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f0017ac0 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f74f0030050 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7509c81740 10 client.19359 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f70149960 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.847+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f0017ac0 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f74f0015040 con 0x558f7040b060 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 10 client.19359 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 10 client.19359 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 20 client.19359 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f704280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 4 client.19359 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:16.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f7037db90 con 0x558f70425d00 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7505644640 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f704280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f7505644640 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f704280f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c978812266ed5a9f server_cookie=eeb4b76f7a1edbb8 in_seq=0 out_seq=0 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.848+0000 7f74e8ff9640 10 client.19359 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2704034321 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f7037db90 con 0x558f70425d00 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f74e8ff9640 10 client.19359 handle_command_reply: tid=0 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 client.19359 shutdown 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 msgr2=0x558f70413200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f74f0001eb0 tx=0x7f74f000cf70 comp rx=0 tx=0).stop 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 shutdown_connections 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f704280f0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f74e8ff9640 0 client.19359 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f70413c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f70413740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 --2- 172.21.15.86:0/2704034321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70413200 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 >> 172.21.15.86:0/2704034321 conn(0x558f704075f0 msgr2=0x558f7040d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 shutdown_connections 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 1 -- 172.21.15.86:0/2704034321 wait complete. 2024-09-05T06:58:16.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.849+0000 7f7509c81740 20 client.19359 trim_cache size 0 max 16384 2024-09-05T06:58:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 Processor -- start 2024-09-05T06:58:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 -- start start 2024-09-05T06:58:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b060 0x558f70406a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7042d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75001af390 con 0x558f70416240 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75001b0f30 con 0x558f7040b980 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7509c81740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75001afa60 con 0x558f7040b060 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34904/0 (socket says 172.21.15.86:34904) 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7504e43640 1 -- 172.21.15.86:0/2786807119 learned_addr learned my addr 172.21.15.86:0/2786807119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7505644640 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7042d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.866+0000 7f7504e43640 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b060 0x558f70406a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 -- 172.21.15.86:0/2786807119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b060 msgr2=0x558f70406a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b060 0x558f70406a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 -- 172.21.15.86:0/2786807119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 msgr2=0x558f7042d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7042d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 -- 172.21.15.86:0/2786807119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f7040b7b0 con 0x558f70416240 2024-09-05T06:58:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7504e43640 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c890 tx=0x7f74f0006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f94cdc3a64fdd3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f74eaffd640 1 -- 172.21.15.86:0/2786807119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0016dc0 con 0x558f70416240 2024-09-05T06:58:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f74eaffd640 1 -- 172.21.15.86:0/2786807119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f000a070 con 0x558f70416240 2024-09-05T06:58:16.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f74eaffd640 1 -- 172.21.15.86:0/2786807119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0015040 con 0x558f70416240 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7509c81740 1 -- 172.21.15.86:0/2786807119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 msgr2=0x558f7042dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.867+0000 7f7509c81740 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c890 tx=0x7f74f0006a30 comp rx=0 tx=0).stop 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- 172.21.15.86:0/2786807119 shutdown_connections 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f70416240 0x558f7042dc60 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7042d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- 172.21.15.86:0/2786807119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f7040b060 0x558f70406a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- 172.21.15.86:0/2786807119 >> 172.21.15.86:0/2786807119 conn(0x558f704075f0 msgr2=0x558f704131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- 172.21.15.86:0/2786807119 shutdown_connections 2024-09-05T06:58:16.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- 172.21.15.86:0/2786807119 wait complete. 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- start start 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 Processor -- start 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7041a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f7041a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75001b0860 con 0x558f7040b060 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75001b0190 con 0x558f7040b980 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75001ad390 con 0x558f70416240 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.868+0000 7f7509c81740 10 client.0 _ll_register_callbacks cb 0x7f74fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:16.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34918/0 (socket says 172.21.15.86:34918) 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 -- 172.21.15.86:0/2152991370 learned_addr learned my addr 172.21.15.86:0/2152991370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7505644640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7041a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f7041a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 -- 172.21.15.86:0/2152991370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 msgr2=0x558f7041a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f7041a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 -- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 msgr2=0x558f7041a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7041a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 -- 172.21.15.86:0/2152991370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7048f6f0 con 0x558f7040b060 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7504e43640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f74f0004700 tx=0x7f74f0022760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2188c14dac732e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f74e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f001c070 con 0x558f7040b060 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9693+0+0 (secure 0 0 0) 0x7f74f0022a80 con 0x558f7040b060 2024-09-05T06:58:16.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f000a070 con 0x558f7040b060 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f74f0047050 con 0x558f7040b060 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7509c81740 10 client.19365 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.869+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f70490930 con 0x558f7040b060 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f000a070 con 0x558f7040b060 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f74f0016660 con 0x558f7040b060 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 10 client.19365 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 10 client.19365 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 20 client.19365 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f70492f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 4 client.19365 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "96a4659f", "release": true, "await": true}] 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "96a4659f", "release": true, "await": true}) -- 0x558f703780a0 con 0x558f70425d00 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7505644640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f70492f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f7505644640 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f70492f50 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492ab01c7e76cf63 server_cookie=177b85477ee876e8 in_seq=0 out_seq=0 2024-09-05T06:58:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.870+0000 7f74e8ff9640 10 client.19365 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f74e8ff9640 1 -- 172.21.15.86:0/2152991370 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558f703780a0 con 0x558f70425d00 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f74e8ff9640 10 client.19365 handle_command_reply: tid=0 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 client.19365 shutdown 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 msgr2=0x558f70419e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f74f0004700 tx=0x7f74f0022760 comp rx=0 tx=0).stop 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 shutdown_connections 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558f70425d00 0x558f70492f50 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f70416240 0x558f7041a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f7040b980 0x558f7041a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 --2- 172.21.15.86:0/2152991370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7040b060 0x558f70419e60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 >> 172.21.15.86:0/2152991370 conn(0x558f704075f0 msgr2=0x558f7040cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f74e8ff9640 0 client.19365 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 shutdown_connections 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 1 -- 172.21.15.86:0/2152991370 wait complete. 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.902+0000 7f7509c81740 20 client.19365 trim_cache size 0 max 16384 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 43, 2024-09-05T06:58:16.904 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 64, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "96a4659f": { 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 64, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:58:16.905 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:16.906 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 msgr2=0x7f74ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 0x7f74ec0a51b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74f80cc2d0 tx=0x7f74f806cfb0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 msgr2=0x7f750019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:16.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b970 tx=0x7f74f800be30 comp rx=0 tx=0).stop 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 shutdown_connections 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2cf0 0x7f74ec0a51b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500106f10 0x7f750019cdb0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500102790 0x7f7500198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 --2- 172.21.15.86:0/820701590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7500101de0 0x7f75001984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.903+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 >> 172.21.15.86:0/820701590 conn(0x7f75000fd8c0 msgr2=0x7f75000fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.904+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 shutdown_connections 2024-09-05T06:58:16.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:16.904+0000 7f75078a8640 1 -- 172.21.15.86:0/820701590 wait complete. 2024-09-05T06:58:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:17 smithi086 ceph-mon[30943]: pgmap v534: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:17 smithi086 ceph-mon[30943]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T06:58:17.658 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '96a4659f', total seconds elapsed: 33.3 2024-09-05T06:58:17.658 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T06:58:17.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:58:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:17 smithi115 ceph-mon[34791]: pgmap v534: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:17 smithi115 ceph-mon[34791]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T06:58:17.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26912 6.56 MB/sec execute 95 sec latency 20079.324 ms 2024-09-05T06:58:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:17 smithi096 ceph-mon[34468]: pgmap v534: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 151 KiB/s wr, 19 op/s 2024-09-05T06:58:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:17 smithi096 ceph-mon[34468]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T06:58:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 -- 172.21.15.86:0/383945488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 msgr2=0x7f4958101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 --2- 172.21.15.86:0/383945488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f4958101200 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4948009900 tx=0x7f494802a480 comp rx=0 tx=0).stop 2024-09-05T06:58:18.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 -- 172.21.15.86:0/383945488 shutdown_connections 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 --2- 172.21.15.86:0/383945488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 0x7f495810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 --2- 172.21.15.86:0/383945488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 --2- 172.21.15.86:0/383945488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f4958101200 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 -- 172.21.15.86:0/383945488 >> 172.21.15.86:0/383945488 conn(0x7f49580fc7d0 msgr2=0x7f49580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 -- 172.21.15.86:0/383945488 shutdown_connections 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.520+0000 7f495cfb8640 1 -- 172.21.15.86:0/383945488 wait complete. 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.521+0000 7f495cfb8640 1 Processor -- start 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 -- start start 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 0x7f495819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:18.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f495810fd70 con 0x7f4958100e20 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f495810fbf0 con 0x7f4958102570 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495cfb8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f495810fef0 con 0x7f4958102ef0 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34938/0 (socket says 172.21.15.86:34938) 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f495659c640 1 -- 172.21.15.86:0/784092160 learned_addr learned my addr 172.21.15.86:0/784092160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f4955d9b640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.522+0000 7f4956d9d640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 0x7f495819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 -- 172.21.15.86:0/784092160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 msgr2=0x7f495819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 0x7f495819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 -- 172.21.15.86:0/784092160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 msgr2=0x7f49581a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:18.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 -- 172.21.15.86:0/784092160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819efe0 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f4955d9b640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495659c640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f494802aa70 tx=0x7f4948003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494803f070 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819f270 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f494803bb20 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948035030 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.523+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49581ab520 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.525+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49581021b0 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.526+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4948042070 con 0x7f4958100e20 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.527+0000 7f49477fe640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 0x7f49400a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:18.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.527+0000 7f4955d9b640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 0x7f49400a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:18.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.527+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f49480efed0 con 0x7f4958100e20 2024-09-05T06:58:18.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.528+0000 7f4955d9b640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 0x7f49400a52b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49580ffda0 tx=0x7f494c005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:18.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.533+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49480bc000 con 0x7f4958100e20 2024-09-05T06:58:18.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27130 6.68 MB/sec execute 96 sec latency 84.937 ms 2024-09-05T06:58:18.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.870+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49581ab6f0 con 0x7f4958100e20 2024-09-05T06:58:18.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.871+0000 7f49477fe640 1 -- 172.21.15.86:0/784092160 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7f49480c0300 con 0x7f4958100e20 2024-09-05T06:58:18.871 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:58:18.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:58:18.872 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.873+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 msgr2=0x7f49400a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.873+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 0x7f49400a52b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49580ffda0 tx=0x7f494c005f90 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.873+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 msgr2=0x7f49581a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.873+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f494802aa70 tx=0x7f4948003c10 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 shutdown_connections 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49400a2df0 0x7f49400a52b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4958102ef0 0x7f495819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4958102570 0x7f49581a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 --2- 172.21.15.86:0/784092160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4958100e20 0x7f49581a46c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:18.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 >> 172.21.15.86:0/784092160 conn(0x7f49580fc7d0 msgr2=0x7f495810c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:18.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 shutdown_connections 2024-09-05T06:58:18.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:18.874+0000 7f495cfb8640 1 -- 172.21.15.86:0/784092160 wait complete. 2024-09-05T06:58:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:19 smithi086 ceph-mon[30943]: pgmap v536: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/784092160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:19 smithi115 ceph-mon[34791]: pgmap v536: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/784092160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:19.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27352 6.73 MB/sec execute 97 sec latency 59.215 ms 2024-09-05T06:58:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:19 smithi096 ceph-mon[34468]: pgmap v536: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/784092160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:19.879 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T06:58:19.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T06:58:20.236 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:20.679 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:20 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:58:20.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:20 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:58:20.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27496 6.74 MB/sec execute 98 sec latency 123.885 ms 2024-09-05T06:58:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:20 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:58:20.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 -- 172.21.15.86:0/591911169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454103fe0 msgr2=0x7f44541043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 --2- 172.21.15.86:0/591911169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454103fe0 0x7f44541043c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4448009900 tx=0x7f444802a410 comp rx=0 tx=0).stop 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 -- 172.21.15.86:0/591911169 shutdown_connections 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 --2- 172.21.15.86:0/591911169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454109110 0x7f4454110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 --2- 172.21.15.86:0/591911169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 0x7f44541089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 --2- 172.21.15.86:0/591911169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454103fe0 0x7f44541043c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 -- 172.21.15.86:0/591911169 >> 172.21.15.86:0/591911169 conn(0x7f44540ffa40 msgr2=0x7f4454101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 -- 172.21.15.86:0/591911169 shutdown_connections 2024-09-05T06:58:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.807+0000 7f445aeeb640 1 -- 172.21.15.86:0/591911169 wait complete. 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 Processor -- start 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 -- start start 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 0x7f445419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 0x7f445419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4454114040 con 0x7f4454104990 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4454113ec0 con 0x7f4454103fe0 2024-09-05T06:58:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f445aeeb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44541141c0 con 0x7f4454109110 2024-09-05T06:58:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f4458c87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f4458c87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:51646/0 (socket says 172.21.15.86:51646) 2024-09-05T06:58:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f4458c87640 1 -- 172.21.15.86:0/3785231697 learned_addr learned my addr 172.21.15.86:0/3785231697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.808+0000 7f4459488640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 0x7f445419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4453fff640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 0x7f445419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 -- 172.21.15.86:0/3785231697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 msgr2=0x7f445419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 0x7f445419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 -- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 msgr2=0x7f445419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 0x7f445419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 -- 172.21.15.86:0/3785231697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445419f610 con 0x7f4454103fe0 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4459488640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 0x7f445419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4458c87640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4448005ab0 tx=0x7f4448003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4451ffb640 1 -- 172.21.15.86:0/3785231697 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444803f070 con 0x7f4454103fe0 2024-09-05T06:58:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445419f8a0 con 0x7f4454103fe0 2024-09-05T06:58:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4451ffb640 1 -- 172.21.15.86:0/3785231697 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4448031d60 con 0x7f4454103fe0 2024-09-05T06:58:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.809+0000 7f4451ffb640 1 -- 172.21.15.86:0/3785231697 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448035030 con 0x7f4454103fe0 2024-09-05T06:58:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.810+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44541a71e0 con 0x7f4454103fe0 2024-09-05T06:58:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.812+0000 7f4451ffb640 1 -- 172.21.15.86:0/3785231697 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4448042070 con 0x7f4454103fe0 2024-09-05T06:58:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 Processor -- start 2024-09-05T06:58:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 -- start start 2024-09-05T06:58:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8675382a0 con 0x55d8675d2fa0 2024-09-05T06:58:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d8674dfb10 con 0x55d8675e6150 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f445d2c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d867534260 con 0x55d8675d2680 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.813+0000 7f4459488640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:51648/0 (socket says 172.21.15.86:51648) 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 -- 172.21.15.86:0/2054651307 learned_addr learned my addr 172.21.15.86:0/2054651307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4451ffb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 0x7f44440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4458c87640 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 -- 172.21.15.86:0/2054651307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 msgr2=0x55d8675d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 -- 172.21.15.86:0/2054651307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 msgr2=0x55d8675e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 -- 172.21.15.86:0/2054651307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8672a9b90 con 0x55d8675e6150 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4451ffb640 1 -- 172.21.15.86:0/3785231697 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f44480f0050 con 0x7f4454103fe0 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4459488640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 0x7f44440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.814+0000 7f4453fff640 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f444000bc90 tx=0x7f44400107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ca1a36b7d81e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.815+0000 7f443effd640 1 -- 172.21.15.86:0/2054651307 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440005220 con 0x55d8675e6150 2024-09-05T06:58:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.815+0000 7f443effd640 1 -- 172.21.15.86:0/2054651307 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f444000be40 con 0x55d8675e6150 2024-09-05T06:58:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.815+0000 7f443effd640 1 -- 172.21.15.86:0/2054651307 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400117c0 con 0x55d8675e6150 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- 172.21.15.86:0/2054651307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 msgr2=0x55d8675e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f444000bc90 tx=0x7f44400107a0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f4459488640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 0x7f44440a5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f444c005fd0 tx=0x7f444c004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- 172.21.15.86:0/2054651307 shutdown_connections 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675e6150 0x55d8675e8540 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f444000bc90 tx=0x7f44400107a0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2054651307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- 172.21.15.86:0/2054651307 >> 172.21.15.86:0/2054651307 conn(0x55d8675cecf0 msgr2=0x55d8675cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- 172.21.15.86:0/2054651307 shutdown_connections 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- 172.21.15.86:0/2054651307 wait complete. 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 -- start start 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.816+0000 7f445d2c4740 1 Processor -- start 2024-09-05T06:58:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 0x55d8675e3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8675e8cc0 con 0x55d8675d2680 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d8674c06e0 con 0x55d8675d2fa0 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d8675e8e40 con 0x55d8675e3e00 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f4453fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f445d2c4740 10 client.0 _ll_register_callbacks cb 0x55d8674e4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f4453fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:51652/0 (socket says 172.21.15.86:51652) 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.817+0000 7f4453fff640 1 -- 172.21.15.86:0/2313679720 learned_addr learned my addr 172.21.15.86:0/2313679720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4459488640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4458c87640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 0x55d8675e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 -- 172.21.15.86:0/2313679720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 msgr2=0x55d8675f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 -- 172.21.15.86:0/2313679720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 msgr2=0x55d8675e3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 0x55d8675e3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 -- 172.21.15.86:0/2313679720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8675f28e0 con 0x55d8675d2fa0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4458c87640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 0x55d8675e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4453fff640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4440005730 tx=0x7f4440011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc76656990215d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4433fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.818+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440011ad0 con 0x55d8675d2fa0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8675459d0 con 0x55d8675d2fa0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f4440011c70 con 0x55d8675d2fa0 2024-09-05T06:58:20.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f444002f050 con 0x55d8675d2fa0 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 10 client.28265 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d8675f5290 con 0x55d8675d2fa0 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d8675459d0 con 0x55d8675d2fa0 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f444001b430 con 0x55d8675d2fa0 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 10 client.28265 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 10 client.28265 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 20 client.28265 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8675f69b0 0x55d8675f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 4 client.28265 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:20.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.819+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d86753fd20 con 0x55d8675f69b0 2024-09-05T06:58:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.820+0000 7f4458c87640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8675f69b0 0x55d8675f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.820+0000 7f4433fff640 10 client.28265 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.820+0000 7f4458c87640 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8675f69b0 0x55d8675f8e30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cfd211f892069a5 server_cookie=3d76b04b00861906 in_seq=0 out_seq=0 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f4433fff640 1 -- 172.21.15.86:0/2313679720 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d86753fd20 con 0x55d8675f69b0 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f4433fff640 10 client.28265 handle_command_reply: tid=0 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 client.28265 shutdown 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 msgr2=0x55d8675e3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4440005730 tx=0x7f4440011750 comp rx=0 tx=0).stop 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 shutdown_connections 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8675f69b0 0x55d8675f8e30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675e3870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 --2- 172.21.15.86:0/2313679720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2680 0x55d8675e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 >> 172.21.15.86:0/2313679720 conn(0x55d8675cecf0 msgr2=0x55d8675d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.822+0000 7f4433fff640 0 client.28265 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:20.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.823+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 shutdown_connections 2024-09-05T06:58:20.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.823+0000 7f445d2c4740 1 -- 172.21.15.86:0/2313679720 wait complete. 2024-09-05T06:58:20.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.823+0000 7f445d2c4740 20 client.28265 trim_cache size 0 max 16384 2024-09-05T06:58:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.848+0000 7f445d2c4740 1 Processor -- start 2024-09-05T06:58:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.848+0000 7f445d2c4740 1 -- start start 2024-09-05T06:58:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44541acf70 con 0x55d8675d2fa0 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44541aeb10 con 0x55d8675d2680 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f445d2c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44541ad640 con 0x55d8675e3e00 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:51662/0 (socket says 172.21.15.86:51662) 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 -- 172.21.15.86:0/1617231989 learned_addr learned my addr 172.21.15.86:0/1617231989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4459488640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4453fff640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 -- 172.21.15.86:0/1617231989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 msgr2=0x55d8675e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 -- 172.21.15.86:0/1617231989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 msgr2=0x55d8675f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 -- 172.21.15.86:0/1617231989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8675e22e0 con 0x55d8675d2680 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4453fff640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:20.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.849+0000 7f4458c87640 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4448031440 tx=0x7f4448090720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa20bf1fef6a3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.850+0000 7f443e7fc640 1 -- 172.21.15.86:0/1617231989 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448049040 con 0x55d8675d2680 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.850+0000 7f443e7fc640 1 -- 172.21.15.86:0/1617231989 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44480c04e0 con 0x55d8675d2680 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.850+0000 7f443e7fc640 1 -- 172.21.15.86:0/1617231989 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480b87a0 con 0x55d8675d2680 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- 172.21.15.86:0/1617231989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 msgr2=0x55d8675de570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4448031440 tx=0x7f4448090720 comp rx=0 tx=0).stop 2024-09-05T06:58:20.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- 172.21.15.86:0/1617231989 shutdown_connections 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675e3e00 0x55d8675e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675d2fa0 0x55d8675f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 --2- 172.21.15.86:0/1617231989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2680 0x55d8675de570 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- 172.21.15.86:0/1617231989 >> 172.21.15.86:0/1617231989 conn(0x55d8675cecf0 msgr2=0x55d8675e4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- 172.21.15.86:0/1617231989 shutdown_connections 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- 172.21.15.86:0/1617231989 wait complete. 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.851+0000 7f445d2c4740 1 -- start start 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 Processor -- start 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675e3e00 0x55d8675eac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44541ae440 con 0x55d8675e3e00 2024-09-05T06:58:20.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44541add70 con 0x55d8675d2fa0 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44541aaf70 con 0x55d8675d2680 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f4458c87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f445d2c4740 10 client.0 _ll_register_callbacks cb 0x7f4440005570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f4458c87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51134/0 (socket says 172.21.15.86:51134) 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f4459488640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675e3e00 0x55d8675eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f4458c87640 1 -- 172.21.15.86:0/3438655592 learned_addr learned my addr 172.21.15.86:0/3438655592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.852+0000 7f4453fff640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 -- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 msgr2=0x55d8675ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 -- 172.21.15.86:0/3438655592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675e3e00 msgr2=0x55d8675eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675e3e00 0x55d8675eac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 -- 172.21.15.86:0/3438655592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d867660a00 con 0x55d8675d2680 2024-09-05T06:58:20.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4453fff640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4458c87640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4448009660 tx=0x7f44480bc2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce445f689c9aba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4433fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448049040 con 0x55d8675d2680 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f44480b87a0 con 0x55d8675d2680 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.853+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44480c04e0 con 0x55d8675d2680 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f44480d50d0 con 0x55d8675d2680 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 10 client.28246 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d867661c50 con 0x55d8675d2680 2024-09-05T06:58:20.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44480c04e0 con 0x55d8675d2680 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f44480f2110 con 0x55d8675d2680 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 10 client.28246 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 10 client.28246 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 20 client.28246 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8676622d0 0x55d8676646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 4 client.28246 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55d8674bbec0 con 0x55d8676622d0 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.854+0000 7f4453fff640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8676622d0 0x55d8676646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.855+0000 7f4433fff640 10 client.28246 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:20.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:20.855+0000 7f4453fff640 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8676622d0 0x55d8676646c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b6e7c80d7d09db server_cookie=ccd3b74e8c038ac8 in_seq=0 out_seq=0 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f4433fff640 1 -- 172.21.15.86:0/3438655592 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d8674bbec0 con 0x55d8676622d0 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f4433fff640 10 client.28246 handle_command_reply: tid=0 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 client.28246 shutdown 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 msgr2=0x55d8675ea170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4448009660 tx=0x7f44480bc2b0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 shutdown_connections 2024-09-05T06:58:21.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55d8676622d0 0x55d8676646c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f4433fff640 0 client.28246 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8675e3e00 0x55d8675eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d8675d2fa0 0x55d8675ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 --2- 172.21.15.86:0/3438655592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d8675d2680 0x55d8675ea170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 >> 172.21.15.86:0/3438655592 conn(0x55d8675cecf0 msgr2=0x55d8675e53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 shutdown_connections 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.085+0000 7f445d2c4740 1 -- 172.21.15.86:0/3438655592 wait complete. 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.086+0000 7f445d2c4740 20 client.28246 trim_cache size 0 max 16384 2024-09-05T06:58:21.087 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 43, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 66, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "57cec38": { 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 66, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:58:21.088 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:21.089 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.086+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 msgr2=0x7f44440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.086+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 0x7f44440a5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f444c005fd0 tx=0x7f444c004ab0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.086+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 msgr2=0x7f445419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.086+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4448005ab0 tx=0x7f4448003c80 comp rx=0 tx=0).stop 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 shutdown_connections 2024-09-05T06:58:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f44440a2d70 0x7f44440a5230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4454109110 0x7f445419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454104990 0x7f445419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 --2- 172.21.15.86:0/3785231697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4454103fe0 0x7f445419a660 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 >> 172.21.15.86:0/3785231697 conn(0x7f44540ffa40 msgr2=0x7f4454100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 shutdown_connections 2024-09-05T06:58:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:21.089+0000 7f445aeeb640 1 -- 172.21.15.86:0/3785231697 wait complete. 2024-09-05T06:58:21.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:21 smithi115 ceph-mon[34791]: pgmap v537: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:58:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:58:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:58:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:21 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:58:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:21 smithi086 ceph-mon[30943]: pgmap v537: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:58:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:58:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:58:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:21 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:58:21.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.68 MB/sec execute 99 sec latency 901.651 ms 2024-09-05T06:58:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:21 smithi096 ceph-mon[34468]: pgmap v537: 129 pgs: 129 active+clean; 780 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-09-05T06:58:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:58:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:58:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:58:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:21 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:58:21.942 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '57cec38', quiesce took 4.3 seconds. Will release after: 25.7 2024-09-05T06:58:22.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.61 MB/sec execute 100 sec latency 1901.746 ms 2024-09-05T06:58:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:23 smithi115 ceph-mon[34791]: pgmap v538: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:23 smithi086 ceph-mon[30943]: pgmap v538: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:23.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.55 MB/sec execute 101 sec latency 2901.863 ms 2024-09-05T06:58:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:23 smithi096 ceph-mon[34468]: pgmap v538: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:24.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.48 MB/sec execute 102 sec latency 3901.959 ms 2024-09-05T06:58:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:25 smithi115 ceph-mon[34791]: pgmap v539: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:25.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:25 smithi086 ceph-mon[30943]: pgmap v539: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:25.759 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.42 MB/sec execute 103 sec latency 4902.096 ms 2024-09-05T06:58:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:25 smithi096 ceph-mon[34468]: pgmap v539: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:26.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.36 MB/sec execute 104 sec latency 5902.212 ms 2024-09-05T06:58:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:27 smithi115 ceph-mon[34791]: pgmap v540: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:27 smithi086 ceph-mon[30943]: pgmap v540: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:27.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.30 MB/sec execute 105 sec latency 6902.297 ms 2024-09-05T06:58:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:27 smithi096 ceph-mon[34468]: pgmap v540: 129 pgs: 129 active+clean; 800 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-09-05T06:58:28.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.24 MB/sec execute 106 sec latency 7902.391 ms 2024-09-05T06:58:29.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:29 smithi086 ceph-mon[30943]: pgmap v541: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.5 MiB/s wr, 128 op/s 2024-09-05T06:58:29.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.18 MB/sec execute 107 sec latency 8902.494 ms 2024-09-05T06:58:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:29 smithi096 ceph-mon[34468]: pgmap v541: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.5 MiB/s wr, 128 op/s 2024-09-05T06:58:29.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:29 smithi115 ceph-mon[34791]: pgmap v541: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.5 MiB/s wr, 128 op/s 2024-09-05T06:58:30.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.12 MB/sec execute 108 sec latency 9902.560 ms 2024-09-05T06:58:31.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:31 smithi115 ceph-mon[34791]: pgmap v542: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.2 MiB/s wr, 112 op/s 2024-09-05T06:58:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:31 smithi086 ceph-mon[30943]: pgmap v542: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.2 MiB/s wr, 112 op/s 2024-09-05T06:58:31.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.06 MB/sec execute 109 sec latency 10902.665 ms 2024-09-05T06:58:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:31 smithi096 ceph-mon[34468]: pgmap v542: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 2.2 MiB/s wr, 112 op/s 2024-09-05T06:58:32.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 6.01 MB/sec execute 110 sec latency 11902.754 ms 2024-09-05T06:58:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:33 smithi086 ceph-mon[30943]: pgmap v543: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-09-05T06:58:33.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.96 MB/sec execute 111 sec latency 12902.795 ms 2024-09-05T06:58:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:33 smithi096 ceph-mon[34468]: pgmap v543: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-09-05T06:58:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:33 smithi115 ceph-mon[34791]: pgmap v543: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-09-05T06:58:34.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.90 MB/sec execute 112 sec latency 13902.886 ms 2024-09-05T06:58:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:35 smithi086 ceph-mon[30943]: pgmap v544: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:35.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.85 MB/sec execute 113 sec latency 14903.007 ms 2024-09-05T06:58:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:35 smithi096 ceph-mon[34468]: pgmap v544: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:35 smithi115 ceph-mon[34791]: pgmap v544: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:36.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.80 MB/sec execute 114 sec latency 15903.089 ms 2024-09-05T06:58:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:37 smithi086 ceph-mon[30943]: pgmap v545: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:37.760 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.75 MB/sec execute 115 sec latency 16903.150 ms 2024-09-05T06:58:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:37 smithi096 ceph-mon[34468]: pgmap v545: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:37 smithi115 ceph-mon[34791]: pgmap v545: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 353 KiB/s wr, 20 op/s 2024-09-05T06:58:38.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.70 MB/sec execute 116 sec latency 17903.207 ms 2024-09-05T06:58:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:39 smithi086 ceph-mon[30943]: pgmap v546: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-09-05T06:58:39.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.65 MB/sec execute 117 sec latency 18903.266 ms 2024-09-05T06:58:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:39 smithi096 ceph-mon[34468]: pgmap v546: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-09-05T06:58:39.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:39 smithi115 ceph-mon[34791]: pgmap v546: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 372 KiB/s wr, 26 op/s 2024-09-05T06:58:40.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.60 MB/sec execute 118 sec latency 19903.389 ms 2024-09-05T06:58:41.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:58:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:41 smithi086 ceph-mon[30943]: pgmap v547: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-09-05T06:58:41.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.56 MB/sec execute 119 sec latency 20903.453 ms 2024-09-05T06:58:41.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:41 smithi096 ceph-mon[34468]: pgmap v547: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-09-05T06:58:41.930 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:41 smithi115 ceph-mon[34791]: pgmap v547: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-09-05T06:58:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 -- 172.21.15.86:0/219708842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 msgr2=0x7f9fb81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/219708842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb81021c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009900 tx=0x7f9fac02a410 comp rx=0 tx=0).stop 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 -- 172.21.15.86:0/219708842 shutdown_connections 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/219708842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8106f10 0x7f9fb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/219708842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8102790 0x7f9fb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/219708842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb81021c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 -- 172.21.15.86:0/219708842 >> 172.21.15.86:0/219708842 conn(0x7f9fb80fd8c0 msgr2=0x7f9fb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 -- 172.21.15.86:0/219708842 shutdown_connections 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.412+0000 7f9fbcd06640 1 -- 172.21.15.86:0/219708842 wait complete. 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 Processor -- start 2024-09-05T06:58:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 -- start start 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8102790 0x7f9fb81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8106f10 0x7f9fb819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111e70 con 0x7f9fb8101de0 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111cf0 con 0x7f9fb8102790 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fbcd06640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111ff0 con 0x7f9fb8106f10 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fb659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fb659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41810/0 (socket says 172.21.15.86:41810) 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fb659c640 1 -- 172.21.15.86:0/3305490928 learned_addr learned my addr 172.21.15.86:0/3305490928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.413+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8102790 0x7f9fb81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8106f10 0x7f9fb819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 -- 172.21.15.86:0/3305490928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8106f10 msgr2=0x7f9fb819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8106f10 0x7f9fb819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 -- 172.21.15.86:0/3305490928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8102790 msgr2=0x7f9fb81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8102790 0x7f9fb81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 -- 172.21.15.86:0/3305490928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb819d320 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fb659c640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9fac005ab0 tx=0x7f9fac003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fa77fe640 1 -- 172.21.15.86:0/3305490928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac03f070 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fa77fe640 1 -- 172.21.15.86:0/3305490928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac031d60 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb819d5b0 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fa77fe640 1 -- 172.21.15.86:0/3305490928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035030 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.414+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a4ef0 con 0x7f9fb8101de0 2024-09-05T06:58:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.416+0000 7f9fa77fe640 1 -- 172.21.15.86:0/3305490928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9fac042070 con 0x7f9fb8101de0 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.416+0000 7f9fbf0df740 1 Processor -- start 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 -- start start 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fa77fe640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 0x7f9fa00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca68dc0 0x55a1bca73140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc9ce2e0 con 0x55a1bca73680 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc975ce0 con 0x55a1bca684a0 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc9ca2a0 con 0x55a1bca68dc0 2024-09-05T06:58:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fa77fe640 1 -- 172.21.15.86:0/3305490928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f9fac0f0240 con 0x7f9fb8101de0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 0x7f9fa00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55718/0 (socket says 172.21.15.86:55718) 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb659c640 1 -- 172.21.15.86:0/1303790498 learned_addr learned my addr 172.21.15.86:0/1303790498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca68dc0 0x55a1bca73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1303790498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca68dc0 msgr2=0x55a1bca73140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca68dc0 0x55a1bca73140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1303790498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 msgr2=0x55a1bca68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1303790498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bc95a440 con 0x55a1bca73680 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.417+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.418+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca75a70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fd0 tx=0x7f9fb0012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe893fcccf562f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.418+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 0x7f9fa00a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0010800 tx=0x7f9fb001d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.418+0000 7f9fa4ff9640 1 -- 172.21.15.86:0/1303790498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0022070 con 0x55a1bca73680 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.418+0000 7f9fa4ff9640 1 -- 172.21.15.86:0/1303790498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fb0009630 con 0x55a1bca73680 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.418+0000 7f9fa4ff9640 1 -- 172.21.15.86:0/1303790498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb001e940 con 0x55a1bca73680 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1303790498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 msgr2=0x55a1bca75a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca75a70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fd0 tx=0x7f9fb0012f90 comp rx=0 tx=0).stop 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1303790498 shutdown_connections 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca75a70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca68dc0 0x55a1bca73140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1303790498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca684a0 0x55a1bca68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1303790498 >> 172.21.15.86:0/1303790498 conn(0x55a1bca64a30 msgr2=0x55a1bca64e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1303790498 shutdown_connections 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1303790498 wait complete. 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- start start 2024-09-05T06:58:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 Processor -- start 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1bca76250 con 0x55a1bca68dc0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1bc989050 con 0x55a1bca73680 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a1bca763d0 con 0x55a1bca684a0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fbf0df740 10 client.0 _ll_register_callbacks cb 0x55a1bc9c9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fb5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fb5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41832/0 (socket says 172.21.15.86:41832) 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.419+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1728514271 learned_addr learned my addr 172.21.15.86:0/1728514271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1728514271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 msgr2=0x55a1bca73050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca73050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 msgr2=0x55a1bca708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1728514271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bc73f3c0 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800be40 tx=0x7f9fa8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992f1a50e2177716 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa801f060 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb0009630 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f9fa8010960 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fa8026050 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fbf0df740 10 client.19398 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1bca81ce0 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fb0009630 con 0x55a1bca68dc0 2024-09-05T06:58:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.420+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9fa8019030 con 0x55a1bca68dc0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 10 client.19398 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 10 client.19398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 20 client.19398 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bca831f0 0x55a1bca85630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 4 client.19398 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1bc9db850 con 0x55a1bca831f0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bca831f0 0x55a1bca85630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9f9dffb640 10 client.19398 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.421+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bca831f0 0x55a1bca85630 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63967010c501165 server_cookie=9617b48a843f7509 in_seq=0 out_seq=0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1728514271 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1bc9db850 con 0x55a1bca831f0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9f9dffb640 10 client.19398 handle_command_reply: tid=0 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 client.19398 shutdown 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 msgr2=0x55a1bca70300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800be40 tx=0x7f9fa8012820 comp rx=0 tx=0).stop 2024-09-05T06:58:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 shutdown_connections 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bca831f0 0x55a1bca85630 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca70300 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1728514271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9f9dffb640 0 client.19398 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 >> 172.21.15.86:0/1728514271 conn(0x55a1bca64a30 msgr2=0x55a1bca6a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 shutdown_connections 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1728514271 wait complete. 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.422+0000 7f9fbf0df740 20 client.19398 trim_cache size 0 max 16384 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.439+0000 7f9fbf0df740 1 Processor -- start 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 -- start start 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca8b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81af110 con 0x55a1bca68dc0 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b0cb0 con 0x55a1bca73680 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81af7e0 con 0x55a1bca684a0 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41846/0 (socket says 172.21.15.86:41846) 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1981365339 learned_addr learned my addr 172.21.15.86:0/1981365339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca8b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1981365339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 msgr2=0x55a1bca7f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1981365339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 msgr2=0x55a1bca8b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca8b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 -- 172.21.15.86:0/1981365339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bca682d0 con 0x55a1bca68dc0 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca8b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.440+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800fe40 tx=0x7f9fa8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be08cf3170c167 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9f9ffff640 1 -- 172.21.15.86:0/1981365339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800f5c0 con 0x55a1bca68dc0 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9f9ffff640 1 -- 172.21.15.86:0/1981365339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa80054e0 con 0x55a1bca68dc0 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9f9ffff640 1 -- 172.21.15.86:0/1981365339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8010ad0 con 0x55a1bca68dc0 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1981365339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 msgr2=0x55a1bca8af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800fe40 tx=0x7f9fa8005c50 comp rx=0 tx=0).stop 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1981365339 shutdown_connections 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca73680 0x55a1bca8b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca68dc0 0x55a1bca8af00 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1981365339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.441+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1981365339 >> 172.21.15.86:0/1981365339 conn(0x55a1bca64a30 msgr2=0x55a1bca6f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1981365339 shutdown_connections 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1981365339 wait complete. 2024-09-05T06:58:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- start start 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 Processor -- start 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 0x55a1bca76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b05e0 con 0x55a1bca73680 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81aff10 con 0x55a1bca68dc0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ad110 con 0x55a1bca684a0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fbf0df740 10 client.0 _ll_register_callbacks cb 0x7f9fa800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41858/0 (socket says 172.21.15.86:41858) 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1646374202 learned_addr learned my addr 172.21.15.86:0/1646374202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.442+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 0x55a1bca76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1646374202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 msgr2=0x55a1bca7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 msgr2=0x55a1bca76ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 0x55a1bca76ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 -- 172.21.15.86:0/1646374202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bca779d0 con 0x55a1bca73680 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb5d9b640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 0x55a1bca76ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fb6d9d640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x55a1bca65db0 tx=0x7f9fb0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383dc4e75d9e1e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0022040 con 0x55a1bca73680 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa80054e0 con 0x55a1bca73680 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f9fb0004bf0 con 0x55a1bca73680 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0039050 con 0x55a1bca73680 2024-09-05T06:58:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fbf0df740 10 client.19404 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1bcaedce0 con 0x55a1bca73680 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.443+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa80054e0 con 0x55a1bca73680 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9fb0005140 con 0x55a1bca73680 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 10 client.19404 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 20 client.19404 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bcaee210 0x55a1bcaf0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 4 client.19404 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1bc9d5d60 con 0x55a1bcaee210 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bcaee210 0x55a1bcaf0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9fb659c640 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bcaee210 0x55a1bcaf0600 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c1061951eb352c server_cookie=6a529c5881e43063 in_seq=0 out_seq=0 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.444+0000 7f9f9dffb640 10 client.19404 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9f9dffb640 1 -- 172.21.15.86:0/1646374202 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a1bc9d5d60 con 0x55a1bcaee210 2024-09-05T06:58:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9f9dffb640 10 client.19404 handle_command_reply: tid=0 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 client.19404 shutdown 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 msgr2=0x55a1bca773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x55a1bca65db0 tx=0x7f9fb0009f30 comp rx=0 tx=0).stop 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 shutdown_connections 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a1bcaee210 0x55a1bcaf0600 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1bca73680 0x55a1bca773e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bca68dc0 0x55a1bca76ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 --2- 172.21.15.86:0/1646374202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1bca684a0 0x55a1bca7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9f9dffb640 0 client.19404 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 >> 172.21.15.86:0/1646374202 conn(0x55a1bca64a30 msgr2=0x55a1bca69f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 shutdown_connections 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 1 -- 172.21.15.86:0/1646374202 wait complete. 2024-09-05T06:58:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbf0df740 20 client.19404 trim_cache size 0 max 16384 2024-09-05T06:58:42.463 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:58:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:58:42.463 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 msgr2=0x7f9fa00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 0x7f9fa00a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0010800 tx=0x7f9fb001d000 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 msgr2=0x7f9fb8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.445+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9fac005ab0 tx=0x7f9fac003cd0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 shutdown_connections 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9fa00a2cf0 0x7f9fa00a51b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb8106f10 0x7f9fb819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb8102790 0x7f9fb81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 --2- 172.21.15.86:0/3305490928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fb8101de0 0x7f9fb8198370 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 >> 172.21.15.86:0/3305490928 conn(0x7f9fb80fd8c0 msgr2=0x7f9fb80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 shutdown_connections 2024-09-05T06:58:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:42.446+0000 7f9fbcd06640 1 -- 172.21.15.86:0/3305490928 wait complete. 2024-09-05T06:58:42.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.51 MB/sec execute 120 sec latency 21903.494 ms 2024-09-05T06:58:43.425 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:58:43.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:58:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:43 smithi086 ceph-mon[30943]: pgmap v548: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-05T06:58:43.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.46 MB/sec execute 121 sec latency 22903.557 ms 2024-09-05T06:58:43.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:43 smithi096 ceph-mon[34468]: pgmap v548: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-05T06:58:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:43 smithi115 ceph-mon[34791]: pgmap v548: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-05T06:58:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 -- 172.21.15.86:0/2766864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 msgr2=0x7fc1c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/2766864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c810da40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009ca0 tx=0x7fc1c002a950 comp rx=0 tx=0).stop 2024-09-05T06:58:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 -- 172.21.15.86:0/2766864598 shutdown_connections 2024-09-05T06:58:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/2766864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c810da40 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/2766864598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8102570 0x7fc1c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/2766864598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8100e20 0x7fc1c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 -- 172.21.15.86:0/2766864598 >> 172.21.15.86:0/2766864598 conn(0x7fc1c80fc7d0 msgr2=0x7fc1c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 -- 172.21.15.86:0/2766864598 shutdown_connections 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.197+0000 7fc1cdd70640 1 -- 172.21.15.86:0/2766864598 wait complete. 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 Processor -- start 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 -- start start 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8100e20 0x7fc1c8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8102570 0x7fc1c8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8110e80 con 0x7fc1c8102ef0 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8110d00 con 0x7fc1c8100e20 2024-09-05T06:58:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1cdd70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8111000 con 0x7fc1c8102570 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43822/0 (socket says 172.21.15.86:43822) 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 -- 172.21.15.86:0/1664310833 learned_addr learned my addr 172.21.15.86:0/1664310833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c6ffd640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8102570 0x7fc1c8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c77fe640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8100e20 0x7fc1c8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 -- 172.21.15.86:0/1664310833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8102570 msgr2=0x7fc1c8199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8102570 0x7fc1c8199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 -- 172.21.15.86:0/1664310833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8100e20 msgr2=0x7fc1c8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8100e20 0x7fc1c8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.198+0000 7fc1c7fff640 1 -- 172.21.15.86:0/1664310833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c81a4810 con 0x7fc1c8102ef0 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1c7fff640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098d0 tx=0x7fc1c0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0034040 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c81a49e0 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c0030800 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004790 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.199+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81a4e90 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.200+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8101200 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.201+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1c0037020 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.202+0000 7fc1c4ff9640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 0x7fc1b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.202+0000 7fc1c77fe640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 0x7fc1b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.202+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7fc1c00f0880 con 0x7fc1c8102ef0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.202+0000 7fc1c77fe640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 0x7fc1b00a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80058c0 tx=0x7fc1b800a4c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.206+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1c00bd110 con 0x7fc1c8102ef0 2024-09-05T06:58:44.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.405+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c81a50c0 con 0x7fc1c8102ef0 2024-09-05T06:58:44.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.406+0000 7fc1c4ff9640 1 -- 172.21.15.86:0/1664310833 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7fc1c00bca10 con 0x7fc1c8102ef0 2024-09-05T06:58:44.407 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:58:44.407 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:58:44.407 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 msgr2=0x7fc1b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 0x7fc1b00a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80058c0 tx=0x7fc1b800a4c0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 msgr2=0x7fc1c81a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098d0 tx=0x7fc1c0003ef0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 shutdown_connections 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1b00a2cf0 0x7fc1b00a51b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8102ef0 0x7fc1c81a42d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c8102570 0x7fc1c8199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 --2- 172.21.15.86:0/1664310833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1c8100e20 0x7fc1c8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 >> 172.21.15.86:0/1664310833 conn(0x7fc1c80fc7d0 msgr2=0x7fc1c81068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 shutdown_connections 2024-09-05T06:58:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:44.408+0000 7fc1cdd70640 1 -- 172.21.15.86:0/1664310833 wait complete. 2024-09-05T06:58:44.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:44 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1664310833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:44.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.42 MB/sec execute 122 sec latency 23903.635 ms 2024-09-05T06:58:44.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:44 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1664310833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:44.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:44 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1664310833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:45.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:58:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:45 smithi086 ceph-mon[30943]: pgmap v549: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:45.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.37 MB/sec execute 123 sec latency 24903.708 ms 2024-09-05T06:58:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:45 smithi096 ceph-mon[34468]: pgmap v549: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 -- 172.21.15.86:0/3208701837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c106a80 msgr2=0x7faa6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 --2- 172.21.15.86:0/3208701837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c106a80 0x7faa6c106e60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7faa60009930 tx=0x7faa6002a420 comp rx=0 tx=0).stop 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 -- 172.21.15.86:0/3208701837 shutdown_connections 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 --2- 172.21.15.86:0/3208701837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c101420 0x7faa6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 --2- 172.21.15.86:0/3208701837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c100a80 0x7faa6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 --2- 172.21.15.86:0/3208701837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c106a80 0x7faa6c106e60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 -- 172.21.15.86:0/3208701837 >> 172.21.15.86:0/3208701837 conn(0x7faa6c0fc7a0 msgr2=0x7faa6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 -- 172.21.15.86:0/3208701837 shutdown_connections 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.865+0000 7faa72bec640 1 -- 172.21.15.86:0/3208701837 wait complete. 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 Processor -- start 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 -- start start 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c101420 0x7faa6c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c106a80 0x7faa6c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c110d30 con 0x7faa6c100a80 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c110bb0 con 0x7faa6c106a80 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa72bec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c110eb0 con 0x7faa6c101420 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43826/0 (socket says 172.21.15.86:43826) 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 -- 172.21.15.86:0/2281747440 learned_addr learned my addr 172.21.15.86:0/2281747440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa71189640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c106a80 0x7faa6c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa6bfff640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c101420 0x7faa6c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 -- 172.21.15.86:0/2281747440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c101420 msgr2=0x7faa6c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c101420 0x7faa6c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 -- 172.21.15.86:0/2281747440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c106a80 msgr2=0x7faa6c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c106a80 0x7faa6c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.866+0000 7faa70988640 1 -- 172.21.15.86:0/2281747440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c19c200 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa70988640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7faa60004770 tx=0x7faa600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa69ffb640 1 -- 172.21.15.86:0/2281747440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6003f070 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c19c490 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa69ffb640 1 -- 172.21.15.86:0/2281747440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa6003b460 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa69ffb640 1 -- 172.21.15.86:0/2281747440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60035070 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.867+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1a3dd0 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.869+0000 7faa69ffb640 1 -- 172.21.15.86:0/2281747440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faa6003b610 con 0x7faa6c100a80 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 Processor -- start 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 -- start start 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b7444d50 0x5601b74432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7443c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601b73a72f0 con 0x5601b744e580 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5601b734ecf0 con 0x5601b74437e0 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa74fc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5601b73a32b0 con 0x5601b7444d50 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa69ffb640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 0x7faa5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43836/0 (socket says 172.21.15.86:43836) 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 -- 172.21.15.86:0/735081514 learned_addr learned my addr 172.21.15.86:0/735081514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa71189640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 0x7faa5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa70988640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7443c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa71189640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b7444d50 0x5601b74432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa69ffb640 1 -- 172.21.15.86:0/2281747440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7faa600efc20 con 0x7faa6c100a80 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 -- 172.21.15.86:0/735081514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b7444d50 msgr2=0x5601b74432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b7444d50 0x5601b74432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 -- 172.21.15.86:0/735081514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 msgr2=0x5601b7443c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7443c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa6bfff640 1 -- 172.21.15.86:0/735081514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b73333a0 con 0x5601b744e580 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.870+0000 7faa70988640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7443c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa71189640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 0x7faa5c0a51b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa64007fd0 tx=0x7faa64007d20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa6bfff640 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc90 tx=0x7faa5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beba5dec7ff7e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa56ffd640 1 -- 172.21.15.86:0/735081514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58015070 con 0x5601b744e580 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa56ffd640 1 -- 172.21.15.86:0/735081514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5801a910 con 0x5601b744e580 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa56ffd640 1 -- 172.21.15.86:0/735081514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580117d0 con 0x5601b744e580 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 -- 172.21.15.86:0/735081514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 msgr2=0x5601b7450970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc90 tx=0x7faa5800f730 comp rx=0 tx=0).stop 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 -- 172.21.15.86:0/735081514 shutdown_connections 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7450970 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7443c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.871+0000 7faa74fc5740 1 --2- 172.21.15.86:0/735081514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b7444d50 0x5601b74432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- 172.21.15.86:0/735081514 >> 172.21.15.86:0/735081514 conn(0x5601b740a310 msgr2=0x5601b743fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- 172.21.15.86:0/735081514 shutdown_connections 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- 172.21.15.86:0/735081514 wait complete. 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- start start 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 Processor -- start 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b744b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b744b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601b7451280 con 0x5601b74437e0 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5601b7362060 con 0x5601b7444d50 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5601b7451400 con 0x5601b744e580 2024-09-05T06:58:45.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa74fc5740 10 client.0 _ll_register_callbacks cb 0x5601b70b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa70988640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.872+0000 7faa70988640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43846/0 (socket says 172.21.15.86:43846) 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 -- 172.21.15.86:0/3681586237 learned_addr learned my addr 172.21.15.86:0/3681586237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa71189640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b744b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 -- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 msgr2=0x5601b744b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b744b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 -- 172.21.15.86:0/3681586237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 msgr2=0x5601b744b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b744b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 -- 172.21.15.86:0/3681586237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b71183c0 con 0x5601b74437e0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa70988640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7faa600ed530 tx=0x7faa60086dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d568a8503f84bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60048070 con 0x5601b74437e0 2024-09-05T06:58:45.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7faa6003b940 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5801a910 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faa600b7fa0 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa74fc5740 10 client.19422 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5601b745cce0 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.873+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5801a910 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7faa6004b060 con 0x5601b74437e0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 10 client.19422 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 10 client.19422 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 20 client.19422 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b745e2a0 0x5601b7460710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 4 client.19422 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b73b4860 con 0x5601b745e2a0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa6bfff640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b745e2a0 0x5601b7460710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa4bfff640 10 client.19422 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.874+0000 7faa6bfff640 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b745e2a0 0x5601b7460710 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9226070d349c1236 server_cookie=96f77ca2485394df in_seq=0 out_seq=0 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa4bfff640 1 -- 172.21.15.86:0/3681586237 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601b73b4860 con 0x5601b745e2a0 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa4bfff640 10 client.19422 handle_command_reply: tid=0 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 client.19422 shutdown 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 msgr2=0x5601b744df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7faa600ed530 tx=0x7faa60086dd0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 shutdown_connections 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b745e2a0 0x5601b7460710 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b744b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b744b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 --2- 172.21.15.86:0/3681586237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b74437e0 0x5601b744df40 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa4bfff640 0 client.19422 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 >> 172.21.15.86:0/3681586237 conn(0x5601b740a310 msgr2=0x5601b7445b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 shutdown_connections 2024-09-05T06:58:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.875+0000 7faa74fc5740 1 -- 172.21.15.86:0/3681586237 wait complete. 2024-09-05T06:58:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.876+0000 7faa74fc5740 20 client.19422 trim_cache size 0 max 16384 2024-09-05T06:58:45.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.888+0000 7faa74fc5740 1 Processor -- start 2024-09-05T06:58:45.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.888+0000 7faa74fc5740 1 -- start start 2024-09-05T06:58:45.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7446330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b7466020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ace70 con 0x5601b7444d50 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1aea10 con 0x5601b74437e0 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa74fc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ad540 con 0x5601b744e580 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43862/0 (socket says 172.21.15.86:43862) 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 -- 172.21.15.86:0/2585231509 learned_addr learned my addr 172.21.15.86:0/2585231509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa70988640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7446330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa71189640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b7466020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 -- 172.21.15.86:0/2585231509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 msgr2=0x5601b7466020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b7466020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 -- 172.21.15.86:0/2585231509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 msgr2=0x5601b7446330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7446330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 -- 172.21.15.86:0/2585231509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b74445b0 con 0x5601b7444d50 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa71189640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b7466020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa6bfff640 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc90 tx=0x7faa5800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883902268c96a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa567fc640 1 -- 172.21.15.86:0/2585231509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58015040 con 0x5601b7444d50 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa567fc640 1 -- 172.21.15.86:0/2585231509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa58004640 con 0x5601b7444d50 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.889+0000 7faa567fc640 1 -- 172.21.15.86:0/2585231509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58026680 con 0x5601b7444d50 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- 172.21.15.86:0/2585231509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 msgr2=0x5601b7446870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc90 tx=0x7faa5800b9e0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- 172.21.15.86:0/2585231509 shutdown_connections 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b744e580 0x5601b7466020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b7444d50 0x5601b7446870 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 --2- 172.21.15.86:0/2585231509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b74437e0 0x5601b7446330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- 172.21.15.86:0/2585231509 >> 172.21.15.86:0/2585231509 conn(0x5601b740a310 msgr2=0x5601b743cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- 172.21.15.86:0/2585231509 shutdown_connections 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- 172.21.15.86:0/2585231509 wait complete. 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 -- start start 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.890+0000 7faa74fc5740 1 Processor -- start 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b74437e0 0x5601b7456690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ae340 con 0x5601b744e580 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1adc70 con 0x5601b7444d50 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1aae70 con 0x5601b74437e0 2024-09-05T06:58:45.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa74fc5740 10 client.0 _ll_register_callbacks cb 0x7faa60005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43870/0 (socket says 172.21.15.86:43870) 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48012/0 (socket says 172.21.15.86:48012) 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 -- 172.21.15.86:0/85643749 learned_addr learned my addr 172.21.15.86:0/85643749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa70988640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b74437e0 0x5601b7456690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 -- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b74437e0 msgr2=0x5601b7456690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b74437e0 0x5601b7456690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 -- 172.21.15.86:0/85643749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 msgr2=0x5601b7456bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa71189640 1 -- 172.21.15.86:0/85643749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b7457210 con 0x5601b744e580 2024-09-05T06:58:45.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.891+0000 7faa6bfff640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa71189640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7faa64014680 tx=0x7faa64007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a40c95fddc70464 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64012e60 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7faa64004b20 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58004640 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faa64014e50 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 10 client.19428 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5601b74c94b0 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa58004640 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7faa64029480 con 0x5601b744e580 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 10 client.19428 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 10 client.19428 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:58:45.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 20 client.19428 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b74c9bd0 0x5601b74cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 4 client.19428 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.892+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601b73aed70 con 0x5601b74c9bd0 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa70988640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b74c9bd0 0x5601b74cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa70988640 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b74c9bd0 0x5601b74cbff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6ff5549ee763de1 server_cookie=4c446fe1910a1083 in_seq=0 out_seq=0 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa4bfff640 10 client.19428 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa4bfff640 1 -- 172.21.15.86:0/85643749 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601b73aed70 con 0x5601b74c9bd0 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa4bfff640 10 client.19428 handle_command_reply: tid=0 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 client.19428 shutdown 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 msgr2=0x5601b7453660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7faa64014680 tx=0x7faa64007520 comp rx=0 tx=0).stop 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 shutdown_connections 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5601b74c9bd0 0x5601b74cbff0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b744e580 0x5601b7453660 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601b7444d50 0x5601b7456bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 --2- 172.21.15.86:0/85643749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601b74437e0 0x5601b7456690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 >> 172.21.15.86:0/85643749 conn(0x5601b740a310 msgr2=0x5601b744ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa4bfff640 0 client.19428 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 shutdown_connections 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.893+0000 7faa74fc5740 1 -- 172.21.15.86:0/85643749 wait complete. 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa74fc5740 20 client.19428 trim_cache size 0 max 16384 2024-09-05T06:58:45.907 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:45.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 msgr2=0x7faa5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 0x7faa5c0a51b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa64007fd0 tx=0x7faa64007d20 comp rx=0 tx=0).stop 2024-09-05T06:58:45.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 msgr2=0x7faa6c197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:45.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7faa60004770 tx=0x7faa600042f0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 shutdown_connections 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faa5c0a2cf0 0x7faa5c0a51b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa6c106a80 0x7faa6c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa6c101420 0x7faa6c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 --2- 172.21.15.86:0/2281747440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa6c100a80 0x7faa6c197250 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 >> 172.21.15.86:0/2281747440 conn(0x7faa6c0fc7a0 msgr2=0x7faa6c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 shutdown_connections 2024-09-05T06:58:45.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:45.894+0000 7faa72bec640 1 -- 172.21.15.86:0/2281747440 wait complete. 2024-09-05T06:58:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:45 smithi115 ceph-mon[34791]: pgmap v549: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:46.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:58:46.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.33 MB/sec execute 124 sec latency 25903.767 ms 2024-09-05T06:58:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 -- 172.21.15.86:0/86600474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 msgr2=0x7f48841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 --2- 172.21.15.86:0/86600474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841067e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02a410 comp rx=0 tx=0).stop 2024-09-05T06:58:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 -- 172.21.15.86:0/86600474 shutdown_connections 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 --2- 172.21.15.86:0/86600474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 0x7f488410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 --2- 172.21.15.86:0/86600474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841067e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 --2- 172.21.15.86:0/86600474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 -- 172.21.15.86:0/86600474 >> 172.21.15.86:0/86600474 conn(0x7f48840fd8c0 msgr2=0x7f48840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 -- 172.21.15.86:0/86600474 shutdown_connections 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 -- 172.21.15.86:0/86600474 wait complete. 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.401+0000 7f4889505640 1 Processor -- start 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 -- start start 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 0x7f488419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4884111e40 con 0x7f4884102790 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4884111cc0 con 0x7f4884101de0 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4889505640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4884111fc0 con 0x7f4884106f10 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43896/0 (socket says 172.21.15.86:43896) 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 -- 172.21.15.86:0/603412823 learned_addr learned my addr 172.21.15.86:0/603412823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f48837fe640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 0x7f488419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4882ffd640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 -- 172.21.15.86:0/603412823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 msgr2=0x7f488419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 0x7f488419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 -- 172.21.15.86:0/603412823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 msgr2=0x7f48841984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 -- 172.21.15.86:0/603412823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488419d460 con 0x7f4884102790 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4882ffd640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:47.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f487bfff640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f486c006010 tx=0x7f486c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.402+0000 7f4880ff9640 1 -- 172.21.15.86:0/603412823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c03f030 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.403+0000 7f4880ff9640 1 -- 172.21.15.86:0/603412823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c0035f0 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.403+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488419d6f0 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.403+0000 7f4880ff9640 1 -- 172.21.15.86:0/603412823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c035070 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.403+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841a5030 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.405+0000 7f488b8de740 1 Processor -- start 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.405+0000 7f488b8de740 1 -- start start 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f4880ff9640 1 -- 172.21.15.86:0/603412823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f486c003b10 con 0x7f4884102790 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 0x5563c6749090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 0x5563c6749a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c6756760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563c66af2f0 con 0x5563c6754370 2024-09-05T06:58:47.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5563c6656cf0 con 0x5563c67495d0 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f488b8de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5563c66ab2b0 con 0x5563c674ab40 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c6756760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c6756760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43912/0 (socket says 172.21.15.86:43912) 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f487bfff640 1 -- 172.21.15.86:0/761514531 learned_addr learned my addr 172.21.15.86:0/761514531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f4882ffd640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 0x5563c6749a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.406+0000 7f48837fe640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 0x5563c6749090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 -- 172.21.15.86:0/761514531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 msgr2=0x5563c6749a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 0x5563c6749a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 -- 172.21.15.86:0/761514531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 msgr2=0x5563c6756760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c6756760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 -- 172.21.15.86:0/761514531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c663b3a0 con 0x5563c674ab40 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f4882ffd640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 0x5563c6749a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f4880ff9640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 0x7f48700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f4882ffd640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 0x7f48700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f4880ff9640 1 -- 172.21.15.86:0/603412823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f486c0f0410 con 0x7f4884102790 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.407+0000 7f48837fe640 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 0x5563c6749090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f487c009f70 tx=0x7f487c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ba0e8d7d3e091f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.408+0000 7f4882ffd640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 0x7f48700a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5563c6746ac0 tx=0x7f487400c390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.408+0000 7f48797fa640 1 -- 172.21.15.86:0/761514531 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c013850 con 0x5563c674ab40 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.408+0000 7f48797fa640 1 -- 172.21.15.86:0/761514531 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f487c019070 con 0x5563c674ab40 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.408+0000 7f48797fa640 1 -- 172.21.15.86:0/761514531 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c014d50 con 0x5563c674ab40 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 -- 172.21.15.86:0/761514531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 msgr2=0x5563c6749090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 0x5563c6749090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f487c009f70 tx=0x7f487c009650 comp rx=0 tx=0).stop 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 -- 172.21.15.86:0/761514531 shutdown_connections 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c6756760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c67495d0 0x5563c6749a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 --2- 172.21.15.86:0/761514531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c674ab40 0x5563c6749090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 -- 172.21.15.86:0/761514531 >> 172.21.15.86:0/761514531 conn(0x5563c6712310 msgr2=0x5563c6745c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 -- 172.21.15.86:0/761514531 shutdown_connections 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.409+0000 7f488b8de740 1 -- 172.21.15.86:0/761514531 wait complete. 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 -- start start 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 Processor -- start 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c67514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563c675b4d0 con 0x5563c6754370 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5563c666a060 con 0x5563c674ab40 2024-09-05T06:58:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5563c675b650 con 0x5563c67495d0 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f488b8de740 10 client.0 _ll_register_callbacks cb 0x5563c63bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f4882ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f487bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f487bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:48046/0 (socket says 172.21.15.86:48046) 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f4882ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49992/0 (socket says 172.21.15.86:49992) 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f487bfff640 1 -- 172.21.15.86:0/1566772787 learned_addr learned my addr 172.21.15.86:0/1566772787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.410+0000 7f48837fe640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c67514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 -- 172.21.15.86:0/1566772787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 msgr2=0x5563c6753a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 -- 172.21.15.86:0/1566772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 msgr2=0x5563c67514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c67514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 -- 172.21.15.86:0/1566772787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c64203c0 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f4882ffd640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f48837fe640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c67514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f487bfff640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c4c30 tx=0x7f486c0f3b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4f8cc7c40b23a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f486a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c049040 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f487c019070 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f486c0b8b60 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f486c0bc2c0 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f488b8de740 10 client.28307 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.411+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c676b340 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f487c019070 con 0x5563c674ab40 2024-09-05T06:58:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f486c031aa0 con 0x5563c674ab40 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 10 client.28307 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 10 client.28307 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 20 client.28307 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c676c7a0 0x5563c676ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 4 client.28307 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c66bc860 con 0x5563c676c7a0 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.412+0000 7f4882ffd640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c676c7a0 0x5563c676ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.413+0000 7f486a7fc640 10 client.28307 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.413+0000 7f4882ffd640 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c676c7a0 0x5563c676ebb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1dea757a1fe40540 server_cookie=9b3884c8ca4ba7dd in_seq=0 out_seq=0 2024-09-05T06:58:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.413+0000 7f486a7fc640 1 -- 172.21.15.86:0/1566772787 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563c66bc860 con 0x5563c676c7a0 2024-09-05T06:58:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.413+0000 7f486a7fc640 10 client.28307 handle_command_reply: tid=0 2024-09-05T06:58:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 client.28307 shutdown 2024-09-05T06:58:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 msgr2=0x5563c6750fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c4c30 tx=0x7f486c0f3b50 comp rx=0 tx=0).stop 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 shutdown_connections 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c676c7a0 0x5563c676ebb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c6754370 0x5563c67514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c674ab40 0x5563c6750fa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f486a7fc640 0 client.28307 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 --2- 172.21.15.86:0/1566772787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6753a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 >> 172.21.15.86:0/1566772787 conn(0x5563c6712310 msgr2=0x5563c674b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 shutdown_connections 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 1 -- 172.21.15.86:0/1566772787 wait complete. 2024-09-05T06:58:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.414+0000 7f488b8de740 20 client.28307 trim_cache size 0 max 16384 2024-09-05T06:58:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 Processor -- start 2024-09-05T06:58:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 -- start start 2024-09-05T06:58:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c67532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c674c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48841acf70 con 0x5563c674ab40 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48841aeb10 con 0x5563c6754370 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f488b8de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ad640 con 0x5563c67495d0 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43934/0 (socket says 172.21.15.86:43934) 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 -- 172.21.15.86:0/2659245992 learned_addr learned my addr 172.21.15.86:0/2659245992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f4882ffd640 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c67532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f48837fe640 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c674c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 -- 172.21.15.86:0/2659245992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 msgr2=0x5563c67532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c67532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 -- 172.21.15.86:0/2659245992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 msgr2=0x5563c674c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.427+0000 7f487bfff640 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c674c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.428+0000 7f487bfff640 1 -- 172.21.15.86:0/2659245992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c6746c80 con 0x5563c674ab40 2024-09-05T06:58:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.428+0000 7f487bfff640 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c0a60 tx=0x7f486c0b8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b85c6f029795360 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.428+0000 7f4878ff9640 1 -- 172.21.15.86:0/2659245992 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c049070 con 0x5563c674ab40 2024-09-05T06:58:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.428+0000 7f4878ff9640 1 -- 172.21.15.86:0/2659245992 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c0bc9b0 con 0x5563c674ab40 2024-09-05T06:58:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.428+0000 7f4878ff9640 1 -- 172.21.15.86:0/2659245992 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c0037a0 con 0x5563c674ab40 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- 172.21.15.86:0/2659245992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 msgr2=0x5563c6759780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c0a60 tx=0x7f486c0b8190 comp rx=0 tx=0).stop 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- 172.21.15.86:0/2659245992 shutdown_connections 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c674c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6759780 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- 172.21.15.86:0/2659245992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c67532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- 172.21.15.86:0/2659245992 >> 172.21.15.86:0/2659245992 conn(0x5563c6712310 msgr2=0x5563c6744d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- 172.21.15.86:0/2659245992 shutdown_connections 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- 172.21.15.86:0/2659245992 wait complete. 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- start start 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 Processor -- start 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6769820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c6765090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ae440 con 0x5563c674ab40 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48841add70 con 0x5563c6754370 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48841aaf70 con 0x5563c67495d0 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f488b8de740 10 client.0 _ll_register_callbacks cb 0x7f486c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:47.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f487bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43950/0 (socket says 172.21.15.86:43950) 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.429+0000 7f487bfff640 1 -- 172.21.15.86:0/2297378238 learned_addr learned my addr 172.21.15.86:0/2297378238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f4882ffd640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6769820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 -- 172.21.15.86:0/2297378238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 msgr2=0x5563c6769820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6769820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 -- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 msgr2=0x5563c6765090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c6765090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 -- 172.21.15.86:0/2297378238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c6765680 con 0x5563c674ab40 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f487bfff640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f486c031440 tx=0x7f486c0c0910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3140fe307f7306 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c049070 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f486c0c4ed0 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486c0bc9b0 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f486c0bbe50 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f488b8de740 10 client.19446 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563c6765680 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486c0bc9b0 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.430+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f486c0b8240 con 0x5563c674ab40 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 10 client.19446 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 10 client.19446 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 20 client.19446 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c67d74e0 0x5563c67d9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 4 client.19446 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563c66b6d70 con 0x5563c67d74e0 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f4882ffd640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c67d74e0 0x5563c67d9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f4882ffd640 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c67d74e0 0x5563c67d9900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca01b14042345c32 server_cookie=5f9a9758be23b364 in_seq=0 out_seq=0 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f486a7fc640 10 client.19446 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f486a7fc640 1 -- 172.21.15.86:0/2297378238 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563c66b6d70 con 0x5563c67d74e0 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f486a7fc640 10 client.19446 handle_command_reply: tid=0 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 1 client.19446 shutdown 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 msgr2=0x5563c6764b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.431+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f486c031440 tx=0x7f486c0c0910 comp rx=0 tx=0).stop 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 shutdown_connections 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5563c67d74e0 0x5563c67d9900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563c6754370 0x5563c6765090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563c674ab40 0x5563c6764b50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f486a7fc640 0 client.19446 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 --2- 172.21.15.86:0/2297378238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5563c67495d0 0x5563c6769820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 >> 172.21.15.86:0/2297378238 conn(0x5563c6712310 msgr2=0x5563c6758770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 shutdown_connections 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 1 -- 172.21.15.86:0/2297378238 wait complete. 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f488b8de740 20 client.19446 trim_cache size 0 max 16384 2024-09-05T06:58:47.439 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 msgr2=0x7f48700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 0x7f48700a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5563c6746ac0 tx=0x7f487400c390 comp rx=0 tx=0).stop 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 msgr2=0x7f48841989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f486c006010 tx=0x7f486c004270 comp rx=0 tx=0).stop 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 shutdown_connections 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f48700a2cf0 0x7f48700a51b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4884106f10 0x7f488419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4884102790 0x7f48841989f0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 --2- 172.21.15.86:0/603412823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884101de0 0x7f48841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 >> 172.21.15.86:0/603412823 conn(0x7f48840fd8c0 msgr2=0x7f48840fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 shutdown_connections 2024-09-05T06:58:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:47.432+0000 7f4889505640 1 -- 172.21.15.86:0/603412823 wait complete. 2024-09-05T06:58:47.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:58:47.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:47 smithi086 ceph-mon[30943]: pgmap v550: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:47.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.29 MB/sec execute 125 sec latency 26903.819 ms 2024-09-05T06:58:47.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:47 smithi096 ceph-mon[34468]: pgmap v550: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:47.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:47 smithi115 ceph-mon[34791]: pgmap v550: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2024-09-05T06:58:48.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:58:48.332 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:48.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.25 MB/sec execute 126 sec latency 27903.846 ms 2024-09-05T06:58:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 -- 172.21.15.86:0/4236540178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 msgr2=0x7f66401021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 --2- 172.21.15.86:0/4236540178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f66401021d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6630009900 tx=0x7f663002a400 comp rx=0 tx=0).stop 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 -- 172.21.15.86:0/4236540178 shutdown_connections 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 --2- 172.21.15.86:0/4236540178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6640106f20 0x7f664010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 --2- 172.21.15.86:0/4236540178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66401027a0 0x7f66401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 --2- 172.21.15.86:0/4236540178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f66401021d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 -- 172.21.15.86:0/4236540178 >> 172.21.15.86:0/4236540178 conn(0x7f66400fd8b0 msgr2=0x7f66400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 -- 172.21.15.86:0/4236540178 shutdown_connections 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.836+0000 7f6644f4d640 1 -- 172.21.15.86:0/4236540178 wait complete. 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 Processor -- start 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 -- start start 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66401027a0 0x7f664019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 0x7f664010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6640111df0 con 0x7f6640101df0 2024-09-05T06:58:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6640111c70 con 0x7f6640106f20 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f6644f4d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6640111f70 con 0x7f66401027a0 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f663e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f663e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43970/0 (socket says 172.21.15.86:43970) 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.837+0000 7f663e59c640 1 -- 172.21.15.86:0/2458977964 learned_addr learned my addr 172.21.15.86:0/2458977964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663dd9b640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66401027a0 0x7f664019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663ed9d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 0x7f664010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 -- 172.21.15.86:0/2458977964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66401027a0 msgr2=0x7f664019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66401027a0 0x7f664019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 -- 172.21.15.86:0/2458977964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 msgr2=0x7f664010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 0x7f664010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 -- 172.21.15.86:0/2458977964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664010e730 con 0x7f6640101df0 2024-09-05T06:58:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663ed9d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 0x7f664010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f663e59c640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6630005940 tx=0x7f6630004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663003f030 con 0x7f6640101df0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6630003aa0 con 0x7f6640101df0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664010e9c0 con 0x7f6640101df0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035070 con 0x7f6640101df0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.838+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664019abe0 con 0x7f6640101df0 2024-09-05T06:58:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.840+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66401032d0 con 0x7f6640101df0 2024-09-05T06:58:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.842+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6630042030 con 0x7f6640101df0 2024-09-05T06:58:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.842+0000 7f662f7fe640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 0x7f66280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:48.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.843+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f66300f0320 con 0x7f6640101df0 2024-09-05T06:58:48.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.843+0000 7f663dd9b640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 0x7f66280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:48.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.844+0000 7f663dd9b640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 0x7f66280a5230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f66401977d0 tx=0x7f663400b590 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:48.846+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66300bcb00 con 0x7f6640101df0 2024-09-05T06:58:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.057+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66401021d0 con 0x7f6640101df0 2024-09-05T06:58:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.057+0000 7f662f7fe640 1 -- 172.21.15.86:0/2458977964 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7f66401021d0 con 0x7f6640101df0 2024-09-05T06:58:49.058 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:58:49.058 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:58:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 msgr2=0x7f66280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 0x7f66280a5230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f66401977d0 tx=0x7f663400b590 comp rx=0 tx=0).stop 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 msgr2=0x7f664019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6630005940 tx=0x7f6630004270 comp rx=0 tx=0).stop 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 shutdown_connections 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f66280a2d70 0x7f66280a5230 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6640106f20 0x7f664010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66401027a0 0x7f664019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 --2- 172.21.15.86:0/2458977964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6640101df0 0x7f664019f030 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 >> 172.21.15.86:0/2458977964 conn(0x7f66400fd8b0 msgr2=0x7f66400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 shutdown_connections 2024-09-05T06:58:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:49.059+0000 7f6644f4d640 1 -- 172.21.15.86:0/2458977964 wait complete. 2024-09-05T06:58:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:49 smithi086 ceph-mon[30943]: pgmap v551: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-05T06:58:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:49 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2458977964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:49.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.21 MB/sec execute 127 sec latency 28903.890 ms 2024-09-05T06:58:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:49 smithi096 ceph-mon[34468]: pgmap v551: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-05T06:58:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:49 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2458977964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:49.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 57cec38 --release --await' 2024-09-05T06:58:49.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 57cec38 --release --await 2024-09-05T06:58:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:49 smithi115 ceph-mon[34791]: pgmap v551: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-05T06:58:49.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:49 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2458977964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:58:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 -- 172.21.15.86:0/3527870315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106b80 msgr2=0x7f244c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 --2- 172.21.15.86:0/3527870315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106b80 0x7f244c106f60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2448009900 tx=0x7f244802a410 comp rx=0 tx=0).stop 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 -- 172.21.15.86:0/3527870315 shutdown_connections 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 --2- 172.21.15.86:0/3527870315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 0x7f244c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 --2- 172.21.15.86:0/3527870315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c100100 0x7f244c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 --2- 172.21.15.86:0/3527870315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c106b80 0x7f244c106f60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 -- 172.21.15.86:0/3527870315 >> 172.21.15.86:0/3527870315 conn(0x7f244c077230 msgr2=0x7f244c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 -- 172.21.15.86:0/3527870315 shutdown_connections 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.015+0000 7f24537d1640 1 -- 172.21.15.86:0/3527870315 wait complete. 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 Processor -- start 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 -- start start 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 0x7f244c101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 0x7f244c101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c111f40 con 0x7f244c100100 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f244c111dc0 con 0x7f244c106b80 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f24537d1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1120c0 con 0x7f244c100aa0 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43988/0 (socket says 172.21.15.86:43988) 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f245156d640 1 -- 172.21.15.86:0/3733681887 learned_addr learned my addr 172.21.15.86:0/3733681887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.016+0000 7f2450d6c640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 0x7f244c101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 0x7f244c101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 -- 172.21.15.86:0/3733681887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 msgr2=0x7f244c101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 0x7f244c101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 -- 172.21.15.86:0/3733681887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 msgr2=0x7f244c101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 0x7f244c101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 -- 172.21.15.86:0/3733681887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1a6900 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 0x7f244c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f245156d640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f2448006010 tx=0x7f2448004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f24427fc640 1 -- 172.21.15.86:0/3733681887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244803f030 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f24427fc640 1 -- 172.21.15.86:0/3733681887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24480035f0 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c1a6b90 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f24427fc640 1 -- 172.21.15.86:0/3733681887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448035070 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.017+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1a7070 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f24427fc640 1 -- 172.21.15.86:0/3733681887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2448003b10 con 0x7f244c100100 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 Processor -- start 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 -- start start 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6feb5b40 0x561e6feb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fec1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e6fe1a2f0 con 0x561e6feb45d0 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561e6fdc1cf0 con 0x561e6feb5b40 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.020+0000 7f2455baa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561e6fe162b0 con 0x561e6febf370 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44004/0 (socket says 172.21.15.86:44004) 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 -- 172.21.15.86:0/1990827212 learned_addr learned my addr 172.21.15.86:0/1990827212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f2450d6c640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fec1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f2451d6e640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6feb5b40 0x561e6feb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f24427fc640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 0x7f24380ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 -- 172.21.15.86:0/1990827212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 msgr2=0x561e6fec1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fec1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 0x7f24380ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 -- 172.21.15.86:0/1990827212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6feb5b40 msgr2=0x561e6feb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6feb5b40 0x561e6feb4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 -- 172.21.15.86:0/1990827212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6fda63a0 con 0x561e6feb45d0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f2450d6c640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fec1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f24427fc640 1 -- 172.21.15.86:0/3733681887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f24480f03e0 con 0x7f244c100100 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f245156d640 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f24480b8920 tx=0x7f24480b8890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cb64f8f777e824 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 0x7f24380ad9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f244c1077c0 tx=0x7f2444004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f24377fe640 1 -- 172.21.15.86:0/1990827212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448049070 con 0x561e6feb45d0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f24377fe640 1 -- 172.21.15.86:0/1990827212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24480bcbc0 con 0x561e6feb45d0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.021+0000 7f24377fe640 1 -- 172.21.15.86:0/1990827212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480bc4c0 con 0x561e6feb45d0 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- 172.21.15.86:0/1990827212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 msgr2=0x561e6feb4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f24480b8920 tx=0x7f24480b8890 comp rx=0 tx=0).stop 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- 172.21.15.86:0/1990827212 shutdown_connections 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fec1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb45d0 0x561e6feb4a10 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 --2- 172.21.15.86:0/1990827212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6feb5b40 0x561e6feb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- 172.21.15.86:0/1990827212 >> 172.21.15.86:0/1990827212 conn(0x561e6fe7d310 msgr2=0x561e6feb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- 172.21.15.86:0/1990827212 shutdown_connections 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- 172.21.15.86:0/1990827212 wait complete. 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.022+0000 7f2455baa740 1 -- start start 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 Processor -- start 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6febc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e6fe1a2f0 con 0x561e6febf370 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561e6fdd5060 con 0x561e6febcbd0 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561e6fec1e30 con 0x561e6feb5b40 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2455baa740 10 client.0 _ll_register_callbacks cb 0x561e6fb288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44008/0 (socket says 172.21.15.86:44008) 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 -- 172.21.15.86:0/3535276010 learned_addr learned my addr 172.21.15.86:0/3535276010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 -- 172.21.15.86:0/3535276010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 msgr2=0x561e6febc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6febc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 -- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 msgr2=0x561e6fecb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2450d6c640 1 -- 172.21.15.86:0/3535276010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6fb8b3c0 con 0x561e6febf370 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.023+0000 7f2451d6e640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2450d6c640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f243c010a50 tx=0x7f243c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb9be2177feec09 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c00fbd0 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f243c00fd70 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24480bcbc0 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f243c005020 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 10 client.19464 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e6fecdcf0 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24480bcbc0 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f243c019410 con 0x561e6febf370 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 10 client.19464 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 10 client.19464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 20 client.19464 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6fed16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 4 client.19464 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:50.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.024+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6fe27860 con 0x561e6fecf270 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.025+0000 7f245156d640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6fed16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.025+0000 7f245156d640 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6fed16e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bc3f3380c1bbd62 server_cookie=b062ddb5140a7770 in_seq=0 out_seq=0 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.025+0000 7f2434ff9640 10 client.19464 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2434ff9640 1 -- 172.21.15.86:0/3535276010 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e6fe27860 con 0x561e6fecf270 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2434ff9640 10 client.19464 handle_command_reply: tid=0 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 client.19464 shutdown 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 msgr2=0x561e6febc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f243c010a50 tx=0x7f243c010f10 comp rx=0 tx=0).stop 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 shutdown_connections 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6fed16e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2434ff9640 0 client.19464 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febf370 0x561e6febc690 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 --2- 172.21.15.86:0/3535276010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6febc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 >> 172.21.15.86:0/3535276010 conn(0x561e6fe7d310 msgr2=0x561e6feb6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 shutdown_connections 2024-09-05T06:58:50.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 1 -- 172.21.15.86:0/3535276010 wait complete. 2024-09-05T06:58:50.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.026+0000 7f2455baa740 20 client.19464 trim_cache size 0 max 16384 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 Processor -- start 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 -- start start 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fecb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1acf20 con 0x561e6feb5b40 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1aeac0 con 0x561e6febcbd0 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.040+0000 7f2455baa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ad5f0 con 0x561e6febf370 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44016/0 (socket says 172.21.15.86:44016) 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 -- 172.21.15.86:0/2256553433 learned_addr learned my addr 172.21.15.86:0/2256553433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f2451d6e640 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fecb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f2450d6c640 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 -- 172.21.15.86:0/2256553433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 msgr2=0x561e6fecb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fecb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 -- 172.21.15.86:0/2256553433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 msgr2=0x561e6fecaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 -- 172.21.15.86:0/2256553433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6feb79d0 con 0x561e6feb5b40 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f245156d640 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f24480b87d0 tx=0x7f24480f3f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e52002ff9cd6119 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f2436ffd640 1 -- 172.21.15.86:0/2256553433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448049040 con 0x561e6feb5b40 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f2436ffd640 1 -- 172.21.15.86:0/2256553433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24480f3d00 con 0x561e6feb5b40 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.041+0000 7f2436ffd640 1 -- 172.21.15.86:0/2256553433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480bc4c0 con 0x561e6feb5b40 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 -- 172.21.15.86:0/2256553433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 msgr2=0x561e6febb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f24480b87d0 tx=0x7f24480f3f30 comp rx=0 tx=0).stop 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 -- 172.21.15.86:0/2256553433 shutdown_connections 2024-09-05T06:58:50.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6febf370 0x561e6fecb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febcbd0 0x561e6fecaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 --2- 172.21.15.86:0/2256553433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6feb5b40 0x561e6febb130 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 -- 172.21.15.86:0/2256553433 >> 172.21.15.86:0/2256553433 conn(0x561e6fe7d310 msgr2=0x561e6febaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 -- 172.21.15.86:0/2256553433 shutdown_connections 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.042+0000 7f2455baa740 1 -- 172.21.15.86:0/2256553433 wait complete. 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 -- start start 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 Processor -- start 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6fec7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 0x561e6feb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1ae3f0 con 0x561e6febcbd0 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1add20 con 0x561e6febf370 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1aaf20 con 0x561e6feb5b40 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2455baa740 10 client.0 _ll_register_callbacks cb 0x7f243c00a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2450d6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2450d6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44030/0 (socket says 172.21.15.86:44030) 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f2450d6c640 1 -- 172.21.15.86:0/603577518 learned_addr learned my addr 172.21.15.86:0/603577518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.043+0000 7f245156d640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6fec7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2451d6e640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 0x561e6feb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 -- 172.21.15.86:0/603577518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 msgr2=0x561e6fec7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6fec7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 -- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 msgr2=0x561e6feb92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 0x561e6feb92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 -- 172.21.15.86:0/603577518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6feb99d0 con 0x561e6febcbd0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2451d6e640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 0x561e6feb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2450d6c640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f243c0198b0 tx=0x7f243c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6f1d8a7e3d1b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c00fe80 con 0x561e6febcbd0 2024-09-05T06:58:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24480f3d00 con 0x561e6febcbd0 2024-09-05T06:58:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f243c019cb0 con 0x561e6febcbd0 2024-09-05T06:58:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f243c03a050 con 0x561e6febcbd0 2024-09-05T06:58:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2455baa740 10 client.19470 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.044+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e6ff39ea0 con 0x561e6febcbd0 2024-09-05T06:58:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24480f3d00 con 0x561e6febcbd0 2024-09-05T06:58:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f243c004c40 con 0x561e6febcbd0 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 10 client.19470 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 10 client.19470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 20 client.19470 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6ff3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 4 client.19470 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e6fe21d70 con 0x561e6fecf270 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.045+0000 7f245156d640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6ff3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2434ff9640 10 client.19470 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f245156d640 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6ff3c6c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef0526a8ba492d84 server_cookie=d54c7f9ecd6eac3e in_seq=0 out_seq=0 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2434ff9640 1 -- 172.21.15.86:0/603577518 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e6fe21d70 con 0x561e6fecf270 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2434ff9640 10 client.19470 handle_command_reply: tid=0 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 client.19470 shutdown 2024-09-05T06:58:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 msgr2=0x561e6feb8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f243c0198b0 tx=0x7f243c00fbd0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 shutdown_connections 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x561e6fecf270 0x561e6ff3c6c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e6febf370 0x561e6feb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e6febcbd0 0x561e6feb8da0 secure :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f243c0198b0 tx=0x7f243c00fbd0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2434ff9640 0 client.19470 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 --2- 172.21.15.86:0/603577518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561e6feb5b40 0x561e6fec7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.046+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 >> 172.21.15.86:0/603577518 conn(0x561e6fe7d310 msgr2=0x561e6febb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 shutdown_connections 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f2455baa740 1 -- 172.21.15.86:0/603577518 wait complete. 2024-09-05T06:58:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f2455baa740 20 client.19470 trim_cache size 0 max 16384 2024-09-05T06:58:50.053 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 msgr2=0x7f24380ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 0x7f24380ad9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f244c1077c0 tx=0x7f2444004a40 comp rx=0 tx=0).stop 2024-09-05T06:58:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 msgr2=0x7f244c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f2448006010 tx=0x7f2448004270 comp rx=0 tx=0).stop 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 shutdown_connections 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24380ab500 0x7f24380ad9c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f244c106b80 0x7f244c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f244c100aa0 0x7f244c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 --2- 172.21.15.86:0/3733681887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f244c100100 0x7f244c1012e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.047+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 >> 172.21.15.86:0/3733681887 conn(0x7f244c077230 msgr2=0x7f244c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.048+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 shutdown_connections 2024-09-05T06:58:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:50.048+0000 7f24537d1640 1 -- 172.21.15.86:0/3733681887 wait complete. 2024-09-05T06:58:50.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27522 5.16 MB/sec execute 128 sec latency 29903.925 ms 2024-09-05T06:58:50.884 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:58:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 -- 172.21.15.86:0/3467659721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc102570 msgr2=0x7f26bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 --2- 172.21.15.86:0/3467659721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc102570 0x7f26bc1029b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b002a480 comp rx=0 tx=0).stop 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 -- 172.21.15.86:0/3467659721 shutdown_connections 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 --2- 172.21.15.86:0/3467659721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 --2- 172.21.15.86:0/3467659721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc102570 0x7f26bc1029b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 --2- 172.21.15.86:0/3467659721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc100e20 0x7f26bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 -- 172.21.15.86:0/3467659721 >> 172.21.15.86:0/3467659721 conn(0x7f26bc0fc7d0 msgr2=0x7f26bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.494+0000 7f26c37f3640 1 -- 172.21.15.86:0/3467659721 shutdown_connections 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.495+0000 7f26c37f3640 1 -- 172.21.15.86:0/3467659721 wait complete. 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.495+0000 7f26c37f3640 1 Processor -- start 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.495+0000 7f26c37f3640 1 -- start start 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 0x7f26bc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc110e30 con 0x7f26bc100e20 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc110cb0 con 0x7f26bc102ef0 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc110fb0 con 0x7f26bc102570 2024-09-05T06:58:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44054/0 (socket says 172.21.15.86:44054) 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 -- 172.21.15.86:0/987212778 learned_addr learned my addr 172.21.15.86:0/987212778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 0x7f26bc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c1d90640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 -- 172.21.15.86:0/987212778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 msgr2=0x7f26bc197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 0x7f26bc197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 -- 172.21.15.86:0/987212778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 msgr2=0x7f26bc19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 -- 172.21.15.86:0/987212778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc19c280 con 0x7f26bc100e20 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 0x7f26bc197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c1d90640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c158f640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00ea80 tx=0x7f26ac00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26ba7fc640 1 -- 172.21.15.86:0/987212778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00ce40 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26ba7fc640 1 -- 172.21.15.86:0/987212778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26ac005020 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc19c570 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26ba7fc640 1 -- 172.21.15.86:0/987212778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0116d0 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.496+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a3e50 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26ba7fc640 1 -- 172.21.15.86:0/987212778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26ac0051d0 con 0x7f26bc100e20 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 Processor -- start 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 -- start start 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f60f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ab5eb9620 con 0x561ab5f61450 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ab5e60e50 con 0x561ab5f53ee0 2024-09-05T06:58:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561ab5eb55e0 con 0x561ab5f54800 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44056/0 (socket says 172.21.15.86:44056) 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1943312052 learned_addr learned my addr 172.21.15.86:0/1943312052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26c1d90640 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.499+0000 7f26ba7fc640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 0x7f26a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c158f640 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f60f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c1d90640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 0x7f26a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1943312052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 msgr2=0x561ab5f60f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f60f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1943312052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 msgr2=0x561ab5f542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1943312052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab5e51d40 con 0x561ab5f61450 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26ba7fc640 1 -- 172.21.15.86:0/987212778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f26ac0cfb90 con 0x7f26bc100e20 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b0032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872d8313dd3d3456 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26c1d90640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 0x7f26a40a5230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005fd0 tx=0x7f26b4004730 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26a37fe640 1 -- 172.21.15.86:0/1943312052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b003f070 con 0x561ab5f61450 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26a37fe640 1 -- 172.21.15.86:0/1943312052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26b0032e50 con 0x561ab5f61450 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.500+0000 7f26a37fe640 1 -- 172.21.15.86:0/1943312052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0035070 con 0x561ab5f61450 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1943312052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 msgr2=0x561ab5f63840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b0032c60 comp rx=0 tx=0).stop 2024-09-05T06:58:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1943312052 shutdown_connections 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f61450 0x561ab5f63840 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f60f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1943312052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1943312052 >> 172.21.15.86:0/1943312052 conn(0x561ab5f505f0 msgr2=0x561ab5f509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1943312052 shutdown_connections 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1943312052 wait complete. 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 -- start start 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.501+0000 7f26c5bcc740 1 Processor -- start 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f53ee0 0x561ab5f5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ab5f64020 con 0x561ab5f54800 2024-09-05T06:58:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ab5e41f10 con 0x561ab5f61450 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561ab5f641a0 con 0x561ab5f53ee0 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 10 client.0 _ll_register_callbacks cb 0x561ab5e65db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44058/0 (socket says 172.21.15.86:44058) 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1906351118 learned_addr learned my addr 172.21.15.86:0/1906351118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c1d90640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c158f640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f53ee0 0x561ab5f5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1906351118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f53ee0 msgr2=0x561ab5f5e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f53ee0 0x561ab5f5e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 msgr2=0x561ab5f5edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f5edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 -- 172.21.15.86:0/1906351118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab5bc8580 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15331cd948bf51b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b004b060 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f26b0004440 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.502+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b0032e50 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f26b003f030 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 10 client.19482 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561ab5c92960 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26b0032e50 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f26b0004990 con 0x561ab5f54800 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 10 client.19482 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 10 client.19482 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 20 client.19482 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5f70ea0 0x561ab5f73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 4 client.19482 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab5ec6b90 con 0x561ab5f70ea0 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.503+0000 7f26c158f640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5f70ea0 0x561ab5f73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.504+0000 7f26c158f640 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5f70ea0 0x561ab5f73290 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a559d270c2c1797 server_cookie=16773415fe1ca90e in_seq=0 out_seq=0 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.504+0000 7f26a0ff9640 10 client.19482 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.504+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1906351118 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ab5ec6b90 con 0x561ab5f70ea0 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.504+0000 7f26a0ff9640 10 client.19482 handle_command_reply: tid=0 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.504+0000 7f26c5bcc740 1 client.19482 shutdown 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 msgr2=0x561ab5f5e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b0005570 comp rx=0 tx=0).stop 2024-09-05T06:58:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 shutdown_connections 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5f70ea0 0x561ab5f73290 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f5edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f5e890 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26a0ff9640 0 client.19482 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1906351118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f53ee0 0x561ab5f5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 >> 172.21.15.86:0/1906351118 conn(0x561ab5f505f0 msgr2=0x561ab5f562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 shutdown_connections 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1906351118 wait complete. 2024-09-05T06:58:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.505+0000 7f26c5bcc740 20 client.19482 trim_cache size 0 max 16384 2024-09-05T06:58:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 Processor -- start 2024-09-05T06:58:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 -- start start 2024-09-05T06:58:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f4f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f61450 0x561ab5f78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1acd70 con 0x561ab5f54800 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ae910 con 0x561ab5f53ee0 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.522+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ad440 con 0x561ab5f61450 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44068/0 (socket says 172.21.15.86:44068) 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 -- 172.21.15.86:0/2479865365 learned_addr learned my addr 172.21.15.86:0/2479865365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c158f640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f4f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c1d90640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f61450 0x561ab5f78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 -- 172.21.15.86:0/2479865365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f61450 msgr2=0x561ab5f78d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f61450 0x561ab5f78d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 -- 172.21.15.86:0/2479865365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 msgr2=0x561ab5f4f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f4f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 -- 172.21.15.86:0/2479865365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab5f73600 con 0x561ab5f54800 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c158f640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f4f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f26b002a9a0 tx=0x7f26b00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03096036ad23b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26a2ffd640 1 -- 172.21.15.86:0/2479865365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b004b040 con 0x561ab5f54800 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26a2ffd640 1 -- 172.21.15.86:0/2479865365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26b0032c60 con 0x561ab5f54800 2024-09-05T06:58:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.523+0000 7f26a2ffd640 1 -- 172.21.15.86:0/2479865365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0035040 con 0x561ab5f54800 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 -- 172.21.15.86:0/2479865365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 msgr2=0x561ab5f78830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f26b002a9a0 tx=0x7f26b00056e0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 -- 172.21.15.86:0/2479865365 shutdown_connections 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f61450 0x561ab5f78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f54800 0x561ab5f78830 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/2479865365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f53ee0 0x561ab5f4f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 -- 172.21.15.86:0/2479865365 >> 172.21.15.86:0/2479865365 conn(0x561ab5f505f0 msgr2=0x561ab5f4ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 -- 172.21.15.86:0/2479865365 shutdown_connections 2024-09-05T06:58:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.524+0000 7f26c5bcc740 1 -- 172.21.15.86:0/2479865365 wait complete. 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 -- start start 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 Processor -- start 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ae240 con 0x561ab5f53ee0 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1adb70 con 0x561ab5f61450 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1aad70 con 0x561ab5f54800 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c5bcc740 10 client.0 _ll_register_callbacks cb 0x7f26b0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c158f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c158f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44076/0 (socket says 172.21.15.86:44076) 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.525+0000 7f26c158f640 1 -- 172.21.15.86:0/1006240524 learned_addr learned my addr 172.21.15.86:0/1006240524 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c1d90640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 -- 172.21.15.86:0/1006240524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 msgr2=0x561ab5f69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f69930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 -- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 msgr2=0x561ab5f663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 -- 172.21.15.86:0/1006240524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab5f66900 con 0x561ab5f53ee0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c1d90640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c158f640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0980d0 tx=0x7f26ac0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af98b8fde8d5489 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01f040 con 0x561ab5f53ee0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f26ac0cd120 con 0x561ab5f53ee0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b0032c60 con 0x561ab5f53ee0 2024-09-05T06:58:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f26ac0982e0 con 0x561ab5f53ee0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c5bcc740 10 client.19488 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.526+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561ab5fdbeb0 con 0x561ab5f53ee0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26ac0962a0 con 0x561ab5f53ee0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f26ac00f040 con 0x561ab5f53ee0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 10 client.19488 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 10 client.19488 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 20 client.19488 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5fdc5b0 0x561ab5fde9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 4 client.19488 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "57cec38", "release": true, "await": true}] 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "57cec38", "release": true, "await": true}) -- 0x7f26b0032c60 con 0x561ab5fdc5b0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.527+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5fdc5b0 0x561ab5fde9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.528+0000 7f26c0d8e640 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5fdc5b0 0x561ab5fde9d0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d85de660986c16 server_cookie=9f1968e17e06246c in_seq=0 out_seq=0 2024-09-05T06:58:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.528+0000 7f26a0ff9640 10 client.19488 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:51.545 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26a0ff9640 1 -- 172.21.15.86:0/1006240524 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f26b0032c60 con 0x561ab5fdc5b0 2024-09-05T06:58:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26a0ff9640 10 client.19488 handle_command_reply: tid=0 2024-09-05T06:58:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26c5bcc740 1 client.19488 shutdown 2024-09-05T06:58:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 msgr2=0x561ab5f693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0980d0 tx=0x7f26ac0d1040 comp rx=0 tx=0).stop 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 shutdown_connections 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.557+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561ab5fdc5b0 0x561ab5fde9d0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab5f61450 0x561ab5f663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ab5f54800 0x561ab5f69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 --2- 172.21.15.86:0/1006240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab5f53ee0 0x561ab5f693f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 >> 172.21.15.86:0/1006240524 conn(0x561ab5f505f0 msgr2=0x561ab5f4f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26a0ff9640 0 client.19488 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 shutdown_connections 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 1 -- 172.21.15.86:0/1006240524 wait complete. 2024-09-05T06:58:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.558+0000 7f26c5bcc740 20 client.19488 trim_cache size 0 max 16384 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 43, 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 68, 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "57cec38": { 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 68, 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:51.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:58:51.561 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 msgr2=0x7f26a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 0x7f26a40a5230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005fd0 tx=0x7f26b4004730 comp rx=0 tx=0).stop 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 msgr2=0x7f26bc1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00ea80 tx=0x7f26ac00ef40 comp rx=0 tx=0).stop 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 shutdown_connections 2024-09-05T06:58:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f26a40a2d70 0x7f26a40a5230 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26bc102ef0 0x7f26bc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.559+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26bc102570 0x7f26bc197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.560+0000 7f26c37f3640 1 --2- 172.21.15.86:0/987212778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26bc100e20 0x7f26bc1972d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.560+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 >> 172.21.15.86:0/987212778 conn(0x7f26bc0fc7d0 msgr2=0x7f26bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.560+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 shutdown_connections 2024-09-05T06:58:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:51.560+0000 7f26c37f3640 1 -- 172.21.15.86:0/987212778 wait complete. 2024-09-05T06:58:51.675 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:51 smithi086 ceph-mon[30943]: pgmap v552: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:58:51.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27545 5.13 MB/sec execute 129 sec latency 30685.412 ms 2024-09-05T06:58:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:51 smithi096 ceph-mon[34468]: pgmap v552: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:58:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:51 smithi115 ceph-mon[34791]: pgmap v552: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T06:58:52.328 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '57cec38', total seconds elapsed: 34.7 2024-09-05T06:58:52.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:52 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:58:52.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27702 5.18 MB/sec execute 130 sec latency 109.474 ms 2024-09-05T06:58:52.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:52 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:58:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 -- 172.21.15.86:0/3998007697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102570 msgr2=0x7feae41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 --2- 172.21.15.86:0/3998007697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102570 0x7feae41029b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802a480 comp rx=0 tx=0).stop 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 -- 172.21.15.86:0/3998007697 shutdown_connections 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 --2- 172.21.15.86:0/3998007697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4102ef0 0x7feae410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 --2- 172.21.15.86:0/3998007697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102570 0x7feae41029b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 --2- 172.21.15.86:0/3998007697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4100e20 0x7feae4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 -- 172.21.15.86:0/3998007697 >> 172.21.15.86:0/3998007697 conn(0x7feae40fc7d0 msgr2=0x7feae40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 -- 172.21.15.86:0/3998007697 shutdown_connections 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.826+0000 7feae91eb640 1 -- 172.21.15.86:0/3998007697 wait complete. 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.827+0000 7feae91eb640 1 Processor -- start 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.827+0000 7feae91eb640 1 -- start start 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.827+0000 7feae91eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4100e20 0x7feae4073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.827+0000 7feae91eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 0x7feae406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.827+0000 7feae91eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae91eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae4110e80 con 0x7feae4102ef0 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae91eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feae4110d00 con 0x7feae4102570 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae91eb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feae4111000 con 0x7feae4100e20 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41178/0 (socket says 172.21.15.86:41178) 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 -- 172.21.15.86:0/611983864 learned_addr learned my addr 172.21.15.86:0/611983864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae259c640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 0x7feae406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 -- 172.21.15.86:0/611983864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4100e20 msgr2=0x7feae4073ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae2d9d640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4100e20 0x7feae4073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4100e20 0x7feae4073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 -- 172.21.15.86:0/611983864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 msgr2=0x7feae406db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 0x7feae406db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 -- 172.21.15.86:0/611983864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae406e750 con 0x7feae4102ef0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae259c640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 0x7feae406db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feae359e640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7feadc00eb10 tx=0x7feadc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.828+0000 7feacbfff640 1 -- 172.21.15.86:0/611983864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004b20 con 0x7feae4102ef0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.829+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41ad350 con 0x7feae4102ef0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.829+0000 7feacbfff640 1 -- 172.21.15.86:0/611983864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feadc004cc0 con 0x7feae4102ef0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.829+0000 7feacbfff640 1 -- 172.21.15.86:0/611983864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc010690 con 0x7feae4102ef0 2024-09-05T06:58:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.829+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41ad750 con 0x7feae4102ef0 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.832+0000 7feaeb5c4740 1 Processor -- start 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.832+0000 7feacbfff640 1 -- 172.21.15.86:0/611983864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feadc005020 con 0x7feae4102ef0 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.832+0000 7feaeb5c4740 1 -- start start 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.832+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fdd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.832+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdd4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fdd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d95fd3a2f0 con 0x55d95fdd4120 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d95fce1cf0 con 0x55d95fdd4b80 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d95fd362b0 con 0x55d95fdd5840 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fdd5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36002/0 (socket says 172.21.15.86:36002) 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae259c640 1 -- 172.21.15.86:0/3666406989 learned_addr learned my addr 172.21.15.86:0/3666406989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdd4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 -- 172.21.15.86:0/3666406989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 msgr2=0x55d95fdd5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fdd5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 -- 172.21.15.86:0/3666406989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 msgr2=0x55d95fde9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 -- 172.21.15.86:0/3666406989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95fcc63a0 con 0x55d95fdd4120 2024-09-05T06:58:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feacbfff640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 0x7feacc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdd4560 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fead4009b90 tx=0x7fead4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743f657d8b444e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feae259c640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 0x7feacc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.833+0000 7feac97fa640 1 -- 172.21.15.86:0/3666406989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4004020 con 0x55d95fdd4120 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feac97fa640 1 -- 172.21.15.86:0/3666406989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead40041c0 con 0x55d95fdd4120 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feac97fa640 1 -- 172.21.15.86:0/3666406989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40126b0 con 0x55d95fdd4120 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feacbfff640 1 -- 172.21.15.86:0/611983864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7feadc0cff50 con 0x7feae4102ef0 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3666406989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 msgr2=0x55d95fdd4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feae259c640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 0x7feacc0a51b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fead8009230 tx=0x7fead8009620 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.834+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdd4560 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fead4009b90 tx=0x7fead4010740 comp rx=0 tx=0).stop 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3666406989 shutdown_connections 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdd4560 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3666406989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fdd5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3666406989 >> 172.21.15.86:0/3666406989 conn(0x55d95fd9d310 msgr2=0x55d95fdd0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3666406989 shutdown_connections 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3666406989 wait complete. 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 -- start start 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.835+0000 7feaeb5c4740 1 Processor -- start 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41190/0 (socket says 172.21.15.86:41190) 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 -- 172.21.15.86:0/1953798034 learned_addr learned my addr 172.21.15.86:0/1953798034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fde7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d95fdea930 con 0x55d95fdd4120 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d95fcf5060 con 0x55d95fdd4b80 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d95fdeaab0 con 0x55d95fdd5840 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae259c640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae359e640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fde7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feaeb5c4740 10 client.0 _ll_register_callbacks cb 0x55d95fa488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 -- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 msgr2=0x55d95fde7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fde7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 -- 172.21.15.86:0/1953798034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 msgr2=0x55d95fde70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.836+0000 7feae2d9d640 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95faab3c0 con 0x55d95fdd4120 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead40041c0 con 0x55d95fdd4120 2024-09-05T06:58:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feae2d9d640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fead4005940 tx=0x7fead4009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be2cfb03fd7c930 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feac6ff5640 10 client.19500 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feac6ff5640 1 -- 172.21.15.86:0/1953798034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead402a060 con 0x55d95fdd4120 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feac6ff5640 1 -- 172.21.15.86:0/1953798034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7fead40035f0 con 0x55d95fdd4120 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feac6ff5640 1 -- 172.21.15.86:0/1953798034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fead40037f0 con 0x55d95fdd4120 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feaeb5c4740 10 client.19500 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.837+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55d95fdf6260 con 0x55d95fdd4120 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feac6ff5640 1 -- 172.21.15.86:0/1953798034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fead400f040 con 0x55d95fdd4120 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 10 client.19500 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 10 client.19500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 20 client.19500 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fdf75b0 0x55d95fdf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 4 client.19500 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95fd47860 con 0x55d95fdf75b0 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feae259c640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fdf75b0 0x55d95fdf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feac6ff5640 10 client.19500 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.838+0000 7feae259c640 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fdf75b0 0x55d95fdf99a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af327e877b5aaa91 server_cookie=5e87c9bdcf7df27c in_seq=0 out_seq=0 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feac6ff5640 1 -- 172.21.15.86:0/1953798034 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d95fd47860 con 0x55d95fdf75b0 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feac6ff5640 10 client.19500 handle_command_reply: tid=0 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 client.19500 shutdown 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 msgr2=0x55d95fde9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fead4005940 tx=0x7fead4009fb0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 shutdown_connections 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fdf75b0 0x55d95fdf99a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd5840 0x55d95fde7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd4b80 0x55d95fde70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/1953798034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde9e10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 >> 172.21.15.86:0/1953798034 conn(0x55d95fd9d310 msgr2=0x55d95fdd69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.839+0000 7feac6ff5640 0 client.19500 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.840+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 shutdown_connections 2024-09-05T06:58:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.840+0000 7feaeb5c4740 1 -- 172.21.15.86:0/1953798034 wait complete. 2024-09-05T06:58:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.840+0000 7feaeb5c4740 20 client.19500 trim_cache size 0 max 16384 2024-09-05T06:58:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.856+0000 7feaeb5c4740 1 Processor -- start 2024-09-05T06:58:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.856+0000 7feaeb5c4740 1 -- start start 2024-09-05T06:58:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdd6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41206/0 (socket says 172.21.15.86:41206) 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fdd66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b5860 con 0x55d95fdd4120 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b7400 con 0x55d95fdd5840 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 -- 172.21.15.86:0/3544448934 learned_addr learned my addr 172.21.15.86:0/3544448934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b5f30 con 0x55d95fdd4b80 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae259c640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdd6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 -- 172.21.15.86:0/3544448934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 msgr2=0x55d95fdd6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdd6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 -- 172.21.15.86:0/3544448934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 msgr2=0x55d95fdd66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae359e640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fdd66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fdd66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae2d9d640 1 -- 172.21.15.86:0/3544448934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95fdd1730 con 0x55d95fdd4120 2024-09-05T06:58:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.857+0000 7feae359e640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fdd66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.858+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fead4009920 tx=0x7fead4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d0f359d2ee1c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.858+0000 7feac8ff9640 1 -- 172.21.15.86:0/3544448934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4010080 con 0x55d95fdd4120 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.858+0000 7feac8ff9640 1 -- 172.21.15.86:0/3544448934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead40041c0 con 0x55d95fdd4120 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.858+0000 7feac8ff9640 1 -- 172.21.15.86:0/3544448934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40049e0 con 0x55d95fdd4120 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.859+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 msgr2=0x55d95fde6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.859+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fead4009920 tx=0x7fead4021000 comp rx=0 tx=0).stop 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 shutdown_connections 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fdd66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdd6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3544448934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fde6d10 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 >> 172.21.15.86:0/3544448934 conn(0x55d95fd9d310 msgr2=0x55d95fde6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 shutdown_connections 2024-09-05T06:58:52.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3544448934 wait complete. 2024-09-05T06:58:52.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.860+0000 7feaeb5c4740 1 -- start start 2024-09-05T06:58:52.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.861+0000 7feaeb5c4740 1 Processor -- start 2024-09-05T06:58:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.861+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.861+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdf8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feaeb5c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fe5fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feaeb5c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b6d30 con 0x55d95fdd4120 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41218/0 (socket says 172.21.15.86:41218) 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 -- 172.21.15.86:0/3199052290 learned_addr learned my addr 172.21.15.86:0/3199052290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae259c640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdf8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b6660 con 0x55d95fdd5840 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feae41b3860 con 0x55d95fdd4b80 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feaeb5c4740 10 client.0 _ll_register_callbacks cb 0x7feadc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 -- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 msgr2=0x55d95fdf8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdf8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 -- 172.21.15.86:0/3199052290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 msgr2=0x55d95fe5fed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fe5fed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae2d9d640 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95fe60410 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.862+0000 7feae259c640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdf8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feae2d9d640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fead4010da0 tx=0x7fead40126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83fd4472c192d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead402a070 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7fead4012a50 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead40041c0 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.863+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fead4005850 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 10 client.19506 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d95fe61570 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead40041c0 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fead400e420 con 0x55d95fdd4120 2024-09-05T06:58:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 10 client.19506 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 10 client.19506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 20 client.19506 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fe61b30 0x55d95fe63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 4 client.19506 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d95fd41d70 con 0x55d95fe61b30 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.864+0000 7feae259c640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fe61b30 0x55d95fe63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.865+0000 7feac6ff5640 10 client.19506 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:52.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.865+0000 7feae259c640 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fe61b30 0x55d95fe63f20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a14560ee1eaa8169 server_cookie=e2498e288d058b75 in_seq=0 out_seq=0 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feac6ff5640 1 -- 172.21.15.86:0/3199052290 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d95fd41d70 con 0x55d95fe61b30 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feac6ff5640 10 client.19506 handle_command_reply: tid=0 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 client.19506 shutdown 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 msgr2=0x55d95fdf7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fead4010da0 tx=0x7fead40126d0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 shutdown_connections 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55d95fe61b30 0x55d95fe63f20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feac6ff5640 0 client.19506 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d95fdd5840 0x55d95fe5fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d95fdd4b80 0x55d95fdf8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 --2- 172.21.15.86:0/3199052290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d95fdd4120 0x55d95fdf7b50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.866+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 >> 172.21.15.86:0/3199052290 conn(0x55d95fd9d310 msgr2=0x55d95fde6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.867+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 shutdown_connections 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.867+0000 7feaeb5c4740 1 -- 172.21.15.86:0/3199052290 wait complete. 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.867+0000 7feaeb5c4740 20 client.19506 trim_cache size 0 max 16384 2024-09-05T06:58:52.870 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 msgr2=0x7feacc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 0x7feacc0a51b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fead8009230 tx=0x7fead8009620 comp rx=0 tx=0).stop 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 msgr2=0x7feae406e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7feadc00eb10 tx=0x7feadc00c2d0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 shutdown_connections 2024-09-05T06:58:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7feacc0a2cf0 0x7feacc0a51b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae4102ef0 0x7feae406e070 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feae4102570 0x7feae406db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 --2- 172.21.15.86:0/611983864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4100e20 0x7feae4073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 >> 172.21.15.86:0/611983864 conn(0x7feae40fc7d0 msgr2=0x7feae4103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 shutdown_connections 2024-09-05T06:58:52.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:52.868+0000 7feae91eb640 1 -- 172.21.15.86:0/611983864 wait complete. 2024-09-05T06:58:52.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:52 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:58:53.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:53 smithi086 ceph-mon[30943]: pgmap v553: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-05T06:58:53.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:58:53.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:58:53.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:58:53.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27876 5.22 MB/sec execute 131 sec latency 72.596 ms 2024-09-05T06:58:53.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:58:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:53 smithi096 ceph-mon[34468]: pgmap v553: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-05T06:58:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:58:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:58:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:58:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:53 smithi115 ceph-mon[34791]: pgmap v553: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-05T06:58:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:58:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:58:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:58:54.130 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:54.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 -- 172.21.15.86:0/418547356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4106a80 msgr2=0x7f08a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 --2- 172.21.15.86:0/418547356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4106a80 0x7f08a4106e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0898009930 tx=0x7f089802a420 comp rx=0 tx=0).stop 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 -- 172.21.15.86:0/418547356 shutdown_connections 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 --2- 172.21.15.86:0/418547356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 0x7f08a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 --2- 172.21.15.86:0/418547356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a4100a80 0x7f08a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 --2- 172.21.15.86:0/418547356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4106a80 0x7f08a4106e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.697+0000 7f08ab14d640 1 -- 172.21.15.86:0/418547356 >> 172.21.15.86:0/418547356 conn(0x7f08a40fc7a0 msgr2=0x7f08a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.698+0000 7f08ab14d640 1 -- 172.21.15.86:0/418547356 shutdown_connections 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.698+0000 7f08ab14d640 1 -- 172.21.15.86:0/418547356 wait complete. 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.698+0000 7f08ab14d640 1 Processor -- start 2024-09-05T06:58:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 -- start start 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 0x7f08a419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a4106a80 0x7f08a419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08a4110d30 con 0x7f08a4101420 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a4110bb0 con 0x7f08a4106a80 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08ab14d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08a4110eb0 con 0x7f08a4100a80 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 0x7f08a419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36048/0 (socket says 172.21.15.86:36048) 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.699+0000 7f08a8ee9640 1 -- 172.21.15.86:0/3556906428 learned_addr learned my addr 172.21.15.86:0/3556906428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 -- 172.21.15.86:0/3556906428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a4106a80 msgr2=0x7f08a419ee60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a4106a80 0x7f08a419ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 -- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 msgr2=0x7f08a419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 0x7f08a419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 -- 172.21.15.86:0/3556906428 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a419f540 con 0x7f08a4100a80 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0898005940 tx=0x7f08980314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a1ffb640 1 -- 172.21.15.86:0/3556906428 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089804c070 con 0x7f08a4100a80 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a419f7d0 con 0x7f08a4100a80 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a1ffb640 1 -- 172.21.15.86:0/3556906428 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08980318a0 con 0x7f08a4100a80 2024-09-05T06:58:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08a1ffb640 1 -- 172.21.15.86:0/3556906428 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898035050 con 0x7f08a4100a80 2024-09-05T06:58:54.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.700+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a41078e0 con 0x7f08a4100a80 2024-09-05T06:58:54.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.704+0000 7f08a1ffb640 1 -- 172.21.15.86:0/3556906428 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f089804f070 con 0x7f08a4100a80 2024-09-05T06:58:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.705+0000 7f08a1ffb640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 0x7f08940a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.705+0000 7f08a1ffb640 1 -- 172.21.15.86:0/3556906428 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f08980f8380 con 0x7f08a4100a80 2024-09-05T06:58:54.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.705+0000 7f08a3fff640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 0x7f08940a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.705+0000 7f08ad526740 1 Processor -- start 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.705+0000 7f08ad526740 1 -- start start 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d7354ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7356a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d734b22f0 con 0x557d7354db40 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d73459cf0 con 0x557d73568380 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08ad526740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d734ae2b0 con 0x557d7354c5d0 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41250/0 (socket says 172.21.15.86:41250) 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08a8ee9640 1 -- 172.21.15.86:0/408441166 learned_addr learned my addr 172.21.15.86:0/408441166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.706+0000 7f08a3fff640 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d7354ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a3fff640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 0x7f08940a52b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f08a40febe0 tx=0x7f0890005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 -- 172.21.15.86:0/408441166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 msgr2=0x557d7354ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d7354ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 -- 172.21.15.86:0/408441166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 msgr2=0x557d7356a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7356a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 -- 172.21.15.86:0/408441166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7343e3a0 con 0x557d7354db40 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a3fff640 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d7354ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f08980ccb40 tx=0x7f08980c0050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cf854ddd36e16e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f088effd640 1 -- 172.21.15.86:0/408441166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980c4400 con 0x557d7354db40 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f088effd640 1 -- 172.21.15.86:0/408441166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08980c87b0 con 0x557d7354db40 2024-09-05T06:58:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.707+0000 7f088effd640 1 -- 172.21.15.86:0/408441166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980cce40 con 0x557d7354db40 2024-09-05T06:58:54.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 -- 172.21.15.86:0/408441166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 msgr2=0x557d7354c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f08980ccb40 tx=0x7f08980c0050 comp rx=0 tx=0).stop 2024-09-05T06:58:54.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 -- 172.21.15.86:0/408441166 shutdown_connections 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7356a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d7354ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 --2- 172.21.15.86:0/408441166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354db40 0x557d7354c090 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 -- 172.21.15.86:0/408441166 >> 172.21.15.86:0/408441166 conn(0x557d73515310 msgr2=0x557d73548c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 -- 172.21.15.86:0/408441166 shutdown_connections 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.708+0000 7f08ad526740 1 -- 172.21.15.86:0/408441166 wait complete. 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.709+0000 7f08ad526740 1 -- start start 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.709+0000 7f08ad526740 1 Processor -- start 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.709+0000 7f08ad526740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.709+0000 7f08ad526740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 0x557d73565850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.709+0000 7f08ad526740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d735741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08ad526740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d734b22f0 con 0x557d73568380 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08ad526740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d7346d060 con 0x557d73565d90 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08ad526740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d7356ae40 con 0x557d7354c5d0 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36056/0 (socket says 172.21.15.86:36056) 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 -- 172.21.15.86:0/166515314 learned_addr learned my addr 172.21.15.86:0/166515314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a3fff640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 0x557d73565850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 -- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 msgr2=0x557d735741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d735741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 -- 172.21.15.86:0/166515314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 msgr2=0x557d73565850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 0x557d73565850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 -- 172.21.15.86:0/166515314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d732233c0 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a3fff640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 0x557d73565850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08ad526740 10 client.0 _ll_register_callbacks cb 0x557d731c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08980c87b0 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.710+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08980c4390 tx=0x7f089802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a204951b2da29e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f0883fff640 10 client.28357 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f0883fff640 1 -- 172.21.15.86:0/166515314 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980dc070 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f0883fff640 1 -- 172.21.15.86:0/166515314 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f08980c3a60 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f0883fff640 1 -- 172.21.15.86:0/166515314 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08980cd070 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 10 client.28357 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=43}) -- 0x557d73576950 con 0x557d7354c5d0 2024-09-05T06:58:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f0883fff640 1 -- 172.21.15.86:0/166515314 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f08980dd260 con 0x557d7354c5d0 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 10 client.28357 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 10 client.28357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 20 client.28357 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d73577e30 0x557d7357a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 4 client.28357 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.711+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d734bf860 con 0x557d73577e30 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.712+0000 7f08a3fff640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d73577e30 0x557d7357a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.712+0000 7f0883fff640 10 client.28357 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.712+0000 7f08a3fff640 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d73577e30 0x557d7357a2f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec647f19421a4cd7 server_cookie=82ee21f134367482 in_seq=0 out_seq=0 2024-09-05T06:58:54.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.713+0000 7f0883fff640 1 -- 172.21.15.86:0/166515314 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d734bf860 con 0x557d73577e30 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.713+0000 7f0883fff640 10 client.28357 handle_command_reply: tid=0 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.713+0000 7f08ad526740 1 client.28357 shutdown 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.713+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 msgr2=0x557d73565310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.713+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08980c4390 tx=0x7f089802a420 comp rx=0 tx=0).stop 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 shutdown_connections 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d73577e30 0x557d7357a2f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d735741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d73568380 0x557d73565850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 --2- 172.21.15.86:0/166515314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d7354c5d0 0x557d73565310 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 >> 172.21.15.86:0/166515314 conn(0x557d73515310 msgr2=0x557d7354e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f0883fff640 0 client.28357 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 shutdown_connections 2024-09-05T06:58:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 1 -- 172.21.15.86:0/166515314 wait complete. 2024-09-05T06:58:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.714+0000 7f08ad526740 20 client.28357 trim_cache size 0 max 16384 2024-09-05T06:58:54.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.737+0000 7f08ad526740 1 Processor -- start 2024-09-05T06:58:54.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.737+0000 7f08ad526740 1 -- start start 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41280/0 (socket says 172.21.15.86:41280) 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 -- 172.21.15.86:0/2472392119 learned_addr learned my addr 172.21.15.86:0/2472392119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73565d90 0x557d73573fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7357fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41ad050 con 0x557d7354c5d0 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41aebf0 con 0x557d73568380 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41ad720 con 0x557d73565d90 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a96ea640 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7357fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a3fff640 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73565d90 0x557d73573fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 -- 172.21.15.86:0/2472392119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73565d90 msgr2=0x557d73573fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73565d90 0x557d73573fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 -- 172.21.15.86:0/2472392119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 msgr2=0x557d7357fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.738+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7357fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.739+0000 7f08a8ee9640 1 -- 172.21.15.86:0/2472392119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d73549bc0 con 0x557d7354c5d0 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.739+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f08980ccb40 tx=0x7f0898048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68208c52c171686 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.739+0000 7f088e7fc640 1 -- 172.21.15.86:0/2472392119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980dc040 con 0x557d7354c5d0 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.739+0000 7f088e7fc640 1 -- 172.21.15.86:0/2472392119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08980c87b0 con 0x557d7354c5d0 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.739+0000 7f088e7fc640 1 -- 172.21.15.86:0/2472392119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898003810 con 0x557d7354c5d0 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 msgr2=0x557d73560a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f08980ccb40 tx=0x7f0898048000 comp rx=0 tx=0).stop 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 shutdown_connections 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73568380 0x557d7357fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73565d90 0x557d73573fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 --2- 172.21.15.86:0/2472392119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d73560a30 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.740+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 >> 172.21.15.86:0/2472392119 conn(0x557d73515310 msgr2=0x557d73560320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.741+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 shutdown_connections 2024-09-05T06:58:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.741+0000 7f08ad526740 1 -- 172.21.15.86:0/2472392119 wait complete. 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.741+0000 7f08ad526740 1 -- start start 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.741+0000 7f08ad526740 1 Processor -- start 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.741+0000 7f08ad526740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d73568fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 0x557d735694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41288/0 (socket says 172.21.15.86:41288) 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41ae520 con 0x557d7354c5d0 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 -- 172.21.15.86:0/816513396 learned_addr learned my addr 172.21.15.86:0/816513396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41ade50 con 0x557d73565d90 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08a41ab0b0 con 0x557d73568380 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a3fff640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d73568fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08ad526740 10 client.0 _ll_register_callbacks cb 0x7f08980c0790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a96ea640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 0x557d735694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 -- 172.21.15.86:0/816513396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 msgr2=0x557d735694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 0x557d735694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 -- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 msgr2=0x557d73568fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d73568fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a8ee9640 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d73569a30 con 0x557d7354c5d0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a96ea640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 0x557d735694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.742+0000 7f08a3fff640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d73568fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08a8ee9640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0898000c00 tx=0x7f0898003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f42bf43f13e0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980dc030 con 0x557d7354c5d0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f08980c3a60 con 0x557d7354c5d0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08980c87b0 con 0x557d7354c5d0 2024-09-05T06:58:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08980c0360 con 0x557d7354c5d0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 10 client.19524 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557d73569a30 con 0x557d7354c5d0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08980c9940 con 0x557d7354c5d0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f0898031a50 con 0x557d7354c5d0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 10 client.19524 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 10 client.19524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 20 client.19524 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d735e2bd0 0x557d735e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 4 client.19524 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.743+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f08980c87b0 con 0x557d735e2bd0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f08a3fff640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d735e2bd0 0x557d735e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:54.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f0883fff640 10 client.19524 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f08a3fff640 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d735e2bd0 0x557d735e4fc0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ced6345179749 server_cookie=d36108f441a51d7d in_seq=0 out_seq=0 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f0883fff640 1 -- 172.21.15.86:0/816513396 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f08980c87b0 con 0x557d735e2bd0 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f0883fff640 10 client.19524 handle_command_reply: tid=0 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.744+0000 7f08ad526740 1 client.19524 shutdown 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 msgr2=0x557d735705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0898000c00 tx=0x7f0898003940 comp rx=0 tx=0).stop 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 shutdown_connections 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557d735e2bd0 0x557d735e4fc0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d73568380 0x557d735694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d73565d90 0x557d73568fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f0883fff640 0 client.19524 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 --2- 172.21.15.86:0/816513396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d7354c5d0 0x557d735705e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 >> 172.21.15.86:0/816513396 conn(0x557d73515310 msgr2=0x557d735474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 shutdown_connections 2024-09-05T06:58:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 1 -- 172.21.15.86:0/816513396 wait complete. 2024-09-05T06:58:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.745+0000 7f08ad526740 20 client.19524 trim_cache size 0 max 16384 2024-09-05T06:58:54.751 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 msgr2=0x7f08940a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 0x7f08940a52b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f08a40febe0 tx=0x7f0890005f90 comp rx=0 tx=0).stop 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 msgr2=0x7f08a419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0898005940 tx=0x7f08980314a0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 shutdown_connections 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f08940a2df0 0x7f08940a52b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a4106a80 0x7f08a419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a4101420 0x7f08a419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 --2- 172.21.15.86:0/3556906428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4100a80 0x7f08a419a590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 >> 172.21.15.86:0/3556906428 conn(0x7f08a40fc7a0 msgr2=0x7f08a406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 shutdown_connections 2024-09-05T06:58:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:54.746+0000 7f08ab14d640 1 -- 172.21.15.86:0/3556906428 wait complete. 2024-09-05T06:58:54.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28031 5.24 MB/sec execute 132 sec latency 73.768 ms 2024-09-05T06:58:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:55 smithi086 ceph-mon[30943]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:55.761 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28189 5.26 MB/sec execute 133 sec latency 82.471 ms 2024-09-05T06:58:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:55 smithi096 ceph-mon[34468]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:55 smithi115 ceph-mon[34791]: pgmap v554: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:56.486 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:58:56.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:58:56.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28343 5.26 MB/sec execute 134 sec latency 74.894 ms 2024-09-05T06:58:56.846 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.454+0000 7f969e675640 1 -- 172.21.15.86:0/1449820491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698100a80 msgr2=0x7f9698100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.454+0000 7f969e675640 1 --2- 172.21.15.86:0/1449820491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698100a80 0x7f9698100ee0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9688009cd0 tx=0x7f968802a8e0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 -- 172.21.15.86:0/1449820491 shutdown_connections 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 --2- 172.21.15.86:0/1449820491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698101420 0x7f969810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 --2- 172.21.15.86:0/1449820491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698100a80 0x7f9698100ee0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 --2- 172.21.15.86:0/1449820491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698106a80 0x7f9698106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 -- 172.21.15.86:0/1449820491 >> 172.21.15.86:0/1449820491 conn(0x7f96980fc7a0 msgr2=0x7f96980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 -- 172.21.15.86:0/1449820491 shutdown_connections 2024-09-05T06:58:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.455+0000 7f969e675640 1 -- 172.21.15.86:0/1449820491 wait complete. 2024-09-05T06:58:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 Processor -- start 2024-09-05T06:58:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 -- start start 2024-09-05T06:58:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698100a80 0x7f969810f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 0x7f969810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9698114090 con 0x7f9698101420 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9698113f10 con 0x7f9698106a80 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.456+0000 7f969e675640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9698114210 con 0x7f9698100a80 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41306/0 (socket says 172.21.15.86:41306) 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 -- 172.21.15.86:0/3063532265 learned_addr learned my addr 172.21.15.86:0/3063532265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f9697fff640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698100a80 0x7f969810f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f969cc12640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 0x7f969810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 -- 172.21.15.86:0/3063532265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698100a80 msgr2=0x7f969810f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698100a80 0x7f969810f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 -- 172.21.15.86:0/3063532265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 msgr2=0x7f969810fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 0x7f969810fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f96977fe640 1 -- 172.21.15.86:0/3063532265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981aac70 con 0x7f9698101420 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.457+0000 7f969cc12640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 0x7f969810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f96977fe640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9688009df0 tx=0x7f9688031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f96957fa640 1 -- 172.21.15.86:0/3063532265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688004040 con 0x7f9698101420 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f96957fa640 1 -- 172.21.15.86:0/3063532265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9688004690 con 0x7f9698101420 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981aaf00 con 0x7f9698101420 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f96957fa640 1 -- 172.21.15.86:0/3063532265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688030d90 con 0x7f9698101420 2024-09-05T06:58:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.458+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981ab3b0 con 0x7f9698101420 2024-09-05T06:58:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.462+0000 7f96957fa640 1 -- 172.21.15.86:0/3063532265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f96880305a0 con 0x7f9698101420 2024-09-05T06:58:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.464+0000 7f96957fa640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 0x7f96840a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.464+0000 7f9697fff640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 0x7f96840a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.465+0000 7f96957fa640 1 -- 172.21.15.86:0/3063532265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f968802e020 con 0x7f9698101420 2024-09-05T06:58:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.465+0000 7f96a0a4e740 1 Processor -- start 2024-09-05T06:58:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.465+0000 7f96a0a4e740 1 -- start start 2024-09-05T06:58:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.465+0000 7f9697fff640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 0x7f96840a53c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9690009b50 tx=0x7f9690005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d111f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55929d077280 con 0x55929d112480 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55929d01eb10 con 0x55929d111b60 2024-09-05T06:58:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55929d073240 con 0x55929d11cd40 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f969cc12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41308/0 (socket says 172.21.15.86:41308) 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 -- 172.21.15.86:0/2866969672 learned_addr learned my addr 172.21.15.86:0/2866969672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f9697fff640 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 -- 172.21.15.86:0/2866969672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 msgr2=0x55929d11f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 -- 172.21.15.86:0/2866969672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 msgr2=0x55929d111f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.466+0000 7f96977fe640 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d111f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.467+0000 7f96977fe640 1 -- 172.21.15.86:0/2866969672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929d02b1f0 con 0x55929d112480 2024-09-05T06:58:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.467+0000 7f96977fe640 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f9688004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f1c244984dfe25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.467+0000 7f968e7fc640 1 -- 172.21.15.86:0/2866969672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880bcd60 con 0x55929d112480 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.467+0000 7f968e7fc640 1 -- 172.21.15.86:0/2866969672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9688036030 con 0x55929d112480 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.467+0000 7f968e7fc640 1 -- 172.21.15.86:0/2866969672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880bc1b0 con 0x55929d112480 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.468+0000 7f96a0a4e740 1 -- 172.21.15.86:0/2866969672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 msgr2=0x55929d11c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.468+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f9688004840 comp rx=0 tx=0).stop 2024-09-05T06:58:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.469+0000 7f96a0a4e740 1 -- 172.21.15.86:0/2866969672 shutdown_connections 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.469+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.469+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11c800 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.469+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/2866969672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d111f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.469+0000 7f96a0a4e740 1 -- 172.21.15.86:0/2866969672 >> 172.21.15.86:0/2866969672 conn(0x55929d10e0f0 msgr2=0x55929d10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.470+0000 7f96a0a4e740 1 -- 172.21.15.86:0/2866969672 shutdown_connections 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.470+0000 7f96a0a4e740 1 -- 172.21.15.86:0/2866969672 wait complete. 2024-09-05T06:58:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.470+0000 7f96a0a4e740 1 -- start start 2024-09-05T06:58:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 Processor -- start 2024-09-05T06:58:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55929d11f6d0 con 0x55929d112480 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55929d031f10 con 0x55929d111b60 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55929d11f850 con 0x55929d11cd40 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.471+0000 7f96a0a4e740 10 client.0 _ll_register_callbacks cb 0x55929cffaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.472+0000 7f969cc12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.472+0000 7f9697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.472+0000 7f9697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33758/0 (socket says 172.21.15.86:33758) 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.472+0000 7f9697fff640 1 -- 172.21.15.86:0/3649239367 learned_addr learned my addr 172.21.15.86:0/3649239367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.472+0000 7f96977fe640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f9697fff640 1 -- 172.21.15.86:0/3649239367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 msgr2=0x55929d11a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f9697fff640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f9697fff640 1 -- 172.21.15.86:0/3649239367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 msgr2=0x55929d11a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f9697fff640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f9697fff640 1 -- 172.21.15.86:0/3649239367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929cde86a0 con 0x55929d111b60 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.473+0000 7f969cc12640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.474+0000 7f9697fff640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9690003970 tx=0x7f9690004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8104647a8ab20acb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.474+0000 7f96777fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969001eb40 con 0x55929d111b60 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f969001ece0 con 0x55929d111b60 2024-09-05T06:58:57.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9688036030 con 0x55929d111b60 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9690028430 con 0x55929d111b60 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96a0a4e740 10 client.28355 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.475+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55929d12b3d0 con 0x55929d111b60 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9688036030 con 0x55929d111b60 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9690032580 con 0x55929d111b60 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96a0a4e740 10 client.28355 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96a0a4e740 10 client.28355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96a0a4e740 20 client.28355 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.476+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d12ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.477+0000 7f96a0a4e740 4 client.28355 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:58:57.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.477+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55929d0847f0 con 0x55929d12c7e0 2024-09-05T06:58:57.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.478+0000 7f96977fe640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d12ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.478+0000 7f96977fe640 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d12ec00 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30133429b93f051b server_cookie=f95ad65e0ff7e315 in_seq=0 out_seq=0 2024-09-05T06:58:57.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.478+0000 7f96777fe640 10 client.28355 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.480+0000 7f96777fe640 1 -- 172.21.15.86:0/3649239367 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55929d0847f0 con 0x55929d12c7e0 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.480+0000 7f96777fe640 10 client.28355 handle_command_reply: tid=0 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.480+0000 7f96a0a4e740 1 client.28355 shutdown 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 msgr2=0x55929d119d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9690003970 tx=0x7f9690004730 comp rx=0 tx=0).stop 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 shutdown_connections 2024-09-05T06:58:57.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d12ec00 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96777fe640 0 client.28355 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d11a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/3649239367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d119d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 >> 172.21.15.86:0/3649239367 conn(0x55929d10e0f0 msgr2=0x55929d113e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 shutdown_connections 2024-09-05T06:58:57.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 1 -- 172.21.15.86:0/3649239367 wait complete. 2024-09-05T06:58:57.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.481+0000 7f96a0a4e740 20 client.28355 trim_cache size 0 max 16384 2024-09-05T06:58:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.512+0000 7f96a0a4e740 1 Processor -- start 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 -- start start 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d11bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d134520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96981acf70 con 0x55929d112480 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96981aeb10 con 0x55929d111b60 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96981ad640 con 0x55929d11cd40 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f9697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d11bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f9697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d11bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33764/0 (socket says 172.21.15.86:33764) 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f96977fe640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f9697fff640 1 -- 172.21.15.86:0/1243712908 learned_addr learned my addr 172.21.15.86:0/1243712908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.513+0000 7f969cc12640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 -- 172.21.15.86:0/1243712908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 msgr2=0x55929d11bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d11bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 -- 172.21.15.86:0/1243712908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 msgr2=0x55929d11a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 -- 172.21.15.86:0/1243712908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55929d118cf0 con 0x55929d11cd40 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f96977fe640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f969cc12640 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d134520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55929d10f470 tx=0x7f9680009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a0d39714c4c0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f968dffb640 1 -- 172.21.15.86:0/1243712908 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680005020 con 0x55929d11cd40 2024-09-05T06:58:57.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f968dffb640 1 -- 172.21.15.86:0/1243712908 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96800047a0 con 0x55929d11cd40 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.514+0000 7f968dffb640 1 -- 172.21.15.86:0/1243712908 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968001a700 con 0x55929d11cd40 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 -- 172.21.15.86:0/1243712908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 msgr2=0x55929d134520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d134520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55929d10f470 tx=0x7f9680009740 comp rx=0 tx=0).stop 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 -- 172.21.15.86:0/1243712908 shutdown_connections 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d11cd40 0x55929d134520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/1243712908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d111b60 0x55929d11bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 -- 172.21.15.86:0/1243712908 >> 172.21.15.86:0/1243712908 conn(0x55929d10e0f0 msgr2=0x55929d11b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 -- 172.21.15.86:0/1243712908 shutdown_connections 2024-09-05T06:58:57.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.515+0000 7f96a0a4e740 1 -- 172.21.15.86:0/1243712908 wait complete. 2024-09-05T06:58:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.516+0000 7f96a0a4e740 1 -- start start 2024-09-05T06:58:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.516+0000 7f96a0a4e740 1 Processor -- start 2024-09-05T06:58:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.516+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 0x55929d121730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.516+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d121c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.516+0000 7f96a0a4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 0x55929d1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96981ae440 con 0x55929d112480 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96981add70 con 0x55929d11cd40 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96a0a4e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96981a2490 con 0x55929d111b60 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96a0a4e740 10 client.0 _ll_register_callbacks cb 0x7f9690009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f969cc12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 0x55929d1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d121c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41338/0 (socket says 172.21.15.86:41338) 2024-09-05T06:58:57.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96977fe640 1 -- 172.21.15.86:0/24499569 learned_addr learned my addr 172.21.15.86:0/24499569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 0x55929d121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 -- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 msgr2=0x55929d1221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 0x55929d1221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 -- 172.21.15.86:0/24499569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 msgr2=0x55929d121c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d121c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 -- 172.21.15.86:0/24499569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55929d196740 con 0x55929d111b60 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f969cc12640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 0x55929d1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:58:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f9697fff640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 0x55929d121730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96900118d0 tx=0x7f9690004110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2894efd6f1c90613 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96777fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.517+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690038070 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9605+0+0 (secure 0 0 0) 0x7f9690032470 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96800047a0 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9690049050 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 10 client.28381 fetch_fsmap learned FSMap version 43 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55929d197a80 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9690032690 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9690044030 con 0x55929d111b60 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 10 client.28381 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 10 client.28381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 20 client.28381 populate_metadata read hostname 'smithi086' 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:58:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 4 client.28381 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.518+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f96800047a0 con 0x55929d12c7e0 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.519+0000 7f96977fe640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.519+0000 7f96977fe640 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d19a1a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79a31db1a18d789 server_cookie=b39505b5ab024c5c in_seq=0 out_seq=0 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.519+0000 7f96777fe640 10 client.28381 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.520+0000 7f96777fe640 1 -- 172.21.15.86:0/24499569 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f96800047a0 con 0x55929d12c7e0 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.520+0000 7f96777fe640 10 client.28381 handle_command_reply: tid=0 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.520+0000 7f96a0a4e740 1 client.28381 shutdown 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 msgr2=0x55929d121730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 0x55929d121730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96900118d0 tx=0x7f9690004110 comp rx=0 tx=0).stop 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 shutdown_connections 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55929d12c7e0 0x55929d19a1a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55929d11cd40 0x55929d1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55929d112480 0x55929d121c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 --2- 172.21.15.86:0/24499569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55929d111b60 0x55929d121730 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 >> 172.21.15.86:0/24499569 conn(0x55929d10e0f0 msgr2=0x55929d113dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96777fe640 0 client.28381 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 shutdown_connections 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 1 -- 172.21.15.86:0/24499569 wait complete. 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.521+0000 7f96a0a4e740 20 client.28381 trim_cache size 0 max 16384 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "100b8bc7-ee00-4b3d-a49e-6c024f40dadb", 2024-09-05T06:58:57.527 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:58:57.528 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:58:57.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 msgr2=0x7f96840a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 0x7f96840a53c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9690009b50 tx=0x7f9690005f90 comp rx=0 tx=0).stop 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 msgr2=0x7f969810f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f9688009df0 tx=0x7f9688031df0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 shutdown_connections 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96840a2f00 0x7f96840a53c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9698106a80 0x7f969810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9698101420 0x7f969810f640 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 --2- 172.21.15.86:0/3063532265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9698100a80 0x7f969810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:58:57.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.522+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 >> 172.21.15.86:0/3063532265 conn(0x7f96980fc7a0 msgr2=0x7f96980fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:58:57.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.523+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 shutdown_connections 2024-09-05T06:58:57.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:58:57.523+0000 7f969e675640 1 -- 172.21.15.86:0/3063532265 wait complete. 2024-09-05T06:58:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:57 smithi086 ceph-mon[30943]: pgmap v555: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:57.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28484 5.25 MB/sec execute 135 sec latency 94.606 ms 2024-09-05T06:58:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:57 smithi096 ceph-mon[34468]: pgmap v555: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:57.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:57 smithi115 ceph-mon[34791]: pgmap v555: 129 pgs: 129 active+clean; 804 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-05T06:58:58.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28652 5.28 MB/sec execute 136 sec latency 96.390 ms 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:58 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:58 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:58.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:58.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:58 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28811 5.29 MB/sec execute 137 sec latency 143.858 ms 2024-09-05T06:58:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:59 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T06:58:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:59 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:59 smithi096 ceph-mon[34468]: pgmap v556: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-05T06:58:59.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:58:59 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:59 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T06:58:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:59 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:59 smithi115 ceph-mon[34791]: pgmap v556: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-05T06:58:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:58:59 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:59 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T06:58:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:59 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:58:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:59 smithi086 ceph-mon[30943]: pgmap v556: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-05T06:58:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:58:59 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T06:59:00.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28970 5.31 MB/sec execute 138 sec latency 155.577 ms 2024-09-05T06:59:01.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:59:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:01 smithi086 ceph-mon[30943]: pgmap v557: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-05T06:59:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:01 smithi086 ceph-mon[30943]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T06:59:01.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29105 5.31 MB/sec execute 139 sec latency 152.158 ms 2024-09-05T06:59:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:01 smithi096 ceph-mon[34468]: pgmap v557: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-05T06:59:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:01 smithi096 ceph-mon[34468]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T06:59:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:01 smithi115 ceph-mon[34791]: pgmap v557: 129 pgs: 129 active+clean; 820 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-05T06:59:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:01 smithi115 ceph-mon[34791]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 -- 172.21.15.86:0/3704878359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1454103ff0 msgr2=0x7f14541043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 --2- 172.21.15.86:0/3704878359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1454103ff0 0x7f14541043d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1444009900 tx=0x7f144402a410 comp rx=0 tx=0).stop 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 -- 172.21.15.86:0/3704878359 shutdown_connections 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 --2- 172.21.15.86:0/3704878359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454109120 0x7f1454110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 --2- 172.21.15.86:0/3704878359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14541049a0 0x7f14541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 --2- 172.21.15.86:0/3704878359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1454103ff0 0x7f14541043d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.477+0000 7f1458a62640 1 -- 172.21.15.86:0/3704878359 >> 172.21.15.86:0/3704878359 conn(0x7f14540ffa90 msgr2=0x7f1454101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.478+0000 7f1458a62640 1 -- 172.21.15.86:0/3704878359 shutdown_connections 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.478+0000 7f1458a62640 1 -- 172.21.15.86:0/3704878359 wait complete. 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.478+0000 7f1458a62640 1 Processor -- start 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 -- start start 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 0x7f145419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 0x7f145419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1454113ff0 con 0x7f14541049a0 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1454113e70 con 0x7f1454103ff0 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1458a62640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1454114170 con 0x7f1454109120 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f145259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 0x7f145419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 0x7f145419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f145259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 0x7f145419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:33788/0 (socket says 172.21.15.86:33788) 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36140/0 (socket says 172.21.15.86:36140) 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f145259c640 1 -- 172.21.15.86:0/3032507400 learned_addr learned my addr 172.21.15.86:0/3032507400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 -- 172.21.15.86:0/3032507400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 msgr2=0x7f145419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 -- 172.21.15.86:0/3032507400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 msgr2=0x7f145419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 0x7f145419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f144bfff640 1 -- 172.21.15.86:0/3032507400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145419f4e0 con 0x7f14541049a0 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.479+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f144bfff640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 0x7f145419aa70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9f0 tx=0x7f143c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f144b7fe640 1 -- 172.21.15.86:0/3032507400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00ccc0 con 0x7f14541049a0 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f144b7fe640 1 -- 172.21.15.86:0/3032507400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143c005020 con 0x7f14541049a0 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145419f7d0 con 0x7f14541049a0 2024-09-05T06:59:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f144b7fe640 1 -- 172.21.15.86:0/3032507400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c004a00 con 0x7f14541049a0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.480+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1454076630 con 0x7f14541049a0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.483+0000 7f144b7fe640 1 -- 172.21.15.86:0/3032507400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f143c004ba0 con 0x7f14541049a0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.483+0000 7f145ae3b740 1 Processor -- start 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 -- start start 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e9bf1a2e0 con 0x558e9bfb44a0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e9bec1ce0 con 0x558e9bfbf680 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f145ae3b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e9bf162a0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53654/0 (socket says 172.21.15.86:53654) 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f1452d9d640 1 -- 172.21.15.86:0/3767868088 learned_addr learned my addr 172.21.15.86:0/3767868088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.484+0000 7f144b7fe640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 0x7f14400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 -- 172.21.15.86:0/3767868088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 msgr2=0x558e9bfbf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 -- 172.21.15.86:0/3767868088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 msgr2=0x558e9bfc1a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc1a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 -- 172.21.15.86:0/3767868088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9bea6440 con 0x558e9bfb44a0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f144b7fe640 1 -- 172.21.15.86:0/3032507400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9693+0+0 (secure 0 0 0) 0x7f143c0cf7e0 con 0x7f14541049a0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f144c009ef0 tx=0x7f144c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b789581d55cd5a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 0x7f14400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1448ff9640 1 -- 172.21.15.86:0/3767868088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c024030 con 0x558e9bfb44a0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1448ff9640 1 -- 172.21.15.86:0/3767868088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f144c003e20 con 0x558e9bfb44a0 2024-09-05T06:59:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.485+0000 7f1448ff9640 1 -- 172.21.15.86:0/3767868088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0047a0 con 0x558e9bfb44a0 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.486+0000 7f145ae3b740 1 -- 172.21.15.86:0/3767868088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 msgr2=0x558e9bfb4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.486+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f144c009ef0 tx=0x7f144c00b690 comp rx=0 tx=0).stop 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 0x7f14400a5380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f144c024000 tx=0x7f144c020610 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 -- 172.21.15.86:0/3767868088 shutdown_connections 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3767868088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfb4880 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 -- 172.21.15.86:0/3767868088 >> 172.21.15.86:0/3767868088 conn(0x558e9bfb0a30 msgr2=0x558e9bfb0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 -- 172.21.15.86:0/3767868088 shutdown_connections 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 -- 172.21.15.86:0/3767868088 wait complete. 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.487+0000 7f145ae3b740 1 -- start start 2024-09-05T06:59:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145ae3b740 1 Processor -- start 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46802/0 (socket says 172.21.15.86:46802) 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.488+0000 7f145259c640 1 -- 172.21.15.86:0/2764979636 learned_addr learned my addr 172.21.15.86:0/2764979636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 0x558e9bfbc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e9bfc2250 con 0x558e9bfbf680 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f144bfff640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e9bed5050 con 0x558e9bfb44a0 2024-09-05T06:59:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f1452d9d640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 0x558e9bfbc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e9bfc23d0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145ae3b740 10 client.0 _ll_register_callbacks cb 0x558e9bf15160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145259c640 1 -- 172.21.15.86:0/2764979636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 msgr2=0x558e9bfbc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145259c640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145259c640 1 -- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 msgr2=0x558e9bfbc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145259c640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 0x558e9bfbc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f145259c640 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9bc8b3c0 con 0x558e9bfb44a0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.489+0000 7f1452d9d640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 0x558e9bfbc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f145259c640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1444000c00 tx=0x7f1444031d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4895535464cfa4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f1439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444040070 con 0x558e9bfb44a0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9693+0+0 (secure 0 0 0) 0x7f1444004040 con 0x558e9bfb44a0 2024-09-05T06:59:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144c003e20 con 0x558e9bfb44a0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1444035040 con 0x558e9bfb44a0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f145ae3b740 10 client.28370 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558e9bfcdc00 con 0x558e9bfb44a0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.490+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f144c003e20 con 0x558e9bfb44a0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f14440047a0 con 0x558e9bfb44a0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 10 client.28370 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 10 client.28370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 20 client.28370 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9bfcf0c0 0x558e9bfd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 4 client.28370 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9bf27850 con 0x558e9bfcf0c0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f144bfff640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9bfcf0c0 0x558e9bfd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f144bfff640 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9bfcf0c0 0x558e9bfd14b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3cbec0332bb7daa server_cookie=87672180e9f8d45 in_seq=0 out_seq=0 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.491+0000 7f1439ffb640 10 client.28370 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f1439ffb640 1 -- 172.21.15.86:0/2764979636 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e9bf27850 con 0x558e9bfcf0c0 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f1439ffb640 10 client.28370 handle_command_reply: tid=0 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 client.28370 shutdown 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 msgr2=0x558e9bfbefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1444000c00 tx=0x7f1444031d20 comp rx=0 tx=0).stop 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 shutdown_connections 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9bfcf0c0 0x558e9bfd14b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfbf680 0x558e9bfbc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f1439ffb640 0 client.28370 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfbc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 --2- 172.21.15.86:0/2764979636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfb44a0 0x558e9bfbefd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.492+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 >> 172.21.15.86:0/2764979636 conn(0x558e9bfb0a30 msgr2=0x558e9bfb6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.493+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 shutdown_connections 2024-09-05T06:59:02.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.493+0000 7f145ae3b740 1 -- 172.21.15.86:0/2764979636 wait complete. 2024-09-05T06:59:02.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.493+0000 7f145ae3b740 20 client.28370 trim_cache size 0 max 16384 2024-09-05T06:59:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 Processor -- start 2024-09-05T06:59:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 -- start start 2024-09-05T06:59:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfcb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfd6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfd72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14541ad000 con 0x558e9bfb44a0 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14541aeba0 con 0x558e9bfbf680 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145ae3b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14541ad6d0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfcb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfcb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53678/0 (socket says 172.21.15.86:53678) 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f145259c640 1 -- 172.21.15.86:0/3126026945 learned_addr learned my addr 172.21.15.86:0/3126026945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfd72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.516+0000 7f144bfff640 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfd6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 -- 172.21.15.86:0/3126026945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 msgr2=0x558e9bfd6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfd6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 -- 172.21.15.86:0/3126026945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 msgr2=0x558e9bfcb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfcb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 -- 172.21.15.86:0/3126026945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9bfbb230 con 0x558e9bfbf680 2024-09-05T06:59:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfd72c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f144c016010 tx=0x7f144c02e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2599b0fc83ff085b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f143bfff640 1 -- 172.21.15.86:0/3126026945 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c02bd60 con 0x558e9bfbf680 2024-09-05T06:59:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f143bfff640 1 -- 172.21.15.86:0/3126026945 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f144c039030 con 0x558e9bfbf680 2024-09-05T06:59:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.517+0000 7f143bfff640 1 -- 172.21.15.86:0/3126026945 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0045a0 con 0x558e9bfbf680 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 -- 172.21.15.86:0/3126026945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 msgr2=0x558e9bfd72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfd72c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f144c016010 tx=0x7f144c02e000 comp rx=0 tx=0).stop 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 -- 172.21.15.86:0/3126026945 shutdown_connections 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfd72c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb4dc0 0x558e9bfd6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3126026945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb44a0 0x558e9bfcb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 -- 172.21.15.86:0/3126026945 >> 172.21.15.86:0/3126026945 conn(0x558e9bfb0a30 msgr2=0x558e9bfb78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 -- 172.21.15.86:0/3126026945 shutdown_connections 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.518+0000 7f145ae3b740 1 -- 172.21.15.86:0/3126026945 wait complete. 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 -- start start 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 Processor -- start 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb44a0 0x558e9bfc7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14541ae4d0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f144bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14541ade00 con 0x558e9bfbf680 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f144bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53684/0 (socket says 172.21.15.86:53684) 2024-09-05T06:59:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14541ab000 con 0x558e9bfb44a0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f144bfff640 1 -- 172.21.15.86:0/3554991404 learned_addr learned my addr 172.21.15.86:0/3554991404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145259c640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb44a0 0x558e9bfc7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f1452d9d640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.519+0000 7f145ae3b740 10 client.0 _ll_register_callbacks cb 0x7f143c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 -- 172.21.15.86:0/3554991404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb44a0 msgr2=0x558e9bfc7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb44a0 0x558e9bfc7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 -- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 msgr2=0x558e9bfc3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 -- 172.21.15.86:0/3554991404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9bfc3d40 con 0x558e9bfb4dc0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f144bfff640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f143c06ba10 tx=0x7f143c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1083a18e21b8afa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f1439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c01c040 con 0x558e9bfb4dc0 2024-09-05T06:59:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9693+0+0 (secure 0 0 0) 0x7f143c0ba120 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144c039030 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f143c09fbc0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f145ae3b740 10 client.19560 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.520+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558e9c039cd0 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f144c039030 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f143c097e80 con 0x558e9bfb4dc0 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 10 client.19560 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 20 client.19560 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9c03a3e0 0x558e9c03c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 4 client.19560 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:59:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e9bf21d60 con 0x558e9c03a3e0 2024-09-05T06:59:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.521+0000 7f145259c640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9c03a3e0 0x558e9c03c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f1439ffb640 10 client.19560 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145259c640 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9c03a3e0 0x558e9c03c800 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cb6b687ac4d2df server_cookie=d4e82a00d89e122b in_seq=0 out_seq=0 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f1439ffb640 1 -- 172.21.15.86:0/3554991404 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e9bf21d60 con 0x558e9c03a3e0 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f1439ffb640 10 client.19560 handle_command_reply: tid=0 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 client.19560 shutdown 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 msgr2=0x558e9bfc3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f143c06ba10 tx=0x7f143c009f60 comp rx=0 tx=0).stop 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 shutdown_connections 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558e9c03a3e0 0x558e9c03c800 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9bfbf680 0x558e9bfc3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9bfb4dc0 0x558e9bfc3000 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 --2- 172.21.15.86:0/3554991404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e9bfb44a0 0x558e9bfc7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 >> 172.21.15.86:0/3554991404 conn(0x558e9bfb0a30 msgr2=0x558e9bfaf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f1439ffb640 0 client.19560 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.522+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 shutdown_connections 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f145ae3b740 1 -- 172.21.15.86:0/3554991404 wait complete. 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "100b8bc7-ee00-4b3d-a49e-6c024f40dadb": { 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "100b8bc7-ee00-4b3d-a49e-6c024f40dadb", 2024-09-05T06:59:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T06:59:02.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:59:02.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T06:59:02.529 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:59:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f145ae3b740 20 client.19560 trim_cache size 0 max 16384 2024-09-05T06:59:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 msgr2=0x7f14400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 0x7f14400a5380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f144c024000 tx=0x7f144c020610 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 msgr2=0x7f145419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.523+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 0x7f145419aa70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9f0 tx=0x7f143c00eeb0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 shutdown_connections 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14400a2ec0 0x7f14400a5380 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1454109120 0x7f145419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14541049a0 0x7f145419aa70 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 --2- 172.21.15.86:0/3032507400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1454103ff0 0x7f145419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 >> 172.21.15.86:0/3032507400 conn(0x7f14540ffa90 msgr2=0x7f1454100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 shutdown_connections 2024-09-05T06:59:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:02.524+0000 7f1458a62640 1 -- 172.21.15.86:0/3032507400 wait complete. 2024-09-05T06:59:02.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29265 5.35 MB/sec execute 140 sec latency 578.674 ms 2024-09-05T06:59:03.371 INFO:tasks.cephfs.filesystem:scrub status for tag:100b8bc7-ee00-4b3d-a49e-6c024f40dadb - {'path': '/', 'tag': '100b8bc7-ee00-4b3d-a49e-6c024f40dadb', 'options': 'recursive,force'} 2024-09-05T06:59:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:03 smithi086 ceph-mon[30943]: pgmap v559: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:03.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29450 5.38 MB/sec execute 141 sec latency 1578.705 ms 2024-09-05T06:59:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:03 smithi096 ceph-mon[34468]: pgmap v559: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:03 smithi115 ceph-mon[34791]: pgmap v559: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:04.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29663 5.42 MB/sec execute 142 sec latency 2578.739 ms 2024-09-05T06:59:05.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29846 5.43 MB/sec execute 143 sec latency 3578.769 ms 2024-09-05T06:59:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:05 smithi096 ceph-mon[34468]: pgmap v560: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:05.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:05 smithi115 ceph-mon[34791]: pgmap v560: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:05 smithi086 ceph-mon[30943]: pgmap v560: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:06.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30029 5.46 MB/sec execute 144 sec latency 4578.795 ms 2024-09-05T06:59:07.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30193 5.49 MB/sec execute 145 sec latency 5578.824 ms 2024-09-05T06:59:07.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:07 smithi086 ceph-mon[30943]: pgmap v561: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:07.788 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:07 smithi086 ceph-mon[30943]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T06:59:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:07 smithi096 ceph-mon[34468]: pgmap v561: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:07 smithi096 ceph-mon[34468]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T06:59:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:07 smithi115 ceph-mon[34791]: pgmap v561: 129 pgs: 129 active+clean; 823 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 294 op/s 2024-09-05T06:59:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:07 smithi115 ceph-mon[34791]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T06:59:08.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30399 5.57 MB/sec execute 146 sec latency 5826.866 ms 2024-09-05T06:59:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:09 smithi086 ceph-mon[30943]: pgmap v563: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 370 op/s 2024-09-05T06:59:09.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30613 5.64 MB/sec execute 147 sec latency 99.853 ms 2024-09-05T06:59:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:09 smithi096 ceph-mon[34468]: pgmap v563: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 370 op/s 2024-09-05T06:59:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:09 smithi115 ceph-mon[34791]: pgmap v563: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 370 op/s 2024-09-05T06:59:10.762 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30771 5.65 MB/sec execute 148 sec latency 58.423 ms 2024-09-05T06:59:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:10 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:59:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:10 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:59:10.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:10 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:59:10.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:10 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:59:10.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:10 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:59:10.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:10 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:59:11.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30903 5.66 MB/sec execute 149 sec latency 114.486 ms 2024-09-05T06:59:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:11 smithi096 ceph-mon[34468]: pgmap v564: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 325 op/s 2024-09-05T06:59:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:59:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:59:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T06:59:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:11 smithi115 ceph-mon[34791]: pgmap v564: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 325 op/s 2024-09-05T06:59:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:59:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:59:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T06:59:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:11 smithi086 ceph-mon[30943]: pgmap v564: 129 pgs: 129 active+clean; 843 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 325 op/s 2024-09-05T06:59:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:59:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:59:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T06:59:12.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31100 5.72 MB/sec execute 150 sec latency 97.028 ms 2024-09-05T06:59:13.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31285 5.74 MB/sec execute 151 sec latency 96.121 ms 2024-09-05T06:59:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:13 smithi096 ceph-mon[34468]: pgmap v565: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:13 smithi115 ceph-mon[34791]: pgmap v565: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:13.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:13 smithi086 ceph-mon[30943]: pgmap v565: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:14.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31445 5.77 MB/sec execute 152 sec latency 145.010 ms 2024-09-05T06:59:15.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31609 5.78 MB/sec execute 153 sec latency 80.227 ms 2024-09-05T06:59:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:15 smithi096 ceph-mon[34468]: pgmap v566: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:15 smithi115 ceph-mon[34791]: pgmap v566: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:15.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:15 smithi086 ceph-mon[30943]: pgmap v566: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:16.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31784 5.77 MB/sec execute 154 sec latency 110.189 ms 2024-09-05T06:59:17.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31953 5.80 MB/sec execute 155 sec latency 63.059 ms 2024-09-05T06:59:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:17 smithi096 ceph-mon[34468]: pgmap v567: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:17 smithi115 ceph-mon[34791]: pgmap v567: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:17.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:17 smithi086 ceph-mon[30943]: pgmap v567: 129 pgs: 129 active+clean; 866 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 283 op/s 2024-09-05T06:59:18.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32115 5.80 MB/sec execute 156 sec latency 109.590 ms 2024-09-05T06:59:19.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32301 5.82 MB/sec execute 157 sec latency 65.555 ms 2024-09-05T06:59:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:19 smithi096 ceph-mon[34468]: pgmap v568: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 275 op/s 2024-09-05T06:59:19.858 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:19 smithi086 ceph-mon[30943]: pgmap v568: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 275 op/s 2024-09-05T06:59:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:19 smithi115 ceph-mon[34791]: pgmap v568: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 275 op/s 2024-09-05T06:59:20.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32501 5.87 MB/sec execute 158 sec latency 63.979 ms 2024-09-05T06:59:21.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32677 5.89 MB/sec execute 159 sec latency 62.418 ms 2024-09-05T06:59:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:21 smithi096 ceph-mon[34468]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 241 op/s 2024-09-05T06:59:21.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:21 smithi115 ceph-mon[34791]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 241 op/s 2024-09-05T06:59:21.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:21 smithi086 ceph-mon[30943]: pgmap v569: 129 pgs: 129 active+clean; 877 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 241 op/s 2024-09-05T06:59:22.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32854 5.89 MB/sec execute 160 sec latency 61.902 ms 2024-09-05T06:59:23.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33028 5.89 MB/sec execute 161 sec latency 60.765 ms 2024-09-05T06:59:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:23 smithi096 ceph-mon[34468]: pgmap v570: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 370 op/s 2024-09-05T06:59:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:23 smithi115 ceph-mon[34791]: pgmap v570: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 370 op/s 2024-09-05T06:59:23.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:23 smithi086 ceph-mon[30943]: pgmap v570: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 370 op/s 2024-09-05T06:59:24.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33196 5.90 MB/sec execute 162 sec latency 96.817 ms 2024-09-05T06:59:25.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33405 5.97 MB/sec execute 163 sec latency 64.696 ms 2024-09-05T06:59:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:25 smithi096 ceph-mon[34468]: pgmap v571: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:25 smithi115 ceph-mon[34791]: pgmap v571: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:25 smithi086 ceph-mon[30943]: pgmap v571: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:26.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33590 5.98 MB/sec execute 164 sec latency 105.422 ms 2024-09-05T06:59:27.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33747 6.01 MB/sec execute 165 sec latency 56.815 ms 2024-09-05T06:59:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:27 smithi096 ceph-mon[34468]: pgmap v572: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:27 smithi115 ceph-mon[34791]: pgmap v572: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:27 smithi086 ceph-mon[30943]: pgmap v572: 129 pgs: 129 active+clean; 880 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 245 op/s 2024-09-05T06:59:28.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33958 6.06 MB/sec execute 166 sec latency 59.148 ms 2024-09-05T06:59:29.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34164 6.11 MB/sec execute 167 sec latency 159.915 ms 2024-09-05T06:59:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:29 smithi096 ceph-mon[34468]: pgmap v573: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-05T06:59:29.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:29 smithi115 ceph-mon[34791]: pgmap v573: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-05T06:59:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:29 smithi086 ceph-mon[30943]: pgmap v573: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 367 op/s 2024-09-05T06:59:30.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34319 6.11 MB/sec execute 168 sec latency 54.643 ms 2024-09-05T06:59:31.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34458 6.13 MB/sec execute 169 sec latency 73.310 ms 2024-09-05T06:59:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:31 smithi096 ceph-mon[34468]: pgmap v574: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 251 op/s 2024-09-05T06:59:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:31 smithi115 ceph-mon[34791]: pgmap v574: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 251 op/s 2024-09-05T06:59:31.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:31 smithi086 ceph-mon[30943]: pgmap v574: 129 pgs: 129 active+clean; 892 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 251 op/s 2024-09-05T06:59:32.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34643 6.17 MB/sec execute 170 sec latency 71.721 ms 2024-09-05T06:59:33.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T06:59:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:33 smithi086 ceph-mon[30943]: pgmap v575: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 366 op/s 2024-09-05T06:59:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:33.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34808 6.20 MB/sec execute 171 sec latency 67.802 ms 2024-09-05T06:59:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:33 smithi096 ceph-mon[34468]: pgmap v575: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 366 op/s 2024-09-05T06:59:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:33 smithi115 ceph-mon[34791]: pgmap v575: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 366 op/s 2024-09-05T06:59:34.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34930 6.20 MB/sec execute 172 sec latency 197.321 ms 2024-09-05T06:59:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 -- 172.21.15.86:0/1194842905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 msgr2=0x7f8530101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 --2- 172.21.15.86:0/1194842905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 0x7f8530101200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8520009930 tx=0x7f852002a4d0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 -- 172.21.15.86:0/1194842905 shutdown_connections 2024-09-05T06:59:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 --2- 172.21.15.86:0/1194842905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102ef0 0x7f853010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 --2- 172.21.15.86:0/1194842905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8530102570 0x7f85301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 --2- 172.21.15.86:0/1194842905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 0x7f8530101200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 -- 172.21.15.86:0/1194842905 >> 172.21.15.86:0/1194842905 conn(0x7f85300fc7d0 msgr2=0x7f85300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 -- 172.21.15.86:0/1194842905 shutdown_connections 2024-09-05T06:59:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.481+0000 7f8535606640 1 -- 172.21.15.86:0/1194842905 wait complete. 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.482+0000 7f8535606640 1 Processor -- start 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 -- start start 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 0x7f8530198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8530102ef0 0x7f853019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110e30 con 0x7f8530102ef0 2024-09-05T06:59:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f852e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f852e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:49458/0 (socket says 172.21.15.86:49458) 2024-09-05T06:59:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f852e7fc640 1 -- 172.21.15.86:0/3541031446 learned_addr learned my addr 172.21.15.86:0/3541031446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110cb0 con 0x7f8530102570 2024-09-05T06:59:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.483+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110fb0 con 0x7f8530100e20 2024-09-05T06:59:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 -- 172.21.15.86:0/3541031446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 msgr2=0x7f8530198400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 0x7f8530198400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 -- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8530102ef0 msgr2=0x7f853019ccd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8530102ef0 0x7f853019ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 -- 172.21.15.86:0/3541031446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853019d3b0 con 0x7f8530102570 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.484+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8524009f20 tx=0x7f852400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.485+0000 7f8517fff640 1 -- 172.21.15.86:0/3541031446 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524020070 con 0x7f8530102570 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.485+0000 7f8517fff640 1 -- 172.21.15.86:0/3541031446 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f852400cd80 con 0x7f8530102570 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.485+0000 7f8517fff640 1 -- 172.21.15.86:0/3541031446 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852401bb20 con 0x7f8530102570 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.485+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853019d6a0 con 0x7f8530102570 2024-09-05T06:59:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.485+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85300fce70 con 0x7f8530102570 2024-09-05T06:59:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.488+0000 7f8517fff640 1 -- 172.21.15.86:0/3541031446 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8524023070 con 0x7f8530102570 2024-09-05T06:59:35.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.489+0000 7f8517fff640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 0x7f85180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.490+0000 7f852effd640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 0x7f85180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.490+0000 7f8517fff640 1 -- 172.21.15.86:0/3541031446 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f85240d38e0 con 0x7f8530102570 2024-09-05T06:59:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.490+0000 7f852effd640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 0x7f85180a5230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f852002a9f0 tx=0x7f8520005fd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 Processor -- start 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 -- start start 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb4e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f53faa12e0 con 0x55f53fb445d0 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.494+0000 7f85379df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f53fa48ce0 con 0x55f53fb43cb0 2024-09-05T06:59:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f85379df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f53fa9d2a0 con 0x55f53fb4ee90 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb44090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:49462/0 (socket says 172.21.15.86:49462) 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852effd640 1 -- 172.21.15.86:0/4178246066 learned_addr learned my addr 172.21.15.86:0/4178246066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 -- 172.21.15.86:0/4178246066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 msgr2=0x55f53fb44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb44090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 -- 172.21.15.86:0/4178246066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 msgr2=0x55f53fb4e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb4e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.495+0000 7f852f7fe640 1 -- 172.21.15.86:0/4178246066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f53fa2d440 con 0x55f53fb4ee90 2024-09-05T06:59:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.496+0000 7f852f7fe640 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb51280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8528009d90 tx=0x7f852800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2251548489c2e24d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.496+0000 7f85157fa640 1 -- 172.21.15.86:0/4178246066 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528003970 con 0x55f53fb4ee90 2024-09-05T06:59:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.496+0000 7f85157fa640 1 -- 172.21.15.86:0/4178246066 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8528004020 con 0x55f53fb4ee90 2024-09-05T06:59:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.496+0000 7f85157fa640 1 -- 172.21.15.86:0/4178246066 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528010720 con 0x55f53fb4ee90 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 -- 172.21.15.86:0/4178246066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 msgr2=0x55f53fb51280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb51280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8528009d90 tx=0x7f852800e720 comp rx=0 tx=0).stop 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 -- 172.21.15.86:0/4178246066 shutdown_connections 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb51280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb4e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 --2- 172.21.15.86:0/4178246066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.497+0000 7f85379df740 1 -- 172.21.15.86:0/4178246066 >> 172.21.15.86:0/4178246066 conn(0x55f53fb40240 msgr2=0x55f53fb40620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.498+0000 7f85379df740 1 -- 172.21.15.86:0/4178246066 shutdown_connections 2024-09-05T06:59:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.498+0000 7f85379df740 1 -- 172.21.15.86:0/4178246066 wait complete. 2024-09-05T06:59:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.498+0000 7f85379df740 1 -- start start 2024-09-05T06:59:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.498+0000 7f85379df740 1 Processor -- start 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f53fb62a70 con 0x55f53fb43cb0 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f53fa5c050 con 0x55f53fb445d0 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f53fb62bf0 con 0x55f53fb4ee90 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f85379df740 10 client.0 _ll_register_callbacks cb 0x55f53fa9c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f852effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f852f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f852effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48194/0 (socket says 172.21.15.86:48194) 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f852effd640 1 -- 172.21.15.86:0/3528839843 learned_addr learned my addr 172.21.15.86:0/3528839843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.499+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852effd640 1 -- 172.21.15.86:0/3528839843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 msgr2=0x55f53fb4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852effd640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852effd640 1 -- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 msgr2=0x55f53fb4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852effd640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852effd640 1 -- 172.21.15.86:0/3528839843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53f8123c0 con 0x55f53fb43cb0 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852f7fe640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.500+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f852effd640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f8520032e80 tx=0x7f8520031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5e84f6af60ac7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f8512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520035040 con 0x55f53fb43cb0 2024-09-05T06:59:35.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f85200043c0 con 0x55f53fb43cb0 2024-09-05T06:59:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8528004020 con 0x55f53fb43cb0 2024-09-05T06:59:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.501+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85200035f0 con 0x55f53fb43cb0 2024-09-05T06:59:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.502+0000 7f85379df740 10 client.19566 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.502+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f53fb6e4c0 con 0x55f53fb43cb0 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.502+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520003c40 con 0x55f53fb43cb0 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.502+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8520050a90 con 0x55f53fb43cb0 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 10 client.19566 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 10 client.19566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 20 client.19566 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fb71ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 4 client.19566 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8528004020 con 0x55f53fb6f8d0 2024-09-05T06:59:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.503+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fb71ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.504+0000 7f852e7fc640 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fb71ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8473fa8c22be63e0 server_cookie=a0314a038721e105 in_seq=0 out_seq=0 2024-09-05T06:59:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.504+0000 7f8512ff5640 10 client.19566 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.505+0000 7f8512ff5640 1 -- 172.21.15.86:0/3528839843 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8528004020 con 0x55f53fb6f8d0 2024-09-05T06:59:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.505+0000 7f8512ff5640 10 client.19566 handle_command_reply: tid=0 2024-09-05T06:59:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.505+0000 7f85379df740 1 client.19566 shutdown 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.506+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 msgr2=0x55f53fb4e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.506+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f8520032e80 tx=0x7f8520031400 comp rx=0 tx=0).stop 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 shutdown_connections 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fb71ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f8512ff5640 0 client.19566 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fb4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 --2- 172.21.15.86:0/3528839843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb43cb0 0x55f53fb4e840 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.507+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 >> 172.21.15.86:0/3528839843 conn(0x55f53fb40240 msgr2=0x55f53fb36b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.511+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 shutdown_connections 2024-09-05T06:59:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.511+0000 7f85379df740 1 -- 172.21.15.86:0/3528839843 wait complete. 2024-09-05T06:59:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.511+0000 7f85379df740 20 client.19566 trim_cache size 0 max 16384 2024-09-05T06:59:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.537+0000 7f85379df740 1 Processor -- start 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.537+0000 7f85379df740 1 -- start start 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.537+0000 7f85379df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 0x55f53fb6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.537+0000 7f85379df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.537+0000 7f85379df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f85379df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8530073cf0 con 0x55f53fb4ee90 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f85379df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85301a6400 con 0x55f53fb445d0 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f85379df740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85301a4f80 con 0x55f53fb43cb0 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48204/0 (socket says 172.21.15.86:48204) 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 -- 172.21.15.86:0/500421519 learned_addr learned my addr 172.21.15.86:0/500421519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852effd640 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 0x55f53fb6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 -- 172.21.15.86:0/500421519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 msgr2=0x55f53fb6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 0x55f53fb6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 -- 172.21.15.86:0/500421519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 msgr2=0x55f53fb775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852f7fe640 1 -- 172.21.15.86:0/500421519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f53fb413b0 con 0x55f53fb4ee90 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.538+0000 7f852effd640 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 0x55f53fb6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.539+0000 7f852f7fe640 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8528005e30 tx=0x7f8528005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4621ae8582386ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.539+0000 7f8514ff9640 1 -- 172.21.15.86:0/500421519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528025070 con 0x55f53fb4ee90 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.540+0000 7f8514ff9640 1 -- 172.21.15.86:0/500421519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f852800ed50 con 0x55f53fb4ee90 2024-09-05T06:59:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.540+0000 7f8514ff9640 1 -- 172.21.15.86:0/500421519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528018e60 con 0x55f53fb4ee90 2024-09-05T06:59:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 -- 172.21.15.86:0/500421519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 msgr2=0x55f53fb77af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8528005e30 tx=0x7f8528005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 -- 172.21.15.86:0/500421519 shutdown_connections 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb4ee90 0x55f53fb77af0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb445d0 0x55f53fb775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 --2- 172.21.15.86:0/500421519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb43cb0 0x55f53fb6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 -- 172.21.15.86:0/500421519 >> 172.21.15.86:0/500421519 conn(0x55f53fb40240 msgr2=0x55f53fb4dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.541+0000 7f85379df740 1 -- 172.21.15.86:0/500421519 shutdown_connections 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.542+0000 7f85379df740 1 -- 172.21.15.86:0/500421519 wait complete. 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.542+0000 7f85379df740 1 -- start start 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.542+0000 7f85379df740 1 Processor -- start 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f852effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fbd2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f852effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:49480/0 (socket says 172.21.15.86:49480) 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f852effd640 1 -- 172.21.15.86:0/637534303 learned_addr learned my addr 172.21.15.86:0/637534303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85301a5d30 con 0x55f53fb445d0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f852e7fc640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85301a5660 con 0x55f53fb43cb0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.543+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8530071cf0 con 0x55f53fb4ee90 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f85379df740 10 client.0 _ll_register_callbacks cb 0x7f85280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852f7fe640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fbd2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 -- 172.21.15.86:0/637534303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 msgr2=0x55f53fbd2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fbd2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 -- 172.21.15.86:0/637534303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 msgr2=0x55f53fb64d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb64d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53fb37230 con 0x55f53fb43cb0 2024-09-05T06:59:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852f7fe640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fbd2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:35.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f852effd640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8520032e80 tx=0x7f8520031c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ec44429f3b80d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:35.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.544+0000 7f8512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520035050 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852800ed50 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f8520059c60 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f852006a050 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f85379df740 10 client.28391 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f53fbdaef0 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.545+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f852800ed50 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f8520003c40 con 0x55f53fb43cb0 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 10 client.28391 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 10 client.28391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 20 client.28391 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fbdd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 4 client.28391 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f53faae850 con 0x55f53fb6f8d0 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.546+0000 7f852e7fc640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fbdd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f852e7fc640 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fbdd650 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02a1409f46daac8 server_cookie=dbec20a4e6f950c5 in_seq=0 out_seq=0 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f8512ff5640 10 client.28391 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f8512ff5640 1 -- 172.21.15.86:0/637534303 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f53faae850 con 0x55f53fb6f8d0 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f8512ff5640 10 client.28391 handle_command_reply: tid=0 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 client.28391 shutdown 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 msgr2=0x55f53fb647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8520032e80 tx=0x7f8520031c50 comp rx=0 tx=0).stop 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 shutdown_connections 2024-09-05T06:59:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f53fb6f8d0 0x55f53fbdd650 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f8512ff5640 0 client.28391 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f53fb4ee90 0x55f53fbd2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f53fb445d0 0x55f53fb64d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 --2- 172.21.15.86:0/637534303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f53fb43cb0 0x55f53fb647c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.547+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 >> 172.21.15.86:0/637534303 conn(0x55f53fb40240 msgr2=0x55f53fb45750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.548+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 shutdown_connections 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.548+0000 7f85379df740 1 -- 172.21.15.86:0/637534303 wait complete. 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.548+0000 7f85379df740 20 client.28391 trim_cache size 0 max 16384 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T06:59:35.551 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.550+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 msgr2=0x7f85180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.550+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 0x7f85180a5230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f852002a9f0 tx=0x7f8520005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.550+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 msgr2=0x7f8530198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.550+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8524009f20 tx=0x7f852400ee80 comp rx=0 tx=0).stop 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 shutdown_connections 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f85180a2d70 0x7f85180a5230 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f852002a9f0 tx=0x7f8520005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8530102ef0 0x7f853019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530102570 0x7f8530198940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 --2- 172.21.15.86:0/3541031446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8530100e20 0x7f8530198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:35.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 >> 172.21.15.86:0/3541031446 conn(0x7f85300fc7d0 msgr2=0x7f853006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:35.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.551+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 shutdown_connections 2024-09-05T06:59:35.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:35.552+0000 7f8535606640 1 -- 172.21.15.86:0/3541031446 wait complete. 2024-09-05T06:59:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:35 smithi086 ceph-mon[30943]: pgmap v576: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:35.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35065 6.19 MB/sec execute 173 sec latency 112.088 ms 2024-09-05T06:59:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:35 smithi096 ceph-mon[34468]: pgmap v576: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:35 smithi115 ceph-mon[34791]: pgmap v576: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:36.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35219 6.17 MB/sec execute 174 sec latency 108.018 ms 2024-09-05T06:59:37.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T06:59:37.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T06:59:37.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35374 6.18 MB/sec execute 175 sec latency 102.171 ms 2024-09-05T06:59:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:37 smithi096 ceph-mon[34468]: pgmap v577: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:37 smithi115 ceph-mon[34791]: pgmap v577: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:37.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:37 smithi086 ceph-mon[30943]: pgmap v577: 129 pgs: 129 active+clean; 910 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 236 op/s 2024-09-05T06:59:38.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35513 6.18 MB/sec execute 176 sec latency 150.207 ms 2024-09-05T06:59:39.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35659 6.19 MB/sec execute 177 sec latency 164.498 ms 2024-09-05T06:59:39.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:39 smithi096 ceph-mon[34468]: pgmap v578: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 341 op/s 2024-09-05T06:59:39.888 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:39 smithi086 ceph-mon[30943]: pgmap v578: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 341 op/s 2024-09-05T06:59:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:39 smithi115 ceph-mon[34791]: pgmap v578: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 341 op/s 2024-09-05T06:59:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 -- 172.21.15.86:0/1920362779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 msgr2=0x7f8be01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 --2- 172.21.15.86:0/1920362779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be01089f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009900 tx=0x7f8bd402a410 comp rx=0 tx=0).stop 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 -- 172.21.15.86:0/1920362779 shutdown_connections 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 --2- 172.21.15.86:0/1920362779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 0x7f8be0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 --2- 172.21.15.86:0/1920362779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be01089f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 --2- 172.21.15.86:0/1920362779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 -- 172.21.15.86:0/1920362779 >> 172.21.15.86:0/1920362779 conn(0x7f8be00ffab0 msgr2=0x7f8be0101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 -- 172.21.15.86:0/1920362779 shutdown_connections 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.012+0000 7f8be5179640 1 -- 172.21.15.86:0/1920362779 wait complete. 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.013+0000 7f8be5179640 1 Processor -- start 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.013+0000 7f8be5179640 1 -- start start 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48234/0 (socket says 172.21.15.86:48234) 2024-09-05T06:59:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 0x7f8be019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0114040 con 0x7f8be0103ff0 2024-09-05T06:59:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0113ec0 con 0x7f8be01049a0 2024-09-05T06:59:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8be5179640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01141c0 con 0x7f8be0109120 2024-09-05T06:59:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 -- 172.21.15.86:0/2294386853 learned_addr learned my addr 172.21.15.86:0/2294386853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bdf59e640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 0x7f8be019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bde59c640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 -- 172.21.15.86:0/2294386853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 msgr2=0x7f8be019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 0x7f8be019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 -- 172.21.15.86:0/2294386853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 msgr2=0x7f8be019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.014+0000 7f8bded9d640 1 -- 172.21.15.86:0/2294386853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be019f4f0 con 0x7f8be0103ff0 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8bded9d640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ea60 tx=0x7f8bd000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000cde0 con 0x7f8be0103ff0 2024-09-05T06:59:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd00035f0 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be019f7e0 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0010690 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.015+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01a70c0 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.017+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be01043d0 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.018+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8bd00037a0 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.018+0000 7f8bc7fff640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 0x7f8bc80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.019+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f8bd00cfb80 con 0x7f8be0103ff0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.019+0000 7f8bde59c640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 0x7f8bc80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.019+0000 7f8bde59c640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 0x7f8bc80a5230 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bd402a930 tx=0x7f8bd4005fd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.026+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bd009c360 con 0x7f8be0103ff0 2024-09-05T06:59:40.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.393+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8be010c360 con 0x7f8be0103ff0 2024-09-05T06:59:40.395 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T06:59:40.395 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"btime":"2024-09-05T06:57:51:240434+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:57:50.408880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T06:59:40.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.394+0000 7f8bc7fff640 1 -- 172.21.15.86:0/2294386853 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11977 (secure 0 0 0) 0x7f8bd009bc60 con 0x7f8be0103ff0 2024-09-05T06:59:40.396 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 msgr2=0x7f8bc80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 0x7f8bc80a5230 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bd402a930 tx=0x7f8bd4005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 msgr2=0x7f8be019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ea60 tx=0x7f8bd000ef20 comp rx=0 tx=0).stop 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 shutdown_connections 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8bc80a2d70 0x7f8bc80a5230 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bd402a930 tx=0x7f8bd4005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be0109120 0x7f8be019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8be01049a0 0x7f8be019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 --2- 172.21.15.86:0/2294386853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be0103ff0 0x7f8be019a540 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.397+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 >> 172.21.15.86:0/2294386853 conn(0x7f8be00ffab0 msgr2=0x7f8be0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.398+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 shutdown_connections 2024-09-05T06:59:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:40.398+0000 7f8be5179640 1 -- 172.21.15.86:0/2294386853 wait complete. 2024-09-05T06:59:40.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:40 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2294386853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:59:40.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35822 6.22 MB/sec execute 178 sec latency 66.758 ms 2024-09-05T06:59:40.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:40 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2294386853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:59:40.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:40 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2294386853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T06:59:41.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T06:59:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:41.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35994 6.25 MB/sec execute 179 sec latency 75.189 ms 2024-09-05T06:59:41.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:41 smithi096 ceph-mon[34468]: pgmap v579: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 219 op/s 2024-09-05T06:59:41.835 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:41 smithi086 ceph-mon[30943]: pgmap v579: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 219 op/s 2024-09-05T06:59:41.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:41 smithi115 ceph-mon[34791]: pgmap v579: 129 pgs: 129 active+clean; 922 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 219 op/s 2024-09-05T06:59:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 -- 172.21.15.86:0/3347083162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108450 msgr2=0x7f1c98108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3347083162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108450 0x7f1c98108830 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009950 tx=0x7f1c8002a440 comp rx=0 tx=0).stop 2024-09-05T06:59:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 -- 172.21.15.86:0/3347083162 shutdown_connections 2024-09-05T06:59:42.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3347083162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c981150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3347083162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108e00 0x7f1c9810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3347083162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108450 0x7f1c98108830 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.110+0000 7f1c9df36640 1 -- 172.21.15.86:0/3347083162 >> 172.21.15.86:0/3347083162 conn(0x7f1c98077ba0 msgr2=0x7f1c98079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.111+0000 7f1c9df36640 1 -- 172.21.15.86:0/3347083162 shutdown_connections 2024-09-05T06:59:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.111+0000 7f1c9df36640 1 -- 172.21.15.86:0/3347083162 wait complete. 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.111+0000 7f1c9df36640 1 Processor -- start 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 -- start start 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 0x7f1c981a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c9819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98118430 con 0x7f1c98108e00 2024-09-05T06:59:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981182b0 con 0x7f1c98108450 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48252/0 (socket says 172.21.15.86:48252) 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c96ffd640 1 -- 172.21.15.86:0/3800540609 learned_addr learned my addr 172.21.15.86:0/3800540609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c97fff640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c9819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.112+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981185b0 con 0x7f1c9810d530 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c977fe640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 0x7f1c981a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 -- 172.21.15.86:0/3800540609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 msgr2=0x7f1c9819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c9819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 -- 172.21.15.86:0/3800540609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 msgr2=0x7f1c981a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 0x7f1c981a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 -- 172.21.15.86:0/3800540609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9819ef60 con 0x7f1c98108e00 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c977fe640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 0x7f1c981a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:59:42.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c97fff640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c9819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800ebc0 tx=0x7f1c8800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c94ff9640 1 -- 172.21.15.86:0/3800540609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88010630 con 0x7f1c98108e00 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9819f250 con 0x7f1c98108e00 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c94ff9640 1 -- 172.21.15.86:0/3800540609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c88010c80 con 0x7f1c98108e00 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c94ff9640 1 -- 172.21.15.86:0/3800540609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88018bd0 con 0x7f1c98108e00 2024-09-05T06:59:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.113+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c981ab4a0 con 0x7f1c98108e00 2024-09-05T06:59:42.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.116+0000 7f1c94ff9640 1 -- 172.21.15.86:0/3800540609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1c880107d0 con 0x7f1c98108e00 2024-09-05T06:59:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c94ff9640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 0x7f1c840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c94ff9640 1 -- 172.21.15.86:0/3800540609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f1c880cfff0 con 0x7f1c98108e00 2024-09-05T06:59:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c977fe640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 0x7f1c840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 Processor -- start 2024-09-05T06:59:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 -- start start 2024-09-05T06:59:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57eb40 0x56140c57d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 0x56140c57da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 0x56140c58a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56140c4e32f0 con 0x56140c588370 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56140c48acf0 con 0x56140c57d5d0 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1ca030f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56140c4df2b0 con 0x56140c57eb40 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 0x56140c58a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 0x56140c57da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 0x56140c58a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48258/0 (socket says 172.21.15.86:48258) 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c97fff640 1 -- 172.21.15.86:0/1468511522 learned_addr learned my addr 172.21.15.86:0/1468511522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.117+0000 7f1c977fe640 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57eb40 0x56140c57d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c977fe640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 0x7f1c840a5230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005ab0 tx=0x7f1c80005fd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 -- 172.21.15.86:0/1468511522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57eb40 msgr2=0x56140c57d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57eb40 0x56140c57d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 -- 172.21.15.86:0/1468511522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 msgr2=0x56140c58a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 0x56140c58a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 -- 172.21.15.86:0/1468511522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140c46f3a0 con 0x56140c57d5d0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 0x56140c57da10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c880cd2a0 tx=0x7f1c8809b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec3d711bbace003 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c8dffb640 1 -- 172.21.15.86:0/1468511522 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8809c770 con 0x56140c57d5d0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c8dffb640 1 -- 172.21.15.86:0/1468511522 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c8809c070 con 0x56140c57d5d0 2024-09-05T06:59:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.118+0000 7f1c8dffb640 1 -- 172.21.15.86:0/1468511522 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88098870 con 0x56140c57d5d0 2024-09-05T06:59:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 -- 172.21.15.86:0/1468511522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 msgr2=0x56140c57da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 0x56140c57da10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c880cd2a0 tx=0x7f1c8809b9d0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 -- 172.21.15.86:0/1468511522 shutdown_connections 2024-09-05T06:59:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c588370 0x56140c58a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57d5d0 0x56140c57da10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 --2- 172.21.15.86:0/1468511522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57eb40 0x56140c57d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 -- 172.21.15.86:0/1468511522 >> 172.21.15.86:0/1468511522 conn(0x56140c546310 msgr2=0x56140c579c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 -- 172.21.15.86:0/1468511522 shutdown_connections 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.119+0000 7f1ca030f740 1 -- 172.21.15.86:0/1468511522 wait complete. 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 -- start start 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 Processor -- start 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57d5d0 0x56140c587e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 0x56140c585840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56140c58b0c0 con 0x56140c57eb40 2024-09-05T06:59:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56140c49e060 con 0x56140c588370 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56140c58b240 con 0x56140c57d5d0 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.120+0000 7f1ca030f740 10 client.0 _ll_register_callbacks cb 0x56140c1f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48268/0 (socket says 172.21.15.86:48268) 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 -- 172.21.15.86:0/2471995766 learned_addr learned my addr 172.21.15.86:0/2471995766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 0x56140c585840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57d5d0 0x56140c587e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 -- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57d5d0 msgr2=0x56140c587e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57d5d0 0x56140c587e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 -- 172.21.15.86:0/2471995766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 msgr2=0x56140c585840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 0x56140c585840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 -- 172.21.15.86:0/2471995766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140c2543c0 con 0x56140c57eb40 2024-09-05T06:59:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 0x56140c585840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800cba0 tx=0x7f1c880cd7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebef5589ac0de83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8809c770 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c8809c070 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f1c8809b4c0 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.121+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c880a4ce0 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 10 client.19590 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56140c596c10 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c8809c070 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1c880a15b0 con 0x56140c57eb40 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 10 client.19590 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 10 client.19590 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:59:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 20 client.19590 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c59a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 4 client.19590 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56140c4f0860 con 0x56140c597d90 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.122+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c59a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.123+0000 7f1c76ffd640 10 client.19590 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:59:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.123+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c59a180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba9776ee71a73907 server_cookie=6670f6946ad6f5c0 in_seq=0 out_seq=0 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.123+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2471995766 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56140c4f0860 con 0x56140c597d90 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.123+0000 7f1c76ffd640 10 client.19590 handle_command_reply: tid=0 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 client.19590 shutdown 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 msgr2=0x56140c5850b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800cba0 tx=0x7f1c880cd7b0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 shutdown_connections 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c59a180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1c76ffd640 0 client.19590 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c588370 0x56140c585840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57eb40 0x56140c5850b0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2471995766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c57d5d0 0x56140c587e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 >> 172.21.15.86:0/2471995766 conn(0x56140c546310 msgr2=0x56140c57f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 shutdown_connections 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.124+0000 7f1ca030f740 1 -- 172.21.15.86:0/2471995766 wait complete. 2024-09-05T06:59:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.125+0000 7f1ca030f740 20 client.19590 trim_cache size 0 max 16384 2024-09-05T06:59:42.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.148+0000 7f1ca030f740 1 Processor -- start 2024-09-05T06:59:42.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 -- start start 2024-09-05T06:59:42.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c593be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b5870 con 0x56140c57d5d0 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b7410 con 0x56140c57eb40 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1ca030f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b5f40 con 0x56140c588370 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59218/0 (socket says 172.21.15.86:59218) 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48284/0 (socket says 172.21.15.86:48284) 2024-09-05T06:59:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c97fff640 1 -- 172.21.15.86:0/4126453009 learned_addr learned my addr 172.21.15.86:0/4126453009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.149+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c593be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 -- 172.21.15.86:0/4126453009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 msgr2=0x56140c593be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c593be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 -- 172.21.15.86:0/4126453009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 msgr2=0x56140c584330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 -- 172.21.15.86:0/4126453009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140c579510 con 0x56140c588370 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c977fe640 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c97fff640 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c90012920 tx=0x7f1c90005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952fa5cb019f3240 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c8d7fa640 1 -- 172.21.15.86:0/4126453009 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90012190 con 0x56140c588370 2024-09-05T06:59:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c8d7fa640 1 -- 172.21.15.86:0/4126453009 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c9000fdd0 con 0x56140c588370 2024-09-05T06:59:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.150+0000 7f1c8d7fa640 1 -- 172.21.15.86:0/4126453009 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90004a00 con 0x56140c588370 2024-09-05T06:59:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 -- 172.21.15.86:0/4126453009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 msgr2=0x56140c59d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c90012920 tx=0x7f1c90005f90 comp rx=0 tx=0).stop 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 -- 172.21.15.86:0/4126453009 shutdown_connections 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59d1a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c593be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 --2- 172.21.15.86:0/4126453009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c584330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.151+0000 7f1ca030f740 1 -- 172.21.15.86:0/4126453009 >> 172.21.15.86:0/4126453009 conn(0x56140c546310 msgr2=0x56140c5934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.152+0000 7f1ca030f740 1 -- 172.21.15.86:0/4126453009 shutdown_connections 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.152+0000 7f1ca030f740 1 -- 172.21.15.86:0/4126453009 wait complete. 2024-09-05T06:59:42.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.152+0000 7f1ca030f740 1 -- start start 2024-09-05T06:59:42.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.153+0000 7f1ca030f740 1 Processor -- start 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c59b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48298/0 (socket says 172.21.15.86:48298) 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1c977fe640 1 -- 172.21.15.86:0/2407959204 learned_addr learned my addr 172.21.15.86:0/2407959204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b6d40 con 0x56140c57d5d0 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b6670 con 0x56140c57eb40 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1c96ffd640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c59b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.154+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981b3870 con 0x56140c588370 2024-09-05T06:59:42.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1ca030f740 10 client.0 _ll_register_callbacks cb 0x7f1c90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 -- 172.21.15.86:0/2407959204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 msgr2=0x56140c59b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c59b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 -- 172.21.15.86:0/2407959204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 msgr2=0x56140c59aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140c6126e0 con 0x56140c588370 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.155+0000 7f1c97fff640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59b920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56140c57a9d0 tx=0x7f1c90005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81a3ba8441f944e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1c76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:59:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9002a020 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c9000fdd0 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f1c90004050 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1c90035050 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.156+0000 7f1ca030f740 10 client.28432 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.157+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56140c613900 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.157+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c9000fdd0 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f1c90004b70 con 0x56140c588370 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 10 client.28432 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 10 client.28432 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 20 client.28432 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:42.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c616030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 4 client.28432 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:59:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56140c4ead70 con 0x56140c597d90 2024-09-05T06:59:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.158+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c616030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.159+0000 7f1c977fe640 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c616030 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=426a6b5227af466d server_cookie=eec732bf749070eb in_seq=0 out_seq=0 2024-09-05T06:59:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.159+0000 7f1c76ffd640 10 client.28432 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T06:59:42.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1c76ffd640 1 -- 172.21.15.86:0/2407959204 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56140c4ead70 con 0x56140c597d90 2024-09-05T06:59:42.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1c76ffd640 10 client.28432 handle_command_reply: tid=0 2024-09-05T06:59:42.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 client.28432 shutdown 2024-09-05T06:59:42.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 msgr2=0x56140c59b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59b920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56140c57a9d0 tx=0x7f1c90005660 comp rx=0 tx=0).stop 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 shutdown_connections 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56140c597d90 0x56140c616030 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56140c588370 0x56140c59b920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56140c57eb40 0x56140c59b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1c76ffd640 0 client.28432 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.161+0000 7f1ca030f740 1 --2- 172.21.15.86:0/2407959204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56140c57d5d0 0x56140c59aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.162+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 >> 172.21.15.86:0/2407959204 conn(0x56140c546310 msgr2=0x56140c578d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.162+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 shutdown_connections 2024-09-05T06:59:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.162+0000 7f1ca030f740 1 -- 172.21.15.86:0/2407959204 wait complete. 2024-09-05T06:59:42.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.162+0000 7f1ca030f740 20 client.28432 trim_cache size 0 max 16384 2024-09-05T06:59:42.164 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:59:42.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.163+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 msgr2=0x7f1c840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.163+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 0x7f1c840a5230 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005ab0 tx=0x7f1c80005fd0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.163+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 msgr2=0x7f1c981a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.163+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800ebc0 tx=0x7f1c8800c6a0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 shutdown_connections 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1c840a2d70 0x7f1c840a5230 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c9810d530 0x7f1c9819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c98108e00 0x7f1c981a4b80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 --2- 172.21.15.86:0/3800540609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c98108450 0x7f1c981a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 >> 172.21.15.86:0/3800540609 conn(0x7f1c98077ba0 msgr2=0x7f1c98078f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 shutdown_connections 2024-09-05T06:59:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:42.164+0000 7f1c9df36640 1 -- 172.21.15.86:0/3800540609 wait complete. 2024-09-05T06:59:42.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36180 6.27 MB/sec execute 180 sec latency 85.861 ms 2024-09-05T06:59:42.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T06:59:43.262 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:43.704 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:43 smithi086 ceph-mon[30943]: pgmap v580: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 339 op/s 2024-09-05T06:59:43.763 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36343 6.26 MB/sec execute 181 sec latency 67.922 ms 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 -- 172.21.15.86:0/1629844406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74101470 msgr2=0x7f5f74101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 --2- 172.21.15.86:0/1629844406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74101470 0x7f5f74101850 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f6402a410 comp rx=0 tx=0).stop 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 -- 172.21.15.86:0/1629844406 shutdown_connections 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 --2- 172.21.15.86:0/1629844406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74105de0 0x7f5f7410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 --2- 172.21.15.86:0/1629844406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 0x7f5f741056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 --2- 172.21.15.86:0/1629844406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74101470 0x7f5f74101850 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 -- 172.21.15.86:0/1629844406 >> 172.21.15.86:0/1629844406 conn(0x7f5f74077230 msgr2=0x7f5f740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 -- 172.21.15.86:0/1629844406 shutdown_connections 2024-09-05T06:59:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.824+0000 7f5f788bc640 1 -- 172.21.15.86:0/1629844406 wait complete. 2024-09-05T06:59:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 Processor -- start 2024-09-05T06:59:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 -- start start 2024-09-05T06:59:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74101470 0x7f5f7419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 0x7f5f7419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7410fc30 con 0x7f5f74105de0 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7410fab0 con 0x7f5f74101e20 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.825+0000 7f5f788bc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7410fdb0 con 0x7f5f74101470 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 0x7f5f7419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74101470 0x7f5f7419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34082/0 (socket says 172.21.15.86:34082) 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 -- 172.21.15.86:0/26181920 learned_addr learned my addr 172.21.15.86:0/26181920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 -- 172.21.15.86:0/26181920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74101470 msgr2=0x7f5f7419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74101470 0x7f5f7419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 -- 172.21.15.86:0/26181920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 msgr2=0x7f5f7419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 0x7f5f7419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 -- 172.21.15.86:0/26181920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7419f4d0 con 0x7f5f74105de0 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.826+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00eb10 tx=0x7f5f6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.827+0000 7f5f637fe640 1 -- 172.21.15.86:0/26181920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004b20 con 0x7f5f74105de0 2024-09-05T06:59:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.827+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7419f7c0 con 0x7f5f74105de0 2024-09-05T06:59:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.827+0000 7f5f637fe640 1 -- 172.21.15.86:0/26181920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f6c004cc0 con 0x7f5f74105de0 2024-09-05T06:59:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.827+0000 7f5f637fe640 1 -- 172.21.15.86:0/26181920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c010690 con 0x7f5f74105de0 2024-09-05T06:59:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.827+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a70a0 con 0x7f5f74105de0 2024-09-05T06:59:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.830+0000 7f5f637fe640 1 -- 172.21.15.86:0/26181920 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f6c005020 con 0x7f5f74105de0 2024-09-05T06:59:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 Processor -- start 2024-09-05T06:59:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 -- start start 2024-09-05T06:59:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f637fe640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 0x7f5f5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557652d8b2f0 con 0x557652e26b40 2024-09-05T06:59:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557652d32cf0 con 0x557652e30370 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557652d872b0 con 0x557652e255d0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 0x7f5f5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.831+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34088/0 (socket says 172.21.15.86:34088) 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f637fe640 1 -- 172.21.15.86:0/26181920 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5f6c0cff60 con 0x7f5f74105de0 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f7259c640 1 -- 172.21.15.86:0/2828791287 learned_addr learned my addr 172.21.15.86:0/2828791287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 -- 172.21.15.86:0/2828791287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 msgr2=0x557652e25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 -- 172.21.15.86:0/2828791287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 msgr2=0x557652e25090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f72d9d640 1 -- 172.21.15.86:0/2828791287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557652d173a0 con 0x557652e30370 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.832+0000 7f5f7259c640 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.833+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 0x7f5f5c0a5230 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800b800 tx=0x7f5f6800bb10 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.833+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e32760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0cd210 tx=0x7f5f6c005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=827c43c1ce99ebd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.833+0000 7f5f60ff9640 1 -- 172.21.15.86:0/2828791287 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c09c6e0 con 0x557652e30370 2024-09-05T06:59:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.833+0000 7f5f60ff9640 1 -- 172.21.15.86:0/2828791287 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f6c05fe30 con 0x557652e30370 2024-09-05T06:59:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.833+0000 7f5f60ff9640 1 -- 172.21.15.86:0/2828791287 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c05e0d0 con 0x557652e30370 2024-09-05T06:59:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2828791287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 msgr2=0x557652e32760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e32760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0cd210 tx=0x7f5f6c005810 comp rx=0 tx=0).stop 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2828791287 shutdown_connections 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e32760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2828791287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.834+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2828791287 >> 172.21.15.86:0/2828791287 conn(0x557652dee310 msgr2=0x557652e21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2828791287 shutdown_connections 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2828791287 wait complete. 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 -- start start 2024-09-05T06:59:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 Processor -- start 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.835+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e2dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557652e33010 con 0x557652e30370 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557652d46060 con 0x557652e26b40 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557652e33190 con 0x557652e255d0 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47864/0 (socket says 172.21.15.86:47864) 2024-09-05T06:59:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 -- 172.21.15.86:0/1078161090 learned_addr learned my addr 172.21.15.86:0/1078161090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7ac95740 10 client.0 _ll_register_callbacks cb 0x557652a998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e2dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 -- 172.21.15.86:0/1078161090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 msgr2=0x557652e2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 -- 172.21.15.86:0/1078161090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 msgr2=0x557652e2dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e2dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 -- 172.21.15.86:0/1078161090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557652afc3c0 con 0x557652e26b40 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f7259c640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f6801ac10 tx=0x7f5f6800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4694ad84fbc68671 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.836+0000 7f5f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T06:59:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68022040 con 0x557652e26b40 2024-09-05T06:59:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c05fe30 con 0x557652e26b40 2024-09-05T06:59:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5f68009040 con 0x557652e26b40 2024-09-05T06:59:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5f68034050 con 0x557652e26b40 2024-09-05T06:59:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 10 client.28421 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557652e3ebf0 con 0x557652e26b40 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f6c05fe30 con 0x557652e26b40 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f5f68026790 con 0x557652e26b40 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 10 client.28421 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 10 client.28421 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 20 client.28421 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652e40440 0x557652e42900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 4 client.28421 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.837+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557652d98860 con 0x557652e40440 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.838+0000 7f5f7259c640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652e40440 0x557652e42900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.838+0000 7f5f59ffb640 10 client.28421 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.838+0000 7f5f7259c640 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652e40440 0x557652e42900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aa8d8eb2bcf3533b server_cookie=20591c5ef347750 in_seq=0 out_seq=0 2024-09-05T06:59:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f59ffb640 1 -- 172.21.15.86:0/1078161090 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557652d98860 con 0x557652e40440 2024-09-05T06:59:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f59ffb640 10 client.28421 handle_command_reply: tid=0 2024-09-05T06:59:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f7ac95740 1 client.28421 shutdown 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 msgr2=0x557652e2d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f6801ac10 tx=0x7f5f6800d040 comp rx=0 tx=0).stop 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 shutdown_connections 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.839+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652e40440 0x557652e42900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e2dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e26b40 0x557652e2d660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/1078161090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 >> 172.21.15.86:0/1078161090 conn(0x557652dee310 msgr2=0x557652e279c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f59ffb640 0 client.28421 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 shutdown_connections 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 1 -- 172.21.15.86:0/1078161090 wait complete. 2024-09-05T06:59:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.840+0000 7f5f7ac95740 20 client.28421 trim_cache size 0 max 16384 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.863+0000 7f5f7ac95740 1 Processor -- start 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.863+0000 7f5f7ac95740 1 -- start start 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 0x557652e281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 0x557652e286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e48210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741acf90 con 0x557652e30370 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741aeb30 con 0x557652e255d0 2024-09-05T06:59:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741ad660 con 0x557652e26b40 2024-09-05T06:59:43.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 0x557652e286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 0x557652e281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 0x557652e286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:54420/0 (socket says 172.21.15.86:54420) 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f71d9b640 1 -- 172.21.15.86:0/2596573393 learned_addr learned my addr 172.21.15.86:0/2596573393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e48210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 -- 172.21.15.86:0/2596573393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 msgr2=0x557652e286f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 0x557652e286f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 -- 172.21.15.86:0/2596573393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 msgr2=0x557652e48210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e48210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.864+0000 7f5f7259c640 1 -- 172.21.15.86:0/2596573393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557652e33180 con 0x557652e255d0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.865+0000 7f5f7259c640 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 0x557652e281b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000c00 tx=0x7f5f64005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbe00ea93685b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.865+0000 7f5f5bfff640 1 -- 172.21.15.86:0/2596573393 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64035040 con 0x557652e255d0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.865+0000 7f5f5bfff640 1 -- 172.21.15.86:0/2596573393 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f640054f0 con 0x557652e255d0 2024-09-05T06:59:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.865+0000 7f5f5bfff640 1 -- 172.21.15.86:0/2596573393 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64004d00 con 0x557652e255d0 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2596573393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 msgr2=0x557652e281b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 0x557652e281b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000c00 tx=0x7f5f64005a20 comp rx=0 tx=0).stop 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2596573393 shutdown_connections 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e30370 0x557652e48210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e26b40 0x557652e286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/2596573393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e255d0 0x557652e281b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2596573393 >> 172.21.15.86:0/2596573393 conn(0x557652dee310 msgr2=0x557652e2f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2596573393 shutdown_connections 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- 172.21.15.86:0/2596573393 wait complete. 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 -- start start 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.866+0000 7f5f7ac95740 1 Processor -- start 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741ae460 con 0x557652e26b40 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741add90 con 0x557652e30370 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741aaf90 con 0x557652e255d0 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7ac95740 10 client.0 _ll_register_callbacks cb 0x7f5f6801f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f7259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34112/0 (socket says 172.21.15.86:34112) 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 -- 172.21.15.86:0/4148032994 learned_addr learned my addr 172.21.15.86:0/4148032994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 -- 172.21.15.86:0/4148032994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 msgr2=0x557652e388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 -- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 msgr2=0x557652e34130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e34130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f71d9b640 1 -- 172.21.15.86:0/4148032994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557652e34670 con 0x557652e26b40 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.867+0000 7f5f72d9d640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e34130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f71d9b640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5f68026c30 tx=0x7f5f6801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc303cde552ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68022040 con 0x557652e26b40 2024-09-05T06:59:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5f68026420 con 0x557652e26b40 2024-09-05T06:59:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f640054f0 con 0x557652e26b40 2024-09-05T06:59:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5f68048050 con 0x557652e26b40 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f7ac95740 10 client.19614 fetch_fsmap learned FSMap version 43 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.868+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557652eaaf30 con 0x557652e26b40 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f640054f0 con 0x557652e26b40 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f5f6803a620 con 0x557652e26b40 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 10 client.19614 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 10 client.19614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 20 client.19614 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652eab680 0x557652eadaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7259c640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652eab680 0x557652eadaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 4 client.19614 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.869+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557652d92d70 con 0x557652eab680 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7259c640 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652eab680 0x557652eadaa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=146fafb48f327e4a server_cookie=aabaacd6606a75d2 in_seq=0 out_seq=0 2024-09-05T06:59:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f59ffb640 10 client.19614 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f59ffb640 1 -- 172.21.15.86:0/4148032994 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557652d92d70 con 0x557652eab680 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f59ffb640 10 client.19614 handle_command_reply: tid=0 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 client.19614 shutdown 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 msgr2=0x557652e33bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5f68026c30 tx=0x7f5f6801cbe0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 shutdown_connections 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x557652eab680 0x557652eadaa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557652e30370 0x557652e34130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557652e26b40 0x557652e33bf0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 --2- 172.21.15.86:0/4148032994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557652e255d0 0x557652e388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.870+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 >> 172.21.15.86:0/4148032994 conn(0x557652dee310 msgr2=0x557652e2d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.875 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.871+0000 7f5f59ffb640 0 client.19614 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.871+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 shutdown_connections 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.871+0000 7f5f7ac95740 1 -- 172.21.15.86:0/4148032994 wait complete. 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.871+0000 7f5f7ac95740 20 client.19614 trim_cache size 0 max 16384 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 msgr2=0x7f5f5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 0x7f5f5c0a5230 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800b800 tx=0x7f5f6800bb10 comp rx=0 tx=0).stop 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 msgr2=0x7f5f7419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00eb10 tx=0x7f5f6c00c2d0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 shutdown_connections 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5f5c0a2d70 0x7f5f5c0a5230 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f74105de0 0x7f5f7419edf0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f74101e20 0x7f5f7419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 --2- 172.21.15.86:0/26181920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f74101470 0x7f5f7419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 >> 172.21.15.86:0/26181920 conn(0x7f5f74077230 msgr2=0x7f5f740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 shutdown_connections 2024-09-05T06:59:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:43.872+0000 7f5f788bc640 1 -- 172.21.15.86:0/26181920 wait complete. 2024-09-05T06:59:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:43 smithi115 ceph-mon[34791]: pgmap v580: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 339 op/s 2024-09-05T06:59:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:43 smithi096 ceph-mon[34468]: pgmap v580: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 339 op/s 2024-09-05T06:59:44.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36470 6.23 MB/sec execute 182 sec latency 175.384 ms 2024-09-05T06:59:44.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:44 smithi115 ceph-mon[34791]: pgmap v581: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:44.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:44 smithi086 ceph-mon[30943]: pgmap v581: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:45.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:44 smithi096 ceph-mon[34468]: pgmap v581: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:45.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36623 6.23 MB/sec execute 183 sec latency 197.959 ms 2024-09-05T06:59:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:45 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:45.986 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:45 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:45 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:46.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36778 6.26 MB/sec execute 184 sec latency 139.751 ms 2024-09-05T06:59:46.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:46 smithi115 ceph-mon[34791]: pgmap v582: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:46.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:46 smithi086 ceph-mon[30943]: pgmap v582: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:47.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:46 smithi096 ceph-mon[34468]: pgmap v582: 129 pgs: 129 active+clean; 920 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-09-05T06:59:47.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36958 6.27 MB/sec execute 185 sec latency 91.164 ms 2024-09-05T06:59:48.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37099 6.30 MB/sec execute 186 sec latency 170.398 ms 2024-09-05T06:59:48.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:48 smithi115 ceph-mon[34791]: pgmap v583: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 331 op/s 2024-09-05T06:59:48.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:48 smithi086 ceph-mon[30943]: pgmap v583: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 331 op/s 2024-09-05T06:59:48.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T06:59:49.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:48 smithi096 ceph-mon[34468]: pgmap v583: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 331 op/s 2024-09-05T06:59:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:49.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37279 6.34 MB/sec execute 187 sec latency 107.673 ms 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 -- 172.21.15.86:0/1545257560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e41012f0 msgr2=0x7ff2e410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/1545257560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e41012f0 0x7ff2e410dfb0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009900 tx=0x7ff2d402a3f0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 -- 172.21.15.86:0/1545257560 shutdown_connections 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/1545257560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e410e4f0 0x7ff2e41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/1545257560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e41012f0 0x7ff2e410dfb0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/1545257560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e4100940 0x7ff2e4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 -- 172.21.15.86:0/1545257560 >> 172.21.15.86:0/1545257560 conn(0x7ff2e40fc7d0 msgr2=0x7ff2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 -- 172.21.15.86:0/1545257560 shutdown_connections 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.912+0000 7ff2ebf25640 1 -- 172.21.15.86:0/1545257560 wait complete. 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.913+0000 7ff2ebf25640 1 Processor -- start 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.913+0000 7ff2ebf25640 1 -- start start 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 0x7ff2e419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e41012f0 0x7ff2e419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 0x7ff2e419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4114020 con 0x7ff2e410e4f0 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4113ea0 con 0x7ff2e41012f0 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ebf25640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41141a0 con 0x7ff2e4100940 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 0x7ff2e419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 0x7ff2e419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:54454/0 (socket says 172.21.15.86:54454) 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/2155637542 learned_addr learned my addr 172.21.15.86:0/2155637542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 0x7ff2e419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e41012f0 0x7ff2e419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 -- 172.21.15.86:0/2155637542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 msgr2=0x7ff2e419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 0x7ff2e419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 -- 172.21.15.86:0/2155637542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e41012f0 msgr2=0x7ff2e419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e41012f0 0x7ff2e419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.914+0000 7ff2ea4c2640 1 -- 172.21.15.86:0/2155637542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e419f550 con 0x7ff2e410e4f0 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 0x7ff2e419ee70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005fd0 tx=0x7ff2dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2daffd640 1 -- 172.21.15.86:0/2155637542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00cdf0 con 0x7ff2e410e4f0 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e419f840 con 0x7ff2e410e4f0 2024-09-05T06:59:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2daffd640 1 -- 172.21.15.86:0/2155637542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2dc005020 con 0x7ff2e410e4f0 2024-09-05T06:59:49.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a7120 con 0x7ff2e410e4f0 2024-09-05T06:59:49.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.915+0000 7ff2daffd640 1 -- 172.21.15.86:0/2155637542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc010690 con 0x7ff2e410e4f0 2024-09-05T06:59:49.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.918+0000 7ff2ee2fe740 1 Processor -- start 2024-09-05T06:59:49.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2daffd640 1 -- 172.21.15.86:0/2155637542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff2dc004a40 con 0x7ff2e410e4f0 2024-09-05T06:59:49.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 -- start start 2024-09-05T06:59:49.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866ae0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b866a462f0 con 0x55b866ae1b40 2024-09-05T06:59:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b8669edcf0 con 0x55b866aeb370 2024-09-05T06:59:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b866a422b0 con 0x55b866ae05d0 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2e94c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2e94c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34132/0 (socket says 172.21.15.86:34132) 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.919+0000 7ff2e94c0640 1 -- 172.21.15.86:0/625219424 learned_addr learned my addr 172.21.15.86:0/625219424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866ae0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e94c0640 1 -- 172.21.15.86:0/625219424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 msgr2=0x55b866ae0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866ae0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2daffd640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 0x7ff2d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e94c0640 1 -- 172.21.15.86:0/625219424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 msgr2=0x55b866aed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 0x7ff2d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2e94c0640 1 -- 172.21.15.86:0/625219424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8669d23a0 con 0x55b866ae1b40 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.920+0000 7ff2daffd640 1 -- 172.21.15.86:0/2155637542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7ff2dc0cfef0 con 0x7ff2e410e4f0 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.921+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402a910 tx=0x7ff2d4032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58678efc0c1baa1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.921+0000 7ff2ea4c2640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 0x7ff2d00a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09bc70 tx=0x7ff2dc097b70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.921+0000 7ff2cffff640 1 -- 172.21.15.86:0/625219424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4040070 con 0x55b866ae1b40 2024-09-05T06:59:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.921+0000 7ff2cffff640 1 -- 172.21.15.86:0/625219424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d4004030 con 0x55b866ae1b40 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.921+0000 7ff2cffff640 1 -- 172.21.15.86:0/625219424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4035040 con 0x55b866ae1b40 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/625219424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 msgr2=0x55b866ae0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff2d402a910 tx=0x7ff2d4032bb0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/625219424 shutdown_connections 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866ae0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/625219424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae1b40 0x55b866ae0090 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/625219424 >> 172.21.15.86:0/625219424 conn(0x55b866aa9310 msgr2=0x55b866adcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/625219424 shutdown_connections 2024-09-05T06:59:49.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.922+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/625219424 wait complete. 2024-09-05T06:59:49.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 -- start start 2024-09-05T06:59:49.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 Processor -- start 2024-09-05T06:59:49.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 0x55b866ae8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b866aee060 con 0x55b866ae8cb0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b866a01060 con 0x55b866ae05d0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.923+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b866aee1e0 con 0x55b866aeb370 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2ee2fe740 10 client.0 _ll_register_callbacks cb 0x55b8667548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:54458/0 (socket says 172.21.15.86:54458) 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/3250571255 learned_addr learned my addr 172.21.15.86:0/3250571255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 msgr2=0x55b866ae8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 0x55b866ae8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 0x55b866ae8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/3250571255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 msgr2=0x55b866af7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/3250571255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8667b73c0 con 0x55b866aeb370 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.924+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 0x55b866ae8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0012760 tx=0x7ff2e0012c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567f8324c0f93705 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000f920 con 0x55b866aeb370 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7ff2e000fac0 con 0x55b866aeb370 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d4004030 con 0x55b866aeb370 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff2e000fcc0 con 0x55b866aeb370 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2ee2fe740 10 client.28462 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b866afa160 con 0x55b866aeb370 2024-09-05T06:59:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.925+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e00108e0 con 0x55b866aeb370 2024-09-05T06:59:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7ff2e001c420 con 0x55b866aeb370 2024-09-05T06:59:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 10 client.28462 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 10 client.28462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:59:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 20 client.28462 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866afb6c0 0x55b866afdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 4 client.28462 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2d4004030 con 0x55b866afb6c0 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.926+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866afb6c0 0x55b866afdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.927+0000 7ff2cd7fa640 10 client.28462 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.927+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866afb6c0 0x55b866afdab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5fb65a53fb2e0cb server_cookie=d301d5206e9aa5d3 in_seq=0 out_seq=0 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3250571255 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff2d4004030 con 0x55b866afb6c0 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2cd7fa640 10 client.28462 handle_command_reply: tid=0 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 client.28462 shutdown 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 msgr2=0x55b866ae8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0012760 tx=0x7ff2e0012c20 comp rx=0 tx=0).stop 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 shutdown_connections 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866afb6c0 0x55b866afdab0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866aeb370 0x55b866ae8770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3250571255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866ae05d0 0x55b866ae8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2cd7fa640 0 client.28462 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 >> 172.21.15.86:0/3250571255 conn(0x55b866aa9310 msgr2=0x55b866ae2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 shutdown_connections 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3250571255 wait complete. 2024-09-05T06:59:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.928+0000 7ff2ee2fe740 20 client.28462 trim_cache size 0 max 16384 2024-09-05T06:59:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 Processor -- start 2024-09-05T06:59:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 -- start start 2024-09-05T06:59:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866adbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41b1330 con 0x55b866ae8cb0 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41b2ea0 con 0x55b866aeb370 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41b1a00 con 0x55b866ae05d0 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34160/0 (socket says 172.21.15.86:34160) 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.949+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/2974103043 learned_addr learned my addr 172.21.15.86:0/2974103043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866adbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/2974103043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 msgr2=0x55b866adbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866adbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/2974103043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 msgr2=0x55b866aec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/2974103043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b866adb310 con 0x55b866ae8cb0 2024-09-05T06:59:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009e90 tx=0x7ff2e000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93c70ec7944750f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2cf7fe640 1 -- 172.21.15.86:0/2974103043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00122e0 con 0x55b866ae8cb0 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2cf7fe640 1 -- 172.21.15.86:0/2974103043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e0005560 con 0x55b866ae8cb0 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.950+0000 7ff2cf7fe640 1 -- 172.21.15.86:0/2974103043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001c680 con 0x55b866ae8cb0 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/2974103043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 msgr2=0x55b866adc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009e90 tx=0x7ff2e000f810 comp rx=0 tx=0).stop 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/2974103043 shutdown_connections 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866adc150 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/2974103043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866adbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/2974103043 >> 172.21.15.86:0/2974103043 conn(0x55b866aa9310 msgr2=0x55b866adb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.951+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/2974103043 shutdown_connections 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.952+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/2974103043 wait complete. 2024-09-05T06:59:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.952+0000 7ff2ee2fe740 1 -- start start 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.952+0000 7ff2ee2fe740 1 Processor -- start 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866af3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41b27d0 con 0x55b866ae8cb0 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41b2100 con 0x55b866aeb370 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41af330 con 0x55b866ae05d0 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2ee2fe740 10 client.0 _ll_register_callbacks cb 0x7ff2e0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866af3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e9cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34166/0 (socket says 172.21.15.86:34166) 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e9cc1640 1 -- 172.21.15.86:0/3154157952 learned_addr learned my addr 172.21.15.86:0/3154157952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 -- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 msgr2=0x55b866aee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 -- 172.21.15.86:0/3154157952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 msgr2=0x55b866af39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 -- 172.21.15.86:0/3154157952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b866ae91d0 con 0x55b866ae05d0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2e94c0640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866af3480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2d403e000 tx=0x7ff2d4005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2df3f2051d95a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.953+0000 7ff2cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d404e070 con 0x55b866ae05d0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7ff2d4004950 con 0x55b866ae05d0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e0005560 con 0x55b866ae05d0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff2d4035040 con 0x55b866ae05d0 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 10 client.28468 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b866b6f160 con 0x55b866ae05d0 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e0005560 con 0x55b866ae05d0 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7ff2d4031a40 con 0x55b866ae05d0 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 10 client.28468 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 10 client.28468 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 20 client.28468 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866b6f3f0 0x55b866b717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 4 client.28468 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.954+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b866ae17b0 con 0x55b866b6f3f0 2024-09-05T06:59:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866b6f3f0 0x55b866b717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2cd7fa640 10 client.28468 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T06:59:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2e9cc1640 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866b6f3f0 0x55b866b717e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb972d0435bf707a server_cookie=e501882e7b24e5fb in_seq=0 out_seq=0 2024-09-05T06:59:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2cd7fa640 1 -- 172.21.15.86:0/3154157952 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b866ae17b0 con 0x55b866b6f3f0 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2cd7fa640 10 client.28468 handle_command_reply: tid=0 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.955+0000 7ff2ee2fe740 1 client.28468 shutdown 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 msgr2=0x55b866af3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866af3480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2d403e000 tx=0x7ff2d4005db0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 shutdown_connections 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55b866b6f3f0 0x55b866b717e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b866aeb370 0x55b866aee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b866ae8cb0 0x55b866af39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 --2- 172.21.15.86:0/3154157952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b866ae05d0 0x55b866af3480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 >> 172.21.15.86:0/3154157952 conn(0x55b866aa9310 msgr2=0x55b866adbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2cd7fa640 0 client.28468 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 shutdown_connections 2024-09-05T06:59:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.956+0000 7ff2ee2fe740 1 -- 172.21.15.86:0/3154157952 wait complete. 2024-09-05T06:59:49.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ee2fe740 20 client.28468 trim_cache size 0 max 16384 2024-09-05T06:59:49.961 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 msgr2=0x7ff2d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 0x7ff2d00a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09bc70 tx=0x7ff2dc097b70 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 msgr2=0x7ff2e419ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 0x7ff2e419ee70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005fd0 tx=0x7ff2dc00ef10 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 shutdown_connections 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff2d00a2d70 0x7ff2d00a5230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e410e4f0 0x7ff2e419ee70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e41012f0 0x7ff2e419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 --2- 172.21.15.86:0/2155637542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2e4100940 0x7ff2e419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 >> 172.21.15.86:0/2155637542 conn(0x7ff2e40fc7d0 msgr2=0x7ff2e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.957+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 shutdown_connections 2024-09-05T06:59:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:49.958+0000 7ff2ebf25640 1 -- 172.21.15.86:0/2155637542 wait complete. 2024-09-05T06:59:50.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37504 6.39 MB/sec execute 188 sec latency 87.040 ms 2024-09-05T06:59:50.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:50 smithi115 ceph-mon[34791]: pgmap v584: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-05T06:59:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:50 smithi086 ceph-mon[30943]: pgmap v584: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-05T06:59:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:50 smithi096 ceph-mon[34468]: pgmap v584: 129 pgs: 129 active+clean; 927 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-05T06:59:51.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37683 6.39 MB/sec execute 189 sec latency 105.658 ms 2024-09-05T06:59:52.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T06:59:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:52.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37800 6.38 MB/sec execute 190 sec latency 112.295 ms 2024-09-05T06:59:52.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:52 smithi086 ceph-mon[30943]: pgmap v585: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-05T06:59:52.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:52 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:59:53.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:52 smithi096 ceph-mon[34468]: pgmap v585: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-05T06:59:53.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:52 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:59:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.178+0000 7f7f08914640 1 -- 172.21.15.86:0/1597192193 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40035f0 con 0x7f7f041091f0 2024-09-05T06:59:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/1597192193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041091f0 msgr2=0x7f7f04110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/1597192193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041091f0 0x7f7f04110250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009930 tx=0x7f7ef402a440 comp rx=0 tx=0).stop 2024-09-05T06:59:53.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/1597192193 shutdown_connections 2024-09-05T06:59:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/1597192193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f0411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/1597192193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041095d0 0x7f7f04109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/1597192193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041091f0 0x7f7f04110250 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009930 tx=0x7f7ef402a440 comp rx=0 tx=0).stop 2024-09-05T06:59:53.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.179+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/1597192193 >> 172.21.15.86:0/1597192193 conn(0x7f7f0406e970 msgr2=0x7f7f0406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:52 smithi115 ceph-mon[34791]: pgmap v585: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-05T06:59:53.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:52 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.180+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/1597192193 shutdown_connections 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.181+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/1597192193 wait complete. 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.181+0000 7f7f0bb7a640 1 Processor -- start 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.181+0000 7f7f0bb7a640 1 -- start start 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 0x7f7f041a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f041a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041abb60 0x7f7f041ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04120b70 con 0x7f7f041abb60 2024-09-05T06:59:53.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041209f0 con 0x7f7f0411b140 2024-09-05T06:59:53.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.182+0000 7f7f0bb7a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04120cf0 con 0x7f7f041095d0 2024-09-05T06:59:53.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.183+0000 7f7f09115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f041a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.183+0000 7f7f09115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f041a77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37538/0 (socket says 172.21.15.86:37538) 2024-09-05T06:59:53.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.183+0000 7f7f09115640 1 -- 172.21.15.86:0/2861179939 learned_addr learned my addr 172.21.15.86:0/2861179939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:53.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.183+0000 7f7f0a117640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041abb60 0x7f7f041ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 0x7f7f041a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 -- 172.21.15.86:0/2861179939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 msgr2=0x7f7f041a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f041a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 -- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041abb60 msgr2=0x7f7f041ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041abb60 0x7f7f041ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.185+0000 7f7f09916640 1 -- 172.21.15.86:0/2861179939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f041ac620 con 0x7f7f041095d0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.186+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 0x7f7f041a7290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4008000 tx=0x7f7ef4005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.186+0000 7f7efaffd640 1 -- 172.21.15.86:0/2861179939 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4040030 con 0x7f7f041095d0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.187+0000 7f7efaffd640 1 -- 172.21.15.86:0/2861179939 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef403c8f0 con 0x7f7f041095d0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.187+0000 7f7efaffd640 1 -- 172.21.15.86:0/2861179939 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031980 con 0x7f7f041095d0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.187+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041ac8b0 con 0x7f7f041095d0 2024-09-05T06:59:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.187+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f04112b00 con 0x7f7f041095d0 2024-09-05T06:59:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.191+0000 7f7f0df53740 1 Processor -- start 2024-09-05T06:59:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.191+0000 7f7f0df53740 1 -- start start 2024-09-05T06:59:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639beb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555639b512f0 con 0x555639beb780 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555639af8cf0 con 0x555639beccf0 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0df53740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555639b4d2b0 con 0x555639bf6520 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f09115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f09115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51514/0 (socket says 172.21.15.86:51514) 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f09115640 1 -- 172.21.15.86:0/1022050387 learned_addr learned my addr 172.21.15.86:0/1022050387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.192+0000 7f7f0a117640 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639beb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 -- 172.21.15.86:0/1022050387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 msgr2=0x555639bf8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 -- 172.21.15.86:0/1022050387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 msgr2=0x555639bebbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bebbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 -- 172.21.15.86:0/1022050387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555639add3a0 con 0x555639beccf0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7f0a117640 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639beb240 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ada0 tx=0x7f7efc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be4ba2c9b633646 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7efaffd640 1 -- 172.21.15.86:0/2861179939 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7ef4043030 con 0x7f7f041095d0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.193+0000 7f7eeffff640 1 -- 172.21.15.86:0/1022050387 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003b70 con 0x555639beccf0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.194+0000 7f7eeffff640 1 -- 172.21.15.86:0/1022050387 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7efc003d10 con 0x555639beccf0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.194+0000 7f7eeffff640 1 -- 172.21.15.86:0/1022050387 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0109c0 con 0x555639beccf0 2024-09-05T06:59:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.194+0000 7f7efaffd640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 0x7f7ef00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- 172.21.15.86:0/1022050387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 msgr2=0x555639beb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 0x7f7ef00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639beb240 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ada0 tx=0x7f7efc00f790 comp rx=0 tx=0).stop 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- 172.21.15.86:0/1022050387 shutdown_connections 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 --2- 172.21.15.86:0/1022050387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639beb240 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- 172.21.15.86:0/1022050387 >> 172.21.15.86:0/1022050387 conn(0x555639bb4310 msgr2=0x555639be7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- 172.21.15.86:0/1022050387 shutdown_connections 2024-09-05T06:59:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- 172.21.15.86:0/1022050387 wait complete. 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7efaffd640 1 -- 172.21.15.86:0/2861179939 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f7ef40efdf0 con 0x7f7f041095d0 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f09916640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 0x7f7ef00a5340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x555639be8c70 tx=0x7f7ef402a440 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.195+0000 7f7f0df53740 1 -- start start 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0df53740 1 Processor -- start 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639bf3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0a117640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0a117640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50918/0 (socket says 172.21.15.86:50918) 2024-09-05T06:59:53.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0a117640 1 -- 172.21.15.86:0/2508788108 learned_addr learned my addr 172.21.15.86:0/2508788108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.196+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f09115640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639bf3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555639bf92c0 con 0x555639beb780 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555639b0c060 con 0x555639beccf0 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555639bf9440 con 0x555639bf6520 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0df53740 10 client.0 _ll_register_callbacks cb 0x55563985f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 -- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 msgr2=0x555639bf36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 -- 172.21.15.86:0/2508788108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 msgr2=0x555639bf3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639bf3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f0a117640 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556398c23c0 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.197+0000 7f7f09115640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639bf3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7f0a117640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ab60 tx=0x7f7efc005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee68b9a481489dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7eed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc02c060 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f7efc0108c0 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7efc003d10 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7efc033050 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7f0df53740 10 client.19641 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.198+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555639c04cb0 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7efc018440 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7f7efc027430 con 0x555639beb780 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 10 client.19641 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 10 client.19641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 20 client.19641 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:53.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 4 client.19641 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:53.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7efc003d10 con 0x555639c060b0 2024-09-05T06:59:53.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.199+0000 7f7f09115640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.200+0000 7f7eed7fa640 10 client.19641 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:59:53.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.200+0000 7f7f09115640 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c084a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64737c7322128186 server_cookie=9be4e5d331c8987 in_seq=0 out_seq=0 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.201+0000 7f7eed7fa640 1 -- 172.21.15.86:0/2508788108 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7efc003d10 con 0x555639c060b0 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.201+0000 7f7eed7fa640 10 client.19641 handle_command_reply: tid=0 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.201+0000 7f7f0df53740 1 client.19641 shutdown 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.201+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 msgr2=0x555639bf5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.201+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ab60 tx=0x7f7efc005f60 comp rx=0 tx=0).stop 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 shutdown_connections 2024-09-05T06:59:53.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c084a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7eed7fa640 0 client.19641 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639bf6520 0x555639bf36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beccf0 0x555639bf3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 --2- 172.21.15.86:0/2508788108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beb780 0x555639bf5ea0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 >> 172.21.15.86:0/2508788108 conn(0x555639bb4310 msgr2=0x555639bedae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.202+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 shutdown_connections 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.203+0000 7f7f0df53740 1 -- 172.21.15.86:0/2508788108 wait complete. 2024-09-05T06:59:53.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.203+0000 7f7f0df53740 20 client.19641 trim_cache size 0 max 16384 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 Processor -- start 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 -- start start 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 0x555639bf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 0x555639bee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639bf6520 0x555639bee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c6b90 con 0x555639bf6520 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c8700 con 0x555639beb780 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.226+0000 7f7f0df53740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c7260 con 0x555639beccf0 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f09115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 0x555639bee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 0x555639bf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f09115640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 0x555639bee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51520/0 (socket says 172.21.15.86:51520) 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f09115640 1 -- 172.21.15.86:0/328988432 learned_addr learned my addr 172.21.15.86:0/328988432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639bf6520 0x555639bee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 -- 172.21.15.86:0/328988432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 msgr2=0x555639bee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 0x555639bee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 -- 172.21.15.86:0/328988432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639bf6520 msgr2=0x555639bee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639bf6520 0x555639bee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 -- 172.21.15.86:0/328988432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555639beed50 con 0x555639beb780 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7f0a117640 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 0x555639bf50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7efc002230 tx=0x7f7efc012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d01d12750a4192 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.227+0000 7f7eef7fe640 1 -- 172.21.15.86:0/328988432 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc02c070 con 0x555639beb780 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7eef7fe640 1 -- 172.21.15.86:0/328988432 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7efc00fc90 con 0x555639beb780 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7eef7fe640 1 -- 172.21.15.86:0/328988432 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc01d040 con 0x555639beb780 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 -- 172.21.15.86:0/328988432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 msgr2=0x555639bf50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 0x555639bf50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7efc002230 tx=0x7f7efc012040 comp rx=0 tx=0).stop 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 -- 172.21.15.86:0/328988432 shutdown_connections 2024-09-05T06:59:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639bf6520 0x555639bee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beccf0 0x555639bee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 --2- 172.21.15.86:0/328988432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639beb780 0x555639bf50c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 -- 172.21.15.86:0/328988432 >> 172.21.15.86:0/328988432 conn(0x555639bb4310 msgr2=0x555639bf49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.228+0000 7f7f0df53740 1 -- 172.21.15.86:0/328988432 shutdown_connections 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.229+0000 7f7f0df53740 1 -- 172.21.15.86:0/328988432 wait complete. 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.229+0000 7f7f0df53740 1 -- start start 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.229+0000 7f7f0df53740 1 Processor -- start 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 0x555639bfa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639bf6520 0x555639bfa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c8030 con 0x555639beccf0 2024-09-05T06:59:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c7960 con 0x555639bf6520 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51530/0 (socket says 172.21.15.86:51530) 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041c4b90 con 0x555639beb780 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 -- 172.21.15.86:0/771700147 learned_addr learned my addr 172.21.15.86:0/771700147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f09115640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 0x555639bfa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639bf6520 0x555639bfa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 -- 172.21.15.86:0/771700147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639bf6520 msgr2=0x555639bfa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639bf6520 0x555639bfa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 -- 172.21.15.86:0/771700147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 msgr2=0x555639bfa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 0x555639bfa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 -- 172.21.15.86:0/771700147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555639bfaf50 con 0x555639beb780 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f09115640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 0x555639bfa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0df53740 10 client.0 _ll_register_callbacks cb 0x7f7efc00ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7f0a117640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005e50 tx=0x7f7efc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714655a4bc29ed22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.230+0000 7f7eed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7efc00fc90 con 0x555639beb780 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7eed7fa640 1 -- 172.21.15.86:0/771700147 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc02c070 con 0x555639beb780 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7eed7fa640 1 -- 172.21.15.86:0/771700147 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f7efc004a00 con 0x555639beb780 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7eed7fa640 1 -- 172.21.15.86:0/771700147 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7efc004c00 con 0x555639beb780 2024-09-05T06:59:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 10 client.28486 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=44}) -- 0x555639c70820 con 0x555639beb780 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7eed7fa640 1 -- 172.21.15.86:0/771700147 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7f7efc010640 con 0x555639beb780 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 10 client.28486 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 10 client.28486 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 20 client.28486 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c72f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 4 client.28486 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.231+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555639b5e860 con 0x555639c060b0 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7f09115640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c72f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7f09115640 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c72f80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cf4ef1dd52b30ae server_cookie=bb2a753e5447acbe in_seq=0 out_seq=0 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7eed7fa640 10 client.28486 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7eed7fa640 1 -- 172.21.15.86:0/771700147 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555639b5e860 con 0x555639c060b0 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7eed7fa640 10 client.28486 handle_command_reply: tid=0 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.232+0000 7f7f0df53740 1 client.28486 shutdown 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 msgr2=0x555639bfee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005e50 tx=0x7f7efc000f40 comp rx=0 tx=0).stop 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 shutdown_connections 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x555639c060b0 0x555639c72f80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555639bf6520 0x555639bfa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555639beccf0 0x555639bfa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 --2- 172.21.15.86:0/771700147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555639beb780 0x555639bfee90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 >> 172.21.15.86:0/771700147 conn(0x555639bb4310 msgr2=0x555639be6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7eed7fa640 0 client.28486 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 shutdown_connections 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 1 -- 172.21.15.86:0/771700147 wait complete. 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.233+0000 7f7f0df53740 20 client.28486 trim_cache size 0 max 16384 2024-09-05T06:59:53.239 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:59:53.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 msgr2=0x7f7ef00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 0x7f7ef00a5340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x555639be8c70 tx=0x7f7ef402a440 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 msgr2=0x7f7f041a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 0x7f7f041a7290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4008000 tx=0x7f7ef4005360 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 shutdown_connections 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7ef00a2e80 0x7f7ef00a5340 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f041abb60 0x7f7f041ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f0411b140 0x7f7f041a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 --2- 172.21.15.86:0/2861179939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f041095d0 0x7f7f041a7290 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.234+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 >> 172.21.15.86:0/2861179939 conn(0x7f7f0406e970 msgr2=0x7f7f0411bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.235+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 shutdown_connections 2024-09-05T06:59:53.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:53.235+0000 7f7f0bb7a640 1 -- 172.21.15.86:0/2861179939 wait complete. 2024-09-05T06:59:53.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37941 6.39 MB/sec execute 191 sec latency 111.537 ms 2024-09-05T06:59:53.964 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:59:53.964 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:59:53.964 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:53 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:59:54.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T06:59:54.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:59:54.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:59:54.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:53 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:59:54.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T06:59:54.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T06:59:54.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:53 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T06:59:54.356 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:54.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38126 6.45 MB/sec execute 192 sec latency 88.084 ms 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 -- 172.21.15.86:0/2065230942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c100a80 msgr2=0x7f5b0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/2065230942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c100a80 0x7f5b0c100ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009890 tx=0x7f5b0002a380 comp rx=0 tx=0).stop 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 -- 172.21.15.86:0/2065230942 shutdown_connections 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/2065230942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c101420 0x7f5b0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/2065230942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c100a80 0x7f5b0c100ee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/2065230942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c106a80 0x7f5b0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.905+0000 7f5b11c0f640 1 -- 172.21.15.86:0/2065230942 >> 172.21.15.86:0/2065230942 conn(0x7f5b0c0fc7a0 msgr2=0x7f5b0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.906+0000 7f5b11c0f640 1 -- 172.21.15.86:0/2065230942 shutdown_connections 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.906+0000 7f5b11c0f640 1 -- 172.21.15.86:0/2065230942 wait complete. 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.906+0000 7f5b11c0f640 1 Processor -- start 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b11c0f640 1 -- start start 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b11c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b11c0f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 0x7f5b0c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b11c0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50960/0 (socket says 172.21.15.86:50960) 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 0x7f5b0c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 0x7f5b0c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51544/0 (socket says 172.21.15.86:51544) 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.907+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37576/0 (socket says 172.21.15.86:37576) 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b11c0f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110cf0 con 0x7f5b0c100a80 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0b7fe640 1 -- 172.21.15.86:0/1677203546 learned_addr learned my addr 172.21.15.86:0/1677203546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110b70 con 0x7f5b0c106a80 2024-09-05T06:59:54.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110e70 con 0x7f5b0c101420 2024-09-05T06:59:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0bfff640 1 -- 172.21.15.86:0/1677203546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 msgr2=0x7f5b0c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 0x7f5b0c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0bfff640 1 -- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 msgr2=0x7f5b0c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0bfff640 1 -- 172.21.15.86:0/1677203546 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1a47d0 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.908+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009fd0 tx=0x7f5b0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b08ff9640 1 -- 172.21.15.86:0/1677203546 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400ce20 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b08ff9640 1 -- 172.21.15.86:0/1677203546 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b04005020 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b08ff9640 1 -- 172.21.15.86:0/1677203546 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04010690 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c1a49a0 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.909+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c1a4e00 con 0x7f5b0c106a80 2024-09-05T06:59:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.911+0000 7f5b08ff9640 1 -- 172.21.15.86:0/1677203546 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b04004b20 con 0x7f5b0c106a80 2024-09-05T06:59:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.912+0000 7f5b13fe8740 1 Processor -- start 2024-09-05T06:59:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.912+0000 7f5b13fe8740 1 -- start start 2024-09-05T06:59:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.912+0000 7f5b08ff9640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 0x7f5af40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 0x555f00140a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b08ff9640 1 -- 172.21.15.86:0/1677203546 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5b040cff00 con 0x7f5b0c106a80 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f0014d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f000a62f0 con 0x555f001405d0 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f0004dcf0 con 0x555f00141b40 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f000a22b0 con 0x555f0014b370 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37578/0 (socket says 172.21.15.86:37578) 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 -- 172.21.15.86:0/1437895301 learned_addr learned my addr 172.21.15.86:0/1437895301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 0x555f00140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0affd640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 0x7f5af40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f0014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 -- 172.21.15.86:0/1437895301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 msgr2=0x555f0014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f0014d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 -- 172.21.15.86:0/1437895301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 msgr2=0x555f00140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 0x555f00140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0b7fe640 1 -- 172.21.15.86:0/1437895301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f000323a0 con 0x555f00141b40 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.913+0000 7f5b0affd640 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 0x555f00140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.914+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5afc011540 tx=0x7f5afc011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15eef51f1e82101c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.914+0000 7f5b0affd640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 0x7f5af40a5230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b0002a970 tx=0x7f5b00030340 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.914+0000 7f5af1ffb640 1 -- 172.21.15.86:0/1437895301 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00f640 con 0x555f00141b40 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.914+0000 7f5af1ffb640 1 -- 172.21.15.86:0/1437895301 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5afc00fc90 con 0x555f00141b40 2024-09-05T06:59:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.914+0000 7f5af1ffb640 1 -- 172.21.15.86:0/1437895301 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc019e60 con 0x555f00141b40 2024-09-05T06:59:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- 172.21.15.86:0/1437895301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 msgr2=0x555f00140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5afc011540 tx=0x7f5afc011a00 comp rx=0 tx=0).stop 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- 172.21.15.86:0/1437895301 shutdown_connections 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f0014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f001405d0 0x555f00140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/1437895301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00140090 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- 172.21.15.86:0/1437895301 >> 172.21.15.86:0/1437895301 conn(0x555f00109310 msgr2=0x555f0013cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- 172.21.15.86:0/1437895301 shutdown_connections 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- 172.21.15.86:0/1437895301 wait complete. 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.915+0000 7f5b13fe8740 1 -- start start 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 Processor -- start 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f001405d0 0x555f0014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00148000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f0014e070 con 0x555f0014b370 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f00061060 con 0x555f00141b40 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f0014e1f0 con 0x555f001405d0 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b13fe8740 10 client.0 _ll_register_callbacks cb 0x555effdb48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50976/0 (socket says 172.21.15.86:50976) 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b0bfff640 1 -- 172.21.15.86:0/3042560576 learned_addr learned my addr 172.21.15.86:0/3042560576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.916+0000 7f5b0affd640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f001405d0 0x555f0014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 -- 172.21.15.86:0/3042560576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f001405d0 msgr2=0x555f0014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f001405d0 0x555f0014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 -- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 msgr2=0x555f00148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 -- 172.21.15.86:0/3042560576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555effe173c0 con 0x555f0014b370 2024-09-05T06:59:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0affd640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5b040cd810 tx=0x7f5b0409c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92954bbcc4ebfd89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040b4030 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5b04005540 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5afc00fc90 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5b04082050 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b13fe8740 10 client.19659 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.917+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f00159b50 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5afc00fc90 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7f5b040a47c0 con 0x555f0014b370 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 10 client.19659 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 20 client.19659 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f0015b010 0x555f0015d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 4 client.19659 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f000b3860 con 0x555f0015b010 2024-09-05T06:59:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.918+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f0015b010 0x555f0015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.919+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f0015b010 0x555f0015d480 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f6c5ead69cce16 server_cookie=50cc95290b787254 in_seq=0 out_seq=0 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.919+0000 7f5aeaffd640 10 client.19659 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.919+0000 7f5aeaffd640 1 -- 172.21.15.86:0/3042560576 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f000b3860 con 0x555f0015b010 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.919+0000 7f5aeaffd640 10 client.19659 handle_command_reply: tid=0 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.919+0000 7f5b13fe8740 1 client.19659 shutdown 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 msgr2=0x555f001485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5b040cd810 tx=0x7f5b0409c7a0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 shutdown_connections 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f0015b010 0x555f0015d480 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f001485d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f00141b40 0x555f00148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/3042560576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f001405d0 0x555f0014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5aeaffd640 0 client.19659 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 >> 172.21.15.86:0/3042560576 conn(0x555f00109310 msgr2=0x555f00142930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 shutdown_connections 2024-09-05T06:59:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 1 -- 172.21.15.86:0/3042560576 wait complete. 2024-09-05T06:59:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.920+0000 7f5b13fe8740 20 client.19659 trim_cache size 0 max 16384 2024-09-05T06:59:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 Processor -- start 2024-09-05T06:59:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 -- start start 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f00157310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f00141b40 0x555f001468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1af170 con 0x555f0014b370 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1b0ce0 con 0x555f001405d0 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1af840 con 0x555f00141b40 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50986/0 (socket says 172.21.15.86:50986) 2024-09-05T06:59:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.941+0000 7f5b0bfff640 1 -- 172.21.15.86:0/2227259726 learned_addr learned my addr 172.21.15.86:0/2227259726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:54.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 -- 172.21.15.86:0/2227259726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f00141b40 msgr2=0x555f001468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f00141b40 0x555f001468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 -- 172.21.15.86:0/2227259726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 msgr2=0x555f00157310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f00157310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 -- 172.21.15.86:0/2227259726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f0013c4f0 con 0x555f0014b370 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5b0bfff640 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5b0409bf70 tx=0x7f5b0409a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b485b29a9946a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.942+0000 7f5af17fa640 1 -- 172.21.15.86:0/2227259726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040b4070 con 0x555f0014b370 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.943+0000 7f5af17fa640 1 -- 172.21.15.86:0/2227259726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b040a4c50 con 0x555f0014b370 2024-09-05T06:59:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.943+0000 7f5af17fa640 1 -- 172.21.15.86:0/2227259726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040a47c0 con 0x555f0014b370 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- 172.21.15.86:0/2227259726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 msgr2=0x555f00146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5b0409bf70 tx=0x7f5b0409a4f0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- 172.21.15.86:0/2227259726 shutdown_connections 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f0014b370 0x555f00146e10 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f00141b40 0x555f001468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/2227259726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f00157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- 172.21.15.86:0/2227259726 >> 172.21.15.86:0/2227259726 conn(0x555f00109310 msgr2=0x555f00147fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- 172.21.15.86:0/2227259726 shutdown_connections 2024-09-05T06:59:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- 172.21.15.86:0/2227259726 wait complete. 2024-09-05T06:59:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.944+0000 7f5b13fe8740 1 -- start start 2024-09-05T06:59:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.945+0000 7f5b13fe8740 1 Processor -- start 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f001c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f001539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1b0610 con 0x555f00141b40 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1aff40 con 0x555f001405d0 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ad170 con 0x555f0014b370 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.946+0000 7f5b13fe8740 10 client.0 _ll_register_callbacks cb 0x7f5b04098280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50994/0 (socket says 172.21.15.86:50994) 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 -- 172.21.15.86:0/491551295 learned_addr learned my addr 172.21.15.86:0/491551295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 -- 172.21.15.86:0/491551295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 msgr2=0x555f001539c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f001539c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 -- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 msgr2=0x555f001c17f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f001c17f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 -- 172.21.15.86:0/491551295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f00153fb0 con 0x555f00141b40 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.947+0000 7f5b0affd640 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009890 tx=0x7f5b00005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6badc1b48a5b125f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0004f070 con 0x555f00141b40 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f5b00003d40 con 0x555f00141b40 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b040a4c50 con 0x555f00141b40 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5b00059050 con 0x555f00141b40 2024-09-05T06:59:54.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5b13fe8740 10 client.19665 fetch_fsmap learned FSMap version 44 2024-09-05T06:59:54.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.948+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f001ce940 con 0x555f00141b40 2024-09-05T06:59:54.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b040a4c50 con 0x555f00141b40 2024-09-05T06:59:54.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8374+0+0 (secure 0 0 0) 0x7f5b0004a040 con 0x555f00141b40 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 10 client.19665 fetch_fsmap finished waiting for FSMap version 44 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 10 client.19665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 20 client.19665 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f001cf110 0x555f001d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 4 client.19665 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.949+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f000add70 con 0x555f001cf110 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.950+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f001cf110 0x555f001d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.951+0000 7f5b0b7fe640 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f001cf110 0x555f001d15d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d57a82af66655d5 server_cookie=706db6e9a4521790 in_seq=0 out_seq=0 2024-09-05T06:59:54.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.951+0000 7f5aeaffd640 10 client.19665 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.954+0000 7f5aeaffd640 1 -- 172.21.15.86:0/491551295 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f000add70 con 0x555f001cf110 2024-09-05T06:59:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.954+0000 7f5aeaffd640 10 client.19665 handle_command_reply: tid=0 2024-09-05T06:59:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.954+0000 7f5b13fe8740 1 client.19665 shutdown 2024-09-05T06:59:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.954+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 msgr2=0x555f00153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.954+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009890 tx=0x7f5b00005ad0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 shutdown_connections 2024-09-05T06:59:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f001cf110 0x555f001d15d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f0014b370 0x555f001539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f00141b40 0x555f00153480 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 --2- 172.21.15.86:0/491551295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f001405d0 0x555f001c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 >> 172.21.15.86:0/491551295 conn(0x555f00109310 msgr2=0x555f00146890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5aeaffd640 0 client.19665 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 shutdown_connections 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.955+0000 7f5b13fe8740 1 -- 172.21.15.86:0/491551295 wait complete. 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.956+0000 7f5b13fe8740 20 client.19665 trim_cache size 0 max 16384 2024-09-05T06:59:54.957 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 msgr2=0x7f5af40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 0x7f5af40a5230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b0002a970 tx=0x7f5b00030340 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 msgr2=0x7f5b0c1a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009fd0 tx=0x7f5b0400ef10 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 shutdown_connections 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af40a2d70 0x7f5af40a5230 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b0c106a80 0x7f5b0c1a4290 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0c101420 0x7f5b0c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 --2- 172.21.15.86:0/1677203546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b0c100a80 0x7f5b0c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 >> 172.21.15.86:0/1677203546 conn(0x7f5b0c0fc7a0 msgr2=0x7f5b0c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 shutdown_connections 2024-09-05T06:59:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:54.957+0000 7f5b11c0f640 1 -- 172.21.15.86:0/1677203546 wait complete. 2024-09-05T06:59:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:55 smithi115 ceph-mon[34791]: pgmap v586: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:55.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:55 smithi086 ceph-mon[30943]: pgmap v586: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:55.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38289 6.46 MB/sec execute 193 sec latency 83.939 ms 2024-09-05T06:59:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:55 smithi096 ceph-mon[34468]: pgmap v586: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:56.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:56 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:56.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:56 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:56.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38408 6.46 MB/sec execute 194 sec latency 515.873 ms 2024-09-05T06:59:56.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:56 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T06:59:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:57 smithi115 ceph-mon[34791]: pgmap v587: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:57 smithi086 ceph-mon[30943]: pgmap v587: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:57.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38568 6.46 MB/sec execute 195 sec latency 107.708 ms 2024-09-05T06:59:57.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:57 smithi096 ceph-mon[34468]: pgmap v587: 129 pgs: 129 active+clean; 950 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 215 op/s 2024-09-05T06:59:58.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T06:59:58.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T06:59:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T06:59:58.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38719 6.45 MB/sec execute 196 sec latency 97.552 ms 2024-09-05T06:59:59.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 06:59:59 smithi086 ceph-mon[30943]: pgmap v588: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 322 op/s 2024-09-05T06:59:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 -- 172.21.15.86:0/3712988080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0106a80 msgr2=0x7f01f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 --2- 172.21.15.86:0/3712988080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0106a80 0x7f01f0106e60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a3f0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 -- 172.21.15.86:0/3712988080 shutdown_connections 2024-09-05T06:59:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 --2- 172.21.15.86:0/3712988080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0101420 0x7f01f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 --2- 172.21.15.86:0/3712988080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0100a80 0x7f01f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 --2- 172.21.15.86:0/3712988080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0106a80 0x7f01f0106e60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 -- 172.21.15.86:0/3712988080 >> 172.21.15.86:0/3712988080 conn(0x7f01f00fc7a0 msgr2=0x7f01f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 -- 172.21.15.86:0/3712988080 shutdown_connections 2024-09-05T06:59:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.681+0000 7f01f5b34640 1 -- 172.21.15.86:0/3712988080 wait complete. 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 Processor -- start 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 -- start start 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 0x7f01f019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 0x7f01f019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fbe0 con 0x7f01f0101420 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fa60 con 0x7f01f0100a80 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01f5b34640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010fd60 con 0x7f01f0106a80 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01ef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01ef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37626/0 (socket says 172.21.15.86:37626) 2024-09-05T06:59:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01ef7fe640 1 -- 172.21.15.86:0/489415486 learned_addr learned my addr 172.21.15.86:0/489415486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01eeffd640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 0x7f01f019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.682+0000 7f01effff640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 0x7f01f019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 -- 172.21.15.86:0/489415486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 msgr2=0x7f01f019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 0x7f01f019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 -- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 msgr2=0x7f01f019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 0x7f01f019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 -- 172.21.15.86:0/489415486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f019f500 con 0x7f01f0100a80 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01effff640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 0x7f01f019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01eeffd640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 0x7f01f019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01e0006010 tx=0x7f01e00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ecff9640 1 -- 172.21.15.86:0/489415486 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e003f030 con 0x7f01f0100a80 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f019f790 con 0x7f01f0100a80 2024-09-05T06:59:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a70d0 con 0x7f01f0100a80 2024-09-05T06:59:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ecff9640 1 -- 172.21.15.86:0/489415486 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e00044c0 con 0x7f01f0100a80 2024-09-05T06:59:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.683+0000 7f01ecff9640 1 -- 172.21.15.86:0/489415486 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0035070 con 0x7f01f0100a80 2024-09-05T06:59:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 06:59:59 smithi115 ceph-mon[34791]: pgmap v588: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 322 op/s 2024-09-05T06:59:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.686+0000 7f01ecff9640 1 -- 172.21.15.86:0/489415486 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f01e0042030 con 0x7f01f0100a80 2024-09-05T06:59:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 Processor -- start 2024-09-05T06:59:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 -- start start 2024-09-05T06:59:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e1d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c02d78280 con 0x564c02e13480 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c02d1fb10 con 0x564c02e1dd40 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c02d74240 con 0x564c02e12b60 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.687+0000 7f01ecff9640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 0x7f01d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51588/0 (socket says 172.21.15.86:51588) 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 -- 172.21.15.86:0/2896394294 learned_addr learned my addr 172.21.15.86:0/2896394294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01eeffd640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e1d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 0x7f01d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 -- 172.21.15.86:0/2896394294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 msgr2=0x564c02e20130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e20130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 -- 172.21.15.86:0/2896394294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 msgr2=0x564c02e1d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e1d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 -- 172.21.15.86:0/2896394294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c02d2c1f0 con 0x564c02e12b60 2024-09-05T06:59:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e1d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01ecff9640 1 -- 172.21.15.86:0/489415486 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f01e00eff20 con 0x7f01f0100a80 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01e8012a80 tx=0x7f01e8012f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c5ed4a93a40f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01effff640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 0x7f01d80a5230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f01e801d000 tx=0x7f01e801f000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01d5ffb640 1 -- 172.21.15.86:0/2896394294 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8010cb0 con 0x564c02e12b60 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01d5ffb640 1 -- 172.21.15.86:0/2896394294 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e8022070 con 0x564c02e12b60 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.688+0000 7f01d5ffb640 1 -- 172.21.15.86:0/2896394294 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8011c00 con 0x564c02e12b60 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 -- 172.21.15.86:0/2896394294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 msgr2=0x564c02e12f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01e8012a80 tx=0x7f01e8012f40 comp rx=0 tx=0).stop 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 -- 172.21.15.86:0/2896394294 shutdown_connections 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e20130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e1d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/2896394294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e12f40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 -- 172.21.15.86:0/2896394294 >> 172.21.15.86:0/2896394294 conn(0x564c02e0f0f0 msgr2=0x564c02e0f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 -- 172.21.15.86:0/2896394294 shutdown_connections 2024-09-05T06:59:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.689+0000 7f01f7f0d740 1 -- 172.21.15.86:0/2896394294 wait complete. 2024-09-05T06:59:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 -- start start 2024-09-05T06:59:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 Processor -- start 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e1abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e1b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c02e20910 con 0x564c02e1dd40 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c02d32f10 con 0x564c02e12b60 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c02e20a90 con 0x564c02e13480 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01f7f0d740 10 client.0 _ll_register_callbacks cb 0x564c02cfba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01eeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.690+0000 7f01effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e1b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51598/0 (socket says 172.21.15.86:51598) 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 -- 172.21.15.86:0/976911422 learned_addr learned my addr 172.21.15.86:0/976911422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e1abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 -- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 msgr2=0x564c02e1abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e1abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 -- 172.21.15.86:0/976911422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 msgr2=0x564c02e1b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e1b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 -- 172.21.15.86:0/976911422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c02ae96a0 con 0x564c02e13480 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e1abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01eeffd640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c80 tx=0x7f01e40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a183abf51110c6af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4012860 con 0x564c02e13480 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f01e4012a00 con 0x564c02e13480 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e8022070 con 0x564c02e13480 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.691+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f01e4025050 con 0x564c02e13480 2024-09-05T06:59:59.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 10 client.28510 fetch_fsmap learned FSMap version 45 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564c02e34d60 con 0x564c02e13480 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e4012c00 con 0x564c02e13480 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f01e401a020 con 0x564c02e13480 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 10 client.28510 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 10 client.28510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 20 client.28510 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02e36280 0x564c02e38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 4 client.28510 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f01e8022070 con 0x564c02e36280 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.692+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02e36280 0x564c02e38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.693+0000 7f01ceffd640 10 client.28510 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.693+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02e36280 0x564c02e38670 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87126d9379687d3b server_cookie=db8e3be43fc6af40 in_seq=0 out_seq=0 2024-09-05T06:59:59.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.693+0000 7f01ceffd640 1 -- 172.21.15.86:0/976911422 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f01e8022070 con 0x564c02e36280 2024-09-05T06:59:59.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.693+0000 7f01ceffd640 10 client.28510 handle_command_reply: tid=0 2024-09-05T06:59:59.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.693+0000 7f01f7f0d740 1 client.28510 shutdown 2024-09-05T06:59:59.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 msgr2=0x564c02e1b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c80 tx=0x7f01e40039c0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 shutdown_connections 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02e36280 0x564c02e38670 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e1b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e1b100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/976911422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e1abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 >> 172.21.15.86:0/976911422 conn(0x564c02e0f0f0 msgr2=0x564c02e0e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01ceffd640 0 client.28510 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 shutdown_connections 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 1 -- 172.21.15.86:0/976911422 wait complete. 2024-09-05T06:59:59.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.694+0000 7f01f7f0d740 20 client.28510 trim_cache size 0 max 16384 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 Processor -- start 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 -- start start 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e19a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e3df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0100e0 con 0x564c02e1dd40 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc011c20 con 0x564c02e12b60 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.720+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0107b0 con 0x564c02e13480 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37658/0 (socket says 172.21.15.86:37658) 2024-09-05T06:59:59.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01eeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e19a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 -- 172.21.15.86:0/521164514 learned_addr learned my addr 172.21.15.86:0/521164514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 -- 172.21.15.86:0/521164514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 msgr2=0x564c02e19a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e19a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 -- 172.21.15.86:0/521164514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 msgr2=0x564c02e3df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e3df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 -- 172.21.15.86:0/521164514 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c02e1d490 con 0x564c02e12b60 2024-09-05T06:59:59.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01eeffd640 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e19a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.721+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ed830 tx=0x7f01e00b7c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad577a79df20957 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.722+0000 7f01d57fa640 1 -- 172.21.15.86:0/521164514 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00dc0b0 con 0x564c02e12b60 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.722+0000 7f01d57fa640 1 -- 172.21.15.86:0/521164514 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e00ba300 con 0x564c02e12b60 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.722+0000 7f01d57fa640 1 -- 172.21.15.86:0/521164514 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0096770 con 0x564c02e12b60 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- 172.21.15.86:0/521164514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 msgr2=0x564c02e15610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ed830 tx=0x7f01e00b7c30 comp rx=0 tx=0).stop 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- 172.21.15.86:0/521164514 shutdown_connections 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e1dd40 0x564c02e3df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e13480 0x564c02e19a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/521164514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e12b60 0x564c02e15610 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- 172.21.15.86:0/521164514 >> 172.21.15.86:0/521164514 conn(0x564c02e0f0f0 msgr2=0x564c02e19360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- 172.21.15.86:0/521164514 shutdown_connections 2024-09-05T06:59:59.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- 172.21.15.86:0/521164514 wait complete. 2024-09-05T06:59:59.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.723+0000 7f01f7f0d740 1 -- start start 2024-09-05T06:59:59.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 Processor -- start 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e2e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e2b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc011550 con 0x564c02e13480 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc010e80 con 0x564c02e1dd40 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.724+0000 7f01f7f0d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc00e0e0 con 0x564c02e12b60 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51060/0 (socket says 172.21.15.86:51060) 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 -- 172.21.15.86:0/3783828791 learned_addr learned my addr 172.21.15.86:0/3783828791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e2e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01f7f0d740 10 client.0 _ll_register_callbacks cb 0x7f01e4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 -- 172.21.15.86:0/3783828791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 msgr2=0x564c02e2e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e2e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 -- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 msgr2=0x564c02e2b5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e2b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 -- 172.21.15.86:0/3783828791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c02e2bbe0 con 0x564c02e13480 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01eeffd640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x564c02e1a450 tx=0x7f01e4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b0c523d1fca8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T06:59:59.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.725+0000 7f01ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T06:59:59.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011da0 con 0x564c02e13480 2024-09-05T06:59:59.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7f01e4005030 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e00ba300 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f01e4005270 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01f7f0d740 10 client.19680 fetch_fsmap learned FSMap version 45 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564c02ea18f0 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01e402d020 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.726+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f01e403b080 con 0x564c02e13480 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 10 client.19680 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 10 client.19680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 20 client.19680 populate_metadata read hostname 'smithi086' 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02ea20a0 0x564c02ea44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 4 client.19680 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c02d857f0 con 0x564c02ea20a0 2024-09-05T06:59:59.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02ea20a0 0x564c02ea44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01ceffd640 10 client.19680 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.727+0000 7f01ef7fe640 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02ea20a0 0x564c02ea44c0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41a61228f5887ff server_cookie=5db8ae1c187c5079 in_seq=0 out_seq=0 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.728+0000 7f01ceffd640 1 -- 172.21.15.86:0/3783828791 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c02d857f0 con 0x564c02ea20a0 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.728+0000 7f01ceffd640 10 client.19680 handle_command_reply: tid=0 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.728+0000 7f01f7f0d740 1 client.19680 shutdown 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.728+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 msgr2=0x564c02e2eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.728+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x564c02e1a450 tx=0x7f01e4011940 comp rx=0 tx=0).stop 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 shutdown_connections 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x564c02ea20a0 0x564c02ea44c0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c02e1dd40 0x564c02e2b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c02e13480 0x564c02e2eb10 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 --2- 172.21.15.86:0/3783828791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c02e12b60 0x564c02e2e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01ceffd640 0 client.19680 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 >> 172.21.15.86:0/3783828791 conn(0x564c02e0f0f0 msgr2=0x564c02e19b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 shutdown_connections 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 1 -- 172.21.15.86:0/3783828791 wait complete. 2024-09-05T06:59:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.729+0000 7f01f7f0d740 20 client.19680 trim_cache size 0 max 16384 2024-09-05T06:59:59.732 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T06:59:59.732 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T06:59:59.732 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "81c9f59b-883f-4d52-ae87-4c4681be01ad", 2024-09-05T06:59:59.732 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T06:59:59.732 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.730+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 msgr2=0x7f01d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.730+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 0x7f01d80a5230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f01e801d000 tx=0x7f01e801f000 comp rx=0 tx=0).stop 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.730+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 msgr2=0x7f01f019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.730+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01e0006010 tx=0x7f01e00040c0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 shutdown_connections 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01d80a2d70 0x7f01d80a5230 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f0106a80 0x7f01f019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0101420 0x7f01f019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 --2- 172.21.15.86:0/489415486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0100a80 0x7f01f019a550 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T06:59:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 >> 172.21.15.86:0/489415486 conn(0x7f01f00fc7a0 msgr2=0x7f01f00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T06:59:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 shutdown_connections 2024-09-05T06:59:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T06:59:59.731+0000 7f01f5b34640 1 -- 172.21.15.86:0/489415486 wait complete. 2024-09-05T06:59:59.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38861 6.44 MB/sec execute 197 sec latency 83.139 ms 2024-09-05T06:59:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 06:59:59 smithi096 ceph-mon[34468]: pgmap v588: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 322 op/s 2024-09-05T07:00:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:00:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:00:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:00.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:00 smithi115 ceph-mon[34791]: overall HEALTH_OK 2024-09-05T07:00:00.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:00:00.724 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:00:00.724 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:00.724 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:00 smithi086 ceph-mon[30943]: overall HEALTH_OK 2024-09-05T07:00:00.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39008 6.44 MB/sec execute 198 sec latency 196.526 ms 2024-09-05T07:00:00.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:00:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:00:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:00 smithi096 ceph-mon[34468]: overall HEALTH_OK 2024-09-05T07:00:01.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:00:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:01.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: pgmap v589: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:01 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: pgmap v589: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-09-05T07:00:01.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:01 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: pgmap v589: 129 pgs: 129 active+clean; 957 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 215 op/s 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:01 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:00:01.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39179 6.47 MB/sec execute 199 sec latency 122.857 ms 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 -- 172.21.15.86:0/1480612221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100100 msgr2=0x7efd58100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 --2- 172.21.15.86:0/1480612221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100100 0x7efd58100560 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7efd48009900 tx=0x7efd4802a3f0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 -- 172.21.15.86:0/1480612221 shutdown_connections 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 --2- 172.21.15.86:0/1480612221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100aa0 0x7efd5810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 --2- 172.21.15.86:0/1480612221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100100 0x7efd58100560 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 --2- 172.21.15.86:0/1480612221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd58106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 -- 172.21.15.86:0/1480612221 >> 172.21.15.86:0/1480612221 conn(0x7efd58077230 msgr2=0x7efd580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 -- 172.21.15.86:0/1480612221 shutdown_connections 2024-09-05T07:00:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.236+0000 7efd6062a640 1 -- 172.21.15.86:0/1480612221 wait complete. 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.237+0000 7efd6062a640 1 Processor -- start 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.237+0000 7efd6062a640 1 -- start start 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.237+0000 7efd6062a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.237+0000 7efd6062a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100aa0 0x7efd581966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd6062a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd5819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd6062a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd58102630 con 0x7efd58100aa0 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd6062a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd581024b0 con 0x7efd58100100 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd6062a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd581027b0 con 0x7efd58106b80 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37674/0 (socket says 172.21.15.86:37674) 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3208685990 learned_addr learned my addr 172.21.15.86:0/3208685990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100aa0 0x7efd581966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd5819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3208685990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 msgr2=0x7efd5819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd5819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100aa0 msgr2=0x7efd581966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100aa0 0x7efd581966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3208685990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5819b140 con 0x7efd58100100 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd5819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e9f0 tx=0x7efd5400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd4f7fe640 1 -- 172.21.15.86:0/3208685990 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5400cd80 con 0x7efd58100100 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.238+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5819b430 con 0x7efd58100100 2024-09-05T07:00:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.239+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581a2d10 con 0x7efd58100100 2024-09-05T07:00:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.239+0000 7efd4f7fe640 1 -- 172.21.15.86:0/3208685990 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd540054d0 con 0x7efd58100100 2024-09-05T07:00:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.239+0000 7efd4f7fe640 1 -- 172.21.15.86:0/3208685990 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54010490 con 0x7efd58100100 2024-09-05T07:00:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.241+0000 7efd4f7fe640 1 -- 172.21.15.86:0/3208685990 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd54005020 con 0x7efd58100100 2024-09-05T07:00:02.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 Processor -- start 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 -- start start 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd4f7fe640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 0x7efd440a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 0x5622b277b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 0x5622b278e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 0x7efd440a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622b26e12e0 con 0x5622b278ee90 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5622b2688ce0 con 0x5622b277b3c0 2024-09-05T07:00:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.242+0000 7efd62a03740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5622b26dd2a0 con 0x5622b277bce0 2024-09-05T07:00:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5ebc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5ebc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51086/0 (socket says 172.21.15.86:51086) 2024-09-05T07:00:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5ebc7640 1 -- 172.21.15.86:0/4031651736 learned_addr learned my addr 172.21.15.86:0/4031651736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 0x5622b278e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd4f7fe640 1 -- 172.21.15.86:0/3208685990 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7efd540cfc90 con 0x7efd58100100 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 0x5622b277b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4031651736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 msgr2=0x5622b277b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 0x5622b277b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4031651736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 msgr2=0x5622b2791280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4031651736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b266d440 con 0x5622b277bce0 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 0x5622b277b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.243+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 0x5622b278e950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd4802a9e0 tx=0x7efd48005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90e8f6fd19e3f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.244+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 0x7efd440a9640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd50009940 tx=0x7efd50009f50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.244+0000 7efd4cff9640 1 -- 172.21.15.86:0/4031651736 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035040 con 0x5622b277bce0 2024-09-05T07:00:02.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.244+0000 7efd4cff9640 1 -- 172.21.15.86:0/4031651736 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd480054f0 con 0x5622b277bce0 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.244+0000 7efd4cff9640 1 -- 172.21.15.86:0/4031651736 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd480035f0 con 0x5622b277bce0 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 -- 172.21.15.86:0/4031651736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 msgr2=0x5622b278e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 0x5622b278e950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd4802a9e0 tx=0x7efd48005c00 comp rx=0 tx=0).stop 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 -- 172.21.15.86:0/4031651736 shutdown_connections 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2791280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277bce0 0x5622b278e950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 --2- 172.21.15.86:0/4031651736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277b3c0 0x5622b277b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 -- 172.21.15.86:0/4031651736 >> 172.21.15.86:0/4031651736 conn(0x5622b2777a30 msgr2=0x5622b2777e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 -- 172.21.15.86:0/4031651736 shutdown_connections 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.245+0000 7efd62a03740 1 -- 172.21.15.86:0/4031651736 wait complete. 2024-09-05T07:00:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd62a03740 1 -- start start 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd62a03740 1 Processor -- start 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd62a03740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd62a03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51668/0 (socket says 172.21.15.86:51668) 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd62a03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b278c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.246+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37686/0 (socket says 172.21.15.86:37686) 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5e3c6640 1 -- 172.21.15.86:0/4133728853 learned_addr learned my addr 172.21.15.86:0/4133728853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622b27919a0 con 0x5622b278ee90 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5622b269c050 con 0x5622b277bce0 2024-09-05T07:00:02.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5622b2791b20 con 0x5622b277b3c0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b278c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd62a03740 10 client.0 _ll_register_callbacks cb 0x5622b26dc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4133728853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 msgr2=0x5622b278e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4133728853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 msgr2=0x5622b278c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b278c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b24523c0 con 0x5622b277bce0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd48005ab0 tx=0x7efd48000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11932db9e4f1b972 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.247+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd480054f0 con 0x5622b277bce0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd41ffb640 1 -- 172.21.15.86:0/4133728853 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48053060 con 0x5622b277bce0 2024-09-05T07:00:02.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd41ffb640 1 -- 172.21.15.86:0/4133728853 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7efd48004ba0 con 0x5622b277bce0 2024-09-05T07:00:02.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd41ffb640 1 -- 172.21.15.86:0/4133728853 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efd48035050 con 0x5622b277bce0 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 10 client.28487 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5622b279d1e0 con 0x5622b277bce0 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd41ffb640 1 -- 172.21.15.86:0/4133728853 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7efd48004da0 con 0x5622b277bce0 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 10 client.28487 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 10 client.28487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 20 client.28487 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b279e530 0x5622b27a0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 4 client.28487 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.248+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd480054f0 con 0x5622b279e530 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.249+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b279e530 0x5622b27a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.249+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b279e530 0x5622b27a0920 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bae25459d6afce0 server_cookie=752695069b6b27ca in_seq=0 out_seq=0 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.249+0000 7efd41ffb640 10 client.28487 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.250+0000 7efd41ffb640 1 -- 172.21.15.86:0/4133728853 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efd480054f0 con 0x5622b279e530 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.250+0000 7efd41ffb640 10 client.28487 handle_command_reply: tid=0 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.250+0000 7efd62a03740 1 client.28487 shutdown 2024-09-05T07:00:02.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.250+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 msgr2=0x5622b278baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.250+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd48005ab0 tx=0x7efd48000bc0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 shutdown_connections 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b279e530 0x5622b27a0920 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b278c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b278baf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 --2- 172.21.15.86:0/4133728853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 >> 172.21.15.86:0/4133728853 conn(0x5622b2777a30 msgr2=0x5622b2776b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd41ffb640 0 client.28487 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 shutdown_connections 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 1 -- 172.21.15.86:0/4133728853 wait complete. 2024-09-05T07:00:02.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.251+0000 7efd62a03740 20 client.28487 trim_cache size 0 max 16384 2024-09-05T07:00:02.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 Processor -- start 2024-09-05T07:00:02.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 -- start start 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b2776a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2788d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ace50 con 0x5622b278ee90 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ae990 con 0x5622b277bce0 2024-09-05T07:00:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd62a03740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ad520 con 0x5622b277b3c0 2024-09-05T07:00:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b2776a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd5e3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51682/0 (socket says 172.21.15.86:51682) 2024-09-05T07:00:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b2776a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:37702/0 (socket says 172.21.15.86:37702) 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.278+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3277729216 learned_addr learned my addr 172.21.15.86:0/3277729216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3277729216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 msgr2=0x5622b2776a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b2776a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3277729216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 msgr2=0x5622b2788d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2788d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 -- 172.21.15.86:0/3277729216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b27915f0 con 0x5622b277b3c0 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd54004a40 tx=0x7efd54005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dbd932e9b2005ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.279+0000 7efd43fff640 1 -- 172.21.15.86:0/3277729216 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5409bd70 con 0x5622b277b3c0 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.280+0000 7efd43fff640 1 -- 172.21.15.86:0/3277729216 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd540a0120 con 0x5622b277b3c0 2024-09-05T07:00:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.280+0000 7efd43fff640 1 -- 172.21.15.86:0/3277729216 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540983e0 con 0x5622b277b3c0 2024-09-05T07:00:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- 172.21.15.86:0/3277729216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 msgr2=0x5622b278a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd54004a40 tx=0x7efd54005430 comp rx=0 tx=0).stop 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- 172.21.15.86:0/3277729216 shutdown_connections 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b278ee90 0x5622b2788d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b277bce0 0x5622b2776a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 --2- 172.21.15.86:0/3277729216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b278a3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- 172.21.15.86:0/3277729216 >> 172.21.15.86:0/3277729216 conn(0x5622b2777a30 msgr2=0x5622b2776370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- 172.21.15.86:0/3277729216 shutdown_connections 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- 172.21.15.86:0/3277729216 wait complete. 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.281+0000 7efd62a03740 1 -- start start 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 Processor -- start 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b27a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b278ee90 0x5622b27a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd581ae2c0 con 0x5622b277bce0 2024-09-05T07:00:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd581adbf0 con 0x5622b278ee90 2024-09-05T07:00:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd581aae50 con 0x5622b277b3c0 2024-09-05T07:00:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd5dbc5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51124/0 (socket says 172.21.15.86:51124) 2024-09-05T07:00:02.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd5dbc5640 1 -- 172.21.15.86:0/1600474793 learned_addr learned my addr 172.21.15.86:0/1600474793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:02.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.282+0000 7efd62a03740 10 client.0 _ll_register_callbacks cb 0x7efd50012890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5ebc7640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b278ee90 0x5622b27a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b27a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 -- 172.21.15.86:0/1600474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 msgr2=0x5622b27a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b27a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 -- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b278ee90 msgr2=0x5622b27a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b278ee90 0x5622b27a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 -- 172.21.15.86:0/1600474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b27a2960 con 0x5622b277bce0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd5dbc5640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7efd48006010 tx=0x7efd48005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694cc46e38b6c60d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48009520 con 0x5622b277bce0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9605+0+0 (secure 0 0 0) 0x7efd480035f0 con 0x5622b277bce0 2024-09-05T07:00:02.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.283+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd540a0120 con 0x5622b277bce0 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efd48053020 con 0x5622b277bce0 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 10 client.19695 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5622b2809af0 con 0x5622b277bce0 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd48003cc0 con 0x5622b277bce0 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7efd480315c0 con 0x5622b277bce0 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 10 client.19695 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 20 client.19695 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b280a150 0x5622b280c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 4 client.19695 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efd540a0120 con 0x5622b280a150 2024-09-05T07:00:02.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.284+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b280a150 0x5622b280c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd41ffb640 10 client.19695 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd5e3c6640 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b280a150 0x5622b280c570 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4932540d69908615 server_cookie=fb441d70fcc52989 in_seq=0 out_seq=0 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd41ffb640 1 -- 172.21.15.86:0/1600474793 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7efd540a0120 con 0x5622b280a150 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd41ffb640 10 client.19695 handle_command_reply: tid=0 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 client.19695 shutdown 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 msgr2=0x5622b27a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7efd48006010 tx=0x7efd48005a20 comp rx=0 tx=0).stop 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 shutdown_connections 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5622b280a150 0x5622b280c570 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5622b278ee90 0x5622b27a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b277bce0 0x5622b27a4190 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 --2- 172.21.15.86:0/1600474793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5622b277b3c0 0x5622b27a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd41ffb640 0 client.19695 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.285+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 >> 172.21.15.86:0/1600474793 conn(0x5622b2777a30 msgr2=0x5622b2776b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 shutdown_connections 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd62a03740 1 -- 172.21.15.86:0/1600474793 wait complete. 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd62a03740 20 client.19695 trim_cache size 0 max 16384 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout: "81c9f59b-883f-4d52-ae87-4c4681be01ad": { 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:00:02.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "81c9f59b-883f-4d52-ae87-4c4681be01ad", 2024-09-05T07:00:02.292 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:00:02.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:02.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:02.292 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 msgr2=0x7efd440a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 0x7efd440a9640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd50009940 tx=0x7efd50009f50 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 msgr2=0x7efd58196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.286+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e9f0 tx=0x7efd5400edf0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 shutdown_connections 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7efd440a7180 0x7efd440a9640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd58106b80 0x7efd5819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd58100aa0 0x7efd581966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 --2- 172.21.15.86:0/3208685990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58100100 0x7efd58196190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 >> 172.21.15.86:0/3208685990 conn(0x7efd58077230 msgr2=0x7efd58104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 shutdown_connections 2024-09-05T07:00:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:02.287+0000 7efd6062a640 1 -- 172.21.15.86:0/3208685990 wait complete. 2024-09-05T07:00:02.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39332 6.47 MB/sec execute 200 sec latency 86.355 ms 2024-09-05T07:00:03.075 INFO:tasks.cephfs.filesystem:scrub status for tag:81c9f59b-883f-4d52-ae87-4c4681be01ad - {'path': '/', 'tag': '81c9f59b-883f-4d52-ae87-4c4681be01ad', 'options': 'recursive,force'} 2024-09-05T07:00:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:03 smithi115 ceph-mon[34791]: pgmap v590: 129 pgs: 129 active+clean; 960 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 319 op/s 2024-09-05T07:00:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:03 smithi086 ceph-mon[30943]: pgmap v590: 129 pgs: 129 active+clean; 960 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 319 op/s 2024-09-05T07:00:03.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39527 6.51 MB/sec execute 201 sec latency 61.486 ms 2024-09-05T07:00:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:03 smithi096 ceph-mon[34468]: pgmap v590: 129 pgs: 129 active+clean; 960 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 319 op/s 2024-09-05T07:00:04.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39720 6.53 MB/sec execute 202 sec latency 63.898 ms 2024-09-05T07:00:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:05 smithi115 ceph-mon[34791]: pgmap v591: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:05 smithi086 ceph-mon[30943]: pgmap v591: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:05.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39880 6.52 MB/sec execute 203 sec latency 149.569 ms 2024-09-05T07:00:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:05 smithi096 ceph-mon[34468]: pgmap v591: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:06.764 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40034 6.50 MB/sec execute 204 sec latency 110.688 ms 2024-09-05T07:00:07.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:07 smithi115 ceph-mon[34791]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:07 smithi115 ceph-mon[34791]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T07:00:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:07 smithi086 ceph-mon[30943]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:07 smithi086 ceph-mon[30943]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T07:00:07.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40227 6.53 MB/sec execute 205 sec latency 65.205 ms 2024-09-05T07:00:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:07 smithi096 ceph-mon[34468]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-05T07:00:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:07 smithi096 ceph-mon[34468]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T07:00:08.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40421 6.54 MB/sec execute 206 sec latency 96.690 ms 2024-09-05T07:00:09.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:09 smithi115 ceph-mon[34791]: pgmap v594: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:09 smithi086 ceph-mon[30943]: pgmap v594: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:09.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40583 6.57 MB/sec execute 207 sec latency 62.823 ms 2024-09-05T07:00:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:09 smithi096 ceph-mon[34468]: pgmap v594: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:10.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40742 6.57 MB/sec execute 208 sec latency 65.264 ms 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: pgmap v595: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:00:11.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:00:11.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: pgmap v595: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:00:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:00:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:00:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:00:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:00:11.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40959 6.63 MB/sec execute 209 sec latency 61.366 ms 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: pgmap v595: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:00:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:00:12.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41157 6.66 MB/sec execute 210 sec latency 204.326 ms 2024-09-05T07:00:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:13 smithi115 ceph-mon[34791]: pgmap v596: 129 pgs: 129 active+clean; 978 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-09-05T07:00:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:13 smithi086 ceph-mon[30943]: pgmap v596: 129 pgs: 129 active+clean; 978 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-09-05T07:00:13.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41314 6.66 MB/sec execute 211 sec latency 104.325 ms 2024-09-05T07:00:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:13 smithi096 ceph-mon[34468]: pgmap v596: 129 pgs: 129 active+clean; 978 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 299 op/s 2024-09-05T07:00:14.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41447 6.65 MB/sec execute 212 sec latency 85.400 ms 2024-09-05T07:00:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:15 smithi115 ceph-mon[34791]: pgmap v597: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:15 smithi086 ceph-mon[30943]: pgmap v597: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:15.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41627 6.69 MB/sec execute 213 sec latency 59.769 ms 2024-09-05T07:00:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:15 smithi096 ceph-mon[34468]: pgmap v597: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:16.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41794 6.70 MB/sec execute 214 sec latency 63.108 ms 2024-09-05T07:00:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:17 smithi115 ceph-mon[34791]: pgmap v598: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:17 smithi115 ceph-mon[34791]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T07:00:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:17 smithi086 ceph-mon[30943]: pgmap v598: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:17 smithi086 ceph-mon[30943]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T07:00:17.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41970 6.71 MB/sec execute 215 sec latency 98.222 ms 2024-09-05T07:00:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:17 smithi096 ceph-mon[34468]: pgmap v598: 129 pgs: 129 active+clean; 979 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 273 op/s 2024-09-05T07:00:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:17 smithi096 ceph-mon[34468]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T07:00:18.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42120 6.71 MB/sec execute 216 sec latency 102.891 ms 2024-09-05T07:00:19.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:19 smithi115 ceph-mon[34791]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:19 smithi086 ceph-mon[30943]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:19.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42283 6.70 MB/sec execute 217 sec latency 59.145 ms 2024-09-05T07:00:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:19 smithi096 ceph-mon[34468]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:20.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42466 6.71 MB/sec execute 218 sec latency 66.237 ms 2024-09-05T07:00:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:21 smithi086 ceph-mon[30943]: pgmap v601: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:21.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:21 smithi115 ceph-mon[34791]: pgmap v601: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:21.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42664 6.73 MB/sec execute 219 sec latency 86.119 ms 2024-09-05T07:00:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:21 smithi096 ceph-mon[34468]: pgmap v601: 129 pgs: 129 active+clean; 993 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 282 op/s 2024-09-05T07:00:22.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42816 6.73 MB/sec execute 220 sec latency 125.080 ms 2024-09-05T07:00:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:23 smithi115 ceph-mon[34791]: pgmap v602: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-09-05T07:00:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:23 smithi086 ceph-mon[30943]: pgmap v602: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-09-05T07:00:23.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42998 6.76 MB/sec execute 221 sec latency 86.274 ms 2024-09-05T07:00:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:23 smithi096 ceph-mon[34468]: pgmap v602: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 293 op/s 2024-09-05T07:00:24.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43180 6.77 MB/sec execute 222 sec latency 61.466 ms 2024-09-05T07:00:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:25 smithi115 ceph-mon[34791]: pgmap v603: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:25.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:25 smithi086 ceph-mon[30943]: pgmap v603: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:25.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43351 6.77 MB/sec execute 223 sec latency 127.901 ms 2024-09-05T07:00:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:25 smithi096 ceph-mon[34468]: pgmap v603: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:26.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43547 6.78 MB/sec execute 224 sec latency 57.581 ms 2024-09-05T07:00:27.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:27 smithi115 ceph-mon[34791]: pgmap v604: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:27 smithi086 ceph-mon[30943]: pgmap v604: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:27.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43720 6.80 MB/sec execute 225 sec latency 100.732 ms 2024-09-05T07:00:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:27 smithi096 ceph-mon[34468]: pgmap v604: 129 pgs: 129 active+clean; 1005 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 293 op/s 2024-09-05T07:00:28.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43906 6.80 MB/sec execute 226 sec latency 55.941 ms 2024-09-05T07:00:29.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:29 smithi115 ceph-mon[34791]: pgmap v605: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-09-05T07:00:29.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:29 smithi086 ceph-mon[30943]: pgmap v605: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-09-05T07:00:29.765 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44079 6.83 MB/sec execute 227 sec latency 134.859 ms 2024-09-05T07:00:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:29 smithi096 ceph-mon[34468]: pgmap v605: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 282 op/s 2024-09-05T07:00:30.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44269 6.84 MB/sec execute 228 sec latency 66.862 ms 2024-09-05T07:00:31.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:31 smithi115 ceph-mon[34791]: pgmap v606: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 247 op/s 2024-09-05T07:00:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:31 smithi086 ceph-mon[30943]: pgmap v606: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 247 op/s 2024-09-05T07:00:31.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44466 6.88 MB/sec execute 229 sec latency 67.641 ms 2024-09-05T07:00:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:31 smithi096 ceph-mon[34468]: pgmap v606: 129 pgs: 129 active+clean; 1012 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 247 op/s 2024-09-05T07:00:32.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44654 6.88 MB/sec execute 230 sec latency 68.543 ms 2024-09-05T07:00:33.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:00:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:33 smithi115 ceph-mon[34791]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 359 op/s 2024-09-05T07:00:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:33 smithi086 ceph-mon[30943]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 359 op/s 2024-09-05T07:00:33.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44803 6.90 MB/sec execute 231 sec latency 60.330 ms 2024-09-05T07:00:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:33 smithi096 ceph-mon[34468]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 359 op/s 2024-09-05T07:00:34.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44971 6.93 MB/sec execute 232 sec latency 60.583 ms 2024-09-05T07:00:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:35.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:35 smithi086 ceph-mon[30943]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-05T07:00:35.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:35 smithi115 ceph-mon[34791]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-05T07:00:35.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45142 6.93 MB/sec execute 233 sec latency 102.252 ms 2024-09-05T07:00:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:35 smithi096 ceph-mon[34468]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 -- 172.21.15.86:0/1817859417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434106f10 msgr2=0x7f443410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 --2- 172.21.15.86:0/1817859417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434106f10 0x7f443410ea90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f442c009ca0 tx=0x7f442c02a8c0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 -- 172.21.15.86:0/1817859417 shutdown_connections 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 --2- 172.21.15.86:0/1817859417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434106f10 0x7f443410ea90 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 --2- 172.21.15.86:0/1817859417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 0x7f44341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 --2- 172.21.15.86:0/1817859417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434101de0 0x7f44341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 -- 172.21.15.86:0/1817859417 >> 172.21.15.86:0/1817859417 conn(0x7f44340fd880 msgr2=0x7f44340ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 -- 172.21.15.86:0/1817859417 shutdown_connections 2024-09-05T07:00:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.127+0000 7f44398c9640 1 -- 172.21.15.86:0/1817859417 wait complete. 2024-09-05T07:00:36.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 Processor -- start 2024-09-05T07:00:36.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 -- start start 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 0x7f443410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 0x7f443410bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 0x7f443410c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4434075f90 con 0x7f4434101de0 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4434075e10 con 0x7f4434102790 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f44398c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4434076110 con 0x7f4434106f10 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.128+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 0x7f443410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 0x7f443410ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52776/0 (socket says 172.21.15.86:52776) 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f4432ffd640 1 -- 172.21.15.86:0/377746074 learned_addr learned my addr 172.21.15.86:0/377746074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 0x7f443410c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44327fc640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 0x7f443410bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 -- 172.21.15.86:0/377746074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 msgr2=0x7f443410bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 0x7f443410bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 -- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 msgr2=0x7f443410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 0x7f443410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 -- 172.21.15.86:0/377746074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44341aac90 con 0x7f4434106f10 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44337fe640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 0x7f443410c280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f442c031040 tx=0x7f442c032f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44388c7640 1 -- 172.21.15.86:0/377746074 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003b70 con 0x7f4434106f10 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341aae60 con 0x7f4434106f10 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44388c7640 1 -- 172.21.15.86:0/377746074 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f442c003d10 con 0x7f4434106f10 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.129+0000 7f44388c7640 1 -- 172.21.15.86:0/377746074 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c030af0 con 0x7f4434106f10 2024-09-05T07:00:36.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.130+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341ab310 con 0x7f4434106f10 2024-09-05T07:00:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.132+0000 7f44388c7640 1 -- 172.21.15.86:0/377746074 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f442c030c90 con 0x7f4434106f10 2024-09-05T07:00:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.132+0000 7f44388c7640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 0x7f441c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.133+0000 7f44388c7640 1 -- 172.21.15.86:0/377746074 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f442c0f18f0 con 0x7f4434106f10 2024-09-05T07:00:36.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.133+0000 7f4432ffd640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 0x7f441c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.156+0000 7f4432ffd640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 0x7f441c0a51b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4424009c90 tx=0x7f4424005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 Processor -- start 2024-09-05T07:00:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 -- start start 2024-09-05T07:00:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 0x55939338e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x559393399140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 0x55939339ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593932f42e0 con 0x55939338e4a0 2024-09-05T07:00:36.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55939329bce0 con 0x55939338edc0 2024-09-05T07:00:36.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f443bca2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5593932f02a0 con 0x559393399680 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 0x55939338e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 0x55939338e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52788/0 (socket says 172.21.15.86:52788) 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f4432ffd640 1 -- 172.21.15.86:0/3321425167 learned_addr learned my addr 172.21.15.86:0/3321425167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f44337fe640 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 0x55939339ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.158+0000 7f44327fc640 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x559393399140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 -- 172.21.15.86:0/3321425167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 msgr2=0x559393399140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x559393399140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 -- 172.21.15.86:0/3321425167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 msgr2=0x55939338e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 0x55939338e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 -- 172.21.15.86:0/3321425167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559393280440 con 0x559393399680 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f44337fe640 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 0x55939339ba70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f442c0eed10 tx=0x7f442c0bd970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570433b4cdadf12d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f4419ffb640 1 -- 172.21.15.86:0/3321425167 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0bdae0 con 0x559393399680 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.159+0000 7f4419ffb640 1 -- 172.21.15.86:0/3321425167 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f442c036040 con 0x559393399680 2024-09-05T07:00:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f4419ffb640 1 -- 172.21.15.86:0/3321425167 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0bdd40 con 0x559393399680 2024-09-05T07:00:36.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 -- 172.21.15.86:0/3321425167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 msgr2=0x55939339ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 0x55939339ba70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f442c0eed10 tx=0x7f442c0bd970 comp rx=0 tx=0).stop 2024-09-05T07:00:36.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 -- 172.21.15.86:0/3321425167 shutdown_connections 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559393399680 0x55939339ba70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x559393399140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 --2- 172.21.15.86:0/3321425167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338e4a0 0x55939338e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 -- 172.21.15.86:0/3321425167 >> 172.21.15.86:0/3321425167 conn(0x55939338aa30 msgr2=0x55939338ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 -- 172.21.15.86:0/3321425167 shutdown_connections 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.160+0000 7f443bca2740 1 -- 172.21.15.86:0/3321425167 wait complete. 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.161+0000 7f443bca2740 1 -- start start 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.161+0000 7f443bca2740 1 Processor -- start 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x559393399000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x559393396830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593932f02a0 con 0x559393399680 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5593932af050 con 0x55939338edc0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34104/0 (socket says 172.21.15.86:34104) 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 -- 172.21.15.86:0/3507706651 learned_addr learned my addr 172.21.15.86:0/3507706651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55939339c010 con 0x55939338e4a0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44337fe640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x559393396830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f443bca2740 10 client.0 _ll_register_callbacks cb 0x5593932ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f4432ffd640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x559393399000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 -- 172.21.15.86:0/3507706651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 msgr2=0x559393399000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x559393399000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.162+0000 7f44327fc640 1 -- 172.21.15.86:0/3507706651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 msgr2=0x559393396830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f44327fc640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x559393396830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f44327fc640 1 -- 172.21.15.86:0/3507706651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593930653c0 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f44337fe640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x559393396830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f44327fc640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4428012700 tx=0x7f4428012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06ee4e51f33118b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f4412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428004030 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f44280041d0 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c036040 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f44280043d0 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f443bca2740 10 client.28505 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.163+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5593933a7830 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f442c036040 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4428003b90 con 0x55939338edc0 2024-09-05T07:00:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 10 client.28505 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 10 client.28505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 20 client.28505 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5593933a9160 0x5593933ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 4 client.28505 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559393301850 con 0x5593933a9160 2024-09-05T07:00:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.164+0000 7f4432ffd640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5593933a9160 0x5593933ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.165+0000 7f4412ffd640 10 client.28505 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.165+0000 7f4432ffd640 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5593933a9160 0x5593933ab5a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868f4d8a3a1c5d75 server_cookie=be19bc369ec35d16 in_seq=0 out_seq=0 2024-09-05T07:00:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f4412ffd640 1 -- 172.21.15.86:0/3507706651 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559393301850 con 0x5593933a9160 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f4412ffd640 10 client.28505 handle_command_reply: tid=0 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 client.28505 shutdown 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 msgr2=0x5593933962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4428012700 tx=0x7f4428012bc0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 shutdown_connections 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5593933a9160 0x5593933ab5a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x559393396830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338edc0 0x5593933962b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 --2- 172.21.15.86:0/3507706651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x559393399000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 >> 172.21.15.86:0/3507706651 conn(0x55939338aa30 msgr2=0x559393390760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f4412ffd640 0 client.28505 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 shutdown_connections 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.166+0000 7f443bca2740 1 -- 172.21.15.86:0/3507706651 wait complete. 2024-09-05T07:00:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.167+0000 7f443bca2740 20 client.28505 trim_cache size 0 max 16384 2024-09-05T07:00:36.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.199+0000 7f443bca2740 1 Processor -- start 2024-09-05T07:00:36.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.199+0000 7f443bca2740 1 -- start start 2024-09-05T07:00:36.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.199+0000 7f443bca2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 0x559393394b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.199+0000 7f443bca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338edc0 0x5593933a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.200+0000 7f443bca2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.200+0000 7f44337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.200+0000 7f44337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52804/0 (socket says 172.21.15.86:52804) 2024-09-05T07:00:36.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.200+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 0x559393394b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 0x559393394b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34120/0 (socket says 172.21.15.86:34120) 2024-09-05T07:00:36.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 -- 172.21.15.86:0/2420755465 learned_addr learned my addr 172.21.15.86:0/2420755465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b5810 con 0x559393399680 2024-09-05T07:00:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b7350 con 0x55939338e4a0 2024-09-05T07:00:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b5ee0 con 0x55939338edc0 2024-09-05T07:00:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44327fc640 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338edc0 0x5593933a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 -- 172.21.15.86:0/2420755465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338edc0 msgr2=0x5593933a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338edc0 0x5593933a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 -- 172.21.15.86:0/2420755465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 msgr2=0x559393394b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 0x559393394b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.201+0000 7f44337fe640 1 -- 172.21.15.86:0/2420755465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559393395260 con 0x559393399680 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.202+0000 7f44337fe640 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f442c0bd5b0 tx=0x7f442c0bd540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c54e903840b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.203+0000 7f44197fa640 1 -- 172.21.15.86:0/2420755465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0db070 con 0x559393399680 2024-09-05T07:00:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.203+0000 7f44197fa640 1 -- 172.21.15.86:0/2420755465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f442c036040 con 0x559393399680 2024-09-05T07:00:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.203+0000 7f44197fa640 1 -- 172.21.15.86:0/2420755465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0bdd40 con 0x559393399680 2024-09-05T07:00:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 msgr2=0x5593933917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f442c0bd5b0 tx=0x7f442c0bd540 comp rx=0 tx=0).stop 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 shutdown_connections 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559393399680 0x5593933917a0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338edc0 0x5593933a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 --2- 172.21.15.86:0/2420755465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55939338e4a0 0x559393394b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 >> 172.21.15.86:0/2420755465 conn(0x55939338aa30 msgr2=0x559393391090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 shutdown_connections 2024-09-05T07:00:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.204+0000 7f443bca2740 1 -- 172.21.15.86:0/2420755465 wait complete. 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.205+0000 7f443bca2740 1 -- start start 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.205+0000 7f443bca2740 1 Processor -- start 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.205+0000 7f443bca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f443bca2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338edc0 0x55939339caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59868/0 (socket says 172.21.15.86:59868) 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f4432ffd640 1 -- 172.21.15.86:0/1187731444 learned_addr learned my addr 172.21.15.86:0/1187731444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559393399680 0x559393411e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b6c80 con 0x55939338edc0 2024-09-05T07:00:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b65b0 con 0x559393399680 2024-09-05T07:00:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f44337fe640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559393399680 0x559393411e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f44327fc640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338edc0 0x55939339caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.206+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b3810 con 0x55939338e4a0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f443bca2740 10 client.0 _ll_register_callbacks cb 0x7f4428009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 -- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559393399680 msgr2=0x559393411e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559393399680 0x559393411e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 -- 172.21.15.86:0/1187731444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338edc0 msgr2=0x55939339caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338edc0 0x55939339caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559393412440 con 0x55939338e4a0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4432ffd640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5593933abde0 tx=0x7f4424004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee65acb870ca1d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:00:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c036040 con 0x55939338e4a0 2024-09-05T07:00:36.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.207+0000 7f4412ffd640 1 -- 172.21.15.86:0/1187731444 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442400e590 con 0x55939338e4a0 2024-09-05T07:00:36.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f4412ffd640 1 -- 172.21.15.86:0/1187731444 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f442400e730 con 0x55939338e4a0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f4412ffd640 1 -- 172.21.15.86:0/1187731444 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f442400e990 con 0x55939338e4a0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 10 client.28549 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5593934136c0 con 0x55939338e4a0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f4412ffd640 1 -- 172.21.15.86:0/1187731444 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f442400eb30 con 0x55939338e4a0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 10 client.28549 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 10 client.28549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 20 client.28549 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559393413cd0 0x5593934160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 4 client.28549 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.208+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f442c036040 con 0x559393413cd0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.209+0000 7f44327fc640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559393413cd0 0x5593934160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.209+0000 7f44327fc640 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559393413cd0 0x5593934160c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a35cf125c861873 server_cookie=ec1948a92f81402a in_seq=0 out_seq=0 2024-09-05T07:00:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.209+0000 7f4412ffd640 10 client.28549 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f4412ffd640 1 -- 172.21.15.86:0/1187731444 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f442c036040 con 0x559393413cd0 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f4412ffd640 10 client.28549 handle_command_reply: tid=0 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 client.28549 shutdown 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 msgr2=0x55939339c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5593933abde0 tx=0x7f4424004730 comp rx=0 tx=0).stop 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 shutdown_connections 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x559393413cd0 0x5593934160c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559393399680 0x559393411e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55939338edc0 0x55939339caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f4412ffd640 0 client.28549 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 --2- 172.21.15.86:0/1187731444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55939338e4a0 0x55939339c5b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 >> 172.21.15.86:0/1187731444 conn(0x55939338aa30 msgr2=0x559393389350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 shutdown_connections 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 1 -- 172.21.15.86:0/1187731444 wait complete. 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.210+0000 7f443bca2740 20 client.28549 trim_cache size 0 max 16384 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:00:36.213 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 msgr2=0x7f441c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 0x7f441c0a51b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4424009c90 tx=0x7f4424005f90 comp rx=0 tx=0).stop 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 msgr2=0x7f443410c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 0x7f443410c280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f442c031040 tx=0x7f442c032f80 comp rx=0 tx=0).stop 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 shutdown_connections 2024-09-05T07:00:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f441c0a2cf0 0x7f441c0a51b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434106f10 0x7f443410c280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4434102790 0x7f443410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 --2- 172.21.15.86:0/377746074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4434101de0 0x7f443410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.211+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 >> 172.21.15.86:0/377746074 conn(0x7f44340fd880 msgr2=0x7f44340feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.212+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 shutdown_connections 2024-09-05T07:00:36.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:36.212+0000 7f44398c9640 1 -- 172.21.15.86:0/377746074 wait complete. 2024-09-05T07:00:36.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45286 6.93 MB/sec execute 234 sec latency 71.564 ms 2024-09-05T07:00:37.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:00:37.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:00:37.421 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:37 smithi086 ceph-mon[30943]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 237 op/s 2024-09-05T07:00:37.609 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:37.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:37 smithi115 ceph-mon[34791]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 237 op/s 2024-09-05T07:00:37.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45422 6.92 MB/sec execute 235 sec latency 118.085 ms 2024-09-05T07:00:37.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:37 smithi096 ceph-mon[34468]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 237 op/s 2024-09-05T07:00:38.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45583 6.93 MB/sec execute 236 sec latency 127.360 ms 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 -- 172.21.15.86:0/4070235384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f8106f20 msgr2=0x7f96f810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 --2- 172.21.15.86:0/4070235384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f8106f20 0x7f96f810eaa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009be0 tx=0x7f96f002a6c0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 -- 172.21.15.86:0/4070235384 shutdown_connections 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 --2- 172.21.15.86:0/4070235384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f8106f20 0x7f96f810eaa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 --2- 172.21.15.86:0/4070235384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f81027a0 0x7f96f81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 --2- 172.21.15.86:0/4070235384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8101df0 0x7f96f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 -- 172.21.15.86:0/4070235384 >> 172.21.15.86:0/4070235384 conn(0x7f96f80fd8b0 msgr2=0x7f96f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:38.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.856+0000 7f96fff10640 1 -- 172.21.15.86:0/4070235384 shutdown_connections 2024-09-05T07:00:38.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.857+0000 7f96fff10640 1 -- 172.21.15.86:0/4070235384 wait complete. 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.857+0000 7f96fff10640 1 Processor -- start 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 -- start start 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81027a0 0x7f96f819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8106f20 0x7f96f819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8111e30 con 0x7f96f8106f20 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8111cb0 con 0x7f96f81027a0 2024-09-05T07:00:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fff10640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8111fb0 con 0x7f96f8101df0 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59880/0 (socket says 172.21.15.86:59880) 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 -- 172.21.15.86:0/961444623 learned_addr learned my addr 172.21.15.86:0/961444623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fd4ab640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81027a0 0x7f96f819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fe4ad640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8106f20 0x7f96f819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 -- 172.21.15.86:0/961444623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81027a0 msgr2=0x7f96f819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81027a0 0x7f96f819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 -- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8106f20 msgr2=0x7f96f819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8106f20 0x7f96f819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.858+0000 7f96fdcac640 1 -- 172.21.15.86:0/961444623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f819f4b0 con 0x7f96f8101df0 2024-09-05T07:00:38.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96fdcac640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96f400e9c0 tx=0x7f96f400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400cde0 con 0x7f96f8101df0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f819f7a0 con 0x7f96f8101df0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f40035f0 con 0x7f96f8101df0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4010690 con 0x7f96f8101df0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.859+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f81a7080 con 0x7f96f8101df0 2024-09-05T07:00:38.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.860+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f81021d0 con 0x7f96f8101df0 2024-09-05T07:00:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.861+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f96f40037a0 con 0x7f96f8101df0 2024-09-05T07:00:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.862+0000 7f96eeffd640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 0x7f96e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.862+0000 7f96fd4ab640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 0x7f96e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.863+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f96f40cfb60 con 0x7f96f8101df0 2024-09-05T07:00:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.863+0000 7f96fd4ab640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 0x7f96e40a51b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f96f80ff760 tx=0x7f96e8005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:38.867+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96f4009510 con 0x7f96f8101df0 2024-09-05T07:00:39.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.235+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f81a7310 con 0x7f96f8101df0 2024-09-05T07:00:39.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.236+0000 7f96eeffd640 1 -- 172.21.15.86:0/961444623 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7f96f409c340 con 0x7f96f8101df0 2024-09-05T07:00:39.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:00:39.237 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:00:39.237 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 msgr2=0x7f96e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 0x7f96e40a51b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f96f80ff760 tx=0x7f96e8005f90 comp rx=0 tx=0).stop 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 msgr2=0x7f96f819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96f400e9c0 tx=0x7f96f400ee80 comp rx=0 tx=0).stop 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 shutdown_connections 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96e40a2cf0 0x7f96e40a51b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96f8106f20 0x7f96f819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:39.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81027a0 0x7f96f819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:39.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 --2- 172.21.15.86:0/961444623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96f8101df0 0x7f96f819a500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:39.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 >> 172.21.15.86:0/961444623 conn(0x7f96f80fd8b0 msgr2=0x7f96f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:39.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 shutdown_connections 2024-09-05T07:00:39.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:39.240+0000 7f96fff10640 1 -- 172.21.15.86:0/961444623 wait complete. 2024-09-05T07:00:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:39 smithi086 ceph-mon[30943]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 356 op/s 2024-09-05T07:00:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:39 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/961444623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:39.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:39 smithi115 ceph-mon[34791]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 356 op/s 2024-09-05T07:00:39.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:39 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/961444623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:39.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45723 6.92 MB/sec execute 237 sec latency 89.403 ms 2024-09-05T07:00:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:39 smithi096 ceph-mon[34468]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 356 op/s 2024-09-05T07:00:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:39 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/961444623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:39.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:00:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:40.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45863 6.92 MB/sec execute 238 sec latency 80.839 ms 2024-09-05T07:00:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 -- 172.21.15.86:0/1263846998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0106f00 msgr2=0x7f61c010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 --2- 172.21.15.86:0/1263846998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0106f00 0x7f61c010ea80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009ca0 tx=0x7f61b802a8c0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 -- 172.21.15.86:0/1263846998 shutdown_connections 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 --2- 172.21.15.86:0/1263846998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0106f00 0x7f61c010ea80 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 --2- 172.21.15.86:0/1263846998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0102780 0x7f61c01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 --2- 172.21.15.86:0/1263846998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101dd0 0x7f61c01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 -- 172.21.15.86:0/1263846998 >> 172.21.15.86:0/1263846998 conn(0x7f61c00fd870 msgr2=0x7f61c00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 -- 172.21.15.86:0/1263846998 shutdown_connections 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.917+0000 7f61c5386640 1 -- 172.21.15.86:0/1263846998 wait complete. 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 Processor -- start 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 -- start start 2024-09-05T07:00:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0102780 0x7f61c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 0x7f61c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111e10 con 0x7f61c0102780 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111c90 con 0x7f61c0106f00 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59900/0 (socket says 172.21.15.86:59900) 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 -- 172.21.15.86:0/3950893733 learned_addr learned my addr 172.21.15.86:0/3950893733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 0x7f61c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.918+0000 7f61c5386640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111f90 con 0x7f61c0101dd0 2024-09-05T07:00:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0102780 0x7f61c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 -- 172.21.15.86:0/3950893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 msgr2=0x7f61c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 0x7f61c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 -- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0102780 msgr2=0x7f61c0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0102780 0x7f61c0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 -- 172.21.15.86:0/3950893733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01aadc0 con 0x7f61c0101dd0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 0x7f61c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61beffd640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9f0 tx=0x7f61b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.919+0000 7f61a7fff640 1 -- 172.21.15.86:0/3950893733 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b000cd80 con 0x7f61c0101dd0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.920+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01aaff0 con 0x7f61c0101dd0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.920+0000 7f61a7fff640 1 -- 172.21.15.86:0/3950893733 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b00108e0 con 0x7f61c0101dd0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.920+0000 7f61a7fff640 1 -- 172.21.15.86:0/3950893733 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0018920 con 0x7f61c0101dd0 2024-09-05T07:00:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.920+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01ab4f0 con 0x7f61c0101dd0 2024-09-05T07:00:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.922+0000 7f61a7fff640 1 -- 172.21.15.86:0/3950893733 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f61b0010a90 con 0x7f61c0101dd0 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61a7fff640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 0x7f61a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61a7fff640 1 -- 172.21.15.86:0/3950893733 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f61b00cfb70 con 0x7f61c0101dd0 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 0x7f61a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 Processor -- start 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 -- start start 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563452792b40 0x563452791090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 0x563452791a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634526f72f0 con 0x56345279c370 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 0x7f61a80a5230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61c00fddd0 tx=0x7f61b4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56345269ecf0 con 0x563452792b40 2024-09-05T07:00:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.923+0000 7f61c775f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5634526f32b0 con 0x5634527915d0 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52870/0 (socket says 172.21.15.86:52870) 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61bf7fe640 1 -- 172.21.15.86:0/3069091081 learned_addr learned my addr 172.21.15.86:0/3069091081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 0x563452791a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61beffd640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563452792b40 0x563452791090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 -- 172.21.15.86:0/3069091081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563452792b40 msgr2=0x563452791090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563452792b40 0x563452791090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 -- 172.21.15.86:0/3069091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 msgr2=0x56345279e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 -- 172.21.15.86:0/3069091081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634526833a0 con 0x5634527915d0 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.924+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 0x563452791a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009fd0 tx=0x7f61b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73adc7715d86f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.925+0000 7f61a57fa640 1 -- 172.21.15.86:0/3069091081 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4023070 con 0x5634527915d0 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.925+0000 7f61a57fa640 1 -- 172.21.15.86:0/3069091081 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b40035f0 con 0x5634527915d0 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.925+0000 7f61a57fa640 1 -- 172.21.15.86:0/3069091081 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b401f680 con 0x5634527915d0 2024-09-05T07:00:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 -- 172.21.15.86:0/3069091081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 msgr2=0x563452791a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 0x563452791a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009fd0 tx=0x7f61b4014000 comp rx=0 tx=0).stop 2024-09-05T07:00:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 -- 172.21.15.86:0/3069091081 shutdown_connections 2024-09-05T07:00:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56345279c370 0x56345279e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634527915d0 0x563452791a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 --2- 172.21.15.86:0/3069091081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563452792b40 0x563452791090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 -- 172.21.15.86:0/3069091081 >> 172.21.15.86:0/3069091081 conn(0x56345275a310 msgr2=0x56345278dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 -- 172.21.15.86:0/3069091081 shutdown_connections 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.926+0000 7f61c775f740 1 -- 172.21.15.86:0/3069091081 wait complete. 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.927+0000 7f61c775f740 1 -- start start 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.927+0000 7f61c775f740 1 Processor -- start 2024-09-05T07:00:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 0x56345279bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 0x563452799570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59930/0 (socket says 172.21.15.86:59930) 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61be7fc640 1 -- 172.21.15.86:0/3899122849 learned_addr learned my addr 172.21.15.86:0/3899122849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56345279f0c0 con 0x5634527915d0 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634526b2060 con 0x56345279c370 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.928+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56345279f240 con 0x563452792b40 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 0x56345279bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61c775f740 10 client.0 _ll_register_callbacks cb 0x5634524058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 0x563452799570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 -- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 msgr2=0x563452798fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 -- 172.21.15.86:0/3899122849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 msgr2=0x563452799570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 0x563452799570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61beffd640 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634524683c0 con 0x5634527915d0 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 0x563452799570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:40.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.929+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61beffd640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 0x56345279bd20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f61b00cce80 tx=0x7f61b0010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b9c74176adeeb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b1070 con 0x5634527915d0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f61b0010630 con 0x5634527915d0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b40035f0 con 0x5634527915d0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f61b00a48c0 con 0x5634527915d0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61c775f740 10 client.19731 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.930+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5634527aab80 con 0x5634527915d0 2024-09-05T07:00:40.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b40035f0 con 0x5634527915d0 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f61b0018ac0 con 0x5634527915d0 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 10 client.19731 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 10 client.19731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 20 client.19731 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5634527ac1d0 0x5634527ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 4 client.19731 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.931+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563452704860 con 0x5634527ac1d0 2024-09-05T07:00:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.932+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5634527ac1d0 0x5634527ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.933+0000 7f61be7fc640 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5634527ac1d0 0x5634527ae5c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4cb63cdf6e09de6 server_cookie=1c356e4c7ee36633 in_seq=0 out_seq=0 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.933+0000 7f61a2ff5640 10 client.19731 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61a2ff5640 1 -- 172.21.15.86:0/3899122849 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563452704860 con 0x5634527ac1d0 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61a2ff5640 10 client.19731 handle_command_reply: tid=0 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 client.19731 shutdown 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 msgr2=0x56345279bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 0x56345279bd20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f61b00cce80 tx=0x7f61b0010f20 comp rx=0 tx=0).stop 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 shutdown_connections 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x5634527ac1d0 0x5634527ae5c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56345279c370 0x563452799570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563452792b40 0x563452798fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 --2- 172.21.15.86:0/3899122849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634527915d0 0x56345279bd20 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 >> 172.21.15.86:0/3899122849 conn(0x56345275a310 msgr2=0x563452793960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61a2ff5640 0 client.19731 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:00:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 shutdown_connections 2024-09-05T07:00:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.934+0000 7f61c775f740 1 -- 172.21.15.86:0/3899122849 wait complete. 2024-09-05T07:00:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.935+0000 7f61c775f740 20 client.19731 trim_cache size 0 max 16384 2024-09-05T07:00:40.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.954+0000 7f61c775f740 1 Processor -- start 2024-09-05T07:00:40.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.954+0000 7f61c775f740 1 -- start start 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452796230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b57f0 con 0x563452792b40 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b7330 con 0x5634527b4290 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61c775f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b5ec0 con 0x56345279c370 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52892/0 (socket says 172.21.15.86:52892) 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61beffd640 1 -- 172.21.15.86:0/708336594 learned_addr learned my addr 172.21.15.86:0/708336594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452796230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 -- 172.21.15.86:0/708336594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 msgr2=0x563452796230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452796230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 -- 172.21.15.86:0/708336594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 msgr2=0x5634527a8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 -- 172.21.15.86:0/708336594 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56345279c090 con 0x56345279c370 2024-09-05T07:00:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452796230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.955+0000 7f61be7fc640 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527b3d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56345279a610 tx=0x7f61b4011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f0b29076f5e641 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.956+0000 7f61a4ff9640 1 -- 172.21.15.86:0/708336594 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4023040 con 0x56345279c370 2024-09-05T07:00:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.956+0000 7f61a4ff9640 1 -- 172.21.15.86:0/708336594 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b4005520 con 0x56345279c370 2024-09-05T07:00:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.956+0000 7f61a4ff9640 1 -- 172.21.15.86:0/708336594 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4028990 con 0x56345279c370 2024-09-05T07:00:40.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- 172.21.15.86:0/708336594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 msgr2=0x5634527b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527b3d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56345279a610 tx=0x7f61b4011af0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- 172.21.15.86:0/708336594 shutdown_connections 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452796230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527b3d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 --2- 172.21.15.86:0/708336594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- 172.21.15.86:0/708336594 >> 172.21.15.86:0/708336594 conn(0x56345275a310 msgr2=0x5634527942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- 172.21.15.86:0/708336594 shutdown_connections 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- 172.21.15.86:0/708336594 wait complete. 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 -- start start 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.957+0000 7f61c775f740 1 Processor -- start 2024-09-05T07:00:40.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452815d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b6c60 con 0x563452792b40 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b6590 con 0x5634527b4290 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01b37f0 con 0x56345279c370 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61c775f740 10 client.0 _ll_register_callbacks cb 0x7f61b0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61beffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59948/0 (socket says 172.21.15.86:59948) 2024-09-05T07:00:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 -- 172.21.15.86:0/1617974158 learned_addr learned my addr 172.21.15.86:0/1617974158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452815d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 -- 172.21.15.86:0/1617974158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 msgr2=0x563452815d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452815d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 -- 172.21.15.86:0/1617974158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 msgr2=0x5634527a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61be7fc640 1 -- 172.21.15.86:0/1617974158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634527a1350 con 0x56345279c370 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61bf7fe640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452815d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.958+0000 7f61beffd640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61be7fc640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x563452797700 tx=0x7f61b4009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9597a4e3d75ff72 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4042060 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f61b40052d0 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b4005520 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f61b4049050 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 10 client.28573 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563452817330 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61b40035f0 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f61b403d5a0 con 0x56345279c370 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 10 client.28573 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 10 client.28573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.959+0000 7f61c775f740 20 client.28573 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:40.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563452817950 0x563452819d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61c775f740 4 client.28573 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f61b4005520 con 0x563452817950 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61beffd640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563452817950 0x563452819d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61beffd640 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563452817950 0x563452819d70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f8f8efd95acb80b server_cookie=ddc1c873439a7fd8 in_seq=0 out_seq=0 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.960+0000 7f61a2ff5640 10 client.28573 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61a2ff5640 1 -- 172.21.15.86:0/1617974158 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f61b4005520 con 0x563452817950 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61a2ff5640 10 client.28573 handle_command_reply: tid=0 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 client.28573 shutdown 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 msgr2=0x5634527a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x563452797700 tx=0x7f61b4009f70 comp rx=0 tx=0).stop 2024-09-05T07:00:40.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 shutdown_connections 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563452817950 0x563452819d70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634527b4290 0x563452815d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56345279c370 0x5634527a0ac0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 --2- 172.21.15.86:0/1617974158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563452792b40 0x5634527a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 >> 172.21.15.86:0/1617974158 conn(0x56345275a310 msgr2=0x56345278cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61a2ff5640 0 client.28573 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 shutdown_connections 2024-09-05T07:00:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 1 -- 172.21.15.86:0/1617974158 wait complete. 2024-09-05T07:00:40.969 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:00:40.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.962+0000 7f61c775f740 20 client.28573 trim_cache size 0 max 16384 2024-09-05T07:00:40.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 msgr2=0x7f61a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 0x7f61a80a5230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f61c00fddd0 tx=0x7f61b4005f90 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 msgr2=0x7f61c00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9f0 tx=0x7f61b000eeb0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 shutdown_connections 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f61a80a2d70 0x7f61a80a5230 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0106f00 0x7f61c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c0102780 0x7f61c0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 --2- 172.21.15.86:0/3950893733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61c0101dd0 0x7f61c00744f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.963+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 >> 172.21.15.86:0/3950893733 conn(0x7f61c00fd870 msgr2=0x7f61c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.964+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 shutdown_connections 2024-09-05T07:00:40.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:40.964+0000 7f61c5386640 1 -- 172.21.15.86:0/3950893733 wait complete. 2024-09-05T07:00:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:41 smithi115 ceph-mon[34791]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 237 op/s 2024-09-05T07:00:41.727 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:41 smithi086 ceph-mon[30943]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 237 op/s 2024-09-05T07:00:41.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46016 6.92 MB/sec execute 239 sec latency 100.380 ms 2024-09-05T07:00:41.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:00:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:41 smithi096 ceph-mon[34468]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 237 op/s 2024-09-05T07:00:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 -- 172.21.15.86:0/317270336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0102d90 msgr2=0x7f90f0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 --2- 172.21.15.86:0/317270336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0102d90 0x7f90f0103170 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009900 tx=0x7f90d802a480 comp rx=0 tx=0).stop 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 -- 172.21.15.86:0/317270336 shutdown_connections 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 --2- 172.21.15.86:0/317270336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f010c170 0x7f90f010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 --2- 172.21.15.86:0/317270336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f01036b0 0x7f90f010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 --2- 172.21.15.86:0/317270336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0102d90 0x7f90f0103170 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 -- 172.21.15.86:0/317270336 >> 172.21.15.86:0/317270336 conn(0x7f90f00fc7d0 msgr2=0x7f90f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 -- 172.21.15.86:0/317270336 shutdown_connections 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 -- 172.21.15.86:0/317270336 wait complete. 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.672+0000 7f90f59bf640 1 Processor -- start 2024-09-05T07:00:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 -- start start 2024-09-05T07:00:42.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 0x7f90f01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0111e00 con 0x7f90f010c170 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0111c80 con 0x7f90f0102d90 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90f59bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0111f80 con 0x7f90f01036b0 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42830/0 (socket says 172.21.15.86:42830) 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90e7fff640 1 -- 172.21.15.86:0/2409748566 learned_addr learned my addr 172.21.15.86:0/2409748566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37176/0 (socket says 172.21.15.86:37176) 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.673+0000 7f90eeffd640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 0x7f90f01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2409748566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 msgr2=0x7f90f0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2409748566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 msgr2=0x7f90f01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 0x7f90f01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2409748566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019d270 con 0x7f90f010c170 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90eeffd640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 0x7f90f01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007db0 tx=0x7f90e8010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90f49bd640 1 -- 172.21.15.86:0/2409748566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8010d40 con 0x7f90f010c170 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90f49bd640 1 -- 172.21.15.86:0/2409748566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e800c640 con 0x7f90f010c170 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019d560 con 0x7f90f010c170 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90f49bd640 1 -- 172.21.15.86:0/2409748566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8017650 con 0x7f90f010c170 2024-09-05T07:00:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.674+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f0100920 con 0x7f90f010c170 2024-09-05T07:00:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.677+0000 7f90f49bd640 1 -- 172.21.15.86:0/2409748566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f90e800c7f0 con 0x7f90f010c170 2024-09-05T07:00:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.677+0000 7f90f7d98740 1 Processor -- start 2024-09-05T07:00:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.677+0000 7f90f7d98740 1 -- start start 2024-09-05T07:00:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f49bd640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 0x7f90dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a257f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56332a1a82f0 con 0x56332a255b80 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f49bd640 1 -- 172.21.15.86:0/2409748566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f90e80d34a0 con 0x7f90f010c170 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56332a14fcf0 con 0x56332a24ade0 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90f7d98740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56332a1a42b0 con 0x56332a24c350 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55706/0 (socket says 172.21.15.86:55706) 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 -- 172.21.15.86:0/3999044734 learned_addr learned my addr 172.21.15.86:0/3999044734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a257f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90eeffd640 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 -- 172.21.15.86:0/3999044734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 msgr2=0x56332a24a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 -- 172.21.15.86:0/3999044734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 msgr2=0x56332a257f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a257f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.678+0000 7f90e7fff640 1 -- 172.21.15.86:0/3999044734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56332a1343a0 con 0x56332a24ade0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.679+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 0x7f90dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.679+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90e0011760 tx=0x7f90e0011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fa69c11283c2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.679+0000 7f90e5ffb640 1 -- 172.21.15.86:0/3999044734 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e001cda0 con 0x56332a24ade0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.679+0000 7f90e5ffb640 1 -- 172.21.15.86:0/3999044734 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e000fc60 con 0x56332a24ade0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.679+0000 7f90e5ffb640 1 -- 172.21.15.86:0/3999044734 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e001b360 con 0x56332a24ade0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 -- 172.21.15.86:0/3999044734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 msgr2=0x56332a24b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 0x7f90dc0a5230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90e0023000 tx=0x7f90e0020000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90e0011760 tx=0x7f90e0011c20 comp rx=0 tx=0).stop 2024-09-05T07:00:42.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 -- 172.21.15.86:0/3999044734 shutdown_connections 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a257f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a24b220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3999044734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 -- 172.21.15.86:0/3999044734 >> 172.21.15.86:0/3999044734 conn(0x56332a20b310 msgr2=0x56332a247470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 -- 172.21.15.86:0/3999044734 shutdown_connections 2024-09-05T07:00:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.680+0000 7f90f7d98740 1 -- 172.21.15.86:0/3999044734 wait complete. 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.681+0000 7f90f7d98740 1 -- start start 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.681+0000 7f90f7d98740 1 Processor -- start 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.681+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a255560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.681+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a255b80 0x56332a252ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90f7d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56332a258880 con 0x56332a24ade0 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90f7d98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56332a163060 con 0x56332a24c350 2024-09-05T07:00:42.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90f7d98740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56332a258a00 con 0x56332a255b80 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55720/0 (socket says 172.21.15.86:55720) 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 -- 172.21.15.86:0/3568212387 learned_addr learned my addr 172.21.15.86:0/3568212387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90eeffd640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a255560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a255b80 0x56332a252ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90f7d98740 10 client.0 _ll_register_callbacks cb 0x563329eb68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 -- 172.21.15.86:0/3568212387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a255b80 msgr2=0x56332a252ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a255b80 0x56332a252ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 -- 172.21.15.86:0/3568212387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 msgr2=0x56332a255560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a255560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.682+0000 7f90e7fff640 1 -- 172.21.15.86:0/3568212387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563329f193c0 con 0x56332a24c350 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90e7fff640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e750 tx=0x7f90e000e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628d446446d69e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e002d040 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f90e0011070 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e000fc60 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90e001cda0 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90f7d98740 10 client.28544 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.683+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56332a264330 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e000fe10 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f90e0004680 con 0x56332a24c350 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 10 client.28544 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 10 client.28544 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 20 client.28544 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a265680 0x56332a267a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 4 client.28544 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f90e000fc60 con 0x56332a265680 2024-09-05T07:00:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.684+0000 7f90eeffd640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a265680 0x56332a267a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.685+0000 7f90ceffd640 10 client.28544 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:00:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.685+0000 7f90eeffd640 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a265680 0x56332a267a70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=410352a77e4639b server_cookie=2591d2497ac30f95 in_seq=0 out_seq=0 2024-09-05T07:00:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.685+0000 7f90ceffd640 1 -- 172.21.15.86:0/3568212387 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f90e000fc60 con 0x56332a265680 2024-09-05T07:00:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.685+0000 7f90ceffd640 10 client.28544 handle_command_reply: tid=0 2024-09-05T07:00:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.685+0000 7f90f7d98740 1 client.28544 shutdown 2024-09-05T07:00:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 msgr2=0x56332a252860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e750 tx=0x7f90e000e780 comp rx=0 tx=0).stop 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 shutdown_connections 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a265680 0x56332a267a70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a255b80 0x56332a252ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24c350 0x56332a252860 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 --2- 172.21.15.86:0/3568212387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a255560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 >> 172.21.15.86:0/3568212387 conn(0x56332a20b310 msgr2=0x56332a24d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90ceffd640 0 client.28544 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 shutdown_connections 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 1 -- 172.21.15.86:0/3568212387 wait complete. 2024-09-05T07:00:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.686+0000 7f90f7d98740 20 client.28544 trim_cache size 0 max 16384 2024-09-05T07:00:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.707+0000 7f90f7d98740 1 Processor -- start 2024-09-05T07:00:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.707+0000 7f90f7d98740 1 -- start start 2024-09-05T07:00:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a261b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 0x56332a24de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ad000 con 0x56332a24ade0 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01aeb40 con 0x56332a255b80 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90f7d98740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ad6d0 con 0x56332a24c350 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42860/0 (socket says 172.21.15.86:42860) 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a261b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90e7fff640 1 -- 172.21.15.86:0/2780134513 learned_addr learned my addr 172.21.15.86:0/2780134513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:42.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 0x56332a24de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2780134513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 msgr2=0x56332a24d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2780134513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 msgr2=0x56332a261b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a261b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.708+0000 7f90ef7fe640 1 -- 172.21.15.86:0/2780134513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56332a253230 con 0x56332a255b80 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.709+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 0x56332a24de70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90e80d07b0 tx=0x7f90e808a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6778d070bb9a14f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.709+0000 7f90e57fa640 1 -- 172.21.15.86:0/2780134513 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e809f580 con 0x56332a255b80 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.709+0000 7f90e57fa640 1 -- 172.21.15.86:0/2780134513 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e80a3930 con 0x56332a255b80 2024-09-05T07:00:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.709+0000 7f90e57fa640 1 -- 172.21.15.86:0/2780134513 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80d3120 con 0x56332a255b80 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 -- 172.21.15.86:0/2780134513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 msgr2=0x56332a24de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 0x56332a24de70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90e80d07b0 tx=0x7f90e808a040 comp rx=0 tx=0).stop 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 -- 172.21.15.86:0/2780134513 shutdown_connections 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a255b80 0x56332a24de70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a24d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2780134513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a24ade0 0x56332a261b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 -- 172.21.15.86:0/2780134513 >> 172.21.15.86:0/2780134513 conn(0x56332a20b310 msgr2=0x56332a251970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 -- 172.21.15.86:0/2780134513 shutdown_connections 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.710+0000 7f90f7d98740 1 -- 172.21.15.86:0/2780134513 wait complete. 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.711+0000 7f90f7d98740 1 -- start start 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.711+0000 7f90f7d98740 1 Processor -- start 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.711+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a25de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.711+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.711+0000 7f90f7d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a24f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90f7d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ae470 con 0x56332a255b80 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90f7d98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01adda0 con 0x56332a24ade0 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90f7d98740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ab000 con 0x56332a24c350 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90f7d98740 10 client.0 _ll_register_callbacks cb 0x7f90e002a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42870/0 (socket says 172.21.15.86:42870) 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 -- 172.21.15.86:0/2997598982 learned_addr learned my addr 172.21.15.86:0/2997598982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a24f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90eeffd640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a25de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 -- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 msgr2=0x56332a25de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a25de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 -- 172.21.15.86:0/2997598982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 msgr2=0x56332a24f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a24f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 -- 172.21.15.86:0/2997598982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56332a24f8c0 con 0x56332a24c350 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90ef7fe640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a24f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90e7fff640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90e0004bf0 tx=0x7f90e0009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c55ed772981a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.712+0000 7f90ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e002d040 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f90e000fe00 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e80a3930 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90e004f050 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90f7d98740 10 client.28588 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.713+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56332a2cfbf0 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e80a3930 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f90e00050f0 con 0x56332a24c350 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 10 client.28588 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 10 client.28588 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:00:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 20 client.28588 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a2d0280 0x56332a2d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 4 client.28588 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:00:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.714+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56332a1b5860 con 0x56332a2d0280 2024-09-05T07:00:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90eeffd640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a2d0280 0x56332a2d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90eeffd640 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a2d0280 0x56332a2d26a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dd5eaecb70da02d server_cookie=b18a160f7b371248 in_seq=0 out_seq=0 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90ceffd640 10 client.28588 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90ceffd640 1 -- 172.21.15.86:0/2997598982 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56332a1b5860 con 0x56332a2d0280 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90ceffd640 10 client.28588 handle_command_reply: tid=0 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.715+0000 7f90f7d98740 1 client.28588 shutdown 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 msgr2=0x56332a25e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90e0004bf0 tx=0x7f90e0009e20 comp rx=0 tx=0).stop 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 shutdown_connections 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56332a2d0280 0x56332a2d26a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56332a255b80 0x56332a24f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56332a24c350 0x56332a25e3b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 --2- 172.21.15.86:0/2997598982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56332a24ade0 0x56332a25de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90ceffd640 0 client.28588 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 >> 172.21.15.86:0/2997598982 conn(0x56332a20b310 msgr2=0x56332a23d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 shutdown_connections 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 1 -- 172.21.15.86:0/2997598982 wait complete. 2024-09-05T07:00:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.716+0000 7f90f7d98740 20 client.28588 trim_cache size 0 max 16384 2024-09-05T07:00:42.722 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:00:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 msgr2=0x7f90dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 0x7f90dc0a5230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90e0023000 tx=0x7f90e0020000 comp rx=0 tx=0).stop 2024-09-05T07:00:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 msgr2=0x7f90f019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007db0 tx=0x7f90e8010420 comp rx=0 tx=0).stop 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 shutdown_connections 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f90dc0a2d70 0x7f90dc0a5230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f010c170 0x7f90f019cb90 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f01036b0 0x7f90f0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 --2- 172.21.15.86:0/2409748566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f0102d90 0x7f90f01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.717+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 >> 172.21.15.86:0/2409748566 conn(0x7f90f00fc7d0 msgr2=0x7f90f00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.718+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 shutdown_connections 2024-09-05T07:00:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:42.718+0000 7f90f59bf640 1 -- 172.21.15.86:0/2409748566 wait complete. 2024-09-05T07:00:42.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46176 6.93 MB/sec execute 240 sec latency 97.225 ms 2024-09-05T07:00:43.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:43 smithi086 ceph-mon[30943]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 335 op/s 2024-09-05T07:00:43.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:00:43.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:43 smithi115 ceph-mon[34791]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 335 op/s 2024-09-05T07:00:43.758 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T07:00:43.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:00:43.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46373 6.95 MB/sec execute 241 sec latency 71.363 ms 2024-09-05T07:00:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:43 smithi096 ceph-mon[34468]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 335 op/s 2024-09-05T07:00:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 -- 172.21.15.86:0/3004330231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830103ff0 msgr2=0x7f88301043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 --2- 172.21.15.86:0/3004330231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830103ff0 0x7f88301043d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8820009900 tx=0x7f882002a3f0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 -- 172.21.15.86:0/3004330231 shutdown_connections 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 --2- 172.21.15.86:0/3004330231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8830109120 0x7f8830110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 --2- 172.21.15.86:0/3004330231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88301049a0 0x7f88301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 --2- 172.21.15.86:0/3004330231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830103ff0 0x7f88301043d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 -- 172.21.15.86:0/3004330231 >> 172.21.15.86:0/3004330231 conn(0x7f88300ffab0 msgr2=0x7f8830101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 -- 172.21.15.86:0/3004330231 shutdown_connections 2024-09-05T07:00:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.425+0000 7f88369e1640 1 -- 172.21.15.86:0/3004330231 wait complete. 2024-09-05T07:00:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.426+0000 7f88369e1640 1 Processor -- start 2024-09-05T07:00:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.426+0000 7f88369e1640 1 -- start start 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.426+0000 7f88369e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8830103ff0 0x7f883019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.426+0000 7f88369e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f88369e1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830109120 0x7f883019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f88369e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8830114040 con 0x7f88301049a0 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f88369e1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8830113ec0 con 0x7f8830103ff0 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f88369e1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88301141c0 con 0x7f8830109120 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37234/0 (socket says 172.21.15.86:37234) 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8830103ff0 0x7f883019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 -- 172.21.15.86:0/3953418131 learned_addr learned my addr 172.21.15.86:0/3953418131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f8834f7e640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830109120 0x7f883019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 -- 172.21.15.86:0/3953418131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830109120 msgr2=0x7f883019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830109120 0x7f883019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 -- 172.21.15.86:0/3953418131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8830103ff0 msgr2=0x7f883019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8830103ff0 0x7f883019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 -- 172.21.15.86:0/3953418131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019f4f0 con 0x7f88301049a0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.427+0000 7f882f7fe640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f882400e9f0 tx=0x7f882400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.428+0000 7f882d7fa640 1 -- 172.21.15.86:0/3953418131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882400cd80 con 0x7f88301049a0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.428+0000 7f882d7fa640 1 -- 172.21.15.86:0/3953418131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88240054d0 con 0x7f88301049a0 2024-09-05T07:00:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.428+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019f7e0 con 0x7f88301049a0 2024-09-05T07:00:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.429+0000 7f882d7fa640 1 -- 172.21.15.86:0/3953418131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824010430 con 0x7f88301049a0 2024-09-05T07:00:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.429+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301a70c0 con 0x7f88301049a0 2024-09-05T07:00:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.431+0000 7f882d7fa640 1 -- 172.21.15.86:0/3953418131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8824005020 con 0x7f88301049a0 2024-09-05T07:00:44.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.432+0000 7f882d7fa640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 0x7f88180a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.432+0000 7f882d7fa640 1 -- 172.21.15.86:0/3953418131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f88240cfdc0 con 0x7f88301049a0 2024-09-05T07:00:44.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.432+0000 7f882ffff640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 0x7f88180a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.432+0000 7f8838dba740 1 Processor -- start 2024-09-05T07:00:44.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.433+0000 7f8838dba740 1 -- start start 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.433+0000 7f8838dba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777f1350 0x55a5777ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.433+0000 7f8838dba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5777f0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.433+0000 7f8838dba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8838dba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a57774d2f0 con 0x55a5777efde0 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8838dba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a5776f4cf0 con 0x55a5777f1350 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8838dba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a5777492b0 con 0x55a5777fab80 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5777f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42892/0 (socket says 172.21.15.86:42892) 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 -- 172.21.15.86:0/1526792499 learned_addr learned my addr 172.21.15.86:0/1526792499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:44.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 -- 172.21.15.86:0/1526792499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777f1350 msgr2=0x55a5777ef8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777f1350 0x55a5777ef8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 -- 172.21.15.86:0/1526792499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 msgr2=0x55a5777f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5777f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 -- 172.21.15.86:0/1526792499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5776d93a0 con 0x55a5777fab80 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f882f7fe640 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5777f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.434+0000 7f8834f7e640 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f40 tx=0x7f8828010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff6cca7c811e579 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.435+0000 7f882ffff640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 0x7f88180a52b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820031400 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.435+0000 7f88167fc640 1 -- 172.21.15.86:0/1526792499 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828003b70 con 0x55a5777fab80 2024-09-05T07:00:44.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.435+0000 7f88167fc640 1 -- 172.21.15.86:0/1526792499 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8828003d10 con 0x55a5777fab80 2024-09-05T07:00:44.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.435+0000 7f88167fc640 1 -- 172.21.15.86:0/1526792499 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280119a0 con 0x55a5777fab80 2024-09-05T07:00:44.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 -- 172.21.15.86:0/1526792499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 msgr2=0x55a5777fcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f40 tx=0x7f8828010860 comp rx=0 tx=0).stop 2024-09-05T07:00:44.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 -- 172.21.15.86:0/1526792499 shutdown_connections 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777fab80 0x55a5777fcf70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5777f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 --2- 172.21.15.86:0/1526792499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777f1350 0x55a5777ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.436+0000 7f8838dba740 1 -- 172.21.15.86:0/1526792499 >> 172.21.15.86:0/1526792499 conn(0x55a5777b0310 msgr2=0x55a5777ec470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.437+0000 7f8838dba740 1 -- 172.21.15.86:0/1526792499 shutdown_connections 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.437+0000 7f8838dba740 1 -- 172.21.15.86:0/1526792499 wait complete. 2024-09-05T07:00:44.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.437+0000 7f8838dba740 1 -- start start 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.437+0000 7f8838dba740 1 Processor -- start 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 0x55a5777f7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777fab80 0x55a5777f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 0x55a5777f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 0x55a5777f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55766/0 (socket says 172.21.15.86:55766) 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882ffff640 1 -- 172.21.15.86:0/1857657069 learned_addr learned my addr 172.21.15.86:0/1857657069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5777fd880 con 0x55a5777fab80 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a577708060 con 0x55a5777efde0 2024-09-05T07:00:44.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a5777fda00 con 0x55a5777f1350 2024-09-05T07:00:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f8838dba740 10 client.0 _ll_register_callbacks cb 0x55a57745b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 -- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 msgr2=0x55a5777f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 0x55a5777f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 -- 172.21.15.86:0/1857657069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777fab80 msgr2=0x55a5777f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777fab80 0x55a5777f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.438+0000 7f882f7fe640 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5774be3c0 con 0x55a5777f1350 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.439+0000 7f882f7fe640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f7e70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88240cd070 tx=0x7f8824005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df25f6a96e357ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.439+0000 7f880f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.440+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882409e030 con 0x55a5777f1350 2024-09-05T07:00:44.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.440+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f88240cd2f0 con 0x55a5777f1350 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.440+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8828003d10 con 0x55a5777f1350 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.441+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f88240cd4f0 con 0x55a5777f1350 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.441+0000 7f8838dba740 10 client.28600 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.441+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a5778093c0 con 0x55a5777f1350 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.441+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f882409be40 con 0x55a5777f1350 2024-09-05T07:00:44.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.441+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f8824080050 con 0x55a5777f1350 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 10 client.28600 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 10 client.28600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 20 client.28600 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a57780cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 4 client.28600 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8828003d10 con 0x55a57780a820 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.442+0000 7f882ffff640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a57780cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.443+0000 7f880f7fe640 10 client.28600 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:00:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.443+0000 7f882ffff640 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a57780cc70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b69959e741583be9 server_cookie=f00527a404ca29be in_seq=0 out_seq=0 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f880f7fe640 1 -- 172.21.15.86:0/1857657069 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8828003d10 con 0x55a57780a820 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f880f7fe640 10 client.28600 handle_command_reply: tid=0 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 client.28600 shutdown 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 msgr2=0x55a5777f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f7e70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88240cd070 tx=0x7f8824005450 comp rx=0 tx=0).stop 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 shutdown_connections 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a57780cc70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f880f7fe640 0 client.28600 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777fab80 0x55a5777f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f7e70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 --2- 172.21.15.86:0/1857657069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777efde0 0x55a5777f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 >> 172.21.15.86:0/1857657069 conn(0x55a5777b0310 msgr2=0x55a5777f2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 shutdown_connections 2024-09-05T07:00:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.444+0000 7f8838dba740 1 -- 172.21.15.86:0/1857657069 wait complete. 2024-09-05T07:00:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.445+0000 7f8838dba740 20 client.28600 trim_cache size 0 max 16384 2024-09-05T07:00:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 Processor -- start 2024-09-05T07:00:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 -- start start 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 0x55a577812540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 0x55a577812a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88301acfa0 con 0x55a5777f1350 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88301aeae0 con 0x55a5777fab80 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f8838dba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88301ad670 con 0x55a5777efde0 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 0x55a577812a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f882ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 0x55a577812540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f882ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 0x55a577812540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42918/0 (socket says 172.21.15.86:42918) 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.465+0000 7f882ffff640 1 -- 172.21.15.86:0/3470320993 learned_addr learned my addr 172.21.15.86:0/3470320993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 -- 172.21.15.86:0/3470320993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 msgr2=0x55a577812540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 0x55a577812540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 -- 172.21.15.86:0/3470320993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 msgr2=0x55a577812a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 0x55a577812a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 -- 172.21.15.86:0/3470320993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5777fd2e0 con 0x55a5777fab80 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f882f7fe640 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 0x55a577812a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8834f7e640 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f1920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8828010d60 tx=0x7f8828009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90f37ad47e6abab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8815ffb640 1 -- 172.21.15.86:0/3470320993 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828004bb0 con 0x55a5777fab80 2024-09-05T07:00:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8815ffb640 1 -- 172.21.15.86:0/3470320993 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8828004d50 con 0x55a5777fab80 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.466+0000 7f8815ffb640 1 -- 172.21.15.86:0/3470320993 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828005490 con 0x55a5777fab80 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 -- 172.21.15.86:0/3470320993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 msgr2=0x55a5777f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f1920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8828010d60 tx=0x7f8828009c00 comp rx=0 tx=0).stop 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 -- 172.21.15.86:0/3470320993 shutdown_connections 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f1920 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777f1350 0x55a577812a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 --2- 172.21.15.86:0/3470320993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777efde0 0x55a577812540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.467+0000 7f8838dba740 1 -- 172.21.15.86:0/3470320993 >> 172.21.15.86:0/3470320993 conn(0x55a5777b0310 msgr2=0x55a5777f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.468+0000 7f8838dba740 1 -- 172.21.15.86:0/3470320993 shutdown_connections 2024-09-05T07:00:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.468+0000 7f8838dba740 1 -- 172.21.15.86:0/3470320993 wait complete. 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.468+0000 7f8838dba740 1 -- start start 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.468+0000 7f8838dba740 1 Processor -- start 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5778032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88301ae410 con 0x55a5777efde0 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f882ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5778032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88301add40 con 0x55a5777fab80 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f882ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5778032c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37270/0 (socket says 172.21.15.86:37270) 2024-09-05T07:00:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f882ffff640 1 -- 172.21.15.86:0/2387847075 learned_addr learned my addr 172.21.15.86:0/2387847075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:42926/0 (socket says 172.21.15.86:42926) 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8834f7e640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88301aafa0 con 0x55a5777f1350 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.469+0000 7f8838dba740 10 client.0 _ll_register_callbacks cb 0x7f882400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 -- 172.21.15.86:0/2387847075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 msgr2=0x55a5777f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 -- 172.21.15.86:0/2387847075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 msgr2=0x55a5778032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5778032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 -- 172.21.15.86:0/2387847075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5777f5160 con 0x55a5777fab80 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8834f7e640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f4a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8828026040 tx=0x7f8828011c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fee6df93d3c97b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f880f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828004bb0 con 0x55a5777fab80 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f8828021040 con 0x55a5777fab80 2024-09-05T07:00:44.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8828004d50 con 0x55a5777fab80 2024-09-05T07:00:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f882803a050 con 0x55a5777fab80 2024-09-05T07:00:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.470+0000 7f8838dba740 10 client.28562 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:44.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a5778753d0 con 0x55a5777fab80 2024-09-05T07:00:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8828004d50 con 0x55a5777fab80 2024-09-05T07:00:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f882802e640 con 0x55a5777fab80 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 10 client.28562 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 10 client.28562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 20 client.28562 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a577877860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 4 client.28562 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f882ffff640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a577877860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.471+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a57775a860 con 0x55a57780a820 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f880f7fe640 10 client.28562 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f882ffff640 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a577877860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c352848bafc1339 server_cookie=6af3a5eb1856ece1 in_seq=0 out_seq=0 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f880f7fe640 1 -- 172.21.15.86:0/2387847075 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a57775a860 con 0x55a57780a820 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f880f7fe640 10 client.28562 handle_command_reply: tid=0 2024-09-05T07:00:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f8838dba740 1 client.28562 shutdown 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 msgr2=0x55a5777f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f4a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8828026040 tx=0x7f8828011c10 comp rx=0 tx=0).stop 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 shutdown_connections 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.472+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55a57780a820 0x55a577877860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a5777fab80 0x55a5777f4a70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5777f1350 0x55a5777f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 --2- 172.21.15.86:0/2387847075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5777efde0 0x55a5778032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f880f7fe640 0 client.28562 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 >> 172.21.15.86:0/2387847075 conn(0x55a5777b0310 msgr2=0x55a5777f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 shutdown_connections 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 1 -- 172.21.15.86:0/2387847075 wait complete. 2024-09-05T07:00:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.473+0000 7f8838dba740 20 client.28562 trim_cache size 0 max 16384 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 msgr2=0x7f88180a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 0x7f88180a52b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820031400 comp rx=0 tx=0).stop 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 msgr2=0x7f883019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f882400e9f0 tx=0x7f882400edf0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 shutdown_connections 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f88180a2df0 0x7f88180a52b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8830109120 0x7f883019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88301049a0 0x7f883019aa80 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 --2- 172.21.15.86:0/3953418131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8830103ff0 0x7f883019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 >> 172.21.15.86:0/3953418131 conn(0x7f88300ffab0 msgr2=0x7f8830100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 shutdown_connections 2024-09-05T07:00:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:44.474+0000 7f88369e1640 1 -- 172.21.15.86:0/3953418131 wait complete. 2024-09-05T07:00:44.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46553 6.97 MB/sec execute 242 sec latency 101.107 ms 2024-09-05T07:00:45.630 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:45 smithi096 ceph-mon[34468]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 240 op/s 2024-09-05T07:00:45.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:45 smithi115 ceph-mon[34791]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 240 op/s 2024-09-05T07:00:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:45 smithi086 ceph-mon[30943]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 240 op/s 2024-09-05T07:00:45.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46640 6.95 MB/sec execute 243 sec latency 511.062 ms 2024-09-05T07:00:46.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46742 6.92 MB/sec execute 244 sec latency 522.181 ms 2024-09-05T07:00:47.633 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:47 smithi086 ceph-mon[30943]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-05T07:00:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:47 smithi115 ceph-mon[34791]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-05T07:00:47.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46912 6.93 MB/sec execute 245 sec latency 131.330 ms 2024-09-05T07:00:47.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:47 smithi096 ceph-mon[34468]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-05T07:00:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.737+0000 7fd2f1391640 1 -- 172.21.15.86:0/2742447791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 msgr2=0x7fd2ec10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.737+0000 7fd2f1391640 1 --2- 172.21.15.86:0/2742447791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec10bbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009900 tx=0x7fd2e002a480 comp rx=0 tx=0).stop 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 -- 172.21.15.86:0/2742447791 shutdown_connections 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 --2- 172.21.15.86:0/2742447791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 0x7fd2ec10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 --2- 172.21.15.86:0/2742447791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec10bbb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 --2- 172.21.15.86:0/2742447791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 -- 172.21.15.86:0/2742447791 >> 172.21.15.86:0/2742447791 conn(0x7fd2ec0fc7a0 msgr2=0x7fd2ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 -- 172.21.15.86:0/2742447791 shutdown_connections 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.738+0000 7fd2f1391640 1 -- 172.21.15.86:0/2742447791 wait complete. 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 Processor -- start 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 -- start start 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:48.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 0x7fd2ec19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec111e10 con 0x7fd2ec102da0 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec111c90 con 0x7fd2ec10c180 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2f1391640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec111f90 con 0x7fd2ec1036c0 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37282/0 (socket says 172.21.15.86:37282) 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2eaffd640 1 -- 172.21.15.86:0/3600835237 learned_addr learned my addr 172.21.15.86:0/3600835237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.739+0000 7fd2eb7fe640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 0x7fd2ec19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2ea7fc640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 -- 172.21.15.86:0/3600835237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 msgr2=0x7fd2ec198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 -- 172.21.15.86:0/3600835237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 msgr2=0x7fd2ec19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 0x7fd2ec19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 -- 172.21.15.86:0/3600835237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec19d300 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2eaffd640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00ea60 tx=0x7fd2dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00cde0 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2dc0035f0 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec19d5f0 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc010690 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.740+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec100930 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.742+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec103180 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.743+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd2dc0037a0 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.744+0000 7fd2d3fff640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 0x7fd2d40a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.744+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7fd2dc0cfb30 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.744+0000 7fd2ea7fc640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 0x7fd2d40a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.745+0000 7fd2ea7fc640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 0x7fd2d40a9640 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0031440 tx=0x7fd2e00095f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:47.750+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2dc09c310 con 0x7fd2ec102da0 2024-09-05T07:00:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.048+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2ec1071c0 con 0x7fd2ec102da0 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.051+0000 7fd2d3fff640 1 -- 172.21.15.86:0/3600835237 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7fd2dc09bc10 con 0x7fd2ec102da0 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.057+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 msgr2=0x7fd2d40a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.057+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 0x7fd2d40a9640 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0031440 tx=0x7fd2e00095f0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.057+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 msgr2=0x7fd2ec198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.057+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00ea60 tx=0x7fd2dc00ef20 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 shutdown_connections 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2d40a7180 0x7fd2d40a9640 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2ec10c180 0x7fd2ec19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2ec1036c0 0x7fd2ec198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 --2- 172.21.15.86:0/3600835237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2ec102da0 0x7fd2ec198350 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:48.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.058+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 >> 172.21.15.86:0/3600835237 conn(0x7fd2ec0fc7a0 msgr2=0x7fd2ec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:48.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.059+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 shutdown_connections 2024-09-05T07:00:48.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:48.059+0000 7fd2f1391640 1 -- 172.21.15.86:0/3600835237 wait complete. 2024-09-05T07:00:48.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3600835237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:48.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3600835237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:48.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47055 6.94 MB/sec execute 246 sec latency 142.294 ms 2024-09-05T07:00:48.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3600835237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:00:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:49 smithi115 ceph-mon[34791]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.5 MiB/s wr, 333 op/s 2024-09-05T07:00:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:49 smithi086 ceph-mon[30943]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.5 MiB/s wr, 333 op/s 2024-09-05T07:00:49.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47225 6.95 MB/sec execute 247 sec latency 240.852 ms 2024-09-05T07:00:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:49 smithi096 ceph-mon[34468]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.5 MiB/s wr, 333 op/s 2024-09-05T07:00:50.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47368 6.95 MB/sec execute 248 sec latency 219.944 ms 2024-09-05T07:00:51.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:51 smithi115 ceph-mon[34791]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-05T07:00:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:51 smithi086 ceph-mon[30943]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-05T07:00:51.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47463 6.95 MB/sec execute 249 sec latency 282.018 ms 2024-09-05T07:00:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:51 smithi096 ceph-mon[34468]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-05T07:00:52.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47585 6.96 MB/sec execute 250 sec latency 148.835 ms 2024-09-05T07:00:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:53 smithi115 ceph-mon[34791]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2024-09-05T07:00:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:53 smithi086 ceph-mon[30943]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2024-09-05T07:00:53.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47773 6.98 MB/sec execute 251 sec latency 116.104 ms 2024-09-05T07:00:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:53 smithi096 ceph-mon[34468]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2024-09-05T07:00:54.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:54 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:00:54.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:54 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:00:54.766 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47955 7.00 MB/sec execute 252 sec latency 174.158 ms 2024-09-05T07:00:54.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:54 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:00:54.871 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T07:00:54.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T07:00:55.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:00:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:55.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:55 smithi086 ceph-mon[30943]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-05T07:00:55.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:55 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:00:55.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:55 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:00:55.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:55 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:00:55.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:55 smithi115 ceph-mon[34791]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-05T07:00:55.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:55 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:00:55.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:55 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:00:55.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:55 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:00:55.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48114 7.00 MB/sec execute 253 sec latency 87.182 ms 2024-09-05T07:00:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:55 smithi096 ceph-mon[34468]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-05T07:00:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:55 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:00:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:55 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:00:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:55 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:00:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.164+0000 7f07a0ec7640 1 -- 172.21.15.86:0/2920778287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 msgr2=0x7f079c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.164+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/2920778287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1067f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0790009900 tx=0x7f079002a410 comp rx=0 tx=0).stop 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 -- 172.21.15.86:0/2920778287 shutdown_connections 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/2920778287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 0x7f079c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/2920778287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1067f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/2920778287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 -- 172.21.15.86:0/2920778287 >> 172.21.15.86:0/2920778287 conn(0x7f079c0fd8f0 msgr2=0x7f079c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 -- 172.21.15.86:0/2920778287 shutdown_connections 2024-09-05T07:00:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.165+0000 7f07a0ec7640 1 -- 172.21.15.86:0/2920778287 wait complete. 2024-09-05T07:00:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.166+0000 7f07a0ec7640 1 Processor -- start 2024-09-05T07:00:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.166+0000 7f07a0ec7640 1 -- start start 2024-09-05T07:00:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.166+0000 7f07a0ec7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f07a0ec7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f07a0ec7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 0x7f079c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55570/0 (socket says 172.21.15.86:55570) 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 -- 172.21.15.86:0/638304480 learned_addr learned my addr 172.21.15.86:0/638304480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f079c111df0 con 0x7f079c1027a0 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079ad9d640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 0x7f079c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f079c111c70 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f0799d9b640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f079c111f70 con 0x7f079c106f20 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 -- 172.21.15.86:0/638304480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 msgr2=0x7f079c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 0x7f079c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 -- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 msgr2=0x7f079c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c19d360 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.167+0000 7f079a59c640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ea60 tx=0x7f078c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.168+0000 7f078b7fe640 1 -- 172.21.15.86:0/638304480 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c00cd20 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.168+0000 7f078b7fe640 1 -- 172.21.15.86:0/638304480 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f078c0035f0 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.168+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c19d650 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.168+0000 7f078b7fe640 1 -- 172.21.15.86:0/638304480 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c010690 con 0x7f079c101df0 2024-09-05T07:00:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.168+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c074420 con 0x7f079c101df0 2024-09-05T07:00:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.171+0000 7f078b7fe640 1 -- 172.21.15.86:0/638304480 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f078c004020 con 0x7f079c101df0 2024-09-05T07:00:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.171+0000 7f078b7fe640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 0x7f07840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f078b7fe640 1 -- 172.21.15.86:0/638304480 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f078c0cfa90 con 0x7f079c101df0 2024-09-05T07:00:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f0799d9b640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 0x7f07840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 Processor -- start 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 -- start start 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f0799d9b640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 0x7f07840a5230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0790031440 tx=0x7f079003c040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cdcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561eb2c2f2a0 con 0x561eb2cd1f70 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561eb2bd6b10 con 0x561eb2cd2890 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.172+0000 7f07a32a0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561eb2c2b260 con 0x561eb2cdd150 2024-09-05T07:00:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38888/0 (socket says 172.21.15.86:38888) 2024-09-05T07:00:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 -- 172.21.15.86:0/405336276 learned_addr learned my addr 172.21.15.86:0/405336276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51208/0 (socket says 172.21.15.86:51208) 2024-09-05T07:00:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f0799d9b640 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cdcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 -- 172.21.15.86:0/405336276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 msgr2=0x561eb2cdf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 -- 172.21.15.86:0/405336276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 msgr2=0x561eb2cdcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cdcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f0799d9b640 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cdcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 -- 172.21.15.86:0/405336276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb29a0b90 con 0x561eb2cd1f70 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f079a59c640 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f078c098540 tx=0x7f078c09b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c743fe19e4b21e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f0788ff9640 1 -- 172.21.15.86:0/405336276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c004310 con 0x561eb2cd1f70 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f0788ff9640 1 -- 172.21.15.86:0/405336276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f078c09c270 con 0x561eb2cd1f70 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.173+0000 7f0788ff9640 1 -- 172.21.15.86:0/405336276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0037a0 con 0x561eb2cd1f70 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.174+0000 7f07a32a0740 1 -- 172.21.15.86:0/405336276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 msgr2=0x561eb2cd2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.174+0000 7f07a32a0740 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f078c098540 tx=0x7f078c09b770 comp rx=0 tx=0).stop 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 -- 172.21.15.86:0/405336276 shutdown_connections 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cdcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 --2- 172.21.15.86:0/405336276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2cd2350 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 -- 172.21.15.86:0/405336276 >> 172.21.15.86:0/405336276 conn(0x561eb2cce500 msgr2=0x561eb2cce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 -- 172.21.15.86:0/405336276 shutdown_connections 2024-09-05T07:00:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 -- 172.21.15.86:0/405336276 wait complete. 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.175+0000 7f07a32a0740 1 -- start start 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.176+0000 7f07a32a0740 1 Processor -- start 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.176+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.176+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f0799d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f0799d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55598/0 (socket says 172.21.15.86:55598) 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f079a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f0799d9b640 1 -- 172.21.15.86:0/1339744222 learned_addr learned my addr 172.21.15.86:0/1339744222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f079ad9d640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561eb2cc5960 con 0x561eb2cd1f70 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561eb2bb76e0 con 0x561eb2cd2890 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561eb2cc5ae0 con 0x561eb2cdd150 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.177+0000 7f07a32a0740 10 client.0 _ll_register_callbacks cb 0x561eb2bdba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 -- 172.21.15.86:0/1339744222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 msgr2=0x561eb2cdbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 -- 172.21.15.86:0/1339744222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 msgr2=0x561eb2ccb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb2cdc140 con 0x561eb2cd2890 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f079ad9d640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f079a59c640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0799d9b640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0790009900 tx=0x7f0790009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152825cf28f43230 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:00:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c09c270 con 0x561eb2cd2890 2024-09-05T07:00:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0781ffb640 1 -- 172.21.15.86:0/1339744222 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790041070 con 0x561eb2cd2890 2024-09-05T07:00:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.178+0000 7f0781ffb640 1 -- 172.21.15.86:0/1339744222 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f0790004450 con 0x561eb2cd2890 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f0781ffb640 1 -- 172.21.15.86:0/1339744222 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0790059050 con 0x561eb2cd2890 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 10 client.28580 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=45}) -- 0x561eb2cf4360 con 0x561eb2cd2890 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f0781ffb640 1 -- 172.21.15.86:0/1339744222 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0790005040 con 0x561eb2cd2890 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 10 client.28580 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 10 client.28580 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 20 client.28580 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cf7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 4 client.28580 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.179+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb2c3c9d0 con 0x561eb2cf5710 2024-09-05T07:00:56.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.180+0000 7f079a59c640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cf7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.180+0000 7f079a59c640 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cf7b00 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d547b96d5aa20d6a server_cookie=35671b6532d395eb in_seq=0 out_seq=0 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.180+0000 7f0781ffb640 10 client.28580 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.181+0000 7f0781ffb640 1 -- 172.21.15.86:0/1339744222 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561eb2c3c9d0 con 0x561eb2cf5710 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f0781ffb640 10 client.28580 handle_command_reply: tid=0 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 client.28580 shutdown 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 msgr2=0x561eb2ccb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0790009900 tx=0x7f0790009260 comp rx=0 tx=0).stop 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 shutdown_connections 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cf7b00 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cdbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2ccb9e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 --2- 172.21.15.86:0/1339744222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f0781ffb640 0 client.28580 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 >> 172.21.15.86:0/1339744222 conn(0x561eb2cce500 msgr2=0x561eb2cd4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 shutdown_connections 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 1 -- 172.21.15.86:0/1339744222 wait complete. 2024-09-05T07:00:56.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.182+0000 7f07a32a0740 20 client.28580 trim_cache size 0 max 16384 2024-09-05T07:00:56.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.212+0000 7f07a32a0740 1 Processor -- start 2024-09-05T07:00:56.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.212+0000 7f07a32a0740 1 -- start start 2024-09-05T07:00:56.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cfd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cfd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ad050 con 0x561eb2cd1f70 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1aeb90 con 0x561eb2cd2890 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f07a32a0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ad720 con 0x561eb2cdd150 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f079a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f079a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38904/0 (socket says 172.21.15.86:38904) 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f079a59c640 1 -- 172.21.15.86:0/2601860228 learned_addr learned my addr 172.21.15.86:0/2601860228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f0799d9b640 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cfd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f079a59c640 1 -- 172.21.15.86:0/2601860228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 msgr2=0x561eb2cfd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.213+0000 7f079a59c640 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cfd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f079a59c640 1 -- 172.21.15.86:0/2601860228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 msgr2=0x561eb2cfd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f079a59c640 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cfd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f079a59c640 1 -- 172.21.15.86:0/2601860228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb2cd26c0 con 0x561eb2cd1f70 2024-09-05T07:00:56.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f079a59c640 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f078c0cd3a0 tx=0x7f078c09b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1567ee725f43a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f0783fff640 1 -- 172.21.15.86:0/2601860228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c07e070 con 0x561eb2cd1f70 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f0783fff640 1 -- 172.21.15.86:0/2601860228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f078c004310 con 0x561eb2cd1f70 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.214+0000 7f0783fff640 1 -- 172.21.15.86:0/2601860228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c099f20 con 0x561eb2cd1f70 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 -- 172.21.15.86:0/2601860228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 msgr2=0x561eb2ccd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f078c0cd3a0 tx=0x7f078c09b7e0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 -- 172.21.15.86:0/2601860228 shutdown_connections 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cdd150 0x561eb2cfd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cd2890 0x561eb2cfd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 --2- 172.21.15.86:0/2601860228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd1f70 0x561eb2ccd230 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.215+0000 7f07a32a0740 1 -- 172.21.15.86:0/2601860228 >> 172.21.15.86:0/2601860228 conn(0x561eb2cce500 msgr2=0x561eb2cccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 -- 172.21.15.86:0/2601860228 shutdown_connections 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 -- 172.21.15.86:0/2601860228 wait complete. 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 -- start start 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 Processor -- start 2024-09-05T07:00:56.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.216+0000 7f07a32a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd2890 0x561eb2cd7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f079a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f079a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51228/0 (socket says 172.21.15.86:51228) 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f079a59c640 1 -- 172.21.15.86:0/705082007 learned_addr learned my addr 172.21.15.86:0/705082007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 0x561eb2ce8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ae4c0 con 0x561eb2cd2890 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1addf0 con 0x561eb2cdd150 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1ab050 con 0x561eb2cd1f70 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f07a32a0740 10 client.0 _ll_register_callbacks cb 0x7f0790032e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f079ad9d640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 0x561eb2ce8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f0799d9b640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd2890 0x561eb2cd7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.217+0000 7f079a59c640 1 -- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 msgr2=0x561eb2ce8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079a59c640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 0x561eb2ce8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079a59c640 1 -- 172.21.15.86:0/705082007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd2890 msgr2=0x561eb2cd7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079a59c640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd2890 0x561eb2cd7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079a59c640 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb2ce8fe0 con 0x561eb2cd1f70 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079ad9d640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 0x561eb2ce8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f079a59c640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f078c097f80 tx=0x7f078c003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78566a2301749da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f0781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0a8070 con 0x561eb2cd1f70 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f078c0cd020 con 0x561eb2cd1f70 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561eb2c36d20 con 0x561eb2cd1f70 2024-09-05T07:00:56.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f078c07e070 con 0x561eb2cd1f70 2024-09-05T07:00:56.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.218+0000 7f07a32a0740 10 client.28624 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561eb2cea2a0 con 0x561eb2cd1f70 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561eb2c36d20 con 0x561eb2cd1f70 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f078c0ba2b0 con 0x561eb2cd1f70 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 10 client.28624 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 10 client.28624 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:00:56.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 20 client.28624 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:56.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 4 client.28624 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T07:00:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x561eb2bb2ec0 con 0x561eb2cf5710 2024-09-05T07:00:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.219+0000 7f0799d9b640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.220+0000 7f0781ffb640 10 client.28624 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.220+0000 7f0799d9b640 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cec5f0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc06f3286895b18 server_cookie=28043422b7e565d1 in_seq=0 out_seq=0 2024-09-05T07:00:56.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.450+0000 7f0781ffb640 1 -- 172.21.15.86:0/705082007 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561eb2bb2ec0 con 0x561eb2cf5710 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.450+0000 7f0781ffb640 10 client.28624 handle_command_reply: tid=0 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 client.28624 shutdown 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 msgr2=0x561eb2cd6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f078c097f80 tx=0x7f078c003ea0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 shutdown_connections 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561eb2cf5710 0x561eb2cec5f0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb2cdd150 0x561eb2ce8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561eb2cd2890 0x561eb2cd7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 --2- 172.21.15.86:0/705082007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb2cd1f70 0x561eb2cd6e50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f0781ffb640 0 client.28624 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 >> 172.21.15.86:0/705082007 conn(0x561eb2cce500 msgr2=0x561eb2cc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 shutdown_connections 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 1 -- 172.21.15.86:0/705082007 wait complete. 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.451+0000 7f07a32a0740 20 client.28624 trim_cache size 0 max 16384 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:00:56.454 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 70, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "a17ee4b7": { 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 70, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:00:56.455 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:00:56.456 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 msgr2=0x7f07840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 0x7f07840a5230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0790031440 tx=0x7f079003c040 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 msgr2=0x7f079c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ea60 tx=0x7f078c00ee60 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 shutdown_connections 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f07840a2d70 0x7f07840a5230 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c106f20 0x7f079c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f079c1027a0 0x7f079c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 --2- 172.21.15.86:0/638304480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f079c101df0 0x7f079c1983b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.452+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 >> 172.21.15.86:0/638304480 conn(0x7f079c0fd8f0 msgr2=0x7f079c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.453+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 shutdown_connections 2024-09-05T07:00:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.453+0000 7f07a0ec7640 1 -- 172.21.15.86:0/638304480 wait complete. 2024-09-05T07:00:56.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.98 MB/sec execute 254 sec latency 465.900 ms 2024-09-05T07:00:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 -- 172.21.15.86:0/1920784878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b81037a0 msgr2=0x7f57b810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/1920784878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b81037a0 0x7f57b810bc00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009900 tx=0x7f57a002a410 comp rx=0 tx=0).stop 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 -- 172.21.15.86:0/1920784878 shutdown_connections 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/1920784878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810c2f0 0x7f57b810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/1920784878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b81037a0 0x7f57b810bc00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/1920784878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b8102df0 0x7f57b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 -- 172.21.15.86:0/1920784878 >> 172.21.15.86:0/1920784878 conn(0x7f57b80fc7d0 msgr2=0x7f57b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 -- 172.21.15.86:0/1920784878 shutdown_connections 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 -- 172.21.15.86:0/1920784878 wait complete. 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.981+0000 7f57bd4bc640 1 Processor -- start 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 -- start start 2024-09-05T07:00:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b8102df0 0x7f57b8100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b81037a0 0x7f57b8100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8111f60 con 0x7f57b810c2f0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8111de0 con 0x7f57b81037a0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57bd4bc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81120e0 con 0x7f57b8102df0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38930/0 (socket says 172.21.15.86:38930) 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b8102df0 0x7f57b8100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 -- 172.21.15.86:0/112709322 learned_addr learned my addr 172.21.15.86:0/112709322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b67fc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b81037a0 0x7f57b8100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 -- 172.21.15.86:0/112709322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b8102df0 msgr2=0x7f57b8100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b8102df0 0x7f57b8100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 -- 172.21.15.86:0/112709322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b81037a0 msgr2=0x7f57b8100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b81037a0 0x7f57b8100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.982+0000 7f57b77fe640 1 -- 172.21.15.86:0/112709322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b81a69a0 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57b77fe640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f57b000cce0 tx=0x7f57b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57affff640 1 -- 172.21.15.86:0/112709322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b000f450 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57affff640 1 -- 172.21.15.86:0/112709322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b000faa0 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81a6c90 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57affff640 1 -- 172.21.15.86:0/112709322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0018b70 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.983+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a71a0 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.985+0000 7f57affff640 1 -- 172.21.15.86:0/112709322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f57b0018d10 con 0x7f57b810c2f0 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 Processor -- start 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 -- start start 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574d1ca32f0 con 0x5574d1d3d5d0 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574d1c4acf0 con 0x5574d1d3eb40 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57bf895740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574d1c9f2b0 con 0x5574d1d48370 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38940/0 (socket says 172.21.15.86:38940) 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 -- 172.21.15.86:0/3555348957 learned_addr learned my addr 172.21.15.86:0/3555348957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57affff640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 0x7f57a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 -- 172.21.15.86:0/3555348957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 msgr2=0x5574d1d4a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d4a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 -- 172.21.15.86:0/3555348957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 msgr2=0x5574d1d3d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d3d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b6ffd640 1 -- 172.21.15.86:0/3555348957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d1c2f3a0 con 0x5574d1d3d5d0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.986+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57b77fe640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 0x7f57a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57affff640 1 -- 172.21.15.86:0/112709322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f57b00cfb30 con 0x7f57b810c2f0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ac20 tx=0x7f57a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8248aae8d8abfba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57ad7fa640 1 -- 172.21.15.86:0/3555348957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a801d070 con 0x5574d1d3d5d0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57ad7fa640 1 -- 172.21.15.86:0/3555348957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a8018af0 con 0x5574d1d3d5d0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.987+0000 7f57ad7fa640 1 -- 172.21.15.86:0/3555348957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010af0 con 0x5574d1d3d5d0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- 172.21.15.86:0/3555348957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 msgr2=0x5574d1d3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57b77fe640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 0x7f57a40a5230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57b00cd440 tx=0x7f57b00c67a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ac20 tx=0x7f57a800f790 comp rx=0 tx=0).stop 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- 172.21.15.86:0/3555348957 shutdown_connections 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3d5d0 0x5574d1d3da10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 --2- 172.21.15.86:0/3555348957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- 172.21.15.86:0/3555348957 >> 172.21.15.86:0/3555348957 conn(0x5574d1d06310 msgr2=0x5574d1d39c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- 172.21.15.86:0/3555348957 shutdown_connections 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- 172.21.15.86:0/3555348957 wait complete. 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 -- start start 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.988+0000 7f57bf895740 1 Processor -- start 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d47d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 0x5574d1d44fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574d1d4b070 con 0x5574d1d48370 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574d1c5e060 con 0x5574d1d3d5d0 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574d1d4b1f0 con 0x5574d1d3eb40 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 10 client.0 _ll_register_callbacks cb 0x5574d19b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:57.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38948/0 (socket says 172.21.15.86:38948) 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 -- 172.21.15.86:0/3942093871 learned_addr learned my addr 172.21.15.86:0/3942093871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 0x5574d1d44fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d47d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 -- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 msgr2=0x5574d1d44fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 0x5574d1d44fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 -- 172.21.15.86:0/3942093871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 msgr2=0x5574d1d47d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d47d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 -- 172.21.15.86:0/3942093871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d1a143c0 con 0x5574d1d48370 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 0x5574d1d44fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f57a00096c0 tx=0x7f57a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a3fb5d0d08fdb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f579e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0035040 con 0x5574d1d48370 2024-09-05T07:00:57.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f57a0004230 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.989+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a8018af0 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f57a0051050 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 10 client.19800 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5574d1d56ac0 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a8018af0 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f57a004a060 con 0x5574d1d48370 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 10 client.19800 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 10 client.19800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 20 client.19800 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1d57f80 0x5574d1d5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 4 client.19800 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574d1cb0860 con 0x5574d1d57f80 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.990+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1d57f80 0x5574d1d5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f579e7fc640 10 client.19800 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:00:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1d57f80 0x5574d1d5a370 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44921bc81674acbf server_cookie=7dbbfebf8bb5acb8 in_seq=0 out_seq=0 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f579e7fc640 1 -- 172.21.15.86:0/3942093871 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574d1cb0860 con 0x5574d1d57f80 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f579e7fc640 10 client.19800 handle_command_reply: tid=0 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 client.19800 shutdown 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 msgr2=0x5574d1d45570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f57a00096c0 tx=0x7f57a002a410 comp rx=0 tx=0).stop 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 shutdown_connections 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1d57f80 0x5574d1d5a370 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d45570 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3eb40 0x5574d1d44fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 --2- 172.21.15.86:0/3942093871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d47d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f579e7fc640 0 client.19800 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 >> 172.21.15.86:0/3942093871 conn(0x5574d1d06310 msgr2=0x5574d1d3f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.991+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 shutdown_connections 2024-09-05T07:00:57.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.992+0000 7f57bf895740 1 -- 172.21.15.86:0/3942093871 wait complete. 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:56.992+0000 7f57bf895740 20 client.19800 trim_cache size 0 max 16384 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.004+0000 7f57bf895740 1 Processor -- start 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.004+0000 7f57bf895740 1 -- start start 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b1210 con 0x5574d1d3eb40 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b2d50 con 0x5574d1d3d5d0 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57bf895740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b18e0 con 0x5574d1d48370 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38956/0 (socket says 172.21.15.86:38956) 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 -- 172.21.15.86:0/972288013 learned_addr learned my addr 172.21.15.86:0/972288013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b67fc640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 -- 172.21.15.86:0/972288013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 msgr2=0x5574d1d40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d40120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 -- 172.21.15.86:0/972288013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 msgr2=0x5574d1d44890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b67fc640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d44890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 -- 172.21.15.86:0/972288013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d1d43f80 con 0x5574d1d3eb40 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b67fc640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d44890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57a800a9e0 tx=0x7f57a8010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f5bed337d3434 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57acff9640 1 -- 172.21.15.86:0/972288013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a801d040 con 0x5574d1d3eb40 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57acff9640 1 -- 172.21.15.86:0/972288013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a8004020 con 0x5574d1d3eb40 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.005+0000 7f57acff9640 1 -- 172.21.15.86:0/972288013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a80052c0 con 0x5574d1d3eb40 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- 172.21.15.86:0/972288013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 msgr2=0x5574d1d38c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57a800a9e0 tx=0x7f57a8010430 comp rx=0 tx=0).stop 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- 172.21.15.86:0/972288013 shutdown_connections 2024-09-05T07:00:57.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d48370 0x5574d1d40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d3eb40 0x5574d1d38c10 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- 172.21.15.86:0/972288013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3d5d0 0x5574d1d44890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- 172.21.15.86:0/972288013 >> 172.21.15.86:0/972288013 conn(0x5574d1d06310 msgr2=0x5574d1d38500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- 172.21.15.86:0/972288013 shutdown_connections 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- 172.21.15.86:0/972288013 wait complete. 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- start start 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 Processor -- start 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3d5d0 0x5574d1d50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b2680 con 0x5574d1d48370 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b1fb0 con 0x5574d1d3eb40 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.006+0000 7f57bf895740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81af210 con 0x5574d1d3d5d0 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57bf895740 10 client.0 _ll_register_callbacks cb 0x7f57a0031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:57.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38972/0 (socket says 172.21.15.86:38972) 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 -- 172.21.15.86:0/3131834003 learned_addr learned my addr 172.21.15.86:0/3131834003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3d5d0 0x5574d1d50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 -- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3d5d0 msgr2=0x5574d1d50ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3d5d0 0x5574d1d50ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 -- 172.21.15.86:0/3131834003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 msgr2=0x5574d1d41e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d41e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 -- 172.21.15.86:0/3131834003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d1d42c30 con 0x5574d1d48370 2024-09-05T07:00:57.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d41e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57b67fc640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f57a0031fa0 tx=0x7f57a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc49f7c3472d7757 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f579e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0035070 con 0x5574d1d48370 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f57a003c860 con 0x5574d1d48370 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a8004020 con 0x5574d1d48370 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f57a003ca60 con 0x5574d1d48370 2024-09-05T07:00:57.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57bf895740 10 client.19806 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.007+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5574d1dc2c60 con 0x5574d1d48370 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57a8004020 con 0x5574d1d48370 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f57a004a040 con 0x5574d1d48370 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 10 client.19806 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 10 client.19806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 20 client.19806 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1dc33e0 0x5574d1dc5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 4 client.19806 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574d1caad70 con 0x5574d1dc33e0 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1dc33e0 0x5574d1dc5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f579e7fc640 10 client.19806 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.008+0000 7f57b6ffd640 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1dc33e0 0x5574d1dc5800 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b07c763e347eaa24 server_cookie=8f908c7c77aeb04b in_seq=0 out_seq=0 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f579e7fc640 1 -- 172.21.15.86:0/3131834003 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574d1caad70 con 0x5574d1dc33e0 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f579e7fc640 10 client.19806 handle_command_reply: tid=0 2024-09-05T07:00:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 client.19806 shutdown 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 msgr2=0x5574d1d42430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f57a0031fa0 tx=0x7f57a002a410 comp rx=0 tx=0).stop 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 shutdown_connections 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5574d1dc33e0 0x5574d1dc5800 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574d1d48370 0x5574d1d42430 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574d1d3eb40 0x5574d1d41e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 --2- 172.21.15.86:0/3131834003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574d1d3d5d0 0x5574d1d50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f579e7fc640 0 client.19806 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 >> 172.21.15.86:0/3131834003 conn(0x5574d1d06310 msgr2=0x5574d1d47050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 shutdown_connections 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 1 -- 172.21.15.86:0/3131834003 wait complete. 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bf895740 20 client.19806 trim_cache size 0 max 16384 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 msgr2=0x7f57a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 0x7f57a40a5230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57b00cd440 tx=0x7f57b00c67a0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 msgr2=0x7f57b81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f57b000cce0 tx=0x7f57b000a620 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 shutdown_connections 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f57a40a2d70 0x7f57a40a5230 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57b810c2f0 0x7f57b81014d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57b81037a0 0x7f57b8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 --2- 172.21.15.86:0/112709322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b8102df0 0x7f57b8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 >> 172.21.15.86:0/112709322 conn(0x7f57b80fc7d0 msgr2=0x7f57b80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 shutdown_connections 2024-09-05T07:00:57.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:57.009+0000 7f57bd4bc640 1 -- 172.21.15.86:0/112709322 wait complete. 2024-09-05T07:00:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:57 smithi115 ceph-mon[34791]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-09-05T07:00:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:57 smithi086 ceph-mon[30943]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-09-05T07:00:57.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.95 MB/sec execute 255 sec latency 1465.989 ms 2024-09-05T07:00:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:57 smithi096 ceph-mon[34468]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 196 op/s 2024-09-05T07:00:57.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:00:58.320 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a17ee4b7', quiesce took 14.6 seconds. Will release after: -4.6 2024-09-05T07:00:58.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:00:58.414 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:58.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.92 MB/sec execute 256 sec latency 2466.069 ms 2024-09-05T07:00:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 -- 172.21.15.86:0/1649564478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380106a80 msgr2=0x7f9380106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 --2- 172.21.15.86:0/1649564478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380106a80 0x7f9380106e60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9370009930 tx=0x7f937002a420 comp rx=0 tx=0).stop 2024-09-05T07:00:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 -- 172.21.15.86:0/1649564478 shutdown_connections 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 --2- 172.21.15.86:0/1649564478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f938010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 --2- 172.21.15.86:0/1649564478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380100a80 0x7f9380100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 --2- 172.21.15.86:0/1649564478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380106a80 0x7f9380106e60 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 -- 172.21.15.86:0/1649564478 >> 172.21.15.86:0/1649564478 conn(0x7f93800fc7a0 msgr2=0x7f93800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 -- 172.21.15.86:0/1649564478 shutdown_connections 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.894+0000 7f9385334640 1 -- 172.21.15.86:0/1649564478 wait complete. 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 Processor -- start 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 -- start start 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f9380199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380106a80 0x7f93801a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9380110cf0 con 0x7f9380100a80 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9380110b70 con 0x7f9380106a80 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f9385334640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9380110e70 con 0x7f9380101420 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39002/0 (socket says 172.21.15.86:39002) 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 -- 172.21.15.86:0/3369430319 learned_addr learned my addr 172.21.15.86:0/3369430319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937f7fe640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380106a80 0x7f93801a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937e7fc640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f9380199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 -- 172.21.15.86:0/3369430319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 msgr2=0x7f9380199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f9380199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 -- 172.21.15.86:0/3369430319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380106a80 msgr2=0x7f93801a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380106a80 0x7f93801a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937effd640 1 -- 172.21.15.86:0/3369430319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801a4700 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.895+0000 7f937e7fc640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f9380199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f937effd640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f9370005940 tx=0x7f93700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f9367fff640 1 -- 172.21.15.86:0/3369430319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937003f030 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f9367fff640 1 -- 172.21.15.86:0/3369430319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9370003cb0 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f9367fff640 1 -- 172.21.15.86:0/3369430319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370035070 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801a48d0 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.896+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801a4d20 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f9367fff640 1 -- 172.21.15.86:0/3369430319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9370042030 con 0x7f9380100a80 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 Processor -- start 2024-09-05T07:00:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 -- start start 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 0x561b12ea0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 0x561b12ea0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12eab370 0x561b12ead760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b12e062f0 con 0x561b12ea05d0 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b12dadcf0 con 0x561b12ea1b40 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f938770d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b12e022b0 con 0x561b12eab370 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.898+0000 7f9367fff640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 0x7f93680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 0x561b12ea0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 0x561b12ea0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 0x561b12ea0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:55696/0 (socket says 172.21.15.86:55696) 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937f7fe640 1 -- 172.21.15.86:0/2940313351 learned_addr learned my addr 172.21.15.86:0/2940313351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937f7fe640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 0x7f93680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937e7fc640 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12eab370 0x561b12ead760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f9367fff640 1 -- 172.21.15.86:0/3369430319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f93700efeb0 con 0x7f9380100a80 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 -- 172.21.15.86:0/2940313351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12eab370 msgr2=0x561b12ead760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12eab370 0x561b12ead760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 -- 172.21.15.86:0/2940313351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 msgr2=0x561b12ea0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 0x561b12ea0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 -- 172.21.15.86:0/2940313351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b12d923a0 con 0x561b12ea05d0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937f7fe640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 0x7f93680a51b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9378013bb0 tx=0x7f937800b950 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f937effd640 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 0x561b12ea0a10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f93700ed1c0 tx=0x7f9370043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e33e2d368127e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f93657fa640 1 -- 172.21.15.86:0/2940313351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370049070 con 0x561b12ea05d0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f93657fa640 1 -- 172.21.15.86:0/2940313351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93700bc690 con 0x561b12ea05d0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.899+0000 7f93657fa640 1 -- 172.21.15.86:0/2940313351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700ed440 con 0x561b12ea05d0 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- 172.21.15.86:0/2940313351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 msgr2=0x561b12ea0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 0x561b12ea0a10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f93700ed1c0 tx=0x7f9370043000 comp rx=0 tx=0).stop 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- 172.21.15.86:0/2940313351 shutdown_connections 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12eab370 0x561b12ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea05d0 0x561b12ea0a10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 --2- 172.21.15.86:0/2940313351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12ea1b40 0x561b12ea0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- 172.21.15.86:0/2940313351 >> 172.21.15.86:0/2940313351 conn(0x561b12e69310 msgr2=0x561b12e9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- 172.21.15.86:0/2940313351 shutdown_connections 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- 172.21.15.86:0/2940313351 wait complete. 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 -- start start 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.900+0000 7f938770d740 1 Processor -- start 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eaadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b12eade30 con 0x561b12ea1b40 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b12dc1060 con 0x561b12eab370 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b12eadfb0 con 0x561b12ea05d0 2024-09-05T07:00:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 10 client.0 _ll_register_callbacks cb 0x561b12b148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39024/0 (socket says 172.21.15.86:39024) 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 -- 172.21.15.86:0/1963426127 learned_addr learned my addr 172.21.15.86:0/1963426127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937effd640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eaadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937f7fe640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 -- 172.21.15.86:0/1963426127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 msgr2=0x561b12eaadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eaadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 -- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 msgr2=0x561b12ea8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 -- 172.21.15.86:0/1963426127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b12b773c0 con 0x561b12ea1b40 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937f7fe640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9374010820 tx=0x7f9374010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65fa9708b5f4a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f9362ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937401d060 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f937400eda0 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.901+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93700bc690 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9374024050 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 10 client.19818 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b12eb9940 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93700bc690 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f9374016450 con 0x561b12ea1b40 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 10 client.19818 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 10 client.19818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 20 client.19818 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12ebae50 0x561b12ebd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 4 client.19818 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b12e13860 con 0x561b12ebae50 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.902+0000 7f937effd640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12ebae50 0x561b12ebd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.903+0000 7f9362ff5640 10 client.19818 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.903+0000 7f937effd640 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12ebae50 0x561b12ebd290 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cafe08daac6ce4 server_cookie=f2351f373257f57a in_seq=0 out_seq=0 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f9362ff5640 1 -- 172.21.15.86:0/1963426127 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b12e13860 con 0x561b12ebae50 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f9362ff5640 10 client.19818 handle_command_reply: tid=0 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 client.19818 shutdown 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 msgr2=0x561b12ea8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9374010820 tx=0x7f9374010ce0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 shutdown_connections 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12ebae50 0x561b12ebd290 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f9362ff5640 0 client.19818 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea8080 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 --2- 172.21.15.86:0/1963426127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eaadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 >> 172.21.15.86:0/1963426127 conn(0x561b12e69310 msgr2=0x561b12ea2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 shutdown_connections 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 1 -- 172.21.15.86:0/1963426127 wait complete. 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.904+0000 7f938770d740 20 client.19818 trim_cache size 0 max 16384 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.916+0000 7f938770d740 1 Processor -- start 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 -- start start 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12ead760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93801af070 con 0x561b12ea1b40 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b0bb0 con 0x561b12eab370 2024-09-05T07:00:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f938770d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f93801af740 con 0x561b12ea05d0 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39036/0 (socket says 172.21.15.86:39036) 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 -- 172.21.15.86:0/1750831091 learned_addr learned my addr 172.21.15.86:0/1750831091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937f7fe640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937effd640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12ead760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 -- 172.21.15.86:0/1750831091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 msgr2=0x561b12ead760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12ead760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 -- 172.21.15.86:0/1750831091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 msgr2=0x561b12ea3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 -- 172.21.15.86:0/1750831091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b12e9d960 con 0x561b12ea1b40 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937f7fe640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937effd640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f937e7fc640 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f93740168a0 tx=0x7f93740056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624bd8ceb9e00443 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f9364ff9640 1 -- 172.21.15.86:0/1750831091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937401d020 con 0x561b12ea1b40 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f9364ff9640 1 -- 172.21.15.86:0/1750831091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9374005e50 con 0x561b12ea1b40 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.917+0000 7f9364ff9640 1 -- 172.21.15.86:0/1750831091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374016590 con 0x561b12ea1b40 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- 172.21.15.86:0/1750831091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 msgr2=0x561b12ea7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f93740168a0 tx=0x7f93740056f0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- 172.21.15.86:0/1750831091 shutdown_connections 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12ea7170 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 --2- 172.21.15.86:0/1750831091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- 172.21.15.86:0/1750831091 >> 172.21.15.86:0/1750831091 conn(0x561b12e69310 msgr2=0x561b12e97a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- 172.21.15.86:0/1750831091 shutdown_connections 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- 172.21.15.86:0/1750831091 wait complete. 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 -- start start 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.918+0000 7f938770d740 1 Processor -- start 2024-09-05T07:00:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eb3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b04e0 con 0x561b12ea1b40 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93801afe10 con 0x561b12eab370 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f93801ad070 con 0x561b12ea05d0 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 10 client.0 _ll_register_callbacks cb 0x7f9374010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39040/0 (socket says 172.21.15.86:39040) 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 -- 172.21.15.86:0/2245542110 learned_addr learned my addr 172.21.15.86:0/2245542110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937f7fe640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 -- 172.21.15.86:0/2245542110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 msgr2=0x561b12eb3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eb3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 -- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 msgr2=0x561b12ea4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 -- 172.21.15.86:0/2245542110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b12ea54b0 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937f7fe640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f937e7fc640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9374010820 tx=0x7f93740168d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84f3841b0b28144 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f9362ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374038020 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f9374016330 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.919+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9374005e50 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9374043050 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 10 client.19824 fetch_fsmap learned FSMap version 45 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b12f25730 con 0x561b12ea1b40 2024-09-05T07:00:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9374005e50 con 0x561b12ea1b40 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f937402a440 con 0x561b12ea1b40 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 10 client.19824 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 10 client.19824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 20 client.19824 populate_metadata read hostname 'smithi086' 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12f25d90 0x561b12f28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 4 client.19824 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b12e0dd70 con 0x561b12f25d90 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.920+0000 7f937effd640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12f25d90 0x561b12f28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f937effd640 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12f25d90 0x561b12f28180 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705185112ea9e55e server_cookie=7b4706b87afa2187 in_seq=0 out_seq=0 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f9362ff5640 10 client.19824 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f9362ff5640 1 -- 172.21.15.86:0/2245542110 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b12e0dd70 con 0x561b12f25d90 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f9362ff5640 10 client.19824 handle_command_reply: tid=0 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 client.19824 shutdown 2024-09-05T07:00:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 msgr2=0x561b12eb3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9374010820 tx=0x7f93740168d0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 shutdown_connections 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b12f25d90 0x561b12f28180 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b12eab370 0x561b12ea4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b12ea1b40 0x561b12eb3ab0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f9362ff5640 0 client.19824 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 --2- 172.21.15.86:0/2245542110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b12ea05d0 0x561b12eb3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 >> 172.21.15.86:0/2245542110 conn(0x561b12e69310 msgr2=0x561b12ea7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 shutdown_connections 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 1 -- 172.21.15.86:0/2245542110 wait complete. 2024-09-05T07:00:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.921+0000 7f938770d740 20 client.19824 trim_cache size 0 max 16384 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 msgr2=0x7f93680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 0x7f93680a51b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9378013bb0 tx=0x7f937800b950 comp rx=0 tx=0).stop 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 msgr2=0x7f9380199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f9370005940 tx=0x7f93700042f0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 shutdown_connections 2024-09-05T07:00:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f93680a2cf0 0x7f93680a51b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9380106a80 0x7f93801a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9380101420 0x7f9380199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 --2- 172.21.15.86:0/3369430319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380100a80 0x7f9380199140 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.922+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 >> 172.21.15.86:0/3369430319 conn(0x7f93800fc7a0 msgr2=0x7f93800fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.923+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 shutdown_connections 2024-09-05T07:00:58.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:00:58.923+0000 7f9385334640 1 -- 172.21.15.86:0/3369430319 wait complete. 2024-09-05T07:00:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:00:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:00:59 smithi115 ceph-mon[34791]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-09-05T07:00:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:00:59 smithi086 ceph-mon[30943]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-09-05T07:00:59.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.90 MB/sec execute 257 sec latency 3466.142 ms 2024-09-05T07:00:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:00:59 smithi096 ceph-mon[34468]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-09-05T07:01:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.319+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1821598898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34106f10 msgr2=0x7f4e3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.319+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1821598898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34106f10 0x7f4e3410ea90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009ca0 tx=0x7f4e2c02a8c0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1821598898 shutdown_connections 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1821598898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34106f10 0x7f4e3410ea90 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1821598898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34102790 0x7f4e341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1821598898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1821598898 >> 172.21.15.86:0/1821598898 conn(0x7f4e340fd8c0 msgr2=0x7f4e340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1821598898 shutdown_connections 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1821598898 wait complete. 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 Processor -- start 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.320+0000 7f4e39cd6640 1 -- start start 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e3419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34106f10 0x7f4e3419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34111e70 con 0x7f4e34102790 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34111cf0 con 0x7f4e34106f10 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34111ff0 con 0x7f4e34101de0 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39048/0 (socket says 172.21.15.86:39048) 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 -- 172.21.15.86:0/1931291403 learned_addr learned my addr 172.21.15.86:0/1931291403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e33fff640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34106f10 0x7f4e3419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e337fe640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e3419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 -- 172.21.15.86:0/1931291403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 msgr2=0x7f4e3419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e3419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 -- 172.21.15.86:0/1931291403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34106f10 msgr2=0x7f4e3419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34106f10 0x7f4e3419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 -- 172.21.15.86:0/1931291403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3419f510 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e337fe640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e3419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e2affd640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c890 tx=0x7f4e1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004220 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1c0043c0 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3419f800 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004a00 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.321+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341a70e0 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.322+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e341021c0 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.324+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4e1c004ba0 con 0x7f4e34102790 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.324+0000 7f4e317fa640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 0x7f4e200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.325+0000 7f4e337fe640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 0x7f4e200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.325+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f4e1c0cf520 con 0x7f4e34102790 2024-09-05T07:01:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.325+0000 7f4e337fe640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 0x7f4e200a51b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005fd0 tx=0x7f4e24005ea0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.328+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e1c014030 con 0x7f4e34102790 2024-09-05T07:01:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.555+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e341a7410 con 0x7f4e34102790 2024-09-05T07:01:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.556+0000 7f4e317fa640 1 -- 172.21.15.86:0/1931291403 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7f4e1c09bd00 con 0x7f4e34102790 2024-09-05T07:01:00.557 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:01:00.557 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:01:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:01:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 msgr2=0x7f4e200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 0x7f4e200a51b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005fd0 tx=0x7f4e24005ea0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 msgr2=0x7f4e3419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c890 tx=0x7f4e1c00cd50 comp rx=0 tx=0).stop 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 shutdown_connections 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4e200a2cf0 0x7f4e200a51b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e34106f10 0x7f4e3419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e34102790 0x7f4e3419aaa0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 --2- 172.21.15.86:0/1931291403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e34101de0 0x7f4e3419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 >> 172.21.15.86:0/1931291403 conn(0x7f4e340fd8c0 msgr2=0x7f4e340feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 shutdown_connections 2024-09-05T07:01:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:00.558+0000 7f4e39cd6640 1 -- 172.21.15.86:0/1931291403 wait complete. 2024-09-05T07:01:00.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.87 MB/sec execute 258 sec latency 4466.211 ms 2024-09-05T07:01:00.816 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:01:00.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:01:01.325 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id a17ee4b7 --release --await' 2024-09-05T07:01:01.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id a17ee4b7 --release --await 2024-09-05T07:01:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:01.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:01 smithi086 ceph-mon[30943]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 211 op/s 2024-09-05T07:01:01.592 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:01 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1931291403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:01 smithi115 ceph-mon[34791]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 211 op/s 2024-09-05T07:01:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:01 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1931291403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:01.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.84 MB/sec execute 259 sec latency 5466.253 ms 2024-09-05T07:01:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:01 smithi096 ceph-mon[34468]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 211 op/s 2024-09-05T07:01:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:01 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1931291403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:01.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- 172.21.15.86:0/3659512881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008100a80 msgr2=0x7f2008100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 --2- 172.21.15.86:0/3659512881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008100a80 0x7f2008100ee0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009930 tx=0x7f1ff002a420 comp rx=0 tx=0).stop 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- 172.21.15.86:0/3659512881 shutdown_connections 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 --2- 172.21.15.86:0/3659512881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f200810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 --2- 172.21.15.86:0/3659512881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008100a80 0x7f2008100ee0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 --2- 172.21.15.86:0/3659512881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008106a80 0x7f2008106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- 172.21.15.86:0/3659512881 >> 172.21.15.86:0/3659512881 conn(0x7f20080fc7a0 msgr2=0x7f20080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- 172.21.15.86:0/3659512881 shutdown_connections 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- 172.21.15.86:0/3659512881 wait complete. 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 Processor -- start 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.973+0000 7f200c9fd640 1 -- start start 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008100a80 0x7f20081983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f2008198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2008110d30 con 0x7f2008106a80 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2008110bb0 con 0x7f2008101420 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2008110eb0 con 0x7f2008100a80 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39068/0 (socket says 172.21.15.86:39068) 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 -- 172.21.15.86:0/457888065 learned_addr learned my addr 172.21.15.86:0/457888065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f1ffffff640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f2008198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200659c640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008100a80 0x7f20081983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 -- 172.21.15.86:0/457888065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008100a80 msgr2=0x7f20081983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008100a80 0x7f20081983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 -- 172.21.15.86:0/457888065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 msgr2=0x7f2008198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f2008198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 -- 172.21.15.86:0/457888065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819d370 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f1ffffff640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f2008198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f2006d9d640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2000009fd0 tx=0x7f200000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f1fff7fe640 1 -- 172.21.15.86:0/457888065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000017070 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f1fff7fe640 1 -- 172.21.15.86:0/457888065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2000003e20 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819d660 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f1fff7fe640 1 -- 172.21.15.86:0/457888065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000010460 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.974+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a4f40 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f1fff7fe640 1 -- 172.21.15.86:0/457888065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2000003970 con 0x7f2008106a80 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 Processor -- start 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 -- start start 2024-09-05T07:01:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bceeef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629bce54280 con 0x5629bceef480 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629bcdfbb10 con 0x5629bcef9d40 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200edd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629bce50240 con 0x5629bceeeb60 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.977+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39076/0 (socket says 172.21.15.86:39076) 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 -- 172.21.15.86:0/2114580041 learned_addr learned my addr 172.21.15.86:0/2114580041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1fff7fe640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 0x7f1ff40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffffff640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffffff640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bceeef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f2006d9d640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 0x7f1ff40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 -- 172.21.15.86:0/2114580041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 msgr2=0x5629bceeef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bceeef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 -- 172.21.15.86:0/2114580041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 msgr2=0x5629bcefc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 -- 172.21.15.86:0/2114580041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bce081f0 con 0x5629bceef480 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffffff640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bceeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1fff7fe640 1 -- 172.21.15.86:0/457888065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f20000cf7f0 con 0x7f2008106a80 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f200659c640 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800cb40 tx=0x7f1ff800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc9ae3ea4786689 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffcff9640 1 -- 172.21.15.86:0/2114580041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8010430 con 0x5629bceef480 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f2006d9d640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 0x7f1ff40a51b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f200009b570 tx=0x7f200006a7a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffcff9640 1 -- 172.21.15.86:0/2114580041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff8010a80 con 0x5629bceef480 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.978+0000 7f1ffcff9640 1 -- 172.21.15.86:0/2114580041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800fb00 con 0x5629bceef480 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- 172.21.15.86:0/2114580041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 msgr2=0x5629bcef9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800cb40 tx=0x7f1ff800e350 comp rx=0 tx=0).stop 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- 172.21.15.86:0/2114580041 shutdown_connections 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcef9800 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 --2- 172.21.15.86:0/2114580041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bceeef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- 172.21.15.86:0/2114580041 >> 172.21.15.86:0/2114580041 conn(0x5629bceeb0f0 msgr2=0x5629bceeb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- 172.21.15.86:0/2114580041 shutdown_connections 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- 172.21.15.86:0/2114580041 wait complete. 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 -- start start 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.979+0000 7f200edd6740 1 Processor -- start 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bceeeb60 0x5629bcef6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceef480 0x5629bcef71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629bcefc910 con 0x5629bcef9d40 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629bce0ef10 con 0x5629bceeeb60 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629bcefca90 con 0x5629bceef480 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 10 client.0 _ll_register_callbacks cb 0x5629bcdd7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39088/0 (socket says 172.21.15.86:39088) 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 -- 172.21.15.86:0/4075396364 learned_addr learned my addr 172.21.15.86:0/4075396364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200659c640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceef480 0x5629bcef71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bceeeb60 0x5629bcef6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 -- 172.21.15.86:0/4075396364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceef480 msgr2=0x5629bcef71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceef480 0x5629bcef71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 -- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bceeeb60 msgr2=0x5629bcef6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bceeeb60 0x5629bcef6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 -- 172.21.15.86:0/4075396364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bcbc56a0 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002a940 tx=0x7f1ff002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce920777f00bf511 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0035040 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f1ff0004140 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff8010a80 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1ff0051050 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 10 client.19842 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.980+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5629bcf084c0 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff8010a80 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f1ff003a040 con 0x5629bcef9d40 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 10 client.19842 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 10 client.19842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 20 client.19842 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf0bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 4 client.19842 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629bce617f0 con 0x5629bcf098d0 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200659c640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf0bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f1fedffb640 10 client.19842 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.981+0000 7f200659c640 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf0bcc0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f77e026905c3936 server_cookie=48d8393bb3cc975 in_seq=0 out_seq=0 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f1fedffb640 1 -- 172.21.15.86:0/4075396364 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629bce617f0 con 0x5629bcf098d0 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f1fedffb640 10 client.19842 handle_command_reply: tid=0 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 client.19842 shutdown 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 msgr2=0x5629bcef76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002a940 tx=0x7f1ff002a420 comp rx=0 tx=0).stop 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 shutdown_connections 2024-09-05T07:01:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf0bcc0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bcef9d40 0x5629bcef76e0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceef480 0x5629bcef71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f1fedffb640 0 client.19842 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 --2- 172.21.15.86:0/4075396364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bceeeb60 0x5629bcef6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 >> 172.21.15.86:0/4075396364 conn(0x5629bceeb0f0 msgr2=0x5629bceea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 shutdown_connections 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 1 -- 172.21.15.86:0/4075396364 wait complete. 2024-09-05T07:01:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:01.982+0000 7f200edd6740 20 client.19842 trim_cache size 0 max 16384 2024-09-05T07:01:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 Processor -- start 2024-09-05T07:01:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 -- start start 2024-09-05T07:01:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcef15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcf11ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20081af150 con 0x5629bceef480 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20081b0c90 con 0x5629bcef9d40 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200edd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20081af820 con 0x5629bceeeb60 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39104/0 (socket says 172.21.15.86:39104) 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 -- 172.21.15.86:0/1439384481 learned_addr learned my addr 172.21.15.86:0/1439384481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f1ffffff640 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcef15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f1ffffff640 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcf11ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 -- 172.21.15.86:0/1439384481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 msgr2=0x5629bcef15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcef15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 -- 172.21.15.86:0/1439384481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 msgr2=0x5629bcf11ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcf11ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.000+0000 7f200659c640 1 -- 172.21.15.86:0/1439384481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bcf0c030 con 0x5629bceef480 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200659c640 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8013040 tx=0x7f1ff80058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe83b324f6dbf8b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f1feffff640 1 -- 172.21.15.86:0/1439384481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8004220 con 0x5629bceef480 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f1feffff640 1 -- 172.21.15.86:0/1439384481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff80043c0 con 0x5629bceef480 2024-09-05T07:01:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f1feffff640 1 -- 172.21.15.86:0/1439384481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800fad0 con 0x5629bceef480 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 -- 172.21.15.86:0/1439384481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 msgr2=0x5629bcf11570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8013040 tx=0x7f1ff80058e0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 -- 172.21.15.86:0/1439384481 shutdown_connections 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcf11ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcf11570 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 --2- 172.21.15.86:0/1439384481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcef15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 -- 172.21.15.86:0/1439384481 >> 172.21.15.86:0/1439384481 conn(0x5629bceeb0f0 msgr2=0x5629bcef8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 -- 172.21.15.86:0/1439384481 shutdown_connections 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.001+0000 7f200edd6740 1 -- 172.21.15.86:0/1439384481 wait complete. 2024-09-05T07:01:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 -- start start 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 Processor -- start 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcefdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20081b05c0 con 0x5629bceef480 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20081afef0 con 0x5629bcef9d40 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20081ad150 con 0x5629bceeeb60 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200edd6740 10 client.0 _ll_register_callbacks cb 0x7f1ff0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39118/0 (socket says 172.21.15.86:39118) 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f200659c640 1 -- 172.21.15.86:0/4187724485 learned_addr learned my addr 172.21.15.86:0/4187724485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcefdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.002+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 -- 172.21.15.86:0/4187724485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 msgr2=0x5629bcefdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcefdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 -- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 msgr2=0x5629bcefe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 -- 172.21.15.86:0/4187724485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bcf77ba0 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200659c640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1ff802b000 tx=0x7f1ff800c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1909d865496f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f1fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8005240 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f1ff80053e0 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff80043c0 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1ff8042050 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200edd6740 10 client.19848 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5629bcf78ea0 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.003+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff80043c0 con 0x5629bceef480 2024-09-05T07:01:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f1ff8004660 con 0x5629bceef480 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 10 client.19848 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 10 client.19848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 20 client.19848 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 4 client.19848 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629bce5bd00 con 0x5629bcf098d0 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f1fedffb640 10 client.19848 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.004+0000 7f1ffffff640 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf7b3c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece37563ded5e008 server_cookie=4298a398dbd79e7d in_seq=0 out_seq=0 2024-09-05T07:01:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f1fedffb640 1 -- 172.21.15.86:0/4187724485 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629bce5bd00 con 0x5629bcf098d0 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f1fedffb640 10 client.19848 handle_command_reply: tid=0 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f200edd6740 1 client.19848 shutdown 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 msgr2=0x5629bcefe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1ff802b000 tx=0x7f1ff800c3b0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 shutdown_connections 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.005+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5629bcf098d0 0x5629bcf7b3c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629bcef9d40 0x5629bcefe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f1fedffb640 0 client.19848 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629bceef480 0x5629bcefe1a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 --2- 172.21.15.86:0/4187724485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5629bceeeb60 0x5629bcefdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 >> 172.21.15.86:0/4187724485 conn(0x5629bceeb0f0 msgr2=0x5629bcef81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 shutdown_connections 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 1 -- 172.21.15.86:0/4187724485 wait complete. 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200edd6740 20 client.19848 trim_cache size 0 max 16384 2024-09-05T07:01:02.016 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:02.017 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:01:02.017 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "4f33e9fe-a4bd-4adf-b798-fa8ae9b0118e", 2024-09-05T07:01:02.017 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:01:02.017 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 msgr2=0x7f1ff40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 0x7f1ff40a51b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f200009b570 tx=0x7f200006a7a0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 msgr2=0x7f200819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2000009fd0 tx=0x7f200000b260 comp rx=0 tx=0).stop 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 shutdown_connections 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f1ff40a2cf0 0x7f1ff40a51b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008106a80 0x7f200819cc90 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2008101420 0x7f2008198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 --2- 172.21.15.86:0/457888065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2008100a80 0x7f20081983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 >> 172.21.15.86:0/457888065 conn(0x7f20080fc7a0 msgr2=0x7f200806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 shutdown_connections 2024-09-05T07:01:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:02.006+0000 7f200c9fd640 1 -- 172.21.15.86:0/457888065 wait complete. 2024-09-05T07:01:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:02.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48177 6.82 MB/sec execute 260 sec latency 6466.373 ms 2024-09-05T07:01:02.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:01:03.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 -- 172.21.15.86:0/937580681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 msgr2=0x7f118410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 --2- 172.21.15.86:0/937580681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f118410ea20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f117c009ca0 tx=0x7f117c02a8c0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 -- 172.21.15.86:0/937580681 shutdown_connections 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 --2- 172.21.15.86:0/937580681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f118410ea20 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 --2- 172.21.15.86:0/937580681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 0x7f1184100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 --2- 172.21.15.86:0/937580681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 0x7f1184106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.149+0000 7f1188b18640 1 -- 172.21.15.86:0/937580681 >> 172.21.15.86:0/937580681 conn(0x7f1184077230 msgr2=0x7f11840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.150+0000 7f1188b18640 1 -- 172.21.15.86:0/937580681 shutdown_connections 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.150+0000 7f1188b18640 1 -- 172.21.15.86:0/937580681 wait complete. 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.150+0000 7f1188b18640 1 Processor -- start 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.150+0000 7f1188b18640 1 -- start start 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 0x7f11841984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 0x7f118419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1184111fa0 con 0x7f1184100aa0 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1184111e20 con 0x7f1184100100 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1188b18640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1184112120 con 0x7f1184106b80 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50958/0 (socket says 172.21.15.86:50958) 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 -- 172.21.15.86:0/4283779869 learned_addr learned my addr 172.21.15.86:0/4283779869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f118259c640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 0x7f11841984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1182d9d640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 0x7f118419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 -- 172.21.15.86:0/4283779869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 msgr2=0x7f118419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 0x7f118419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 -- 172.21.15.86:0/4283779869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 msgr2=0x7f11841984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 0x7f11841984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 -- 172.21.15.86:0/4283779869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118419d470 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.151+0000 7f1181d9b640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b4a0 tx=0x7f116c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.152+0000 7f117b7fe640 1 -- 172.21.15.86:0/4283779869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c015020 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.152+0000 7f117b7fe640 1 -- 172.21.15.86:0/4283779869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f116c011ce0 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.152+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118419d760 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.152+0000 7f117b7fe640 1 -- 172.21.15.86:0/4283779869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c010dc0 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.152+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11841a5040 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.154+0000 7f117b7fe640 1 -- 172.21.15.86:0/4283779869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f116c018020 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.155+0000 7f117b7fe640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 0x7f11700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.155+0000 7f117b7fe640 1 -- 172.21.15.86:0/4283779869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f116c0ceff0 con 0x7f1184100aa0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.155+0000 7f118259c640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 0x7f11700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.155+0000 7f118aef1740 1 Processor -- start 2024-09-05T07:01:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 -- start start 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118259c640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 0x7f11700a51b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1174009990 tx=0x7f1174009110 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff2634c620 con 0x55ff263e7980 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff262f3e50 con 0x55ff263f2240 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118aef1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff263485e0 con 0x55ff263e7060 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50974/0 (socket says 172.21.15.86:50974) 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 -- 172.21.15.86:0/299710952 learned_addr learned my addr 172.21.15.86:0/299710952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1182d9d640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f118259c640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 -- 172.21.15.86:0/299710952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 msgr2=0x55ff263e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 -- 172.21.15.86:0/299710952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 msgr2=0x55ff263f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1181d9b640 1 -- 172.21.15.86:0/299710952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff262e4d40 con 0x55ff263e7980 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.156+0000 7f1182d9d640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.157+0000 7f1181d9b640 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f116c097b50 tx=0x7f116c09adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c2ed6593d44b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.157+0000 7f1178ff9640 1 -- 172.21.15.86:0/299710952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c09b180 con 0x55ff263e7980 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.157+0000 7f1178ff9640 1 -- 172.21.15.86:0/299710952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f116c09f530 con 0x55ff263e7980 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.157+0000 7f1178ff9640 1 -- 172.21.15.86:0/299710952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c09f160 con 0x55ff263e7980 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- 172.21.15.86:0/299710952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 msgr2=0x55ff263f1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f116c097b50 tx=0x7f116c09adc0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- 172.21.15.86:0/299710952 shutdown_connections 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263f1d00 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 --2- 172.21.15.86:0/299710952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- 172.21.15.86:0/299710952 >> 172.21.15.86:0/299710952 conn(0x55ff263e35f0 msgr2=0x55ff263e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- 172.21.15.86:0/299710952 shutdown_connections 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- 172.21.15.86:0/299710952 wait complete. 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 -- start start 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.158+0000 7f118aef1740 1 Processor -- start 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 0x55ff263ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff263f9220 con 0x55ff263e7060 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff262d4f10 con 0x55ff263f2240 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff263f93a0 con 0x55ff263e7980 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118aef1740 10 client.0 _ll_register_callbacks cb 0x55ff262f8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50978/0 (socket says 172.21.15.86:50978) 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 -- 172.21.15.86:0/3699272153 learned_addr learned my addr 172.21.15.86:0/3699272153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 0x55ff263ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f1182d9d640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 -- 172.21.15.86:0/3699272153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 msgr2=0x55ff263ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 0x55ff263ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 -- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 msgr2=0x55ff263efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 -- 172.21.15.86:0/3699272153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2605b580 con 0x55ff263e7060 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 0x55ff263ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f118259c640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1174015710 tx=0x7f11740047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a46394e87a225fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.159+0000 7f1169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174025020 con 0x55ff263e7060 2024-09-05T07:01:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f1174013690 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c09f530 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f117401b040 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 10 client.19860 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ff26125960 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f116c09f530 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f1174021a60 con 0x55ff263e7060 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 10 client.19860 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 10 client.19860 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.160+0000 7f118aef1740 20 client.19860 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff26406230 0x55ff264086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f118aef1740 4 client.19860 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff26359b90 con 0x55ff26406230 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff26406230 0x55ff264086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f1169ffb640 10 client.19860 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.161+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff26406230 0x55ff264086a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b062608c53da903b server_cookie=ac83fac2677ac1df in_seq=0 out_seq=0 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.162+0000 7f1169ffb640 1 -- 172.21.15.86:0/3699272153 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff26359b90 con 0x55ff26406230 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.162+0000 7f1169ffb640 10 client.19860 handle_command_reply: tid=0 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.162+0000 7f118aef1740 1 client.19860 shutdown 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 msgr2=0x55ff263ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1174015710 tx=0x7f11740047d0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 shutdown_connections 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff26406230 0x55ff264086a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f1169ffb640 0 client.19860 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7980 0x55ff263ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 --2- 172.21.15.86:0/3699272153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7060 0x55ff263ef1f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 >> 172.21.15.86:0/3699272153 conn(0x55ff263e35f0 msgr2=0x55ff263e2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 shutdown_connections 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 1 -- 172.21.15.86:0/3699272153 wait complete. 2024-09-05T07:01:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.163+0000 7f118aef1740 20 client.19860 trim_cache size 0 max 16384 2024-09-05T07:01:03.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.180+0000 7f118aef1740 1 Processor -- start 2024-09-05T07:01:03.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.180+0000 7f118aef1740 1 -- start start 2024-09-05T07:01:03.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 0x55ff2640dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ad090 con 0x55ff263f2240 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11841aebd0 con 0x55ff263e7980 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118aef1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ad760 con 0x55ff263e7060 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50992/0 (socket says 172.21.15.86:50992) 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 -- 172.21.15.86:0/1070791255 learned_addr learned my addr 172.21.15.86:0/1070791255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f118259c640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1181d9b640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 0x55ff2640dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 -- 172.21.15.86:0/1070791255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 msgr2=0x55ff263f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 -- 172.21.15.86:0/1070791255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 msgr2=0x55ff2640dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 0x55ff2640dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1182d9d640 1 -- 172.21.15.86:0/1070791255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff263e4280 con 0x55ff263f2240 2024-09-05T07:01:03.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.181+0000 7f1181d9b640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 0x55ff2640dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.182+0000 7f1182d9d640 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f11841077c0 tx=0x7f117c031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f570769a7f37260e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.182+0000 7f116bfff640 1 -- 172.21.15.86:0/1070791255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c031450 con 0x55ff263f2240 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.182+0000 7f116bfff640 1 -- 172.21.15.86:0/1070791255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f117c036050 con 0x55ff263f2240 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.182+0000 7f116bfff640 1 -- 172.21.15.86:0/1070791255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c004590 con 0x55ff263f2240 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- 172.21.15.86:0/1070791255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 msgr2=0x55ff2640e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f11841077c0 tx=0x7f117c031b80 comp rx=0 tx=0).stop 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- 172.21.15.86:0/1070791255 shutdown_connections 2024-09-05T07:01:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263f2240 0x55ff2640e500 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263e7980 0x55ff2640dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- 172.21.15.86:0/1070791255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- 172.21.15.86:0/1070791255 >> 172.21.15.86:0/1070791255 conn(0x55ff263e35f0 msgr2=0x55ff263f0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- 172.21.15.86:0/1070791255 shutdown_connections 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- 172.21.15.86:0/1070791255 wait complete. 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 -- start start 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 Processor -- start 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.183+0000 7f118aef1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118aef1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ae500 con 0x55ff263e7980 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118aef1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ade30 con 0x55ff263f2240 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118aef1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11841ab090 con 0x55ff263e7060 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118aef1740 10 client.0 _ll_register_callbacks cb 0x7f117400ef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60950/0 (socket says 172.21.15.86:60950) 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 -- 172.21.15.86:0/3086902705 learned_addr learned my addr 172.21.15.86:0/3086902705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f1182d9d640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 -- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 msgr2=0x55ff263fb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263fb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 -- 172.21.15.86:0/3086902705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 msgr2=0x55ff263faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 -- 172.21.15.86:0/3086902705 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff263fba40 con 0x55ff263e7060 2024-09-05T07:01:03.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f118259c640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f117400e5a0 tx=0x7f1174021cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2289574043599f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f1169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.184+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174025040 con 0x55ff263e7060 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f1174013aa0 con 0x55ff263e7060 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c036050 con 0x55ff263e7060 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1174006d00 con 0x55ff263e7060 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 10 client.28675 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ff26471e30 con 0x55ff263e7060 2024-09-05T07:01:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f117c036050 con 0x55ff263e7060 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f117402a3f0 con 0x55ff263e7060 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 10 client.28675 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 10 client.28675 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 20 client.28675 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff264725a0 0x55ff264749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 4 client.28675 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a17ee4b7", "release": true, "await": true}] 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.185+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a17ee4b7", "release": true, "await": true}) -- 0x55ff263540a0 con 0x55ff264725a0 2024-09-05T07:01:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.186+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff264725a0 0x55ff264749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:03.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.186+0000 7f1181d9b640 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff264725a0 0x55ff264749e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d753b85e190bbf server_cookie=de9cf9f7ad4d9e86 in_seq=0 out_seq=0 2024-09-05T07:01:03.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.186+0000 7f1169ffb640 10 client.28675 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f1169ffb640 1 -- 172.21.15.86:0/3086902705 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ff263540a0 con 0x55ff264725a0 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f1169ffb640 10 client.28675 handle_command_reply: tid=0 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 client.28675 shutdown 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 msgr2=0x55ff263feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f117400e5a0 tx=0x7f1174021cb0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 shutdown_connections 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ff264725a0 0x55ff264749e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff263f2240 0x55ff263fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff263e7980 0x55ff263faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f1169ffb640 0 client.28675 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 --2- 172.21.15.86:0/3086902705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff263e7060 0x55ff263feb80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.218+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 >> 172.21.15.86:0/3086902705 conn(0x55ff263e35f0 msgr2=0x55ff263e92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.219+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 shutdown_connections 2024-09-05T07:01:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.219+0000 7f118aef1740 1 -- 172.21.15.86:0/3086902705 wait complete. 2024-09-05T07:01:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.220+0000 7f118aef1740 20 client.28675 trim_cache size 0 max 16384 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 72, 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "a17ee4b7": { 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 72, 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 130.0, 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:03.223 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.220+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 msgr2=0x7f11700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.220+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 0x7f11700a51b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1174009990 tx=0x7f1174009110 comp rx=0 tx=0).stop 2024-09-05T07:01:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 msgr2=0x7f1184198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b4a0 tx=0x7f116c00b8a0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 shutdown_connections 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f11700a2cf0 0x7f11700a51b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1184106b80 0x7f118419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1184100aa0 0x7f1184198a00 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 --2- 172.21.15.86:0/4283779869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1184100100 0x7f11841984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 >> 172.21.15.86:0/4283779869 conn(0x7f1184077230 msgr2=0x7f1184104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 shutdown_connections 2024-09-05T07:01:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:03.221+0000 7f1188b18640 1 -- 172.21.15.86:0/4283779869 wait complete. 2024-09-05T07:01:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-05T07:01:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:01:03.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:03.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48260 6.81 MB/sec execute 261 sec latency 6898.027 ms 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:03.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:04.003 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a17ee4b7', total seconds elapsed: 20.2 2024-09-05T07:01:04.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48399 6.82 MB/sec execute 262 sec latency 139.138 ms 2024-09-05T07:01:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 -- 172.21.15.86:0/930862910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac101dd0 msgr2=0x7f9aac1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 --2- 172.21.15.86:0/930862910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac101dd0 0x7f9aac1021b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009900 tx=0x7f9a9c02a400 comp rx=0 tx=0).stop 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 -- 172.21.15.86:0/930862910 shutdown_connections 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 --2- 172.21.15.86:0/930862910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac106f00 0x7f9aac10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 --2- 172.21.15.86:0/930862910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac102780 0x7f9aac1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 --2- 172.21.15.86:0/930862910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac101dd0 0x7f9aac1021b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 -- 172.21.15.86:0/930862910 >> 172.21.15.86:0/930862910 conn(0x7f9aac0fd870 msgr2=0x7f9aac0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 -- 172.21.15.86:0/930862910 shutdown_connections 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.587+0000 7f9ab1982640 1 -- 172.21.15.86:0/930862910 wait complete. 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.588+0000 7f9ab1982640 1 Processor -- start 2024-09-05T07:01:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.588+0000 7f9ab1982640 1 -- start start 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 0x7f9aac19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac106f00 0x7f9aac19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac114040 con 0x7f9aac101dd0 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac113ec0 con 0x7f9aac106f00 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9ab1982640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1141c0 con 0x7f9aac102780 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 0x7f9aac19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:60970/0 (socket says 172.21.15.86:60970) 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaa7fc640 1 -- 172.21.15.86:0/4249105965 learned_addr learned my addr 172.21.15.86:0/4249105965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac106f00 0x7f9aac19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 -- 172.21.15.86:0/4249105965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 msgr2=0x7f9aac19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 -- 172.21.15.86:0/4249105965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac106f00 msgr2=0x7f9aac19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac106f00 0x7f9aac19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaaffd640 1 -- 172.21.15.86:0/4249105965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac19f510 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.589+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9aaaffd640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 0x7f9aac19a560 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c02a9f0 tx=0x7f9a9c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9ab0980640 1 -- 172.21.15.86:0/4249105965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c03f070 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9ab0980640 1 -- 172.21.15.86:0/4249105965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c031b50 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9ab0980640 1 -- 172.21.15.86:0/4249105965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c035030 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac19f7a0 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.590+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac19fc00 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 Processor -- start 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab0980640 1 -- 172.21.15.86:0/4249105965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9a9c042070 con 0x7f9aac101dd0 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 -- start start 2024-09-05T07:01:05.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414784950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x560414787280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604146d72e0 con 0x560414784e90 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56041467ece0 con 0x5604147713a0 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.594+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5604146d32a0 con 0x560414771cc0 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414784950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50220/0 (socket says 172.21.15.86:50220) 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 -- 172.21.15.86:0/247584944 learned_addr learned my addr 172.21.15.86:0/247584944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 -- 172.21.15.86:0/247584944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 msgr2=0x560414784950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414784950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.595+0000 7f9aab7fe640 1 -- 172.21.15.86:0/247584944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 msgr2=0x560414787280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:01:05.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x560414787280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aab7fe640 1 -- 172.21.15.86:0/247584944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560414663440 con 0x5604147713a0 2024-09-05T07:01:05.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009f50 tx=0x7f9aa40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9c701a3096593d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aa1ffb640 1 -- 172.21.15.86:0/247584944 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004d20 con 0x5604147713a0 2024-09-05T07:01:05.048 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "4f33e9fe-a4bd-4adf-b798-fa8ae9b0118e": { 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "4f33e9fe-a4bd-4adf-b798-fa8ae9b0118e", 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:05.049 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9ab0980640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 0x7f9a980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aa1ffb640 1 -- 172.21.15.86:0/247584944 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aa4004050 con 0x5604147713a0 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aa1ffb640 1 -- 172.21.15.86:0/247584944 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4005280 con 0x5604147713a0 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.596+0000 7f9aaaffd640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 0x7f9a980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab0980640 1 -- 172.21.15.86:0/4249105965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f9a9c0ef950 con 0x7f9aac101dd0 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/247584944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 msgr2=0x560414771780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009f50 tx=0x7f9aa40118d0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9aaaffd640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 0x7f9a980a5300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0ecd10 tx=0x7f9a9c0f5550 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/247584944 shutdown_connections 2024-09-05T07:01:05.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x560414787280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414784950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/247584944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x560414771780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.597+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/247584944 >> 172.21.15.86:0/247584944 conn(0x56041476da30 msgr2=0x56041476de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.598+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/247584944 shutdown_connections 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.598+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/247584944 wait complete. 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.598+0000 7f9ab3d5b740 1 -- start start 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.598+0000 7f9ab3d5b740 1 Processor -- start 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51018/0 (socket says 172.21.15.86:51018) 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9aab7fe640 1 -- 172.21.15.86:0/3920673869 learned_addr learned my addr 172.21.15.86:0/3920673869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x560414781ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414784e90 0x560414782010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560414787ac0 con 0x5604147713a0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560414692050 con 0x560414771cc0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x560414781ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.599+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560414787c40 con 0x560414784e90 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414784e90 0x560414782010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 -- 172.21.15.86:0/3920673869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414784e90 msgr2=0x560414782010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414784e90 0x560414782010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 -- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 msgr2=0x560414781ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x560414781ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604144483c0 con 0x5604147713a0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x560414781ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003fd0 tx=0x7f9aa4004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453c7cf8777096f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9a8affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.600+0000 7f9ab3d5b740 10 client.0 _ll_register_callbacks cb 0x5604146d2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9a8affd640 1 -- 172.21.15.86:0/3920673869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402b020 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9a8affd640 1 -- 172.21.15.86:0/3920673869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f9aa4005040 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4004050 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9a8affd640 1 -- 172.21.15.86:0/3920673869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9aa4005260 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 10 client.19875 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=45}) -- 0x560414793290 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9a8affd640 1 -- 172.21.15.86:0/3920673869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f9aa4005400 con 0x5604147713a0 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 10 client.19875 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 10 client.19875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 20 client.19875 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560414794a70 0x560414796f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 4 client.19875 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:05.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.601+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604146e4850 con 0x560414794a70 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.602+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560414794a70 0x560414796f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.602+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560414794a70 0x560414796f30 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38434d54b43efab server_cookie=27160ffc968f7fe1 in_seq=0 out_seq=0 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.602+0000 7f9a8affd640 10 client.19875 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9a8affd640 1 -- 172.21.15.86:0/3920673869 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604146e4850 con 0x560414794a70 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9a8affd640 10 client.19875 handle_command_reply: tid=0 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 client.19875 shutdown 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 msgr2=0x560414784800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003fd0 tx=0x7f9aa4004820 comp rx=0 tx=0).stop 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 shutdown_connections 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560414794a70 0x560414796f30 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414784e90 0x560414782010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x560414781ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/3920673869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604147713a0 0x560414784800 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9a8affd640 0 client.19875 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 >> 172.21.15.86:0/3920673869 conn(0x56041476da30 msgr2=0x560414773790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 shutdown_connections 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/3920673869 wait complete. 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.603+0000 7f9ab3d5b740 20 client.19875 trim_cache size 0 max 16384 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 Processor -- start 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 -- start start 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604147713a0 0x560414790d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x56041477d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1aced0 con 0x560414784e90 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1aea10 con 0x560414771cc0 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1ad5a0 con 0x5604147713a0 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.628+0000 7f9aaa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x56041477d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aaa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x56041477d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:50244/0 (socket says 172.21.15.86:50244) 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51028/0 (socket says 172.21.15.86:51028) 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aaa7fc640 1 -- 172.21.15.86:0/2406838329 learned_addr learned my addr 172.21.15.86:0/2406838329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604147713a0 0x560414790d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2406838329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604147713a0 msgr2=0x560414790d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604147713a0 0x560414790d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2406838329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 msgr2=0x56041477d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x56041477d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2406838329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560414782380 con 0x560414784e90 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003a70 tx=0x7f9aa4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b35183168beba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aa17fa640 1 -- 172.21.15.86:0/2406838329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402b070 con 0x560414784e90 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aa17fa640 1 -- 172.21.15.86:0/2406838329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aa4004050 con 0x560414784e90 2024-09-05T07:01:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.629+0000 7f9aa17fa640 1 -- 172.21.15.86:0/2406838329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4005400 con 0x560414784e90 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/2406838329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 msgr2=0x56041479c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003a70 tx=0x7f9aa4025000 comp rx=0 tx=0).stop 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/2406838329 shutdown_connections 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x56041479c840 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560414771cc0 0x56041477d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/2406838329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5604147713a0 0x560414790d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/2406838329 >> 172.21.15.86:0/2406838329 conn(0x56041476da30 msgr2=0x56041477c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/2406838329 shutdown_connections 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.630+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/2406838329 wait complete. 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 -- start start 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 Processor -- start 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x56041478d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414787e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1ae340 con 0x560414784e90 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1adc70 con 0x5604147713a0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1aaed0 con 0x560414771cc0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9ab3d5b740 10 client.0 _ll_register_callbacks cb 0x7f9a9c0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9aab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51038/0 (socket says 172.21.15.86:51038) 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9aab7fe640 1 -- 172.21.15.86:0/1168638582 learned_addr learned my addr 172.21.15.86:0/1168638582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x56041478d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.631+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414787e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 -- 172.21.15.86:0/1168638582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 msgr2=0x560414787e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414787e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 -- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 msgr2=0x56041478d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x56041478d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 -- 172.21.15.86:0/1168638582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604147fd800 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40045b0 tx=0x7f9aa4005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7837f5b462ea273c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9a8affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402b070 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9605+0+0 (secure 0 0 0) 0x7f9aa40426a0 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4004050 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9aa40428a0 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9ab3d5b740 10 client.19881 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.632+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5604147fea30 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aa4042200 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f9aa403e440 con 0x560414784e90 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 10 client.19881 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 10 client.19881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 20 client.19881 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5604147ff080 0x560414801470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 4 client.19881 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9aa4004050 con 0x5604147ff080 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5604147ff080 0x560414801470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9a8affd640 10 client.19881 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.633+0000 7f9aaa7fc640 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5604147ff080 0x560414801470 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ba9afd6540ee21 server_cookie=10310231c24a8ab7 in_seq=0 out_seq=0 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9a8affd640 1 -- 172.21.15.86:0/1168638582 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f9aa4004050 con 0x5604147ff080 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9a8affd640 10 client.19881 handle_command_reply: tid=0 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 client.19881 shutdown 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 msgr2=0x5604147885f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40045b0 tx=0x7f9aa4005cf0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 shutdown_connections 2024-09-05T07:01:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5604147ff080 0x560414801470 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560414784e90 0x5604147885f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560414771cc0 0x560414787e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 --2- 172.21.15.86:0/1168638582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604147713a0 0x56041478d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 >> 172.21.15.86:0/1168638582 conn(0x56041476da30 msgr2=0x56041477d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9a8affd640 0 client.19881 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 shutdown_connections 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 1 -- 172.21.15.86:0/1168638582 wait complete. 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.634+0000 7f9ab3d5b740 20 client.19881 trim_cache size 0 max 16384 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 msgr2=0x7f9a980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 0x7f9a980a5300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0ecd10 tx=0x7f9a9c0f5550 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 msgr2=0x7f9aac19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 0x7f9aac19a560 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c02a9f0 tx=0x7f9a9c003b90 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 shutdown_connections 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a980a2e40 0x7f9a980a5300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9aac106f00 0x7f9aac19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9aac102780 0x7f9aac19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 --2- 172.21.15.86:0/4249105965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aac101dd0 0x7f9aac19a560 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:05.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 >> 172.21.15.86:0/4249105965 conn(0x7f9aac0fd870 msgr2=0x7f9aac0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:05.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 shutdown_connections 2024-09-05T07:01:05.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:04.635+0000 7f9ab1982640 1 -- 172.21.15.86:0/4249105965 wait complete. 2024-09-05T07:01:05.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:05 smithi115 ceph-mon[34791]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-05T07:01:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:05 smithi086 ceph-mon[30943]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-05T07:01:05.767 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48557 6.84 MB/sec execute 263 sec latency 184.927 ms 2024-09-05T07:01:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:05 smithi096 ceph-mon[34468]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-05T07:01:06.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48681 6.84 MB/sec execute 264 sec latency 170.719 ms 2024-09-05T07:01:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:07 smithi086 ceph-mon[30943]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-05T07:01:07.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:07 smithi086 ceph-mon[30943]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T07:01:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:07 smithi115 ceph-mon[34791]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-05T07:01:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:07 smithi115 ceph-mon[34791]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T07:01:07.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48826 6.84 MB/sec execute 265 sec latency 298.239 ms 2024-09-05T07:01:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:07 smithi096 ceph-mon[34468]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-05T07:01:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:07 smithi096 ceph-mon[34468]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T07:01:08.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48966 6.83 MB/sec execute 266 sec latency 220.539 ms 2024-09-05T07:01:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:09 smithi115 ceph-mon[34791]: pgmap v626: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:09 smithi086 ceph-mon[30943]: pgmap v626: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:09.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49109 6.83 MB/sec execute 267 sec latency 130.047 ms 2024-09-05T07:01:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:09 smithi096 ceph-mon[34468]: pgmap v626: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:09.986 INFO:tasks.cephfs.filesystem:scrub status for tag:4f33e9fe-a4bd-4adf-b798-fa8ae9b0118e - {'path': '/', 'tag': '4f33e9fe-a4bd-4adf-b798-fa8ae9b0118e', 'options': 'recursive,force'} 2024-09-05T07:01:10.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49256 6.83 MB/sec execute 268 sec latency 100.945 ms 2024-09-05T07:01:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:11 smithi115 ceph-mon[34791]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:11 smithi086 ceph-mon[30943]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:11.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49394 6.82 MB/sec execute 269 sec latency 249.049 ms 2024-09-05T07:01:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:11 smithi096 ceph-mon[34468]: pgmap v627: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-09-05T07:01:12.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49591 6.85 MB/sec execute 270 sec latency 66.493 ms 2024-09-05T07:01:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:13 smithi115 ceph-mon[34791]: pgmap v628: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-09-05T07:01:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:13 smithi086 ceph-mon[30943]: pgmap v628: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-09-05T07:01:13.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49781 6.86 MB/sec execute 271 sec latency 66.790 ms 2024-09-05T07:01:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:13 smithi096 ceph-mon[34468]: pgmap v628: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-09-05T07:01:14.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49961 6.87 MB/sec execute 272 sec latency 63.351 ms 2024-09-05T07:01:15.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:15 smithi115 ceph-mon[34791]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:15 smithi086 ceph-mon[30943]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:15.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50126 6.87 MB/sec execute 273 sec latency 103.709 ms 2024-09-05T07:01:15.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:15 smithi096 ceph-mon[34468]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:16.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:01:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:01:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:01:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:01:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:01:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:01:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:01:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:01:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:01:16.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:01:16.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50313 6.88 MB/sec execute 274 sec latency 57.339 ms 2024-09-05T07:01:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:01:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:01:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:01:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:01:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:01:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:17 smithi115 ceph-mon[34791]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:17 smithi115 ceph-mon[34791]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T07:01:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:17 smithi086 ceph-mon[30943]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:17 smithi086 ceph-mon[30943]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T07:01:17.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50493 6.88 MB/sec execute 275 sec latency 120.715 ms 2024-09-05T07:01:17.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:17 smithi096 ceph-mon[34468]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 218 op/s 2024-09-05T07:01:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:17 smithi096 ceph-mon[34468]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T07:01:18.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50681 6.89 MB/sec execute 276 sec latency 72.339 ms 2024-09-05T07:01:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:19 smithi086 ceph-mon[30943]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:19.768 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50856 6.90 MB/sec execute 277 sec latency 127.491 ms 2024-09-05T07:01:19.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:19 smithi096 ceph-mon[34468]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:19 smithi115 ceph-mon[34791]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:20.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51048 6.93 MB/sec execute 278 sec latency 59.457 ms 2024-09-05T07:01:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:21 smithi086 ceph-mon[30943]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:21.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51243 6.96 MB/sec execute 279 sec latency 72.783 ms 2024-09-05T07:01:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:21 smithi096 ceph-mon[34468]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:21 smithi115 ceph-mon[34791]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.2 MiB/s wr, 280 op/s 2024-09-05T07:01:22.576 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T07:01:22.576 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.35 2024-09-05T07:01:22.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:01:22.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51466 6.98 MB/sec execute 280 sec latency 66.063 ms 2024-09-05T07:01:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 -- 172.21.15.86:0/912031554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 msgr2=0x7fb56410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 --2- 172.21.15.86:0/912031554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56410ea90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009ca0 tx=0x7fb55c02a8c0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 -- 172.21.15.86:0/912031554 shutdown_connections 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 --2- 172.21.15.86:0/912031554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56410ea90 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 --2- 172.21.15.86:0/912031554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb5641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 --2- 172.21.15.86:0/912031554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 -- 172.21.15.86:0/912031554 >> 172.21.15.86:0/912031554 conn(0x7fb5640fd880 msgr2=0x7fb5640ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:23.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 -- 172.21.15.86:0/912031554 shutdown_connections 2024-09-05T07:01:23.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.540+0000 7fb56ada0640 1 -- 172.21.15.86:0/912031554 wait complete. 2024-09-05T07:01:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 Processor -- start 2024-09-05T07:01:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 -- start start 2024-09-05T07:01:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb564198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb564111e20 con 0x7fb564106f10 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb564111ca0 con 0x7fb564102790 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.541+0000 7fb56ada0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb564111fa0 con 0x7fb564101de0 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36336/0 (socket says 172.21.15.86:36336) 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 -- 172.21.15.86:0/4177695107 learned_addr learned my addr 172.21.15.86:0/4177695107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb563fff640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb564198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb56933d640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 -- 172.21.15.86:0/4177695107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 msgr2=0x7fb564198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb564198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 -- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 msgr2=0x7fb56419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 -- 172.21.15.86:0/4177695107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56419d280 con 0x7fb564101de0 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb56933d640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb563fff640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb564198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb568b3c640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb55800e9f0 tx=0x7fb55800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55800cd80 con 0x7fb564101de0 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56419d570 con 0x7fb564101de0 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5580054d0 con 0x7fb564101de0 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558010490 con 0x7fb564101de0 2024-09-05T07:01:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.542+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb564074420 con 0x7fb564101de0 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.544+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5641032c0 con 0x7fb564101de0 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.545+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb558005020 con 0x7fb564101de0 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.546+0000 7fb561ffb640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 0x7fb5540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.546+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fb5580cfdd0 con 0x7fb564101de0 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.546+0000 7fb563fff640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 0x7fb5540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.546+0000 7fb563fff640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 0x7fb5540a5230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb5640fdca0 tx=0x7fb550005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.550+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5580d3030 con 0x7fb564101de0 2024-09-05T07:01:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:23 smithi086 ceph-mon[30943]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 285 op/s 2024-09-05T07:01:23.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51619 6.98 MB/sec execute 281 sec latency 104.734 ms 2024-09-05T07:01:23.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:23 smithi096 ceph-mon[34468]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 285 op/s 2024-09-05T07:01:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.903+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb56410a150 con 0x7fb564101de0 2024-09-05T07:01:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.904+0000 7fb561ffb640 1 -- 172.21.15.86:0/4177695107 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7fb55809c550 con 0x7fb564101de0 2024-09-05T07:01:23.905 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:01:23.905 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:01:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:01:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 msgr2=0x7fb5540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 0x7fb5540a5230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb5640fdca0 tx=0x7fb550005f90 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 msgr2=0x7fb5641982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb55800e9f0 tx=0x7fb55800edf0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 shutdown_connections 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb5540a2d70 0x7fb5540a5230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb564106f10 0x7fb56419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb564102790 0x7fb564198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 --2- 172.21.15.86:0/4177695107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb564101de0 0x7fb5641982d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 >> 172.21.15.86:0/4177695107 conn(0x7fb5640fd880 msgr2=0x7fb5640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 shutdown_connections 2024-09-05T07:01:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:23.908+0000 7fb56ada0640 1 -- 172.21.15.86:0/4177695107 wait complete. 2024-09-05T07:01:23.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:23 smithi115 ceph-mon[34791]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 285 op/s 2024-09-05T07:01:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4177695107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:24.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51745 6.98 MB/sec execute 282 sec latency 76.037 ms 2024-09-05T07:01:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4177695107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:24.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4177695107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:25.708 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:25 smithi086 ceph-mon[30943]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:25.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51860 6.98 MB/sec execute 283 sec latency 497.256 ms 2024-09-05T07:01:25.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:25 smithi096 ceph-mon[34468]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:25.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:25 smithi115 ceph-mon[34791]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:26.020 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 139.35 --await-for 45.0' 2024-09-05T07:01:26.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 139.35 --await-for 45.0 2024-09-05T07:01:26.407 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:26.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52012 6.99 MB/sec execute 284 sec latency 115.739 ms 2024-09-05T07:01:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:27 smithi086 ceph-mon[30943]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:27.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52145 6.98 MB/sec execute 285 sec latency 151.728 ms 2024-09-05T07:01:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:27 smithi096 ceph-mon[34468]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:27.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:27 smithi115 ceph-mon[34791]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 285 op/s 2024-09-05T07:01:28.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52265 6.98 MB/sec execute 286 sec latency 278.634 ms 2024-09-05T07:01:29.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:29 smithi086 ceph-mon[30943]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-05T07:01:29.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 -- 172.21.15.86:0/819646082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4101de0 msgr2=0x7f20b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 --2- 172.21.15.86:0/819646082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4101de0 0x7f20b41021c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009900 tx=0x7f20b002a410 comp rx=0 tx=0).stop 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 -- 172.21.15.86:0/819646082 shutdown_connections 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 --2- 172.21.15.86:0/819646082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4106f10 0x7f20b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 --2- 172.21.15.86:0/819646082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 0x7f20b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 --2- 172.21.15.86:0/819646082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4101de0 0x7f20b41021c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 -- 172.21.15.86:0/819646082 >> 172.21.15.86:0/819646082 conn(0x7f20b40fd8c0 msgr2=0x7f20b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 -- 172.21.15.86:0/819646082 shutdown_connections 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.218+0000 7f20bbc66640 1 -- 172.21.15.86:0/819646082 wait complete. 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 Processor -- start 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 -- start start 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 0x7f20b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 0x7f20b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4111e30 con 0x7f20b4102790 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4111cb0 con 0x7f20b4106f10 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20bbc66640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4111fb0 con 0x7f20b4101de0 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20ba203640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 0x7f20b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.219+0000 7f20ba203640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:58576/0 (socket says 172.21.15.86:58576) 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20ba203640 1 -- 172.21.15.86:0/734170315 learned_addr learned my addr 172.21.15.86:0/734170315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9201640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 0x7f20b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 -- 172.21.15.86:0/734170315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 msgr2=0x7f20b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 -- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 msgr2=0x7f20b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 0x7f20b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 -- 172.21.15.86:0/734170315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41a67f0 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20ba203640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20b9a02640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 0x7f20b40744f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20b002aa00 tx=0x7f20b0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20aaffd640 1 -- 172.21.15.86:0/734170315 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b003f070 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20aaffd640 1 -- 172.21.15.86:0/734170315 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b0031b50 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.220+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41a6a80 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.221+0000 7f20aaffd640 1 -- 172.21.15.86:0/734170315 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0035030 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.221+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a6f30 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.223+0000 7f20aaffd640 1 -- 172.21.15.86:0/734170315 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20b0042070 con 0x7f20b4101de0 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.224+0000 7f20be03f740 1 Processor -- start 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.224+0000 7f20aaffd640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 0x7f20a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.224+0000 7f20be03f740 1 -- start start 2024-09-05T07:01:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20aaffd640 1 -- 172.21.15.86:0/734170315 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f20b00efd80 con 0x7f20b4101de0 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b182c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b18210160 con 0x561b182bf240 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b181b7720 con 0x561b182b4980 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20be03f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b1820c120 con 0x561b182b4060 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20ba203640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 0x7f20a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36364/0 (socket says 172.21.15.86:36364) 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 -- 172.21.15.86:0/3907008400 learned_addr learned my addr 172.21.15.86:0/3907008400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9201640 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20ba203640 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b182c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 -- 172.21.15.86:0/3907008400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 msgr2=0x561b182bed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182bed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 -- 172.21.15.86:0/3907008400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 msgr2=0x561b182c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b182c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.225+0000 7f20b9a02640 1 -- 172.21.15.86:0/3907008400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b17f81ba0 con 0x561b182b4060 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.226+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20b00c45a0 tx=0x7f20b0003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9491304abdb0ba9e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.226+0000 7f20ba203640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 0x7f20a00a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0049d0 tx=0x7f20ac013460 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.226+0000 7f209ffff640 1 -- 172.21.15.86:0/3907008400 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0049050 con 0x561b182b4060 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.226+0000 7f209ffff640 1 -- 172.21.15.86:0/3907008400 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b00c0210 con 0x561b182b4060 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.226+0000 7f209ffff640 1 -- 172.21.15.86:0/3907008400 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b00ed490 con 0x561b182b4060 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 -- 172.21.15.86:0/3907008400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 msgr2=0x561b182b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20b00c45a0 tx=0x7f20b0003f90 comp rx=0 tx=0).stop 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 -- 172.21.15.86:0/3907008400 shutdown_connections 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b182c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182bed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 --2- 172.21.15.86:0/3907008400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182b4440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 -- 172.21.15.86:0/3907008400 >> 172.21.15.86:0/3907008400 conn(0x561b182b05f0 msgr2=0x561b182b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 -- 172.21.15.86:0/3907008400 shutdown_connections 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.227+0000 7f20be03f740 1 -- 172.21.15.86:0/3907008400 wait complete. 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.228+0000 7f20be03f740 1 -- start start 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.228+0000 7f20be03f740 1 Processor -- start 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.228+0000 7f20be03f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4980 0x561b182bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 0x561b182bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b182c1bd0 con 0x561b182b4060 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b18198900 con 0x561b182bf240 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561b182c1d50 con 0x561b182b4980 2024-09-05T07:01:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34136/0 (socket says 172.21.15.86:34136) 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 -- 172.21.15.86:0/3412443251 learned_addr learned my addr 172.21.15.86:0/3412443251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20be03f740 10 client.0 _ll_register_callbacks cb 0x561b18196780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9201640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4980 0x561b182bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20ba203640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 0x561b182bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 -- 172.21.15.86:0/3412443251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4980 msgr2=0x561b182bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4980 0x561b182bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 -- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 msgr2=0x561b182bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 0x561b182bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 -- 172.21.15.86:0/3412443251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b181d8c50 con 0x561b182b4060 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20ba203640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 0x561b182bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ed690 tx=0x7f20b00bc090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cef4d9eb1e6e76c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.229+0000 7f209d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b00b9fb0 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f20b00b8580 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b00c0210 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f20b0049020 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f20be03f740 10 client.19899 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b182c1bd0 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b00c0210 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f20b00bb360 con 0x561b182b4060 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.230+0000 7f20be03f740 10 client.19899 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20be03f740 10 client.19899 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20be03f740 20 client.19899 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b182cea30 0x561b182d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20be03f740 4 client.19899 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b1821d890 con 0x561b182cea30 2024-09-05T07:01:29.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20b9201640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b182cea30 0x561b182d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f20b9201640 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b182cea30 0x561b182d0e20 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ba75a5501a6fd4 server_cookie=755a485f9f238ab9 in_seq=0 out_seq=0 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.231+0000 7f209d7fa640 10 client.19899 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f209d7fa640 1 -- 172.21.15.86:0/3412443251 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b1821d890 con 0x561b182cea30 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f209d7fa640 10 client.19899 handle_command_reply: tid=0 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 client.19899 shutdown 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 msgr2=0x561b182bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ed690 tx=0x7f20b00bc090 comp rx=0 tx=0).stop 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 shutdown_connections 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b182cea30 0x561b182d0e20 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182bf240 0x561b182bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4980 0x561b182bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 --2- 172.21.15.86:0/3412443251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4060 0x561b182bc1f0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 >> 172.21.15.86:0/3412443251 conn(0x561b182b05f0 msgr2=0x561b182a6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f209d7fa640 0 client.19899 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 shutdown_connections 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 1 -- 172.21.15.86:0/3412443251 wait complete. 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.232+0000 7f20be03f740 20 client.19899 trim_cache size 0 max 16384 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.261+0000 7f20be03f740 1 Processor -- start 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.261+0000 7f20be03f740 1 -- start start 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.261+0000 7f20be03f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4060 0x561b182bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.261+0000 7f20be03f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4980 0x561b182d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.261+0000 7f20be03f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20be03f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41aceb0 con 0x561b182b4980 2024-09-05T07:01:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20be03f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41ae9f0 con 0x561b182b4060 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20be03f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41ad580 con 0x561b182bf240 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36378/0 (socket says 172.21.15.86:36378) 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 -- 172.21.15.86:0/3935184551 learned_addr learned my addr 172.21.15.86:0/3935184551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20b9201640 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4980 0x561b182d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20b9a02640 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4060 0x561b182bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 -- 172.21.15.86:0/3935184551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4060 msgr2=0x561b182bb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4060 0x561b182bb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 -- 172.21.15.86:0/3935184551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4980 msgr2=0x561b182d66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4980 0x561b182d66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 -- 172.21.15.86:0/3935184551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b182b47b0 con 0x561b182bf240 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f20ba203640 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20ac010710 tx=0x7f20ac008000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fdb65e35a913d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.262+0000 7f209f7fe640 1 -- 172.21.15.86:0/3935184551 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac025070 con 0x561b182bf240 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f209f7fe640 1 -- 172.21.15.86:0/3935184551 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20ac017040 con 0x561b182bf240 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f209f7fe640 1 -- 172.21.15.86:0/3935184551 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac021970 con 0x561b182bf240 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 -- 172.21.15.86:0/3935184551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 msgr2=0x561b182d6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20ac010710 tx=0x7f20ac008000 comp rx=0 tx=0).stop 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 -- 172.21.15.86:0/3935184551 shutdown_connections 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182bf240 0x561b182d6c00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182b4980 0x561b182d66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 --2- 172.21.15.86:0/3935184551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4060 0x561b182bb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 -- 172.21.15.86:0/3935184551 >> 172.21.15.86:0/3935184551 conn(0x561b182b05f0 msgr2=0x561b182bc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 -- 172.21.15.86:0/3935184551 shutdown_connections 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.263+0000 7f20be03f740 1 -- 172.21.15.86:0/3935184551 wait complete. 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 -- start start 2024-09-05T07:01:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 Processor -- start 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41ae320 con 0x561b182bf240 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41adc50 con 0x561b182b4980 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41aaeb0 con 0x561b182b4060 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20be03f740 10 client.0 _ll_register_callbacks cb 0x7f20a40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20ba203640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20b9a02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:36390/0 (socket says 172.21.15.86:36390) 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20ba203640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34158/0 (socket says 172.21.15.86:34158) 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20b9a02640 1 -- 172.21.15.86:0/475006921 learned_addr learned my addr 172.21.15.86:0/475006921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:29.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.264+0000 7f20b9201640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 -- 172.21.15.86:0/475006921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 msgr2=0x561b182c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 -- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 msgr2=0x561b182c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 -- 172.21.15.86:0/475006921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b18338bd0 con 0x561b182bf240 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20b9a02640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20b9201640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20ba203640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f20ac013960 tx=0x7f20ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1804ea077735e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f209d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac025040 con 0x561b182bf240 2024-09-05T07:01:29.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20ac017040 con 0x561b182bf240 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 74, 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "49e8861d": { 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 74, 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:29.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 139.3, 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:29.769 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f20ac0217b0 con 0x561b182bf240 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f20ac0219f0 con 0x561b182bf240 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20be03f740 10 client.19905 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b18339db0 con 0x561b182bf240 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.265+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20ac017040 con 0x561b182bf240 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f20ac031070 con 0x561b182bf240 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 10 client.19905 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 10 client.19905 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 20 client.19905 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b1833a440 0x561b1833c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 4 client.19905 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.35, "await_for": 45.0}] 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.35, "await_for": 45.0}) -- 0x561b18217be0 con 0x561b1833a440 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20b9a02640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b1833a440 0x561b1833c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f209d7fa640 10 client.19905 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.266+0000 7f20b9a02640 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b1833a440 0x561b1833c830 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234a7f2c529cee37 server_cookie=b8e34ee243157fd0 in_seq=0 out_seq=0 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f209d7fa640 1 -- 172.21.15.86:0/475006921 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561b18217be0 con 0x561b1833a440 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f209d7fa640 10 client.19905 handle_command_reply: tid=0 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 client.19905 shutdown 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 msgr2=0x561b18338690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f20ac013960 tx=0x7f20ac005bd0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 shutdown_connections 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x561b1833a440 0x561b1833c830 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b182bf240 0x561b18338690 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b182b4980 0x561b182c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f209d7fa640 0 client.19905 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 --2- 172.21.15.86:0/475006921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561b182b4060 0x561b182c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 >> 172.21.15.86:0/475006921 conn(0x561b182b05f0 msgr2=0x561b182a66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 shutdown_connections 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 1 -- 172.21.15.86:0/475006921 wait complete. 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.590+0000 7f20be03f740 20 client.19905 trim_cache size 0 max 16384 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 msgr2=0x7f20a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 0x7f20a00a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0049d0 tx=0x7f20ac013460 comp rx=0 tx=0).stop 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 msgr2=0x7f20b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 0x7f20b40744f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20b002aa00 tx=0x7f20b0003b90 comp rx=0 tx=0).stop 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 shutdown_connections 2024-09-05T07:01:29.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f20a00a2cf0 0x7f20a00a51b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b4106f10 0x7f20b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b4102790 0x7f20b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 --2- 172.21.15.86:0/734170315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20b4101de0 0x7f20b40744f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 >> 172.21.15.86:0/734170315 conn(0x7f20b40fd8c0 msgr2=0x7f20b40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 shutdown_connections 2024-09-05T07:01:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:29.591+0000 7f20bbc66640 1 -- 172.21.15.86:0/734170315 wait complete. 2024-09-05T07:01:29.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.96 MB/sec execute 287 sec latency 507.172 ms 2024-09-05T07:01:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:29 smithi096 ceph-mon[34468]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-05T07:01:29.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:29 smithi115 ceph-mon[34791]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-05T07:01:30.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.94 MB/sec execute 288 sec latency 1507.219 ms 2024-09-05T07:01:31.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '49e8861d', quiesce took 8.8 seconds. Will release after: 10.55 2024-09-05T07:01:31.735 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:31 smithi115 ceph-mon[34791]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:01:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:31 smithi086 ceph-mon[30943]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:01:31.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.92 MB/sec execute 289 sec latency 2507.297 ms 2024-09-05T07:01:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:31 smithi096 ceph-mon[34468]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:01:32.769 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.89 MB/sec execute 290 sec latency 3507.371 ms 2024-09-05T07:01:33.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:33 smithi086 ceph-mon[30943]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 306 op/s 2024-09-05T07:01:33.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.87 MB/sec execute 291 sec latency 4507.461 ms 2024-09-05T07:01:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:33 smithi096 ceph-mon[34468]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 306 op/s 2024-09-05T07:01:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:33 smithi115 ceph-mon[34791]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 306 op/s 2024-09-05T07:01:34.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.85 MB/sec execute 292 sec latency 5507.564 ms 2024-09-05T07:01:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:35 smithi086 ceph-mon[30943]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-05T07:01:35.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.82 MB/sec execute 293 sec latency 6507.676 ms 2024-09-05T07:01:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:35 smithi096 ceph-mon[34468]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-05T07:01:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:35 smithi115 ceph-mon[34791]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-05T07:01:36.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.80 MB/sec execute 294 sec latency 7507.781 ms 2024-09-05T07:01:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:37 smithi086 ceph-mon[30943]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-09-05T07:01:37.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.78 MB/sec execute 295 sec latency 8507.868 ms 2024-09-05T07:01:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:37 smithi096 ceph-mon[34468]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-09-05T07:01:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:37 smithi115 ceph-mon[34791]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-09-05T07:01:38.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.75 MB/sec execute 296 sec latency 9507.953 ms 2024-09-05T07:01:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:39 smithi086 ceph-mon[30943]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-09-05T07:01:39.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.73 MB/sec execute 297 sec latency 10508.022 ms 2024-09-05T07:01:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:39 smithi096 ceph-mon[34468]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-09-05T07:01:39.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:39 smithi115 ceph-mon[34791]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 203 op/s 2024-09-05T07:01:39.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:01:40.249 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:40.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.71 MB/sec execute 298 sec latency 11508.054 ms 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/24876880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8101450 msgr2=0x7fd2c8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/24876880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8101450 0x7fd2c8105920 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009900 tx=0x7fd2b802a470 comp rx=0 tx=0).stop 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/24876880 shutdown_connections 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/24876880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8105f40 0x7fd2c81063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/24876880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8101450 0x7fd2c8105920 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/24876880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8100b30 0x7fd2c8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/24876880 >> 172.21.15.86:0/24876880 conn(0x7fd2c80fc7a0 msgr2=0x7fd2c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/24876880 shutdown_connections 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.768+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/24876880 wait complete. 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 Processor -- start 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 -- start start 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 0x7fd2c81a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 0x7fd2c819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81184c0 con 0x7fd2c8100b30 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8118340 con 0x7fd2c8105f40 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cf2ba640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8118640 con 0x7fd2c8101450 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58126/0 (socket says 172.21.15.86:58126) 2024-09-05T07:01:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 -- 172.21.15.86:0/4228820211 learned_addr learned my addr 172.21.15.86:0/4228820211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd857640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 0x7fd2c819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 -- 172.21.15.86:0/4228820211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 msgr2=0x7fd2c81a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 0x7fd2c81a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 0x7fd2c81a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 -- 172.21.15.86:0/4228820211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 msgr2=0x7fd2c819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 0x7fd2c819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd056640 1 -- 172.21.15.86:0/4228820211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c819ef90 con 0x7fd2c8100b30 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cd857640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 0x7fd2c819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.769+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 0x7fd2c81a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c920 tx=0x7fd2c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2be7fc640 1 -- 172.21.15.86:0/4228820211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4007c70 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2be7fc640 1 -- 172.21.15.86:0/4228820211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c4007e10 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c819f280 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2be7fc640 1 -- 172.21.15.86:0/4228820211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4018820 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.770+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c81ab4d0 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.772+0000 7fd2be7fc640 1 -- 172.21.15.86:0/4228820211 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd2c40189c0 con 0x7fd2c8100b30 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 Processor -- start 2024-09-05T07:01:40.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 -- start start 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b192880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f6b0f82e0 con 0x555f6b192dc0 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f6b09fce0 con 0x555f6b19d680 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2d1693740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f6b0f42a0 con 0x555f6b1924a0 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58136/0 (socket says 172.21.15.86:58136) 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 -- 172.21.15.86:0/2983481120 learned_addr learned my addr 172.21.15.86:0/2983481120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2be7fc640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 0x7fd2b40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd857640 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b192880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cc855640 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd857640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 0x7fd2b40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2be7fc640 1 -- 172.21.15.86:0/4228820211 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fd2c40cfa70 con 0x7fd2c8100b30 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 -- 172.21.15.86:0/2983481120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 msgr2=0x555f6b192880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b192880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 -- 172.21.15.86:0/2983481120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 msgr2=0x555f6b19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.773+0000 7fd2cd056640 1 -- 172.21.15.86:0/2983481120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6b084440 con 0x555f6b192dc0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.774+0000 7fd2cd056640 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fd2c409b7f0 tx=0x7fd2c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f237afe0031c4157 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.774+0000 7fd2cd857640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 0x7fd2b40a5230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009fd0 tx=0x7fd2c0008220 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.774+0000 7fd2b37fe640 1 -- 172.21.15.86:0/2983481120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40cfe60 con 0x555f6b192dc0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.774+0000 7fd2b37fe640 1 -- 172.21.15.86:0/2983481120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c400fb00 con 0x555f6b192dc0 2024-09-05T07:01:40.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.774+0000 7fd2b37fe640 1 -- 172.21.15.86:0/2983481120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40a4760 con 0x555f6b192dc0 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- 172.21.15.86:0/2983481120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 msgr2=0x555f6b19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fd2c409b7f0 tx=0x7fd2c4000bc0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- 172.21.15.86:0/2983481120 shutdown_connections 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19d140 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- 172.21.15.86:0/2983481120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b192880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- 172.21.15.86:0/2983481120 >> 172.21.15.86:0/2983481120 conn(0x555f6b18ea30 msgr2=0x555f6b18ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- 172.21.15.86:0/2983481120 shutdown_connections 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- 172.21.15.86:0/2983481120 wait complete. 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- start start 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 Processor -- start 2024-09-05T07:01:40.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b1924a0 0x555f6b19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b192dc0 0x555f6b19a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f6b1a0250 con 0x555f6b19d680 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f6b0b3050 con 0x555f6b1924a0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f6b1a03d0 con 0x555f6b192dc0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.775+0000 7fd2d1693740 10 client.0 _ll_register_callbacks cb 0x555f6b0f3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58150/0 (socket says 172.21.15.86:58150) 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 -- 172.21.15.86:0/3545841983 learned_addr learned my addr 172.21.15.86:0/3545841983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cc855640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b192dc0 0x555f6b19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd056640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b1924a0 0x555f6b19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 -- 172.21.15.86:0/3545841983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b192dc0 msgr2=0x555f6b19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b192dc0 0x555f6b19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 -- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b1924a0 msgr2=0x555f6b19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b1924a0 0x555f6b19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 -- 172.21.15.86:0/3545841983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6ae693c0 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2cd857640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0015ca0 tx=0x7fd2c0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8805f8cce00d2954 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00253f0 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fd2c0025590 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f6b105850 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd2c00257d0 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2d1693740 10 client.19917 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.776+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555f6b1abc20 con 0x555f6b19d680 2024-09-05T07:01:40.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555f6b105850 con 0x555f6b19d680 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fd2c001a850 con 0x555f6b19d680 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 10 client.19917 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 10 client.19917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 20 client.19917 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b1ad0e0 0x555f6b1af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 4 client.19917 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f6b0ffd60 con 0x555f6b1ad0e0 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2cd056640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b1ad0e0 0x555f6b1af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2cd056640 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b1ad0e0 0x555f6b1af540 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b5015d6608a93a server_cookie=142a093556dd604d in_seq=0 out_seq=0 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.777+0000 7fd2b0ff9640 10 client.19917 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/3545841983 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f6b0ffd60 con 0x555f6b1ad0e0 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2b0ff9640 10 client.19917 handle_command_reply: tid=0 2024-09-05T07:01:40.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 client.19917 shutdown 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 msgr2=0x555f6b19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0015ca0 tx=0x7fd2c0004b50 comp rx=0 tx=0).stop 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 shutdown_connections 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b1ad0e0 0x555f6b1af540 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b19d680 0x555f6b19a7e0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b192dc0 0x555f6b19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 --2- 172.21.15.86:0/3545841983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b1924a0 0x555f6b19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2b0ff9640 0 client.19917 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.778+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 >> 172.21.15.86:0/3545841983 conn(0x555f6b18ea30 msgr2=0x555f6b194710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.779+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 shutdown_connections 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.779+0000 7fd2d1693740 1 -- 172.21.15.86:0/3545841983 wait complete. 2024-09-05T07:01:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.779+0000 7fd2d1693740 20 client.19917 trim_cache size 0 max 16384 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.795+0000 7fd2d1693740 1 Processor -- start 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.795+0000 7fd2d1693740 1 -- start start 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b57f0 con 0x555f6b192dc0 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b7330 con 0x555f6b19d680 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2d1693740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b5ec0 con 0x555f6b1924a0 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58158/0 (socket says 172.21.15.86:58158) 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 -- 172.21.15.86:0/4139715775 learned_addr learned my addr 172.21.15.86:0/4139715775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 -- 172.21.15.86:0/4139715775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 msgr2=0x555f6b1a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 -- 172.21.15.86:0/4139715775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 msgr2=0x555f6b1b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 -- 172.21.15.86:0/4139715775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6b19d340 con 0x555f6b192dc0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009900 tx=0x7fd2b80043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e742f3ee1ea92c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2b2ffd640 1 -- 172.21.15.86:0/4139715775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8035040 con 0x555f6b192dc0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2b2ffd640 1 -- 172.21.15.86:0/4139715775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2b8041070 con 0x555f6b192dc0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.796+0000 7fd2b2ffd640 1 -- 172.21.15.86:0/4139715775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8003850 con 0x555f6b192dc0 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- 172.21.15.86:0/4139715775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 msgr2=0x555f6b19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009900 tx=0x7fd2b80043f0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- 172.21.15.86:0/4139715775 shutdown_connections 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b19c1f0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4139715775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- 172.21.15.86:0/4139715775 >> 172.21.15.86:0/4139715775 conn(0x555f6b18ea30 msgr2=0x555f6b19bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- 172.21.15.86:0/4139715775 shutdown_connections 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- 172.21.15.86:0/4139715775 wait complete. 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 -- start start 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.797+0000 7fd2d1693740 1 Processor -- start 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b6c60 con 0x555f6b192dc0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b6590 con 0x555f6b19d680 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81b37f0 con 0x555f6b1924a0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2d1693740 10 client.0 _ll_register_callbacks cb 0x7fd2c000e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58160/0 (socket says 172.21.15.86:58160) 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 -- 172.21.15.86:0/4171090537 learned_addr learned my addr 172.21.15.86:0/4171090537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cd857640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 -- 172.21.15.86:0/4171090537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 msgr2=0x555f6b1a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 -- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 msgr2=0x555f6b1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.798+0000 7fd2cc855640 1 -- 172.21.15.86:0/4171090537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6b1a1ab0 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2cc855640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8006010 tx=0x7fd2b80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2b64424edcd67f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8035030 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fd2b8005040 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b8041070 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd2b805a050 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 10 client.19923 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555f6b217c20 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2b8041070 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fd2b8003ce0 con 0x555f6b192dc0 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 10 client.19923 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 10 client.19923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 20 client.19923 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b218270 0x555f6b21a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 4 client.19923 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f6b07c210 con 0x555f6b218270 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.799+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b218270 0x555f6b21a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2cd056640 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b218270 0x555f6b21a690 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b10003582c236a server_cookie=7aa88cfeb4e104c6 in_seq=0 out_seq=0 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2b0ff9640 10 client.19923 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2b0ff9640 1 -- 172.21.15.86:0/4171090537 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f6b07c210 con 0x555f6b218270 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2b0ff9640 10 client.19923 handle_command_reply: tid=0 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 client.19923 shutdown 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 msgr2=0x555f6b1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8006010 tx=0x7fd2b80048f0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 shutdown_connections 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x555f6b218270 0x555f6b21a690 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:01:40.821 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f6b19d680 0x555f6b1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f6b192dc0 0x555f6b1a0f00 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2b0ff9640 0 client.19923 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 --2- 172.21.15.86:0/4171090537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f6b1924a0 0x555f6b1a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 >> 172.21.15.86:0/4171090537 conn(0x555f6b18ea30 msgr2=0x555f6b18d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 shutdown_connections 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 1 -- 172.21.15.86:0/4171090537 wait complete. 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.800+0000 7fd2d1693740 20 client.19923 trim_cache size 0 max 16384 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 msgr2=0x7fd2b40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 0x7fd2b40a5230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009fd0 tx=0x7fd2c0008220 comp rx=0 tx=0).stop 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 msgr2=0x7fd2c81a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c920 tx=0x7fd2c400cde0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 shutdown_connections 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd2b40a2d70 0x7fd2b40a5230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c8105f40 0x7fd2c819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2c8101450 0x7fd2c81a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 --2- 172.21.15.86:0/4228820211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c8100b30 0x7fd2c81a4670 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 >> 172.21.15.86:0/4228820211 conn(0x7fd2c80fc7a0 msgr2=0x7fd2c81038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 shutdown_connections 2024-09-05T07:01:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:40.801+0000 7fd2cf2ba640 1 -- 172.21.15.86:0/4228820211 wait complete. 2024-09-05T07:01:41.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:01:41.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:01:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:41 smithi086 ceph-mon[30943]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-09-05T07:01:41.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.68 MB/sec execute 299 sec latency 12508.082 ms 2024-09-05T07:01:41.802 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:41.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:41 smithi096 ceph-mon[34468]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-09-05T07:01:41.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:01:41.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:41 smithi115 ceph-mon[34791]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-09-05T07:01:42.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.323+0000 7f3b3c121640 1 -- 172.21.15.86:0/1176227403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34105f40 msgr2=0x7f3b341063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- 172.21.15.86:0/1176227403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34105f40 0x7f3b341063d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009900 tx=0x7f3b2c02a470 comp rx=0 tx=0).stop 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- 172.21.15.86:0/1176227403 shutdown_connections 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- 172.21.15.86:0/1176227403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34105f40 0x7f3b341063d0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- 172.21.15.86:0/1176227403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b34105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- 172.21.15.86:0/1176227403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34100b30 0x7f3b34100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- 172.21.15.86:0/1176227403 >> 172.21.15.86:0/1176227403 conn(0x7f3b340fc7a0 msgr2=0x7f3b340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- 172.21.15.86:0/1176227403 shutdown_connections 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- 172.21.15.86:0/1176227403 wait complete. 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 Processor -- start 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- start start 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b341a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34105f40 0x7f3b3419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.324+0000 7f3b3c121640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34118410 con 0x7f3b34100b30 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b3c121640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34118290 con 0x7f3b34101450 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b3c121640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34118590 con 0x7f3b34105f40 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58184/0 (socket says 172.21.15.86:58184) 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 -- 172.21.15.86:0/2807258374 learned_addr learned my addr 172.21.15.86:0/2807258374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b3a6be640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34105f40 0x7f3b3419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b396bc640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b341a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 -- 172.21.15.86:0/2807258374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34105f40 msgr2=0x7f3b3419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34105f40 0x7f3b3419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 -- 172.21.15.86:0/2807258374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 msgr2=0x7f3b341a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b341a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 -- 172.21.15.86:0/2807258374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419ef30 con 0x7f3b34100b30 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b396bc640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b341a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b39ebd640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400cce0 tx=0x7f3b2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24004b20 con 0x7f3b34100b30 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b24004cc0 con 0x7f3b34100b30 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3419f220 con 0x7f3b34100b30 2024-09-05T07:01:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400f6b0 con 0x7f3b34100b30 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.325+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341ab570 con 0x7f3b34100b30 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.326+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3406b070 con 0x7f3b34100b30 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.327+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b2400f850 con 0x7f3b34100b30 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.328+0000 7f3b32ffd640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 0x7f3b1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.328+0000 7f3b396bc640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 0x7f3b1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.328+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f3b240cf950 con 0x7f3b34100b30 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.329+0000 7f3b396bc640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 0x7f3b1c0a51b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b28005dc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.332+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b24014030 con 0x7f3b34100b30 2024-09-05T07:01:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.547+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b341ab740 con 0x7f3b34100b30 2024-09-05T07:01:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.547+0000 7f3b32ffd640 1 -- 172.21.15.86:0/2807258374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7f3b2409c130 con 0x7f3b34100b30 2024-09-05T07:01:42.549 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:01:42.549 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:01:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 msgr2=0x7f3b1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 0x7f3b1c0a51b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b28005dc0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 msgr2=0x7f3b341a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400cce0 tx=0x7f3b2400a620 comp rx=0 tx=0).stop 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 shutdown_connections 2024-09-05T07:01:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3b1c0a2cf0 0x7f3b1c0a51b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b34105f40 0x7f3b3419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b34101450 0x7f3b341a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 --2- 172.21.15.86:0/2807258374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34100b30 0x7f3b341a45f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 >> 172.21.15.86:0/2807258374 conn(0x7f3b340fc7a0 msgr2=0x7f3b341038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 shutdown_connections 2024-09-05T07:01:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.550+0000 7f3b3c121640 1 -- 172.21.15.86:0/2807258374 wait complete. 2024-09-05T07:01:42.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.66 MB/sec execute 300 sec latency 13508.108 ms 2024-09-05T07:01:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 -- 172.21.15.86:0/3355732167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390102ef0 msgr2=0x7fa39010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 --2- 172.21.15.86:0/3355732167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390102ef0 0x7fa39010da40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa388009ca0 tx=0x7fa38802a950 comp rx=0 tx=0).stop 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 -- 172.21.15.86:0/3355732167 shutdown_connections 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 --2- 172.21.15.86:0/3355732167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390102ef0 0x7fa39010da40 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 --2- 172.21.15.86:0/3355732167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 0x7fa3901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 --2- 172.21.15.86:0/3355732167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390100e20 0x7fa390101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.929+0000 7fa397887640 1 -- 172.21.15.86:0/3355732167 >> 172.21.15.86:0/3355732167 conn(0x7fa3900fc7d0 msgr2=0x7fa3900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- 172.21.15.86:0/3355732167 shutdown_connections 2024-09-05T07:01:42.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- 172.21.15.86:0/3355732167 wait complete. 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 Processor -- start 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- start start 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 0x7fa39019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390102ef0 0x7fa39019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa390110e30 con 0x7fa390100e20 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa390110cb0 con 0x7fa390102ef0 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa397887640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa390110fb0 con 0x7fa390102570 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa395623640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.930+0000 7fa395623640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38282/0 (socket says 172.21.15.86:38282) 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 -- 172.21.15.86:0/2580050155 learned_addr learned my addr 172.21.15.86:0/2580050155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395e24640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390102ef0 0x7fa39019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa394e22640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 0x7fa39019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 -- 172.21.15.86:0/2580050155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 msgr2=0x7fa39019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 0x7fa39019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 -- 172.21.15.86:0/2580050155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390102ef0 msgr2=0x7fa39019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390102ef0 0x7fa39019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 -- 172.21.15.86:0/2580050155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39019f580 con 0x7fa390100e20 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa395623640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c890 tx=0x7fa38c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c004220 con 0x7fa390100e20 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa38c0043c0 con 0x7fa390100e20 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39019f870 con 0x7fa390100e20 2024-09-05T07:01:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c004a00 con 0x7fa390100e20 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.931+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3901a7150 con 0x7fa390100e20 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.932+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa390101200 con 0x7fa390100e20 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.934+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa38c004ba0 con 0x7fa390100e20 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.935+0000 7fa3867fc640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 0x7fa37c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.935+0000 7fa394e22640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 0x7fa37c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.935+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fa38c0cf990 con 0x7fa390100e20 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.935+0000 7fa394e22640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 0x7fa37c0a51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa39010df10 tx=0x7fa38000c4a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:42.938+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa38c014030 con 0x7fa390100e20 2024-09-05T07:01:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.142+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa39019b930 con 0x7fa390100e20 2024-09-05T07:01:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.142+0000 7fa3867fc640 1 -- 172.21.15.86:0/2580050155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7fa38c09c110 con 0x7fa390100e20 2024-09-05T07:01:43.145 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:01:43.145 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:01:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:01:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.144+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 msgr2=0x7fa37c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.144+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 0x7fa37c0a51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa39010df10 tx=0x7fa38000c4a0 comp rx=0 tx=0).stop 2024-09-05T07:01:43.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.144+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 msgr2=0x7fa39019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.144+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c890 tx=0x7fa38c00cd50 comp rx=0 tx=0).stop 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 shutdown_connections 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa37c0a2cf0 0x7fa37c0a51b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa390102ef0 0x7fa39019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa390102570 0x7fa39019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 --2- 172.21.15.86:0/2580050155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa390100e20 0x7fa39019a5d0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 >> 172.21.15.86:0/2580050155 conn(0x7fa3900fc7d0 msgr2=0x7fa390069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 shutdown_connections 2024-09-05T07:01:43.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:43.145+0000 7fa397887640 1 -- 172.21.15.86:0/2580050155 wait complete. 2024-09-05T07:01:43.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:01:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:43 smithi086 ceph-mon[30943]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-09-05T07:01:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:43 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2807258374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:43 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2580050155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:43.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.64 MB/sec execute 301 sec latency 14508.159 ms 2024-09-05T07:01:43.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:43 smithi096 ceph-mon[34468]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-09-05T07:01:43.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:43 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2807258374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:43.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:43 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2580050155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:43 smithi115 ceph-mon[34791]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-09-05T07:01:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2807258374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:43 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2580050155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:44.001 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:44.454 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 49e8861d --release --await' 2024-09-05T07:01:44.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 49e8861d --release --await 2024-09-05T07:01:44.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.62 MB/sec execute 302 sec latency 15508.182 ms 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 -- 172.21.15.86:0/2439257173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 msgr2=0x7f676810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 --2- 172.21.15.86:0/2439257173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676810da40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6760009ca0 tx=0x7f676002a950 comp rx=0 tx=0).stop 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 -- 172.21.15.86:0/2439257173 shutdown_connections 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 --2- 172.21.15.86:0/2439257173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676810da40 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 --2- 172.21.15.86:0/2439257173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768102570 0x7f67681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 --2- 172.21.15.86:0/2439257173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768100e20 0x7f6768101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 -- 172.21.15.86:0/2439257173 >> 172.21.15.86:0/2439257173 conn(0x7f67680fc7d0 msgr2=0x7f67680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 -- 172.21.15.86:0/2439257173 shutdown_connections 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 -- 172.21.15.86:0/2439257173 wait complete. 2024-09-05T07:01:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.920+0000 7f676cb49640 1 Processor -- start 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 -- start start 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 0x7f6768198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768102570 0x7f6768198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6768111f70 con 0x7f6768102ef0 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6768111df0 con 0x7f6768102570 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676cb49640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67681120f0 con 0x7f6768100e20 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38314/0 (socket says 172.21.15.86:38314) 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 -- 172.21.15.86:0/2014954713 learned_addr learned my addr 172.21.15.86:0/2014954713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6765d9b640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768102570 0x7f6768198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676659c640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 0x7f6768198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 -- 172.21.15.86:0/2014954713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 msgr2=0x7f6768198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 0x7f6768198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 -- 172.21.15.86:0/2014954713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768102570 msgr2=0x7f6768198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768102570 0x7f6768198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 -- 172.21.15.86:0/2014954713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676819d3c0 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f676659c640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 0x7f6768198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.921+0000 7f6766d9d640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6760006010 tx=0x7f676002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.922+0000 7f67577fe640 1 -- 172.21.15.86:0/2014954713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760034040 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.922+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676819d650 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.922+0000 7f67577fe640 1 -- 172.21.15.86:0/2014954713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6760003e00 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.922+0000 7f67577fe640 1 -- 172.21.15.86:0/2014954713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760045480 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.922+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67681a4f90 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.924+0000 7f67577fe640 1 -- 172.21.15.86:0/2014954713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6760037020 con 0x7f6768102ef0 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.924+0000 7f676ef22740 1 Processor -- start 2024-09-05T07:01:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.924+0000 7f676ef22740 1 -- start start 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe15a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56499fd7b2f0 con 0x56499fe16b40 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56499fd22cf0 con 0x56499fe155d0 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676ef22740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56499fd772b0 con 0x56499fe20370 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f67577fe640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 0x7f67500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38318/0 (socket says 172.21.15.86:38318) 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 -- 172.21.15.86:0/1010554637 learned_addr learned my addr 172.21.15.86:0/1010554637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6766d9d640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 0x7f67500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676659c640 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe15a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f67577fe640 1 -- 172.21.15.86:0/2014954713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f67600f08a0 con 0x7f6768102ef0 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 -- 172.21.15.86:0/1010554637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 msgr2=0x56499fe22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 -- 172.21.15.86:0/1010554637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 msgr2=0x56499fe15a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe15a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 -- 172.21.15.86:0/1010554637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499fd073a0 con 0x56499fe16b40 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f676659c640 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe15a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6766d9d640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 0x7f67500a51b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f67600c5170 tx=0x7f6760095040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.925+0000 7f6765d9b640 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f675c00a500 tx=0x7f675c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34f3376bc3e3f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f6754ff9640 1 -- 172.21.15.86:0/1010554637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c005020 con 0x56499fe16b40 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f6754ff9640 1 -- 172.21.15.86:0/1010554637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f675c0047a0 con 0x56499fe16b40 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f6754ff9640 1 -- 172.21.15.86:0/1010554637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c012650 con 0x56499fe16b40 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 -- 172.21.15.86:0/1010554637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 msgr2=0x56499fe15090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f675c00a500 tx=0x7f675c0117b0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 -- 172.21.15.86:0/1010554637 shutdown_connections 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe15a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 --2- 172.21.15.86:0/1010554637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe15090 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.926+0000 7f676ef22740 1 -- 172.21.15.86:0/1010554637 >> 172.21.15.86:0/1010554637 conn(0x56499fdde310 msgr2=0x56499fe11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- 172.21.15.86:0/1010554637 shutdown_connections 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- 172.21.15.86:0/1010554637 wait complete. 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- start start 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 Processor -- start 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe1cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56499fe23070 con 0x56499fe20370 2024-09-05T07:01:44.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56499fd36060 con 0x56499fe155d0 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56499fe231f0 con 0x56499fe16b40 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f676ef22740 10 client.0 _ll_register_callbacks cb 0x56499fa898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.927+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38332/0 (socket says 172.21.15.86:38332) 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 -- 172.21.15.86:0/3237536336 learned_addr learned my addr 172.21.15.86:0/3237536336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f676659c640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe1cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 -- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 msgr2=0x56499fe1cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe1cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 -- 172.21.15.86:0/3237536336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 msgr2=0x56499fe1fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 -- 172.21.15.86:0/3237536336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499faec3c0 con 0x56499fe20370 2024-09-05T07:01:44.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f6765d9b640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f675c005c90 tx=0x7f675c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94b5050f9e16542 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f674dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c023040 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c0047a0 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f675c012450 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f675c004cb0 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f676ef22740 10 client.19947 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56499fe2ead0 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.928+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f675c0047a0 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f675c00f4b0 con 0x56499fe20370 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 10 client.19947 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 10 client.19947 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 20 client.19947 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe2fed0 0x56499fe322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 4 client.19947 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499fd88860 con 0x56499fe2fed0 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676659c640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe2fed0 0x56499fe322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f674dffb640 10 client.19947 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.929+0000 7f676659c640 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe2fed0 0x56499fe322c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df98ad42c2ced436 server_cookie=91a6f0fc87953f6f in_seq=0 out_seq=0 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f674dffb640 1 -- 172.21.15.86:0/3237536336 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56499fd88860 con 0x56499fe2fed0 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f674dffb640 10 client.19947 handle_command_reply: tid=0 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 client.19947 shutdown 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 msgr2=0x56499fe1d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f675c005c90 tx=0x7f675c00a570 comp rx=0 tx=0).stop 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 shutdown_connections 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe2fed0 0x56499fe322c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe1d5d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe1cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f674dffb640 0 client.19947 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 --2- 172.21.15.86:0/3237536336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 >> 172.21.15.86:0/3237536336 conn(0x56499fdde310 msgr2=0x56499fe17990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 shutdown_connections 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.930+0000 7f676ef22740 1 -- 172.21.15.86:0/3237536336 wait complete. 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.931+0000 7f676ef22740 20 client.19947 trim_cache size 0 max 16384 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.943+0000 7f676ef22740 1 Processor -- start 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.943+0000 7f676ef22740 1 -- start start 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67681af0f0 con 0x56499fe20370 2024-09-05T07:01:44.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b0c30 con 0x56499fe155d0 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676ef22740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67681af7c0 con 0x56499fe16b40 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38342/0 (socket says 172.21.15.86:38342) 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 -- 172.21.15.86:0/2196587590 learned_addr learned my addr 172.21.15.86:0/2196587590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f676659c640 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 -- 172.21.15.86:0/2196587590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 msgr2=0x56499fe18120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe18120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 -- 172.21.15.86:0/2196587590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 msgr2=0x56499fe1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 -- 172.21.15.86:0/2196587590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499fe16300 con 0x56499fe20370 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f6765d9b640 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f675c00ac60 tx=0x7f675c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b099706b10e40244 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f674ffff640 1 -- 172.21.15.86:0/2196587590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c032040 con 0x56499fe20370 2024-09-05T07:01:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f674ffff640 1 -- 172.21.15.86:0/2196587590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f675c00a690 con 0x56499fe20370 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.944+0000 7f674ffff640 1 -- 172.21.15.86:0/2196587590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c00f420 con 0x56499fe20370 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- 172.21.15.86:0/2196587590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 msgr2=0x56499fe18660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f675c00ac60 tx=0x7f675c00a340 comp rx=0 tx=0).stop 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- 172.21.15.86:0/2196587590 shutdown_connections 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe20370 0x56499fe18660 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe16b40 0x56499fe18120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 --2- 172.21.15.86:0/2196587590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- 172.21.15.86:0/2196587590 >> 172.21.15.86:0/2196587590 conn(0x56499fdde310 msgr2=0x56499fe1e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- 172.21.15.86:0/2196587590 shutdown_connections 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- 172.21.15.86:0/2196587590 wait complete. 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 -- start start 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.945+0000 7f676ef22740 1 Processor -- start 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b0560 con 0x56499fe16b40 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67681afe90 con 0x56499fe155d0 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67681ad0f0 con 0x56499fe20370 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 10 client.0 _ll_register_callbacks cb 0x7f67580071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38348/0 (socket says 172.21.15.86:38348) 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 -- 172.21.15.86:0/724736470 learned_addr learned my addr 172.21.15.86:0/724736470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f6765d9b640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 -- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 msgr2=0x56499fe24d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe24d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f6765d9b640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 -- 172.21.15.86:0/724736470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 msgr2=0x56499fe242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 -- 172.21.15.86:0/724736470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499fe99a50 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676659c640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x56499fe12ac0 tx=0x7f675801a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1000f4ad1d7b9313 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f674dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580198c0 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f6758019a60 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.946+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c00a690 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6758019c60 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 10 client.19953 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56499fe9ad60 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f675c00a690 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f6758006600 con 0x56499fe16b40 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 10 client.19953 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:44.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 10 client.19953 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 20 client.19953 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe9b530 0x56499fe9d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 4 client.19953 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56499fd82d70 con 0x56499fe9b530 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.947+0000 7f6765d9b640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe9b530 0x56499fe9d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f674dffb640 10 client.19953 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f6765d9b640 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe9b530 0x56499fe9d950 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e64604306466cc8 server_cookie=bba6a4bc047aa004 in_seq=0 out_seq=0 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f674dffb640 1 -- 172.21.15.86:0/724736470 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56499fd82d70 con 0x56499fe9b530 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f674dffb640 10 client.19953 handle_command_reply: tid=0 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 client.19953 shutdown 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 msgr2=0x56499fe247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x56499fe12ac0 tx=0x7f675801a7d0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 shutdown_connections 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x56499fe9b530 0x56499fe9d950 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56499fe20370 0x56499fe24d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56499fe16b40 0x56499fe247f0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f674dffb640 0 client.19953 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 --2- 172.21.15.86:0/724736470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56499fe155d0 0x56499fe242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 >> 172.21.15.86:0/724736470 conn(0x56499fdde310 msgr2=0x56499fe10d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 shutdown_connections 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 1 -- 172.21.15.86:0/724736470 wait complete. 2024-09-05T07:01:44.985 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.948+0000 7f676ef22740 20 client.19953 trim_cache size 0 max 16384 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 msgr2=0x7f67500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 0x7f67500a51b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f67600c5170 tx=0x7f6760095040 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 msgr2=0x7f676819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6760006010 tx=0x7f676002f710 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 shutdown_connections 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f67500a2cf0 0x7f67500a51b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6768102ef0 0x7f676819cce0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6768102570 0x7f6768198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 --2- 172.21.15.86:0/2014954713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6768100e20 0x7f6768198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 >> 172.21.15.86:0/2014954713 conn(0x7f67680fc7d0 msgr2=0x7f67680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 shutdown_connections 2024-09-05T07:01:44.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:44.949+0000 7f676cb49640 1 -- 172.21.15.86:0/2014954713 wait complete. 2024-09-05T07:01:45.285 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:45 smithi086 ceph-mon[30943]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 179 KiB/s wr, 19 op/s 2024-09-05T07:01:45.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52346 6.60 MB/sec execute 303 sec latency 16508.236 ms 2024-09-05T07:01:45.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:01:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:45 smithi096 ceph-mon[34468]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 179 KiB/s wr, 19 op/s 2024-09-05T07:01:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:45 smithi115 ceph-mon[34791]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 179 KiB/s wr, 19 op/s 2024-09-05T07:01:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- 172.21.15.86:0/1541180521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28102ef0 msgr2=0x7f0c2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1541180521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28102ef0 0x7f0c2810da40 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c2002a480 comp rx=0 tx=0).stop 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- 172.21.15.86:0/1541180521 shutdown_connections 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1541180521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28102ef0 0x7f0c2810da40 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1541180521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 0x7f0c281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1541180521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c28100e20 0x7f0c28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- 172.21.15.86:0/1541180521 >> 172.21.15.86:0/1541180521 conn(0x7f0c280fc7d0 msgr2=0x7f0c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- 172.21.15.86:0/1541180521 shutdown_connections 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- 172.21.15.86:0/1541180521 wait complete. 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 Processor -- start 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.311+0000 7f0c2e053640 1 -- start start 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 0x7f0c28199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c28102ef0 0x7f0c281a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28110ed0 con 0x7f0c28100e20 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28110d50 con 0x7f0c28102ef0 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111050 con 0x7f0c28102570 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38356/0 (socket says 172.21.15.86:38356) 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 -- 172.21.15.86:0/1234504506 learned_addr learned my addr 172.21.15.86:0/1234504506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 0x7f0c28199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 -- 172.21.15.86:0/1234504506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 msgr2=0x7f0c28199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 0x7f0c28199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 -- 172.21.15.86:0/1234504506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c28102ef0 msgr2=0x7f0c281a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c28102ef0 0x7f0c281a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 -- 172.21.15.86:0/1234504506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c281a48c0 con 0x7f0c28100e20 2024-09-05T07:01:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ba50 tx=0x7f0c1800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c24ff9640 1 -- 172.21.15.86:0/1234504506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18019070 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c24ff9640 1 -- 172.21.15.86:0/1234504506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c1800cbe0 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c24ff9640 1 -- 172.21.15.86:0/1234504506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c180075d0 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c281a4a90 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.312+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281a4f50 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c24ff9640 1 -- 172.21.15.86:0/1234504506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0c1801c070 con 0x7f0c28100e20 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 Processor -- start 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 -- start start 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 0x5611e0a14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 0x5611e0a17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5611e096f620 con 0x5611e0a0a060 2024-09-05T07:01:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5611e0916e50 con 0x5611e0a15240 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c3042c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5611e096b5e0 con 0x5611e0a0a980 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38358/0 (socket says 172.21.15.86:38358) 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c27fff640 1 -- 172.21.15.86:0/137243250 learned_addr learned my addr 172.21.15.86:0/137243250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c24ff9640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c277fe640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 0x5611e0a14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c27fff640 1 -- 172.21.15.86:0/137243250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 msgr2=0x5611e0a14d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.315+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 0x5611e0a17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 0x5611e0a14d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 -- 172.21.15.86:0/137243250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 msgr2=0x5611e0a17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 0x5611e0a17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c24ff9640 1 -- 172.21.15.86:0/1234504506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f0c180cede0 con 0x7f0c28100e20 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 -- 172.21.15.86:0/137243250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e0907d40 con 0x5611e0a0a060 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 0x5611e0a17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c277fe640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 0x5611e0a14d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009a20 tx=0x7f0c20032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c302419b063ca053 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c0dffb640 1 -- 172.21.15.86:0/137243250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20035040 con 0x5611e0a0a060 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c0dffb640 1 -- 172.21.15.86:0/137243250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c200046c0 con 0x5611e0a0a060 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c0dffb640 1 -- 172.21.15.86:0/137243250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200318b0 con 0x5611e0a0a060 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.316+0000 7f0c27fff640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 0x7f0c100a51b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c20048000 tx=0x7f0c20045000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- 172.21.15.86:0/137243250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 msgr2=0x5611e0a0a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009a20 tx=0x7f0c20032cc0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- 172.21.15.86:0/137243250 shutdown_connections 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a15240 0x5611e0a17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a980 0x5611e0a14d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 --2- 172.21.15.86:0/137243250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a0a440 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- 172.21.15.86:0/137243250 >> 172.21.15.86:0/137243250 conn(0x5611e0a065f0 msgr2=0x5611e0a069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- 172.21.15.86:0/137243250 shutdown_connections 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- 172.21.15.86:0/137243250 wait complete. 2024-09-05T07:01:46.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.317+0000 7f0c3042c740 1 -- start start 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 Processor -- start 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a12620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a12b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5611e0a17e10 con 0x5611e0a0a060 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5611e08f7f10 con 0x5611e0a0a980 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5611e0a17f90 con 0x5611e0a15240 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c3042c740 10 client.0 _ll_register_callbacks cb 0x5611e091bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38360/0 (socket says 172.21.15.86:38360) 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 -- 172.21.15.86:0/1195867038 learned_addr learned my addr 172.21.15.86:0/1195867038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a12620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 -- 172.21.15.86:0/1195867038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 msgr2=0x5611e0a12b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a12b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 -- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 msgr2=0x5611e0a12620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a12620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.318+0000 7f0c277fe640 1 -- 172.21.15.86:0/1195867038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e067e580 con 0x5611e0a0a060 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c277fe640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809acd0 tx=0x7f0c180145e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b13e95cc42dd260 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18023040 con 0x5611e0a0a060 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f0c1800f160 con 0x5611e0a0a060 2024-09-05T07:01:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c200046c0 con 0x5611e0a0a060 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0c18018170 con 0x5611e0a0a060 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c3042c740 10 client.19965 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.319+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5611e0748960 con 0x5611e0a0a060 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c200046c0 con 0x5611e0a0a060 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0c1809b230 con 0x5611e0a0a060 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 10 client.19965 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 10 client.19965 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 20 client.19965 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a24d60 0x5611e0a271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 4 client.19965 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e097cb90 con 0x5611e0a24d60 2024-09-05T07:01:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.320+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a24d60 0x5611e0a271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.321+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a24d60 0x5611e0a271d0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35d37062c1d529e server_cookie=f05d749bcdb351c9 in_seq=0 out_seq=0 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.321+0000 7f0c06ffd640 10 client.19965 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.321+0000 7f0c06ffd640 1 -- 172.21.15.86:0/1195867038 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611e097cb90 con 0x5611e0a24d60 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.321+0000 7f0c06ffd640 10 client.19965 handle_command_reply: tid=0 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.321+0000 7f0c3042c740 1 client.19965 shutdown 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 msgr2=0x5611e0a120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0c1809acd0 tx=0x7f0c180145e0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 shutdown_connections 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a24d60 0x5611e0a271d0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a12b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a12620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 --2- 172.21.15.86:0/1195867038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a120e0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 >> 172.21.15.86:0/1195867038 conn(0x5611e0a065f0 msgr2=0x5611e0a05750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c06ffd640 0 client.19965 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 shutdown_connections 2024-09-05T07:01:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 1 -- 172.21.15.86:0/1195867038 wait complete. 2024-09-05T07:01:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.322+0000 7f0c3042c740 20 client.19965 trim_cache size 0 max 16384 2024-09-05T07:01:46.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.348+0000 7f0c3042c740 1 Processor -- start 2024-09-05T07:01:46.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.348+0000 7f0c3042c740 1 -- start start 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 0x5611e0a14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281af310 con 0x5611e0a15240 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b0e50 con 0x5611e0a0a980 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c3042c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281af9e0 con 0x5611e0a0a060 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38370/0 (socket says 172.21.15.86:38370) 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 -- 172.21.15.86:0/609013653 learned_addr learned my addr 172.21.15.86:0/609013653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c277fe640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 0x5611e0a14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 -- 172.21.15.86:0/609013653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 msgr2=0x5611e0a14050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 0x5611e0a14050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 -- 172.21.15.86:0/609013653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 msgr2=0x5611e0a2cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a2cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c27fff640 1 -- 172.21.15.86:0/609013653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e0a11090 con 0x5611e0a15240 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.349+0000 7f0c277fe640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 0x5611e0a14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.350+0000 7f0c27fff640 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0c20032d30 tx=0x7f0c20005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1874c14ed39083 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.350+0000 7f0c0d7fa640 1 -- 172.21.15.86:0/609013653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20057020 con 0x5611e0a15240 2024-09-05T07:01:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.350+0000 7f0c0d7fa640 1 -- 172.21.15.86:0/609013653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c200046c0 con 0x5611e0a15240 2024-09-05T07:01:46.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.350+0000 7f0c0d7fa640 1 -- 172.21.15.86:0/609013653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20035040 con 0x5611e0a15240 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- 172.21.15.86:0/609013653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 msgr2=0x5611e0a2d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0c20032d30 tx=0x7f0c20005a20 comp rx=0 tx=0).stop 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- 172.21.15.86:0/609013653 shutdown_connections 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a15240 0x5611e0a2d020 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a2cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 --2- 172.21.15.86:0/609013653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a0a060 0x5611e0a14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- 172.21.15.86:0/609013653 >> 172.21.15.86:0/609013653 conn(0x5611e0a065f0 msgr2=0x5611e0a13940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- 172.21.15.86:0/609013653 shutdown_connections 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- 172.21.15.86:0/609013653 wait complete. 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 -- start start 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.351+0000 7f0c3042c740 1 Processor -- start 2024-09-05T07:01:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38384/0 (socket says 172.21.15.86:38384) 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b0780 con 0x5611e0a0a060 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 -- 172.21.15.86:0/2371511431 learned_addr learned my addr 172.21.15.86:0/2371511431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281b00b0 con 0x5611e0a0a980 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ad310 con 0x5611e0a15240 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c3042c740 10 client.0 _ll_register_callbacks cb 0x7f0c18097340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 -- 172.21.15.86:0/2371511431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 msgr2=0x5611e0a972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 -- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 msgr2=0x5611e0a19b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e0a97800 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c277fe640 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0c180a3e00 tx=0x7f0c180a37b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee43d2c39478bd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.352+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1809b6d0 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f0c1800f160 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c200046c0 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0c180a3c40 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 10 client.19971 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5611e0a98a50 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c1809afd0 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0c18096d20 con 0x5611e0a0a060 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 10 client.19971 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 10 client.19971 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 20 client.19971 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a98f90 0x5611e0a9b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 4 client.19971 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "49e8861d", "release": true, "await": true}] 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.353+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "49e8861d", "release": true, "await": true}) -- 0x7f0c200046c0 con 0x5611e0a98f90 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.354+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a98f90 0x5611e0a9b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.354+0000 7f0c26ffd640 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a98f90 0x5611e0a9b3a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e334e4a5c19e4e server_cookie=eea5e08a5ddd8e77 in_seq=0 out_seq=0 2024-09-05T07:01:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.354+0000 7f0c06ffd640 10 client.19971 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c06ffd640 1 -- 172.21.15.86:0/2371511431 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f0c200046c0 con 0x5611e0a98f90 2024-09-05T07:01:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c06ffd640 10 client.19971 handle_command_reply: tid=0 2024-09-05T07:01:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 client.19971 shutdown 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 msgr2=0x5611e0a195d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0c180a3e00 tx=0x7f0c180a37b0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 shutdown_connections 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5611e0a98f90 0x5611e0a9b3a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c06ffd640 0 client.19971 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5611e0a15240 0x5611e0a972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5611e0a0a980 0x5611e0a19b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 --2- 172.21.15.86:0/2371511431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611e0a0a060 0x5611e0a195d0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 >> 172.21.15.86:0/2371511431 conn(0x5611e0a065f0 msgr2=0x5611e0a05750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 shutdown_connections 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.386+0000 7f0c3042c740 1 -- 172.21.15.86:0/2371511431 wait complete. 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c3042c740 20 client.19971 trim_cache size 0 max 16384 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 76, 2024-09-05T07:01:46.389 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "49e8861d": { 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 76, 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 139.3, 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:01:46.390 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:46.391 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 msgr2=0x7f0c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 0x7f0c100a51b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c20048000 tx=0x7f0c20045000 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 msgr2=0x7f0c28199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ba50 tx=0x7f0c1800bf10 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 shutdown_connections 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c28102ef0 0x7f0c281a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c28102570 0x7f0c28199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 --2- 172.21.15.86:0/1234504506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c28100e20 0x7f0c28199300 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.387+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 >> 172.21.15.86:0/1234504506 conn(0x7f0c280fc7d0 msgr2=0x7f0c280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.388+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 shutdown_connections 2024-09-05T07:01:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:46.388+0000 7f0c2e053640 1 -- 172.21.15.86:0/1234504506 wait complete. 2024-09-05T07:01:46.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52407 6.58 MB/sec execute 304 sec latency 17134.746 ms 2024-09-05T07:01:47.541 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '49e8861d', total seconds elapsed: 25.0 2024-09-05T07:01:47.541 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.35 2024-09-05T07:01:47.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:01:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:47 smithi086 ceph-mon[30943]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-05T07:01:47.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52595 6.60 MB/sec execute 305 sec latency 82.889 ms 2024-09-05T07:01:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 -- 172.21.15.86:0/3132560281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 msgr2=0x7f9e701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 --2- 172.21.15.86:0/3132560281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e701021d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e6002a410 comp rx=0 tx=0).stop 2024-09-05T07:01:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 -- 172.21.15.86:0/3132560281 shutdown_connections 2024-09-05T07:01:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 --2- 172.21.15.86:0/3132560281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e70106f20 0x7f9e7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 --2- 172.21.15.86:0/3132560281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e701027a0 0x7f9e701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 --2- 172.21.15.86:0/3132560281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e701021d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 -- 172.21.15.86:0/3132560281 >> 172.21.15.86:0/3132560281 conn(0x7f9e700fd8b0 msgr2=0x7f9e700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 -- 172.21.15.86:0/3132560281 shutdown_connections 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.771+0000 7f9e750da640 1 -- 172.21.15.86:0/3132560281 wait complete. 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.772+0000 7f9e750da640 1 Processor -- start 2024-09-05T07:01:47.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.772+0000 7f9e750da640 1 -- start start 2024-09-05T07:01:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e7019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e701027a0 0x7f9e7019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70111e30 con 0x7f9e70101df0 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70111cb0 con 0x7f9e70106f20 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e750da640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70111fb0 con 0x7f9e701027a0 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e7019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34724/0 (socket says 172.21.15.86:34724) 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 -- 172.21.15.86:0/233453560 learned_addr learned my addr 172.21.15.86:0/233453560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:47.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e701027a0 0x7f9e7019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 -- 172.21.15.86:0/233453560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e701027a0 msgr2=0x7f9e7019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e701027a0 0x7f9e7019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 -- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 msgr2=0x7f9e7019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e7019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.773+0000 7f9e6f59e640 1 -- 172.21.15.86:0/233453560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7019f510 con 0x7f9e70106f20 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e6f59e640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eaa0 tx=0x7f9e6800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e57fff640 1 -- 172.21.15.86:0/233453560 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800cdb0 con 0x7f9e70106f20 2024-09-05T07:01:47.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e57fff640 1 -- 172.21.15.86:0/233453560 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e68005020 con 0x7f9e70106f20 2024-09-05T07:01:47.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e57fff640 1 -- 172.21.15.86:0/233453560 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e680116d0 con 0x7f9e70106f20 2024-09-05T07:01:47.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7019f800 con 0x7f9e70106f20 2024-09-05T07:01:47.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.774+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701a70e0 con 0x7f9e70106f20 2024-09-05T07:01:47.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.778+0000 7f9e774b3740 1 Processor -- start 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.778+0000 7f9e774b3740 1 -- start start 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.778+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.778+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b98415d0 0x5558b9841a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.778+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b984c370 0x5558b984e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e774b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558b97a72f0 con 0x5558b98415d0 2024-09-05T07:01:47.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e774b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558b974ecf0 con 0x5558b984c370 2024-09-05T07:01:47.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e774b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5558b97a32b0 con 0x5558b9842b40 2024-09-05T07:01:47.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49848/0 (socket says 172.21.15.86:49848) 2024-09-05T07:01:47.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 -- 172.21.15.86:0/941418026 learned_addr learned my addr 172.21.15.86:0/941418026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:47.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b984c370 0x5558b984e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e57fff640 1 -- 172.21.15.86:0/233453560 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9e680051d0 con 0x7f9e70106f20 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 -- 172.21.15.86:0/941418026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b984c370 msgr2=0x5558b984e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b984c370 0x5558b984e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 -- 172.21.15.86:0/941418026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b98415d0 msgr2=0x5558b9841a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.779+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b98415d0 0x5558b9841a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.780+0000 7f9e6e59c640 1 -- 172.21.15.86:0/941418026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b97333a0 con 0x5558b9842b40 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.780+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b6c0 tx=0x7f9e6400b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6757cfe3cec53482 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.780+0000 7f9e557fa640 1 -- 172.21.15.86:0/941418026 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640106b0 con 0x5558b9842b40 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.780+0000 7f9e557fa640 1 -- 172.21.15.86:0/941418026 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e64010d00 con 0x5558b9842b40 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.780+0000 7f9e557fa640 1 -- 172.21.15.86:0/941418026 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6401db70 con 0x5558b9842b40 2024-09-05T07:01:47.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.781+0000 7f9e57fff640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 0x7f9e580a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.781+0000 7f9e57fff640 1 -- 172.21.15.86:0/233453560 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f9e680cf900 con 0x7f9e70106f20 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.781+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 0x7f9e580a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- 172.21.15.86:0/941418026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 msgr2=0x5558b9841090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b6c0 tx=0x7f9e6400b650 comp rx=0 tx=0).stop 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- 172.21.15.86:0/941418026 shutdown_connections 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b984c370 0x5558b984e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b98415d0 0x5558b9841a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 0x7f9e580a52b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e60031440 tx=0x7f9e60005bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 --2- 172.21.15.86:0/941418026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b9842b40 0x5558b9841090 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- 172.21.15.86:0/941418026 >> 172.21.15.86:0/941418026 conn(0x5558b980a310 msgr2=0x5558b983dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- 172.21.15.86:0/941418026 shutdown_connections 2024-09-05T07:01:47.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- 172.21.15.86:0/941418026 wait complete. 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 -- start start 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.782+0000 7f9e774b3740 1 Processor -- start 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34728/0 (socket says 172.21.15.86:34728) 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38414/0 (socket says 172.21.15.86:38414) 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 -- 172.21.15.86:0/71023688 learned_addr learned my addr 172.21.15.86:0/71023688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b984c370 0x5558b98495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558b984f070 con 0x5558b9842b40 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558b9762060 con 0x5558b98415d0 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5558b984f1f0 con 0x5558b984c370 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6f59e640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b984c370 0x5558b98495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 -- 172.21.15.86:0/71023688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b984c370 msgr2=0x5558b98495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b984c370 0x5558b98495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e6ed9d640 1 -- 172.21.15.86:0/71023688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 msgr2=0x5558b9849000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.783+0000 7f9e774b3740 10 client.0 _ll_register_callbacks cb 0x5558b94b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e6ed9d640 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b95183c0 con 0x5558b98415d0 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e64010d00 con 0x5558b98415d0 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e6002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40dda6d12c70748c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e52ff5640 10 client.28733 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e52ff5640 1 -- 172.21.15.86:0/71023688 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6004d020 con 0x5558b98415d0 2024-09-05T07:01:47.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e52ff5640 1 -- 172.21.15.86:0/71023688 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f9e60004040 con 0x5558b98415d0 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e52ff5640 1 -- 172.21.15.86:0/71023688 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e60035040 con 0x5558b98415d0 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e774b3740 10 client.28733 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.784+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5558b985aa20 con 0x5558b98415d0 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e52ff5640 1 -- 172.21.15.86:0/71023688 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f9e60004240 con 0x5558b98415d0 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 10 client.28733 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 10 client.28733 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 20 client.28733 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b985c330 0x5558b985e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 4 client.28733 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558b97b4860 con 0x5558b985c330 2024-09-05T07:01:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.785+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b985c330 0x5558b985e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b985c330 0x5558b985e7f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1a905e638e3197f server_cookie=e2f8a43a57ece1b7 in_seq=0 out_seq=0 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e52ff5640 10 client.28733 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e52ff5640 1 -- 172.21.15.86:0/71023688 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558b97b4860 con 0x5558b985c330 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e52ff5640 10 client.28733 handle_command_reply: tid=0 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e774b3740 1 client.28733 shutdown 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 msgr2=0x5558b984bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e6002a410 comp rx=0 tx=0).stop 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.786+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 shutdown_connections 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b985c330 0x5558b985e7f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b984c370 0x5558b98495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b9842b40 0x5558b9849000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 --2- 172.21.15.86:0/71023688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b98415d0 0x5558b984bd50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 >> 172.21.15.86:0/71023688 conn(0x5558b980a310 msgr2=0x5558b9843930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e52ff5640 0 client.28733 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:01:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 shutdown_connections 2024-09-05T07:01:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 1 -- 172.21.15.86:0/71023688 wait complete. 2024-09-05T07:01:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.787+0000 7f9e774b3740 20 client.28733 trim_cache size 0 max 16384 2024-09-05T07:01:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:47 smithi096 ceph-mon[34468]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.811+0000 7f9e774b3740 1 Processor -- start 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.811+0000 7f9e774b3740 1 -- start start 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b983cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9844660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701b13d0 con 0x5558b984c370 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701b2f10 con 0x5558b9842b40 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e774b3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701b1aa0 con 0x5558b98415d0 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34740/0 (socket says 172.21.15.86:34740) 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e6e59c640 1 -- 172.21.15.86:0/260033611 learned_addr learned my addr 172.21.15.86:0/260033611 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:47.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e6e59c640 1 -- 172.21.15.86:0/260033611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 msgr2=0x5558b983cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.812+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b983cbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.813+0000 7f9e6e59c640 1 -- 172.21.15.86:0/260033611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 msgr2=0x5558b9844660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.813+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9844660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.813+0000 7f9e6e59c640 1 -- 172.21.15.86:0/260033611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b98423a0 con 0x5558b9842b40 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.813+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e64012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f784572a456415 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.813+0000 7f9e54ff9640 1 -- 172.21.15.86:0/260033611 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6401d3e0 con 0x5558b9842b40 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.814+0000 7f9e54ff9640 1 -- 172.21.15.86:0/260033611 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e640054f0 con 0x5558b9842b40 2024-09-05T07:01:47.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.814+0000 7f9e54ff9640 1 -- 172.21.15.86:0/260033611 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6401d6f0 con 0x5558b9842b40 2024-09-05T07:01:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 -- 172.21.15.86:0/260033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 msgr2=0x5558b9844120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e64012f90 comp rx=0 tx=0).stop 2024-09-05T07:01:47.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 -- 172.21.15.86:0/260033611 shutdown_connections 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9844660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9844120 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 --2- 172.21.15.86:0/260033611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b983cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 -- 172.21.15.86:0/260033611 >> 172.21.15.86:0/260033611 conn(0x5558b980a310 msgr2=0x5558b983c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 -- 172.21.15.86:0/260033611 shutdown_connections 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.815+0000 7f9e774b3740 1 -- 172.21.15.86:0/260033611 wait complete. 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e774b3740 1 -- start start 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e774b3740 1 Processor -- start 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e774b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9861a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.816+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49870/0 (socket says 172.21.15.86:49870) 2024-09-05T07:01:47.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34754/0 (socket says 172.21.15.86:34754) 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6ed9d640 1 -- 172.21.15.86:0/3480523465 learned_addr learned my addr 172.21.15.86:0/3480523465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701b2840 con 0x5558b984c370 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701b2170 con 0x5558b9842b40 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701af3d0 con 0x5558b98415d0 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e774b3740 10 client.0 _ll_register_callbacks cb 0x7f9e60040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6f59e640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9861a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 -- 172.21.15.86:0/3480523465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 msgr2=0x5558b9860fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 -- 172.21.15.86:0/3480523465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 msgr2=0x5558b9861a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9861a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b98cdaa0 con 0x5558b9842b40 2024-09-05T07:01:47.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e6e59c640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5558b984a640 tx=0x7f9e6401d3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd1ddaaf71af37f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.817+0000 7f9e52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640106b0 con 0x5558b9842b40 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f9e64010850 con 0x5558b9842b40 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e640054f0 con 0x5558b9842b40 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e64010ab0 con 0x5558b9842b40 2024-09-05T07:01:47.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e774b3740 10 client.28739 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5558b98ceda0 con 0x5558b9842b40 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.818+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e6401dba0 con 0x5558b9842b40 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f9e6401c070 con 0x5558b9842b40 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 10 client.28739 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 10 client.28739 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 20 client.28739 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b98cf210 0x5558b98d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 4 client.28739 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9e640054f0 con 0x5558b98cf210 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b98cf210 0x5558b98d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e6ed9d640 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b98cf210 0x5558b98d1600 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af90ee0769b565f2 server_cookie=980290f940effc17 in_seq=0 out_seq=0 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.819+0000 7f9e52ff5640 10 client.28739 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e52ff5640 1 -- 172.21.15.86:0/3480523465 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9e640054f0 con 0x5558b98cf210 2024-09-05T07:01:47.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e52ff5640 10 client.28739 handle_command_reply: tid=0 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 client.28739 shutdown 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 msgr2=0x5558b9861500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5558b984a640 tx=0x7f9e6401d3e0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 shutdown_connections 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5558b98cf210 0x5558b98d1600 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e52ff5640 0 client.28739 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b984c370 0x5558b9861a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558b9842b40 0x5558b9861500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 --2- 172.21.15.86:0/3480523465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5558b98415d0 0x5558b9860fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 >> 172.21.15.86:0/3480523465 conn(0x5558b980a310 msgr2=0x5558b983cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 shutdown_connections 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 1 -- 172.21.15.86:0/3480523465 wait complete. 2024-09-05T07:01:47.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.820+0000 7f9e774b3740 20 client.28739 trim_cache size 0 max 16384 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 msgr2=0x7f9e580a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 0x7f9e580a52b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e60031440 tx=0x7f9e60005bc0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 msgr2=0x7f9e7019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eaa0 tx=0x7f9e6800ef60 comp rx=0 tx=0).stop 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 shutdown_connections 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e580a2df0 0x7f9e580a52b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70106f20 0x7f9e7019ee30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e701027a0 0x7f9e7019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 --2- 172.21.15.86:0/233453560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e70101df0 0x7f9e7019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:47.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 >> 172.21.15.86:0/233453560 conn(0x7f9e700fd8b0 msgr2=0x7f9e700febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:47.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 shutdown_connections 2024-09-05T07:01:47.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:47.821+0000 7f9e750da640 1 -- 172.21.15.86:0/233453560 wait complete. 2024-09-05T07:01:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:47 smithi115 ceph-mon[34791]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-05T07:01:48.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:01:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:48.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52748 6.59 MB/sec execute 306 sec latency 87.206 ms 2024-09-05T07:01:49.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2485684566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd601036e0 msgr2=0x7fbd60107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2485684566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd601036e0 0x7fbd60107b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009890 tx=0x7fbd5402a360 comp rx=0 tx=0).stop 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2485684566 shutdown_connections 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2485684566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601082a0 0x7fbd6010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2485684566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd601036e0 0x7fbd60107b90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2485684566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd60102d30 0x7fbd60103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2485684566 >> 172.21.15.86:0/2485684566 conn(0x7fbd600fc7d0 msgr2=0x7fbd600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2485684566 shutdown_connections 2024-09-05T07:01:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.349+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2485684566 wait complete. 2024-09-05T07:01:49.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.350+0000 7fbd64fa2640 1 Processor -- start 2024-09-05T07:01:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.351+0000 7fbd64fa2640 1 -- start start 2024-09-05T07:01:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 0x7fbd6019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34774/0 (socket says 172.21.15.86:34774) 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49890/0 (socket says 172.21.15.86:49890) 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60111f10 con 0x7fbd601082a0 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 0x7fbd6019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60111d90 con 0x7fbd60102d30 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 0x7fbd6019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38446/0 (socket says 172.21.15.86:38446) 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd5ed9d640 1 -- 172.21.15.86:0/2267727256 learned_addr learned my addr 172.21.15.86:0/2267727256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:49.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.352+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60112090 con 0x7fbd601036e0 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5e59c640 1 -- 172.21.15.86:0/2267727256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 msgr2=0x7fbd60198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5e59c640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5e59c640 1 -- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 msgr2=0x7fbd6019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5e59c640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 0x7fbd6019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5e59c640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6019d480 con 0x7fbd60102d30 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.353+0000 7fbd5dd9b640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:49.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.355+0000 7fbd5e59c640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009c70 tx=0x7fbd5000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:49.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.355+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50012270 con 0x7fbd60102d30 2024-09-05T07:01:49.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.355+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd50011c00 con 0x7fbd60102d30 2024-09-05T07:01:49.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.355+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5001a760 con 0x7fbd60102d30 2024-09-05T07:01:49.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.355+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6019d770 con 0x7fbd60102d30 2024-09-05T07:01:49.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.356+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd601008c0 con 0x7fbd60102d30 2024-09-05T07:01:49.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.357+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6006a850 con 0x7fbd60102d30 2024-09-05T07:01:49.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.358+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd50011750 con 0x7fbd60102d30 2024-09-05T07:01:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.359+0000 7fbd4f7fe640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 0x7fbd480a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.359+0000 7fbd5dd9b640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 0x7fbd480a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.359+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fbd500d3a10 con 0x7fbd60102d30 2024-09-05T07:01:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.360+0000 7fbd5dd9b640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 0x7fbd480a52b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbd540057c0 tx=0x7fbd540056d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.363+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd500a0190 con 0x7fbd60102d30 2024-09-05T07:01:49.398 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:49 smithi086 ceph-mon[30943]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 25 op/s 2024-09-05T07:01:49.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.731+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd6010a230 con 0x7fbd60102d30 2024-09-05T07:01:49.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.732+0000 7fbd4f7fe640 1 -- 172.21.15.86:0/2267727256 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7fbd5009fa90 con 0x7fbd60102d30 2024-09-05T07:01:49.733 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:01:49.734 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:01:49.734 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:01:49.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 msgr2=0x7fbd480a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:49.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 0x7fbd480a52b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbd540057c0 tx=0x7fbd540056d0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 msgr2=0x7fbd601984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009c70 tx=0x7fbd5000c6a0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 shutdown_connections 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbd480a2df0 0x7fbd480a52b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd601082a0 0x7fbd6019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd601036e0 0x7fbd60198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 --2- 172.21.15.86:0/2267727256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd60102d30 0x7fbd601984d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 >> 172.21.15.86:0/2267727256 conn(0x7fbd600fc7d0 msgr2=0x7fbd60105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 shutdown_connections 2024-09-05T07:01:49.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:49.735+0000 7fbd64fa2640 1 -- 172.21.15.86:0/2267727256 wait complete. 2024-09-05T07:01:49.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52880 6.59 MB/sec execute 307 sec latency 134.903 ms 2024-09-05T07:01:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:49 smithi096 ceph-mon[34468]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 25 op/s 2024-09-05T07:01:49.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:49 smithi115 ceph-mon[34791]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 25 op/s 2024-09-05T07:01:50.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:50 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2267727256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:50.637 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 139.35 --await-for 45.0' 2024-09-05T07:01:50.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 139.35 --await-for 45.0 2024-09-05T07:01:50.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53061 6.61 MB/sec execute 308 sec latency 93.761 ms 2024-09-05T07:01:50.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:50 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2267727256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:50.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.864+0000 7f430cf5c640 1 -- 172.21.15.86:0/4089693186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102570 msgr2=0x7f43081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.864+0000 7f430cf5c640 1 --2- 172.21.15.86:0/4089693186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102570 0x7f43081029b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009930 tx=0x7f42fc02a4d0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 -- 172.21.15.86:0/4089693186 shutdown_connections 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 --2- 172.21.15.86:0/4089693186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4308102ef0 0x7f430810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 --2- 172.21.15.86:0/4089693186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102570 0x7f43081029b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 --2- 172.21.15.86:0/4089693186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 -- 172.21.15.86:0/4089693186 >> 172.21.15.86:0/4089693186 conn(0x7f43080fc7d0 msgr2=0x7f43080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 -- 172.21.15.86:0/4089693186 shutdown_connections 2024-09-05T07:01:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.865+0000 7f430cf5c640 1 -- 172.21.15.86:0/4089693186 wait complete. 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 Processor -- start 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 -- start start 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4308102570 0x7f4308107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 0x7f43081084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4308110e80 con 0x7f4308102570 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4308110d00 con 0x7f4308100e20 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430cf5c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4308111000 con 0x7f4308102ef0 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34798/0 (socket says 172.21.15.86:34798) 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.866+0000 7f430659c640 1 -- 172.21.15.86:0/375172726 learned_addr learned my addr 172.21.15.86:0/375172726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f4306d9d640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 0x7f43081084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 -- 172.21.15.86:0/375172726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 msgr2=0x7f43081084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 0x7f43081084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 -- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4308102570 msgr2=0x7f4308107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4308102570 0x7f4308107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 -- 172.21.15.86:0/375172726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430819a400 con 0x7f4308100e20 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f4306d9d640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 0x7f43081084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430659c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f42f800ea20 tx=0x7f42f800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f42f77fe640 1 -- 172.21.15.86:0/375172726 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800cd80 con 0x7f4308100e20 2024-09-05T07:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f42f77fe640 1 -- 172.21.15.86:0/375172726 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f80108e0 con 0x7f4308100e20 2024-09-05T07:01:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f42f77fe640 1 -- 172.21.15.86:0/375172726 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8018840 con 0x7f4308100e20 2024-09-05T07:01:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430819a6f0 con 0x7f4308100e20 2024-09-05T07:01:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.867+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430819abe0 con 0x7f4308100e20 2024-09-05T07:01:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.870+0000 7f42f77fe640 1 -- 172.21.15.86:0/375172726 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f42f8010a90 con 0x7f4308100e20 2024-09-05T07:01:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.871+0000 7f42f77fe640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 0x7f42f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f42f77fe640 1 -- 172.21.15.86:0/375172726 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42f80cfce0 con 0x7f4308100e20 2024-09-05T07:01:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f4305d9b640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 0x7f42f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 Processor -- start 2024-09-05T07:01:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 -- start start 2024-09-05T07:01:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x55643068ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556430697370 0x556430699760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564305f22f0 con 0x55643068db40 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556430599cf0 con 0x55643068c5d0 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.872+0000 7f430f335740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5564305ee2b0 con 0x556430697370 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f4306d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556430697370 0x556430699760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38482/0 (socket says 172.21.15.86:38482) 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 -- 172.21.15.86:0/1796911530 learned_addr learned my addr 172.21.15.86:0/1796911530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 -- 172.21.15.86:0/1796911530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556430697370 msgr2=0x556430699760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556430697370 0x556430699760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f4305d9b640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 0x7f42f00a52b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009230 tx=0x7f42fc031400 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 -- 172.21.15.86:0/1796911530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 msgr2=0x55643068ca10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x55643068ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 -- 172.21.15.86:0/1796911530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643057e3a0 con 0x55643068db40 2024-09-05T07:01:50.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.873+0000 7f430659c640 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f42f80981c0 tx=0x7f42f809c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaf8cdfd5d8ca95 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.874+0000 7f42f4ff9640 1 -- 172.21.15.86:0/1796911530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80a49d0 con 0x55643068db40 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.874+0000 7f42f4ff9640 1 -- 172.21.15.86:0/1796911530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f80983d0 con 0x55643068db40 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.874+0000 7f42f4ff9640 1 -- 172.21.15.86:0/1796911530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8097d10 con 0x55643068db40 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- 172.21.15.86:0/1796911530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 msgr2=0x55643068c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f42f80981c0 tx=0x7f42f809c520 comp rx=0 tx=0).stop 2024-09-05T07:01:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- 172.21.15.86:0/1796911530 shutdown_connections 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556430697370 0x556430699760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x55643068ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 --2- 172.21.15.86:0/1796911530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x55643068c090 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- 172.21.15.86:0/1796911530 >> 172.21.15.86:0/1796911530 conn(0x556430655310 msgr2=0x556430688c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- 172.21.15.86:0/1796911530 shutdown_connections 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- 172.21.15.86:0/1796911530 wait complete. 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 -- start start 2024-09-05T07:01:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.875+0000 7f430f335740 1 Processor -- start 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x556430694070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556430697370 0x556430694800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55643069a0c0 con 0x556430697370 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564305ad060 con 0x55643068c5d0 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55643069a240 con 0x55643068db40 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430f335740 10 client.0 _ll_register_callbacks cb 0x5564303008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:34814/0 (socket says 172.21.15.86:34814) 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 -- 172.21.15.86:0/3536131973 learned_addr learned my addr 172.21.15.86:0/3536131973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f4305d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x556430694070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f4306d9d640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556430697370 0x556430694800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 -- 172.21.15.86:0/3536131973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 msgr2=0x556430694070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x556430694070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 -- 172.21.15.86:0/3536131973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556430697370 msgr2=0x556430694800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556430697370 0x556430694800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.876+0000 7f430659c640 1 -- 172.21.15.86:0/3536131973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564303633c0 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430659c640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f42f8000c00 tx=0x7f42f8098580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68993f7027c459e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80b8030 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f80983d0 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42f809a060 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f42f809b490 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 10 client.28754 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5564306a5bd0 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f80983d0 con 0x55643068c5d0 2024-09-05T07:01:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f42f809ac20 con 0x55643068c5d0 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 10 client.28754 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 10 client.28754 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 20 client.28754 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5564306a6f20 0x5564306a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 4 client.28754 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.877+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564305ff860 con 0x5564306a6f20 2024-09-05T07:01:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.878+0000 7f4305d9b640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5564306a6f20 0x5564306a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.878+0000 7f42edffb640 10 client.28754 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:01:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.878+0000 7f4305d9b640 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5564306a6f20 0x5564306a9310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9603d1bbd17039c4 server_cookie=cc783dbca25beb5 in_seq=0 out_seq=0 2024-09-05T07:01:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f42edffb640 1 -- 172.21.15.86:0/3536131973 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564305ff860 con 0x5564306a6f20 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f42edffb640 10 client.28754 handle_command_reply: tid=0 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 client.28754 shutdown 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 msgr2=0x556430696dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f42f8000c00 tx=0x7f42f8098580 comp rx=0 tx=0).stop 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 shutdown_connections 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x5564306a6f20 0x5564306a9310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f42edffb640 0 client.28754 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556430697370 0x556430694800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x556430694070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 --2- 172.21.15.86:0/3536131973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55643068c5d0 0x556430696dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.879+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 >> 172.21.15.86:0/3536131973 conn(0x556430655310 msgr2=0x55643068e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.880+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 shutdown_connections 2024-09-05T07:01:50.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.880+0000 7f430f335740 1 -- 172.21.15.86:0/3536131973 wait complete. 2024-09-05T07:01:50.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.880+0000 7f430f335740 20 client.28754 trim_cache size 0 max 16384 2024-09-05T07:01:50.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.902+0000 7f430f335740 1 Processor -- start 2024-09-05T07:01:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.902+0000 7f430f335740 1 -- start start 2024-09-05T07:01:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.902+0000 7f430f335740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 0x556430698760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430f335740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x556430698ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 0x556430698760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 0x556430698760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49938/0 (socket says 172.21.15.86:49938) 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430659c640 1 -- 172.21.15.86:0/3635986989 learned_addr learned my addr 172.21.15.86:0/3635986989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430f335740 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x556430692810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43081be090 con 0x55643068db40 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43081bfbd0 con 0x556430697370 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f4306d9d640 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x556430692810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.903+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43081be760 con 0x55643068c5d0 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4305d9b640 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x556430698ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 -- 172.21.15.86:0/3635986989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 msgr2=0x556430698760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 0x556430698760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 -- 172.21.15.86:0/3635986989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 msgr2=0x556430698ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x556430698ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 -- 172.21.15.86:0/3635986989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556430689960 con 0x556430697370 2024-09-05T07:01:50.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.904+0000 7f4306d9d640 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x556430692810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4300005fd0 tx=0x7f4300012930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3f6f1da033dde5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.905+0000 7f42effff640 1 -- 172.21.15.86:0/3635986989 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430001f020 con 0x556430697370 2024-09-05T07:01:50.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.905+0000 7f42effff640 1 -- 172.21.15.86:0/3635986989 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4300005020 con 0x556430697370 2024-09-05T07:01:50.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.905+0000 7f42effff640 1 -- 172.21.15.86:0/3635986989 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43000106f0 con 0x556430697370 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.906+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 msgr2=0x556430692810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.906+0000 7f430f335740 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x556430692810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4300005fd0 tx=0x7f4300012930 comp rx=0 tx=0).stop 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 shutdown_connections 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x556430692810 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068db40 0x556430698ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 --2- 172.21.15.86:0/3635986989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068c5d0 0x556430698760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 >> 172.21.15.86:0/3635986989 conn(0x556430655310 msgr2=0x5564306995f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 shutdown_connections 2024-09-05T07:01:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.907+0000 7f430f335740 1 -- 172.21.15.86:0/3635986989 wait complete. 2024-09-05T07:01:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.908+0000 7f430f335740 1 -- start start 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.908+0000 7f430f335740 1 Processor -- start 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.908+0000 7f430f335740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x5564306a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x5564306a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43081bf500 con 0x55643068c5d0 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38512/0 (socket says 172.21.15.86:38512) 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 -- 172.21.15.86:0/2262326281 learned_addr learned my addr 172.21.15.86:0/2262326281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f4305d9b640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x5564306a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43081bee30 con 0x556430697370 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4308101200 con 0x55643068db40 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430f335740 10 client.0 _ll_register_callbacks cb 0x7f42f8010de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f4306d9d640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x5564306a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 -- 172.21.15.86:0/2262326281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 msgr2=0x5564306a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x5564306a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 -- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 msgr2=0x5564306a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x5564306a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f430659c640 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556430710850 con 0x55643068c5d0 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f4306d9d640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x5564306a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.909+0000 7f4305d9b640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x5564306a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f430659c640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c6a0 tx=0x7f42f8010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7ed09ecd6d7e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f42edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80b8060 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42f807ab80 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4300005020 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f42f8098000 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f430f335740 10 client.20004 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556430711af0 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.910+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f80983d0 con 0x55643068c5d0 2024-09-05T07:01:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f42f80863e0 con 0x55643068c5d0 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 10 client.20004 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 10 client.20004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 20 client.20004 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x556430712010 0x556430714400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 4 client.20004 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:01:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4300005020 con 0x556430712010 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.911+0000 7f4305d9b640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x556430712010 0x556430714400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f42edffb640 10 client.20004 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f4305d9b640 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x556430712010 0x556430714400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63c62ed0413fa6b6 server_cookie=107e2e76d640a5f8 in_seq=0 out_seq=0 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f42edffb640 1 -- 172.21.15.86:0/2262326281 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4300005020 con 0x556430712010 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f42edffb640 10 client.20004 handle_command_reply: tid=0 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f430f335740 1 client.20004 shutdown 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 msgr2=0x5564306a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.912+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c6a0 tx=0x7f42f8010850 comp rx=0 tx=0).stop 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 shutdown_connections 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x556430712010 0x556430714400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f42edffb640 0 client.20004 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556430697370 0x5564306a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55643068db40 0x5564306a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 --2- 172.21.15.86:0/2262326281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55643068c5d0 0x5564306a8090 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 >> 172.21.15.86:0/2262326281 conn(0x556430655310 msgr2=0x55643068e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 shutdown_connections 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 1 -- 172.21.15.86:0/2262326281 wait complete. 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.913+0000 7f430f335740 20 client.20004 trim_cache size 0 max 16384 2024-09-05T07:01:50.917 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 msgr2=0x7f42f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 0x7f42f00a52b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009230 tx=0x7f42fc031400 comp rx=0 tx=0).stop 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 msgr2=0x7f4308107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f42f800ea20 tx=0x7f42f800eee0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 shutdown_connections 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42f00a2df0 0x7f42f00a52b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4308102ef0 0x7f43081084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4308102570 0x7f4308107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 --2- 172.21.15.86:0/375172726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4308100e20 0x7f4308107a50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.914+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 >> 172.21.15.86:0/375172726 conn(0x7f43080fc7d0 msgr2=0x7f430806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.915+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 shutdown_connections 2024-09-05T07:01:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:50.915+0000 7f430cf5c640 1 -- 172.21.15.86:0/375172726 wait complete. 2024-09-05T07:01:50.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:50 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2267727256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:01:51.724 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:51 smithi086 ceph-mon[30943]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 102 KiB/s wr, 15 op/s 2024-09-05T07:01:51.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:01:51.770 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53228 6.62 MB/sec execute 309 sec latency 72.426 ms 2024-09-05T07:01:51.772 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:51.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:51 smithi096 ceph-mon[34468]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 102 KiB/s wr, 15 op/s 2024-09-05T07:01:51.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:51 smithi115 ceph-mon[34791]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 102 KiB/s wr, 15 op/s 2024-09-05T07:01:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 -- 172.21.15.86:0/1535178129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 msgr2=0x7fcdac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 --2- 172.21.15.86:0/1535178129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1067e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009900 tx=0x7fcd9c02a410 comp rx=0 tx=0).stop 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 -- 172.21.15.86:0/1535178129 shutdown_connections 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 --2- 172.21.15.86:0/1535178129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 --2- 172.21.15.86:0/1535178129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1067e0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 --2- 172.21.15.86:0/1535178129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 0x7fcdac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 -- 172.21.15.86:0/1535178129 >> 172.21.15.86:0/1535178129 conn(0x7fcdac0fd8c0 msgr2=0x7fcdac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.342+0000 7fcdb4015640 1 -- 172.21.15.86:0/1535178129 shutdown_connections 2024-09-05T07:01:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.343+0000 7fcdb4015640 1 -- 172.21.15.86:0/1535178129 wait complete. 2024-09-05T07:01:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.343+0000 7fcdb4015640 1 Processor -- start 2024-09-05T07:01:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.343+0000 7fcdb4015640 1 -- start start 2024-09-05T07:01:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 0x7fcdac198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb15b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb15b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38528/0 (socket says 172.21.15.86:38528) 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac111e70 con 0x7fcdac102790 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac111cf0 con 0x7fcdac101de0 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb15b0640 1 -- 172.21.15.86:0/978680134 learned_addr learned my addr 172.21.15.86:0/978680134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac111ff0 con 0x7fcdac106f10 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.344+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 0x7fcdac198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 -- 172.21.15.86:0/978680134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 msgr2=0x7fcdac19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 -- 172.21.15.86:0/978680134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 msgr2=0x7fcdac198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 0x7fcdac198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac19d320 con 0x7fcdac102790 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c005ab0 tx=0x7fcd9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcda2ffd640 1 -- 172.21.15.86:0/978680134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c03f070 con 0x7fcdac102790 2024-09-05T07:01:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcda2ffd640 1 -- 172.21.15.86:0/978680134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9c031b60 con 0x7fcdac102790 2024-09-05T07:01:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcda2ffd640 1 -- 172.21.15.86:0/978680134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c035030 con 0x7fcdac102790 2024-09-05T07:01:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac19d5b0 con 0x7fcdac102790 2024-09-05T07:01:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.345+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac19d9b0 con 0x7fcdac102790 2024-09-05T07:01:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.348+0000 7fcda2ffd640 1 -- 172.21.15.86:0/978680134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd9c042070 con 0x7fcdac102790 2024-09-05T07:01:52.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.349+0000 7fcdb63ee740 1 Processor -- start 2024-09-05T07:01:52.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.349+0000 7fcda2ffd640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 0x7fcd980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.349+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 0x7fcd980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 -- start start 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48d0170 0x5637e48da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5637e4834160 con 0x5637e48d0170 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637e47db720 con 0x5637e48daa30 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5637e4830120 con 0x5637e48cf850 2024-09-05T07:01:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49964/0 (socket says 172.21.15.86:49964) 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 -- 172.21.15.86:0/3001941381 learned_addr learned my addr 172.21.15.86:0/3001941381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48d0170 0x5637e48da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 -- 172.21.15.86:0/3001941381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 msgr2=0x5637e48dce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 -- 172.21.15.86:0/3001941381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48d0170 msgr2=0x5637e48da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48d0170 0x5637e48da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb1db1640 1 -- 172.21.15.86:0/3001941381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e45a5ba0 con 0x5637e48cf850 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcda2ffd640 1 -- 172.21.15.86:0/978680134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fcd9c0efd90 con 0x7fcdac102790 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.350+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 0x7fcd980a5230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005fd0 tx=0x7fcda4009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.351+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009cf0 tx=0x7fcda8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b861f9d56f37c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.351+0000 7fcd97fff640 1 -- 172.21.15.86:0/3001941381 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004030 con 0x5637e48cf850 2024-09-05T07:01:52.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.351+0000 7fcd97fff640 1 -- 172.21.15.86:0/3001941381 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcda80041d0 con 0x5637e48cf850 2024-09-05T07:01:52.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.351+0000 7fcd97fff640 1 -- 172.21.15.86:0/3001941381 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8019770 con 0x5637e48cf850 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 -- 172.21.15.86:0/3001941381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 msgr2=0x5637e48cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009cf0 tx=0x7fcda8011810 comp rx=0 tx=0).stop 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 -- 172.21.15.86:0/3001941381 shutdown_connections 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48dce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48d0170 0x5637e48da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/3001941381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48cf850 0x5637e48cfc30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 -- 172.21.15.86:0/3001941381 >> 172.21.15.86:0/3001941381 conn(0x5637e48cbde0 msgr2=0x5637e48cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 -- 172.21.15.86:0/3001941381 shutdown_connections 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.352+0000 7fcdb63ee740 1 -- 172.21.15.86:0/3001941381 wait complete. 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 -- start start 2024-09-05T07:01:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 Processor -- start 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 0x5637e48d7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 0x5637e48d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5637e48dd600 con 0x5637e48daa30 2024-09-05T07:01:52.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637e47bc900 con 0x5637e48cf850 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5637e48dd780 con 0x5637e48d0170 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb15b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 0x5637e48d7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb15b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49966/0 (socket says 172.21.15.86:49966) 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb15b0640 1 -- 172.21.15.86:0/1911040028 learned_addr learned my addr 172.21.15.86:0/1911040028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb63ee740 10 client.0 _ll_register_callbacks cb 0x5637e47ba780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.353+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 0x5637e48d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb1db1640 1 -- 172.21.15.86:0/1911040028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 msgr2=0x5637e48d7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb1db1640 1 -- 172.21.15.86:0/1911040028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 msgr2=0x5637e48d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 0x5637e48d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb1db1640 1 -- 172.21.15.86:0/1911040028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e47fcc50 con 0x5637e48cf850 2024-09-05T07:01:52.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.354+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 0x5637e48d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.355+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 0x5637e48d7890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009cf0 tx=0x7fcda8003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987533c10b5f20b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.355+0000 7fcd957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8027070 con 0x5637e48cf850 2024-09-05T07:01:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fcda8005040 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda80041d0 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcda8031050 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcdb63ee740 10 client.28766 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5637e48f1a80 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.356+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcda80041d0 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fcda8004b40 con 0x5637e48cf850 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 10 client.28766 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 10 client.28766 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 20 client.28766 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e48f3310 0x5637e48f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 4 client.28766 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e4841890 con 0x5637e48f3310 2024-09-05T07:01:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.357+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e48f3310 0x5637e48f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.358+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e48f3310 0x5637e48f57b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386aecac06052ae3 server_cookie=78bef5e41f08c5a7 in_seq=0 out_seq=0 2024-09-05T07:01:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.358+0000 7fcd957fa640 10 client.28766 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcd957fa640 1 -- 172.21.15.86:0/1911040028 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637e4841890 con 0x5637e48f3310 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcd957fa640 10 client.28766 handle_command_reply: tid=0 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 client.28766 shutdown 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 msgr2=0x5637e48d7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 0x5637e48d7890 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009cf0 tx=0x7fcda8003b80 comp rx=0 tx=0).stop 2024-09-05T07:01:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 shutdown_connections 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e48f3310 0x5637e48f57b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48daa30 0x5637e48d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48d7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1911040028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48cf850 0x5637e48d7890 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 >> 172.21.15.86:0/1911040028 conn(0x5637e48cbde0 msgr2=0x5637e48d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcd957fa640 0 client.28766 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 shutdown_connections 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.359+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1911040028 wait complete. 2024-09-05T07:01:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.360+0000 7fcdb63ee740 20 client.28766 trim_cache size 0 max 16384 2024-09-05T07:01:52.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.385+0000 7fcdb63ee740 1 Processor -- start 2024-09-05T07:01:52.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.385+0000 7fcdb63ee740 1 -- start start 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.385+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.385+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900100e0 con 0x5637e48cf850 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90011c20 con 0x5637e48daa30 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb63ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900107b0 con 0x5637e48d0170 2024-09-05T07:01:52.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48d6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38560/0 (socket says 172.21.15.86:38560) 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb1db1640 1 -- 172.21.15.86:0/1611612371 learned_addr learned my addr 172.21.15.86:0/1611612371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 -- 172.21.15.86:0/1611612371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 msgr2=0x5637e48fb0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48fb0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 -- 172.21.15.86:0/1611612371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 msgr2=0x5637e48d6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48d6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.386+0000 7fcdb25b2640 1 -- 172.21.15.86:0/1611612371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e48cc8a0 con 0x5637e48daa30 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.387+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48fb600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5637e48cd160 tx=0x7fcda4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17e847d8c624ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.387+0000 7fcd977fe640 1 -- 172.21.15.86:0/1611612371 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda401e620 con 0x5637e48daa30 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.387+0000 7fcd977fe640 1 -- 172.21.15.86:0/1611612371 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcda401ec70 con 0x5637e48daa30 2024-09-05T07:01:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.387+0000 7fcd977fe640 1 -- 172.21.15.86:0/1611612371 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda401dc80 con 0x5637e48daa30 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1611612371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 msgr2=0x5637e48fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48fb600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5637e48cd160 tx=0x7fcda4004ab0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1611612371 shutdown_connections 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48daa30 0x5637e48fb600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48d0170 0x5637e48fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/1611612371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48d6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1611612371 >> 172.21.15.86:0/1611612371 conn(0x5637e48cbde0 msgr2=0x5637e48caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1611612371 shutdown_connections 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.388+0000 7fcdb63ee740 1 -- 172.21.15.86:0/1611612371 wait complete. 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb63ee740 1 -- start start 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb63ee740 1 Processor -- start 2024-09-05T07:01:52.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb63ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 0x5637e48d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.389+0000 7fcdb1db1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38564/0 (socket says 172.21.15.86:38564) 2024-09-05T07:01:52.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb1db1640 1 -- 172.21.15.86:0/2897542924 learned_addr learned my addr 172.21.15.86:0/2897542924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48daa30 0x5637e48d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90011550 con 0x5637e48cf850 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90010e80 con 0x5637e48d0170 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9000e0e0 con 0x5637e48daa30 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 0x5637e48d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb25b2640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48daa30 0x5637e48d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 10 client.0 _ll_register_callbacks cb 0x7fcda8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 -- 172.21.15.86:0/2897542924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48daa30 msgr2=0x5637e48d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48daa30 0x5637e48d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 -- 172.21.15.86:0/2897542924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 msgr2=0x5637e48eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e48d6840 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda401ec70 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcdb15b0640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 0x5637e48d43b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0b93b0 tx=0x7fcd9c03b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7776b359c618a8cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.390+0000 7fcd957fa640 10 client.28772 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcd957fa640 1 -- 172.21.15.86:0/2897542924 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c049040 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcd957fa640 1 -- 172.21.15.86:0/2897542924 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fcd9c0bb250 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcd957fa640 1 -- 172.21.15.86:0/2897542924 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcd9c042d90 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 10 client.28772 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5637e495dbb0 con 0x5637e48d0170 2024-09-05T07:01:52.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcd957fa640 1 -- 172.21.15.86:0/2897542924 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fcd9c0ed320 con 0x5637e48d0170 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 10 client.28772 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 10 client.28772 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 20 client.28772 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e495e290 0x5637e49606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 4 client.28772 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.35, "await_for": 45.0}] 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.35, "await_for": 45.0}) -- 0x5637e483bbe0 con 0x5637e495e290 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.391+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e495e290 0x5637e49606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.392+0000 7fcd957fa640 10 client.28772 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:52.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.392+0000 7fcdb1db1640 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e495e290 0x5637e49606b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ce8a8ff29e1479 server_cookie=c04c5920a7b0baa2 in_seq=0 out_seq=0 2024-09-05T07:01:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.616+0000 7fcd957fa640 1 -- 172.21.15.86:0/2897542924 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5637e483bbe0 con 0x5637e495e290 2024-09-05T07:01:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.616+0000 7fcd957fa640 10 client.28772 handle_command_reply: tid=0 2024-09-05T07:01:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.616+0000 7fcdb63ee740 1 client.28772 shutdown 2024-09-05T07:01:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 msgr2=0x5637e48d43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 0x5637e48d43b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0b93b0 tx=0x7fcd9c03b000 comp rx=0 tx=0).stop 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 shutdown_connections 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5637e495e290 0x5637e49606b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5637e48daa30 0x5637e48d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e48d0170 0x5637e48d43b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 --2- 172.21.15.86:0/2897542924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5637e48cf850 0x5637e48eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 >> 172.21.15.86:0/2897542924 conn(0x5637e48cbde0 msgr2=0x5637e48cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcd957fa640 0 client.28772 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 shutdown_connections 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 1 -- 172.21.15.86:0/2897542924 wait complete. 2024-09-05T07:01:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.617+0000 7fcdb63ee740 20 client.28772 trim_cache size 0 max 16384 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 78, 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "af59a075": { 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 78, 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:01:52.619 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 139.3, 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:52.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 msgr2=0x7fcd980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 0x7fcd980a5230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005fd0 tx=0x7fcda4009500 comp rx=0 tx=0).stop 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 msgr2=0x7fcdac1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c005ab0 tx=0x7fcd9c003c10 comp rx=0 tx=0).stop 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 shutdown_connections 2024-09-05T07:01:52.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcd980a2d70 0x7fcd980a5230 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdac106f10 0x7fcdac19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac102790 0x7fcdac1988b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 --2- 172.21.15.86:0/978680134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac101de0 0x7fcdac198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 >> 172.21.15.86:0/978680134 conn(0x7fcdac0fd8c0 msgr2=0x7fcdac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 shutdown_connections 2024-09-05T07:01:52.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:52.618+0000 7fcdb4015640 1 -- 172.21.15.86:0/978680134 wait complete. 2024-09-05T07:01:52.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.63 MB/sec execute 310 sec latency 398.670 ms 2024-09-05T07:01:53.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- 172.21.15.86:0/3980064443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4102570 msgr2=0x7f24c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 --2- 172.21.15.86:0/3980064443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4102570 0x7f24c41029b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002a480 comp rx=0 tx=0).stop 2024-09-05T07:01:53.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- 172.21.15.86:0/3980064443 shutdown_connections 2024-09-05T07:01:53.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 --2- 172.21.15.86:0/3980064443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 0x7f24c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 --2- 172.21.15.86:0/3980064443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4102570 0x7f24c41029b0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 --2- 172.21.15.86:0/3980064443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4100e20 0x7f24c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- 172.21.15.86:0/3980064443 >> 172.21.15.86:0/3980064443 conn(0x7f24c40fc7d0 msgr2=0x7f24c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- 172.21.15.86:0/3980064443 shutdown_connections 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- 172.21.15.86:0/3980064443 wait complete. 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 Processor -- start 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.200+0000 7f24cbb19640 1 -- start start 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4102570 0x7f24c41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 0x7f24c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110e50 con 0x7f24c4100e20 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110cd0 con 0x7f24c4102570 2024-09-05T07:01:53.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4110fd0 con 0x7f24c4102ef0 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56092/0 (socket says 172.21.15.86:56092) 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 -- 172.21.15.86:0/1070202593 learned_addr learned my addr 172.21.15.86:0/1070202593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c90b4640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4102570 0x7f24c41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 0x7f24c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 -- 172.21.15.86:0/1070202593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 msgr2=0x7f24c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 0x7f24c419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 -- 172.21.15.86:0/1070202593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4102570 msgr2=0x7f24c41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4102570 0x7f24c41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 -- 172.21.15.86:0/1070202593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c419d430 con 0x7f24c4100e20 2024-09-05T07:01:53.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24c98b5640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f24b800d950 tx=0x7f24b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24b6ffd640 1 -- 172.21.15.86:0/1070202593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8017030 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24b6ffd640 1 -- 172.21.15.86:0/1070202593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b8004020 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c419d720 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24b6ffd640 1 -- 172.21.15.86:0/1070202593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80047a0 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.201+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a5000 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24b6ffd640 1 -- 172.21.15.86:0/1070202593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f24b80041d0 con 0x7f24c4100e20 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 Processor -- start 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 -- start start 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c336a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c336aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572c32d02f0 con 0x5572c3375370 2024-09-05T07:01:53.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572c3277cf0 con 0x5572c336a5d0 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24cdef2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5572c32cc2b0 con 0x5572c336bb40 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24b6ffd640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.204+0000 7f24c90b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56094/0 (socket says 172.21.15.86:56094) 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 -- 172.21.15.86:0/73163958 learned_addr learned my addr 172.21.15.86:0/73163958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c98b5640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c336aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c336a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 -- 172.21.15.86:0/73163958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 msgr2=0x5572c336a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c336a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 -- 172.21.15.86:0/73163958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 msgr2=0x5572c336aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c336aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 -- 172.21.15.86:0/73163958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c325c3a0 con 0x5572c3375370 2024-09-05T07:01:53.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24b6ffd640 1 -- 172.21.15.86:0/1070202593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f24b80cf380 con 0x7f24c4100e20 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c98b5640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c336aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0070b0 tx=0x7f24bc0093d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24c90b4640 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c0031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a214c9347cfaba1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24affff640 1 -- 172.21.15.86:0/73163958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0040070 con 0x5572c3375370 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24affff640 1 -- 172.21.15.86:0/73163958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c0031630 con 0x5572c3375370 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.205+0000 7f24affff640 1 -- 172.21.15.86:0/73163958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0035040 con 0x5572c3375370 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 -- 172.21.15.86:0/73163958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 msgr2=0x5572c3377760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c0031c60 comp rx=0 tx=0).stop 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 -- 172.21.15.86:0/73163958 shutdown_connections 2024-09-05T07:01:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c3377760 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c336aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 --2- 172.21.15.86:0/73163958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c336a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 -- 172.21.15.86:0/73163958 >> 172.21.15.86:0/73163958 conn(0x5572c3333310 msgr2=0x5572c3366c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 -- 172.21.15.86:0/73163958 shutdown_connections 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.206+0000 7f24cdef2740 1 -- 172.21.15.86:0/73163958 wait complete. 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 -- start start 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 Processor -- start 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336a5d0 0x5572c3374c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336bb40 0x5572c3371eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572c3378010 con 0x5572c3375370 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572c328b060 con 0x5572c336bb40 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5572c3378190 con 0x5572c336a5d0 2024-09-05T07:01:53.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24cdef2740 10 client.0 _ll_register_callbacks cb 0x5572c2fde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24ca0b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24ca0b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56108/0 (socket says 172.21.15.86:56108) 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24ca0b6640 1 -- 172.21.15.86:0/3176513507 learned_addr learned my addr 172.21.15.86:0/3176513507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24c98b5640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336a5d0 0x5572c3374c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.207+0000 7f24c90b4640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336bb40 0x5572c3371eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 -- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336a5d0 msgr2=0x5572c3374c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336a5d0 0x5572c3374c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 -- 172.21.15.86:0/3176513507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336bb40 msgr2=0x5572c3371eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336bb40 0x5572c3371eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 -- 172.21.15.86:0/3176513507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c30413c0 con 0x5572c3375370 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0095f0 tx=0x7f24bc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a033923db2c06fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc029060 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f24bc01b9d0 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c0031630 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f24bc030050 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24cdef2740 10 client.20031 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5572c3383960 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c0031630 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.208+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f24bc024b80 con 0x5572c3375370 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 10 client.20031 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 10 client.20031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 20 client.20031 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c3384e20 0x5572c3387270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 4 client.20031 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:53.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c32dd860 con 0x5572c3384e20 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24c98b5640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c3384e20 0x5572c3387270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24c98b5640 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c3384e20 0x5572c3387270 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef22fd1987a4fd4d server_cookie=ebe7846029ad27dd in_seq=0 out_seq=0 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.209+0000 7f24ad7fa640 10 client.20031 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24ad7fa640 1 -- 172.21.15.86:0/3176513507 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572c32dd860 con 0x5572c3384e20 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24ad7fa640 10 client.20031 handle_command_reply: tid=0 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 client.20031 shutdown 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 msgr2=0x5572c33723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0095f0 tx=0x7f24bc004b50 comp rx=0 tx=0).stop 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 shutdown_connections 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c3384e20 0x5572c3387270 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33723f0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336bb40 0x5572c3371eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3176513507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336a5d0 0x5572c3374c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24ad7fa640 0 client.20031 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:01:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 >> 172.21.15.86:0/3176513507 conn(0x5572c3333310 msgr2=0x5572c336c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 shutdown_connections 2024-09-05T07:01:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 1 -- 172.21.15.86:0/3176513507 wait complete. 2024-09-05T07:01:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.210+0000 7f24cdef2740 20 client.20031 trim_cache size 0 max 16384 2024-09-05T07:01:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 Processor -- start 2024-09-05T07:01:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 -- start start 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c3366700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 0x5572c337dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41af1a0 con 0x5572c336bb40 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b0ce0 con 0x5572c336a5d0 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.223+0000 7f24cdef2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41af870 con 0x5572c3375370 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56122/0 (socket says 172.21.15.86:56122) 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 -- 172.21.15.86:0/3536463127 learned_addr learned my addr 172.21.15.86:0/3536463127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:53.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c98b5640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c3366700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 0x5572c337dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 -- 172.21.15.86:0/3536463127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 msgr2=0x5572c337dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 0x5572c337dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 -- 172.21.15.86:0/3536463127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 msgr2=0x5572c3366700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c3366700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 -- 172.21.15.86:0/3536463127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c3370e60 con 0x5572c336bb40 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 0x5572c337dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24c90b4640 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d54975c63e9cea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24af7fe640 1 -- 172.21.15.86:0/3536463127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0040050 con 0x5572c336bb40 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24af7fe640 1 -- 172.21.15.86:0/3536463127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c00055e0 con 0x5572c336bb40 2024-09-05T07:01:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.224+0000 7f24af7fe640 1 -- 172.21.15.86:0/3536463127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0035040 con 0x5572c336bb40 2024-09-05T07:01:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- 172.21.15.86:0/3536463127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 msgr2=0x5572c337d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002a480 comp rx=0 tx=0).stop 2024-09-05T07:01:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- 172.21.15.86:0/3536463127 shutdown_connections 2024-09-05T07:01:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c3375370 0x5572c337dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c336bb40 0x5572c337d800 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 --2- 172.21.15.86:0/3536463127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c3366700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- 172.21.15.86:0/3536463127 >> 172.21.15.86:0/3536463127 conn(0x5572c3333310 msgr2=0x5572c3373f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- 172.21.15.86:0/3536463127 shutdown_connections 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- 172.21.15.86:0/3536463127 wait complete. 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 -- start start 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.225+0000 7f24cdef2740 1 Processor -- start 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c33ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c33ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b0610 con 0x5572c3375370 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41aff40 con 0x5572c336a5d0 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41ad1a0 con 0x5572c336bb40 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24cdef2740 10 client.0 _ll_register_callbacks cb 0x7f24bc009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56134/0 (socket says 172.21.15.86:56134) 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 -- 172.21.15.86:0/2568457386 learned_addr learned my addr 172.21.15.86:0/2568457386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24c98b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c33ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24c90b4640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c33ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 -- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 msgr2=0x5572c33ec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c33ec370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 -- 172.21.15.86:0/2568457386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 msgr2=0x5572c33ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c33ebe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 -- 172.21.15.86:0/2568457386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c33ed3c0 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24c98b5640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c33ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ca0b6640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f24bc024f50 tx=0x7f24bc01b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd72e6612f1d9e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.226+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc039060 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f24bc019770 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c00055e0 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f24bc040050 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 10 client.20037 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5572c33f4730 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c00055e0 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f24bc0243e0 con 0x5572c3375370 2024-09-05T07:01:53.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 10 client.20037 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 10 client.20037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 20 client.20037 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c33f4df0 0x5572c33f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 4 client.20037 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572c32d7d70 con 0x5572c33f4df0 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.227+0000 7f24c98b5640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c33f4df0 0x5572c33f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24c98b5640 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c33f4df0 0x5572c33f71e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce704e612495af48 server_cookie=eb0212ddfe44d15e in_seq=0 out_seq=0 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24ad7fa640 10 client.20037 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24ad7fa640 1 -- 172.21.15.86:0/2568457386 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572c32d7d70 con 0x5572c33f4df0 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24ad7fa640 10 client.20037 handle_command_reply: tid=0 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 client.20037 shutdown 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 msgr2=0x5572c33ecbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f24bc024f50 tx=0x7f24bc01b350 comp rx=0 tx=0).stop 2024-09-05T07:01:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 shutdown_connections 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5572c33f4df0 0x5572c33f71e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24ad7fa640 0 client.20037 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c3375370 0x5572c33ecbb0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5572c336bb40 0x5572c33ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 --2- 172.21.15.86:0/2568457386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572c336a5d0 0x5572c33ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 >> 172.21.15.86:0/2568457386 conn(0x5572c3333310 msgr2=0x5572c3365d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 shutdown_connections 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 1 -- 172.21.15.86:0/2568457386 wait complete. 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.228+0000 7f24cdef2740 20 client.20037 trim_cache size 0 max 16384 2024-09-05T07:01:53.235 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:01:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 msgr2=0x7f24b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0070b0 tx=0x7f24bc0093d0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 msgr2=0x7f24c4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:53.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f24b800d950 tx=0x7f24b800de10 comp rx=0 tx=0).stop 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 shutdown_connections 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24b00a2cf0 0x7f24b00a51b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24c4102ef0 0x7f24c419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24c4102570 0x7f24c41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 --2- 172.21.15.86:0/1070202593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c4100e20 0x7f24c4198480 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 >> 172.21.15.86:0/1070202593 conn(0x7f24c40fc7d0 msgr2=0x7f24c406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 shutdown_connections 2024-09-05T07:01:53.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:53.229+0000 7f24cbb19640 1 -- 172.21.15.86:0/1070202593 wait complete. 2024-09-05T07:01:53.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.60 MB/sec execute 311 sec latency 1398.770 ms 2024-09-05T07:01:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:53 smithi096 ceph-mon[34468]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-05T07:01:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:53 smithi115 ceph-mon[34791]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-05T07:01:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:53 smithi086 ceph-mon[30943]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-05T07:01:54.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:01:54.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.58 MB/sec execute 312 sec latency 2398.832 ms 2024-09-05T07:01:55.333 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'af59a075', quiesce took 7.8 seconds. Will release after: 11.55 2024-09-05T07:01:55.397 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:55.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:55 smithi086 ceph-mon[30943]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:55.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:55 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:01:55.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.56 MB/sec execute 313 sec latency 3398.911 ms 2024-09-05T07:01:55.798 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:55 smithi115 ceph-mon[34791]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:55.798 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:55 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:01:55.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:55 smithi096 ceph-mon[34468]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:55.809 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:55 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.889+0000 7f4301187640 1 -- 172.21.15.86:0/1851897758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106f10 msgr2=0x7f42fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.889+0000 7f4301187640 1 --2- 172.21.15.86:0/1851897758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106f10 0x7f42fc10ea90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009ca0 tx=0x7f42f402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 -- 172.21.15.86:0/1851897758 shutdown_connections 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 --2- 172.21.15.86:0/1851897758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc106f10 0x7f42fc10ea90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 --2- 172.21.15.86:0/1851897758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc102790 0x7f42fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 --2- 172.21.15.86:0/1851897758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc101de0 0x7f42fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 -- 172.21.15.86:0/1851897758 >> 172.21.15.86:0/1851897758 conn(0x7f42fc0fd8c0 msgr2=0x7f42fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 -- 172.21.15.86:0/1851897758 shutdown_connections 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 -- 172.21.15.86:0/1851897758 wait complete. 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.890+0000 7f4301187640 1 Processor -- start 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 -- start start 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc101de0 0x7f42fc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc106f10 0x7f42fc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc111e20 con 0x7f42fc102790 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc111ca0 con 0x7f42fc106f10 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc111fa0 con 0x7f42fc101de0 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc101de0 0x7f42fc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56148/0 (socket says 172.21.15.86:56148) 2024-09-05T07:01:55.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 -- 172.21.15.86:0/3376311899 learned_addr learned my addr 172.21.15.86:0/3376311899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fb59e640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc106f10 0x7f42fc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 -- 172.21.15.86:0/3376311899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc101de0 msgr2=0x7f42fc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc101de0 0x7f42fc198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 -- 172.21.15.86:0/3376311899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc106f10 msgr2=0x7f42fc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc106f10 0x7f42fc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 -- 172.21.15.86:0/3376311899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc19d2c0 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d950 tx=0x7f42e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42f3fff640 1 -- 172.21.15.86:0/3376311899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4017030 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc19d5b0 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42f3fff640 1 -- 172.21.15.86:0/3376311899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42e4004020 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.891+0000 7f42f3fff640 1 -- 172.21.15.86:0/3376311899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40047a0 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.892+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc1a4e90 con 0x7f42fc102790 2024-09-05T07:01:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.894+0000 7f42f3fff640 1 -- 172.21.15.86:0/3376311899 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f42e40041d0 con 0x7f42fc102790 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.894+0000 7f4303560740 1 Processor -- start 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.894+0000 7f4303560740 1 -- start start 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 0x55c564d23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c564c892f0 con 0x55c564d2e370 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c564c30cf0 con 0x55c564d24b40 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f4303560740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c564c852b0 con 0x55c564d235d0 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56152/0 (socket says 172.21.15.86:56152) 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 -- 172.21.15.86:0/3039017083 learned_addr learned my addr 172.21.15.86:0/3039017083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42f3fff640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 0x7f42e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fb59e640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 0x55c564d23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fad9d640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fb59e640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 0x7f42e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42f3fff640 1 -- 172.21.15.86:0/3376311899 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42e40cf260 con 0x7f42fc102790 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 -- 172.21.15.86:0/3039017083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 msgr2=0x55c564d23a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d23a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 -- 172.21.15.86:0/3039017083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 msgr2=0x55c564d23090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 0x55c564d23090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fa59c640 1 -- 172.21.15.86:0/3039017083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c564c153a0 con 0x55c564d2e370 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fb59e640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 0x55c564d23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.895+0000 7f42fad9d640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d23a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f42fa59c640 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f42e4069b60 tx=0x7f42e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3e0226d108fe08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f42fb59e640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 0x7f42e80a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004af0 tx=0x7f42f402a8c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f42f17fa640 1 -- 172.21.15.86:0/3039017083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e409ac40 con 0x55c564d2e370 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f42f17fa640 1 -- 172.21.15.86:0/3039017083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42e4095e50 con 0x55c564d2e370 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f42f17fa640 1 -- 172.21.15.86:0/3039017083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4097030 con 0x55c564d2e370 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f4303560740 1 -- 172.21.15.86:0/3039017083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 msgr2=0x55c564d30760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.896+0000 7f4303560740 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f42e4069b60 tx=0x7f42e40046c0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- 172.21.15.86:0/3039017083 shutdown_connections 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d2e370 0x55c564d30760 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d23a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- 172.21.15.86:0/3039017083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d24b40 0x55c564d23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- 172.21.15.86:0/3039017083 >> 172.21.15.86:0/3039017083 conn(0x55c564cec310 msgr2=0x55c564d1fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- 172.21.15.86:0/3039017083 shutdown_connections 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- 172.21.15.86:0/3039017083 wait complete. 2024-09-05T07:01:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- start start 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 Processor -- start 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d235d0 0x55c564d2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 0x55c564d2b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c564d30fb0 con 0x55c564d24b40 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c564c44060 con 0x55c564d235d0 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c564d31130 con 0x55c564d2e370 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f4303560740 10 client.0 _ll_register_callbacks cb 0x55c5649978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56162/0 (socket says 172.21.15.86:56162) 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f42fa59c640 1 -- 172.21.15.86:0/343591504 learned_addr learned my addr 172.21.15.86:0/343591504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.897+0000 7f42fb59e640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 0x55c564d2b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fad9d640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d235d0 0x55c564d2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 -- 172.21.15.86:0/343591504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 msgr2=0x55c564d2b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 0x55c564d2b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 -- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d235d0 msgr2=0x55c564d2dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d235d0 0x55c564d2dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 -- 172.21.15.86:0/343591504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5649fa3c0 con 0x55c564d24b40 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fb59e640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 0x55c564d2b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42fa59c640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f42e4097cb0 tx=0x7f42e40a3fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30eb53128fc2e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e409ac40 con 0x55c564d24b40 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42e40cc790 con 0x55c564d24b40 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e4095e50 con 0x55c564d24b40 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f42e4097580 con 0x55c564d24b40 2024-09-05T07:01:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f4303560740 10 client.20049 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c564d3ca90 con 0x55c564d24b40 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.898+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42e4095e50 con 0x55c564d24b40 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f42e40bb110 con 0x55c564d24b40 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 10 client.20049 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 10 client.20049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 20 client.20049 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564d3dde0 0x55c564d401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 4 client.20049 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c564c96860 con 0x55c564d3dde0 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f42fad9d640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564d3dde0 0x55c564d401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f42e27fc640 10 client.20049 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.899+0000 7f42fad9d640 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564d3dde0 0x55c564d401d0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bba0d3758b0d62 server_cookie=27b87679dff317a9 in_seq=0 out_seq=0 2024-09-05T07:01:55.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f42e27fc640 1 -- 172.21.15.86:0/343591504 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c564c96860 con 0x55c564d3dde0 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f42e27fc640 10 client.20049 handle_command_reply: tid=0 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 client.20049 shutdown 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 msgr2=0x55c564d2b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f42e4097cb0 tx=0x7f42e40a3fc0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 shutdown_connections 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564d3dde0 0x55c564d401d0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d2e370 0x55c564d2b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2b050 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f42e27fc640 0 client.20049 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 --2- 172.21.15.86:0/343591504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d235d0 0x55c564d2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 >> 172.21.15.86:0/343591504 conn(0x55c564cec310 msgr2=0x55c564d25990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 shutdown_connections 2024-09-05T07:01:55.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 1 -- 172.21.15.86:0/343591504 wait complete. 2024-09-05T07:01:55.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.900+0000 7f4303560740 20 client.20049 trim_cache size 0 max 16384 2024-09-05T07:01:55.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.913+0000 7f4303560740 1 Processor -- start 2024-09-05T07:01:55.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.913+0000 7f4303560740 1 -- start start 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1af280 con 0x55c564d24b40 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b0dc0 con 0x55c564d2e370 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f4303560740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1af950 con 0x55c564d235d0 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:43902/0 (socket says 172.21.15.86:43902) 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 -- 172.21.15.86:0/1228758799 learned_addr learned my addr 172.21.15.86:0/1228758799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 -- 172.21.15.86:0/1228758799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 msgr2=0x55c564d431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 -- 172.21.15.86:0/1228758799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 msgr2=0x55c564d2a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 -- 172.21.15.86:0/1228758799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c564d20960 con 0x55c564d235d0 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fb59e640 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55c564d31170 tx=0x7f42ec00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521e4f94bf7d9ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.914+0000 7f42f0ff9640 1 -- 172.21.15.86:0/1228758799 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004220 con 0x55c564d235d0 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f42f0ff9640 1 -- 172.21.15.86:0/1228758799 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42ec0043c0 con 0x55c564d235d0 2024-09-05T07:01:55.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f42f0ff9640 1 -- 172.21.15.86:0/1228758799 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0035f0 con 0x55c564d235d0 2024-09-05T07:01:55.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 -- 172.21.15.86:0/1228758799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 msgr2=0x55c564d1fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55c564d31170 tx=0x7f42ec00b410 comp rx=0 tx=0).stop 2024-09-05T07:01:55.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 -- 172.21.15.86:0/1228758799 shutdown_connections 2024-09-05T07:01:55.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d2a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 --2- 172.21.15.86:0/1228758799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d1fd70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 -- 172.21.15.86:0/1228758799 >> 172.21.15.86:0/1228758799 conn(0x55c564cec310 msgr2=0x55c564d29940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 -- 172.21.15.86:0/1228758799 shutdown_connections 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.915+0000 7f4303560740 1 -- 172.21.15.86:0/1228758799 wait complete. 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 -- start start 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 Processor -- start 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d40b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b06f0 con 0x55c564d24b40 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b0020 con 0x55c564d2e370 2024-09-05T07:01:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1ad280 con 0x55c564d235d0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f4303560740 10 client.0 _ll_register_callbacks cb 0x7f42ec00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56176/0 (socket says 172.21.15.86:56176) 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f42fa59c640 1 -- 172.21.15.86:0/1518628346 learned_addr learned my addr 172.21.15.86:0/1518628346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.916+0000 7f42fb59e640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 -- 172.21.15.86:0/1518628346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 msgr2=0x55c564d435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 -- 172.21.15.86:0/1518628346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 msgr2=0x55c564d43b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 -- 172.21.15.86:0/1518628346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c564d26120 con 0x55c564d2e370 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fa59c640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42fb59e640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d40b00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42f4031040 tx=0x7f42f4004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5671ddedc17c5011 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4059060 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ec0043c0 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f42f40305c0 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f42f4060050 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f4303560740 10 client.28808 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.917+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c564da8230 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42ec0043c0 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f42f40547f0 con 0x55c564d2e370 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 10 client.28808 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 10 client.28808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 20 client.28808 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564da8900 0x55c564daacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 4 client.28808 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:01:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c564c90d70 con 0x55c564da8900 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564da8900 0x55c564daacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42fad9d640 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564da8900 0x55c564daacf0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf9cb23bf2a9f54 server_cookie=da8b9408191c949f in_seq=0 out_seq=0 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42e27fc640 10 client.28808 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42e27fc640 1 -- 172.21.15.86:0/1518628346 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c564c90d70 con 0x55c564da8900 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f42e27fc640 10 client.28808 handle_command_reply: tid=0 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.918+0000 7f4303560740 1 client.28808 shutdown 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 msgr2=0x55c564d40b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d40b00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42f4031040 tx=0x7f42f4004a60 comp rx=0 tx=0).stop 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 shutdown_connections 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55c564da8900 0x55c564daacf0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c564d2e370 0x55c564d40b00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f42e27fc640 0 client.28808 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c564d24b40 0x55c564d43b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 --2- 172.21.15.86:0/1518628346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c564d235d0 0x55c564d435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 >> 172.21.15.86:0/1518628346 conn(0x55c564cec310 msgr2=0x55c564d2a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 shutdown_connections 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 1 -- 172.21.15.86:0/1518628346 wait complete. 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4303560740 20 client.28808 trim_cache size 0 max 16384 2024-09-05T07:01:55.925 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 msgr2=0x7f42e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 0x7f42e80a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f42f4004af0 tx=0x7f42f402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 msgr2=0x7f42fc198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.919+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d950 tx=0x7f42e400de10 comp rx=0 tx=0).stop 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 shutdown_connections 2024-09-05T07:01:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42e80a2cf0 0x7f42e80a51b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42fc106f10 0x7f42fc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42fc102790 0x7f42fc198850 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 --2- 172.21.15.86:0/3376311899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc101de0 0x7f42fc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 >> 172.21.15.86:0/3376311899 conn(0x7f42fc0fd8c0 msgr2=0x7f42fc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 shutdown_connections 2024-09-05T07:01:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:55.920+0000 7f4301187640 1 -- 172.21.15.86:0/3376311899 wait complete. 2024-09-05T07:01:56.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.54 MB/sec execute 314 sec latency 4398.997 ms 2024-09-05T07:01:57.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.52 MB/sec execute 315 sec latency 5399.044 ms 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:57 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.927 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:01:57.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:57 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.956 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:57 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:58 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 -- 172.21.15.86:0/2752537374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10c170 msgr2=0x7f013c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 --2- 172.21.15.86:0/2752537374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10c170 0x7f013c10e610 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0134009ca0 tx=0x7f013402a950 comp rx=0 tx=0).stop 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 -- 172.21.15.86:0/2752537374 shutdown_connections 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 --2- 172.21.15.86:0/2752537374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10c170 0x7f013c10e610 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 --2- 172.21.15.86:0/2752537374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c103660 0x7f013c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 --2- 172.21.15.86:0/2752537374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c102d40 0x7f013c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 -- 172.21.15.86:0/2752537374 >> 172.21.15.86:0/2752537374 conn(0x7f013c0fc7a0 msgr2=0x7f013c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 -- 172.21.15.86:0/2752537374 shutdown_connections 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.713+0000 7f0143592640 1 -- 172.21.15.86:0/2752537374 wait complete. 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 Processor -- start 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 -- start start 2024-09-05T07:01:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 0x7f013c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c10c170 0x7f013c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c111e00 con 0x7f013c102d40 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f013c111c80 con 0x7f013c103660 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0143592640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f013c111f80 con 0x7f013c10c170 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56192/0 (socket says 172.21.15.86:56192) 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f014132e640 1 -- 172.21.15.86:0/3772411869 learned_addr learned my addr 172.21.15.86:0/3772411869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0140b2d640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 0x7f013c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.714+0000 7f0141b2f640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c10c170 0x7f013c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 -- 172.21.15.86:0/3772411869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c10c170 msgr2=0x7f013c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c10c170 0x7f013c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 -- 172.21.15.86:0/3772411869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 msgr2=0x7f013c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 0x7f013c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 -- 172.21.15.86:0/3772411869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1a67e0 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f0140b2d640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 0x7f013c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f014132e640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f013800d950 tx=0x7f013800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f01327fc640 1 -- 172.21.15.86:0/3772411869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138017030 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1a6ad0 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f01327fc640 1 -- 172.21.15.86:0/3772411869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0138004020 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f01327fc640 1 -- 172.21.15.86:0/3772411869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380047a0 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.715+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a7060 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.717+0000 7f01327fc640 1 -- 172.21.15.86:0/3772411869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f01380041d0 con 0x7f013c102d40 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 Processor -- start 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 -- start start 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f01327fc640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 0x7f01280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c2964f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2972130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559c28ca280 con 0x5559c296fd40 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559c2871b10 con 0x5559c2965480 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014596b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559c28c6240 con 0x5559c2964b60 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f01327fc640 1 -- 172.21.15.86:0/3772411869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f01380cf320 con 0x7f013c102d40 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f0140b2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f0140b2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47480/0 (socket says 172.21.15.86:47480) 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f0140b2d640 1 -- 172.21.15.86:0/1358472232 learned_addr learned my addr 172.21.15.86:0/1358472232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f014132e640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c2964f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.718+0000 7f0141b2f640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2972130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0141b2f640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 0x7f01280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 -- 172.21.15.86:0/1358472232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 msgr2=0x5559c2964f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c2964f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 -- 172.21.15.86:0/1358472232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 msgr2=0x5559c2972130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2972130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 -- 172.21.15.86:0/1358472232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c287e1f0 con 0x5559c2965480 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0141b2f640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2972130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0140b2d640 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f012c008b30 tx=0x7f012c00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31351a7065e1ebcb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f0141b2f640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 0x7f01280a51b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5559c2962470 tx=0x7f0134005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f01277fe640 1 -- 172.21.15.86:0/1358472232 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c014070 con 0x5559c2965480 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f01277fe640 1 -- 172.21.15.86:0/1358472232 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012c0054d0 con 0x5559c2965480 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.719+0000 7f01277fe640 1 -- 172.21.15.86:0/1358472232 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c010690 con 0x5559c2965480 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- 172.21.15.86:0/1358472232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 msgr2=0x5559c296f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f012c008b30 tx=0x7f012c00e350 comp rx=0 tx=0).stop 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- 172.21.15.86:0/1358472232 shutdown_connections 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2972130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296f800 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 --2- 172.21.15.86:0/1358472232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c2964f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- 172.21.15.86:0/1358472232 >> 172.21.15.86:0/1358472232 conn(0x5559c29610f0 msgr2=0x5559c29614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- 172.21.15.86:0/1358472232 shutdown_connections 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- 172.21.15.86:0/1358472232 wait complete. 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 -- start start 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.720+0000 7f014596b740 1 Processor -- start 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2965480 0x5559c296d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c296fd40 0x5559c296d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559c2972910 con 0x5559c2964b60 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559c2884f10 con 0x5559c296fd40 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559c2972a90 con 0x5559c2965480 2024-09-05T07:01:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014596b740 10 client.0 _ll_register_callbacks cb 0x5559c284da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56224/0 (socket says 172.21.15.86:56224) 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 -- 172.21.15.86:0/615361365 learned_addr learned my addr 172.21.15.86:0/615361365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f0140b2d640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2965480 0x5559c296d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f0141b2f640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c296fd40 0x5559c296d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 -- 172.21.15.86:0/615361365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2965480 msgr2=0x5559c296d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2965480 0x5559c296d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 -- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c296fd40 msgr2=0x5559c296d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c296fd40 0x5559c296d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 -- 172.21.15.86:0/615361365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c263b6a0 con 0x5559c2964b60 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f014132e640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0138004770 tx=0x7f013809ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2d671c6ae52c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.721+0000 7f0124ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013809b400 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f0138097a70 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c0054d0 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f013809f7b0 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 10 client.20067 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5559c297e4d0 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012c0054d0 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0138097150 con 0x5559c2964b60 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 10 client.20067 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 10 client.20067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 20 client.20067 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c297f8e0 0x5559c2981cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 4 client.20067 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:01:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.722+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c28d77f0 con 0x5559c297f8e0 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.723+0000 7f0140b2d640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c297f8e0 0x5559c2981cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.723+0000 7f0140b2d640 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c297f8e0 0x5559c2981cf0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d70900bdc4089ca server_cookie=52925b25c2f42648 in_seq=0 out_seq=0 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.723+0000 7f0124ff9640 10 client.20067 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f0124ff9640 1 -- 172.21.15.86:0/615361365 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559c28d77f0 con 0x5559c297f8e0 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f0124ff9640 10 client.20067 handle_command_reply: tid=0 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 client.20067 shutdown 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 msgr2=0x5559c296cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0138004770 tx=0x7f013809ae80 comp rx=0 tx=0).stop 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 shutdown_connections 2024-09-05T07:01:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c297f8e0 0x5559c2981cf0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c296fd40 0x5559c296d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2965480 0x5559c296d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 --2- 172.21.15.86:0/615361365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c296cbb0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 >> 172.21.15.86:0/615361365 conn(0x5559c29610f0 msgr2=0x5559c2960270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f0124ff9640 0 client.20067 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 shutdown_connections 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 1 -- 172.21.15.86:0/615361365 wait complete. 2024-09-05T07:01:58.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.724+0000 7f014596b740 20 client.20067 trim_cache size 0 max 16384 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.741+0000 7f014596b740 1 Processor -- start 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.741+0000 7f014596b740 1 -- start start 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c29675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b11d0 con 0x5559c296fd40 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b2d10 con 0x5559c2965480 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014596b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b18a0 con 0x5559c2964b60 2024-09-05T07:01:58.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56238/0 (socket says 172.21.15.86:56238) 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 -- 172.21.15.86:0/4104157411 learned_addr learned my addr 172.21.15.86:0/4104157411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f014132e640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c29675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0140b2d640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 -- 172.21.15.86:0/4104157411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 msgr2=0x5559c29675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c29675c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 -- 172.21.15.86:0/4104157411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 msgr2=0x5559c296ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 -- 172.21.15.86:0/4104157411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c29652b0 con 0x5559c296fd40 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0140b2d640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.742+0000 7f0141b2f640 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0134035000 tx=0x7f013403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92147fceb777d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f0126ffd640 1 -- 172.21.15.86:0/4104157411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134034040 con 0x5559c296fd40 2024-09-05T07:01:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f0126ffd640 1 -- 172.21.15.86:0/4104157411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f013404f070 con 0x5559c296fd40 2024-09-05T07:01:58.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f0126ffd640 1 -- 172.21.15.86:0/4104157411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134054890 con 0x5559c296fd40 2024-09-05T07:01:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 -- 172.21.15.86:0/4104157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 msgr2=0x5559c2987610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0134035000 tx=0x7f013403c000 comp rx=0 tx=0).stop 2024-09-05T07:01:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 -- 172.21.15.86:0/4104157411 shutdown_connections 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c296fd40 0x5559c2987610 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c296ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 --2- 172.21.15.86:0/4104157411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c2964b60 0x5559c29675c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 -- 172.21.15.86:0/4104157411 >> 172.21.15.86:0/4104157411 conn(0x5559c29610f0 msgr2=0x5559c296e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 -- 172.21.15.86:0/4104157411 shutdown_connections 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.743+0000 7f014596b740 1 -- 172.21.15.86:0/4104157411 wait complete. 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 -- start start 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 Processor -- start 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c2974050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 0x5559c2974590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b2640 con 0x5559c2964b60 2024-09-05T07:01:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b1f70 con 0x5559c2965480 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1af1d0 con 0x5559c296fd40 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014596b740 10 client.0 _ll_register_callbacks cb 0x7f0138004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014132e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56246/0 (socket says 172.21.15.86:56246) 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f014132e640 1 -- 172.21.15.86:0/72542231 learned_addr learned my addr 172.21.15.86:0/72542231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f0141b2f640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 0x5559c2974590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.744+0000 7f0140b2d640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c2974050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 -- 172.21.15.86:0/72542231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 msgr2=0x5559c2974590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 0x5559c2974590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 -- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 msgr2=0x5559c2974050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c2974050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 -- 172.21.15.86:0/72542231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c29e9690 con 0x5559c2964b60 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0141b2f640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 0x5559c2974590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0140b2d640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c2974050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014132e640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f013809b100 tx=0x7f01380ccae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4601ceab2a31e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0124ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380976a0 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f01380ba260 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013404f070 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f013809b400 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014596b740 10 client.20073 fetch_fsmap learned FSMap version 45 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5559c29ea920 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.745+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f013404f070 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f013807d070 con 0x5559c2964b60 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 10 client.20073 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 10 client.20073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 20 client.20073 populate_metadata read hostname 'smithi086' 2024-09-05T07:01:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c29eb040 0x5559c29ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 4 client.20073 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559c28d1d00 con 0x5559c29eb040 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f0140b2d640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c29eb040 0x5559c29ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f0124ff9640 10 client.20073 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.746+0000 7f0140b2d640 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c29eb040 0x5559c29ed460 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745d85f1fe954d8f server_cookie=bb4a65ffb5b6494e in_seq=0 out_seq=0 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.747+0000 7f0124ff9640 1 -- 172.21.15.86:0/72542231 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559c28d1d00 con 0x5559c29eb040 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.747+0000 7f0124ff9640 10 client.20073 handle_command_reply: tid=0 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.747+0000 7f014596b740 1 client.20073 shutdown 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 msgr2=0x5559c2973b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f013809b100 tx=0x7f01380ccae0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 shutdown_connections 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5559c29eb040 0x5559c29ed460 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559c296fd40 0x5559c2974590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f0124ff9640 0 client.20073 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c2965480 0x5559c2974050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 --2- 172.21.15.86:0/72542231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559c2964b60 0x5559c2973b10 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 >> 172.21.15.86:0/72542231 conn(0x5559c29610f0 msgr2=0x5559c2960270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 shutdown_connections 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 1 -- 172.21.15.86:0/72542231 wait complete. 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.748+0000 7f014596b740 20 client.20073 trim_cache size 0 max 16384 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:01:58.753 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:01:58.754 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "9f943370-5634-47ec-b2e0-7e968b7e3c5d", 2024-09-05T07:01:58.754 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:01:58.754 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:01:58.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 msgr2=0x7f01280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 0x7f01280a51b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5559c2962470 tx=0x7f0134005c50 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 msgr2=0x7f013c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f013800d950 tx=0x7f013800de10 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 shutdown_connections 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f01280a2cf0 0x7f01280a51b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f013c10c170 0x7f013c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c103660 0x7f013c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 --2- 172.21.15.86:0/3772411869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c102d40 0x7f013c1009a0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 >> 172.21.15.86:0/3772411869 conn(0x7f013c0fc7a0 msgr2=0x7f013c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 shutdown_connections 2024-09-05T07:01:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:01:58.749+0000 7f0143592640 1 -- 172.21.15.86:0/3772411869 wait complete. 2024-09-05T07:01:58.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.50 MB/sec execute 316 sec latency 6399.073 ms 2024-09-05T07:01:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T07:01:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T07:01:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:58 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi115 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:58.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:58 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:59.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:01:59.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2024-09-05T07:01:59.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T07:01:59.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T07:01:59.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:59.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:01:59 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:59.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.48 MB/sec execute 317 sec latency 7399.102 ms 2024-09-05T07:01:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2024-09-05T07:01:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T07:01:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T07:01:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:59.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:01:59 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 184 op/s 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi096 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:01:59.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:01:59 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.572+0000 7f2fd6ffd640 1 -- 172.21.15.86:0/2229980150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0031e40 con 0x7f2fd81055b0 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.572+0000 7f2fde50b640 1 -- 172.21.15.86:0/2229980150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 msgr2=0x7f2fd810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.572+0000 7f2fde50b640 1 --2- 172.21.15.86:0/2229980150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd810d130 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009ca0 tx=0x7f2fd002a8c0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 -- 172.21.15.86:0/2229980150 shutdown_connections 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 --2- 172.21.15.86:0/2229980150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd810d130 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 --2- 172.21.15.86:0/2229980150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100e30 0x7f2fd8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 --2- 172.21.15.86:0/2229980150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100480 0x7f2fd8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 -- 172.21.15.86:0/2229980150 >> 172.21.15.86:0/2229980150 conn(0x7f2fd8077220 msgr2=0x7f2fd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 -- 172.21.15.86:0/2229980150 shutdown_connections 2024-09-05T07:02:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 -- 172.21.15.86:0/2229980150 wait complete. 2024-09-05T07:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 Processor -- start 2024-09-05T07:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.573+0000 7f2fde50b640 1 -- start start 2024-09-05T07:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 0x7f2fd80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810fc40 con 0x7f2fd81055b0 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810fac0 con 0x7f2fd8100480 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fde50b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810fdc0 con 0x7f2fd8100e30 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fdcaa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:43968/0 (socket says 172.21.15.86:43968) 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fdcaa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd80fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56262/0 (socket says 172.21.15.86:56262) 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 -- 172.21.15.86:0/323849156 learned_addr learned my addr 172.21.15.86:0/323849156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 0x7f2fd80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 -- 172.21.15.86:0/323849156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 msgr2=0x7f2fd80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 0x7f2fd80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 -- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 msgr2=0x7f2fd80fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd80fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd77fe640 1 -- 172.21.15.86:0/323849156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd81a6710 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.574+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 0x7f2fd80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c920 tx=0x7f2fcc00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fd57fa640 1 -- 172.21.15.86:0/323849156 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc007c70 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd81a6a00 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fd57fa640 1 -- 172.21.15.86:0/323849156 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fcc007e10 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fd57fa640 1 -- 172.21.15.86:0/323849156 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc018820 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.575+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a6f90 con 0x7f2fd8100e30 2024-09-05T07:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.577+0000 7f2fd57fa640 1 -- 172.21.15.86:0/323849156 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2fcc0189c0 con 0x7f2fd8100e30 2024-09-05T07:02:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 Processor -- start 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fd57fa640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 0x7f2fc00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 -- start start 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 0x7f2fc00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fd57fa640 1 -- 172.21.15.86:0/323849156 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f2fcc0cf910 con 0x7f2fd8100e30 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aace0 0x55f6fe2bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe2102e0 con 0x55f6fe2aace0 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe1b7ce0 con 0x55f6fe2bde90 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.578+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe20c2a0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:43976/0 (socket says 172.21.15.86:43976) 2024-09-05T07:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 -- 172.21.15.86:0/480598126 learned_addr learned my addr 172.21.15.86:0/480598126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aace0 0x55f6fe2bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 0x7f2fc00a5230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0004af0 tx=0x7f2fd0031a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 -- 172.21.15.86:0/480598126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 msgr2=0x55f6fe2c0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2c0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 -- 172.21.15.86:0/480598126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aace0 msgr2=0x55f6fe2bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aace0 0x55f6fe2bd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 -- 172.21.15.86:0/480598126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6fe19c440 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800bde0 tx=0x7f2fc800f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a688d9d9ab0bfd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fbe7fc640 1 -- 172.21.15.86:0/480598126 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800d860 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.579+0000 7f2fbe7fc640 1 -- 172.21.15.86:0/480598126 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc80173e0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fbe7fc640 1 -- 172.21.15.86:0/480598126 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800c5d0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 -- 172.21.15.86:0/480598126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 msgr2=0x55f6fe2aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800bde0 tx=0x7f2fc800f830 comp rx=0 tx=0).stop 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 -- 172.21.15.86:0/480598126 shutdown_connections 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2c0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aace0 0x55f6fe2bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/480598126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2aa7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 -- 172.21.15.86:0/480598126 >> 172.21.15.86:0/480598126 conn(0x55f6fe2a6a30 msgr2=0x55f6fe2a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 -- 172.21.15.86:0/480598126 shutdown_connections 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.580+0000 7f2fe08e4740 1 -- 172.21.15.86:0/480598126 wait complete. 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 -- start start 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 Processor -- start 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe2c0a50 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe1cb050 con 0x55f6fe2bde90 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6fe2c0bd0 con 0x55f6fe2aace0 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fe08e4740 10 client.0 _ll_register_callbacks cb 0x55f6fe20b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56284/0 (socket says 172.21.15.86:56284) 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fd7fff640 1 -- 172.21.15.86:0/4227346329 learned_addr learned my addr 172.21.15.86:0/4227346329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.581+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 -- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 msgr2=0x55f6fe2bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 -- 172.21.15.86:0/4227346329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 msgr2=0x55f6fe2bd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 -- 172.21.15.86:0/4227346329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6fdf813c0 con 0x55f6fe2aace0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2baaa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ccc20 tx=0x7f2fcc018d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24604a01c9b4d18b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a43d0 con 0x55f6fe2aace0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f2fcc0cf450 con 0x55f6fe2aace0 2024-09-05T07:02:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc80173e0 con 0x55f6fe2aace0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2fcc07f050 con 0x55f6fe2aace0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fe08e4740 10 client.28861 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f6fe2cc4c0 con 0x55f6fe2aace0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.582+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc80173e0 con 0x55f6fe2aace0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f2fcc0860b0 con 0x55f6fe2aace0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 10 client.28861 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 10 client.28861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 20 client.28861 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe2cd980 0x55f6fe2cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 4 client.28861 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6fe21d850 con 0x55f6fe2cd980 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe2cd980 0x55f6fe2cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fb77fe640 10 client.28861 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:00.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.583+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe2cd980 0x55f6fe2cfdd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e18aec35490ef89 server_cookie=38fbb72dc7803b0c in_seq=0 out_seq=0 2024-09-05T07:02:00.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.584+0000 7f2fb77fe640 1 -- 172.21.15.86:0/4227346329 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6fe21d850 con 0x55f6fe2cd980 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.584+0000 7f2fb77fe640 10 client.28861 handle_command_reply: tid=0 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.584+0000 7f2fe08e4740 1 client.28861 shutdown 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 msgr2=0x55f6fe2baaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2baaa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ccc20 tx=0x7f2fcc018d70 comp rx=0 tx=0).stop 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 shutdown_connections 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe2cd980 0x55f6fe2cfdd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2bde90 0x55f6fe2bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2baaa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/4227346329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 >> 172.21.15.86:0/4227346329 conn(0x55f6fe2a6a30 msgr2=0x55f6fe2a5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fb77fe640 0 client.28861 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 shutdown_connections 2024-09-05T07:02:00.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 1 -- 172.21.15.86:0/4227346329 wait complete. 2024-09-05T07:02:00.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.585+0000 7f2fe08e4740 20 client.28861 trim_cache size 0 max 16384 2024-09-05T07:02:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.603+0000 7f2fe08e4740 1 Processor -- start 2024-09-05T07:02:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.603+0000 7f2fe08e4740 1 -- start start 2024-09-05T07:02:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T07:02:00.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:00 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ace10 con 0x55f6fe2bde90 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ae950 con 0x55f6fe2aace0 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ad4e0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:43996/0 (socket says 172.21.15.86:43996) 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47560/0 (socket says 172.21.15.86:47560) 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 -- 172.21.15.86:0/1952477870 learned_addr learned my addr 172.21.15.86:0/1952477870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 -- 172.21.15.86:0/1952477870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 msgr2=0x55f6fe2cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 -- 172.21.15.86:0/1952477870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 msgr2=0x55f6fe2d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd7fff640 1 -- 172.21.15.86:0/1952477870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6fe2bb370 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.604+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.605+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8010fd0 tx=0x7f2fc8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174a73f354db786e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.605+0000 7f2fbdffb640 1 -- 172.21.15.86:0/1952477870 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc801d040 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.605+0000 7f2fbdffb640 1 -- 172.21.15.86:0/1952477870 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc80173e0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.605+0000 7f2fbdffb640 1 -- 172.21.15.86:0/1952477870 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8004680 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- 172.21.15.86:0/1952477870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 msgr2=0x55f6fe2c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8010fd0 tx=0x7f2fc8005e50 comp rx=0 tx=0).stop 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- 172.21.15.86:0/1952477870 shutdown_connections 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aace0 0x55f6fe2cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/1952477870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c9cf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- 172.21.15.86:0/1952477870 >> 172.21.15.86:0/1952477870 conn(0x55f6fe2a6a30 msgr2=0x55f6fe2b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- 172.21.15.86:0/1952477870 shutdown_connections 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- 172.21.15.86:0/1952477870 wait complete. 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.606+0000 7f2fe08e4740 1 -- start start 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 Processor -- start 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ae280 con 0x55f6fe2bde90 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81adbb0 con 0x55f6fe2aa3c0 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81aae10 con 0x55f6fe2aace0 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fe08e4740 10 client.0 _ll_register_callbacks cb 0x7f2fcc00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:44006/0 (socket says 172.21.15.86:44006) 2024-09-05T07:02:00.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fd77fe640 1 -- 172.21.15.86:0/3807931392 learned_addr learned my addr 172.21.15.86:0/3807931392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.607+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 -- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aa3c0 msgr2=0x55f6fe2c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 -- 172.21.15.86:0/3807931392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 msgr2=0x55f6fe2b7120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 -- 172.21.15.86:0/3807931392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6fe2b7660 con 0x55f6fe2aace0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fdcaa8640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fd77fe640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc09b980 tx=0x7f2fcc09b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c7b5b445b963f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09b9f0 con 0x55f6fe2aace0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f2fcc0a41d0 con 0x55f6fe2aace0 2024-09-05T07:02:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.608+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc80173e0 con 0x55f6fe2aace0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2fcc09fda0 con 0x55f6fe2aace0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 10 client.28867 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f6fe337e80 con 0x55f6fe2aace0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc80173e0 con 0x55f6fe2aace0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f2fcc082050 con 0x55f6fe2aace0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 10 client.28867 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 10 client.28867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 20 client.28867 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe338470 0x55f6fe33a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 4 client.28867 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.609+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6fe217d60 con 0x55f6fe338470 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe338470 0x55f6fe33a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fb77fe640 10 client.28867 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fd7fff640 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe338470 0x55f6fe33a890 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0652fd4c4f8967 server_cookie=2a19690c2c1a2ff3 in_seq=0 out_seq=0 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fb77fe640 1 -- 172.21.15.86:0/3807931392 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f6fe217d60 con 0x55f6fe338470 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fb77fe640 10 client.28867 handle_command_reply: tid=0 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fe08e4740 1 client.28867 shutdown 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.610+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 msgr2=0x55f6fe2c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc09b980 tx=0x7f2fcc09b390 comp rx=0 tx=0).stop 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 shutdown_connections 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f6fe338470 0x55f6fe33a890 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6fe2bde90 0x55f6fe2b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6fe2aace0 0x55f6fe2c63a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 --2- 172.21.15.86:0/3807931392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6fe2aa3c0 0x55f6fe2c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 >> 172.21.15.86:0/3807931392 conn(0x55f6fe2a6a30 msgr2=0x55f6fe2b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fb77fe640 0 client.28867 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 shutdown_connections 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 1 -- 172.21.15.86:0/3807931392 wait complete. 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.611+0000 7f2fe08e4740 20 client.28867 trim_cache size 0 max 16384 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "9f943370-5634-47ec-b2e0-7e968b7e3c5d": { 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "9f943370-5634-47ec-b2e0-7e968b7e3c5d", 2024-09-05T07:02:00.617 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 msgr2=0x7f2fc00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 0x7f2fc00a5230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0004af0 tx=0x7f2fd0031a40 comp rx=0 tx=0).stop 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 msgr2=0x7f2fd80fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c920 tx=0x7f2fcc00cde0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 shutdown_connections 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2fc00a2d70 0x7f2fc00a5230 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd81055b0 0x7f2fd80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd8100e30 0x7f2fd80fcbd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 --2- 172.21.15.86:0/323849156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd8100480 0x7f2fd80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 >> 172.21.15.86:0/323849156 conn(0x7f2fd8077220 msgr2=0x7f2fd80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 shutdown_connections 2024-09-05T07:02:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:00.612+0000 7f2fde50b640 1 -- 172.21.15.86:0/323849156 wait complete. 2024-09-05T07:02:00.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.46 MB/sec execute 318 sec latency 8399.160 ms 2024-09-05T07:02:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T07:02:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T07:02:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:02:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:00 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: Adjusting osd_memory_target on smithi086 to 176.0M 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: Unable to set osd_memory_target on smithi086 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:02:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:00.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:00 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:02:01.423 INFO:tasks.cephfs.filesystem:scrub status for tag:9f943370-5634-47ec-b2e0-7e968b7e3c5d - {'path': '/', 'tag': '9f943370-5634-47ec-b2e0-7e968b7e3c5d', 'options': 'recursive,force'} 2024-09-05T07:02:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:01 smithi086 ceph-mon[30943]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-05T07:02:01.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.44 MB/sec execute 319 sec latency 9399.197 ms 2024-09-05T07:02:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:01 smithi096 ceph-mon[34468]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-05T07:02:01.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:01 smithi115 ceph-mon[34791]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-05T07:02:02.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.42 MB/sec execute 320 sec latency 10399.315 ms 2024-09-05T07:02:03.771 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.40 MB/sec execute 321 sec latency 11399.399 ms 2024-09-05T07:02:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:03 smithi096 ceph-mon[34468]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-05T07:02:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:03 smithi115 ceph-mon[34791]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-05T07:02:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:03 smithi086 ceph-mon[30943]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-05T07:02:04.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.38 MB/sec execute 322 sec latency 12399.510 ms 2024-09-05T07:02:05.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.36 MB/sec execute 323 sec latency 13399.629 ms 2024-09-05T07:02:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:05 smithi096 ceph-mon[34468]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 810 KiB/s wr, 93 op/s 2024-09-05T07:02:05.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:05 smithi115 ceph-mon[34791]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 810 KiB/s wr, 93 op/s 2024-09-05T07:02:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:05 smithi086 ceph-mon[30943]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 810 KiB/s wr, 93 op/s 2024-09-05T07:02:06.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.34 MB/sec execute 324 sec latency 14399.754 ms 2024-09-05T07:02:06.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:02:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:07.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:07 smithi086 ceph-mon[30943]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 567 KiB/s wr, 82 op/s 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 -- 172.21.15.86:0/742165666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0100a80 msgr2=0x7f75e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/742165666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0100a80 0x7f75e0100ee0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009930 tx=0x7f75d402a420 comp rx=0 tx=0).stop 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 -- 172.21.15.86:0/742165666 shutdown_connections 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/742165666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0101420 0x7f75e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/742165666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0100a80 0x7f75e0100ee0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/742165666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 0x7f75e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 -- 172.21.15.86:0/742165666 >> 172.21.15.86:0/742165666 conn(0x7f75e00fc7a0 msgr2=0x7f75e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 -- 172.21.15.86:0/742165666 shutdown_connections 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.618+0000 7f75e5dbd640 1 -- 172.21.15.86:0/742165666 wait complete. 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 Processor -- start 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 -- start start 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0100a80 0x7f75e01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 0x7f75e019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0110d30 con 0x7f75e0101420 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0110bb0 con 0x7f75e0106a80 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.619+0000 7f75e5dbd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0110eb0 con 0x7f75e0100a80 2024-09-05T07:02:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44824/0 (socket says 172.21.15.86:44824) 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 -- 172.21.15.86:0/4060428078 learned_addr learned my addr 172.21.15.86:0/4060428078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75dffff640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 0x7f75e019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75df7fe640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0100a80 0x7f75e01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 -- 172.21.15.86:0/4060428078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0100a80 msgr2=0x7f75e01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0100a80 0x7f75e01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 -- 172.21.15.86:0/4060428078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 msgr2=0x7f75e019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 0x7f75e019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 -- 172.21.15.86:0/4060428078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e019d270 con 0x7f75e0101420 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75deffd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f75d4004770 tx=0x7f75d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d403f070 con 0x7f75e0101420 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75d403b460 con 0x7f75e0101420 2024-09-05T07:02:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e019d500 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4035070 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.620+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a4e40 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.621+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e01021c0 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.622+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f75d403b610 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.623+0000 7f75dcff9640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 0x7f75c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.623+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7f75d40efc50 con 0x7f75e0101420 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.623+0000 7f75df7fe640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 0x7f75c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.623+0000 7f75df7fe640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 0x7f75c80a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005fd0 tx=0x7f75d0005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.627+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75d40f5020 con 0x7f75e0101420 2024-09-05T07:02:07.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.32 MB/sec execute 325 sec latency 15399.824 ms 2024-09-05T07:02:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:07 smithi096 ceph-mon[34468]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 567 KiB/s wr, 82 op/s 2024-09-05T07:02:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.828+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75e010a030 con 0x7f75e0101420 2024-09-05T07:02:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.829+0000 7f75dcff9640 1 -- 172.21.15.86:0/4060428078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7f75d40bc430 con 0x7f75e0101420 2024-09-05T07:02:07.829 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:02:07.830 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:02:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:02:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 msgr2=0x7f75c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 0x7f75c80a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005fd0 tx=0x7f75d0005850 comp rx=0 tx=0).stop 2024-09-05T07:02:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 msgr2=0x7f75e0198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f75d4004770 tx=0x7f75d40042f0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 shutdown_connections 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f75c80a2cf0 0x7f75c80a51b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75e0106a80 0x7f75e019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e0101420 0x7f75e0198800 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 --2- 172.21.15.86:0/4060428078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e0100a80 0x7f75e01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 >> 172.21.15.86:0/4060428078 conn(0x7f75e00fc7a0 msgr2=0x7f75e006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 shutdown_connections 2024-09-05T07:02:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:07.831+0000 7f75e5dbd640 1 -- 172.21.15.86:0/4060428078 wait complete. 2024-09-05T07:02:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:07 smithi115 ceph-mon[34791]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 567 KiB/s wr, 82 op/s 2024-09-05T07:02:08.525 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id af59a075 --release --await' 2024-09-05T07:02:08.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id af59a075 --release --await 2024-09-05T07:02:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:08 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4060428078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:08.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53355 6.30 MB/sec execute 326 sec latency 16399.855 ms 2024-09-05T07:02:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:08 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4060428078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:08 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4060428078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 -- 172.21.15.86:0/1426629558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0100a80 msgr2=0x7fdaf0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 --2- 172.21.15.86:0/1426629558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0100a80 0x7fdaf0100ee0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009930 tx=0x7fdae002a420 comp rx=0 tx=0).stop 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 -- 172.21.15.86:0/1426629558 shutdown_connections 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 --2- 172.21.15.86:0/1426629558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 --2- 172.21.15.86:0/1426629558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0100a80 0x7fdaf0100ee0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 --2- 172.21.15.86:0/1426629558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0106a80 0x7fdaf0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 -- 172.21.15.86:0/1426629558 >> 172.21.15.86:0/1426629558 conn(0x7fdaf00fc7a0 msgr2=0x7fdaf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 -- 172.21.15.86:0/1426629558 shutdown_connections 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.276+0000 7fdaf8207640 1 -- 172.21.15.86:0/1426629558 wait complete. 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 Processor -- start 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 -- start start 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 0x7fdaf0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0110cf0 con 0x7fdaf0106a80 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0110b70 con 0x7fdaf0100a80 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf8207640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0110e70 con 0x7fdaf0101420 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44840/0 (socket says 172.21.15.86:44840) 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 -- 172.21.15.86:0/2297684069 learned_addr learned my addr 172.21.15.86:0/2297684069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 0x7fdaf0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 -- 172.21.15.86:0/2297684069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 msgr2=0x7fdaf01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 -- 172.21.15.86:0/2297684069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 msgr2=0x7fdaf0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 0x7fdaf0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 -- 172.21.15.86:0/2297684069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf01a4820 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 0x7fdaf0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdaf67a4640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdae800eaa0 tx=0x7fdae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.277+0000 7fdae6ffd640 1 -- 172.21.15.86:0/2297684069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800cdb0 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.278+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf01a49f0 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.278+0000 7fdae6ffd640 1 -- 172.21.15.86:0/2297684069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae8005020 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.278+0000 7fdae6ffd640 1 -- 172.21.15.86:0/2297684069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80116d0 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.278+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a4eb0 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.280+0000 7fdae6ffd640 1 -- 172.21.15.86:0/2297684069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdae80051d0 con 0x7fdaf0106a80 2024-09-05T07:02:09.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.280+0000 7fdafa5e0740 1 Processor -- start 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 -- start start 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 0x55a9a8b82630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8ada620 con 0x55a9a8b75980 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8a81e50 con 0x55a9a8b80240 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8ad65e0 con 0x55a9a8b75060 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44850/0 (socket says 172.21.15.86:44850) 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/2432180302 learned_addr learned my addr 172.21.15.86:0/2432180302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 0x55a9a8b82630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdae6ffd640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 0x7fdadc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/2432180302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 msgr2=0x55a9a8b75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/2432180302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 msgr2=0x55a9a8b82630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 0x55a9a8b82630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/2432180302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a8a72d40 con 0x55a9a8b75980 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 0x55a9a8b82630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.281+0000 7fdaf67a4640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 0x7fdadc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.282+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bd0 tx=0x7fdaec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e4b2aec38dff75 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.282+0000 7fdae6ffd640 1 -- 172.21.15.86:0/2297684069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fdae80cfb40 con 0x7fdaf0106a80 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.282+0000 7fdadbfff640 1 -- 172.21.15.86:0/2432180302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec016070 con 0x55a9a8b75980 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.282+0000 7fdadbfff640 1 -- 172.21.15.86:0/2432180302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec0054d0 con 0x55a9a8b75980 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.282+0000 7fdadbfff640 1 -- 172.21.15.86:0/2432180302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec011470 con 0x55a9a8b75980 2024-09-05T07:02:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdaf67a4640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 0x7fdadc0a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdae80985f0 tx=0x7fdae8086750 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- 172.21.15.86:0/2432180302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 msgr2=0x55a9a8b7fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bd0 tx=0x7fdaec00e6c0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- 172.21.15.86:0/2432180302 shutdown_connections 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b80240 0x55a9a8b82630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b7fd00 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/2432180302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- 172.21.15.86:0/2432180302 >> 172.21.15.86:0/2432180302 conn(0x55a9a8b715f0 msgr2=0x55a9a8b719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- 172.21.15.86:0/2432180302 shutdown_connections 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- 172.21.15.86:0/2432180302 wait complete. 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 -- start start 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.283+0000 7fdafa5e0740 1 Processor -- start 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75980 0x55a9a8b7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8b82e10 con 0x55a9a8b80240 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8a62f10 con 0x55a9a8b75980 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8b82f90 con 0x55a9a8b75060 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdafa5e0740 10 client.0 _ll_register_callbacks cb 0x55a9a8a86db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44854/0 (socket says 172.21.15.86:44854) 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 -- 172.21.15.86:0/844663040 learned_addr learned my addr 172.21.15.86:0/844663040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75980 0x55a9a8b7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 -- 172.21.15.86:0/844663040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 msgr2=0x55a9a8b7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b7d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 -- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75980 msgr2=0x55a9a8b7d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75980 0x55a9a8b7d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 -- 172.21.15.86:0/844663040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a87e9580 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdae00096c0 tx=0x7fdae002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d40556ec06d455 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdad97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0035040 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.284+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fdae00035f0 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec0054d0 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdae00037f0 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 10 client.20106 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a9a88b3960 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec0054d0 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fdae003a040 con 0x55a9a8b80240 2024-09-05T07:02:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 10 client.20106 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 10 client.20106 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 20 client.20106 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8b8fcc0 0x55a9a8b920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 4 client.20106 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a8ae7b90 con 0x55a9a8b8fcc0 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.285+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8b8fcc0 0x55a9a8b920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.286+0000 7fdad97fa640 10 client.20106 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.286+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8b8fcc0 0x55a9a8b920d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a015d39b71c0a48 server_cookie=1a393ffbe04acf65 in_seq=0 out_seq=0 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.286+0000 7fdad97fa640 1 -- 172.21.15.86:0/844663040 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9a8ae7b90 con 0x55a9a8b8fcc0 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.286+0000 7fdad97fa640 10 client.20106 handle_command_reply: tid=0 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.286+0000 7fdafa5e0740 1 client.20106 shutdown 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 msgr2=0x55a9a8b7db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdae00096c0 tx=0x7fdae002a420 comp rx=0 tx=0).stop 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 shutdown_connections 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8b8fcc0 0x55a9a8b920d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b80240 0x55a9a8b7db50 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75980 0x55a9a8b7d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdad97fa640 0 client.20106 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/844663040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b75060 0x55a9a8b7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 >> 172.21.15.86:0/844663040 conn(0x55a9a8b715f0 msgr2=0x55a9a8b772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 shutdown_connections 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 1 -- 172.21.15.86:0/844663040 wait complete. 2024-09-05T07:02:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.287+0000 7fdafa5e0740 20 client.20106 trim_cache size 0 max 16384 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.303+0000 7fdafa5e0740 1 Processor -- start 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 -- start start 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8b77ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8b885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01af2f0 con 0x55a9a8b75980 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b0e30 con 0x55a9a8b75060 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01af9c0 con 0x55a9a8b80240 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44866/0 (socket says 172.21.15.86:44866) 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4095169134 learned_addr learned my addr 172.21.15.86:0/4095169134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8b885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8b77ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4095169134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 msgr2=0x55a9a8b885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8b885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4095169134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 msgr2=0x55a9a8b77ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8b77ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4095169134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a8b7c080 con 0x55a9a8b75980 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8b77ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.304+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bd0 tx=0x7fdaec004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a0c2b6e41f0775 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdadb7fe640 1 -- 172.21.15.86:0/4095169134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec016040 con 0x55a9a8b75980 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdadb7fe640 1 -- 172.21.15.86:0/4095169134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec004680 con 0x55a9a8b75980 2024-09-05T07:02:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdadb7fe640 1 -- 172.21.15.86:0/4095169134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004cb0 con 0x55a9a8b75980 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4095169134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 msgr2=0x55a9a8b70a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009bd0 tx=0x7fdaec004c40 comp rx=0 tx=0).stop 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4095169134 shutdown_connections 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8b885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8b70a10 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4095169134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8b77ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4095169134 >> 172.21.15.86:0/4095169134 conn(0x55a9a8b715f0 msgr2=0x55a9a8b7e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4095169134 shutdown_connections 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.305+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4095169134 wait complete. 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 -- start start 2024-09-05T07:02:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 Processor -- start 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8bf6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8bf2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b0760 con 0x55a9a8b75980 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b0090 con 0x55a9a8b75060 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01ad2f0 con 0x55a9a8b80240 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdafa5e0740 10 client.0 _ll_register_callbacks cb 0x7fdae0031880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdaf5fa3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44868/0 (socket says 172.21.15.86:44868) 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.306+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4068155707 learned_addr learned my addr 172.21.15.86:0/4068155707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8bf2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8bf6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4068155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 msgr2=0x55a9a8bf2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8bf2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 msgr2=0x55a9a8bf6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8bf6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 -- 172.21.15.86:0/4068155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a8bf3140 con 0x55a9a8b75980 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8bf2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdaf5fa3640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009510 tx=0x7fdaec009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9168d68aae1a4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdad97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec016040 con 0x55a9a8b75980 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec004680 con 0x55a9a8b75980 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9605+0+0 (secure 0 0 0) 0x7fdaec011470 con 0x55a9a8b75980 2024-09-05T07:02:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdaec044050 con 0x55a9a8b75980 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdafa5e0740 10 client.20112 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a9a8bff720 con 0x55a9a8b75980 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.307+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec004680 con 0x55a9a8b75980 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fdaec00f040 con 0x55a9a8b75980 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 10 client.20112 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 10 client.20112 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:02:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 20 client.20112 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8bffc40 0x55a9a8c02030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 4 client.20112 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "af59a075", "release": true, "await": true}] 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "af59a075", "release": true, "await": true}) -- 0x55a9a8ae20a0 con 0x55a9a8bffc40 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8bffc40 0x55a9a8c02030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdad97fa640 10 client.20112 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:09.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.308+0000 7fdaf57a2640 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8bffc40 0x55a9a8c02030 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a39ca57c743cb3 server_cookie=f2d4f441aa8feaa7 in_seq=0 out_seq=0 2024-09-05T07:02:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.336+0000 7fdad97fa640 1 -- 172.21.15.86:0/4068155707 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a9a8ae20a0 con 0x55a9a8bffc40 2024-09-05T07:02:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.336+0000 7fdad97fa640 10 client.20112 handle_command_reply: tid=0 2024-09-05T07:02:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.336+0000 7fdafa5e0740 1 client.20112 shutdown 2024-09-05T07:02:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 msgr2=0x55a9a8bf7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009510 tx=0x7fdaec009540 comp rx=0 tx=0).stop 2024-09-05T07:02:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 shutdown_connections 2024-09-05T07:02:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55a9a8bffc40 0x55a9a8c02030 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a9a8b80240 0x55a9a8bf2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9a8b75980 0x55a9a8bf7320 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 --2- 172.21.15.86:0/4068155707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9a8b75060 0x55a9a8bf6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 >> 172.21.15.86:0/4068155707 conn(0x55a9a8b715f0 msgr2=0x55a9a8b7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdad97fa640 0 client.20112 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 shutdown_connections 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.337+0000 7fdafa5e0740 1 -- 172.21.15.86:0/4068155707 wait complete. 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.338+0000 7fdafa5e0740 20 client.20112 trim_cache size 0 max 16384 2024-09-05T07:02:09.339 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 45, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 80, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "af59a075": { 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 80, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:02:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 139.3, 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:02:09.341 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 msgr2=0x7fdadc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 0x7fdadc0a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdae80985f0 tx=0x7fdae8086750 comp rx=0 tx=0).stop 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 msgr2=0x7fdaf01a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdae800eaa0 tx=0x7fdae800ef60 comp rx=0 tx=0).stop 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 shutdown_connections 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdadc0a2cf0 0x7fdadc0a51b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0106a80 0x7fdaf01a42e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaf0101420 0x7fdaf01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 --2- 172.21.15.86:0/2297684069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf0100a80 0x7fdaf0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 >> 172.21.15.86:0/2297684069 conn(0x7fdaf00fc7a0 msgr2=0x7fdaf00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 shutdown_connections 2024-09-05T07:02:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:09.339+0000 7fdaf8207640 1 -- 172.21.15.86:0/2297684069 wait complete. 2024-09-05T07:02:09.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53428 6.29 MB/sec execute 327 sec latency 16946.836 ms 2024-09-05T07:02:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:09 smithi096 ceph-mon[34468]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 580 KiB/s wr, 88 op/s 2024-09-05T07:02:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:09 smithi115 ceph-mon[34791]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 580 KiB/s wr, 88 op/s 2024-09-05T07:02:09.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:09 smithi086 ceph-mon[30943]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 580 KiB/s wr, 88 op/s 2024-09-05T07:02:10.078 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'af59a075', total seconds elapsed: 22.5 2024-09-05T07:02:10.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53565 6.28 MB/sec execute 328 sec latency 153.752 ms 2024-09-05T07:02:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:10 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:02:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:10 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:02:10.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:10 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:02:10.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:10 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:02:10.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:10 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:02:10.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:10 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:02:11.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53721 6.28 MB/sec execute 329 sec latency 82.972 ms 2024-09-05T07:02:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:11 smithi096 ceph-mon[34468]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-09-05T07:02:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:02:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:02:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:11 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:02:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:11 smithi096 ceph-mon[34468]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T07:02:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:11 smithi115 ceph-mon[34791]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-09-05T07:02:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:02:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:02:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:11 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:02:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:11 smithi115 ceph-mon[34791]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T07:02:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:11 smithi086 ceph-mon[30943]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-09-05T07:02:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:02:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:02:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:11 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:02:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:11 smithi086 ceph-mon[30943]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T07:02:12.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53914 6.29 MB/sec execute 330 sec latency 61.435 ms 2024-09-05T07:02:13.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54107 6.30 MB/sec execute 331 sec latency 69.474 ms 2024-09-05T07:02:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:13 smithi096 ceph-mon[34468]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 775 KiB/s wr, 69 op/s 2024-09-05T07:02:13.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:13 smithi115 ceph-mon[34791]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 775 KiB/s wr, 69 op/s 2024-09-05T07:02:13.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:13 smithi086 ceph-mon[30943]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 775 KiB/s wr, 69 op/s 2024-09-05T07:02:14.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54267 6.30 MB/sec execute 332 sec latency 68.674 ms 2024-09-05T07:02:15.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54443 6.33 MB/sec execute 333 sec latency 109.322 ms 2024-09-05T07:02:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:15 smithi096 ceph-mon[34468]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:15 smithi115 ceph-mon[34791]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:15.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:15 smithi086 ceph-mon[30943]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:16.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54642 6.35 MB/sec execute 334 sec latency 94.014 ms 2024-09-05T07:02:17.772 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54871 6.38 MB/sec execute 335 sec latency 240.889 ms 2024-09-05T07:02:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:17 smithi096 ceph-mon[34468]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:17 smithi096 ceph-mon[34468]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T07:02:17.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:17 smithi115 ceph-mon[34791]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:17.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:17 smithi115 ceph-mon[34791]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T07:02:17.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:17 smithi086 ceph-mon[30943]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 941 KiB/s wr, 58 op/s 2024-09-05T07:02:17.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:17 smithi086 ceph-mon[30943]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T07:02:18.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55039 6.39 MB/sec execute 336 sec latency 95.102 ms 2024-09-05T07:02:19.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55181 6.40 MB/sec execute 337 sec latency 61.603 ms 2024-09-05T07:02:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:19 smithi096 ceph-mon[34468]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 229 op/s 2024-09-05T07:02:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:19 smithi115 ceph-mon[34791]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 229 op/s 2024-09-05T07:02:19.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:19 smithi086 ceph-mon[30943]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 229 op/s 2024-09-05T07:02:20.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55349 6.41 MB/sec execute 338 sec latency 138.727 ms 2024-09-05T07:02:21.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55518 6.42 MB/sec execute 339 sec latency 181.436 ms 2024-09-05T07:02:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:21 smithi096 ceph-mon[34468]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-05T07:02:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:21 smithi115 ceph-mon[34791]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-05T07:02:21.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:21 smithi086 ceph-mon[30943]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-05T07:02:22.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55682 6.42 MB/sec execute 340 sec latency 157.266 ms 2024-09-05T07:02:23.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55841 6.41 MB/sec execute 341 sec latency 78.082 ms 2024-09-05T07:02:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:23 smithi096 ceph-mon[34468]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 258 op/s 2024-09-05T07:02:23.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:23 smithi115 ceph-mon[34791]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 258 op/s 2024-09-05T07:02:23.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:23 smithi086 ceph-mon[30943]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 258 op/s 2024-09-05T07:02:24.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55992 6.42 MB/sec execute 342 sec latency 82.351 ms 2024-09-05T07:02:25.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56140 6.41 MB/sec execute 343 sec latency 183.832 ms 2024-09-05T07:02:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:25 smithi096 ceph-mon[34468]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:25 smithi115 ceph-mon[34791]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:25.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:25 smithi086 ceph-mon[30943]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:26.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56322 6.42 MB/sec execute 344 sec latency 107.050 ms 2024-09-05T07:02:27.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56523 6.44 MB/sec execute 345 sec latency 245.026 ms 2024-09-05T07:02:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:27 smithi096 ceph-mon[34468]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:27 smithi115 ceph-mon[34791]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:27 smithi086 ceph-mon[30943]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 271 op/s 2024-09-05T07:02:28.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56721 6.46 MB/sec execute 346 sec latency 241.685 ms 2024-09-05T07:02:29.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56865 6.47 MB/sec execute 347 sec latency 161.171 ms 2024-09-05T07:02:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:29 smithi096 ceph-mon[34468]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-05T07:02:29.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:29 smithi115 ceph-mon[34791]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-05T07:02:29.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:29 smithi086 ceph-mon[30943]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-09-05T07:02:30.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57011 6.46 MB/sec execute 348 sec latency 177.498 ms 2024-09-05T07:02:31.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:02:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:31 smithi086 ceph-mon[30943]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 245 op/s 2024-09-05T07:02:31.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57188 6.47 MB/sec execute 349 sec latency 84.797 ms 2024-09-05T07:02:31.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:31 smithi096 ceph-mon[34468]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 245 op/s 2024-09-05T07:02:31.825 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:31.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:31 smithi115 ceph-mon[34791]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 245 op/s 2024-09-05T07:02:32.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.421+0000 7faf83edb640 1 -- 172.21.15.86:0/1943952649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c101de0 msgr2=0x7faf7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.421+0000 7faf83edb640 1 --2- 172.21.15.86:0/1943952649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c101de0 0x7faf7c1021c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faf78009ca0 tx=0x7faf7802a8c0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 -- 172.21.15.86:0/1943952649 shutdown_connections 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 --2- 172.21.15.86:0/1943952649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c106f10 0x7faf7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 --2- 172.21.15.86:0/1943952649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c102790 0x7faf7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 --2- 172.21.15.86:0/1943952649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c101de0 0x7faf7c1021c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 -- 172.21.15.86:0/1943952649 >> 172.21.15.86:0/1943952649 conn(0x7faf7c0fd8e0 msgr2=0x7faf7c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 -- 172.21.15.86:0/1943952649 shutdown_connections 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 -- 172.21.15.86:0/1943952649 wait complete. 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 Processor -- start 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.422+0000 7faf83edb640 1 -- start start 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c102790 0x7faf7c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c106f10 0x7faf7c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111e70 con 0x7faf7c102790 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111cf0 con 0x7faf7c106f10 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf83edb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111ff0 con 0x7faf7c101de0 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf81c77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf81c77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52726/0 (socket says 172.21.15.86:52726) 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf81c77640 1 -- 172.21.15.86:0/361470234 learned_addr learned my addr 172.21.15.86:0/361470234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf81476640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c102790 0x7faf7c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.423+0000 7faf81c77640 1 -- 172.21.15.86:0/361470234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c106f10 msgr2=0x7faf7c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf81c77640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c106f10 0x7faf7c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf81c77640 1 -- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c102790 msgr2=0x7faf7c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf81c77640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c102790 0x7faf7c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf81c77640 1 -- 172.21.15.86:0/361470234 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c19d3c0 con 0x7faf7c101de0 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf81c77640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf78031040 tx=0x7faf78032ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf72ffd640 1 -- 172.21.15.86:0/361470234 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78003b70 con 0x7faf7c101de0 2024-09-05T07:02:32.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c19d650 con 0x7faf7c101de0 2024-09-05T07:02:32.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a4f90 con 0x7faf7c101de0 2024-09-05T07:02:32.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf72ffd640 1 -- 172.21.15.86:0/361470234 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf78003d10 con 0x7faf7c101de0 2024-09-05T07:02:32.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.424+0000 7faf72ffd640 1 -- 172.21.15.86:0/361470234 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78030ac0 con 0x7faf7c101de0 2024-09-05T07:02:32.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.427+0000 7faf72ffd640 1 -- 172.21.15.86:0/361470234 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faf78030cc0 con 0x7faf7c101de0 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 Processor -- start 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 -- start start 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf72ffd640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 0x7faf680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438dee680 0x558438df0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558438d492e0 con 0x558438de3dc0 2024-09-05T07:02:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558438cf0ce0 con 0x558438dee680 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf862b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558438d452a0 con 0x558438de34a0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf81476640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf82478640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf81476640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48678/0 (socket says 172.21.15.86:48678) 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf82478640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:52738/0 (socket says 172.21.15.86:52738) 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf81476640 1 -- 172.21.15.86:0/346011959 learned_addr learned my addr 172.21.15.86:0/346011959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.428+0000 7faf72ffd640 1 -- 172.21.15.86:0/361470234 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7faf780f0810 con 0x7faf7c101de0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438dee680 0x558438df0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf81c77640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 0x7faf680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 -- 172.21.15.86:0/346011959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438dee680 msgr2=0x558438df0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438dee680 0x558438df0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 -- 172.21.15.86:0/346011959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 msgr2=0x558438dee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 -- 172.21.15.86:0/346011959 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558438cd5440 con 0x558438de34a0 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf81476640 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf82478640 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf74009ed0 tx=0x7faf74009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869f72d001d9f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf81c77640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 0x7faf680a5230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf780bc790 tx=0x7faf780095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.429+0000 7faf67fff640 1 -- 172.21.15.86:0/346011959 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74004050 con 0x558438de34a0 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf67fff640 1 -- 172.21.15.86:0/346011959 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf740041f0 con 0x558438de34a0 2024-09-05T07:02:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf67fff640 1 -- 172.21.15.86:0/346011959 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74003ea0 con 0x558438de34a0 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 -- 172.21.15.86:0/346011959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 msgr2=0x558438de3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf74009ed0 tx=0x7faf74009490 comp rx=0 tx=0).stop 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 -- 172.21.15.86:0/346011959 shutdown_connections 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438dee680 0x558438df0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de3dc0 0x558438dee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 --2- 172.21.15.86:0/346011959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de34a0 0x558438de3880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.430+0000 7faf862b4740 1 -- 172.21.15.86:0/346011959 >> 172.21.15.86:0/346011959 conn(0x558438ddfa30 msgr2=0x558438ddfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.431+0000 7faf862b4740 1 -- 172.21.15.86:0/346011959 shutdown_connections 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.431+0000 7faf862b4740 1 -- 172.21.15.86:0/346011959 wait complete. 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.431+0000 7faf862b4740 1 -- start start 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.431+0000 7faf862b4740 1 Processor -- start 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.431+0000 7faf862b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438dedf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438de6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558438df5700 con 0x558438dee680 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558438d04050 con 0x558438de34a0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558438df5880 con 0x558438de3dc0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46388/0 (socket says 172.21.15.86:46388) 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 -- 172.21.15.86:0/4263835274 learned_addr learned my addr 172.21.15.86:0/4263835274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438de6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf862b4740 10 client.0 _ll_register_callbacks cb 0x558438d44160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf81476640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438dedf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 -- 172.21.15.86:0/4263835274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 msgr2=0x558438dedf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438dedf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 -- 172.21.15.86:0/4263835274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 msgr2=0x558438de6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438de6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf82478640 1 -- 172.21.15.86:0/4263835274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558438aba3c0 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.432+0000 7faf81476640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438dedf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf82478640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf74016040 tx=0x7faf74003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690547670f120cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf657fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74003c20 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7faf74003dc0 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf740041f0 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faf74003fc0 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf862b4740 10 client.28868 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558438e055b0 con 0x558438de34a0 2024-09-05T07:02:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.433+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf740041f0 con 0x558438de34a0 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7faf7401e440 con 0x558438de34a0 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 10 client.28868 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 10 client.28868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 20 client.28868 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e06900 0x558438e08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 4 client.28868 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558438d56850 con 0x558438e06900 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf81476640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e06900 0x558438e08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf81476640 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e06900 0x558438e08cf0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ead2384d74634f server_cookie=a0dfefbf986cfe10 in_seq=0 out_seq=0 2024-09-05T07:02:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.434+0000 7faf657fa640 10 client.28868 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.435+0000 7faf657fa640 1 -- 172.21.15.86:0/4263835274 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558438d56850 con 0x558438e06900 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.435+0000 7faf657fa640 10 client.28868 handle_command_reply: tid=0 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.435+0000 7faf862b4740 1 client.28868 shutdown 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 msgr2=0x558438deda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf74016040 tx=0x7faf74003b40 comp rx=0 tx=0).stop 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 shutdown_connections 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e06900 0x558438e08cf0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438de6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438dedf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 --2- 172.21.15.86:0/4263835274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438deda20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf657fa640 0 client.28868 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:32.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 >> 172.21.15.86:0/4263835274 conn(0x558438ddfa30 msgr2=0x558438ddeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 shutdown_connections 2024-09-05T07:02:32.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 1 -- 172.21.15.86:0/4263835274 wait complete. 2024-09-05T07:02:32.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.436+0000 7faf862b4740 20 client.28868 trim_cache size 0 max 16384 2024-09-05T07:02:32.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 Processor -- start 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 -- start start 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de3dc0 0x558438e0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438dee680 0x558438e0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf600100e0 con 0x558438de34a0 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf60011c20 con 0x558438de3dc0 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf862b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf600107b0 con 0x558438dee680 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf82478640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf82478640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48696/0 (socket says 172.21.15.86:48696) 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf82478640 1 -- 172.21.15.86:0/1949126237 learned_addr learned my addr 172.21.15.86:0/1949126237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf82478640 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438dee680 0x558438e0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.463+0000 7faf81476640 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de3dc0 0x558438e0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 -- 172.21.15.86:0/1949126237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438dee680 msgr2=0x558438e0eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438dee680 0x558438e0eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 -- 172.21.15.86:0/1949126237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de3dc0 msgr2=0x558438e0e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de3dc0 0x558438e0e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 -- 172.21.15.86:0/1949126237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558438de3bf0 con 0x558438de34a0 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf82478640 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf74009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a609a124ec5c69f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf677fe640 1 -- 172.21.15.86:0/1949126237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7401ea80 con 0x558438de34a0 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf677fe640 1 -- 172.21.15.86:0/1949126237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf7401ec20 con 0x558438de34a0 2024-09-05T07:02:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.464+0000 7faf677fe640 1 -- 172.21.15.86:0/1949126237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7401e6a0 con 0x558438de34a0 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 -- 172.21.15.86:0/1949126237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 msgr2=0x558438de5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf74009430 comp rx=0 tx=0).stop 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 -- 172.21.15.86:0/1949126237 shutdown_connections 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438dee680 0x558438e0eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de3dc0 0x558438e0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 --2- 172.21.15.86:0/1949126237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438de34a0 0x558438de5560 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 -- 172.21.15.86:0/1949126237 >> 172.21.15.86:0/1949126237 conn(0x558438ddfa30 msgr2=0x558438dead40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 -- 172.21.15.86:0/1949126237 shutdown_connections 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.465+0000 7faf862b4740 1 -- 172.21.15.86:0/1949126237 wait complete. 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.466+0000 7faf862b4740 1 -- start start 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.466+0000 7faf862b4740 1 Processor -- start 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.466+0000 7faf862b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.466+0000 7faf862b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438df0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf862b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438df13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf862b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf60011550 con 0x558438dee680 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf862b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf60010e80 con 0x558438de34a0 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf862b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf6000e0e0 con 0x558438de3dc0 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf862b4740 10 client.0 _ll_register_callbacks cb 0x7faf6c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf81476640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438df0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46412/0 (socket says 172.21.15.86:46412) 2024-09-05T07:02:32.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 -- 172.21.15.86:0/1564878177 learned_addr learned my addr 172.21.15.86:0/1564878177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:32.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438df13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 -- 172.21.15.86:0/1564878177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 msgr2=0x558438df0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438df0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 -- 172.21.15.86:0/1564878177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 msgr2=0x558438df13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438df13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 -- 172.21.15.86:0/1564878177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558438e01a50 con 0x558438de34a0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf82478640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faf740095f0 tx=0x7faf7401e4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a29698de103c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.467+0000 7faf657fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7401ea80 con 0x558438de34a0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7faf74005020 con 0x558438de34a0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7401ec20 con 0x558438de34a0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faf7401e680 con 0x558438de34a0 2024-09-05T07:02:32.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf862b4740 10 client.28874 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558438e71750 con 0x558438de34a0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf7401ec20 con 0x558438de34a0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.468+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7faf74004c90 con 0x558438de34a0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 10 client.28874 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 10 client.28874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 20 client.28874 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e71bc0 0x558438e73fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 4 client.28874 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558438d50d60 con 0x558438e71bc0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf81476640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e71bc0 0x558438e73fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf657fa640 10 client.28874 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf81476640 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e71bc0 0x558438e73fb0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85ba42a2f81fe15 server_cookie=c138e3eb7716799d in_seq=0 out_seq=0 2024-09-05T07:02:32.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf657fa640 1 -- 172.21.15.86:0/1564878177 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558438d50d60 con 0x558438e71bc0 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf657fa640 10 client.28874 handle_command_reply: tid=0 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.469+0000 7faf862b4740 1 client.28874 shutdown 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 msgr2=0x558438e03ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faf740095f0 tx=0x7faf7401e4e0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 shutdown_connections 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x558438e71bc0 0x558438e73fb0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558438dee680 0x558438df13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558438de3dc0 0x558438df0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf657fa640 0 client.28874 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 --2- 172.21.15.86:0/1564878177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558438de34a0 0x558438e03ec0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 >> 172.21.15.86:0/1564878177 conn(0x558438ddfa30 msgr2=0x558438dead40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 shutdown_connections 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 1 -- 172.21.15.86:0/1564878177 wait complete. 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.470+0000 7faf862b4740 20 client.28874 trim_cache size 0 max 16384 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:02:32.476 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 msgr2=0x7faf680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 0x7faf680a5230 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf780bc790 tx=0x7faf780095c0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 msgr2=0x7faf7c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf78031040 tx=0x7faf78032ff0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 shutdown_connections 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7faf680a2d70 0x7faf680a5230 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7c106f10 0x7faf7c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf7c102790 0x7faf7c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 --2- 172.21.15.86:0/361470234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf7c101de0 0x7faf7c198410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:32.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.471+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 >> 172.21.15.86:0/361470234 conn(0x7faf7c0fd8e0 msgr2=0x7faf7c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:32.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.473+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 shutdown_connections 2024-09-05T07:02:32.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:32.473+0000 7faf83edb640 1 -- 172.21.15.86:0/361470234 wait complete. 2024-09-05T07:02:32.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57370 6.47 MB/sec execute 350 sec latency 89.367 ms 2024-09-05T07:02:33.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57530 6.47 MB/sec execute 351 sec latency 115.356 ms 2024-09-05T07:02:33.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:02:33.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:02:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:33 smithi096 ceph-mon[34468]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-05T07:02:33.807 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:33 smithi086 ceph-mon[30943]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-05T07:02:33.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:33 smithi115 ceph-mon[34791]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-05T07:02:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 -- 172.21.15.86:0/823336798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174108290 msgr2=0x7f817410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 --2- 172.21.15.86:0/823336798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174108290 0x7f817410ec30 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f816c009cd0 tx=0x7f816c02a8f0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 -- 172.21.15.86:0/823336798 shutdown_connections 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 --2- 172.21.15.86:0/823336798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174108290 0x7f817410ec30 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 --2- 172.21.15.86:0/823336798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174103740 0x7f8174107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 --2- 172.21.15.86:0/823336798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 -- 172.21.15.86:0/823336798 >> 172.21.15.86:0/823336798 conn(0x7f81740fc7d0 msgr2=0x7f81740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 -- 172.21.15.86:0/823336798 shutdown_connections 2024-09-05T07:02:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.664+0000 7f817a1e2640 1 -- 172.21.15.86:0/823336798 wait complete. 2024-09-05T07:02:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.665+0000 7f817a1e2640 1 Processor -- start 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.665+0000 7f817a1e2640 1 -- start start 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 0x7f81741013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8174111f40 con 0x7f8174103740 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8174111dc0 con 0x7f8174102d90 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f817a1e2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f81741120c0 con 0x7f8174108290 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f81737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57858/0 (socket says 172.21.15.86:57858) 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f81737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:46980/0 (socket says 172.21.15.86:46980) 2024-09-05T07:02:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f8172ffd640 1 -- 172.21.15.86:0/2038697453 learned_addr learned my addr 172.21.15.86:0/2038697453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.666+0000 7f8173fff640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 0x7f81741013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 -- 172.21.15.86:0/2038697453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 msgr2=0x7f81741013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 0x7f81741013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 -- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 msgr2=0x7f8174100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 -- 172.21.15.86:0/2038697453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8174072d30 con 0x7f8174102d90 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f8172ffd640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f8173fff640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 0x7f81741013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f81737fe640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400ccc0 con 0x7f8174102d90 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8174073020 con 0x7f8174102d90 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81640108e0 con 0x7f8174102d90 2024-09-05T07:02:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164018880 con 0x7f8174102d90 2024-09-05T07:02:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.667+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81740735b0 con 0x7f8174102d90 2024-09-05T07:02:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.668+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8174103170 con 0x7f8174102d90 2024-09-05T07:02:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.670+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8164010a90 con 0x7f8174102d90 2024-09-05T07:02:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.671+0000 7f8170ff9640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.671+0000 7f8172ffd640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.671+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f81640cfb40 con 0x7f8174102d90 2024-09-05T07:02:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.671+0000 7f8172ffd640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 0x7f815c0a5230 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8174102090 tx=0x7f8168005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.676+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f816409bcd0 con 0x7f8174102d90 2024-09-05T07:02:34.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57659 6.47 MB/sec execute 352 sec latency 161.469 ms 2024-09-05T07:02:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.985+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8174073fd0 con 0x7f8174102d90 2024-09-05T07:02:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.986+0000 7f8170ff9640 1 -- 172.21.15.86:0/2038697453 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11974 (secure 0 0 0) 0x7f816409ffd0 con 0x7f8174102d90 2024-09-05T07:02:34.988 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:02:34.988 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"btime":"2024-09-05T06:59:55:306724+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T06:59:54.497388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:02:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 msgr2=0x7f815c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 0x7f815c0a5230 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8174102090 tx=0x7f8168005f90 comp rx=0 tx=0).stop 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 msgr2=0x7f8174100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 shutdown_connections 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8174108290 0x7f81741013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8174103740 0x7f8174100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 --2- 172.21.15.86:0/2038697453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8174102d90 0x7f8174100920 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:34.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.989+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 >> 172.21.15.86:0/2038697453 conn(0x7f81740fc7d0 msgr2=0x7f8174105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:34.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.990+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 shutdown_connections 2024-09-05T07:02:34.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:34.990+0000 7f817a1e2640 1 -- 172.21.15.86:0/2038697453 wait complete. 2024-09-05T07:02:35.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57754 6.46 MB/sec execute 353 sec latency 210.457 ms 2024-09-05T07:02:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:35 smithi096 ceph-mon[34468]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-05T07:02:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:35 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2038697453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:35 smithi115 ceph-mon[34791]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-05T07:02:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:35 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2038697453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:35.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:35 smithi086 ceph-mon[30943]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-05T07:02:35.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:35 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2038697453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:02:36.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57922 6.48 MB/sec execute 354 sec latency 265.261 ms 2024-09-05T07:02:37.152 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:02:37.568 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:37.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58151 6.51 MB/sec execute 355 sec latency 70.191 ms 2024-09-05T07:02:37.838 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:37 smithi086 ceph-mon[30943]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-09-05T07:02:37.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:37 smithi115 ceph-mon[34791]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-09-05T07:02:38.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:37 smithi096 ceph-mon[34468]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-09-05T07:02:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 -- 172.21.15.86:0/2621470268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278109110 msgr2=0x7f8278110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 --2- 172.21.15.86:0/2621470268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278109110 0x7f8278110c90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f8270009ca0 tx=0x7f827002a8c0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 -- 172.21.15.86:0/2621470268 shutdown_connections 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 --2- 172.21.15.86:0/2621470268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278109110 0x7f8278110c90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 --2- 172.21.15.86:0/2621470268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278104990 0x7f82781089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 --2- 172.21.15.86:0/2621470268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278103fe0 0x7f82781043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 -- 172.21.15.86:0/2621470268 >> 172.21.15.86:0/2621470268 conn(0x7f82780ffa40 msgr2=0x7f8278101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 -- 172.21.15.86:0/2621470268 shutdown_connections 2024-09-05T07:02:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.143+0000 7f82805e7640 1 -- 172.21.15.86:0/2621470268 wait complete. 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 Processor -- start 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 -- start start 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278104990 0x7f827819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278109110 0x7f827819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f827e383640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f827e383640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47008/0 (socket says 172.21.15.86:47008) 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f827e383640 1 -- 172.21.15.86:0/306575354 learned_addr learned my addr 172.21.15.86:0/306575354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8278113fd0 con 0x7f8278104990 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8278113e50 con 0x7f8278103fe0 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8278114150 con 0x7f8278109110 2024-09-05T07:02:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.144+0000 7f827db82640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278104990 0x7f827819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827eb84640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278109110 0x7f827819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 -- 172.21.15.86:0/306575354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278109110 msgr2=0x7f827819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278109110 0x7f827819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 -- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278104990 msgr2=0x7f827819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278104990 0x7f827819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 -- 172.21.15.86:0/306575354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827819f4c0 con 0x7f8278103fe0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f827e383640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f827400e9f0 tx=0x7f827400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f826f7fe640 1 -- 172.21.15.86:0/306575354 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400cd80 con 0x7f8278103fe0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827819f7b0 con 0x7f8278103fe0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f826f7fe640 1 -- 172.21.15.86:0/306575354 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82740108e0 con 0x7f8278103fe0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a7090 con 0x7f8278103fe0 2024-09-05T07:02:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.145+0000 7f826f7fe640 1 -- 172.21.15.86:0/306575354 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274018920 con 0x7f8278103fe0 2024-09-05T07:02:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.148+0000 7f826f7fe640 1 -- 172.21.15.86:0/306575354 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8274010430 con 0x7f8278103fe0 2024-09-05T07:02:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.148+0000 7f82829c0740 1 Processor -- start 2024-09-05T07:02:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.148+0000 7f826f7fe640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 0x7f82640a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.148+0000 7f82829c0740 1 -- start start 2024-09-05T07:02:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f826f7fe640 1 -- 172.21.15.86:0/306575354 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f82740cfab0 con 0x7f8278103fe0 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827eb84640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 0x7f82640a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4ae12f0 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4a88cf0 con 0x55a1a4b7cb40 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f82829c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4add2b0 con 0x55a1a4b86370 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827eb84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827db82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827eb84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b88760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:59614/0 (socket says 172.21.15.86:59614) 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47018/0 (socket says 172.21.15.86:47018) 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827db82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57892/0 (socket says 172.21.15.86:57892) 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827eb84640 1 -- 172.21.15.86:0/2737477254 learned_addr learned my addr 172.21.15.86:0/2737477254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 -- 172.21.15.86:0/2737477254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 msgr2=0x55a1a4b88760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b88760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 -- 172.21.15.86:0/2737477254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 msgr2=0x55a1a4b7ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 -- 172.21.15.86:0/2737477254 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a4a6d3a0 con 0x55a1a4b7cb40 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827eb84640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 0x7f82640a52b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8270009430 tx=0x7f8270031c00 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827db82640 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.149+0000 7f827e383640 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f82740ccd60 tx=0x7f827409b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a823b100e09d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.150+0000 7f826cff9640 1 -- 172.21.15.86:0/2737477254 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827409ce60 con 0x55a1a4b7cb40 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.150+0000 7f826cff9640 1 -- 172.21.15.86:0/2737477254 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f827409bb30 con 0x55a1a4b7cb40 2024-09-05T07:02:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.150+0000 7f826cff9640 1 -- 172.21.15.86:0/2737477254 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274097880 con 0x55a1a4b7cb40 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.150+0000 7f82829c0740 1 -- 172.21.15.86:0/2737477254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 msgr2=0x55a1a4b7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.150+0000 7f82829c0740 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f82740ccd60 tx=0x7f827409b770 comp rx=0 tx=0).stop 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- 172.21.15.86:0/2737477254 shutdown_connections 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b88760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- 172.21.15.86:0/2737477254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7b090 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- 172.21.15.86:0/2737477254 >> 172.21.15.86:0/2737477254 conn(0x55a1a4b44310 msgr2=0x55a1a4b77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- 172.21.15.86:0/2737477254 shutdown_connections 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- 172.21.15.86:0/2737477254 wait complete. 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- start start 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 Processor -- start 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b83030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b83640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.151+0000 7f82829c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4b89070 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f82829c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4a9c060 con 0x55a1a4b7cb40 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f82829c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a1a4b891f0 con 0x55a1a4b86370 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f82829c0740 10 client.0 _ll_register_callbacks cb 0x55a1a47ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57906/0 (socket says 172.21.15.86:57906) 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 -- 172.21.15.86:0/522759473 learned_addr learned my addr 172.21.15.86:0/522759473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827db82640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b83030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827eb84640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b83640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 -- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 msgr2=0x55a1a4b83640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b83640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 -- 172.21.15.86:0/522759473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 msgr2=0x55a1a4b83030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b83030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 -- 172.21.15.86:0/522759473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a48523c0 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f827e383640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f827409b830 tx=0x7f8274010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b7c9a6a152397e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f8261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827409ce60 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827409bb30 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.152+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f8274097880 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f827409fee0 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 10 client.20148 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a1a4b94b70 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f827409bb30 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f8274010a90 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 10 client.20148 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 10 client.20148 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 20 client.20148 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4b95ec0 0x55a1a4b982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 4 client.20148 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.153+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a4aee860 con 0x55a1a4b95ec0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.154+0000 7f827db82640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4b95ec0 0x55a1a4b982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.154+0000 7f827db82640 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4b95ec0 0x55a1a4b982b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56d29057d8bc2118 server_cookie=d71e53d1b3ac1dda in_seq=0 out_seq=0 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.154+0000 7f8261ffb640 10 client.20148 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f8261ffb640 1 -- 172.21.15.86:0/522759473 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1a4aee860 con 0x55a1a4b95ec0 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f8261ffb640 10 client.20148 handle_command_reply: tid=0 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 client.20148 shutdown 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 msgr2=0x55a1a4b85d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f827409b830 tx=0x7f8274010820 comp rx=0 tx=0).stop 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 shutdown_connections 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4b95ec0 0x55a1a4b982b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b86370 0x55a1a4b83640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f8261ffb640 0 client.20148 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b83030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 --2- 172.21.15.86:0/522759473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b85d80 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 >> 172.21.15.86:0/522759473 conn(0x55a1a4b44310 msgr2=0x55a1a4b7d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 shutdown_connections 2024-09-05T07:02:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 1 -- 172.21.15.86:0/522759473 wait complete. 2024-09-05T07:02:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.155+0000 7f82829c0740 20 client.20148 trim_cache size 0 max 16384 2024-09-05T07:02:38.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 Processor -- start 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 -- start start 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b72c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f825c010160 con 0x55a1a4b7cb40 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f825c011ca0 con 0x55a1a4b86370 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f82829c0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f825c010830 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57922/0 (socket says 172.21.15.86:57922) 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827eb84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 -- 172.21.15.86:0/4152329483 learned_addr learned my addr 172.21.15.86:0/4152329483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827e383640 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b72c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 -- 172.21.15.86:0/4152329483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 msgr2=0x55a1a4b72c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b72c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 -- 172.21.15.86:0/4152329483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 msgr2=0x55a1a4b7e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b7e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.178+0000 7f827db82640 1 -- 172.21.15.86:0/4152329483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a4b81fe0 con 0x55a1a4b7cb40 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.179+0000 7f827db82640 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8268000c00 tx=0x7f8268005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708c2025bcf63707 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.179+0000 7f8263fff640 1 -- 172.21.15.86:0/4152329483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268010430 con 0x55a1a4b7cb40 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.179+0000 7f8263fff640 1 -- 172.21.15.86:0/4152329483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8268010a80 con 0x55a1a4b7cb40 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.179+0000 7f8263fff640 1 -- 172.21.15.86:0/4152329483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826801f8f0 con 0x55a1a4b7cb40 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- 172.21.15.86:0/4152329483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 msgr2=0x55a1a4b7e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8268000c00 tx=0x7f8268005f90 comp rx=0 tx=0).stop 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- 172.21.15.86:0/4152329483 shutdown_connections 2024-09-05T07:02:38.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b7cb40 0x55a1a4b7e120 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 --2- 172.21.15.86:0/4152329483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b72c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- 172.21.15.86:0/4152329483 >> 172.21.15.86:0/4152329483 conn(0x55a1a4b44310 msgr2=0x55a1a4b76d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- 172.21.15.86:0/4152329483 shutdown_connections 2024-09-05T07:02:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- 172.21.15.86:0/4152329483 wait complete. 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.180+0000 7f82829c0740 1 -- start start 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 Processor -- start 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b95ec0 0x55a1a4c08010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f827db82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f827db82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47034/0 (socket says 172.21.15.86:47034) 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f827db82640 1 -- 172.21.15.86:0/3950788927 learned_addr learned my addr 172.21.15.86:0/3950788927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f825c0115d0 con 0x55a1a4b95ec0 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f825c010f00 con 0x55a1a4b86370 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f825c00e160 con 0x55a1a4b7b5d0 2024-09-05T07:02:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f827eb84640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b95ec0 0x55a1a4c08010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f82829c0740 10 client.0 _ll_register_callbacks cb 0x7f82680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.181+0000 7f827e383640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 -- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 msgr2=0x55a1a4b99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 -- 172.21.15.86:0/3950788927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b95ec0 msgr2=0x55a1a4c08010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b95ec0 0x55a1a4c08010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a4b839b0 con 0x55a1a4b86370 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.182+0000 7f827db82640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8268025000 tx=0x7f8268005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a6fdd35228a17cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f8261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268003fb0 con 0x55a1a4b86370 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f8268004420 con 0x55a1a4b86370 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8268010a80 con 0x55a1a4b86370 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f826803b050 con 0x55a1a4b86370 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f82829c0740 10 client.28898 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a1a4c096b0 con 0x55a1a4b86370 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.183+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8268010a80 con 0x55a1a4b86370 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f826801f870 con 0x55a1a4b86370 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 10 client.28898 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 10 client.28898 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 20 client.28898 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4c09cb0 0x55a1a4c0c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 4 client.28898 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1a4ae8d70 con 0x55a1a4c09cb0 2024-09-05T07:02:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.184+0000 7f827e383640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4c09cb0 0x55a1a4c0c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.185+0000 7f827e383640 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4c09cb0 0x55a1a4c0c0d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4421271ca8d2594 server_cookie=2a45f3f98411e148 in_seq=0 out_seq=0 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.185+0000 7f8261ffb640 10 client.28898 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f8261ffb640 1 -- 172.21.15.86:0/3950788927 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1a4ae8d70 con 0x55a1a4c09cb0 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f8261ffb640 10 client.28898 handle_command_reply: tid=0 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 client.28898 shutdown 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 msgr2=0x55a1a4b99cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8268025000 tx=0x7f8268005e40 comp rx=0 tx=0).stop 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 shutdown_connections 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55a1a4c09cb0 0x55a1a4c0c0d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1a4b95ec0 0x55a1a4c08010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1a4b86370 0x55a1a4b99cb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 --2- 172.21.15.86:0/3950788927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a1a4b7b5d0 0x55a1a4b99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 >> 172.21.15.86:0/3950788927 conn(0x55a1a4b44310 msgr2=0x55a1a4b84890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f8261ffb640 0 client.28898 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:02:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 shutdown_connections 2024-09-05T07:02:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 1 -- 172.21.15.86:0/3950788927 wait complete. 2024-09-05T07:02:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.186+0000 7f82829c0740 20 client.28898 trim_cache size 0 max 16384 2024-09-05T07:02:38.190 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 msgr2=0x7f82640a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 0x7f82640a52b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8270009430 tx=0x7f8270031c00 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 msgr2=0x7f827819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f827400e9f0 tx=0x7f827400eeb0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 shutdown_connections 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f82640a2df0 0x7f82640a52b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8278109110 0x7f827819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8278104990 0x7f827819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 --2- 172.21.15.86:0/306575354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8278103fe0 0x7f827819a510 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 >> 172.21.15.86:0/306575354 conn(0x7f82780ffa40 msgr2=0x7f8278100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 shutdown_connections 2024-09-05T07:02:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:38.187+0000 7f82805e7640 1 -- 172.21.15.86:0/306575354 wait complete. 2024-09-05T07:02:38.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58344 6.53 MB/sec execute 356 sec latency 104.896 ms 2024-09-05T07:02:38.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:02:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:39 smithi086 ceph-mon[30943]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 354 op/s 2024-09-05T07:02:39.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58471 6.52 MB/sec execute 357 sec latency 246.029 ms 2024-09-05T07:02:39.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:39 smithi115 ceph-mon[34791]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 354 op/s 2024-09-05T07:02:40.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:39 smithi096 ceph-mon[34468]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.0 MiB/s wr, 354 op/s 2024-09-05T07:02:40.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58581 6.52 MB/sec execute 358 sec latency 146.774 ms 2024-09-05T07:02:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:41 smithi086 ceph-mon[30943]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2024-09-05T07:02:41.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58697 6.53 MB/sec execute 359 sec latency 151.820 ms 2024-09-05T07:02:41.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:41 smithi115 ceph-mon[34791]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2024-09-05T07:02:42.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:41 smithi096 ceph-mon[34468]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 225 op/s 2024-09-05T07:02:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 -- 172.21.15.86:0/3772500848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 msgr2=0x7fce10100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 --2- 172.21.15.86:0/3772500848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce10100f00 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fce00009860 tx=0x7fce0002a3b0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 -- 172.21.15.86:0/3772500848 shutdown_connections 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 --2- 172.21.15.86:0/3772500848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce101063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 --2- 172.21.15.86:0/3772500848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce10105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 --2- 172.21.15.86:0/3772500848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce10100f00 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 -- 172.21.15.86:0/3772500848 >> 172.21.15.86:0/3772500848 conn(0x7fce100fc7d0 msgr2=0x7fce100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 -- 172.21.15.86:0/3772500848 shutdown_connections 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.567+0000 7fce1611b640 1 -- 172.21.15.86:0/3772500848 wait complete. 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 Processor -- start 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 -- start start 2024-09-05T07:02:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce101a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce1019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce101183f0 con 0x7fce10100b20 2024-09-05T07:02:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce10118270 con 0x7fce10101440 2024-09-05T07:02:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce1611b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce10118570 con 0x7fce10105f30 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57958/0 (socket says 172.21.15.86:57958) 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce0f7fe640 1 -- 172.21.15.86:0/955284096 learned_addr learned my addr 172.21.15.86:0/955284096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce0effd640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce101a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.568+0000 7fce0ffff640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce1019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 -- 172.21.15.86:0/955284096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 msgr2=0x7fce101a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce101a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 -- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 msgr2=0x7fce101a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 -- 172.21.15.86:0/955284096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1019ef60 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0effd640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce101a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0ffff640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce1019e7c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fce08009fd0 tx=0x7fce0800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0cff9640 1 -- 172.21.15.86:0/955284096 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0800ce20 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0cff9640 1 -- 172.21.15.86:0/955284096 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce08005020 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1019f250 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce0cff9640 1 -- 172.21.15.86:0/955284096 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08010670 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.569+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce101ab4a0 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.573+0000 7fce0cff9640 1 -- 172.21.15.86:0/955284096 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce08004b20 con 0x7fce10105f30 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 Processor -- start 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 -- start start 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f418090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f425760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e55f37e2f0 con 0x55e55f4185d0 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e55f325cf0 con 0x55e55f423370 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce184f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e55f37a2b0 con 0x55e55f419b40 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57974/0 (socket says 172.21.15.86:57974) 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3677277204 learned_addr learned my addr 172.21.15.86:0/3677277204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.574+0000 7fce0effd640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f418090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0effd640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f425760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3677277204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 msgr2=0x55e55f418090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f418090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3677277204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 msgr2=0x55e55f425760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f425760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3677277204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55f30a3a0 con 0x55e55f4185d0 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0cff9640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 0x7fcdf80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0effd640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f418090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fce00009980 tx=0x7fce000095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741a82f5a0af3acd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0cff9640 1 -- 172.21.15.86:0/955284096 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fce080cff60 con 0x7fce10105f30 2024-09-05T07:02:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fcdf5ffb640 1 -- 172.21.15.86:0/3677277204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00037040 con 0x55e55f4185d0 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fce0effd640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 0x7fcdf80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fcdf5ffb640 1 -- 172.21.15.86:0/3677277204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce00032ca0 con 0x55e55f4185d0 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.575+0000 7fcdf5ffb640 1 -- 172.21.15.86:0/3677277204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000039e0 con 0x55e55f4185d0 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.576+0000 7fce0effd640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 0x7fcdf80a51b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55e55f419070 tx=0x7fce04004730 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.576+0000 7fce184f4740 1 -- 172.21.15.86:0/3677277204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 msgr2=0x55e55f418a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.576+0000 7fce184f4740 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fce00009980 tx=0x7fce000095a0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 -- 172.21.15.86:0/3677277204 shutdown_connections 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f425760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f418a10 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 --2- 172.21.15.86:0/3677277204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f418090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 -- 172.21.15.86:0/3677277204 >> 172.21.15.86:0/3677277204 conn(0x55e55f3e1310 msgr2=0x55e55f414c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 -- 172.21.15.86:0/3677277204 shutdown_connections 2024-09-05T07:02:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.577+0000 7fce184f4740 1 -- 172.21.15.86:0/3677277204 wait complete. 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 -- start start 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 Processor -- start 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f41ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e55f426010 con 0x55e55f423370 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e55f339060 con 0x55e55f4185d0 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e55f426190 con 0x55e55f419b40 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.578+0000 7fce184f4740 10 client.0 _ll_register_callbacks cb 0x55e55f08c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47082/0 (socket says 172.21.15.86:47082) 2024-09-05T07:02:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57976/0 (socket says 172.21.15.86:57976) 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0f7fe640 1 -- 172.21.15.86:0/1087592329 learned_addr learned my addr 172.21.15.86:0/1087592329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0effd640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f41ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 -- 172.21.15.86:0/1087592329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 msgr2=0x55e55f41ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f41ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 -- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 msgr2=0x55e55f422d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 -- 172.21.15.86:0/1087592329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55f0ef3c0 con 0x55e55f423370 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce0ffff640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fce080cd210 tx=0x7fce08004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32aa44bbc666539 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fcdeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce080b4020 con 0x55e55f423370 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fce080cd490 con 0x55e55f423370 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.579+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce00032ca0 con 0x55e55f423370 2024-09-05T07:02:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fce080a0390 con 0x55e55f423370 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 10 client.20166 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e55f431ac0 con 0x55e55f423370 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce00032ca0 con 0x55e55f423370 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fce08097d30 con 0x55e55f423370 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 10 client.20166 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 10 client.20166 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 20 client.20166 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f435900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 4 client.20166 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55f38b860 con 0x55e55f433490 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.580+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f435900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.581+0000 7fcdeeffd640 10 client.20166 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:02:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.581+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f435900 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5379861450be9ba6 server_cookie=b4df023a669e202 in_seq=0 out_seq=0 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fcdeeffd640 1 -- 172.21.15.86:0/1087592329 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e55f38b860 con 0x55e55f433490 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fcdeeffd640 10 client.20166 handle_command_reply: tid=0 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 client.20166 shutdown 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 msgr2=0x55e55f420570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fce080cd210 tx=0x7fce08004ed0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 shutdown_connections 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f435900 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f423370 0x55e55f420570 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f41ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fcdeeffd640 0 client.20166 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 --2- 172.21.15.86:0/1087592329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f422d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 >> 172.21.15.86:0/1087592329 conn(0x55e55f3e1310 msgr2=0x55e55f41a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 shutdown_connections 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 1 -- 172.21.15.86:0/1087592329 wait complete. 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.582+0000 7fce184f4740 20 client.20166 trim_cache size 0 max 16384 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.603+0000 7fce184f4740 1 Processor -- start 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.603+0000 7fce184f4740 1 -- start start 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f424760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f424ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce101aceb0 con 0x55e55f4185d0 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ae9f0 con 0x55e55f423370 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce184f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ad580 con 0x55e55f419b40 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f424760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57990/0 (socket says 172.21.15.86:57990) 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3467838973 learned_addr learned my addr 172.21.15.86:0/3467838973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0ffff640 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f424ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3467838973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 msgr2=0x55e55f424760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f424760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3467838973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 msgr2=0x55e55f424ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f424ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 -- 172.21.15.86:0/3467838973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55f4193a0 con 0x55e55f4185d0 2024-09-05T07:02:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.604+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fce00000c00 tx=0x7fce00005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fedf84580d6ff3c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.605+0000 7fcdf57fa640 1 -- 172.21.15.86:0/3467838973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00056070 con 0x55e55f4185d0 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.605+0000 7fcdf57fa640 1 -- 172.21.15.86:0/3467838973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce00032e50 con 0x55e55f4185d0 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.605+0000 7fcdf57fa640 1 -- 172.21.15.86:0/3467838973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00037040 con 0x55e55f4185d0 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- 172.21.15.86:0/3467838973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 msgr2=0x55e55f42f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fce00000c00 tx=0x7fce00005780 comp rx=0 tx=0).stop 2024-09-05T07:02:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- 172.21.15.86:0/3467838973 shutdown_connections 2024-09-05T07:02:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f423370 0x55e55f424ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f419b40 0x55e55f424760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 --2- 172.21.15.86:0/3467838973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f4185d0 0x55e55f42f2b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- 172.21.15.86:0/3467838973 >> 172.21.15.86:0/3467838973 conn(0x55e55f3e1310 msgr2=0x55e55f41e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- 172.21.15.86:0/3467838973 shutdown_connections 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- 172.21.15.86:0/3467838973 wait complete. 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 -- start start 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.606+0000 7fce184f4740 1 Processor -- start 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f43a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f43a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 0x55e55f43ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f43a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:47102/0 (socket says 172.21.15.86:47102) 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce101ae320 con 0x55e55f419b40 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce101adc50 con 0x55e55f4185d0 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce10105910 con 0x55e55f423370 2024-09-05T07:02:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58004/0 (socket says 172.21.15.86:58004) 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 -- 172.21.15.86:0/2630497559 learned_addr learned my addr 172.21.15.86:0/2630497559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce184f4740 10 client.0 _ll_register_callbacks cb 0x7fce08097b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0ffff640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 0x55e55f43ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 -- 172.21.15.86:0/2630497559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 msgr2=0x55e55f43ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 0x55e55f43ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 -- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 msgr2=0x55e55f43a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f43a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0effd640 1 -- 172.21.15.86:0/2630497559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55f426180 con 0x55e55f419b40 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.607+0000 7fce0ffff640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 0x55e55f43ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fce0effd640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fce04005fd0 tx=0x7fce0400bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3b6631ba7af344 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fcdeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0403d420 con 0x55e55f419b40 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fce04032420 con 0x55e55f419b40 2024-09-05T07:02:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce00032e50 con 0x55e55f419b40 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fce04032620 con 0x55e55f419b40 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fce184f4740 10 client.20172 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.608+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e55f42b6f0 con 0x55e55f419b40 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce00032e50 con 0x55e55f419b40 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fce0403c840 con 0x55e55f419b40 2024-09-05T07:02:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 10 client.20172 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 10 client.20172 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 20 client.20172 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f4a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 4 client.20172 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e55f385d70 con 0x55e55f433490 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f4a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fce0f7fe640 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f4a01f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de0879a804b796cd server_cookie=24af6b4f5e9f1ac3 in_seq=0 out_seq=0 2024-09-05T07:02:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.609+0000 7fcdeeffd640 10 client.20172 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:02:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fcdeeffd640 1 -- 172.21.15.86:0/2630497559 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e55f385d70 con 0x55e55f433490 2024-09-05T07:02:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fcdeeffd640 10 client.20172 handle_command_reply: tid=0 2024-09-05T07:02:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 client.20172 shutdown 2024-09-05T07:02:42.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 msgr2=0x55e55f43a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fce04005fd0 tx=0x7fce0400bad0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.420 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 shutdown_connections 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55e55f433490 0x55e55f4a01f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e55f423370 0x55e55f43ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55f419b40 0x55e55f43a740 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 --2- 172.21.15.86:0/2630497559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e55f4185d0 0x55e55f43a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fcdeeffd640 0 client.20172 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 >> 172.21.15.86:0/2630497559 conn(0x55e55f3e1310 msgr2=0x55e55f421830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 shutdown_connections 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.610+0000 7fce184f4740 1 -- 172.21.15.86:0/2630497559 wait complete. 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.611+0000 7fce184f4740 20 client.20172 trim_cache size 0 max 16384 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.611+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 msgr2=0x7fcdf80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.611+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 0x7fcdf80a51b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55e55f419070 tx=0x7fce04004730 comp rx=0 tx=0).stop 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.611+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 msgr2=0x7fce1019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.611+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce1019e7c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fce08009fd0 tx=0x7fce0800ef10 comp rx=0 tx=0).stop 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 shutdown_connections 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fcdf80a2cf0 0x7fcdf80a51b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce10105f30 0x7fce1019e7c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce10101440 0x7fce101a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 --2- 172.21.15.86:0/955284096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce10100b20 0x7fce101a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 >> 172.21.15.86:0/955284096 conn(0x7fce100fc7d0 msgr2=0x7fce1010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 shutdown_connections 2024-09-05T07:02:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:41.612+0000 7fce1611b640 1 -- 172.21.15.86:0/955284096 wait complete. 2024-09-05T07:02:42.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58873 6.55 MB/sec execute 360 sec latency 84.875 ms 2024-09-05T07:02:43.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59017 6.56 MB/sec execute 361 sec latency 114.360 ms 2024-09-05T07:02:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:43 smithi115 ceph-mon[34791]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 301 op/s 2024-09-05T07:02:43.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:43 smithi086 ceph-mon[30943]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 301 op/s 2024-09-05T07:02:44.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:43 smithi096 ceph-mon[34468]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 301 op/s 2024-09-05T07:02:44.773 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59168 6.56 MB/sec execute 362 sec latency 77.958 ms 2024-09-05T07:02:45.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59325 6.56 MB/sec execute 363 sec latency 72.433 ms 2024-09-05T07:02:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:45 smithi115 ceph-mon[34791]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-09-05T07:02:45.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:45 smithi086 ceph-mon[30943]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-09-05T07:02:46.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:45 smithi096 ceph-mon[34468]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-09-05T07:02:46.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59447 6.55 MB/sec execute 364 sec latency 107.352 ms 2024-09-05T07:02:47.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59556 6.53 MB/sec execute 365 sec latency 142.556 ms 2024-09-05T07:02:47.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:47 smithi086 ceph-mon[30943]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-05T07:02:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:47 smithi115 ceph-mon[34791]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-05T07:02:47.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:02:48.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:47 smithi096 ceph-mon[34468]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-05T07:02:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:48.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59688 6.53 MB/sec execute 366 sec latency 140.599 ms 2024-09-05T07:02:49.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59848 6.54 MB/sec execute 367 sec latency 95.013 ms 2024-09-05T07:02:49.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:49 smithi115 ceph-mon[34791]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 291 op/s 2024-09-05T07:02:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:49 smithi086 ceph-mon[30943]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 291 op/s 2024-09-05T07:02:50.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:49 smithi096 ceph-mon[34468]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.1 MiB/s wr, 291 op/s 2024-09-05T07:02:50.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60017 6.54 MB/sec execute 368 sec latency 99.145 ms 2024-09-05T07:02:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:51 smithi086 ceph-mon[30943]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-05T07:02:51.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60174 6.56 MB/sec execute 369 sec latency 111.883 ms 2024-09-05T07:02:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:51 smithi115 ceph-mon[34791]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-05T07:02:52.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:51 smithi096 ceph-mon[34468]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.564+0000 7f7f74560640 1 -- 172.21.15.86:0/1010777938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 msgr2=0x7f7f6c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.564+0000 7f7f74560640 1 --2- 172.21.15.86:0/1010777938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c100ec0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009930 tx=0x7f7f5c02a420 comp rx=0 tx=0).stop 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 -- 172.21.15.86:0/1010777938 shutdown_connections 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 --2- 172.21.15.86:0/1010777938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 --2- 172.21.15.86:0/1010777938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c100ec0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 --2- 172.21.15.86:0/1010777938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 -- 172.21.15.86:0/1010777938 >> 172.21.15.86:0/1010777938 conn(0x7f7f6c0fc780 msgr2=0x7f7f6c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 -- 172.21.15.86:0/1010777938 shutdown_connections 2024-09-05T07:02:52.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 -- 172.21.15.86:0/1010777938 wait complete. 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.565+0000 7f7f74560640 1 Processor -- start 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 -- start start 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111e30 con 0x7f7f6c100a60 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111cb0 con 0x7f7f6c106a60 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f74560640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111fb0 con 0x7f7f6c101400 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50142/0 (socket says 172.21.15.86:50142) 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 -- 172.21.15.86:0/2096994995 learned_addr learned my addr 172.21.15.86:0/2096994995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 -- 172.21.15.86:0/2096994995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 msgr2=0x7f7f6c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 -- 172.21.15.86:0/2096994995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 msgr2=0x7f7f6c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f722fc640 1 -- 172.21.15.86:0/2096994995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c19d360 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.566+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f722fc640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800d950 tx=0x7f7f6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f637fe640 1 -- 172.21.15.86:0/2096994995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68017070 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f637fe640 1 -- 172.21.15.86:0/2096994995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f68009c30 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f637fe640 1 -- 172.21.15.86:0/2096994995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800f620 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c19d650 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.567+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c19da80 con 0x7f7f6c100a60 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.570+0000 7f7f76939740 1 Processor -- start 2024-09-05T07:02:52.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 -- start start 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f637fe640 1 -- 172.21.15.86:0/2096994995 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f680109a0 con 0x7f7f6c100a60 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 0x55ac1479c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479c5d0 0x55ac1479ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac147022f0 con 0x55ac147a7370 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac146a9cf0 con 0x55ac1479c5d0 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f76939740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac146fe2b0 con 0x55ac1479db40 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f71afb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479c5d0 0x55ac1479ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f72afd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 0x55ac1479c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f71afb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50154/0 (socket says 172.21.15.86:50154) 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.571+0000 7f7f71afb640 1 -- 172.21.15.86:0/2624178994 learned_addr learned my addr 172.21.15.86:0/2624178994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:52.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 -- 172.21.15.86:0/2624178994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 msgr2=0x55ac1479c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 0x55ac1479c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 -- 172.21.15.86:0/2624178994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479c5d0 msgr2=0x55ac1479ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479c5d0 0x55ac1479ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 -- 172.21.15.86:0/2624178994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1468e3a0 con 0x55ac147a7370 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 0x55ac1479c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006010 tx=0x7f7f5c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee366bb734dd8e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f637fe640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 0x7f7f580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f637fe640 1 -- 172.21.15.86:0/2096994995 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f7f680cf470 con 0x7f7f6c100a60 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.572+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 0x7f7f580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.573+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 0x7f7f580a5230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f64009500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f60ff9640 1 -- 172.21.15.86:0/2624178994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c03f070 con 0x55ac147a7370 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f60ff9640 1 -- 172.21.15.86:0/2624178994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c004030 con 0x55ac147a7370 2024-09-05T07:02:52.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f60ff9640 1 -- 172.21.15.86:0/2624178994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c035070 con 0x55ac147a7370 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 -- 172.21.15.86:0/2624178994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 msgr2=0x55ac147a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006010 tx=0x7f7f5c032c10 comp rx=0 tx=0).stop 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 -- 172.21.15.86:0/2624178994 shutdown_connections 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147a9760 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479c5d0 0x55ac1479ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.575+0000 7f7f76939740 1 --2- 172.21.15.86:0/2624178994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479db40 0x55ac1479c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 -- 172.21.15.86:0/2624178994 >> 172.21.15.86:0/2624178994 conn(0x55ac14765310 msgr2=0x55ac14798c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 -- 172.21.15.86:0/2624178994 shutdown_connections 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 -- 172.21.15.86:0/2624178994 wait complete. 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 -- start start 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 Processor -- start 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac147a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.576+0000 7f7f76939740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f76939740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f76939740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac147aa010 con 0x55ac1479c5d0 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f76939740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac146bd060 con 0x55ac1479db40 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f76939740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac147aa190 con 0x55ac147a7370 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac147a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f72afd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f71afb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f72afd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56390/0 (socket says 172.21.15.86:56390) 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f72afd640 1 -- 172.21.15.86:0/3608489050 learned_addr learned my addr 172.21.15.86:0/3608489050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f76939740 10 client.0 _ll_register_callbacks cb 0x55ac144108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 -- 172.21.15.86:0/3608489050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 msgr2=0x55ac147a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 -- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 msgr2=0x55ac147a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 -- 172.21.15.86:0/3608489050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac144733c0 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f72afd640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f722fc640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac147a6d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7f680cc780 tx=0x7f7f680d2b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4590355de6ff937a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.577+0000 7f7f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68020070 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f7f6809a9a0 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c004030 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7f6809aeb0 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f76939740 10 client.20184 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ac147b5a50 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c004030 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.578+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f7f680cf120 con 0x55ac1479c5d0 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 10 client.20184 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:52.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 10 client.20184 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 20 client.20184 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac147b6f10 0x55ac147b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 4 client.20184 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1470f860 con 0x55ac147b6f10 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f71afb640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac147b6f10 0x55ac147b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f71afb640 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac147b6f10 0x55ac147b9380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ad6598951c84ff6 server_cookie=d6107aca4f9c8a1a in_seq=0 out_seq=0 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.579+0000 7f7f55ffb640 10 client.20184 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.580+0000 7f7f55ffb640 1 -- 172.21.15.86:0/3608489050 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac1470f860 con 0x55ac147b6f10 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.580+0000 7f7f55ffb640 10 client.20184 handle_command_reply: tid=0 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.580+0000 7f7f76939740 1 client.20184 shutdown 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.580+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 msgr2=0x55ac147a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac147a6d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7f680cc780 tx=0x7f7f680d2b10 comp rx=0 tx=0).stop 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 shutdown_connections 2024-09-05T07:02:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac147b6f10 0x55ac147b9380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 --2- 172.21.15.86:0/3608489050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac147a6d30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 >> 172.21.15.86:0/3608489050 conn(0x55ac14765310 msgr2=0x55ac1479e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f55ffb640 0 client.20184 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 shutdown_connections 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 1 -- 172.21.15.86:0/3608489050 wait complete. 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.581+0000 7f7f76939740 20 client.20184 trim_cache size 0 max 16384 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 Processor -- start 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 -- start start 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ad040 con 0x55ac1479c5d0 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1aeb80 con 0x55ac1479db40 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f76939740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ad710 con 0x55ac147a7370 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50176/0 (socket says 172.21.15.86:50176) 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f722fc640 1 -- 172.21.15.86:0/2663047395 learned_addr learned my addr 172.21.15.86:0/2663047395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.602+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f72afd640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 -- 172.21.15.86:0/2663047395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 msgr2=0x55ac147a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 -- 172.21.15.86:0/2663047395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 msgr2=0x55ac14798e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 -- 172.21.15.86:0/2663047395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1479f120 con 0x55ac1479db40 2024-09-05T07:02:52.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f722fc640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f71afb640 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147b32b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a940 tx=0x7f7f5c032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5b381daa190638 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.603+0000 7f7f57fff640 1 -- 172.21.15.86:0/2663047395 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c035040 con 0x55ac1479db40 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f57fff640 1 -- 172.21.15.86:0/2663047395 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c0054f0 con 0x55ac1479db40 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f57fff640 1 -- 172.21.15.86:0/2663047395 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c03f040 con 0x55ac1479db40 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 -- 172.21.15.86:0/2663047395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 msgr2=0x55ac147b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147b32b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a940 tx=0x7f7f5c032c10 comp rx=0 tx=0).stop 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 -- 172.21.15.86:0/2663047395 shutdown_connections 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac147a7370 0x55ac147a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147b32b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 --2- 172.21.15.86:0/2663047395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac1479c5d0 0x55ac14798e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 -- 172.21.15.86:0/2663047395 >> 172.21.15.86:0/2663047395 conn(0x55ac14765310 msgr2=0x55ac14798700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.604+0000 7f7f76939740 1 -- 172.21.15.86:0/2663047395 shutdown_connections 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 -- 172.21.15.86:0/2663047395 wait complete. 2024-09-05T07:02:52.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 -- start start 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 Processor -- start 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147aa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae4b0 con 0x55ac147a7370 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1adde0 con 0x55ac1479db40 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ab040 con 0x55ac1479c5d0 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f76939740 10 client.0 _ll_register_callbacks cb 0x7f7f6800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f722fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56398/0 (socket says 172.21.15.86:56398) 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f722fc640 1 -- 172.21.15.86:0/1149376979 learned_addr learned my addr 172.21.15.86:0/1149376979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.605+0000 7f7f71afb640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f72afd640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 -- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 msgr2=0x55ac147aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 -- 172.21.15.86:0/1149376979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 msgr2=0x55ac147aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 -- 172.21.15.86:0/1149376979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1481fe00 con 0x55ac1479c5d0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f722fc640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000c00 tx=0x7f7f680cf330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c720dca960949b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f680b4060 con 0x55ac1479c5d0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f7f680974b0 con 0x55ac1479c5d0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c0054f0 con 0x55ac1479c5d0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7f68014070 con 0x55ac1479c5d0 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f76939740 10 client.28957 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:52.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.606+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ac148210e0 con 0x55ac1479c5d0 2024-09-05T07:02:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f68009de0 con 0x55ac1479c5d0 2024-09-05T07:02:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f7f68010460 con 0x55ac1479c5d0 2024-09-05T07:02:52.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 10 client.28957 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 10 client.28957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 20 client.28957 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac148216e0 0x55ac14823b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 4 client.28957 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7f5c0054f0 con 0x55ac148216e0 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.607+0000 7f7f71afb640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac148216e0 0x55ac14823b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f55ffb640 10 client.28957 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f71afb640 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac148216e0 0x55ac14823b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b25b783a5cf9748d server_cookie=8dec184d56b10c06 in_seq=0 out_seq=0 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f55ffb640 1 -- 172.21.15.86:0/1149376979 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7f5c0054f0 con 0x55ac148216e0 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f55ffb640 10 client.28957 handle_command_reply: tid=0 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 client.28957 shutdown 2024-09-05T07:02:52.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 msgr2=0x55ac147aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000c00 tx=0x7f7f680cf330 comp rx=0 tx=0).stop 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 shutdown_connections 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55ac148216e0 0x55ac14823b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac147a7370 0x55ac147aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1479db40 0x55ac147aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 --2- 172.21.15.86:0/1149376979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac1479c5d0 0x55ac147aa3e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 >> 172.21.15.86:0/1149376979 conn(0x55ac14765310 msgr2=0x55ac147a3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f55ffb640 0 client.28957 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 shutdown_connections 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 1 -- 172.21.15.86:0/1149376979 wait complete. 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.608+0000 7f7f76939740 20 client.28957 trim_cache size 0 max 16384 2024-09-05T07:02:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 msgr2=0x7f7f580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 0x7f7f580a5230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f64009500 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 msgr2=0x7f7f6c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800d950 tx=0x7f7f6800de10 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 shutdown_connections 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7f580a2d70 0x7f7f580a5230 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f6c106a60 0x7f7f6c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f6c101400 0x7f7f6c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 --2- 172.21.15.86:0/2096994995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c100a60 0x7f7f6c1983b0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 >> 172.21.15.86:0/2096994995 conn(0x7f7f6c0fc780 msgr2=0x7f7f6c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 shutdown_connections 2024-09-05T07:02:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:51.609+0000 7f7f74560640 1 -- 172.21.15.86:0/2096994995 wait complete. 2024-09-05T07:02:52.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60302 6.56 MB/sec execute 370 sec latency 206.983 ms 2024-09-05T07:02:53.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60439 6.57 MB/sec execute 371 sec latency 127.515 ms 2024-09-05T07:02:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:53 smithi115 ceph-mon[34791]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 311 op/s 2024-09-05T07:02:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:53 smithi086 ceph-mon[30943]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 311 op/s 2024-09-05T07:02:54.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:53 smithi096 ceph-mon[34468]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 311 op/s 2024-09-05T07:02:54.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60588 6.56 MB/sec execute 372 sec latency 110.616 ms 2024-09-05T07:02:55.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:02:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:55.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60753 6.57 MB/sec execute 373 sec latency 90.443 ms 2024-09-05T07:02:55.815 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:55 smithi086 ceph-mon[30943]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 247 op/s 2024-09-05T07:02:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:55 smithi115 ceph-mon[34791]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 247 op/s 2024-09-05T07:02:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/3470889386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04100a70 msgr2=0x7f3c04100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/3470889386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04100a70 0x7f3c04100ed0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009930 tx=0x7f3bf402a420 comp rx=0 tx=0).stop 2024-09-05T07:02:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/3470889386 shutdown_connections 2024-09-05T07:02:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/3470889386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04101410 0x7f3c0410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/3470889386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04100a70 0x7f3c04100ed0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/3470889386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04106a70 0x7f3c04106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.936+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/3470889386 >> 172.21.15.86:0/3470889386 conn(0x7f3c040fc790 msgr2=0x7f3c040febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.937+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/3470889386 shutdown_connections 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.937+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/3470889386 wait complete. 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.937+0000 7f3c0c7a4640 1 Processor -- start 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.937+0000 7f3c0c7a4640 1 -- start start 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04100a70 0x7f3c04196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 0x7f3c0419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0410fc70 con 0x7f3c04106a70 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0410faf0 con 0x7f3c04101410 2024-09-05T07:02:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0c7a4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0410fdf0 con 0x7f3c04100a70 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0a540640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04100a70 0x7f3c04196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44706/0 (socket says 172.21.15.86:44706) 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1122304589 learned_addr learned my addr 172.21.15.86:0/1122304589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 0x7f3c0419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1122304589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04100a70 msgr2=0x7f3c04196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04100a70 0x7f3c04196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 msgr2=0x7f3c0419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 0x7f3c0419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1122304589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0419b140 con 0x7f3c04101410 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 0x7f3c0419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.938+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4005ab0 tx=0x7f3bf4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.939+0000 7f3bfb7fe640 1 -- 172.21.15.86:0/1122304589 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf403f070 con 0x7f3c04101410 2024-09-05T07:02:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.939+0000 7f3bfb7fe640 1 -- 172.21.15.86:0/1122304589 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bf4031b50 con 0x7f3c04101410 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.939+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0419b3d0 con 0x7f3c04101410 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.939+0000 7f3bfb7fe640 1 -- 172.21.15.86:0/1122304589 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4035030 con 0x7f3c04101410 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.939+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c041a2d10 con 0x7f3c04101410 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.942+0000 7f3bfb7fe640 1 -- 172.21.15.86:0/1122304589 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3bf4042070 con 0x7f3c04101410 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.943+0000 7f3c0eb7d740 1 Processor -- start 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 -- start start 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3bfb7fe640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 0x7f3bf00a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82eb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec82e1e2f0 con 0x55ec82eb85d0 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec82dc5cf0 con 0x55ec82eb9b40 2024-09-05T07:02:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ec82e1a2b0 con 0x55ec82ec3370 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0ad41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0ad41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44710/0 (socket says 172.21.15.86:44710) 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3c0ad41640 1 -- 172.21.15.86:0/886750742 learned_addr learned my addr 172.21.15.86:0/886750742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.944+0000 7f3bfb7fe640 1 -- 172.21.15.86:0/1122304589 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f3bf40efb50 con 0x7f3c04101410 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 0x7f3bf00a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82eb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 -- 172.21.15.86:0/886750742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 msgr2=0x55ec82ec5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 -- 172.21.15.86:0/886750742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 msgr2=0x55ec82eb8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82eb8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 -- 172.21.15.86:0/886750742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec82daa3a0 con 0x55ec82eb9b40 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82eb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0099e0 tx=0x7f3bfc011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afda8b45aa7e17c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 0x7f3bf00a96c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005d00 tx=0x7f3c0000b410 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3bf8ff9640 1 -- 172.21.15.86:0/886750742 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00fd10 con 0x55ec82eb9b40 2024-09-05T07:02:55.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3bf8ff9640 1 -- 172.21.15.86:0/886750742 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc003e20 con 0x55ec82eb9b40 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.945+0000 7f3bf8ff9640 1 -- 172.21.15.86:0/886750742 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004770 con 0x55ec82eb9b40 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.946+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/886750742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 msgr2=0x55ec82eb8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.946+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0099e0 tx=0x7f3bfc011440 comp rx=0 tx=0).stop 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.946+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/886750742 shutdown_connections 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82eb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/886750742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82eb8090 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/886750742 >> 172.21.15.86:0/886750742 conn(0x55ec82e81310 msgr2=0x55ec82eb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/886750742 shutdown_connections 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/886750742 wait complete. 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 -- start start 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.947+0000 7f3c0eb7d740 1 Processor -- start 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ec2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec82ec6070 con 0x55ec82eb85d0 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec82dd9060 con 0x55ec82eb9b40 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ec82ec61f0 con 0x55ec82ec3370 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0eb7d740 10 client.0 _ll_register_callbacks cb 0x55ec82b2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44722/0 (socket says 172.21.15.86:44722) 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1153457004 learned_addr learned my addr 172.21.15.86:0/1153457004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ec2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 msgr2=0x55ec82ec0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1153457004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 msgr2=0x55ec82ec2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ec2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 -- 172.21.15.86:0/1153457004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec82b8f3c0 con 0x55ec82eb9b40 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ec2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000c00 tx=0x7f3bf40433b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9280d0d1c41a3486 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.948+0000 7f3bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4049040 con 0x55ec82eb9b40 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f3bf40b82f0 con 0x55ec82eb9b40 2024-09-05T07:02:55.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc003e20 con 0x55ec82eb9b40 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3bf40c4840 con 0x55ec82eb9b40 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 10 client.28946 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ec82eda350 con 0x55ec82eb9b40 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc003e20 con 0x55ec82eb9b40 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f3bf40c43b0 con 0x55ec82eb9b40 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 10 client.28946 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 10 client.28946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 20 client.28946 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82edb740 0x55ec82eddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 4 client.28946 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.949+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec82e2b860 con 0x55ec82edb740 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.950+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82edb740 0x55ec82eddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.950+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82edb740 0x55ec82eddb90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a772d18e287937f server_cookie=5ba625510320a105 in_seq=0 out_seq=0 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.950+0000 7f3bedffb640 10 client.28946 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3bedffb640 1 -- 172.21.15.86:0/1153457004 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec82e2b860 con 0x55ec82edb740 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3bedffb640 10 client.28946 handle_command_reply: tid=0 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 client.28946 shutdown 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 msgr2=0x55ec82ec0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000c00 tx=0x7f3bf40433b0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 shutdown_connections 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82edb740 0x55ec82eddb90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82ec0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ec0040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3bedffb640 0 client.28946 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1153457004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ec2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 >> 172.21.15.86:0/1153457004 conn(0x55ec82e81310 msgr2=0x55ec82eba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 shutdown_connections 2024-09-05T07:02:55.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1153457004 wait complete. 2024-09-05T07:02:55.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.951+0000 7f3c0eb7d740 20 client.28946 trim_cache size 0 max 16384 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.969+0000 7f3c0eb7d740 1 Processor -- start 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.969+0000 7f3c0eb7d740 1 -- start start 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82eb9b40 0x55ec82ee3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 0x55ec82ee3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041ad020 con 0x55ec82eb85d0 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041aeb60 con 0x55ec82ec3370 2024-09-05T07:02:55.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041ad6f0 con 0x55ec82eb9b40 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0a540640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0a540640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50844/0 (socket says 172.21.15.86:50844) 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0a540640 1 -- 172.21.15.86:0/1887040623 learned_addr learned my addr 172.21.15.86:0/1887040623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 0x55ec82ee3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82eb9b40 0x55ec82ee3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 -- 172.21.15.86:0/1887040623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82eb9b40 msgr2=0x55ec82ee3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82eb9b40 0x55ec82ee3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 -- 172.21.15.86:0/1887040623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 msgr2=0x55ec82ed7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0ad41640 1 -- 172.21.15.86:0/1887040623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec82ec0980 con 0x55ec82ec3370 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.970+0000 7f3c0a540640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.971+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 0x55ec82ee3970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00fc20 tx=0x7f3bfc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68074407ca16489 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.971+0000 7f3beffff640 1 -- 172.21.15.86:0/1887040623 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003970 con 0x55ec82ec3370 2024-09-05T07:02:55.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.971+0000 7f3beffff640 1 -- 172.21.15.86:0/1887040623 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc003fc0 con 0x55ec82ec3370 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.971+0000 7f3beffff640 1 -- 172.21.15.86:0/1887040623 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc005730 con 0x55ec82ec3370 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1887040623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 msgr2=0x55ec82ee3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 0x55ec82ee3970 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00fc20 tx=0x7f3bfc005fd0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1887040623 shutdown_connections 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82ec3370 0x55ec82ee3970 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82eb9b40 0x55ec82ee3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/1887040623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ed7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1887040623 >> 172.21.15.86:0/1887040623 conn(0x55ec82e81310 msgr2=0x55ec82ebb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1887040623 shutdown_connections 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/1887040623 wait complete. 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.972+0000 7f3c0eb7d740 1 -- start start 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 Processor -- start 2024-09-05T07:02:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ed03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82f45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041ae490 con 0x55ec82eb85d0 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041addc0 con 0x55ec82eb9b40 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041ab020 con 0x55ec82ec3370 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0eb7d740 10 client.0 _ll_register_callbacks cb 0x7f3bf4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50856/0 (socket says 172.21.15.86:50856) 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 -- 172.21.15.86:0/3013777039 learned_addr learned my addr 172.21.15.86:0/3013777039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0ad41640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82f45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 -- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 msgr2=0x55ec82f45400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82f45400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 -- 172.21.15.86:0/3013777039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 msgr2=0x55ec82ed03e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ed03e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.973+0000 7f3c0a540640 1 -- 172.21.15.86:0/3013777039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec82f45940 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3c0a540640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005fd0 tx=0x7f3c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ddb9eac03e771a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00028710 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc003fc0 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f3c000288b0 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3c0004c050 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3c0eb7d740 10 client.20208 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ec82ec6520 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.974+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bfc003fc0 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f3c0001fc60 con 0x55ec82eb85d0 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 10 client.20208 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:55.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 10 client.20208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 20 client.20208 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82ec6c70 0x55ec82ec90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 4 client.20208 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec82e25d70 con 0x55ec82ec6c70 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82ec6c70 0x55ec82ec90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3bedffb640 10 client.20208 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.975+0000 7f3c09d3f640 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82ec6c70 0x55ec82ec90b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce96e783f01adc8e server_cookie=a47e86a0cb79c1d0 in_seq=0 out_seq=0 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3bedffb640 1 -- 172.21.15.86:0/3013777039 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec82e25d70 con 0x55ec82ec6c70 2024-09-05T07:02:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3bedffb640 10 client.20208 handle_command_reply: tid=0 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 client.20208 shutdown 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 msgr2=0x55ec82ecfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005fd0 tx=0x7f3c00004730 comp rx=0 tx=0).stop 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 shutdown_connections 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55ec82ec6c70 0x55ec82ec90b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec82ec3370 0x55ec82f45400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3bedffb640 0 client.20208 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec82eb9b40 0x55ec82ed03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 --2- 172.21.15.86:0/3013777039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec82eb85d0 0x55ec82ecfea0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.976+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 >> 172.21.15.86:0/3013777039 conn(0x55ec82e81310 msgr2=0x55ec82eb34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 shutdown_connections 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0eb7d740 1 -- 172.21.15.86:0/3013777039 wait complete. 2024-09-05T07:02:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0eb7d740 20 client.20208 trim_cache size 0 max 16384 2024-09-05T07:02:55.987 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 msgr2=0x7f3bf00a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 0x7f3bf00a96c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005d00 tx=0x7f3c0000b410 comp rx=0 tx=0).stop 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 msgr2=0x7f3c041966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.977+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4005ab0 tx=0x7f3bf4003b90 comp rx=0 tx=0).stop 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 shutdown_connections 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f3bf00a7200 0x7f3bf00a96c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c04106a70 0x7f3c0419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c04101410 0x7f3c041966d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 --2- 172.21.15.86:0/1122304589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c04100a70 0x7f3c04196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:55.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 >> 172.21.15.86:0/1122304589 conn(0x7f3c040fc790 msgr2=0x7f3c040fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:55.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 shutdown_connections 2024-09-05T07:02:55.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:55.978+0000 7f3c0c7a4640 1 -- 172.21.15.86:0/1122304589 wait complete. 2024-09-05T07:02:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:55 smithi096 ceph-mon[34468]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 247 op/s 2024-09-05T07:02:56.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60921 6.56 MB/sec execute 374 sec latency 59.360 ms 2024-09-05T07:02:56.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:02:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:02:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/2486771679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 msgr2=0x7ff8d4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/2486771679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d4103110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002a410 comp rx=0 tx=0).stop 2024-09-05T07:02:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/2486771679 shutdown_connections 2024-09-05T07:02:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/2486771679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/2486771679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 0x7ff8d4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/2486771679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d4103110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.748+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/2486771679 >> 172.21.15.86:0/2486771679 conn(0x7ff8d40fc7d0 msgr2=0x7ff8d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.749+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/2486771679 shutdown_connections 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.749+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/2486771679 wait complete. 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 Processor -- start 2024-09-05T07:02:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 -- start start 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 0x7ff8d410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4111f50 con 0x7ff8d41036e0 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4111dd0 con 0x7ff8d41082a0 2024-09-05T07:02:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.750+0000 7ff8dc1d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41120d0 con 0x7ff8d4102d30 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44746/0 (socket says 172.21.15.86:44746) 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 -- 172.21.15.86:0/3155647299 learned_addr learned my addr 172.21.15.86:0/3155647299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8d9770640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 0x7ff8d410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 -- 172.21.15.86:0/3155647299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 msgr2=0x7ff8d419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 -- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 msgr2=0x7ff8d410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 0x7ff8d410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 -- 172.21.15.86:0/3155647299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d410e8f0 con 0x7ff8d41082a0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8da772640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eb10 tx=0x7ff8cc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.751+0000 7ff8caffd640 1 -- 172.21.15.86:0/3155647299 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc004b20 con 0x7ff8d41082a0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.752+0000 7ff8caffd640 1 -- 172.21.15.86:0/3155647299 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8cc004cc0 con 0x7ff8d41082a0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.752+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d410ebe0 con 0x7ff8d41082a0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.752+0000 7ff8caffd640 1 -- 172.21.15.86:0/3155647299 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc010690 con 0x7ff8d41082a0 2024-09-05T07:02:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.752+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d419ada0 con 0x7ff8d41082a0 2024-09-05T07:02:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.755+0000 7ff8caffd640 1 -- 172.21.15.86:0/3155647299 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff8cc005020 con 0x7ff8d41082a0 2024-09-05T07:02:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.755+0000 7ff8caffd640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 0x7ff8c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 Processor -- start 2024-09-05T07:02:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8caffd640 1 -- 172.21.15.86:0/3155647299 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7ff8cc0cfd20 con 0x7ff8d41082a0 2024-09-05T07:02:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 -- start start 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 0x556bc0b13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bc0a792f0 con 0x556bc0b14b40 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bc0a20cf0 con 0x556bc0b135d0 2024-09-05T07:02:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bc0a752b0 con 0x556bc0b1e370 2024-09-05T07:02:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 0x7ff8c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.756+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44754/0 (socket says 172.21.15.86:44754) 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 -- 172.21.15.86:0/1969133392 learned_addr learned my addr 172.21.15.86:0/1969133392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8da772640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 -- 172.21.15.86:0/1969133392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 msgr2=0x556bc0b20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 0x556bc0b13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 -- 172.21.15.86:0/1969133392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 msgr2=0x556bc0b13090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 0x556bc0b13090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 -- 172.21.15.86:0/1969133392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc0a053a0 con 0x556bc0b135d0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 0x556bc0b13090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009c20 tx=0x7ff8c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31d0f842559cd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.757+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 0x7ff8c00a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009230 tx=0x7ff8d0005b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.758+0000 7ff8bffff640 1 -- 172.21.15.86:0/1969133392 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4004050 con 0x556bc0b135d0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.758+0000 7ff8bffff640 1 -- 172.21.15.86:0/1969133392 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8c40054d0 con 0x556bc0b135d0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.758+0000 7ff8bffff640 1 -- 172.21.15.86:0/1969133392 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4004cf0 con 0x556bc0b135d0 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.758+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1969133392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 msgr2=0x556bc0b13a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.758+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009c20 tx=0x7ff8c4011810 comp rx=0 tx=0).stop 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1969133392 shutdown_connections 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b135d0 0x556bc0b13a10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1969133392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b14b40 0x556bc0b13090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1969133392 >> 172.21.15.86:0/1969133392 conn(0x556bc0adc310 msgr2=0x556bc0b0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1969133392 shutdown_connections 2024-09-05T07:02:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1969133392 wait complete. 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 -- start start 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.759+0000 7ff8de5ae740 1 Processor -- start 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b1e370 0x556bc0b1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bc0b21070 con 0x556bc0b1e370 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bc0a34060 con 0x556bc0b14b40 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bc0b211f0 con 0x556bc0b135d0 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:53558/0 (socket says 172.21.15.86:53558) 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 -- 172.21.15.86:0/3330973588 learned_addr learned my addr 172.21.15.86:0/3330973588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8de5ae740 10 client.0 _ll_register_callbacks cb 0x556bc07878c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8da772640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b1e370 0x556bc0b1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9770640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 -- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 msgr2=0x556bc0b1af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 -- 172.21.15.86:0/3330973588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b1e370 msgr2=0x556bc0b1b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b1e370 0x556bc0b1b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.760+0000 7ff8d9f71640 1 -- 172.21.15.86:0/3330973588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc07ea3c0 con 0x556bc0b135d0 2024-09-05T07:02:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.761+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d00058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb7bab585aa9ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.761+0000 7ff8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:02:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.761+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0035040 con 0x556bc0b135d0 2024-09-05T07:02:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.761+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c40054d0 con 0x556bc0b135d0 2024-09-05T07:02:57.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.761+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7ff8d0004450 con 0x556bc0b135d0 2024-09-05T07:02:57.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.762+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff8d00035f0 con 0x556bc0b135d0 2024-09-05T07:02:57.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.762+0000 7ff8de5ae740 10 client.28987 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:57.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.762+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556bc0b2ca20 con 0x556bc0b135d0 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.762+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8c40054d0 con 0x556bc0b135d0 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7ff8d00488d0 con 0x556bc0b135d0 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 10 client.28987 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 10 client.28987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 20 client.28987 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b2de30 0x556bc0b302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 4 client.28987 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:02:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.763+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc0a86860 con 0x556bc0b2de30 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.765+0000 7ff8d9770640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b2de30 0x556bc0b302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.766+0000 7ff8d9770640 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b2de30 0x556bc0b302c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba05b584b19718b server_cookie=c4ee65a0eaeddc62 in_seq=0 out_seq=0 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.766+0000 7ff8bd7fa640 10 client.28987 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.766+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/3330973588 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bc0a86860 con 0x556bc0b2de30 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.766+0000 7ff8bd7fa640 10 client.28987 handle_command_reply: tid=0 2024-09-05T07:02:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.766+0000 7ff8de5ae740 1 client.28987 shutdown 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 msgr2=0x556bc0b1dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d00058d0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 shutdown_connections 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b2de30 0x556bc0b302c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b1e370 0x556bc0b1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8bd7fa640 0 client.28987 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/3330973588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b135d0 0x556bc0b1dcb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 >> 172.21.15.86:0/3330973588 conn(0x556bc0adc310 msgr2=0x556bc0b15930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 shutdown_connections 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 1 -- 172.21.15.86:0/3330973588 wait complete. 2024-09-05T07:02:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.767+0000 7ff8de5ae740 20 client.28987 trim_cache size 0 max 16384 2024-09-05T07:02:57.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61079 6.56 MB/sec execute 375 sec latency 78.061 ms 2024-09-05T07:02:57.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 Processor -- start 2024-09-05T07:02:57.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 -- start start 2024-09-05T07:02:57.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b2a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b1fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41bded0 con 0x556bc0b135d0 2024-09-05T07:02:57.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41bfa10 con 0x556bc0b14b40 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41be5a0 con 0x556bc0b1e370 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44770/0 (socket says 172.21.15.86:44770) 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8d9770640 1 -- 172.21.15.86:0/2349778132 learned_addr learned my addr 172.21.15.86:0/2349778132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b2a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8da772640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b1fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.789+0000 7ff8d9f71640 1 -- 172.21.15.86:0/2349778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 msgr2=0x556bc0b1fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b1fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9f71640 1 -- 172.21.15.86:0/2349778132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 msgr2=0x556bc0b1f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9f71640 1 -- 172.21.15.86:0/2349778132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc0b14300 con 0x556bc0b135d0 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9770640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b2a310 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0048d10 tx=0x7ff8d0031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3171511d4a9761 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8bf7fe640 1 -- 172.21.15.86:0/2349778132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d005e070 con 0x556bc0b135d0 2024-09-05T07:02:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8bf7fe640 1 -- 172.21.15.86:0/2349778132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8d004d070 con 0x556bc0b135d0 2024-09-05T07:02:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.790+0000 7ff8bf7fe640 1 -- 172.21.15.86:0/2349778132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0035040 con 0x556bc0b135d0 2024-09-05T07:02:57.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 -- 172.21.15.86:0/2349778132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 msgr2=0x556bc0b2a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b2a310 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0048d10 tx=0x7ff8d0031f90 comp rx=0 tx=0).stop 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 -- 172.21.15.86:0/2349778132 shutdown_connections 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b1fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b1f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/2349778132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b2a310 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 -- 172.21.15.86:0/2349778132 >> 172.21.15.86:0/2349778132 conn(0x556bc0adc310 msgr2=0x556bc0b1af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 -- 172.21.15.86:0/2349778132 shutdown_connections 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.791+0000 7ff8de5ae740 1 -- 172.21.15.86:0/2349778132 wait complete. 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 -- start start 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 Processor -- start 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8de5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41bf340 con 0x556bc0b135d0 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8d9770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44774/0 (socket says 172.21.15.86:44774) 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8d9770640 1 -- 172.21.15.86:0/1828868013 learned_addr learned my addr 172.21.15.86:0/1828868013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.792+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41bec70 con 0x556bc0b14b40 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41bbed0 con 0x556bc0b1e370 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8da772640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8de5ae740 10 client.0 _ll_register_callbacks cb 0x7ff8cc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 -- 172.21.15.86:0/1828868013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 msgr2=0x556bc0b363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 -- 172.21.15.86:0/1828868013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 msgr2=0x556bc0b35970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b35970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc0b24d70 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8d9770640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4011d10 tx=0x7ff8c4009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f824abfe7ea8f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4031070 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d004d070 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.793+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7ff8c4004460 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff8c403d050 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 10 client.28967 fetch_fsmap learned FSMap version 45 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556bc0b990a0 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8c4003aa0 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7ff8c402c590 con 0x556bc0b14b40 2024-09-05T07:02:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 10 client.28967 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 10 client.28967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 20 client.28967 populate_metadata read hostname 'smithi086' 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b99620 0x556bc0b9ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 4 client.28967 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.794+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff8d004d070 con 0x556bc0b99620 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b99620 0x556bc0b9ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8d9f71640 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b99620 0x556bc0b9ba40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261595040d7ec985 server_cookie=687b3e47448008f0 in_seq=0 out_seq=0 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8bd7fa640 10 client.28967 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8bd7fa640 1 -- 172.21.15.86:0/1828868013 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff8d004d070 con 0x556bc0b99620 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8bd7fa640 10 client.28967 handle_command_reply: tid=0 2024-09-05T07:02:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8de5ae740 1 client.28967 shutdown 2024-09-05T07:02:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 msgr2=0x556bc0b35eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.795+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4011d10 tx=0x7ff8c4009dd0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 shutdown_connections 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x556bc0b99620 0x556bc0b9ba40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bc0b1e370 0x556bc0b363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bc0b14b40 0x556bc0b35eb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 --2- 172.21.15.86:0/1828868013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bc0b135d0 0x556bc0b35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 >> 172.21.15.86:0/1828868013 conn(0x556bc0adc310 msgr2=0x556bc0b0f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8bd7fa640 0 client.28967 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 shutdown_connections 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 1 -- 172.21.15.86:0/1828868013 wait complete. 2024-09-05T07:02:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.796+0000 7ff8de5ae740 20 client.28967 trim_cache size 0 max 16384 2024-09-05T07:02:57.802 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 msgr2=0x7ff8c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 0x7ff8c00a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009230 tx=0x7ff8d0005b50 comp rx=0 tx=0).stop 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 msgr2=0x7ff8d410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eb10 tx=0x7ff8cc00c2d0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 shutdown_connections 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff8c00a2d70 0x7ff8c00a5230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41082a0 0x7ff8d410e3b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d41036e0 0x7ff8d410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 --2- 172.21.15.86:0/3155647299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8d4102d30 0x7ff8d419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:02:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.797+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 >> 172.21.15.86:0/3155647299 conn(0x7ff8d40fc7d0 msgr2=0x7ff8d4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:02:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.798+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 shutdown_connections 2024-09-05T07:02:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:02:57.798+0000 7ff8dc1d5640 1 -- 172.21.15.86:0/3155647299 wait complete. 2024-09-05T07:02:57.844 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:57 smithi086 ceph-mon[30943]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-05T07:02:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:57 smithi115 ceph-mon[34791]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-05T07:02:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:57 smithi096 ceph-mon[34468]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-05T07:02:58.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61234 6.56 MB/sec execute 376 sec latency 129.513 ms 2024-09-05T07:02:59.771 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:02:59.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:02:59.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61399 6.58 MB/sec execute 377 sec latency 213.641 ms 2024-09-05T07:02:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:59 smithi115 ceph-mon[34791]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 332 op/s 2024-09-05T07:02:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:02:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:02:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:02:59 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:02:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:59 smithi086 ceph-mon[30943]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 332 op/s 2024-09-05T07:02:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:02:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:02:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:02:59 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:59 smithi096 ceph-mon[34468]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 332 op/s 2024-09-05T07:03:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:03:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:02:59 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:00.166 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:00.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:00 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:03:00.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:00 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:03:00.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:00 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:00.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:00 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:03:00.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:00 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:03:00.586 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:00 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:00.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61576 6.59 MB/sec execute 378 sec latency 95.803 ms 2024-09-05T07:03:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:00 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:03:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:00 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:03:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:00 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:03:01.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61759 6.60 MB/sec execute 379 sec latency 121.740 ms 2024-09-05T07:03:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:01 smithi115 ceph-mon[34791]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 230 op/s 2024-09-05T07:03:01.957 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:01 smithi096 ceph-mon[34468]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 230 op/s 2024-09-05T07:03:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:01 smithi086 ceph-mon[30943]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 230 op/s 2024-09-05T07:03:02.774 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61907 6.61 MB/sec execute 380 sec latency 134.088 ms 2024-09-05T07:03:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:02 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:02 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:02 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:02 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:02.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:02 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:02.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:02 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:02.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:02 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:02.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:02 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:02 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:02 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:02 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:02 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/4202620628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b88101df0 msgr2=0x7f4b881021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/4202620628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b88101df0 0x7f4b881021d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009900 tx=0x7f4b8002a410 comp rx=0 tx=0).stop 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/4202620628 shutdown_connections 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/4202620628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88106f20 0x7f4b8810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/4202620628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b881027a0 0x7f4b881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/4202620628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b88101df0 0x7f4b881021d0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/4202620628 >> 172.21.15.86:0/4202620628 conn(0x7f4b880fd8f0 msgr2=0x7f4b880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/4202620628 shutdown_connections 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/4202620628 wait complete. 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.278+0000 7f4b8f8ac640 1 Processor -- start 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 -- start start 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88101df0 0x7f4b88198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b88106f20 0x7f4b8819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88111e30 con 0x7f4b881027a0 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88111cb0 con 0x7f4b88106f20 2024-09-05T07:03:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8f8ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88111fb0 con 0x7f4b88101df0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50908/0 (socket says 172.21.15.86:50908) 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3566995232 learned_addr learned my addr 172.21.15.86:0/3566995232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8de49640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b88106f20 0x7f4b8819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8d648640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88101df0 0x7f4b88198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3566995232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88101df0 msgr2=0x7f4b88198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88101df0 0x7f4b88198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3566995232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b88106f20 msgr2=0x7f4b8819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b88106f20 0x7f4b8819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3566995232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819d2d0 con 0x7f4b881027a0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.279+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e9f0 tx=0x7f4b7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.280+0000 7f4b867fc640 1 -- 172.21.15.86:0/3566995232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00cd80 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.280+0000 7f4b867fc640 1 -- 172.21.15.86:0/3566995232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b7c0108e0 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.280+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8819d5c0 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.280+0000 7f4b867fc640 1 -- 172.21.15.86:0/3566995232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c018920 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.280+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a4ea0 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.282+0000 7f4b867fc640 1 -- 172.21.15.86:0/3566995232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4b7c010a90 con 0x7f4b881027a0 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 Processor -- start 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 -- start start 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 0x56400e766c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e769540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56400e6bd280 con 0x56400e767150 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56400e664b10 con 0x56400e75bf70 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b91c85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56400e6b9240 con 0x56400e75c890 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b867fc640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 0x7f4b700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b8de49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b8de49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:44816/0 (socket says 172.21.15.86:44816) 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b8de49640 1 -- 172.21.15.86:0/1863670015 learned_addr learned my addr 172.21.15.86:0/1863670015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e769540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8de49640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 0x7f4b700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b867fc640 1 -- 172.21.15.86:0/3566995232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f4b7c0cfb60 con 0x7f4b881027a0 2024-09-05T07:03:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 -- 172.21.15.86:0/1863670015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 msgr2=0x56400e766c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.283+0000 7f4b8d648640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 0x56400e766c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 0x56400e766c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 -- 172.21.15.86:0/1863670015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 msgr2=0x56400e75c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 -- 172.21.15.86:0/1863670015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400e6711f0 con 0x56400e767150 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8de49640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8d648640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 0x56400e766c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8de49640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 0x7f4b700a51b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b780099e0 tx=0x7f4b78009c10 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.284+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e769540 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0cce70 tx=0x7f4b7c09bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318b36eb8f5918b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.285+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/1863670015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0cd0f0 con 0x56400e767150 2024-09-05T07:03:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.285+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/1863670015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b7c098360 con 0x56400e767150 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.285+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/1863670015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c09b3a0 con 0x56400e767150 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- 172.21.15.86:0/1863670015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 msgr2=0x56400e769540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e769540 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0cce70 tx=0x7f4b7c09bdb0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- 172.21.15.86:0/1863670015 shutdown_connections 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e769540 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75c890 0x56400e766c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1863670015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e75c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- 172.21.15.86:0/1863670015 >> 172.21.15.86:0/1863670015 conn(0x56400e758500 msgr2=0x56400e7588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- 172.21.15.86:0/1863670015 shutdown_connections 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- 172.21.15.86:0/1863670015 wait complete. 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 -- start start 2024-09-05T07:03:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.286+0000 7f4b91c85740 1 Processor -- start 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e764060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e764ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56400e769d20 con 0x56400e75c890 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56400e677f10 con 0x56400e75bf70 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56400e769ea0 con 0x56400e767150 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b91c85740 10 client.0 _ll_register_callbacks cb 0x56400e640a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50926/0 (socket says 172.21.15.86:50926) 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3421465840 learned_addr learned my addr 172.21.15.86:0/3421465840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3421465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 msgr2=0x56400e764ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e764ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 msgr2=0x56400e764060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e764060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.287+0000 7f4b8ce47640 1 -- 172.21.15.86:0/3421465840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400e42e6a0 con 0x56400e75c890 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.288+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0986c0 tx=0x7f4b7c09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ec68f77c0fc3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.288+0000 7f4b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.288+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0a4670 con 0x56400e75c890 2024-09-05T07:03:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.288+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f4b7c09b3e0 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.288+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7c098360 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.289+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4b7c0a4420 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.289+0000 7f4b91c85740 10 client.20235 fetch_fsmap learned FSMap version 45 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.289+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56400e7758c0 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.289+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b7c098360 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4b7c018be0 con 0x56400e75c890 2024-09-05T07:03:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 10 client.20235 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3ab9887c-3140-4060-b22c-033f9d57f0b4", 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 10 client.20235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 20 client.20235 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e776eb0 0x56400e779330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 4 client.20235 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400e6ca7f0 con 0x56400e776eb0 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b8d648640 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e776eb0 0x56400e779330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b6cff9640 10 client.20235 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.290+0000 7f4b8d648640 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e776eb0 0x56400e779330 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25bd7274da8d02f server_cookie=11d7b9e20c7ab548 in_seq=0 out_seq=0 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b6cff9640 1 -- 172.21.15.86:0/3421465840 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56400e6ca7f0 con 0x56400e776eb0 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b6cff9640 10 client.20235 handle_command_reply: tid=0 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 client.20235 shutdown 2024-09-05T07:03:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 msgr2=0x56400e7645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0986c0 tx=0x7f4b7c09b370 comp rx=0 tx=0).stop 2024-09-05T07:03:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 shutdown_connections 2024-09-05T07:03:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e776eb0 0x56400e779330 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e764ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75c890 0x56400e7645a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 --2- 172.21.15.86:0/3421465840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e75bf70 0x56400e764060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 >> 172.21.15.86:0/3421465840 conn(0x56400e758500 msgr2=0x56400e753270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b6cff9640 0 client.20235 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 shutdown_connections 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.291+0000 7f4b91c85740 1 -- 172.21.15.86:0/3421465840 wait complete. 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.292+0000 7f4b91c85740 20 client.20235 trim_cache size 0 max 16384 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 Processor -- start 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 -- start start 2024-09-05T07:03:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e77f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e752a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881ad0b0 con 0x56400e75bf70 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881aebf0 con 0x56400e763040 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b91c85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881ad780 con 0x56400e767150 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.309+0000 7f4b8d648640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50942/0 (socket says 172.21.15.86:50942) 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 -- 172.21.15.86:0/2749430982 learned_addr learned my addr 172.21.15.86:0/2749430982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8de49640 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e752a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e77f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 -- 172.21.15.86:0/2749430982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 msgr2=0x56400e77f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e77f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 -- 172.21.15.86:0/2749430982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 msgr2=0x56400e752a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e752a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 -- 172.21.15.86:0/2749430982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400e758330 con 0x56400e75bf70 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b8d648640 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009340 tx=0x7f4b80032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4165aaa5edeef3c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b6effd640 1 -- 172.21.15.86:0/2749430982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80035040 con 0x56400e75bf70 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b6effd640 1 -- 172.21.15.86:0/2749430982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b80046020 con 0x56400e75bf70 2024-09-05T07:03:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.310+0000 7f4b6effd640 1 -- 172.21.15.86:0/2749430982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800055b0 con 0x56400e75bf70 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- 172.21.15.86:0/2749430982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 msgr2=0x56400e77ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009340 tx=0x7f4b80032f00 comp rx=0 tx=0).stop 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- 172.21.15.86:0/2749430982 shutdown_connections 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e752a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e767150 0x56400e77f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 --2- 172.21.15.86:0/2749430982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e75bf70 0x56400e77ec40 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- 172.21.15.86:0/2749430982 >> 172.21.15.86:0/2749430982 conn(0x56400e758500 msgr2=0x56400e762930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- 172.21.15.86:0/2749430982 shutdown_connections 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- 172.21.15.86:0/2749430982 wait complete. 2024-09-05T07:03:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 -- start start 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.311+0000 7f4b91c85740 1 Processor -- start 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 0x56400e76f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e76aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881ae520 con 0x56400e767150 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881ade50 con 0x56400e763040 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881ab090 con 0x56400e75bf70 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 10 client.0 _ll_register_callbacks cb 0x7f4b7c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50958/0 (socket says 172.21.15.86:50958) 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 -- 172.21.15.86:0/1445335465 learned_addr learned my addr 172.21.15.86:0/1445335465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8ce47640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e76aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8d648640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 0x56400e76f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 -- 172.21.15.86:0/1445335465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 msgr2=0x56400e76f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 0x56400e76f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 -- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 msgr2=0x56400e76aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e76aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 -- 172.21.15.86:0/1445335465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400e76b910 con 0x56400e767150 2024-09-05T07:03:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8d648640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 0x56400e76f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b8de49640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f4b78003ba0 tx=0x7f4b78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a6e8ea7f711643 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004640 con 0x56400e767150 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7f4b780047e0 con 0x56400e767150 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.312+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b80046020 con 0x56400e767150 2024-09-05T07:03:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4b78032050 con 0x56400e767150 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 10 client.20241 fetch_fsmap learned FSMap version 45 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56400e7e1c40 con 0x56400e767150 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b80046020 con 0x56400e767150 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4b78027020 con 0x56400e767150 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 10 client.20241 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 10 client.20241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 20 client.20241 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e7e23c0 0x56400e7e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 4 client.20241 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56400e6c4d00 con 0x56400e7e23c0 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b8d648640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e7e23c0 0x56400e7e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b6cff9640 10 client.20241 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.313+0000 7f4b8d648640 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e7e23c0 0x56400e7e4800 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebad58b754854f6d server_cookie=4d49576c5ef120bd in_seq=0 out_seq=0 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1445335465 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56400e6c4d00 con 0x56400e7e23c0 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b6cff9640 10 client.20241 handle_command_reply: tid=0 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 client.20241 shutdown 2024-09-05T07:03:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 msgr2=0x56400e76b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f4b78003ba0 tx=0x7f4b78004ab0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 shutdown_connections 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56400e7e23c0 0x56400e7e4800 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400e767150 0x56400e76b0e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b6cff9640 0 client.20241 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e763040 0x56400e76aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 --2- 172.21.15.86:0/1445335465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56400e75bf70 0x56400e76f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 >> 172.21.15.86:0/1445335465 conn(0x56400e758500 msgr2=0x56400e752a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.314+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 shutdown_connections 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b91c85740 1 -- 172.21.15.86:0/1445335465 wait complete. 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b91c85740 20 client.20241 trim_cache size 0 max 16384 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 msgr2=0x7f4b700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 0x7f4b700a51b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b780099e0 tx=0x7f4b78009c10 comp rx=0 tx=0).stop 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 msgr2=0x7f4b88198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e9f0 tx=0x7f4b7c00eeb0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 shutdown_connections 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b700a2cf0 0x7f4b700a51b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b88106f20 0x7f4b8819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b881027a0 0x7f4b88198860 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 --2- 172.21.15.86:0/3566995232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b88101df0 0x7f4b88198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 >> 172.21.15.86:0/3566995232 conn(0x7f4b880fd8f0 msgr2=0x7f4b880fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 shutdown_connections 2024-09-05T07:03:03.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:02.315+0000 7f4b8f8ac640 1 -- 172.21.15.86:0/3566995232 wait complete. 2024-09-05T07:03:03.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62035 6.62 MB/sec execute 381 sec latency 180.145 ms 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:03 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:03.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:03 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:04.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:03 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:04.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:03:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:04.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62207 6.64 MB/sec execute 382 sec latency 544.920 ms 2024-09-05T07:03:05.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 -- 172.21.15.86:0/1200283567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 msgr2=0x7fbc4c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 --2- 172.21.15.86:0/1200283567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c103110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009900 tx=0x7fbc4002a410 comp rx=0 tx=0).stop 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 -- 172.21.15.86:0/1200283567 shutdown_connections 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 --2- 172.21.15.86:0/1200283567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 --2- 172.21.15.86:0/1200283567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 --2- 172.21.15.86:0/1200283567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c103110 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 -- 172.21.15.86:0/1200283567 >> 172.21.15.86:0/1200283567 conn(0x7fbc4c0fc7d0 msgr2=0x7fbc4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 -- 172.21.15.86:0/1200283567 shutdown_connections 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.120+0000 7fbc52f18640 1 -- 172.21.15.86:0/1200283567 wait complete. 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 Processor -- start 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 -- start start 2024-09-05T07:03:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c111f50 con 0x7fbc4c1082a0 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c111dd0 con 0x7fbc4c1036e0 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.121+0000 7fbc52f18640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1120d0 con 0x7fbc4c102d30 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38994/0 (socket says 172.21.15.86:38994) 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc4bfff640 1 -- 172.21.15.86:0/218233270 learned_addr learned my addr 172.21.15.86:0/218233270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 -- 172.21.15.86:0/218233270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 msgr2=0x7fbc4c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 -- 172.21.15.86:0/218233270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 msgr2=0x7fbc4c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 -- 172.21.15.86:0/218233270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c19d500 con 0x7fbc4c1082a0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.122+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c19ce20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009fd0 tx=0x7fbc4400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.123+0000 7fbc49ffb640 1 -- 172.21.15.86:0/218233270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400ce20 con 0x7fbc4c1082a0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.123+0000 7fbc49ffb640 1 -- 172.21.15.86:0/218233270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc440137d0 con 0x7fbc4c1082a0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.123+0000 7fbc49ffb640 1 -- 172.21.15.86:0/218233270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44005540 con 0x7fbc4c1082a0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.123+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c19d7f0 con 0x7fbc4c1082a0 2024-09-05T07:03:05.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.123+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c1008c0 con 0x7fbc4c1082a0 2024-09-05T07:03:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.126+0000 7fbc49ffb640 1 -- 172.21.15.86:0/218233270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc44013980 con 0x7fbc4c1082a0 2024-09-05T07:03:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.126+0000 7fbc552f1740 1 Processor -- start 2024-09-05T07:03:05.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 -- start start 2024-09-05T07:03:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d22d2e0 con 0x55cc1d2dae90 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d1d4ce0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc552f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d2292a0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc49ffb640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 0x7fbc3c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc50cb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:54276/0 (socket says 172.21.15.86:54276) 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc4bfff640 1 -- 172.21.15.86:0/2067909353 learned_addr learned my addr 172.21.15.86:0/2067909353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 0x7fbc3c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.127+0000 7fbc49ffb640 1 -- 172.21.15.86:0/218233270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fbc440d3d30 con 0x7fbc4c1082a0 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 -- 172.21.15.86:0/2067909353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 msgr2=0x55cc1d2c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 -- 172.21.15.86:0/2067909353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 msgr2=0x55cc1d2dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 -- 172.21.15.86:0/2067909353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1d1b9440 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc514b5640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 0x7fbc3c0a5300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbc4409fbc0 tx=0x7fbc4406ab70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2da950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005940 tx=0x7fbc40032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daab5586e24e3ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc36ffd640 1 -- 172.21.15.86:0/2067909353 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40035040 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc36ffd640 1 -- 172.21.15.86:0/2067909353 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc400316b0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.128+0000 7fbc36ffd640 1 -- 172.21.15.86:0/2067909353 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40003d60 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- 172.21.15.86:0/2067909353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 msgr2=0x55cc1d2da950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2da950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005940 tx=0x7fbc40032e60 comp rx=0 tx=0).stop 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- 172.21.15.86:0/2067909353 shutdown_connections 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2da950 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 --2- 172.21.15.86:0/2067909353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- 172.21.15.86:0/2067909353 >> 172.21.15.86:0/2067909353 conn(0x55cc1d2c3a30 msgr2=0x55cc1d2c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- 172.21.15.86:0/2067909353 shutdown_connections 2024-09-05T07:03:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- 172.21.15.86:0/2067909353 wait complete. 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.129+0000 7fbc552f1740 1 -- start start 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 Processor -- start 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2d8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2da820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35414/0 (socket says 172.21.15.86:35414) 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc4bfff640 1 -- 172.21.15.86:0/1405421642 learned_addr learned my addr 172.21.15.86:0/1405421642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d2dd9a0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d1e8050 con 0x55cc1d2dae90 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cc1d2ddb20 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.130+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc552f1740 10 client.0 _ll_register_callbacks cb 0x55cc1d228160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 -- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 msgr2=0x55cc1d2d8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2d8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 -- 172.21.15.86:0/1405421642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 msgr2=0x55cc1d2da820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2da820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1cf9e3c0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2d8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2d7ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c00 tx=0x7fbc400038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ddd3dde989dc238 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4004e030 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc400316b0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fbc400314b0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.131+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbc4005b050 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 10 client.29014 fetch_fsmap learned FSMap version 45 2024-09-05T07:03:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cc1d2dd9a0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc400316b0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fbc4003c6a0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 10 client.29014 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 10 client.29014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 20 client.29014 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d2ea510 0x55cc1d2ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 4 client.29014 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc1d23a850 con 0x55cc1d2ea510 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.132+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d2ea510 0x55cc1d2ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.133+0000 7fbc2bfff640 10 client.29014 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.133+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d2ea510 0x55cc1d2ec900 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c221e24974a0435 server_cookie=f44cc36ee63617d0 in_seq=0 out_seq=0 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc2bfff640 1 -- 172.21.15.86:0/1405421642 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc1d23a850 con 0x55cc1d2ea510 2024-09-05T07:03:05.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc2bfff640 10 client.29014 handle_command_reply: tid=0 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 client.29014 shutdown 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 msgr2=0x55cc1d2d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2d7ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c00 tx=0x7fbc400038a0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 shutdown_connections 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d2ea510 0x55cc1d2ec900 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2d8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2d7ad0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 --2- 172.21.15.86:0/1405421642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2da820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 >> 172.21.15.86:0/1405421642 conn(0x55cc1d2c3a30 msgr2=0x55cc1d2c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc2bfff640 0 client.29014 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 shutdown_connections 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 1 -- 172.21.15.86:0/1405421642 wait complete. 2024-09-05T07:03:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.134+0000 7fbc552f1740 20 client.29014 trim_cache size 0 max 16384 2024-09-05T07:03:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 Processor -- start 2024-09-05T07:03:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 -- start start 2024-09-05T07:03:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1acfa0 con 0x55cc1d2dae90 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1aeae0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.163+0000 7fbc552f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1ad670 con 0x55cc1d2c73c0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.164+0000 7fbc50cb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.164+0000 7fbc50cb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:39026/0 (socket says 172.21.15.86:39026) 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.164+0000 7fbc50cb4640 1 -- 172.21.15.86:0/3866358466 learned_addr learned my addr 172.21.15.86:0/3866358466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.164+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.164+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 -- 172.21.15.86:0/3866358466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 msgr2=0x55cc1d2f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 -- 172.21.15.86:0/3866358466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 msgr2=0x55cc1d2f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 -- 172.21.15.86:0/3866358466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1d2dd5f0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2e6c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc38000c00 tx=0x7fbc38004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ab31333daa9fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc367fc640 1 -- 172.21.15.86:0/3866358466 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38000e60 con 0x55cc1d2c73c0 2024-09-05T07:03:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc367fc640 1 -- 172.21.15.86:0/3866358466 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc3802a9c0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.165+0000 7fbc367fc640 1 -- 172.21.15.86:0/3866358466 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38020910 con 0x55cc1d2c73c0 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.166+0000 7fbc552f1740 1 -- 172.21.15.86:0/3866358466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 msgr2=0x55cc1d2e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.166+0000 7fbc552f1740 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2e6c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc38000c00 tx=0x7fbc38004730 comp rx=0 tx=0).stop 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.167+0000 7fbc552f1740 1 -- 172.21.15.86:0/3866358466 shutdown_connections 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.167+0000 7fbc552f1740 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2dae90 0x55cc1d2f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.167+0000 7fbc552f1740 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d2f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.167+0000 7fbc552f1740 1 --2- 172.21.15.86:0/3866358466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c73c0 0x55cc1d2e6c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.167+0000 7fbc552f1740 1 -- 172.21.15.86:0/3866358466 >> 172.21.15.86:0/3866358466 conn(0x55cc1d2c3a30 msgr2=0x55cc1d2d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.169+0000 7fbc552f1740 1 -- 172.21.15.86:0/3866358466 shutdown_connections 2024-09-05T07:03:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.170+0000 7fbc552f1740 1 -- 172.21.15.86:0/3866358466 wait complete. 2024-09-05T07:03:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.170+0000 7fbc552f1740 1 -- start start 2024-09-05T07:03:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.170+0000 7fbc552f1740 1 Processor -- start 2024-09-05T07:03:05.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d34d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2de330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1ae410 con 0x55cc1d2c73c0 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1add40 con 0x55cc1d2dae90 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1aafa0 con 0x55cc1d2c7ce0 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc552f1740 10 client.0 _ll_register_callbacks cb 0x7fbc40040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35428/0 (socket says 172.21.15.86:35428) 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc4bfff640 1 -- 172.21.15.86:0/540514341 learned_addr learned my addr 172.21.15.86:0/540514341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.171+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 -- 172.21.15.86:0/540514341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 msgr2=0x55cc1d34d890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T07:03:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d34d890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 -- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 msgr2=0x55cc1d2de330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2de330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 -- 172.21.15.86:0/540514341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1d2dea20 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc4bfff640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbc380059f0 tx=0x7fbc38013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c78fda2a4c6262a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.172+0000 7fbc2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3802f040 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc3802a9c0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9605+0+0 (secure 0 0 0) 0x7fbc3802a7b0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbc38005020 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc552f1740 10 client.20259 fetch_fsmap learned FSMap version 45 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.173+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cc1d354b30 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc3802a9c0 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fbc38035070 con 0x55cc1d2c73c0 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 10 client.20259 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T07:03:05.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 10 client.20259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 20 client.20259 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d355160 0x55cc1d357550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 4 client.20259 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:03:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.174+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc1d234d60 con 0x55cc1d355160 2024-09-05T07:03:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.175+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d355160 0x55cc1d357550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:05.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.176+0000 7fbc2bfff640 10 client.20259 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:05.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.176+0000 7fbc50cb4640 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d355160 0x55cc1d357550 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffaa71b41fa3290 server_cookie=c94395f7c34a5efa in_seq=0 out_seq=0 2024-09-05T07:03:05.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.176+0000 7fbc2bfff640 1 -- 172.21.15.86:0/540514341 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cc1d234d60 con 0x55cc1d355160 2024-09-05T07:03:05.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.176+0000 7fbc2bfff640 10 client.20259 handle_command_reply: tid=0 2024-09-05T07:03:05.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.176+0000 7fbc552f1740 1 client.20259 shutdown 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 msgr2=0x55cc1d34d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbc380059f0 tx=0x7fbc38013040 comp rx=0 tx=0).stop 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 shutdown_connections 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55cc1d355160 0x55cc1d357550 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc1d2dae90 0x55cc1d2de330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc1d2c7ce0 0x55cc1d34d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 --2- 172.21.15.86:0/540514341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc1d2c73c0 0x55cc1d34d350 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 >> 172.21.15.86:0/540514341 conn(0x55cc1d2c3a30 msgr2=0x55cc1d2d7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc2bfff640 0 client.20259 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 shutdown_connections 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 1 -- 172.21.15.86:0/540514341 wait complete. 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.177+0000 7fbc552f1740 20 client.20259 trim_cache size 0 max 16384 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:03:05.179 INFO:teuthology.orchestra.run.smithi086.stdout: "3ab9887c-3140-4060-b22c-033f9d57f0b4": { 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3ab9887c-3140-4060-b22c-033f9d57f0b4", 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.178+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 msgr2=0x7fbc3c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.178+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 0x7fbc3c0a5300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbc4409fbc0 tx=0x7fbc4406ab70 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.178+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 msgr2=0x7fbc4c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.178+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c19ce20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009fd0 tx=0x7fbc4400ef10 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 shutdown_connections 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbc3c0a2e40 0x7fbc3c0a5300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4c1082a0 0x7fbc4c19ce20 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc4c1036e0 0x7fbc4c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 --2- 172.21.15.86:0/218233270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4c102d30 0x7fbc4c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 >> 172.21.15.86:0/218233270 conn(0x7fbc4c0fc7d0 msgr2=0x7fbc4c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 shutdown_connections 2024-09-05T07:03:05.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:05.179+0000 7fbc52f18640 1 -- 172.21.15.86:0/218233270 wait complete. 2024-09-05T07:03:05.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62330 6.64 MB/sec execute 383 sec latency 707.974 ms 2024-09-05T07:03:05.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:05 smithi115 ceph-mon[34791]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-09-05T07:03:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:05 smithi086 ceph-mon[30943]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-09-05T07:03:06.043 INFO:tasks.cephfs.filesystem:scrub status for tag:3ab9887c-3140-4060-b22c-033f9d57f0b4 - {'path': '/', 'tag': '3ab9887c-3140-4060-b22c-033f9d57f0b4', 'options': 'recursive,force'} 2024-09-05T07:03:06.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:05 smithi096 ceph-mon[34468]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 239 op/s 2024-09-05T07:03:06.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62475 6.64 MB/sec execute 384 sec latency 139.860 ms 2024-09-05T07:03:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:06 smithi115 ceph-mon[34791]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T07:03:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:06 smithi115 ceph-mon[34791]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 273 op/s 2024-09-05T07:03:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:06 smithi086 ceph-mon[30943]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T07:03:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:06 smithi086 ceph-mon[30943]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 273 op/s 2024-09-05T07:03:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:06 smithi096 ceph-mon[34468]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T07:03:07.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:06 smithi096 ceph-mon[34468]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 273 op/s 2024-09-05T07:03:07.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62651 6.65 MB/sec execute 385 sec latency 52.169 ms 2024-09-05T07:03:08.775 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62803 6.65 MB/sec execute 386 sec latency 85.001 ms 2024-09-05T07:03:09.056 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:08 smithi096 ceph-mon[34468]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:09.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:08 smithi115 ceph-mon[34791]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:09.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:08 smithi086 ceph-mon[30943]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:09.776 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62953 6.65 MB/sec execute 387 sec latency 223.662 ms 2024-09-05T07:03:10.776 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63091 6.64 MB/sec execute 388 sec latency 69.512 ms 2024-09-05T07:03:11.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:10 smithi096 ceph-mon[34468]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:10 smithi115 ceph-mon[34791]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:11.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:10 smithi086 ceph-mon[30943]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.4 MiB/s wr, 264 op/s 2024-09-05T07:03:11.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63250 6.65 MB/sec execute 389 sec latency 126.403 ms 2024-09-05T07:03:12.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63410 6.65 MB/sec execute 390 sec latency 128.648 ms 2024-09-05T07:03:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:13 smithi115 ceph-mon[34791]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-09-05T07:03:13.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:13 smithi115 ceph-mon[34791]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T07:03:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:13 smithi086 ceph-mon[30943]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-09-05T07:03:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:13 smithi086 ceph-mon[30943]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T07:03:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:13 smithi096 ceph-mon[34468]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-09-05T07:03:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:13 smithi096 ceph-mon[34468]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T07:03:13.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63596 6.66 MB/sec execute 391 sec latency 68.808 ms 2024-09-05T07:03:14.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63750 6.66 MB/sec execute 392 sec latency 63.091 ms 2024-09-05T07:03:15.433 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:15 smithi115 ceph-mon[34791]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 309 op/s 2024-09-05T07:03:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:15 smithi086 ceph-mon[30943]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 309 op/s 2024-09-05T07:03:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:15 smithi096 ceph-mon[34468]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 309 op/s 2024-09-05T07:03:15.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63907 6.66 MB/sec execute 393 sec latency 70.241 ms 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:03:16.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:16 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:03:16.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:16 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:03:16.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:16 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:03:16.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64083 6.67 MB/sec execute 394 sec latency 87.245 ms 2024-09-05T07:03:17.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:17 smithi086 ceph-mon[30943]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 249 op/s 2024-09-05T07:03:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:17 smithi096 ceph-mon[34468]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 249 op/s 2024-09-05T07:03:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:17 smithi115 ceph-mon[34791]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 249 op/s 2024-09-05T07:03:17.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64269 6.68 MB/sec execute 395 sec latency 282.802 ms 2024-09-05T07:03:18.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64452 6.68 MB/sec execute 396 sec latency 422.155 ms 2024-09-05T07:03:19.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:19 smithi086 ceph-mon[30943]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:19 smithi096 ceph-mon[34468]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:19 smithi115 ceph-mon[34791]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:19.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64613 6.67 MB/sec execute 397 sec latency 58.958 ms 2024-09-05T07:03:20.777 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64773 6.68 MB/sec execute 398 sec latency 103.152 ms 2024-09-05T07:03:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:21 smithi086 ceph-mon[30943]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:21 smithi096 ceph-mon[34468]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:21.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:21 smithi115 ceph-mon[34791]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 291 op/s 2024-09-05T07:03:21.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65009 6.71 MB/sec execute 399 sec latency 61.283 ms 2024-09-05T07:03:21.950 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T07:03:21.950 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T07:03:21.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:03:22.315 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:22.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65204 6.73 MB/sec execute 400 sec latency 97.316 ms 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 -- 172.21.15.86:0/328334043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a84101df0 msgr2=0x7f2a841021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/328334043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a84101df0 0x7f2a841021d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002a410 comp rx=0 tx=0).stop 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 -- 172.21.15.86:0/328334043 shutdown_connections 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/328334043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/328334043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841027a0 0x7f2a841067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/328334043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a84101df0 0x7f2a841021d0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 -- 172.21.15.86:0/328334043 >> 172.21.15.86:0/328334043 conn(0x7f2a840fd8b0 msgr2=0x7f2a840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 -- 172.21.15.86:0/328334043 shutdown_connections 2024-09-05T07:03:22.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.895+0000 7f2a8bccd640 1 -- 172.21.15.86:0/328334043 wait complete. 2024-09-05T07:03:22.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 Processor -- start 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 -- start start 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 0x7f2a841989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84111e30 con 0x7f2a841027a0 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84111cb0 con 0x7f2a84106f20 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8bccd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84111fb0 con 0x7f2a84101df0 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a89a69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a89268640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 0x7f2a841989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a89a69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55562/0 (socket says 172.21.15.86:55562) 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a89a69640 1 -- 172.21.15.86:0/3945791291 learned_addr learned my addr 172.21.15.86:0/3945791291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:22.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.896+0000 7f2a8a26a640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 -- 172.21.15.86:0/3945791291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 msgr2=0x7f2a84198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 -- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 msgr2=0x7f2a841989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 0x7f2a841989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 -- 172.21.15.86:0/3945791291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8419d430 con 0x7f2a84106f20 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a89268640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 0x7f2a841989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a89a69640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8a26a640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8419cd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009fd0 tx=0x7f2a7c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00ce20 con 0x7f2a84106f20 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a7c005020 con 0x7f2a84106f20 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8419d720 con 0x7f2a84106f20 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a84074420 con 0x7f2a84106f20 2024-09-05T07:03:22.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.897+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c010690 con 0x7f2a84106f20 2024-09-05T07:03:22.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.899+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68005290 con 0x7f2a84106f20 2024-09-05T07:03:22.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.901+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a7c004a40 con 0x7f2a84106f20 2024-09-05T07:03:22.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.902+0000 7f2a7affd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 0x7f2a700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:22.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.902+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f2a7c0cfdb0 con 0x7f2a84106f20 2024-09-05T07:03:22.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.902+0000 7f2a89a69640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 0x7f2a700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:22.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.903+0000 7f2a89a69640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 0x7f2a700a5230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005ab0 tx=0x7f2a80040040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:22.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:22.909+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a7c09c590 con 0x7f2a84106f20 2024-09-05T07:03:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.271+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a68005850 con 0x7f2a84106f20 2024-09-05T07:03:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.272+0000 7f2a7affd640 1 -- 172.21.15.86:0/3945791291 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f2a7c09be90 con 0x7f2a84106f20 2024-09-05T07:03:23.273 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:03:23.273 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:03:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 msgr2=0x7f2a700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 0x7f2a700a5230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005ab0 tx=0x7f2a80040040 comp rx=0 tx=0).stop 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 msgr2=0x7f2a8419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8419cd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009fd0 tx=0x7f2a7c00ef10 comp rx=0 tx=0).stop 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 shutdown_connections 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a700a2d70 0x7f2a700a5230 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84106f20 0x7f2a8419cd50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a841027a0 0x7f2a841989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:23.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 --2- 172.21.15.86:0/3945791291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a84101df0 0x7f2a84198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:23.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.274+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 >> 172.21.15.86:0/3945791291 conn(0x7f2a840fd8b0 msgr2=0x7f2a840fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:23.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.275+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 shutdown_connections 2024-09-05T07:03:23.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:23.275+0000 7f2a8bccd640 1 -- 172.21.15.86:0/3945791291 wait complete. 2024-09-05T07:03:23.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:23 smithi086 ceph-mon[30943]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 299 op/s 2024-09-05T07:03:23.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:23 smithi096 ceph-mon[34468]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 299 op/s 2024-09-05T07:03:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:23 smithi115 ceph-mon[34791]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 299 op/s 2024-09-05T07:03:23.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65346 6.73 MB/sec execute 401 sec latency 86.388 ms 2024-09-05T07:03:24.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T07:03:24.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T07:03:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:24.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3945791291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:24.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3945791291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:24.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3945791291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:24.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65491 6.74 MB/sec execute 402 sec latency 92.254 ms 2024-09-05T07:03:25.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:25 smithi086 ceph-mon[30943]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-05T07:03:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:25 smithi096 ceph-mon[34468]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-05T07:03:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:25 smithi115 ceph-mon[34791]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-05T07:03:25.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65612 6.74 MB/sec execute 403 sec latency 308.719 ms 2024-09-05T07:03:26.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65768 6.76 MB/sec execute 404 sec latency 296.221 ms 2024-09-05T07:03:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:27 smithi086 ceph-mon[30943]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-09-05T07:03:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:27 smithi096 ceph-mon[34468]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-09-05T07:03:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 -- 172.21.15.86:0/3921179409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870106f10 msgr2=0x7f387010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 --2- 172.21.15.86:0/3921179409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870106f10 0x7f387010ea90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3868009ca0 tx=0x7f386802a8c0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 -- 172.21.15.86:0/3921179409 shutdown_connections 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 --2- 172.21.15.86:0/3921179409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870106f10 0x7f387010ea90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 --2- 172.21.15.86:0/3921179409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870102790 0x7f38701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 --2- 172.21.15.86:0/3921179409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870101de0 0x7f38701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 -- 172.21.15.86:0/3921179409 >> 172.21.15.86:0/3921179409 conn(0x7f38700fd880 msgr2=0x7f38700ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 -- 172.21.15.86:0/3921179409 shutdown_connections 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 -- 172.21.15.86:0/3921179409 wait complete. 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.857+0000 7f38781fb640 1 Processor -- start 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 -- start start 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 0x7f3870198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 0x7f387019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3870111e20 con 0x7f3870101de0 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3870111ca0 con 0x7f3870106f10 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f38781fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3870111fa0 con 0x7f3870102790 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57986/0 (socket says 172.21.15.86:57986) 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 -- 172.21.15.86:0/1405981407 learned_addr learned my addr 172.21.15.86:0/1405981407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875796640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 0x7f3870198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3876798640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 0x7f387019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 -- 172.21.15.86:0/1405981407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 msgr2=0x7f3870198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 0x7f3870198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 -- 172.21.15.86:0/1405981407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 msgr2=0x7f387019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 0x7f387019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875f97640 1 -- 172.21.15.86:0/1405981407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387019d280 con 0x7f3870101de0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3876798640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 0x7f387019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.858+0000 7f3875796640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 0x7f3870198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f3875f97640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9f0 tx=0x7f386c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f3866ffd640 1 -- 172.21.15.86:0/1405981407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c00cd80 con 0x7f3870101de0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387019d570 con 0x7f3870101de0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f3866ffd640 1 -- 172.21.15.86:0/1405981407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f386c0054d0 con 0x7f3870101de0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f3866ffd640 1 -- 172.21.15.86:0/1405981407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c010490 con 0x7f3870101de0 2024-09-05T07:03:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.859+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38701a4e50 con 0x7f3870101de0 2024-09-05T07:03:27.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:27 smithi115 ceph-mon[34791]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-09-05T07:03:27.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.861+0000 7f3866ffd640 1 -- 172.21.15.86:0/1405981407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f386c005020 con 0x7f3870101de0 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.861+0000 7f387a5d4740 1 Processor -- start 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 -- start start 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4e21122a0 con 0x55f4e21b7940 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f4e20b9b10 con 0x55f4e21ad080 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f387a5d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4e210e260 con 0x55f4e21ac760 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3866ffd640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 0x7f385c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 0x7f385c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3875796640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3875f97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3866ffd640 1 -- 172.21.15.86:0/1405981407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f386c0cfca0 con 0x7f3870101de0 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55598/0 (socket says 172.21.15.86:55598) 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 -- 172.21.15.86:0/2834253206 learned_addr learned my addr 172.21.15.86:0/2834253206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 -- 172.21.15.86:0/2834253206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 msgr2=0x55f4e21b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 -- 172.21.15.86:0/2834253206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 msgr2=0x55f4e21b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.862+0000 7f3876798640 1 -- 172.21.15.86:0/2834253206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e1e83b90 con 0x55f4e21ac760 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f3876798640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 0x7f385c0a51b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3868006010 tx=0x7f3868009900 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f3876798640 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3868009560 tx=0x7f3868047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a3c3c638873ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f385bfff640 1 -- 172.21.15.86:0/2834253206 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386804d070 con 0x55f4e21ac760 2024-09-05T07:03:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f385bfff640 1 -- 172.21.15.86:0/2834253206 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3868036070 con 0x55f4e21ac760 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f385bfff640 1 -- 172.21.15.86:0/2834253206 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680305c0 con 0x55f4e21ac760 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f387a5d4740 1 -- 172.21.15.86:0/2834253206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 msgr2=0x55f4e21acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.863+0000 7f387a5d4740 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3868009560 tx=0x7f3868047000 comp rx=0 tx=0).stop 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- 172.21.15.86:0/2834253206 shutdown_connections 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- 172.21.15.86:0/2834253206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21acb40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- 172.21.15.86:0/2834253206 >> 172.21.15.86:0/2834253206 conn(0x55f4e21a8cf0 msgr2=0x55f4e21a90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- 172.21.15.86:0/2834253206 shutdown_connections 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- 172.21.15.86:0/2834253206 wait complete. 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- start start 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 Processor -- start 2024-09-05T07:03:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4e21ba390 con 0x55f4e21b7940 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f4e209a6e0 con 0x55f4e21ad080 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.864+0000 7f387a5d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4e21ba510 con 0x55f4e21ac760 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f387a5d4740 10 client.0 _ll_register_callbacks cb 0x55f4e20bea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57996/0 (socket says 172.21.15.86:57996) 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 -- 172.21.15.86:0/694628014 learned_addr learned my addr 172.21.15.86:0/694628014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3875f97640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3875796640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 -- 172.21.15.86:0/694628014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 msgr2=0x55f4e21b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 -- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 msgr2=0x55f4e21b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 -- 172.21.15.86:0/694628014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e21dd0c0 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3875f97640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f3876798640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3868004940 tx=0x7f3868004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804ca77c4fdf6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f38597fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868058070 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3868036070 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f3868003af0 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3868064050 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f387a5d4740 10 client.20277 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.865+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f4e21ba390 con 0x55f4e21b7940 2024-09-05T07:03:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3868036070 con 0x55f4e21b7940 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f3868052450 con 0x55f4e21b7940 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 10 client.20277 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 10 client.20277 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 20 client.20277 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e21e0bb0 0x55f4e21e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 4 client.20277 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4e211f9d0 con 0x55f4e21e0bb0 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f3875f97640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e21e0bb0 0x55f4e21e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f38597fa640 10 client.20277 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.866+0000 7f3875f97640 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e21e0bb0 0x55f4e21e2fa0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62afa53568518dca server_cookie=2a2c7ef0f5d5c71 in_seq=0 out_seq=0 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f38597fa640 1 -- 172.21.15.86:0/694628014 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4e211f9d0 con 0x55f4e21e0bb0 2024-09-05T07:03:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f38597fa640 10 client.20277 handle_command_reply: tid=0 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 client.20277 shutdown 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 msgr2=0x55f4e21b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3868004940 tx=0x7f3868004600 comp rx=0 tx=0).stop 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 shutdown_connections 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e21e0bb0 0x55f4e21e2fa0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21b5260 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f38597fa640 0 client.20277 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 --2- 172.21.15.86:0/694628014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 >> 172.21.15.86:0/694628014 conn(0x55f4e21a8cf0 msgr2=0x55f4e21aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 shutdown_connections 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 1 -- 172.21.15.86:0/694628014 wait complete. 2024-09-05T07:03:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.867+0000 7f387a5d4740 20 client.20277 trim_cache size 0 max 16384 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 Processor -- start 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 -- start start 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 0x55f4e21b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 0x55f4e21af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3870075d00 con 0x55f4e21ac760 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38701a62a0 con 0x55f4e21ad080 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f387a5d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38700763d0 con 0x55f4e21b7940 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f3875f97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 0x55f4e21b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f3875f97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 0x55f4e21b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58008/0 (socket says 172.21.15.86:58008) 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f3875f97640 1 -- 172.21.15.86:0/1649509747 learned_addr learned my addr 172.21.15.86:0/1649509747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f3876798640 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 0x55f4e21af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.891+0000 7f3875796640 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 -- 172.21.15.86:0/1649509747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 msgr2=0x55f4e21b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 -- 172.21.15.86:0/1649509747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 msgr2=0x55f4e21b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 0x55f4e21b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 -- 172.21.15.86:0/1649509747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e21e35d0 con 0x55f4e21b7940 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f3876798640 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 0x55f4e21af1c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3868004910 tx=0x7f3868004670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b9d7f2ff3c8fea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f385b7fe640 1 -- 172.21.15.86:0/1649509747 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868058070 con 0x55f4e21b7940 2024-09-05T07:03:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f385b7fe640 1 -- 172.21.15.86:0/1649509747 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3868036040 con 0x55f4e21b7940 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.892+0000 7f385b7fe640 1 -- 172.21.15.86:0/1649509747 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386804d040 con 0x55f4e21b7940 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 -- 172.21.15.86:0/1649509747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 msgr2=0x55f4e21af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 0x55f4e21af1c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3868004910 tx=0x7f3868004670 comp rx=0 tx=0).stop 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 -- 172.21.15.86:0/1649509747 shutdown_connections 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21b7940 0x55f4e21af1c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1649509747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21ac760 0x55f4e21b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 -- 172.21.15.86:0/1649509747 >> 172.21.15.86:0/1649509747 conn(0x55f4e21a8cf0 msgr2=0x55f4e21a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 -- 172.21.15.86:0/1649509747 shutdown_connections 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.893+0000 7f387a5d4740 1 -- 172.21.15.86:0/1649509747 wait complete. 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.894+0000 7f387a5d4740 1 -- start start 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.894+0000 7f387a5d4740 1 Processor -- start 2024-09-05T07:03:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.894+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.894+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f387a5d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f387a5d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38701a5bd0 con 0x55f4e21b7940 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f387a5d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38701a5500 con 0x55f4e21ad080 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f387a5d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3870073d00 con 0x55f4e21ac760 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f387a5d4740 10 client.0 _ll_register_callbacks cb 0x7f386c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3876798640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875f97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3876798640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58012/0 (socket says 172.21.15.86:58012) 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3876798640 1 -- 172.21.15.86:0/1195172345 learned_addr learned my addr 172.21.15.86:0/1195172345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 -- 172.21.15.86:0/1195172345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 msgr2=0x55f4e21c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 -- 172.21.15.86:0/1195172345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 msgr2=0x55f4e21c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 -- 172.21.15.86:0/1195172345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e21a9980 con 0x55f4e21ad080 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3876798640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f3875796640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21c4cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55f4e21b5d90 tx=0x7f38600096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e3404ef199c076 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.895+0000 7f38597fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860023060 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3868036040 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f3860004230 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f386002a050 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f387a5d4740 10 client.29018 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f4e21a4260 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.896+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f386000fe60 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f3860003c10 con 0x55f4e21ad080 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 10 client.29018 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 10 client.29018 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 20 client.29018 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e224c700 0x55f4e224eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 4 client.29018 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f3868036040 con 0x55f4e224c700 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f3875f97640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e224c700 0x55f4e224eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f38597fa640 10 client.29018 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:26.897+0000 7f3875f97640 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e224c700 0x55f4e224eb20 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a398e18e059f920c server_cookie=d7d4bc673feada4c in_seq=0 out_seq=0 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f38597fa640 1 -- 172.21.15.86:0/1195172345 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f3868036040 con 0x55f4e224c700 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f38597fa640 10 client.29018 handle_command_reply: tid=0 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 client.29018 shutdown 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 msgr2=0x55f4e21c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21c4cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55f4e21b5d90 tx=0x7f38600096c0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 shutdown_connections 2024-09-05T07:03:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55f4e224c700 0x55f4e224eb20 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 46, 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 82, 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "226972e1": { 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 82, 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:27.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:27.703 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4e21b7940 0x55f4e21c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f38597fa640 0 client.29018 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4e21ad080 0x55f4e21c4cb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 --2- 172.21.15.86:0/1195172345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4e21ac760 0x55f4e21c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 >> 172.21.15.86:0/1195172345 conn(0x55f4e21a8cf0 msgr2=0x55f4e21b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 shutdown_connections 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.167+0000 7f387a5d4740 1 -- 172.21.15.86:0/1195172345 wait complete. 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f387a5d4740 20 client.29018 trim_cache size 0 max 16384 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 msgr2=0x7f385c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 0x7f385c0a51b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3868006010 tx=0x7f3868009900 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 msgr2=0x7f38701982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9f0 tx=0x7f386c00edf0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 shutdown_connections 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f385c0a2cf0 0x7f385c0a51b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3870106f10 0x7f387019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3870102790 0x7f3870198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 --2- 172.21.15.86:0/1405981407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3870101de0 0x7f38701982d0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 >> 172.21.15.86:0/1405981407 conn(0x7f38700fd880 msgr2=0x7f38700febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 shutdown_connections 2024-09-05T07:03:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:27.168+0000 7f38781fb640 1 -- 172.21.15.86:0/1405981407 wait complete. 2024-09-05T07:03:27.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.74 MB/sec execute 405 sec latency 876.562 ms 2024-09-05T07:03:28.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.73 MB/sec execute 406 sec latency 1876.627 ms 2024-09-05T07:03:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:29 smithi086 ceph-mon[30943]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 349 op/s 2024-09-05T07:03:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:29 smithi096 ceph-mon[34468]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 349 op/s 2024-09-05T07:03:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:29 smithi115 ceph-mon[34791]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.6 MiB/s wr, 349 op/s 2024-09-05T07:03:29.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.71 MB/sec execute 407 sec latency 2876.753 ms 2024-09-05T07:03:30.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.69 MB/sec execute 408 sec latency 3876.848 ms 2024-09-05T07:03:31.229 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '226972e1', quiesce took 9.3 seconds. Will release after: 20.7 2024-09-05T07:03:31.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:31 smithi115 ceph-mon[34791]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 216 op/s 2024-09-05T07:03:31.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:31 smithi086 ceph-mon[30943]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 216 op/s 2024-09-05T07:03:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:31 smithi096 ceph-mon[34468]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 216 op/s 2024-09-05T07:03:31.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.68 MB/sec execute 409 sec latency 4876.984 ms 2024-09-05T07:03:32.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.66 MB/sec execute 410 sec latency 5877.088 ms 2024-09-05T07:03:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:33 smithi086 ceph-mon[30943]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 264 op/s 2024-09-05T07:03:33.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:33 smithi096 ceph-mon[34468]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 264 op/s 2024-09-05T07:03:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:33 smithi115 ceph-mon[34791]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 264 op/s 2024-09-05T07:03:33.778 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.64 MB/sec execute 411 sec latency 6877.187 ms 2024-09-05T07:03:34.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.63 MB/sec execute 412 sec latency 7877.289 ms 2024-09-05T07:03:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:35 smithi086 ceph-mon[30943]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-09-05T07:03:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:35 smithi096 ceph-mon[34468]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-09-05T07:03:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:35 smithi115 ceph-mon[34791]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-09-05T07:03:35.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.61 MB/sec execute 413 sec latency 8877.384 ms 2024-09-05T07:03:36.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:03:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:36.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.60 MB/sec execute 414 sec latency 9877.459 ms 2024-09-05T07:03:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 -- 172.21.15.86:0/3165663560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c103720 msgr2=0x7fa80c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 --2- 172.21.15.86:0/3165663560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c103720 0x7fa80c10bc10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009900 tx=0x7fa7fc02a480 comp rx=0 tx=0).stop 2024-09-05T07:03:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 -- 172.21.15.86:0/3165663560 shutdown_connections 2024-09-05T07:03:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 --2- 172.21.15.86:0/3165663560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c10c1a0 0x7fa80c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 --2- 172.21.15.86:0/3165663560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c103720 0x7fa80c10bc10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 --2- 172.21.15.86:0/3165663560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c102e00 0x7fa80c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 -- 172.21.15.86:0/3165663560 >> 172.21.15.86:0/3165663560 conn(0x7fa80c0fc7a0 msgr2=0x7fa80c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 -- 172.21.15.86:0/3165663560 shutdown_connections 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.805+0000 7fa8142e9640 1 -- 172.21.15.86:0/3165663560 wait complete. 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 Processor -- start 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 -- start start 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c103720 0x7fa80c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c10c1a0 0x7fa80c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c111e20 con 0x7fa80c102e00 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c111ca0 con 0x7fa80c103720 2024-09-05T07:03:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa8142e9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c111fa0 con 0x7fa80c10c1a0 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59550/0 (socket says 172.21.15.86:59550) 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa812085640 1 -- 172.21.15.86:0/1239384091 learned_addr learned my addr 172.21.15.86:0/1239384091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.806+0000 7fa811884640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c103720 0x7fa80c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812886640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c10c1a0 0x7fa80c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 -- 172.21.15.86:0/1239384091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c10c1a0 msgr2=0x7fa80c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c10c1a0 0x7fa80c1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 -- 172.21.15.86:0/1239384091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c103720 msgr2=0x7fa80c100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c103720 0x7fa80c100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 -- 172.21.15.86:0/1239384091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c1a68c0 con 0x7fa80c102e00 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa812085640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9f0 tx=0x7fa80800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa8037fe640 1 -- 172.21.15.86:0/1239384091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800cd80 con 0x7fa80c102e00 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa8037fe640 1 -- 172.21.15.86:0/1239384091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8080054d0 con 0x7fa80c102e00 2024-09-05T07:03:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c1a6bb0 con 0x7fa80c102e00 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa8037fe640 1 -- 172.21.15.86:0/1239384091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808010490 con 0x7fa80c102e00 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.807+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c1a70c0 con 0x7fa80c102e00 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.809+0000 7fa8037fe640 1 -- 172.21.15.86:0/1239384091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa808005020 con 0x7fa80c102e00 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 Processor -- start 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 -- start start 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a21e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a229140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d6a1842e0 con 0x560d6a229680 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d6a12bce0 con 0x560d6a21e4a0 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8166c2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d6a1802a0 con 0x560d6a21edc0 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa811884640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa811884640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59554/0 (socket says 172.21.15.86:59554) 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa8037fe640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.810+0000 7fa811884640 1 -- 172.21.15.86:0/1774069351 learned_addr learned my addr 172.21.15.86:0/1774069351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa812085640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a229140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa812886640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a21e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa812886640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa8037fe640 1 -- 172.21.15.86:0/1239384091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fa8080cfc80 con 0x7fa80c102e00 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 -- 172.21.15.86:0/1774069351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 msgr2=0x560d6a229140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a229140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 -- 172.21.15.86:0/1774069351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 msgr2=0x560d6a21e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a21e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 -- 172.21.15.86:0/1774069351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6a110440 con 0x560d6a229680 2024-09-05T07:03:36.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa812085640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a229140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa811884640 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009900 tx=0x7fa7fc032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856bc8b7b8e03ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa812886640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 0x7fa7f80a5230 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa804009370 tx=0x7fa804009240 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa800dea640 1 -- 172.21.15.86:0/1774069351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc040070 con 0x560d6a229680 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa800dea640 1 -- 172.21.15.86:0/1774069351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7fc004030 con 0x560d6a229680 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.811+0000 7fa800dea640 1 -- 172.21.15.86:0/1774069351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc035040 con 0x560d6a229680 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- 172.21.15.86:0/1774069351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 msgr2=0x560d6a22ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009900 tx=0x7fa7fc032bb0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- 172.21.15.86:0/1774069351 shutdown_connections 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a22ba70 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a229140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 --2- 172.21.15.86:0/1774069351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a21e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- 172.21.15.86:0/1774069351 >> 172.21.15.86:0/1774069351 conn(0x560d6a21aa30 msgr2=0x560d6a21ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- 172.21.15.86:0/1774069351 shutdown_connections 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- 172.21.15.86:0/1774069351 wait complete. 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 -- start start 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.812+0000 7fa8166c2740 1 Processor -- start 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a2261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a226720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d6a22c250 con 0x560d6a21e4a0 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d6a13f050 con 0x560d6a21edc0 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560d6a22c3d0 con 0x560d6a229680 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 10 client.0 _ll_register_callbacks cb 0x560d6a17f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59564/0 (socket says 172.21.15.86:59564) 2024-09-05T07:03:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 -- 172.21.15.86:0/2774275139 learned_addr learned my addr 172.21.15.86:0/2774275139 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812886640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a226720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa811884640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a2261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 -- 172.21.15.86:0/2774275139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 msgr2=0x560d6a226720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a226720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 -- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 msgr2=0x560d6a2261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a2261e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 -- 172.21.15.86:0/2774275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d69ef53c0 con 0x560d6a21e4a0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa811884640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a2261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa812085640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa80809ba00 tx=0x7fa80800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55d3cdd631c5be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa7f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080cd3b0 con 0x560d6a21e4a0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fa80809bf60 con 0x560d6a21e4a0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7fc004030 con 0x560d6a21e4a0 2024-09-05T07:03:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa80809c400 con 0x560d6a21e4a0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 10 client.20295 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.813+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560d6a237b60 con 0x560d6a21e4a0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7fc004030 con 0x560d6a21e4a0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fa8080a44e0 con 0x560d6a21e4a0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 10 client.20295 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 10 client.20295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 20 client.20295 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a239070 0x560d6a23b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 4 client.20295 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d6a191850 con 0x560d6a239070 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.814+0000 7fa811884640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a239070 0x560d6a23b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa811884640 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a239070 0x560d6a23b4e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b81551da202c2b server_cookie=269160e30a9ad57a in_seq=0 out_seq=0 2024-09-05T07:03:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa7f5ffb640 10 client.20295 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:36.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/2774275139 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d6a191850 con 0x560d6a239070 2024-09-05T07:03:36.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa7f5ffb640 10 client.20295 handle_command_reply: tid=0 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa8166c2740 1 client.20295 shutdown 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.815+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 msgr2=0x560d6a228f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa80809ba00 tx=0x7fa80800a440 comp rx=0 tx=0).stop 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 shutdown_connections 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a239070 0x560d6a23b4e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a226720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a2261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 --2- 172.21.15.86:0/2774275139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a228f30 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 >> 172.21.15.86:0/2774275139 conn(0x560d6a21aa30 msgr2=0x560d6a220710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa7f5ffb640 0 client.20295 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 shutdown_connections 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 1 -- 172.21.15.86:0/2774275139 wait complete. 2024-09-05T07:03:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.816+0000 7fa8166c2740 20 client.20295 trim_cache size 0 max 16384 2024-09-05T07:03:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 Processor -- start 2024-09-05T07:03:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 -- start start 2024-09-05T07:03:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a240db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a2412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b1250 con 0x560d6a229680 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b2d90 con 0x560d6a21e4a0 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.832+0000 7fa8166c2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b1920 con 0x560d6a21edc0 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59580/0 (socket says 172.21.15.86:59580) 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 -- 172.21.15.86:0/4139673655 learned_addr learned my addr 172.21.15.86:0/4139673655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa811884640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a2412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812085640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a240db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 -- 172.21.15.86:0/4139673655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 msgr2=0x560d6a2412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a2412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 -- 172.21.15.86:0/4139673655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 msgr2=0x560d6a240db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a240db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 -- 172.21.15.86:0/4139673655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6a21a4d0 con 0x560d6a229680 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa811884640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a2412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa812886640 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa8040049d0 tx=0x7fa8040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bb8528715c38c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa7f7fff640 1 -- 172.21.15.86:0/4139673655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804024030 con 0x560d6a229680 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa7f7fff640 1 -- 172.21.15.86:0/4139673655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa804019040 con 0x560d6a229680 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.833+0000 7fa7f7fff640 1 -- 172.21.15.86:0/4139673655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040209d0 con 0x560d6a229680 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- 172.21.15.86:0/4139673655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 msgr2=0x560d6a223440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa8040049d0 tx=0x7fa8040095c0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- 172.21.15.86:0/4139673655 shutdown_connections 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a229680 0x560d6a223440 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a21edc0 0x560d6a2412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- 172.21.15.86:0/4139673655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21e4a0 0x560d6a240db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- 172.21.15.86:0/4139673655 >> 172.21.15.86:0/4139673655 conn(0x560d6a21aa30 msgr2=0x560d6a219a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- 172.21.15.86:0/4139673655 shutdown_connections 2024-09-05T07:03:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- 172.21.15.86:0/4139673655 wait complete. 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 -- start start 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 Processor -- start 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a22ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.834+0000 7fa8166c2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a22f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b26c0 con 0x560d6a21e4a0 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b1ff0 con 0x560d6a21edc0 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1af250 con 0x560d6a229680 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 10 client.0 _ll_register_callbacks cb 0x7fa808004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59592/0 (socket says 172.21.15.86:59592) 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 -- 172.21.15.86:0/3868266510 learned_addr learned my addr 172.21.15.86:0/3868266510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812886640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a22f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa811884640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a22ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 -- 172.21.15.86:0/3868266510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 msgr2=0x560d6a22f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a22f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 -- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 msgr2=0x560d6a22ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a22ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 -- 172.21.15.86:0/3868266510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6a22faa0 con 0x560d6a21e4a0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812886640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a22f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa812085640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa808009230 tx=0x7fa8080cd460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b744532cf1381d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa7f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80809bad0 con 0x560d6a21e4a0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fa80809b360 con 0x560d6a21e4a0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa804019040 con 0x560d6a21e4a0 2024-09-05T07:03:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa8080a4970 con 0x560d6a21e4a0 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 10 client.20301 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.835+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560d6a2a3bd0 con 0x560d6a21e4a0 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa804019040 con 0x560d6a21e4a0 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fa8080a4bc0 con 0x560d6a21e4a0 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 10 client.20301 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 10 client.20301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 20 client.20301 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a2a41b0 0x560d6a2a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 4 client.20301 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d6a18bd60 con 0x560d6a2a41b0 2024-09-05T07:03:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa811884640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a2a41b0 0x560d6a2a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa811884640 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a2a41b0 0x560d6a2a65a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabcc914357abc3a server_cookie=3e74b1dac4bf3dab in_seq=0 out_seq=0 2024-09-05T07:03:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.836+0000 7fa7f5ffb640 10 client.20301 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/3868266510 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d6a18bd60 con 0x560d6a2a41b0 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa7f5ffb640 10 client.20301 handle_command_reply: tid=0 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 client.20301 shutdown 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 msgr2=0x560d6a231bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa808009230 tx=0x7fa8080cd460 comp rx=0 tx=0).stop 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 shutdown_connections 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560d6a2a41b0 0x560d6a2a65a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560d6a229680 0x560d6a22f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d6a21edc0 0x560d6a22ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 --2- 172.21.15.86:0/3868266510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d6a21e4a0 0x560d6a231bd0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 >> 172.21.15.86:0/3868266510 conn(0x560d6a21aa30 msgr2=0x560d6a219a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa7f5ffb640 0 client.20301 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 shutdown_connections 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 1 -- 172.21.15.86:0/3868266510 wait complete. 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8166c2740 20 client.20301 trim_cache size 0 max 16384 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:03:36.844 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 msgr2=0x7fa7f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 0x7fa7f80a5230 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa804009370 tx=0x7fa804009240 comp rx=0 tx=0).stop 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 msgr2=0x7fa80c100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.837+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9f0 tx=0x7fa80800edf0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 shutdown_connections 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c10c1a0 0x7fa80c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa80c103720 0x7fa80c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 --2- 172.21.15.86:0/1239384091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80c102e00 0x7fa80c100a60 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 >> 172.21.15.86:0/1239384091 conn(0x7fa80c0fc7a0 msgr2=0x7fa80c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 shutdown_connections 2024-09-05T07:03:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:36.838+0000 7fa8142e9640 1 -- 172.21.15.86:0/1239384091 wait complete. 2024-09-05T07:03:37.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:37 smithi086 ceph-mon[30943]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 146 op/s 2024-09-05T07:03:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:37 smithi096 ceph-mon[34468]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 146 op/s 2024-09-05T07:03:37.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:03:37.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:03:37.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:37 smithi115 ceph-mon[34791]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 146 op/s 2024-09-05T07:03:37.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.58 MB/sec execute 415 sec latency 10877.514 ms 2024-09-05T07:03:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 -- 172.21.15.86:0/1134090279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004106f10 msgr2=0x7f300410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 --2- 172.21.15.86:0/1134090279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004106f10 0x7f300410ea90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009ca0 tx=0x7f2ffc02a8c0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 -- 172.21.15.86:0/1134090279 shutdown_connections 2024-09-05T07:03:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 --2- 172.21.15.86:0/1134090279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004106f10 0x7f300410ea90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 --2- 172.21.15.86:0/1134090279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 0x7f30041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 --2- 172.21.15.86:0/1134090279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004101de0 0x7f30041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 -- 172.21.15.86:0/1134090279 >> 172.21.15.86:0/1134090279 conn(0x7f30040fd8c0 msgr2=0x7f30040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 -- 172.21.15.86:0/1134090279 shutdown_connections 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.362+0000 7f300bb57640 1 -- 172.21.15.86:0/1134090279 wait complete. 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 Processor -- start 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 -- start start 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 0x7f3004198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 0x7f300419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3004111e20 con 0x7f3004101de0 2024-09-05T07:03:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3004111ca0 con 0x7f3004102790 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300bb57640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3004111fa0 con 0x7f3004106f10 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59608/0 (socket says 172.21.15.86:59608) 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 -- 172.21.15.86:0/227626618 learned_addr learned my addr 172.21.15.86:0/227626618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30090f2640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 0x7f3004198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300a0f4640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 0x7f300419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 -- 172.21.15.86:0/227626618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 msgr2=0x7f300419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 0x7f300419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 -- 172.21.15.86:0/227626618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 msgr2=0x7f3004198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 0x7f3004198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 -- 172.21.15.86:0/227626618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300419d2d0 con 0x7f3004101de0 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f300a0f4640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 0x7f300419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.363+0000 7f30098f3640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f300000e9f0 tx=0x7f300000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.364+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300000cd80 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.364+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30000054d0 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.364+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300419d5c0 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.364+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010490 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.364+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30041a4ea0 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.365+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30041021c0 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.366+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3000005020 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.366+0000 7f2ffaffd640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 0x7f2ff00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.367+0000 7f30090f2640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 0x7f2ff00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.367+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f30000cfca0 con 0x7f3004101de0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.367+0000 7f30090f2640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 0x7f2ff00a51b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f30040fdce0 tx=0x7f2ff400b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.371+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f300009c480 con 0x7f3004101de0 2024-09-05T07:03:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.569+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f300410a1b0 con 0x7f3004101de0 2024-09-05T07:03:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.569+0000 7f2ffaffd640 1 -- 172.21.15.86:0/227626618 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f300009bd80 con 0x7f3004101de0 2024-09-05T07:03:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:03:38.570 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:03:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 msgr2=0x7f2ff00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 0x7f2ff00a51b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f30040fdce0 tx=0x7f2ff400b390 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 msgr2=0x7f3004198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f300000e9f0 tx=0x7f300000edf0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 shutdown_connections 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2ff00a2cf0 0x7f2ff00a51b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106f10 0x7f300419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3004102790 0x7f3004198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 --2- 172.21.15.86:0/227626618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3004101de0 0x7f3004198320 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 >> 172.21.15.86:0/227626618 conn(0x7f30040fd8c0 msgr2=0x7f30040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 shutdown_connections 2024-09-05T07:03:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:38.571+0000 7f300bb57640 1 -- 172.21.15.86:0/227626618 wait complete. 2024-09-05T07:03:38.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.56 MB/sec execute 416 sec latency 11877.617 ms 2024-09-05T07:03:39.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:03:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:39 smithi086 ceph-mon[30943]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T07:03:39.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:39 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/227626618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:39.507 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:39 smithi096 ceph-mon[34468]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T07:03:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:39 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/227626618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:39.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:39 smithi115 ceph-mon[34791]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T07:03:39.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:39 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/227626618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:39.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.55 MB/sec execute 417 sec latency 12877.660 ms 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 -- 172.21.15.86:0/3707605875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4101420 msgr2=0x7f4ef410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/3707605875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4101420 0x7f4ef410d950 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009cd0 tx=0x7f4eec02a8e0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 -- 172.21.15.86:0/3707605875 shutdown_connections 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/3707605875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4101420 0x7f4ef410d950 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/3707605875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4100a80 0x7f4ef4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/3707605875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4106a80 0x7f4ef4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 -- 172.21.15.86:0/3707605875 >> 172.21.15.86:0/3707605875 conn(0x7f4ef40fc7a0 msgr2=0x7f4ef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 -- 172.21.15.86:0/3707605875 shutdown_connections 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:39.999+0000 7f4ef8f93640 1 -- 172.21.15.86:0/3707605875 wait complete. 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 Processor -- start 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 -- start start 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4101420 0x7f4ef4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 0x7f4ef41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4110d30 con 0x7f4ef4100a80 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4110bb0 con 0x7f4ef4106a80 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef8f93640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4110eb0 con 0x7f4ef4101420 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59636/0 (socket says 172.21.15.86:59636) 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.000+0000 7f4ef259c640 1 -- 172.21.15.86:0/1469542872 learned_addr learned my addr 172.21.15.86:0/1469542872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4101420 0x7f4ef4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 0x7f4ef41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 -- 172.21.15.86:0/1469542872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4101420 msgr2=0x7f4ef4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4101420 0x7f4ef4199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 -- 172.21.15.86:0/1469542872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 msgr2=0x7f4ef41a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 0x7f4ef41a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 -- 172.21.15.86:0/1469542872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef41a4780 con 0x7f4ef4100a80 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 0x7f4ef41a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef259c640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d770 tx=0x7f4ee400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ee37fe640 1 -- 172.21.15.86:0/1469542872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4017070 con 0x7f4ef4100a80 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ee37fe640 1 -- 172.21.15.86:0/1469542872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee40054d0 con 0x7f4ef4100a80 2024-09-05T07:03:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef41a4950 con 0x7f4ef4100a80 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ee37fe640 1 -- 172.21.15.86:0/1469542872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004cf0 con 0x7f4ef4100a80 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.001+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef41a4e60 con 0x7f4ef4100a80 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.003+0000 7f4ee37fe640 1 -- 172.21.15.86:0/1469542872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4ee4004e90 con 0x7f4ef4100a80 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 Processor -- start 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 -- start start 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ee37fe640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 0x7f4edc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3ea9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f3ea012f0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f3e9a8cf0 con 0x563f3eaa6370 2024-09-05T07:03:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4efb36c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f3e9fd2b0 con 0x563f3ea9cb40 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ee37fe640 1 -- 172.21.15.86:0/1469542872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f4ee40cf2f0 con 0x7f4ef4100a80 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 0x7f4edc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59642/0 (socket says 172.21.15.86:59642) 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/1917016162 learned_addr learned my addr 172.21.15.86:0/1917016162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef259c640 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3ea9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/1917016162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 msgr2=0x563f3ea9b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3ea9b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/1917016162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 msgr2=0x563f3eaa8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.004+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/1917016162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3e98d3a0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.005+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 0x7f4edc0a5230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4eec006010 tx=0x7f4eec031470 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.005+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009b10 tx=0x7f4ee800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d186085927075c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.005+0000 7f4ee0ff9640 1 -- 172.21.15.86:0/1917016162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8016070 con 0x563f3ea9b5d0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.005+0000 7f4ee0ff9640 1 -- 172.21.15.86:0/1917016162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee80054d0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.005+0000 7f4ee0ff9640 1 -- 172.21.15.86:0/1917016162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8011470 con 0x563f3ea9b5d0 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- 172.21.15.86:0/1917016162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 msgr2=0x563f3ea9ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009b10 tx=0x7f4ee800e2f0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- 172.21.15.86:0/1917016162 shutdown_connections 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea9ba10 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- 172.21.15.86:0/1917016162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3ea9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- 172.21.15.86:0/1917016162 >> 172.21.15.86:0/1917016162 conn(0x563f3ea64310 msgr2=0x563f3ea97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- 172.21.15.86:0/1917016162 shutdown_connections 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- 172.21.15.86:0/1917016162 wait complete. 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- start start 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 Processor -- start 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 0x563f3eaa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3eaa6370 0x563f3eaa34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f3eaad480 con 0x563f3ea9b5d0 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f3e9bc060 con 0x563f3ea9cb40 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f3eaad600 con 0x563f3eaa6370 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4efb36c740 10 client.0 _ll_register_callbacks cb 0x563f3e70f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.006+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59656/0 (socket says 172.21.15.86:59656) 2024-09-05T07:03:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 -- 172.21.15.86:0/3069693598 learned_addr learned my addr 172.21.15.86:0/3069693598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 0x563f3eaa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 -- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3eaa6370 msgr2=0x563f3eaa34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3eaa6370 0x563f3eaa34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3eaa6370 0x563f3eaa34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 -- 172.21.15.86:0/3069693598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 msgr2=0x563f3eaa2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 0x563f3eaa2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 -- 172.21.15.86:0/3069693598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3e7723c0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 0x563f3eaa2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005c20 tx=0x7f4ee409adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a278e064d22a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4020040 con 0x563f3ea9b5d0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f4ee40bb270 con 0x563f3ea9b5d0 2024-09-05T07:03:40.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee80054d0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4ee409b370 con 0x563f3ea9b5d0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4efb36c740 10 client.20319 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563f3eab8f50 con 0x563f3ea9b5d0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee80054d0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.007+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f4ee40cc820 con 0x563f3ea9b5d0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 10 client.20319 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 10 client.20319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 20 client.20319 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eaba400 0x563f3eabc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 4 client.20319 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f3ea0e860 con 0x563f3eaba400 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eaba400 0x563f3eabc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eaba400 0x563f3eabc8a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41bf6e82f6e40342 server_cookie=d3abd6a61e88f82 in_seq=0 out_seq=0 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.008+0000 7f4ed9ffb640 10 client.20319 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3069693598 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f3ea0e860 con 0x563f3eaba400 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4ed9ffb640 10 client.20319 handle_command_reply: tid=0 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 client.20319 shutdown 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 msgr2=0x563f3eaa5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005c20 tx=0x7f4ee409adf0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 shutdown_connections 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eaba400 0x563f3eabc8a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3eaa6370 0x563f3eaa34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3ea9cb40 0x563f3eaa2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3069693598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3eaa5cc0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 >> 172.21.15.86:0/3069693598 conn(0x563f3ea64310 msgr2=0x563f3ea9d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4ed9ffb640 0 client.20319 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 shutdown_connections 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.009+0000 7f4efb36c740 1 -- 172.21.15.86:0/3069693598 wait complete. 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.010+0000 7f4efb36c740 20 client.20319 trim_cache size 0 max 16384 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 Processor -- start 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 -- start start 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3eab6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eac2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41acdc0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41ae900 con 0x563f3eaa6370 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4efb36c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41ad490 con 0x563f3ea9cb40 2024-09-05T07:03:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59670/0 (socket says 172.21.15.86:59670) 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.022+0000 7f4ef259c640 1 -- 172.21.15.86:0/3437622214 learned_addr learned my addr 172.21.15.86:0/3437622214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eac2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 -- 172.21.15.86:0/3437622214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 msgr2=0x563f3eab6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3eab6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3eab6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 -- 172.21.15.86:0/3437622214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 msgr2=0x563f3eac2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eac2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 -- 172.21.15.86:0/3437622214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3eaa1ed0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3eab6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4ee409b070 tx=0x7f4ee4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e0bccda795c0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4edbfff640 1 -- 172.21.15.86:0/3437622214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40b9070 con 0x563f3ea9b5d0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4edbfff640 1 -- 172.21.15.86:0/3437622214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee40b4280 con 0x563f3ea9b5d0 2024-09-05T07:03:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4edbfff640 1 -- 172.21.15.86:0/3437622214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4020050 con 0x563f3ea9b5d0 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 -- 172.21.15.86:0/3437622214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 msgr2=0x563f3ea97e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4ee409b070 tx=0x7f4ee4005bb0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 -- 172.21.15.86:0/3437622214 shutdown_connections 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eac2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9cb40 0x563f3eab6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3437622214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9b5d0 0x563f3ea97e10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.023+0000 7f4efb36c740 1 -- 172.21.15.86:0/3437622214 >> 172.21.15.86:0/3437622214 conn(0x563f3ea64310 msgr2=0x563f3ea97700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- 172.21.15.86:0/3437622214 shutdown_connections 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- 172.21.15.86:0/3437622214 wait complete. 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- start start 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 Processor -- start 2024-09-05T07:03:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9b5d0 0x563f3eab3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41ae230 con 0x563f3ea9cb40 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41adb60 con 0x563f3eaa6370 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41aadc0 con 0x563f3ea9b5d0 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4efb36c740 10 client.0 _ll_register_callbacks cb 0x7f4ee409acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59674/0 (socket says 172.21.15.86:59674) 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/3046656066 learned_addr learned my addr 172.21.15.86:0/3046656066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9b5d0 msgr2=0x563f3eab3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9b5d0 0x563f3eab3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/3046656066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 msgr2=0x563f3eaa0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef1d9b640 1 -- 172.21.15.86:0/3046656066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3eaa0b50 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.024+0000 7f4ef2d9d640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ef1d9b640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009a60 tx=0x7f4ee8005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fb13cfd7ba40f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8016040 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f4ee8011680 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee40b4280 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4ee8037050 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 10 client.20325 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563f3eb251b0 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee40b4280 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f4ee800f040 con 0x563f3ea9cb40 2024-09-05T07:03:40.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 10 client.20325 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 10 client.20325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 20 client.20325 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eb25820 0x563f3eb27c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 4 client.20325 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.025+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f3ea08d70 con 0x563f3eb25820 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.026+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eb25820 0x563f3eb27c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4ef259c640 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eb25820 0x563f3eb27c10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccdf618f0781c2f9 server_cookie=3f8a7d99bc4ee0fe in_seq=0 out_seq=0 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4ed9ffb640 10 client.20325 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4ed9ffb640 1 -- 172.21.15.86:0/3046656066 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f3ea08d70 con 0x563f3eb25820 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4ed9ffb640 10 client.20325 handle_command_reply: tid=0 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 client.20325 shutdown 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 msgr2=0x563f3ea9fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009a60 tx=0x7f4ee8005cc0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 shutdown_connections 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x563f3eb25820 0x563f3eb27c10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3eaa6370 0x563f3eaa0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3ea9cb40 0x563f3ea9fd80 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4ed9ffb640 0 client.20325 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 --2- 172.21.15.86:0/3046656066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f3ea9b5d0 0x563f3eab3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 >> 172.21.15.86:0/3046656066 conn(0x563f3ea64310 msgr2=0x563f3ea964e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 shutdown_connections 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 1 -- 172.21.15.86:0/3046656066 wait complete. 2024-09-05T07:03:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.027+0000 7f4efb36c740 20 client.20325 trim_cache size 0 max 16384 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 msgr2=0x7f4edc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 0x7f4edc0a5230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4eec006010 tx=0x7f4eec031470 comp rx=0 tx=0).stop 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 msgr2=0x7f4ef41991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d770 tx=0x7f4ee400dc30 comp rx=0 tx=0).stop 2024-09-05T07:03:40.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 shutdown_connections 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4edc0a2d70 0x7f4edc0a5230 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef4106a80 0x7f4ef41a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ef4101420 0x7f4ef4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 --2- 172.21.15.86:0/1469542872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef4100a80 0x7f4ef41991c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 >> 172.21.15.86:0/1469542872 conn(0x7f4ef40fc7a0 msgr2=0x7f4ef40fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 shutdown_connections 2024-09-05T07:03:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:40.028+0000 7f4ef8f93640 1 -- 172.21.15.86:0/1469542872 wait complete. 2024-09-05T07:03:40.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:03:40.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.53 MB/sec execute 418 sec latency 13877.691 ms 2024-09-05T07:03:41.002 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:41.403 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:41 smithi086 ceph-mon[30943]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 673 KiB/s wr, 59 op/s 2024-09-05T07:03:41.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 -- 172.21.15.86:0/3160925291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 msgr2=0x7f4968106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 --2- 172.21.15.86:0/3160925291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f4968106e60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f495c009930 tx=0x7f495c02a420 comp rx=0 tx=0).stop 2024-09-05T07:03:41.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 -- 172.21.15.86:0/3160925291 shutdown_connections 2024-09-05T07:03:41.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 --2- 172.21.15.86:0/3160925291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 0x7f496810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 --2- 172.21.15.86:0/3160925291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 0x7f4968100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 --2- 172.21.15.86:0/3160925291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f4968106e60 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 -- 172.21.15.86:0/3160925291 >> 172.21.15.86:0/3160925291 conn(0x7f49680fc7a0 msgr2=0x7f49680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 -- 172.21.15.86:0/3160925291 shutdown_connections 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.485+0000 7f496f9d6640 1 -- 172.21.15.86:0/3160925291 wait complete. 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 Processor -- start 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 -- start start 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 0x7f49680fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 0x7f49680fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4968110d30 con 0x7f4968106a80 2024-09-05T07:03:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4968110bb0 con 0x7f4968100a80 2024-09-05T07:03:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496f9d6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4968110eb0 con 0x7f4968101420 2024-09-05T07:03:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59692/0 (socket says 172.21.15.86:59692) 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 -- 172.21.15.86:0/1446691445 learned_addr learned my addr 172.21.15.86:0/1446691445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496d772640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 0x7f49680fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496cf71640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 0x7f49680fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 -- 172.21.15.86:0/1446691445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 msgr2=0x7f49680fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 0x7f49680fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 -- 172.21.15.86:0/1446691445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 msgr2=0x7f49680fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 0x7f49680fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 -- 172.21.15.86:0/1446691445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a6870 con 0x7f4968106a80 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.486+0000 7f496df73640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f496000eaa0 tx=0x7f496000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.487+0000 7f495a7fc640 1 -- 172.21.15.86:0/1446691445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496000cdb0 con 0x7f4968106a80 2024-09-05T07:03:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.487+0000 7f495a7fc640 1 -- 172.21.15.86:0/1446691445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4960005020 con 0x7f4968106a80 2024-09-05T07:03:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.487+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a6b60 con 0x7f4968106a80 2024-09-05T07:03:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.487+0000 7f495a7fc640 1 -- 172.21.15.86:0/1446691445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49600116d0 con 0x7f4968106a80 2024-09-05T07:03:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.487+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a70f0 con 0x7f4968106a80 2024-09-05T07:03:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f495a7fc640 1 -- 172.21.15.86:0/1446691445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49600051d0 con 0x7f4968106a80 2024-09-05T07:03:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 Processor -- start 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 -- start start 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56232ec6b2f0 con 0x56232ed10370 2024-09-05T07:03:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56232ec12cf0 con 0x56232ed06b40 2024-09-05T07:03:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f4971daf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56232ec672b0 con 0x56232ed055d0 2024-09-05T07:03:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59708/0 (socket says 172.21.15.86:59708) 2024-09-05T07:03:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f496cf71640 1 -- 172.21.15.86:0/1938027453 learned_addr learned my addr 172.21.15.86:0/1938027453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f495a7fc640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 0x7f49540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.490+0000 7f496d772640 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 -- 172.21.15.86:0/1938027453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 msgr2=0x56232ed05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 -- 172.21.15.86:0/1938027453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 msgr2=0x56232ed05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 -- 172.21.15.86:0/1938027453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232ebf73a0 con 0x56232ed10370 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496df73640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 0x7f49540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f495a7fc640 1 -- 172.21.15.86:0/1446691445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f49600cf8a0 con 0x7f4968106a80 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f496cf71640 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f496400bee0 tx=0x7f496400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4ed2a8d0967106 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f49537fe640 1 -- 172.21.15.86:0/1938027453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964004030 con 0x56232ed10370 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f49537fe640 1 -- 172.21.15.86:0/1938027453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49640041d0 con 0x56232ed10370 2024-09-05T07:03:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.491+0000 7f49537fe640 1 -- 172.21.15.86:0/1938027453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49640106c0 con 0x56232ed10370 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- 172.21.15.86:0/1938027453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 msgr2=0x56232ed12760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f496df73640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 0x7f49540a5230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4960097e90 tx=0x7f4960086040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f496400bee0 tx=0x7f496400b870 comp rx=0 tx=0).stop 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- 172.21.15.86:0/1938027453 shutdown_connections 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed12760 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 --2- 172.21.15.86:0/1938027453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- 172.21.15.86:0/1938027453 >> 172.21.15.86:0/1938027453 conn(0x56232ecce310 msgr2=0x56232ed01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- 172.21.15.86:0/1938027453 shutdown_connections 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- 172.21.15.86:0/1938027453 wait complete. 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 -- start start 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.492+0000 7f4971daf740 1 Processor -- start 2024-09-05T07:03:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56232ed13070 con 0x56232ed10370 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56232ec26060 con 0x56232ed06b40 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56232ed131f0 con 0x56232ed055d0 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 10 client.0 _ll_register_callbacks cb 0x56232e9798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59716/0 (socket says 172.21.15.86:59716) 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 -- 172.21.15.86:0/577368179 learned_addr learned my addr 172.21.15.86:0/577368179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496d772640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 -- 172.21.15.86:0/577368179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 msgr2=0x56232ed0fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed0fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 -- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 msgr2=0x56232ed0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 -- 172.21.15.86:0/577368179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232e9dc3c0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f496cf71640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4964001eb0 tx=0x7f4964003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca012b72fdcc257 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964026040 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49640041d0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.493+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f49640037f0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f49640039f0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 10 client.20337 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56232ed1eaf0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49640041d0 con 0x56232ed10370 2024-09-05T07:03:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f496401c440 con 0x56232ed10370 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 10 client.20337 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 10 client.20337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 20 client.20337 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed1ff50 0x56232ed22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 4 client.20337 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56232ec78860 con 0x56232ed1ff50 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.494+0000 7f496d772640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed1ff50 0x56232ed22340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f496d772640 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed1ff50 0x56232ed22340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95dfbc3bd001829f server_cookie=6b3eba10195c811 in_seq=0 out_seq=0 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4950ff9640 10 client.20337 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4950ff9640 1 -- 172.21.15.86:0/577368179 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56232ec78860 con 0x56232ed1ff50 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4950ff9640 10 client.20337 handle_command_reply: tid=0 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4971daf740 1 client.20337 shutdown 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 msgr2=0x56232ed0d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.495+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4964001eb0 tx=0x7f4964003c90 comp rx=0 tx=0).stop 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 shutdown_connections 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed1ff50 0x56232ed22340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed0d570 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4950ff9640 0 client.20337 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed06b40 0x56232ed0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 --2- 172.21.15.86:0/577368179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed055d0 0x56232ed0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 >> 172.21.15.86:0/577368179 conn(0x56232ecce310 msgr2=0x56232ed07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 shutdown_connections 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 1 -- 172.21.15.86:0/577368179 wait complete. 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.496+0000 7f4971daf740 20 client.20337 trim_cache size 0 max 16384 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 Processor -- start 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 -- start start 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed1c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed10370 0x56232ed27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b1460 con 0x56232ed06b40 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b2fa0 con 0x56232ed055d0 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4971daf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b1b30 con 0x56232ed10370 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59722/0 (socket says 172.21.15.86:59722) 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 -- 172.21.15.86:0/3830964205 learned_addr learned my addr 172.21.15.86:0/3830964205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496cf71640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed1c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496cf71640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed10370 0x56232ed27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 -- 172.21.15.86:0/3830964205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed10370 msgr2=0x56232ed27a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed10370 0x56232ed27a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 -- 172.21.15.86:0/3830964205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 msgr2=0x56232ed1c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed1c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 -- 172.21.15.86:0/3830964205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232ed0e440 con 0x56232ed06b40 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496cf71640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed1c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f496d772640 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f495c02a940 tx=0x7f495c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19671c711b5ca905 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.509+0000 7f4952ffd640 1 -- 172.21.15.86:0/3830964205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c049030 con 0x56232ed06b40 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4952ffd640 1 -- 172.21.15.86:0/3830964205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f495c0042e0 con 0x56232ed06b40 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4952ffd640 1 -- 172.21.15.86:0/3830964205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c035040 con 0x56232ed06b40 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- 172.21.15.86:0/3830964205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 msgr2=0x56232ed0bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f495c02a940 tx=0x7f495c005fd0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- 172.21.15.86:0/3830964205 shutdown_connections 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed10370 0x56232ed27a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed06b40 0x56232ed0bfb0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 --2- 172.21.15.86:0/3830964205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed1c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- 172.21.15.86:0/3830964205 >> 172.21.15.86:0/3830964205 conn(0x56232ecce310 msgr2=0x56232ed0b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- 172.21.15.86:0/3830964205 shutdown_connections 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- 172.21.15.86:0/3830964205 wait complete. 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.510+0000 7f4971daf740 1 -- start start 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 Processor -- start 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed14150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed06b40 0x56232ed14690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b28d0 con 0x56232ed10370 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b2200 con 0x56232ed055d0 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49681af460 con 0x56232ed06b40 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f4971daf740 10 client.0 _ll_register_callbacks cb 0x7f495c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f496cf71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59724/0 (socket says 172.21.15.86:59724) 2024-09-05T07:03:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f496cf71640 1 -- 172.21.15.86:0/220298519 learned_addr learned my addr 172.21.15.86:0/220298519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.511+0000 7f496cf71640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed14150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496d772640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed06b40 0x56232ed14690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 -- 172.21.15.86:0/220298519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed06b40 msgr2=0x56232ed14690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed06b40 0x56232ed14690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 -- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 msgr2=0x56232ed14150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed14150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 -- 172.21.15.86:0/220298519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232ed82990 con 0x56232ed10370 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f496cf71640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f496401c860 tx=0x7f4964005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d321b8c0494d5234 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496400ba60 con 0x56232ed10370 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f49640037f0 con 0x56232ed10370 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c0042e0 con 0x56232ed10370 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f496401cc60 con 0x56232ed10370 2024-09-05T07:03:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4971daf740 10 client.20343 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56232ed83c70 con 0x56232ed10370 2024-09-05T07:03:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.512+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f495c0042e0 con 0x56232ed10370 2024-09-05T07:03:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f4964010a10 con 0x56232ed10370 2024-09-05T07:03:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 10 client.20343 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 10 client.20343 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:03:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 20 client.20343 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed84390 0x56232ed867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 4 client.20343 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56232ec72d70 con 0x56232ed84390 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f496d772640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed84390 0x56232ed867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f496d772640 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed84390 0x56232ed867b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8337da898335cef server_cookie=96564ce21e4c4514 in_seq=0 out_seq=0 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4950ff9640 10 client.20343 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4950ff9640 1 -- 172.21.15.86:0/220298519 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56232ec72d70 con 0x56232ed84390 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4950ff9640 10 client.20343 handle_command_reply: tid=0 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.513+0000 7f4971daf740 1 client.20343 shutdown 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 msgr2=0x56232ed14bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f496401c860 tx=0x7f4964005850 comp rx=0 tx=0).stop 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 shutdown_connections 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x56232ed84390 0x56232ed867b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56232ed10370 0x56232ed14bd0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56232ed06b40 0x56232ed14690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 --2- 172.21.15.86:0/220298519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56232ed055d0 0x56232ed14150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 >> 172.21.15.86:0/220298519 conn(0x56232ecce310 msgr2=0x56232ed0c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4950ff9640 0 client.20343 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 shutdown_connections 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 1 -- 172.21.15.86:0/220298519 wait complete. 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f4971daf740 20 client.20343 trim_cache size 0 max 16384 2024-09-05T07:03:41.542 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:03:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 msgr2=0x7f49540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 0x7f49540a5230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4960097e90 tx=0x7f4960086040 comp rx=0 tx=0).stop 2024-09-05T07:03:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 msgr2=0x7f49680fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f496000eaa0 tx=0x7f496000ef60 comp rx=0 tx=0).stop 2024-09-05T07:03:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 shutdown_connections 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f49540a2d70 0x7f49540a5230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4968106a80 0x7f49680fd8f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4968101420 0x7f49680fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 --2- 172.21.15.86:0/1446691445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4968100a80 0x7f49680fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.514+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 >> 172.21.15.86:0/1446691445 conn(0x7f49680fc7a0 msgr2=0x7f496806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.515+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 shutdown_connections 2024-09-05T07:03:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:41.515+0000 7f496f9d6640 1 -- 172.21.15.86:0/1446691445 wait complete. 2024-09-05T07:03:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:41 smithi096 ceph-mon[34468]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 673 KiB/s wr, 59 op/s 2024-09-05T07:03:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:41 smithi115 ceph-mon[34791]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 673 KiB/s wr, 59 op/s 2024-09-05T07:03:41.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.52 MB/sec execute 419 sec latency 14877.749 ms 2024-09-05T07:03:42.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:03:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:42.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.50 MB/sec execute 420 sec latency 15877.777 ms 2024-09-05T07:03:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.971+0000 7f7c2325f640 1 -- 172.21.15.86:0/3978157457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c1027a0 msgr2=0x7f7c1c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.971+0000 7f7c2325f640 1 --2- 172.21.15.86:0/3978157457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c1067f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009900 tx=0x7f7c1002a410 comp rx=0 tx=0).stop 2024-09-05T07:03:43.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- 172.21.15.86:0/3978157457 shutdown_connections 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- 172.21.15.86:0/3978157457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c106f20 0x7f7c1c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- 172.21.15.86:0/3978157457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c1067f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- 172.21.15.86:0/3978157457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- 172.21.15.86:0/3978157457 >> 172.21.15.86:0/3978157457 conn(0x7f7c1c0fd8b0 msgr2=0x7f7c1c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- 172.21.15.86:0/3978157457 shutdown_connections 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- 172.21.15.86:0/3978157457 wait complete. 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 Processor -- start 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- start start 2024-09-05T07:03:43.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c111ed0 con 0x7f7c1c106f20 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c111d50 con 0x7f7c1c101df0 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.972+0000 7f7c2325f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c112050 con 0x7f7c1c1027a0 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41730/0 (socket says 172.21.15.86:41730) 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 -- 172.21.15.86:0/2183212951 learned_addr learned my addr 172.21.15.86:0/2183212951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 -- 172.21.15.86:0/2183212951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 msgr2=0x7f7c1c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 -- 172.21.15.86:0/2183212951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 msgr2=0x7f7c1c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 -- 172.21.15.86:0/2183212951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c19d480 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c217fc640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400b970 tx=0x7f7c1400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c19ffb640 1 -- 172.21.15.86:0/2183212951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14019070 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c19ffb640 1 -- 172.21.15.86:0/2183212951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c14014440 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c19d770 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c19ffb640 1 -- 172.21.15.86:0/2183212951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140075d0 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.973+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c1a5050 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.975+0000 7f7c19ffb640 1 -- 172.21.15.86:0/2183212951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c1401c070 con 0x7f7c1c106f20 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 Processor -- start 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 -- start start 2024-09-05T07:03:43.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c19ffb640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 0x7f7c040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 0x55e264827090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264827a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e26478d2f0 con 0x55e2648275d0 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e264734cf0 con 0x55e264832370 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c25638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e2647892b0 con 0x55e264828b40 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 0x7f7c040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c19ffb640 1 -- 172.21.15.86:0/2183212951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f7c140cedd0 con 0x7f7c1c106f20 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264827a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c20ffb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 0x55e264827090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264827a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41734/0 (socket says 172.21.15.86:41734) 2024-09-05T07:03:43.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c20ffb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 0x55e264827090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:48164/0 (socket says 172.21.15.86:48164) 2024-09-05T07:03:43.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c1bfff640 1 -- 172.21.15.86:0/497709705 learned_addr learned my addr 172.21.15.86:0/497709705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:43.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.976+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:35534/0 (socket says 172.21.15.86:35534) 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 -- 172.21.15.86:0/497709705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 msgr2=0x55e264827090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 0x55e264827090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 -- 172.21.15.86:0/497709705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 msgr2=0x55e264827a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264827a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 -- 172.21.15.86:0/497709705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2647193a0 con 0x55e264832370 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 0x7f7c040a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002a930 tx=0x7f7c10032cf0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c217fc640 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c140cc190 tx=0x7f7c14004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4ec83dbf406a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c02ffd640 1 -- 172.21.15.86:0/497709705 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1409b660 con 0x55e264832370 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c02ffd640 1 -- 172.21.15.86:0/497709705 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1409af60 con 0x55e264832370 2024-09-05T07:03:43.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.977+0000 7f7c02ffd640 1 -- 172.21.15.86:0/497709705 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140145f0 con 0x55e264832370 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- 172.21.15.86:0/497709705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 msgr2=0x55e264834760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c140cc190 tx=0x7f7c14004a40 comp rx=0 tx=0).stop 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- 172.21.15.86:0/497709705 shutdown_connections 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264832370 0x55e264834760 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264827a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 --2- 172.21.15.86:0/497709705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264828b40 0x55e264827090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- 172.21.15.86:0/497709705 >> 172.21.15.86:0/497709705 conn(0x55e2647f0310 msgr2=0x55e264823c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- 172.21.15.86:0/497709705 shutdown_connections 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- 172.21.15.86:0/497709705 wait complete. 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 -- start start 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.978+0000 7f7c25638740 1 Processor -- start 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e264831d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e26482f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e264835070 con 0x55e264832370 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e264748060 con 0x55e264828b40 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e2648351f0 con 0x55e2648275d0 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 10 client.0 _ll_register_callbacks cb 0x55e26449b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41738/0 (socket says 172.21.15.86:41738) 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 -- 172.21.15.86:0/918361580 learned_addr learned my addr 172.21.15.86:0/918361580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e264831d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e26482f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 -- 172.21.15.86:0/918361580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 msgr2=0x55e264831d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e264831d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 -- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 msgr2=0x55e26482f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e26482f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 -- 172.21.15.86:0/918361580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2644fe3c0 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c217fc640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c14097930 tx=0x7f7c14004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dcd4cceabc690 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140a3940 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f7c140cc270 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.979+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1409af60 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7c1406b050 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 10 client.20355 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e264840b40 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1409af60 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f7c14065bb0 con 0x55e264832370 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 10 client.20355 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 10 client.20355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 20 client.20355 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:43.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e264841e90 0x55e264844280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 4 client.20355 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.980+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26479a860 con 0x55e264841e90 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.981+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e264841e90 0x55e264844280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.981+0000 7f7bfbfff640 10 client.20355 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.981+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e264841e90 0x55e264844280 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6483720e515dd164 server_cookie=5a9ac214620dfaf6 in_seq=0 out_seq=0 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7bfbfff640 1 -- 172.21.15.86:0/918361580 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e26479a860 con 0x55e264841e90 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7bfbfff640 10 client.20355 handle_command_reply: tid=0 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 client.20355 shutdown 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 msgr2=0x55e26482f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c14097930 tx=0x7f7c14004b50 comp rx=0 tx=0).stop 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 shutdown_connections 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e264841e90 0x55e264844280 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e26482f650 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e26482f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7bfbfff640 0 client.20355 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 --2- 172.21.15.86:0/918361580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e264831d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 >> 172.21.15.86:0/918361580 conn(0x55e2647f0310 msgr2=0x55e2648299c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 shutdown_connections 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 1 -- 172.21.15.86:0/918361580 wait complete. 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.982+0000 7f7c25638740 20 client.20355 trim_cache size 0 max 16384 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 Processor -- start 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 -- start start 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e26483e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264823e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1af230 con 0x55e264832370 2024-09-05T07:03:43.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1b0d70 con 0x55e264828b40 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c25638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1af900 con 0x55e2648275d0 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c217fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41744/0 (socket says 172.21.15.86:41744) 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.995+0000 7f7c217fc640 1 -- 172.21.15.86:0/3142484792 learned_addr learned my addr 172.21.15.86:0/3142484792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e26483e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264823e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 -- 172.21.15.86:0/3142484792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 msgr2=0x55e26483e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e26483e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 -- 172.21.15.86:0/3142484792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 msgr2=0x55e264823e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264823e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 -- 172.21.15.86:0/3142484792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26482a120 con 0x55e264832370 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c217fc640 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7c140a36a0 tx=0x7f7c14014a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed78dc98fdd5d4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c027fc640 1 -- 172.21.15.86:0/3142484792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140cc510 con 0x55e264832370 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c027fc640 1 -- 172.21.15.86:0/3142484792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1409af60 con 0x55e264832370 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.996+0000 7f7c027fc640 1 -- 172.21.15.86:0/3142484792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14097200 con 0x55e264832370 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- 172.21.15.86:0/3142484792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 msgr2=0x55e2648472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7c140a36a0 tx=0x7f7c14014a20 comp rx=0 tx=0).stop 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- 172.21.15.86:0/3142484792 shutdown_connections 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e264832370 0x55e2648472a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 --2- 172.21.15.86:0/3142484792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2648275d0 0x55e26483e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- 172.21.15.86:0/3142484792 >> 172.21.15.86:0/3142484792 conn(0x55e2647f0310 msgr2=0x55e264823700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- 172.21.15.86:0/3142484792 shutdown_connections 2024-09-05T07:03:43.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- 172.21.15.86:0/3142484792 wait complete. 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 -- start start 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.997+0000 7f7c25638740 1 Processor -- start 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264845550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264832370 0x55e264845a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1b06a0 con 0x55e2648275d0 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1affd0 con 0x55e264828b40 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1ad230 con 0x55e264832370 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 10 client.0 _ll_register_callbacks cb 0x7f7c10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41760/0 (socket says 172.21.15.86:41760) 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 -- 172.21.15.86:0/1385757238 learned_addr learned my addr 172.21.15.86:0/1385757238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264845550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c217fc640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264832370 0x55e264845a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 -- 172.21.15.86:0/1385757238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264832370 msgr2=0x55e264845a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264832370 0x55e264845a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 -- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 msgr2=0x55e264845550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264845550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 -- 172.21.15.86:0/1385757238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2648ab7d0 con 0x55e2648275d0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c20ffb640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c000c00 tx=0x7f7c0c007fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6384cc79b93a3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c015e60 con 0x55e2648275d0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f7c0c0145c0 con 0x55e2648275d0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1409af60 con 0x55e2648275d0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7c0c02d050 con 0x55e2648275d0 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 10 client.20361 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:43.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.998+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e2648acab0 con 0x55e2648275d0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1409af60 con 0x55e2648275d0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f7c0c0248a0 con 0x55e2648275d0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 10 client.20361 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 10 client.20361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 20 client.20361 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e2648ad1b0 0x55e2648af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 4 client.20361 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e264794d70 con 0x55e2648ad1b0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e2648ad1b0 0x55e2648af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7c1bfff640 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e2648ad1b0 0x55e2648af5d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db751ac8dc920ab3 server_cookie=7a21383b4d627938 in_seq=0 out_seq=0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:42.999+0000 7f7bfbfff640 10 client.20361 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7bfbfff640 1 -- 172.21.15.86:0/1385757238 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e264794d70 con 0x55e2648ad1b0 2024-09-05T07:03:43.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7bfbfff640 10 client.20361 handle_command_reply: tid=0 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 client.20361 shutdown 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 msgr2=0x55e264845010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c000c00 tx=0x7f7c0c007fd0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 shutdown_connections 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55e2648ad1b0 0x55e2648af5d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7bfbfff640 0 client.20361 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e264832370 0x55e264845a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e264828b40 0x55e264845550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 --2- 172.21.15.86:0/1385757238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2648275d0 0x55e264845010 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 >> 172.21.15.86:0/1385757238 conn(0x55e2647f0310 msgr2=0x55e2648308a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 shutdown_connections 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 1 -- 172.21.15.86:0/1385757238 wait complete. 2024-09-05T07:03:43.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.000+0000 7f7c25638740 20 client.20361 trim_cache size 0 max 16384 2024-09-05T07:03:43.042 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:03:43.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 msgr2=0x7f7c040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 0x7f7c040a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002a930 tx=0x7f7c10032cf0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 msgr2=0x7f7c1c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400b970 tx=0x7f7c1400be30 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 shutdown_connections 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7c040a2d70 0x7f7c040a5230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1c106f20 0x7f7c1c19cda0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c1c1027a0 0x7f7c1c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 --2- 172.21.15.86:0/2183212951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c1c101df0 0x7f7c1c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 >> 172.21.15.86:0/2183212951 conn(0x7f7c1c0fd8b0 msgr2=0x7f7c1c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 shutdown_connections 2024-09-05T07:03:43.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:43.001+0000 7f7c2325f640 1 -- 172.21.15.86:0/2183212951 wait complete. 2024-09-05T07:03:43.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:43 smithi086 ceph-mon[30943]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 682 KiB/s wr, 63 op/s 2024-09-05T07:03:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:43 smithi096 ceph-mon[34468]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 682 KiB/s wr, 63 op/s 2024-09-05T07:03:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:43 smithi115 ceph-mon[34791]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 682 KiB/s wr, 63 op/s 2024-09-05T07:03:43.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.49 MB/sec execute 421 sec latency 16877.848 ms 2024-09-05T07:03:43.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:03:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 -- 172.21.15.86:0/360257775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 msgr2=0x7f14cc10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 --2- 172.21.15.86:0/360257775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc10e790 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009ca0 tx=0x7f14c402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 -- 172.21.15.86:0/360257775 shutdown_connections 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 --2- 172.21.15.86:0/360257775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc10e790 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 --2- 172.21.15.86:0/360257775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14cc1037a0 0x7f14cc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 --2- 172.21.15.86:0/360257775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 -- 172.21.15.86:0/360257775 >> 172.21.15.86:0/360257775 conn(0x7f14cc0fc7d0 msgr2=0x7f14cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 -- 172.21.15.86:0/360257775 shutdown_connections 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.571+0000 7f14d0c14640 1 -- 172.21.15.86:0/360257775 wait complete. 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 Processor -- start 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 -- start start 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14cc1037a0 0x7f14cc100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc111fc0 con 0x7f14cc10c2f0 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc111e40 con 0x7f14cc102df0 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14d0c14640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc112140 con 0x7f14cc1037a0 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41782/0 (socket says 172.21.15.86:41782) 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 -- 172.21.15.86:0/3754015541 learned_addr learned my addr 172.21.15.86:0/3754015541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 -- 172.21.15.86:0/3754015541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14cc1037a0 msgr2=0x7f14cc100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14cc1037a0 0x7f14cc100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 -- 172.21.15.86:0/3754015541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 msgr2=0x7f14cc100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 -- 172.21.15.86:0/3754015541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc1a6a40 con 0x7f14cc10c2f0 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.572+0000 7f14cad9d640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009dc0 tx=0x7f14c4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.573+0000 7f14bb7fe640 1 -- 172.21.15.86:0/3754015541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4003cf0 con 0x7f14cc10c2f0 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.573+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc1a6cd0 con 0x7f14cc10c2f0 2024-09-05T07:03:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.573+0000 7f14bb7fe640 1 -- 172.21.15.86:0/3754015541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c40044f0 con 0x7f14cc10c2f0 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.573+0000 7f14bb7fe640 1 -- 172.21.15.86:0/3754015541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4030bf0 con 0x7f14cc10c2f0 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.573+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc1a7210 con 0x7f14cc10c2f0 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.575+0000 7f14bb7fe640 1 -- 172.21.15.86:0/3754015541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14c4004040 con 0x7f14cc10c2f0 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.575+0000 7f14d2fed740 1 Processor -- start 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 -- start start 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f44905d0 0x5559f4490a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f449d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14bb7fe640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 0x7f14b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559f43f62f0 con 0x5559f4491b40 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559f439dcf0 con 0x5559f449b370 2024-09-05T07:03:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14d2fed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559f43f22b0 con 0x5559f44905d0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14cad9d640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 0x7f14b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41786/0 (socket says 172.21.15.86:41786) 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 -- 172.21.15.86:0/1336722730 learned_addr learned my addr 172.21.15.86:0/1336722730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14bb7fe640 1 -- 172.21.15.86:0/3754015541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f14c40f0e80 con 0x7f14cc10c2f0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f449d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14ca59c640 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f44905d0 0x5559f4490a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 -- 172.21.15.86:0/1336722730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f44905d0 msgr2=0x5559f4490a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14cad9d640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 0x7f14b40a51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14c40ee790 tx=0x7f14c40b2500 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f44905d0 0x5559f4490a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 -- 172.21.15.86:0/1336722730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 msgr2=0x5559f449d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f449d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.576+0000 7f14c9d9b640 1 -- 172.21.15.86:0/1336722730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f43823a0 con 0x5559f4491b40 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f14c000bf60 tx=0x7f14c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fcfb4b0db95625 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14b8ff9640 1 -- 172.21.15.86:0/1336722730 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0005220 con 0x5559f4491b40 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14b8ff9640 1 -- 172.21.15.86:0/1336722730 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c0004c50 con 0x5559f4491b40 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14b8ff9640 1 -- 172.21.15.86:0/1336722730 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0012660 con 0x5559f4491b40 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 -- 172.21.15.86:0/1336722730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 msgr2=0x5559f4490090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f14c000bf60 tx=0x7f14c000b260 comp rx=0 tx=0).stop 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 -- 172.21.15.86:0/1336722730 shutdown_connections 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f449d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f44905d0 0x5559f4490a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 --2- 172.21.15.86:0/1336722730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f4491b40 0x5559f4490090 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.577+0000 7f14d2fed740 1 -- 172.21.15.86:0/1336722730 >> 172.21.15.86:0/1336722730 conn(0x5559f4459310 msgr2=0x5559f448cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- 172.21.15.86:0/1336722730 shutdown_connections 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- 172.21.15.86:0/1336722730 wait complete. 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- start start 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 Processor -- start 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4497fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f4498560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559f449e070 con 0x5559f44905d0 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559f43b1060 con 0x5559f449b370 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5559f449e1f0 con 0x5559f4491b40 2024-09-05T07:03:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14d2fed740 10 client.0 _ll_register_callbacks cb 0x5559f41048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41792/0 (socket says 172.21.15.86:41792) 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14c9d9b640 1 -- 172.21.15.86:0/2550595030 learned_addr learned my addr 172.21.15.86:0/2550595030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.578+0000 7f14ca59c640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4497fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 -- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 msgr2=0x5559f4497fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4497fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 -- 172.21.15.86:0/2550595030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 msgr2=0x5559f4498560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f4498560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 -- 172.21.15.86:0/2550595030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f41673c0 con 0x5559f44905d0 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14ca59c640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4497fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f14c0004700 tx=0x7f14c0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936d25b8f7dcf53e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004040 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f14c00041e0 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0004c50 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f14c002f050 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 10 client.20373 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5559f44a9af0 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c0004c50 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f14c0003920 con 0x5559f44905d0 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 10 client.20373 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 10 client.20373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.579+0000 7f14d2fed740 20 client.20373 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f44aae40 0x5559f44ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14d2fed740 4 client.20373 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f4403860 con 0x5559f44aae40 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14ca59c640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f44aae40 0x5559f44ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14ca59c640 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f44aae40 0x5559f44ad230 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf40165b93fa5bf6 server_cookie=dd99176b65403b5e in_seq=0 out_seq=0 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.580+0000 7f14b1ffb640 10 client.20373 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14b1ffb640 1 -- 172.21.15.86:0/2550595030 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559f4403860 con 0x5559f44aae40 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14b1ffb640 10 client.20373 handle_command_reply: tid=0 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 client.20373 shutdown 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 msgr2=0x5559f449ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f14c0004700 tx=0x7f14c0011e90 comp rx=0 tx=0).stop 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 shutdown_connections 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f44aae40 0x5559f44ad230 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f4498560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4497fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14b1ffb640 0 client.20373 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 --2- 172.21.15.86:0/2550595030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f449ad10 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 >> 172.21.15.86:0/2550595030 conn(0x5559f4459310 msgr2=0x5559f4492930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 shutdown_connections 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.581+0000 7f14d2fed740 1 -- 172.21.15.86:0/2550595030 wait complete. 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.582+0000 7f14d2fed740 20 client.20373 trim_cache size 0 max 16384 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.594+0000 7f14d2fed740 1 Processor -- start 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.594+0000 7f14d2fed740 1 -- start start 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f44905d0 0x5559f44a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f448bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b1310 con 0x5559f449b370 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b2e50 con 0x5559f44905d0 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14d2fed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b19e0 con 0x5559f4491b40 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41804/0 (socket says 172.21.15.86:41804) 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 -- 172.21.15.86:0/355160453 learned_addr learned my addr 172.21.15.86:0/355160453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14ca59c640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f448bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f44905d0 0x5559f44a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 -- 172.21.15.86:0/355160453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 msgr2=0x5559f448bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f448bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 -- 172.21.15.86:0/355160453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f44905d0 msgr2=0x5559f44a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f44905d0 0x5559f44a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 -- 172.21.15.86:0/355160453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f4493120 con 0x5559f449b370 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14ca59c640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f448bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f14c0003fd0 tx=0x7f14c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27322dce0219d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.595+0000 7f14b3fff640 1 -- 172.21.15.86:0/355160453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0029110 con 0x5559f449b370 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14b3fff640 1 -- 172.21.15.86:0/355160453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c0022030 con 0x5559f449b370 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14b3fff640 1 -- 172.21.15.86:0/355160453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003850 con 0x5559f449b370 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- 172.21.15.86:0/355160453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 msgr2=0x5559f44b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f14c0003fd0 tx=0x7f14c000b260 comp rx=0 tx=0).stop 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- 172.21.15.86:0/355160453 shutdown_connections 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f449b370 0x5559f44b2830 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f448bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 --2- 172.21.15.86:0/355160453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f44905d0 0x5559f44a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- 172.21.15.86:0/355160453 >> 172.21.15.86:0/355160453 conn(0x5559f4459310 msgr2=0x5559f448b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- 172.21.15.86:0/355160453 shutdown_connections 2024-09-05T07:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- 172.21.15.86:0/355160453 wait complete. 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 -- start start 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.596+0000 7f14d2fed740 1 Processor -- start 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4495480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f44959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b2780 con 0x5559f44905d0 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1b20b0 con 0x5559f449b370 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1af310 con 0x5559f4491b40 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 10 client.0 _ll_register_callbacks cb 0x7f14c000bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41816/0 (socket says 172.21.15.86:41816) 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 -- 172.21.15.86:0/3789006485 learned_addr learned my addr 172.21.15.86:0/3789006485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f44959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4495480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 -- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 msgr2=0x5559f4495480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4495480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 -- 172.21.15.86:0/3789006485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 msgr2=0x5559f44959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f44959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 -- 172.21.15.86:0/3789006485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f4514800 con 0x5559f44905d0 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4495480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14c9d9b640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f14c0005d80 tx=0x7f14c00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87b4f0d6d1c936b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:44.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0016040 con 0x5559f44905d0 2024-09-05T07:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f14c0004000 con 0x5559f44905d0 2024-09-05T07:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.597+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0022030 con 0x5559f44905d0 2024-09-05T07:03:44.633 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f14c00057f0 con 0x5559f44905d0 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 10 client.20379 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5559f4515a40 con 0x5559f44905d0 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c0022030 con 0x5559f44905d0 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f14c00035f0 con 0x5559f44905d0 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 10 client.20379 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 10 client.20379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 20 client.20379 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f45160b0 0x5559f45184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 4 client.20379 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559f43fdd70 con 0x5559f45160b0 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.598+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f45160b0 0x5559f45184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14b1ffb640 10 client.20379 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:03:44.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14ca59c640 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f45160b0 0x5559f45184a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad3f7dc511b8e8d8 server_cookie=b60c20ecd45d103 in_seq=0 out_seq=0 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14b1ffb640 1 -- 172.21.15.86:0/3789006485 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559f43fdd70 con 0x5559f45160b0 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14b1ffb640 10 client.20379 handle_command_reply: tid=0 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 client.20379 shutdown 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 msgr2=0x5559f4494f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f14c0005d80 tx=0x7f14c00056e0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 shutdown_connections 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x5559f45160b0 0x5559f45184a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559f449b370 0x5559f44959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5559f4491b40 0x5559f4495480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14b1ffb640 0 client.20379 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 --2- 172.21.15.86:0/3789006485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559f44905d0 0x5559f4494f40 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 >> 172.21.15.86:0/3789006485 conn(0x5559f4459310 msgr2=0x5559f4492930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 shutdown_connections 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 1 -- 172.21.15.86:0/3789006485 wait complete. 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.599+0000 7f14d2fed740 20 client.20379 trim_cache size 0 max 16384 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 msgr2=0x7f14b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 0x7f14b40a51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14c40ee790 tx=0x7f14c40b2500 comp rx=0 tx=0).stop 2024-09-05T07:03:44.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 msgr2=0x7f14cc1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009dc0 tx=0x7f14c4032ff0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 shutdown_connections 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14b40a2cf0 0x7f14b40a51b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14cc10c2f0 0x7f14cc1014d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14cc1037a0 0x7f14cc100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 --2- 172.21.15.86:0/3754015541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14cc102df0 0x7f14cc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:44.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 >> 172.21.15.86:0/3754015541 conn(0x7f14cc0fc7d0 msgr2=0x7f14cc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:44.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 shutdown_connections 2024-09-05T07:03:44.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:44.600+0000 7f14d0c14640 1 -- 172.21.15.86:0/3754015541 wait complete. 2024-09-05T07:03:44.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.47 MB/sec execute 422 sec latency 17877.941 ms 2024-09-05T07:03:45.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:45 smithi086 ceph-mon[30943]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-09-05T07:03:45.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:03:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:45 smithi096 ceph-mon[34468]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-09-05T07:03:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:45 smithi115 ceph-mon[34791]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 KiB/s wr, 16 op/s 2024-09-05T07:03:45.776 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:45.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.46 MB/sec execute 423 sec latency 18877.968 ms 2024-09-05T07:03:46.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 -- 172.21.15.86:0/2754334183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101df0 msgr2=0x7f22a4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 --2- 172.21.15.86:0/2754334183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101df0 0x7f22a4105920 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802a410 comp rx=0 tx=0).stop 2024-09-05T07:03:46.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 -- 172.21.15.86:0/2754334183 shutdown_connections 2024-09-05T07:03:46.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 --2- 172.21.15.86:0/2754334183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 --2- 172.21.15.86:0/2754334183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101df0 0x7f22a4105920 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 --2- 172.21.15.86:0/2754334183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101440 0x7f22a4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 -- 172.21.15.86:0/2754334183 >> 172.21.15.86:0/2754334183 conn(0x7f22a4077210 msgr2=0x7f22a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 -- 172.21.15.86:0/2754334183 shutdown_connections 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.267+0000 7f22aa274640 1 -- 172.21.15.86:0/2754334183 wait complete. 2024-09-05T07:03:46.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 Processor -- start 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 -- start start 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101df0 0x7f22a4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112040 con 0x7f22a4101440 2024-09-05T07:03:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4111ec0 con 0x7f22a4106050 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22aa274640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41121c0 con 0x7f22a4101df0 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41832/0 (socket says 172.21.15.86:41832) 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a3fff640 1 -- 172.21.15.86:0/1050702248 learned_addr learned my addr 172.21.15.86:0/1050702248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a37fe640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101df0 0x7f22a4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a8811640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a3fff640 1 -- 172.21.15.86:0/1050702248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101df0 msgr2=0x7f22a4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.268+0000 7f22a3fff640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101df0 0x7f22a4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a3fff640 1 -- 172.21.15.86:0/1050702248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 msgr2=0x7f22a419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a3fff640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a3fff640 1 -- 172.21.15.86:0/1050702248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a419d550 con 0x7f22a4101440 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a8811640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a3fff640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f229400d950 tx=0x7f229400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a17fa640 1 -- 172.21.15.86:0/1050702248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294017070 con 0x7f22a4101440 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a419d840 con 0x7f22a4101440 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a17fa640 1 -- 172.21.15.86:0/1050702248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2294009c30 con 0x7f22a4101440 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22a17fa640 1 -- 172.21.15.86:0/1050702248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229400f620 con 0x7f22a4101440 2024-09-05T07:03:46.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.269+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a5120 con 0x7f22a4101440 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.271+0000 7f22a17fa640 1 -- 172.21.15.86:0/1050702248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f22940109a0 con 0x7f22a4101440 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.271+0000 7f22ac64d740 1 Processor -- start 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 -- start start 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a17fa640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 0x7f228c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434f99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 0x563434fa6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563434eff2f0 con 0x563434fa4370 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563434ea6cf0 con 0x563434f995d0 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22ac64d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563434efb2b0 con 0x563434f9ab40 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a8811640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 0x7f228c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a17fa640 1 -- 172.21.15.86:0/1050702248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f22940cf5f0 con 0x7f22a4101440 2024-09-05T07:03:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:35626/0 (socket says 172.21.15.86:35626) 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 -- 172.21.15.86:0/1029501781 learned_addr learned my addr 172.21.15.86:0/1029501781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a8811640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 0x563434fa6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a3fff640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434f99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 -- 172.21.15.86:0/1029501781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 msgr2=0x563434f99090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434f99090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 -- 172.21.15.86:0/1029501781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 msgr2=0x563434fa6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 0x563434fa6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a37fe640 1 -- 172.21.15.86:0/1029501781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563434e8b3a0 con 0x563434f995d0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a8811640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 0x563434fa6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.272+0000 7f22a8811640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 0x7f228c0a51b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c0139f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22a37fe640 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41983fed5eba0b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f228a7fc640 1 -- 172.21.15.86:0/1029501781 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229803f070 con 0x563434f995d0 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f228a7fc640 1 -- 172.21.15.86:0/1029501781 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2298032e10 con 0x563434f995d0 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f228a7fc640 1 -- 172.21.15.86:0/1029501781 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298035070 con 0x563434f995d0 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 -- 172.21.15.86:0/1029501781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 msgr2=0x563434f99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f2298032c20 comp rx=0 tx=0).stop 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 -- 172.21.15.86:0/1029501781 shutdown_connections 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434fa4370 0x563434fa6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434f99a10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.273+0000 7f22ac64d740 1 --2- 172.21.15.86:0/1029501781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434f99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- 172.21.15.86:0/1029501781 >> 172.21.15.86:0/1029501781 conn(0x563434f62310 msgr2=0x563434f95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- 172.21.15.86:0/1029501781 shutdown_connections 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- 172.21.15.86:0/1029501781 wait complete. 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- start start 2024-09-05T07:03:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 Processor -- start 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434fa15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563434fa7010 con 0x563434f995d0 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563434eba060 con 0x563434fa4370 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563434fa7190 con 0x563434f9ab40 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22ac64d740 10 client.0 _ll_register_callbacks cb 0x563434c0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41852/0 (socket says 172.21.15.86:41852) 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.274+0000 7f22a3fff640 1 -- 172.21.15.86:0/3004831285 learned_addr learned my addr 172.21.15.86:0/3004831285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a8811640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434fa15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 -- 172.21.15.86:0/3004831285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 msgr2=0x563434fa1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 -- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 msgr2=0x563434fa15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434fa15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 -- 172.21.15.86:0/3004831285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563434c703c0 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a8811640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434fa15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2294097a80 tx=0x7f2294009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62df8c03f94632d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294020070 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f2294097ce0 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2298032e10 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f229409fa20 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22ac64d740 10 client.20391 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563434fb2a90 con 0x563434f995d0 2024-09-05T07:03:46.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.275+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2298032e10 con 0x563434f995d0 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f2294084c20 con 0x563434f995d0 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 10 client.20391 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 10 client.20391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 20 client.20391 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563434fb3ef0 0x563434fb6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 4 client.20391 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563434f0c860 con 0x563434fb3ef0 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563434fb3ef0 0x563434fb6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22837fe640 10 client.20391 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.276+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563434fb3ef0 0x563434fb6340 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6c15c292649451 server_cookie=ebdfb80729da8b1b in_seq=0 out_seq=0 2024-09-05T07:03:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22837fe640 1 -- 172.21.15.86:0/3004831285 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563434f0c860 con 0x563434fb3ef0 2024-09-05T07:03:46.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22837fe640 10 client.20391 handle_command_reply: tid=0 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 client.20391 shutdown 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 msgr2=0x563434fa3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2294097a80 tx=0x7f2294009de0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 shutdown_connections 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563434fb3ef0 0x563434fb6340 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434fa15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3004831285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434fa3d50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22837fe640 0 client.20391 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 >> 172.21.15.86:0/3004831285 conn(0x563434f62310 msgr2=0x563434f9b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 shutdown_connections 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 1 -- 172.21.15.86:0/3004831285 wait complete. 2024-09-05T07:03:46.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.277+0000 7f22ac64d740 20 client.20391 trim_cache size 0 max 16384 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 Processor -- start 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 -- start start 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434f9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41af090 con 0x563434f995d0 2024-09-05T07:03:46.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b0bd0 con 0x563434fa4370 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.290+0000 7f22ac64d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41af760 con 0x563434f9ab40 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41854/0 (socket says 172.21.15.86:41854) 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 -- 172.21.15.86:0/2098788921 learned_addr learned my addr 172.21.15.86:0/2098788921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 -- 172.21.15.86:0/2098788921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 msgr2=0x563434fa2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 -- 172.21.15.86:0/2098788921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 msgr2=0x563434f9c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434f9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 -- 172.21.15.86:0/2098788921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563434f96730 con 0x563434f995d0 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f22a3fff640 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2294098040 tx=0x7f229409b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a8c6ace875a9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f2289ffb640 1 -- 172.21.15.86:0/2098788921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294014020 con 0x563434f995d0 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f2289ffb640 1 -- 172.21.15.86:0/2098788921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22940a42e0 con 0x563434f995d0 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.291+0000 7f2289ffb640 1 -- 172.21.15.86:0/2098788921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294020070 con 0x563434f995d0 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- 172.21.15.86:0/2098788921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 msgr2=0x563434f9a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2294098040 tx=0x7f229409b730 comp rx=0 tx=0).stop 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- 172.21.15.86:0/2098788921 shutdown_connections 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434fa4370 0x563434f9c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434f9ab40 0x563434fa2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- 172.21.15.86:0/2098788921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f995d0 0x563434f9a300 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- 172.21.15.86:0/2098788921 >> 172.21.15.86:0/2098788921 conn(0x563434f62310 msgr2=0x563434fa2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- 172.21.15.86:0/2098788921 shutdown_connections 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- 172.21.15.86:0/2098788921 wait complete. 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- start start 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 Processor -- start 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434facbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434fa4370 0x563434f9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b0500 con 0x563434f9ab40 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41afe30 con 0x563434f995d0 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ad090 con 0x563434fa4370 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22ac64d740 10 client.0 _ll_register_callbacks cb 0x7f229409b060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.292+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41862/0 (socket says 172.21.15.86:41862) 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 -- 172.21.15.86:0/3349886794 learned_addr learned my addr 172.21.15.86:0/3349886794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a8811640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434fa4370 0x563434f9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434facbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 -- 172.21.15.86:0/3349886794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434fa4370 msgr2=0x563434f9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434fa4370 0x563434f9e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 -- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 msgr2=0x563434facbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434facbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 -- 172.21.15.86:0/3349886794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563434f9eae0 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22a37fe640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f22980051c0 tx=0x7f2298009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450186c13a67042b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229803f040 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f2298031af0 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22940a42e0 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f229805e050 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22ac64d740 10 client.20397 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56343501ebe0 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.293+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22940a42e0 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f229804c070 con 0x563434f9ab40 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 10 client.20397 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 10 client.20397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:46.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 20 client.20397 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56343501efe0 0x5634350213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 4 client.20397 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563434f06d70 con 0x56343501efe0 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56343501efe0 0x5634350213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22837fe640 10 client.20397 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.294+0000 7f22a3fff640 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56343501efe0 0x5634350213d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd907bee5b4600c server_cookie=7e3b5d6addf88ba8 in_seq=0 out_seq=0 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22837fe640 1 -- 172.21.15.86:0/3349886794 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563434f06d70 con 0x56343501efe0 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22837fe640 10 client.20397 handle_command_reply: tid=0 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 client.20397 shutdown 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 msgr2=0x563434f9dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f22980051c0 tx=0x7f2298009780 comp rx=0 tx=0).stop 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 shutdown_connections 2024-09-05T07:03:46.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56343501efe0 0x5634350213d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563434fa4370 0x563434f9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563434f9ab40 0x563434f9dd70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 --2- 172.21.15.86:0/3349886794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563434f995d0 0x563434facbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 >> 172.21.15.86:0/3349886794 conn(0x563434f62310 msgr2=0x563434fa2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22837fe640 0 client.20397 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 shutdown_connections 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 1 -- 172.21.15.86:0/3349886794 wait complete. 2024-09-05T07:03:46.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22ac64d740 20 client.20397 trim_cache size 0 max 16384 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 msgr2=0x7f228c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 0x7f228c0a51b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c0139f0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 msgr2=0x7f22a41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.295+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f229400d950 tx=0x7f229400de10 comp rx=0 tx=0).stop 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 shutdown_connections 2024-09-05T07:03:46.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f228c0a2cf0 0x7f228c0a51b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22a4106050 0x7f22a419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a4101df0 0x7f22a4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 --2- 172.21.15.86:0/1050702248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101440 0x7f22a41985a0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 >> 172.21.15.86:0/1050702248 conn(0x7f22a4077210 msgr2=0x7f22a4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 shutdown_connections 2024-09-05T07:03:46.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:46.296+0000 7f22aa274640 1 -- 172.21.15.86:0/1050702248 wait complete. 2024-09-05T07:03:46.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.44 MB/sec execute 424 sec latency 19878.028 ms 2024-09-05T07:03:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:47 smithi096 ceph-mon[34468]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-05T07:03:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:47 smithi115 ceph-mon[34791]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-05T07:03:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:47 smithi086 ceph-mon[30943]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 120 KiB/s wr, 14 op/s 2024-09-05T07:03:47.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.42 MB/sec execute 425 sec latency 20878.362 ms 2024-09-05T07:03:47.986 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:03:47.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:03:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 -- 172.21.15.86:0/1677490819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c01027a0 msgr2=0x7f38c01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 --2- 172.21.15.86:0/1677490819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c01027a0 0x7f38c01067f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009900 tx=0x7f38a802a410 comp rx=0 tx=0).stop 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 -- 172.21.15.86:0/1677490819 shutdown_connections 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 --2- 172.21.15.86:0/1677490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 0x7f38c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 --2- 172.21.15.86:0/1677490819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c01027a0 0x7f38c01067f0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 --2- 172.21.15.86:0/1677490819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c0101df0 0x7f38c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 -- 172.21.15.86:0/1677490819 >> 172.21.15.86:0/1677490819 conn(0x7f38c00fd8f0 msgr2=0x7f38c00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 -- 172.21.15.86:0/1677490819 shutdown_connections 2024-09-05T07:03:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.710+0000 7f38c50ab640 1 -- 172.21.15.86:0/1677490819 wait complete. 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.711+0000 7f38c50ab640 1 Processor -- start 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.711+0000 7f38c50ab640 1 -- start start 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.711+0000 7f38c50ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c01027a0 0x7f38c01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 0x7f38c019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38c0111e30 con 0x7f38c0101df0 2024-09-05T07:03:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38c0111cb0 con 0x7f38c0106f20 2024-09-05T07:03:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38c0111fb0 con 0x7f38c01027a0 2024-09-05T07:03:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41882/0 (socket says 172.21.15.86:41882) 2024-09-05T07:03:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 -- 172.21.15.86:0/485983633 learned_addr learned my addr 172.21.15.86:0/485983633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38be59c640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c01027a0 0x7f38c01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bf59e640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 0x7f38c019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 -- 172.21.15.86:0/485983633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c01027a0 msgr2=0x7f38c01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c01027a0 0x7f38c01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 -- 172.21.15.86:0/485983633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 msgr2=0x7f38c019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 0x7f38c019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 -- 172.21.15.86:0/485983633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c019d350 con 0x7f38c0101df0 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38bed9d640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d770 tx=0x7f38b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38b7fff640 1 -- 172.21.15.86:0/485983633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8017070 con 0x7f38c0101df0 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c019d640 con 0x7f38c0101df0 2024-09-05T07:03:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38b7fff640 1 -- 172.21.15.86:0/485983633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b80054d0 con 0x7f38c0101df0 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.712+0000 7f38b7fff640 1 -- 172.21.15.86:0/485983633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004cf0 con 0x7f38c0101df0 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.713+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c01a4f20 con 0x7f38c0101df0 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38b7fff640 1 -- 172.21.15.86:0/485983633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f38b8004e90 con 0x7f38c0101df0 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 Processor -- start 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 -- start start 2024-09-05T07:03:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062846e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x560062855c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600627ac280 con 0x560062856150 2024-09-05T07:03:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560062753b10 con 0x560062847380 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.715+0000 7f38c7484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5600627a8240 con 0x560062846a60 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38b7fff640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 0x7f38ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41892/0 (socket says 172.21.15.86:41892) 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 -- 172.21.15.86:0/2694122041 learned_addr learned my addr 172.21.15.86:0/2694122041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:48.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38bed9d640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x560062855c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38bf59e640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062846e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38bf59e640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 0x7f38ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38b7fff640 1 -- 172.21.15.86:0/485983633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f38b80cf300 con 0x7f38c0101df0 2024-09-05T07:03:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 -- 172.21.15.86:0/2694122041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 msgr2=0x560062846e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062846e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 -- 172.21.15.86:0/2694122041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 msgr2=0x560062855c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x560062855c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 -- 172.21.15.86:0/2694122041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600627601f0 con 0x560062856150 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38bed9d640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x560062855c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38be59c640 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009900 tx=0x7f38a8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e103bdf2754b76e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38bf59e640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 0x7f38ac0a51b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bfd0 tx=0x7f38b0005c20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38b57fa640 1 -- 172.21.15.86:0/2694122041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8040070 con 0x560062856150 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38b57fa640 1 -- 172.21.15.86:0/2694122041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38a80316a0 con 0x560062856150 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.716+0000 7f38b57fa640 1 -- 172.21.15.86:0/2694122041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8035040 con 0x560062856150 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- 172.21.15.86:0/2694122041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 msgr2=0x560062858540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009900 tx=0x7f38a8032e40 comp rx=0 tx=0).stop 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- 172.21.15.86:0/2694122041 shutdown_connections 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x560062858540 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x560062855c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 --2- 172.21.15.86:0/2694122041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062846e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- 172.21.15.86:0/2694122041 >> 172.21.15.86:0/2694122041 conn(0x5600628430f0 msgr2=0x5600628434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- 172.21.15.86:0/2694122041 shutdown_connections 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- 172.21.15.86:0/2694122041 wait complete. 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.717+0000 7f38c7484740 1 -- start start 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 Processor -- start 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062853030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 0x560062853ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560062858d20 con 0x560062847380 2024-09-05T07:03:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560062766f10 con 0x560062856150 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560062858ea0 con 0x560062846a60 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38c7484740 10 client.0 _ll_register_callbacks cb 0x56006272fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41900/0 (socket says 172.21.15.86:41900) 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 -- 172.21.15.86:0/1193420133 learned_addr learned my addr 172.21.15.86:0/1193420133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38bf59e640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 0x560062853ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38bed9d640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062853030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 -- 172.21.15.86:0/1193420133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 msgr2=0x560062853030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062853030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 -- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 msgr2=0x560062853ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 0x560062853ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 -- 172.21.15.86:0/1193420133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006251d6a0 con 0x560062847380 2024-09-05T07:03:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38bf59e640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 0x560062853ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38bed9d640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062853030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38be59c640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f38a80096c0 tx=0x7f38a8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c30ab030d9dd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.718+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8035070 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f38a8004030 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a80316a0 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f38a8055050 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 10 client.20409 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560062864990 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38a80316a0 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f38a80046c0 con 0x560062847380 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 10 client.20409 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 10 client.20409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 20 client.20409 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560062865ce0 0x5600628680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 4 client.20409 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600627b97f0 con 0x560062865ce0 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.719+0000 7f38bed9d640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560062865ce0 0x5600628680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.720+0000 7f38bed9d640 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560062865ce0 0x5600628680d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0ccf7d89e49492 server_cookie=1bbfed38bbdf1b98 in_seq=0 out_seq=0 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.720+0000 7f38a67fc640 10 client.20409 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.720+0000 7f38a67fc640 1 -- 172.21.15.86:0/1193420133 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600627b97f0 con 0x560062865ce0 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.720+0000 7f38a67fc640 10 client.20409 handle_command_reply: tid=0 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.720+0000 7f38c7484740 1 client.20409 shutdown 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 msgr2=0x560062853570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f38a80096c0 tx=0x7f38a8031400 comp rx=0 tx=0).stop 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 shutdown_connections 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560062865ce0 0x5600628680d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062856150 0x560062853ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062847380 0x560062853570 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38a67fc640 0 client.20409 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 --2- 172.21.15.86:0/1193420133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062853030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 >> 172.21.15.86:0/1193420133 conn(0x5600628430f0 msgr2=0x560062848e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 shutdown_connections 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 1 -- 172.21.15.86:0/1193420133 wait complete. 2024-09-05T07:03:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.721+0000 7f38c7484740 20 client.20409 trim_cache size 0 max 16384 2024-09-05T07:03:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.738+0000 7f38c7484740 1 Processor -- start 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.738+0000 7f38c7484740 1 -- start start 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062854e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006286d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01acf20 con 0x560062856150 2024-09-05T07:03:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01aea60 con 0x560062847380 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38c7484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01ad5f0 con 0x560062846a60 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41902/0 (socket says 172.21.15.86:41902) 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 -- 172.21.15.86:0/2811177446 learned_addr learned my addr 172.21.15.86:0/2811177446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38be59c640 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006286d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bed9d640 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062854e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 -- 172.21.15.86:0/2811177446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 msgr2=0x560062854e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062854e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 -- 172.21.15.86:0/2811177446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 msgr2=0x56006286d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006286d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 -- 172.21.15.86:0/2811177446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560062843ca0 con 0x560062856150 2024-09-05T07:03:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.739+0000 7f38bf59e640 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005fb0 tx=0x7f38b000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ad89335d61145a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38b4ff9640 1 -- 172.21.15.86:0/2811177446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b000ebf0 con 0x560062856150 2024-09-05T07:03:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38b4ff9640 1 -- 172.21.15.86:0/2811177446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b000ed90 con 0x560062856150 2024-09-05T07:03:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38b4ff9640 1 -- 172.21.15.86:0/2811177446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b001f750 con 0x560062856150 2024-09-05T07:03:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 -- 172.21.15.86:0/2811177446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 msgr2=0x56006286dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005fb0 tx=0x7f38b000b540 comp rx=0 tx=0).stop 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 -- 172.21.15.86:0/2811177446 shutdown_connections 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062856150 0x56006286dbe0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006286d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 --2- 172.21.15.86:0/2811177446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062846a60 0x560062854e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.740+0000 7f38c7484740 1 -- 172.21.15.86:0/2811177446 >> 172.21.15.86:0/2811177446 conn(0x5600628430f0 msgr2=0x560062854780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- 172.21.15.86:0/2811177446 shutdown_connections 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- 172.21.15.86:0/2811177446 wait complete. 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- start start 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 Processor -- start 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006285a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 0x56006285aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01ae390 con 0x560062846a60 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01adcc0 con 0x560062847380 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f38c01aaf20 con 0x560062856150 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38c7484740 10 client.0 _ll_register_callbacks cb 0x7f38b000bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41914/0 (socket says 172.21.15.86:41914) 2024-09-05T07:03:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38bed9d640 1 -- 172.21.15.86:0/641257280 learned_addr learned my addr 172.21.15.86:0/641257280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38be59c640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006285a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38bf59e640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 0x56006285aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.741+0000 7f38bed9d640 1 -- 172.21.15.86:0/641257280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 msgr2=0x56006285aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bed9d640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 0x56006285aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bed9d640 1 -- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 msgr2=0x56006285a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bed9d640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006285a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bed9d640 1 -- 172.21.15.86:0/641257280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600628cf7d0 con 0x560062846a60 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38be59c640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006285a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bf59e640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 0x56006285aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38bed9d640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f38b8097db0 tx=0x7f38b80664b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529023149f9efcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8020040 con 0x560062846a60 2024-09-05T07:03:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f38b8097a50 con 0x560062846a60 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b000ed90 con 0x560062846a60 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f38b8009c30 con 0x560062846a60 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 10 client.20415 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5600628d0af0 con 0x560062846a60 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b000ed90 con 0x560062846a60 2024-09-05T07:03:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f38b8016170 con 0x560062846a60 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 10 client.20415 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 10 client.20415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 20 client.20415 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600628d11c0 0x5600628d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 4 client.20415 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.742+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600627b3d00 con 0x5600628d11c0 2024-09-05T07:03:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.743+0000 7f38be59c640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600628d11c0 0x5600628d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.743+0000 7f38be59c640 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600628d11c0 0x5600628d35b0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8668cc1a7d5ec4 server_cookie=3224fb856b66bbd3 in_seq=0 out_seq=0 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.743+0000 7f38a67fc640 10 client.20415 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38a67fc640 1 -- 172.21.15.86:0/641257280 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600627b3d00 con 0x5600628d11c0 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38a67fc640 10 client.20415 handle_command_reply: tid=0 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 client.20415 shutdown 2024-09-05T07:03:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 msgr2=0x560062859fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f38b8097db0 tx=0x7f38b80664b0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 shutdown_connections 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5600628d11c0 0x5600628d35b0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560062856150 0x56006285aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560062847380 0x56006285a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 --2- 172.21.15.86:0/641257280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560062846a60 0x560062859fd0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 >> 172.21.15.86:0/641257280 conn(0x5600628430f0 msgr2=0x560062842270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.744+0000 7f38a67fc640 0 client.20415 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 shutdown_connections 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c7484740 1 -- 172.21.15.86:0/641257280 wait complete. 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c7484740 20 client.20415 trim_cache size 0 max 16384 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2db0c633-6dab-40c9-9976-8b7a49220f47", 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:03:48.773 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 msgr2=0x7f38ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 0x7f38ac0a51b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bfd0 tx=0x7f38b0005c20 comp rx=0 tx=0).stop 2024-09-05T07:03:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 msgr2=0x7f38c01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d770 tx=0x7f38b800dc30 comp rx=0 tx=0).stop 2024-09-05T07:03:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 shutdown_connections 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f38ac0a2cf0 0x7f38ac0a51b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38c0106f20 0x7f38c019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c01027a0 0x7f38c01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 --2- 172.21.15.86:0/485983633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38c0101df0 0x7f38c01983a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 >> 172.21.15.86:0/485983633 conn(0x7f38c00fd8f0 msgr2=0x7f38c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 shutdown_connections 2024-09-05T07:03:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:48.745+0000 7f38c50ab640 1 -- 172.21.15.86:0/485983633 wait complete. 2024-09-05T07:03:48.779 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.41 MB/sec execute 426 sec latency 21878.406 ms 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 150 KiB/s wr, 19 op/s 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.511 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:49 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 150 KiB/s wr, 19 op/s 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:49 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 150 KiB/s wr, 19 op/s 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:49 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:49.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.39 MB/sec execute 427 sec latency 22878.466 ms 2024-09-05T07:03:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 -- 172.21.15.86:0/1483950863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 msgr2=0x7f313010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 --2- 172.21.15.86:0/1483950863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313010da40 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3128009ca0 tx=0x7f312802a950 comp rx=0 tx=0).stop 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 -- 172.21.15.86:0/1483950863 shutdown_connections 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 --2- 172.21.15.86:0/1483950863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313010da40 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 --2- 172.21.15.86:0/1483950863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 0x7f31301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 --2- 172.21.15.86:0/1483950863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 0x7f3130101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 -- 172.21.15.86:0/1483950863 >> 172.21.15.86:0/1483950863 conn(0x7f31300fc7d0 msgr2=0x7f31300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 -- 172.21.15.86:0/1483950863 shutdown_connections 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.260+0000 7f3136e74640 1 -- 172.21.15.86:0/1483950863 wait complete. 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 Processor -- start 2024-09-05T07:03:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 -- start start 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 0x7f31301972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 0x7f3130197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3130110e30 con 0x7f3130102ef0 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3130110cb0 con 0x7f3130102570 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3136e74640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3130110fb0 con 0x7f3130100e20 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41932/0 (socket says 172.21.15.86:41932) 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 -- 172.21.15.86:0/339870895 learned_addr learned my addr 172.21.15.86:0/339870895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f312ffff640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 0x7f3130197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3134c10640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 0x7f31301972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 -- 172.21.15.86:0/339870895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 msgr2=0x7f31301972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 0x7f31301972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 -- 172.21.15.86:0/339870895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 msgr2=0x7f3130197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 0x7f3130197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.261+0000 7f3135411640 1 -- 172.21.15.86:0/339870895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313019c2a0 con 0x7f3130102ef0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f3135411640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f312802ae70 tx=0x7f312802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f312dffb640 1 -- 172.21.15.86:0/339870895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128034040 con 0x7f3130102ef0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f312dffb640 1 -- 172.21.15.86:0/339870895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31280308f0 con 0x7f3130102ef0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313019c530 con 0x7f3130102ef0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f312dffb640 1 -- 172.21.15.86:0/339870895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128004780 con 0x7f3130102ef0 2024-09-05T07:03:50.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.262+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301a3e70 con 0x7f3130102ef0 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.264+0000 7f312dffb640 1 -- 172.21.15.86:0/339870895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3128037020 con 0x7f3130102ef0 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 Processor -- start 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 -- start start 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c66dc0 0x5612b9c71140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9c73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612b9bcc2e0 con 0x5612b9c664a0 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612b9b73ce0 con 0x5612b9c66dc0 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f313924d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5612b9bc82a0 con 0x5612b9c71680 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41944/0 (socket says 172.21.15.86:41944) 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 -- 172.21.15.86:0/4218915440 learned_addr learned my addr 172.21.15.86:0/4218915440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312dffb640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 0x7f31200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f3134c10640 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c66dc0 0x5612b9c71140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f3135411640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 0x7f31200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 -- 172.21.15.86:0/4218915440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 msgr2=0x5612b9c73a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9c73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 -- 172.21.15.86:0/4218915440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c66dc0 msgr2=0x5612b9c71140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c66dc0 0x5612b9c71140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312ffff640 1 -- 172.21.15.86:0/4218915440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9b58440 con 0x5612b9c664a0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.265+0000 7f312dffb640 1 -- 172.21.15.86:0/339870895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f31280f0960 con 0x7f3130102ef0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f312ffff640 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f311c00cba0 tx=0x7f311c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6dec6443bfda71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f311affd640 1 -- 172.21.15.86:0/4218915440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c016030 con 0x5612b9c664a0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f3135411640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 0x7f31200a51b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31280bca80 tx=0x7f31280095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f311affd640 1 -- 172.21.15.86:0/4218915440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f311c0054d0 con 0x5612b9c664a0 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f311affd640 1 -- 172.21.15.86:0/4218915440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c012480 con 0x5612b9c664a0 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f313924d740 1 -- 172.21.15.86:0/4218915440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 msgr2=0x5612b9c66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.266+0000 7f313924d740 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f311c00cba0 tx=0x7f311c010740 comp rx=0 tx=0).stop 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- 172.21.15.86:0/4218915440 shutdown_connections 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9c73a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c66dc0 0x5612b9c71140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- 172.21.15.86:0/4218915440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c664a0 0x5612b9c66880 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- 172.21.15.86:0/4218915440 >> 172.21.15.86:0/4218915440 conn(0x5612b9c62a30 msgr2=0x5612b9c62e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- 172.21.15.86:0/4218915440 shutdown_connections 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- 172.21.15.86:0/4218915440 wait complete. 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- start start 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 Processor -- start 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c70f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c66dc0 0x5612b9c6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612b9c74250 con 0x5612b9c71680 2024-09-05T07:03:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612b9b87050 con 0x5612b9c664a0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5612b9c743d0 con 0x5612b9c66dc0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.267+0000 7f313924d740 10 client.0 _ll_register_callbacks cb 0x5612b9bc7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41960/0 (socket says 172.21.15.86:41960) 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 -- 172.21.15.86:0/1667147108 learned_addr learned my addr 172.21.15.86:0/1667147108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c70f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f3134c10640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c66dc0 0x5612b9c6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 -- 172.21.15.86:0/1667147108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c66dc0 msgr2=0x5612b9c6e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c66dc0 0x5612b9c6e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 -- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 msgr2=0x5612b9c70f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c70f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 -- 172.21.15.86:0/1667147108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b993d3c0 con 0x5612b9c71680 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f312ffff640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f311c0057c0 tx=0x7f311c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72733a350c19aff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f310ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c01f020 con 0x5612b9c71680 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f311c0051a0 con 0x5612b9c71680 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311c0054d0 con 0x5612b9c71680 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f311c02f050 con 0x5612b9c71680 2024-09-05T07:03:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f313924d740 10 client.20427 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.268+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5612b9c7fbc0 con 0x5612b9c71680 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f311c0054d0 con 0x5612b9c71680 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f311c003a20 con 0x5612b9c71680 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 10 client.20427 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 10 client.20427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 20 client.20427 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9c81080 0x5612b9c83470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 4 client.20427 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b9bd9850 con 0x5612b9c81080 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f3134c10640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9c81080 0x5612b9c83470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f3134c10640 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9c81080 0x5612b9c83470 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ca61bb4a8540b4 server_cookie=3d60c43c4264713f in_seq=0 out_seq=0 2024-09-05T07:03:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.269+0000 7f310ffff640 10 client.20427 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f310ffff640 1 -- 172.21.15.86:0/1667147108 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612b9bd9850 con 0x5612b9c81080 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f310ffff640 10 client.20427 handle_command_reply: tid=0 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 client.20427 shutdown 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 msgr2=0x5612b9c6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f311c0057c0 tx=0x7f311c005900 comp rx=0 tx=0).stop 2024-09-05T07:03:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 shutdown_connections 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9c81080 0x5612b9c83470 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c71680 0x5612b9c6e760 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f310ffff640 0 client.20427 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c66dc0 0x5612b9c6e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 --2- 172.21.15.86:0/1667147108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c70f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 >> 172.21.15.86:0/1667147108 conn(0x5612b9c62a30 msgr2=0x5612b9c68760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 shutdown_connections 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.270+0000 7f313924d740 1 -- 172.21.15.86:0/1667147108 wait complete. 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.271+0000 7f313924d740 20 client.20427 trim_cache size 0 max 16384 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 Processor -- start 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 -- start start 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c664a0 0x5612b9c70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c71680 0x5612b9c79d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31301adf20 con 0x5612b9c66dc0 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31301afa60 con 0x5612b9c71680 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.287+0000 7f313924d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31301ae5f0 con 0x5612b9c664a0 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41976/0 (socket says 172.21.15.86:41976) 2024-09-05T07:03:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 -- 172.21.15.86:0/1855401541 learned_addr learned my addr 172.21.15.86:0/1855401541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f312ffff640 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c71680 0x5612b9c79d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f312ffff640 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c664a0 0x5612b9c70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 -- 172.21.15.86:0/1855401541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c664a0 msgr2=0x5612b9c70190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c664a0 0x5612b9c70190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 -- 172.21.15.86:0/1855401541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c71680 msgr2=0x5612b9c79d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c71680 0x5612b9c79d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 -- 172.21.15.86:0/1855401541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9c62860 con 0x5612b9c66dc0 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f3134c10640 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f202c0ccdd786ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f311a7fc640 1 -- 172.21.15.86:0/1855401541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240140d0 con 0x5612b9c66dc0 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f311a7fc640 1 -- 172.21.15.86:0/1855401541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3124014270 con 0x5612b9c66dc0 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.288+0000 7f311a7fc640 1 -- 172.21.15.86:0/1855401541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124026730 con 0x5612b9c66dc0 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- 172.21.15.86:0/1855401541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 msgr2=0x5612b9c79810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124014840 comp rx=0 tx=0).stop 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- 172.21.15.86:0/1855401541 shutdown_connections 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c71680 0x5612b9c79d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c79810 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 --2- 172.21.15.86:0/1855401541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c664a0 0x5612b9c70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- 172.21.15.86:0/1855401541 >> 172.21.15.86:0/1855401541 conn(0x5612b9c62a30 msgr2=0x5612b9c6fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- 172.21.15.86:0/1855401541 shutdown_connections 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- 172.21.15.86:0/1855401541 wait complete. 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.289+0000 7f313924d740 1 -- start start 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 Processor -- start 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c6b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9ce4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31301af390 con 0x5612b9c66dc0 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31301aecc0 con 0x5612b9c664a0 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31301abf20 con 0x5612b9c71680 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f313924d740 10 client.0 _ll_register_callbacks cb 0x7f31240092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41984/0 (socket says 172.21.15.86:41984) 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 -- 172.21.15.86:0/3489694083 learned_addr learned my addr 172.21.15.86:0/3489694083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f312ffff640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c6b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f312ffff640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9ce4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 -- 172.21.15.86:0/3489694083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 msgr2=0x5612b9ce4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9ce4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 -- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 msgr2=0x5612b9c6b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c6b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 -- 172.21.15.86:0/3489694083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b9ce55f0 con 0x5612b9c66dc0 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f3134c10640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3124009cd0 tx=0x7f3124014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea46756346735a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.290+0000 7f310ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240140d0 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f3124022070 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3124014270 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3124005240 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 10 client.20433 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5612b9cedbc0 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3124014270 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f312400f040 con 0x5612b9c66dc0 2024-09-05T07:03:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 10 client.20433 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 10 client.20433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 20 client.20433 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9cee370 0x5612b9cf0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 4 client.20433 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612b9c6eaf0 con 0x5612b9cee370 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.291+0000 7f312ffff640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9cee370 0x5612b9cf0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f312ffff640 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9cee370 0x5612b9cf0790 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3bd5fd65d06bb8 server_cookie=9136d086068f393b in_seq=0 out_seq=0 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f310ffff640 10 client.20433 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f310ffff640 1 -- 172.21.15.86:0/3489694083 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5612b9c6eaf0 con 0x5612b9cee370 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f310ffff640 10 client.20433 handle_command_reply: tid=0 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stdout: "2db0c633-6dab-40c9-9976-8b7a49220f47": { 2024-09-05T07:03:50.317 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2db0c633-6dab-40c9-9976-8b7a49220f47", 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 client.20433 shutdown 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 msgr2=0x5612b9c6b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3124009cd0 tx=0x7f3124014f90 comp rx=0 tx=0).stop 2024-09-05T07:03:50.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 shutdown_connections 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5612b9cee370 0x5612b9cf0790 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612b9c71680 0x5612b9ce4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b9c66dc0 0x5612b9c6b740 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 --2- 172.21.15.86:0/3489694083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612b9c664a0 0x5612b9c6b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 >> 172.21.15.86:0/3489694083 conn(0x5612b9c62a30 msgr2=0x5612b9c6fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f310ffff640 0 client.20433 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 shutdown_connections 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 1 -- 172.21.15.86:0/3489694083 wait complete. 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.292+0000 7f313924d740 20 client.20433 trim_cache size 0 max 16384 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 msgr2=0x7f31200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 0x7f31200a51b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31280bca80 tx=0x7f31280095c0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 msgr2=0x7f313019bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:50.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f312802ae70 tx=0x7f312802f5d0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 shutdown_connections 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f31200a2cf0 0x7f31200a51b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130102ef0 0x7f313019bbc0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3130102570 0x7f3130197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 --2- 172.21.15.86:0/339870895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3130100e20 0x7f31301972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 >> 172.21.15.86:0/339870895 conn(0x7f31300fc7d0 msgr2=0x7f31300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 shutdown_connections 2024-09-05T07:03:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:50.293+0000 7f3136e74640 1 -- 172.21.15.86:0/339870895 wait complete. 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:03:50.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.38 MB/sec execute 428 sec latency 23878.592 ms 2024-09-05T07:03:51.087 INFO:tasks.cephfs.filesystem:scrub status for tag:2db0c633-6dab-40c9-9976-8b7a49220f47 - {'path': '/', 'tag': '2db0c633-6dab-40c9-9976-8b7a49220f47', 'options': 'recursive,force'} 2024-09-05T07:03:51.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:51 smithi086 ceph-mon[30943]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-05T07:03:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:51 smithi096 ceph-mon[34468]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-05T07:03:51.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:51 smithi115 ceph-mon[34791]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-05T07:03:51.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.36 MB/sec execute 429 sec latency 24878.689 ms 2024-09-05T07:03:51.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:03:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 -- 172.21.15.86:0/3623388303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 msgr2=0x7f3394100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 --2- 172.21.15.86:0/3623388303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f3394100f10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f338802a470 comp rx=0 tx=0).stop 2024-09-05T07:03:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 -- 172.21.15.86:0/3623388303 shutdown_connections 2024-09-05T07:03:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 --2- 172.21.15.86:0/3623388303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394105f40 0x7f33941063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 --2- 172.21.15.86:0/3623388303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394101450 0x7f3394105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 --2- 172.21.15.86:0/3623388303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f3394100f10 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.727+0000 7f339bfd8640 1 -- 172.21.15.86:0/3623388303 >> 172.21.15.86:0/3623388303 conn(0x7f33940fc7a0 msgr2=0x7f33940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- 172.21.15.86:0/3623388303 shutdown_connections 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- 172.21.15.86:0/3623388303 wait complete. 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 Processor -- start 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- start start 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394101450 0x7f33941a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394105f40 0x7f339419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33941184f0 con 0x7f3394100b30 2024-09-05T07:03:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3394118370 con 0x7f3394105f40 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f339bfd8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3394118670 con 0x7f3394101450 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f3399d74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.728+0000 7f3399d74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60558/0 (socket says 172.21.15.86:60558) 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 -- 172.21.15.86:0/1748848707 learned_addr learned my addr 172.21.15.86:0/1748848707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f339a575640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394105f40 0x7f339419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399573640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394101450 0x7f33941a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 -- 172.21.15.86:0/1748848707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394101450 msgr2=0x7f33941a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394101450 0x7f33941a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 -- 172.21.15.86:0/1748848707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394105f40 msgr2=0x7f339419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394105f40 0x7f339419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 -- 172.21.15.86:0/1748848707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339419f0b0 con 0x7f3394100b30 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3399d74640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f3388006010 tx=0x7f3388004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:52.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338803f030 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33880035f0 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339419f340 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388035070 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.729+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33941ab690 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.730+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33941059a0 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.731+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3388003b10 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.732+0000 7f3386ffd640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.732+0000 7f3399573640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.732+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f33880f06d0 con 0x7f3394100b30 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.732+0000 7f3399573640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33940fff80 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:52.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.736+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33880bceb0 con 0x7f3394100b30 2024-09-05T07:03:52.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.35 MB/sec execute 430 sec latency 25878.736 ms 2024-09-05T07:03:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.939+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33941ab860 con 0x7f3394100b30 2024-09-05T07:03:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.940+0000 7f3386ffd640 1 -- 172.21.15.86:0/1748848707 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f33880bc7b0 con 0x7f3394100b30 2024-09-05T07:03:52.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:03:52.941 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:03:52.941 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 msgr2=0x7f33800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33940fff80 tx=0x7f3390008040 comp rx=0 tx=0).stop 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 msgr2=0x7f33941a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f3388006010 tx=0x7f3388004270 comp rx=0 tx=0).stop 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 shutdown_connections 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3394105f40 0x7f339419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3394101450 0x7f33941a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 --2- 172.21.15.86:0/1748848707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3394100b30 0x7f33941a47c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:52.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.942+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 >> 172.21.15.86:0/1748848707 conn(0x7f33940fc7a0 msgr2=0x7f339410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:52.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.943+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 shutdown_connections 2024-09-05T07:03:52.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:52.943+0000 7f339bfd8640 1 -- 172.21.15.86:0/1748848707 wait complete. 2024-09-05T07:03:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:53 smithi096 ceph-mon[34468]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-05T07:03:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:53 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1748848707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:53.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:53 smithi115 ceph-mon[34791]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-05T07:03:53.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:53 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1748848707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:53.697 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:53 smithi086 ceph-mon[30943]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-05T07:03:53.697 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:53 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1748848707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:53.727 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 226972e1 --release --await' 2024-09-05T07:03:53.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 226972e1 --release --await 2024-09-05T07:03:53.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65792 6.34 MB/sec execute 431 sec latency 26878.789 ms 2024-09-05T07:03:53.982 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:54.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 -- 172.21.15.86:0/289021874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4101de0 msgr2=0x7fbde41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 --2- 172.21.15.86:0/289021874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4101de0 0x7fbde41021c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009900 tx=0x7fbddc02a410 comp rx=0 tx=0).stop 2024-09-05T07:03:54.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 -- 172.21.15.86:0/289021874 shutdown_connections 2024-09-05T07:03:54.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 --2- 172.21.15.86:0/289021874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4106f10 0x7fbde410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 --2- 172.21.15.86:0/289021874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4102790 0x7fbde41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 --2- 172.21.15.86:0/289021874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4101de0 0x7fbde41021c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 -- 172.21.15.86:0/289021874 >> 172.21.15.86:0/289021874 conn(0x7fbde40fd8c0 msgr2=0x7fbde40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 -- 172.21.15.86:0/289021874 shutdown_connections 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.493+0000 7fbde981a640 1 -- 172.21.15.86:0/289021874 wait complete. 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 Processor -- start 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 -- start start 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 0x7fbde41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4106f10 0x7fbde419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4111e20 con 0x7fbde4102790 2024-09-05T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4111ca0 con 0x7fbde4101de0 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.494+0000 7fbde981a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4111fa0 con 0x7fbde4106f10 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60574/0 (socket says 172.21.15.86:60574) 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 -- 172.21.15.86:0/1485093107 learned_addr learned my addr 172.21.15.86:0/1485093107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde37fe640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4106f10 0x7fbde419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 0x7fbde41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 -- 172.21.15.86:0/1485093107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4106f10 msgr2=0x7fbde419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4106f10 0x7fbde419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 -- 172.21.15.86:0/1485093107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 msgr2=0x7fbde41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 0x7fbde41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 -- 172.21.15.86:0/1485093107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde419d390 con 0x7fbde4102790 2024-09-05T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 0x7fbde41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800ea80 tx=0x7fbdd800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde8818640 1 -- 172.21.15.86:0/1485093107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800ce40 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde419d680 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde8818640 1 -- 172.21.15.86:0/1485093107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd8005020 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde8818640 1 -- 172.21.15.86:0/1485093107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80116d0 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.495+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41a4f60 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde8818640 1 -- 172.21.15.86:0/1485093107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbdd80051d0 con 0x7fbde4102790 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 Processor -- start 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 -- start start 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589324f60 0x557589325340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x557589334110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55758928a620 con 0x557589334650 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557589231e50 con 0x557589325880 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5575892865e0 con 0x557589324f60 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60590/0 (socket says 172.21.15.86:60590) 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 -- 172.21.15.86:0/2801631247 learned_addr learned my addr 172.21.15.86:0/2801631247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde37fe640 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589324f60 0x557589325340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x557589334110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde8818640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 0x7fbdcc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 -- 172.21.15.86:0/2801631247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589324f60 msgr2=0x557589325340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589324f60 0x557589325340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 -- 172.21.15.86:0/2801631247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 msgr2=0x557589334110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x557589334110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.498+0000 7fbde27fc640 1 -- 172.21.15.86:0/2801631247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557589222d40 con 0x557589334650 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbde37fe640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 0x7fbdcc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbde8818640 1 -- 172.21.15.86:0/1485093107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fbdd80cfbc0 con 0x7fbde4102790 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbde27fc640 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80cce40 tx=0x7fbdd80668d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457f5a68c214e997 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbdc9ffb640 1 -- 172.21.15.86:0/2801631247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801f070 con 0x557589334650 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbdc9ffb640 1 -- 172.21.15.86:0/2801631247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd809c3a0 con 0x557589334650 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbde37fe640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 0x7fbdcc0a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005fd0 tx=0x7fbdd40046c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.499+0000 7fbdc9ffb640 1 -- 172.21.15.86:0/2801631247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd806f210 con 0x557589334650 2024-09-05T07:03:54.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- 172.21.15.86:0/2801631247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 msgr2=0x557589336a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80cce40 tx=0x7fbdd80668d0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- 172.21.15.86:0/2801631247 shutdown_connections 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589334650 0x557589336a40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x557589334110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/2801631247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589324f60 0x557589325340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- 172.21.15.86:0/2801631247 >> 172.21.15.86:0/2801631247 conn(0x5575893215f0 msgr2=0x5575893219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- 172.21.15.86:0/2801631247 shutdown_connections 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- 172.21.15.86:0/2801631247 wait complete. 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 -- start start 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 Processor -- start 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 0x557589331520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.500+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589334650 0x557589331fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557589337220 con 0x557589325880 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557589212f10 con 0x557589324f60 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5575893373a0 con 0x557589334650 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 10 client.0 _ll_register_callbacks cb 0x557589236db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60606/0 (socket says 172.21.15.86:60606) 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 -- 172.21.15.86:0/1132745634 learned_addr learned my addr 172.21.15.86:0/1132745634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde37fe640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589334650 0x557589331fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 0x557589331520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 -- 172.21.15.86:0/1132745634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589334650 msgr2=0x557589331fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589334650 0x557589331fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 -- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 msgr2=0x557589331520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 0x557589331520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 -- 172.21.15.86:0/1132745634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557588f99580 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 0x557589331520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbde27fc640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8098670 tx=0x7fbdd80d31c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5233e8ba531a1054 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801f040 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fbdd8099d40 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd809c3a0 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbdd809b7e0 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 10 client.20451 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.501+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557589063960 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd809c3a0 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fbdd800f040 con 0x557589325880 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 10 client.20451 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:54.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 10 client.20451 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 20 client.20451 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589344070 0x557589346460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 4 client.20451 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557589297b90 con 0x557589344070 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589344070 0x557589346460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589344070 0x557589346460 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbf3feceb012080 server_cookie=bb97236610d03b18 in_seq=0 out_seq=0 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.502+0000 7fbdc2ffd640 10 client.20451 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/1132745634 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557589297b90 con 0x557589344070 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdc2ffd640 10 client.20451 handle_command_reply: tid=0 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 client.20451 shutdown 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 msgr2=0x557589331a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8098670 tx=0x7fbdd80d31c0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 shutdown_connections 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589344070 0x557589346460 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557589334650 0x557589331fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589325880 0x557589331a60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/1132745634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589324f60 0x557589331520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 >> 172.21.15.86:0/1132745634 conn(0x5575893215f0 msgr2=0x557589327300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdc2ffd640 0 client.20451 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 shutdown_connections 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.503+0000 7fbdebbf3740 1 -- 172.21.15.86:0/1132745634 wait complete. 2024-09-05T07:03:54.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.504+0000 7fbdebbf3740 20 client.20451 trim_cache size 0 max 16384 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 Processor -- start 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 -- start start 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x55758932c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x55758934ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41acf60 con 0x557589324f60 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41aeaa0 con 0x557589325880 2024-09-05T07:03:54.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41ad630 con 0x5575893215f0 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60622/0 (socket says 172.21.15.86:60622) 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.520+0000 7fbde27fc640 1 -- 172.21.15.86:0/4124710491 learned_addr learned my addr 172.21.15.86:0/4124710491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde37fe640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x55758934ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x55758932c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 -- 172.21.15.86:0/4124710491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 msgr2=0x55758932c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x55758932c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 -- 172.21.15.86:0/4124710491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 msgr2=0x55758934ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x55758934ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 -- 172.21.15.86:0/4124710491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557589333460 con 0x557589324f60 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x55758932c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbde27fc640 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b00d0 tx=0x7fbdd80d33c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91291a187e3e7506 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbdc97fa640 1 -- 172.21.15.86:0/4124710491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801f040 con 0x557589324f60 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbdc97fa640 1 -- 172.21.15.86:0/4124710491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd80b9020 con 0x557589324f60 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.521+0000 7fbdc97fa640 1 -- 172.21.15.86:0/4124710491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd809b600 con 0x557589324f60 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- 172.21.15.86:0/4124710491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 msgr2=0x557589320640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b00d0 tx=0x7fbdd80d33c0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- 172.21.15.86:0/4124710491 shutdown_connections 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x55758934ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589320640 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/4124710491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x55758932c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- 172.21.15.86:0/4124710491 >> 172.21.15.86:0/4124710491 conn(0x55758932cf10 msgr2=0x55758931ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- 172.21.15.86:0/4124710491 shutdown_connections 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- 172.21.15.86:0/4124710491 wait complete. 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.522+0000 7fbdebbf3740 1 -- start start 2024-09-05T07:03:54.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 Processor -- start 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x5575893389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x5575893ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41ae3d0 con 0x557589324f60 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41add00 con 0x557589325880 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde41aaf60 con 0x5575893215f0 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdebbf3740 10 client.0 _ll_register_callbacks cb 0x7fbdd8097e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60628/0 (socket says 172.21.15.86:60628) 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 -- 172.21.15.86:0/3201613393 learned_addr learned my addr 172.21.15.86:0/3201613393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde37fe640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x5575893ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x5575893389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 -- 172.21.15.86:0/3201613393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 msgr2=0x5575893389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x5575893389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 -- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 msgr2=0x5575893ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x5575893ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 -- 172.21.15.86:0/3201613393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575893adf10 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbde27fc640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8098670 tx=0x7fbdd809bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c9019d1def8e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.523+0000 7fbdc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd801f070 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7fbdd8099fa0 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd80b9020 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbdd809b600 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 10 client.20457 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5575893adf10 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdd80b9020 con 0x557589324f60 2024-09-05T07:03:54.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fbdd8098310 con 0x557589324f60 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 10 client.20457 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 10 client.20457 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 20 client.20457 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589334650 0x5575893b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 4 client.20457 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "226972e1", "release": true, "await": true}] 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "226972e1", "release": true, "await": true}) -- 0x5575892920a0 con 0x557589334650 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.524+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589334650 0x5575893b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.525+0000 7fbde2ffd640 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589334650 0x5575893b1410 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b91b74d48f8b47 server_cookie=ed6cc6c8cb2e16c7 in_seq=0 out_seq=0 2024-09-05T07:03:54.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.525+0000 7fbdc2ffd640 10 client.20457 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.554+0000 7fbdc2ffd640 1 -- 172.21.15.86:0/3201613393 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5575892920a0 con 0x557589334650 2024-09-05T07:03:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.554+0000 7fbdc2ffd640 10 client.20457 handle_command_reply: tid=0 2024-09-05T07:03:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.554+0000 7fbdebbf3740 1 client.20457 shutdown 2024-09-05T07:03:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.554+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 msgr2=0x557589338f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.554+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8098670 tx=0x7fbdd809bca0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 shutdown_connections 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557589334650 0x5575893b1410 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557589325880 0x5575893ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557589324f60 0x557589338f00 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 --2- 172.21.15.86:0/3201613393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5575893215f0 0x5575893389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 >> 172.21.15.86:0/3201613393 conn(0x55758932cf10 msgr2=0x557589320750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdc2ffd640 0 client.20457 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 shutdown_connections 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 1 -- 172.21.15.86:0/3201613393 wait complete. 2024-09-05T07:03:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.555+0000 7fbdebbf3740 20 client.20457 trim_cache size 0 max 16384 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 46, 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 84, 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "226972e1": { 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 84, 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:54.559 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:54.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.556+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 msgr2=0x7fbdcc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.556+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 0x7fbdcc0a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005fd0 tx=0x7fbdd40046c0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.556+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 msgr2=0x7fbde4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.556+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800ea80 tx=0x7fbdd800ef40 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 shutdown_connections 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fbdcc0a2cf0 0x7fbdcc0a51b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde4106f10 0x7fbde419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde4102790 0x7fbde4198920 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 --2- 172.21.15.86:0/1485093107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbde4101de0 0x7fbde41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 >> 172.21.15.86:0/1485093107 conn(0x7fbde40fd8c0 msgr2=0x7fbde40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 shutdown_connections 2024-09-05T07:03:54.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:54.557+0000 7fbde981a640 1 -- 172.21.15.86:0/1485093107 wait complete. 2024-09-05T07:03:54.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65822 6.32 MB/sec execute 432 sec latency 27651.344 ms 2024-09-05T07:03:55.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:55 smithi086 ceph-mon[30943]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-09-05T07:03:55.499 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '226972e1', total seconds elapsed: 33.5 2024-09-05T07:03:55.499 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T07:03:55.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:03:55.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:55 smithi096 ceph-mon[34468]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-09-05T07:03:55.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:55 smithi115 ceph-mon[34791]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-09-05T07:03:55.780 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65992 6.33 MB/sec execute 433 sec latency 267.255 ms 2024-09-05T07:03:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:56.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 -- 172.21.15.86:0/2778250677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 msgr2=0x7f40701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 --2- 172.21.15.86:0/2778250677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f40701029b0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4064009860 tx=0x7f406402a3d0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 -- 172.21.15.86:0/2778250677 shutdown_connections 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 --2- 172.21.15.86:0/2778250677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f407010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 --2- 172.21.15.86:0/2778250677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f40701029b0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 --2- 172.21.15.86:0/2778250677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 -- 172.21.15.86:0/2778250677 >> 172.21.15.86:0/2778250677 conn(0x7f40700fc7d0 msgr2=0x7f40700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 -- 172.21.15.86:0/2778250677 shutdown_connections 2024-09-05T07:03:56.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.487+0000 7f407666b640 1 -- 172.21.15.86:0/2778250677 wait complete. 2024-09-05T07:03:56.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 Processor -- start 2024-09-05T07:03:56.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 -- start start 2024-09-05T07:03:56.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:56.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:56.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f4070108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60656/0 (socket says 172.21.15.86:60656) 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f406ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f406ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38660/0 (socket says 172.21.15.86:38660) 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f406f7fe640 1 -- 172.21.15.86:0/4055118310 learned_addr learned my addr 172.21.15.86:0/4055118310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4070110e80 con 0x7f4070102570 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.488+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4070110d00 con 0x7f4070100e20 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4070111000 con 0x7f4070102ef0 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f4074c08640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f4070108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 -- 172.21.15.86:0/4055118310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 msgr2=0x7f4070108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f4070108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 -- 172.21.15.86:0/4055118310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 msgr2=0x7f4070107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40701a6980 con 0x7f4070102570 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f4074c08640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f4070108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406ffff640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406f7fe640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4064009980 tx=0x7f4064030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.489+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064040070 con 0x7f4070102570 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.490+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40640055f0 con 0x7f4070102570 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.490+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40701a6c10 con 0x7f4070102570 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.490+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640046c0 con 0x7f4070102570 2024-09-05T07:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.490+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40701a70f0 con 0x7f4070102570 2024-09-05T07:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.491+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4070101200 con 0x7f4070102570 2024-09-05T07:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.492+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4064043070 con 0x7f4070102570 2024-09-05T07:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.493+0000 7f406d7fa640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 0x7f40580a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.493+0000 7f406ffff640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 0x7f40580a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.493+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f40640f0280 con 0x7f4070102570 2024-09-05T07:03:56.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.494+0000 7f406ffff640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 0x7f40580a5300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4060005fd0 tx=0x7f4060009260 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:56.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.500+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40640bca60 con 0x7f4070102570 2024-09-05T07:03:56.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66122 6.33 MB/sec execute 434 sec latency 330.304 ms 2024-09-05T07:03:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.839+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40701a7380 con 0x7f4070102570 2024-09-05T07:03:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.840+0000 7f406d7fa640 1 -- 172.21.15.86:0/4055118310 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f40640bc360 con 0x7f4070102570 2024-09-05T07:03:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:03:56.841 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:03:56.841 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 msgr2=0x7f40580a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 0x7f40580a5300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4060005fd0 tx=0x7f4060009260 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 msgr2=0x7f4070108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4064009980 tx=0x7f4064030fc0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 shutdown_connections 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f40580a2e40 0x7f40580a5300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4070102ef0 0x7f4070108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4070102570 0x7f4070108020 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 --2- 172.21.15.86:0/4055118310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100e20 0x7f4070107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 >> 172.21.15.86:0/4055118310 conn(0x7f40700fc7d0 msgr2=0x7f40700fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:56.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 shutdown_connections 2024-09-05T07:03:56.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:56.843+0000 7f407666b640 1 -- 172.21.15.86:0/4055118310 wait complete. 2024-09-05T07:03:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:57 smithi086 ceph-mon[30943]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-09-05T07:03:57.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:57 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/4055118310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:57 smithi096 ceph-mon[34468]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-09-05T07:03:57.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:57 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/4055118310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:57 smithi115 ceph-mon[34791]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-09-05T07:03:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:57 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/4055118310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:03:57.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66264 6.32 MB/sec execute 435 sec latency 514.491 ms 2024-09-05T07:03:57.973 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T07:03:57.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T07:03:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:03:58.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66436 6.33 MB/sec execute 436 sec latency 144.146 ms 2024-09-05T07:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.899+0000 7f8eedc78640 1 -- 172.21.15.86:0/454035316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 msgr2=0x7f8ee81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.899+0000 7f8eedc78640 1 --2- 172.21.15.86:0/454035316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee81089f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed802a410 comp rx=0 tx=0).stop 2024-09-05T07:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.899+0000 7f8eedc78640 1 -- 172.21.15.86:0/454035316 shutdown_connections 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.899+0000 7f8eedc78640 1 --2- 172.21.15.86:0/454035316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8109120 0x7f8ee8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.899+0000 7f8eedc78640 1 --2- 172.21.15.86:0/454035316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee81089f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 --2- 172.21.15.86:0/454035316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8103ff0 0x7f8ee81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 -- 172.21.15.86:0/454035316 >> 172.21.15.86:0/454035316 conn(0x7f8ee80ffab0 msgr2=0x7f8ee8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 -- 172.21.15.86:0/454035316 shutdown_connections 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 -- 172.21.15.86:0/454035316 wait complete. 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 Processor -- start 2024-09-05T07:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.900+0000 7f8eedc78640 1 -- start start 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8103ff0 0x7f8ee819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 0x7f8ee819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113fd0 con 0x7f8ee8109120 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113e50 con 0x7f8ee81049a0 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8eedc78640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8114150 con 0x7f8ee8103ff0 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38678/0 (socket says 172.21.15.86:38678) 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/1407953023 learned_addr learned my addr 172.21.15.86:0/1407953023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 0x7f8ee819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8103ff0 0x7f8ee819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/1407953023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8103ff0 msgr2=0x7f8ee819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8103ff0 0x7f8ee819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.901+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 msgr2=0x7f8ee819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 0x7f8ee819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/1407953023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee819f500 con 0x7f8ee81049a0 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 0x7f8ee819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005940 tx=0x7f8ed80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee4ff9640 1 -- 172.21.15.86:0/1407953023 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed803f030 con 0x7f8ee81049a0 2024-09-05T07:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee819f790 con 0x7f8ee81049a0 2024-09-05T07:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee4ff9640 1 -- 172.21.15.86:0/1407953023 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed80044c0 con 0x7f8ee81049a0 2024-09-05T07:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8ee4ff9640 1 -- 172.21.15.86:0/1407953023 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8035070 con 0x7f8ee81049a0 2024-09-05T07:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.902+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a70d0 con 0x7f8ee81049a0 2024-09-05T07:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.905+0000 7f8ee4ff9640 1 -- 172.21.15.86:0/1407953023 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ed8042030 con 0x7f8ee81049a0 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 Processor -- start 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 -- start start 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee4ff9640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 0x7f8ed40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 0x55fc01b1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b25940 0x55fc01b27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc01a802a0 con 0x55fc01b1a760 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc01a27b10 con 0x55fc01b1b080 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ef0051740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fc01a7c260 con 0x55fc01b25940 2024-09-05T07:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee4ff9640 1 -- 172.21.15.86:0/1407953023 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f8ed80efb60 con 0x7f8ee81049a0 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 0x55fc01b1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38686/0 (socket says 172.21.15.86:38686) 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/3233238892 learned_addr learned my addr 172.21.15.86:0/3233238892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b25940 0x55fc01b27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.906+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 0x7f8ed40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/3233238892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b25940 msgr2=0x55fc01b27d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b25940 0x55fc01b27d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/3233238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 msgr2=0x55fc01b1ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 0x55fc01b1ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 -- 172.21.15.86:0/3233238892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc017f1b90 con 0x55fc01b1b080 2024-09-05T07:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 0x55fc01b1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8047ff0 tx=0x7f8ed80bb7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37fc065ba61ba752 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8eddffb640 1 -- 172.21.15.86:0/3233238892 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8049040 con 0x55fc01b1b080 2024-09-05T07:03:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8eddffb640 1 -- 172.21.15.86:0/3233238892 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed80bbc90 con 0x55fc01b1b080 2024-09-05T07:03:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.907+0000 7f8eddffb640 1 -- 172.21.15.86:0/3233238892 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b8300 con 0x55fc01b1b080 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 -- 172.21.15.86:0/3233238892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 msgr2=0x55fc01b25400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 0x7f8ed40a5230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005c40 tx=0x7f8ee0005bd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8047ff0 tx=0x7f8ed80bb7d0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 -- 172.21.15.86:0/3233238892 shutdown_connections 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b25940 0x55fc01b27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1b080 0x55fc01b25400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3233238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1a760 0x55fc01b1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 -- 172.21.15.86:0/3233238892 >> 172.21.15.86:0/3233238892 conn(0x55fc01b16cf0 msgr2=0x55fc01b170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 -- 172.21.15.86:0/3233238892 shutdown_connections 2024-09-05T07:03:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.908+0000 7f8ef0051740 1 -- 172.21.15.86:0/3233238892 wait complete. 2024-09-05T07:03:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.909+0000 7f8ef0051740 1 -- start start 2024-09-05T07:03:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.909+0000 7f8ef0051740 1 Processor -- start 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.909+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b22820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.909+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.909+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b22820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b22820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:38052/0 (socket says 172.21.15.86:38052) 2024-09-05T07:03:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60692/0 (socket says 172.21.15.86:60692) 2024-09-05T07:03:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee77fe640 1 -- 172.21.15.86:0/1987886174 learned_addr learned my addr 172.21.15.86:0/1987886174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc01b282d0 con 0x55fc01b1b080 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc01a086e0 con 0x55fc01b25940 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fc01b28450 con 0x55fc01b1a760 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 -- 172.21.15.86:0/1987886174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 msgr2=0x55fc01b22820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b22820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 -- 172.21.15.86:0/1987886174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 msgr2=0x55fc01b22d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc01b31730 con 0x55fc01b25940 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ef0051740 10 client.0 _ll_register_callbacks cb 0x55fc01a2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed80bbc90 con 0x55fc01b25940 2024-09-05T07:03:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b232a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55fc01b18070 tx=0x7f8ee002c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7724179368810d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.910+0000 7f8ec6ffd640 10 client.29156 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1987886174 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003e060 con 0x55fc01b25940 2024-09-05T07:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1987886174 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f8ee002e620 con 0x55fc01b25940 2024-09-05T07:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1987886174 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8ee002e820 con 0x55fc01b25940 2024-09-05T07:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 10 client.29156 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55fc01b33f20 con 0x55fc01b25940 2024-09-05T07:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1987886174 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f8ee002e9c0 con 0x55fc01b25940 2024-09-05T07:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 10 client.29156 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 10 client.29156 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 20 client.29156 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01b357a0 0x55fc01b37c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 4 client.29156 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:03:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.911+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc01a8d9d0 con 0x55fc01b357a0 2024-09-05T07:03:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.912+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01b357a0 0x55fc01b37c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.912+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01b357a0 0x55fc01b37c60 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388bb73f57293428 server_cookie=97cb0a42c6f3a79d in_seq=0 out_seq=0 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.912+0000 7f8ec6ffd640 10 client.29156 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1987886174 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc01a8d9d0 con 0x55fc01b357a0 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ec6ffd640 10 client.29156 handle_command_reply: tid=0 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 client.29156 shutdown 2024-09-05T07:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 msgr2=0x55fc01b232a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b232a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55fc01b18070 tx=0x7f8ee002c000 comp rx=0 tx=0).stop 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 shutdown_connections 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01b357a0 0x55fc01b37c60 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b232a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b22d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ec6ffd640 0 client.29156 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1987886174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b22820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 >> 172.21.15.86:0/1987886174 conn(0x55fc01b16cf0 msgr2=0x55fc01b1ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 shutdown_connections 2024-09-05T07:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 1 -- 172.21.15.86:0/1987886174 wait complete. 2024-09-05T07:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.913+0000 7f8ef0051740 20 client.29156 trim_cache size 0 max 16384 2024-09-05T07:03:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.937+0000 7f8ef0051740 1 Processor -- start 2024-09-05T07:03:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.937+0000 7f8ef0051740 1 -- start start 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.937+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 0x55fc01b246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.937+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1b080 0x55fc01b1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.937+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ef0051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81acfa0 con 0x55fc01b25940 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ef0051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81aeae0 con 0x55fc01b1a760 2024-09-05T07:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ef0051740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81ad670 con 0x55fc01b1b080 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 0x55fc01b246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 0x55fc01b246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38698/0 (socket says 172.21.15.86:38698) 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60698/0 (socket says 172.21.15.86:60698) 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee77fe640 1 -- 172.21.15.86:0/3180662181 learned_addr learned my addr 172.21.15.86:0/3180662181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1b080 0x55fc01b1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 -- 172.21.15.86:0/3180662181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1b080 msgr2=0x55fc01b1d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1b080 0x55fc01b1d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 -- 172.21.15.86:0/3180662181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 msgr2=0x55fc01b246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 0x55fc01b246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 -- 172.21.15.86:0/3180662181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc01b12260 con 0x55fc01b25940 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00099e0 tx=0x7f8ee0033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31c8c2b622f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.938+0000 7f8edd7fa640 1 -- 172.21.15.86:0/3180662181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0050070 con 0x55fc01b25940 2024-09-05T07:03:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8edd7fa640 1 -- 172.21.15.86:0/3180662181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed80bbc90 con 0x55fc01b25940 2024-09-05T07:03:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8edd7fa640 1 -- 172.21.15.86:0/3180662181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003ae50 con 0x55fc01b25940 2024-09-05T07:03:58.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 -- 172.21.15.86:0/3180662181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 msgr2=0x55fc01b3d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00099e0 tx=0x7f8ee0033040 comp rx=0 tx=0).stop 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 -- 172.21.15.86:0/3180662181 shutdown_connections 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b25940 0x55fc01b3d530 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1b080 0x55fc01b1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 --2- 172.21.15.86:0/3180662181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b1a760 0x55fc01b246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.939+0000 7f8ef0051740 1 -- 172.21.15.86:0/3180662181 >> 172.21.15.86:0/3180662181 conn(0x55fc01b16cf0 msgr2=0x55fc01b15e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 -- 172.21.15.86:0/3180662181 shutdown_connections 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 -- 172.21.15.86:0/3180662181 wait complete. 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 -- start start 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 Processor -- start 2024-09-05T07:03:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b2d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.940+0000 7f8ef0051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b29850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b2d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ef0051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81ae410 con 0x55fc01b1b080 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ef0051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81add40 con 0x55fc01b25940 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ef0051740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81aafa0 con 0x55fc01b1a760 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ef0051740 10 client.0 _ll_register_callbacks cb 0x7f8ee0031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:03:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:38082/0 (socket says 172.21.15.86:38082) 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 -- 172.21.15.86:0/1576214900 learned_addr learned my addr 172.21.15.86:0/1576214900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee7fff640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b29850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 -- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 msgr2=0x55fc01b29850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b29850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 -- 172.21.15.86:0/1576214900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 msgr2=0x55fc01b2d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:03:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b2d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 -- 172.21.15.86:0/1576214900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc01b2a080 con 0x55fc01b1a760 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ee77fe640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55fc01b384d0 tx=0x7f8ed0004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d89a9e31bc88e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ec6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0016030 con 0x55fc01b1a760 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed80bbc90 con 0x55fc01b1a760 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.941+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9605+0+0 (secure 0 0 0) 0x7f8ed0010630 con 0x55fc01b1a760 2024-09-05T07:03:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8ed0010830 con 0x55fc01b1a760 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 10 client.29197 fetch_fsmap learned FSMap version 46 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fc01ba1010 con 0x55fc01b1a760 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed80bbc90 con 0x55fc01b1a760 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f8ed0027b20 con 0x55fc01b1a760 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 10 client.29197 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:03:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 10 client.29197 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:03:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 20 client.29197 populate_metadata read hostname 'smithi086' 2024-09-05T07:03:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01ba17a0 0x55fc01ba3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:03:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 4 client.29197 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T07:03:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55fc01a87d20 con 0x55fc01ba17a0 2024-09-05T07:03:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.942+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01ba17a0 0x55fc01ba3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:03:58.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.943+0000 7f8ee6ffd640 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01ba17a0 0x55fc01ba3bb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1aaa1b1ac2b2e9 server_cookie=fbd0f013b5437638 in_seq=0 out_seq=0 2024-09-05T07:03:58.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:58.943+0000 7f8ec6ffd640 10 client.29197 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.195+0000 7f8ec6ffd640 1 -- 172.21.15.86:0/1576214900 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fc01a87d20 con 0x55fc01ba17a0 2024-09-05T07:03:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.195+0000 7f8ec6ffd640 10 client.29197 handle_command_reply: tid=0 2024-09-05T07:03:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.195+0000 7f8ef0051740 1 client.29197 shutdown 2024-09-05T07:03:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 msgr2=0x55fc01b2d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55fc01b384d0 tx=0x7f8ed0004600 comp rx=0 tx=0).stop 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 shutdown_connections 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55fc01ba17a0 0x55fc01ba3bb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ec6ffd640 0 client.29197 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc01b25940 0x55fc01b29850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc01b1b080 0x55fc01b2d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 --2- 172.21.15.86:0/1576214900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc01b1a760 0x55fc01b2d480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 >> 172.21.15.86:0/1576214900 conn(0x55fc01b16cf0 msgr2=0x55fc01b15650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 shutdown_connections 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 1 -- 172.21.15.86:0/1576214900 wait complete. 2024-09-05T07:03:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.196+0000 7f8ef0051740 20 client.29197 trim_cache size 0 max 16384 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 46, 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 86, 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "e131ad72": { 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 86, 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:59.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:59.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 msgr2=0x7f8ed40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 0x7f8ed40a5230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005c40 tx=0x7f8ee0005bd0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 msgr2=0x7f8ee819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:03:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005940 tx=0x7f8ed80040c0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 shutdown_connections 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f8ed40a2d70 0x7f8ed40a5230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee8109120 0x7f8ee819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ee81049a0 0x7f8ee819aa90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 --2- 172.21.15.86:0/1407953023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ee8103ff0 0x7f8ee819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 >> 172.21.15.86:0/1407953023 conn(0x7f8ee80ffab0 msgr2=0x7f8ee8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 shutdown_connections 2024-09-05T07:03:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:03:59.197+0000 7f8eedc78640 1 -- 172.21.15.86:0/1407953023 wait complete. 2024-09-05T07:03:59.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:03:59 smithi086 ceph-mon[30943]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 967 KiB/s wr, 55 op/s 2024-09-05T07:03:59.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:03:59 smithi096 ceph-mon[34468]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 967 KiB/s wr, 55 op/s 2024-09-05T07:03:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:03:59 smithi115 ceph-mon[34791]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 967 KiB/s wr, 55 op/s 2024-09-05T07:03:59.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.32 MB/sec execute 437 sec latency 681.588 ms 2024-09-05T07:03:59.874 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e131ad72', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-05T07:04:00.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:00 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:04:00.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:00 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:04:00.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:00 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:04:00.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.31 MB/sec execute 438 sec latency 1681.635 ms 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 936 KiB/s wr, 49 op/s 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:04:01.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:04:01.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 936 KiB/s wr, 49 op/s 2024-09-05T07:04:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:04:01.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.29 MB/sec execute 439 sec latency 2681.717 ms 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 936 KiB/s wr, 49 op/s 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:04:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:04:01.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:04:02.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.28 MB/sec execute 440 sec latency 3681.817 ms 2024-09-05T07:04:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:03 smithi115 ceph-mon[34791]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-05T07:04:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:03 smithi086 ceph-mon[30943]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-05T07:04:03.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.26 MB/sec execute 441 sec latency 4681.908 ms 2024-09-05T07:04:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:03 smithi096 ceph-mon[34468]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-05T07:04:04.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.25 MB/sec execute 442 sec latency 5682.006 ms 2024-09-05T07:04:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:05 smithi115 ceph-mon[34791]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:05 smithi086 ceph-mon[30943]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:05.781 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.23 MB/sec execute 443 sec latency 6682.125 ms 2024-09-05T07:04:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:05 smithi096 ceph-mon[34468]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:06.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.22 MB/sec execute 444 sec latency 7682.180 ms 2024-09-05T07:04:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:07 smithi115 ceph-mon[34791]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:07 smithi115 ceph-mon[34791]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T07:04:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:07 smithi086 ceph-mon[30943]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:07 smithi086 ceph-mon[30943]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T07:04:07.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.21 MB/sec execute 445 sec latency 8682.306 ms 2024-09-05T07:04:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:07 smithi096 ceph-mon[34468]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-05T07:04:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:07 smithi096 ceph-mon[34468]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T07:04:08.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.19 MB/sec execute 446 sec latency 9682.432 ms 2024-09-05T07:04:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:09 smithi115 ceph-mon[34791]: pgmap v722: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:09 smithi086 ceph-mon[30943]: pgmap v722: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:09.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.18 MB/sec execute 447 sec latency 10682.563 ms 2024-09-05T07:04:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:09 smithi096 ceph-mon[34468]: pgmap v722: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:10.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.16 MB/sec execute 448 sec latency 11682.653 ms 2024-09-05T07:04:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:11 smithi115 ceph-mon[34791]: pgmap v723: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:11 smithi086 ceph-mon[30943]: pgmap v723: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:11.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.15 MB/sec execute 449 sec latency 12682.748 ms 2024-09-05T07:04:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:11 smithi096 ceph-mon[34468]: pgmap v723: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-05T07:04:12.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.14 MB/sec execute 450 sec latency 13682.858 ms 2024-09-05T07:04:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:13 smithi115 ceph-mon[34791]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 321 KiB/s wr, 40 op/s 2024-09-05T07:04:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:13 smithi086 ceph-mon[30943]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 321 KiB/s wr, 40 op/s 2024-09-05T07:04:13.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.12 MB/sec execute 451 sec latency 14682.945 ms 2024-09-05T07:04:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:13 smithi096 ceph-mon[34468]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 321 KiB/s wr, 40 op/s 2024-09-05T07:04:14.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.11 MB/sec execute 452 sec latency 15683.048 ms 2024-09-05T07:04:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:15 smithi115 ceph-mon[34791]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:15 smithi086 ceph-mon[30943]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:15.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.10 MB/sec execute 453 sec latency 16683.158 ms 2024-09-05T07:04:15.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:15 smithi096 ceph-mon[34468]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:16.782 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.08 MB/sec execute 454 sec latency 17683.251 ms 2024-09-05T07:04:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:17 smithi115 ceph-mon[34791]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:17 smithi115 ceph-mon[34791]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T07:04:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:17 smithi086 ceph-mon[30943]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:17 smithi086 ceph-mon[30943]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T07:04:17.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.07 MB/sec execute 455 sec latency 18683.381 ms 2024-09-05T07:04:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:17 smithi096 ceph-mon[34468]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 165 KiB/s wr, 16 op/s 2024-09-05T07:04:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:17 smithi096 ceph-mon[34468]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T07:04:18.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.06 MB/sec execute 456 sec latency 19683.480 ms 2024-09-05T07:04:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:19 smithi115 ceph-mon[34791]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:19 smithi086 ceph-mon[30943]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:19.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.04 MB/sec execute 457 sec latency 20683.572 ms 2024-09-05T07:04:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:19 smithi096 ceph-mon[34468]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:20.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.03 MB/sec execute 458 sec latency 21683.632 ms 2024-09-05T07:04:21.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:04:21.346 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:21 smithi115 ceph-mon[34791]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:21 smithi086 ceph-mon[30943]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:21.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.02 MB/sec execute 459 sec latency 22683.730 ms 2024-09-05T07:04:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:21 smithi096 ceph-mon[34468]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-05T07:04:22.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 -- 172.21.15.86:0/128781898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 msgr2=0x7fc1f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/128781898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f8106e60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009930 tx=0x7fc1e802a420 comp rx=0 tx=0).stop 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 -- 172.21.15.86:0/128781898 shutdown_connections 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/128781898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 0x7fc1f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/128781898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 0x7fc1f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/128781898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f8106e60 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 -- 172.21.15.86:0/128781898 >> 172.21.15.86:0/128781898 conn(0x7fc1f80fc7a0 msgr2=0x7fc1f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 -- 172.21.15.86:0/128781898 shutdown_connections 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 -- 172.21.15.86:0/128781898 wait complete. 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.285+0000 7fc1fe78a640 1 Processor -- start 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 -- start start 2024-09-05T07:04:22.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 0x7fc1f8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 0x7fc1f81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8110cf0 con 0x7fc1f8106a80 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8110b70 con 0x7fc1f8101420 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fe78a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f8110e70 con 0x7fc1f8100a80 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42844/0 (socket says 172.21.15.86:42844) 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 -- 172.21.15.86:0/2094064322 learned_addr learned my addr 172.21.15.86:0/2094064322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1effff640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 0x7fc1f81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 0x7fc1f8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 -- 172.21.15.86:0/2094064322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 msgr2=0x7fc1f8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 0x7fc1f8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 -- 172.21.15.86:0/2094064322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 msgr2=0x7fc1f81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 0x7fc1f81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 -- 172.21.15.86:0/2094064322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81a4740 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1fcd27640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000eaa0 tx=0x7fc1f000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.286+0000 7fc1f5ffb640 1 -- 172.21.15.86:0/2094064322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004d20 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.287+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a4910 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.287+0000 7fc1f5ffb640 1 -- 172.21.15.86:0/2094064322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f0005020 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.287+0000 7fc1f5ffb640 1 -- 172.21.15.86:0/2094064322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00116b0 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.287+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a4e20 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc1f5ffb640 1 -- 172.21.15.86:0/2094064322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1f00051d0 con 0x7fc1f8106a80 2024-09-05T07:04:22.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 Processor -- start 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 -- start start 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83bfb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c06140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b83b612e0 con 0x563b83c06680 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b83b08ce0 con 0x563b83bfbdc0 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.289+0000 7fc200b63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563b83b5d2a0 con 0x563b83bfb4a0 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42860/0 (socket says 172.21.15.86:42860) 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 -- 172.21.15.86:0/2533562594 learned_addr learned my addr 172.21.15.86:0/2533562594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:22.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1f5ffb640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 0x7fc1e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83bfb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c06140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1fcd27640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 0x7fc1e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 -- 172.21.15.86:0/2533562594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 msgr2=0x563b83bfb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83bfb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 -- 172.21.15.86:0/2533562594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 msgr2=0x563b83c06140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c06140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 -- 172.21.15.86:0/2533562594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b83aed440 con 0x563b83c06680 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c06140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1f5ffb640 1 -- 172.21.15.86:0/2094064322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fc1f00cfa70 con 0x7fc1f8106a80 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1effff640 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00135a0 tx=0x7fc1e000a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd63934cb32cd14f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1ee7fc640 1 -- 172.21.15.86:0/2533562594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0019030 con 0x563b83c06680 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1fcd27640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 0x7fc1e40a51b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc1f009bae0 tx=0x7fc1f0066b70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1ee7fc640 1 -- 172.21.15.86:0/2533562594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e00110d0 con 0x563b83c06680 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.290+0000 7fc1ee7fc640 1 -- 172.21.15.86:0/2533562594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000f810 con 0x563b83c06680 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- 172.21.15.86:0/2533562594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 msgr2=0x563b83c08a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00135a0 tx=0x7fc1e000a830 comp rx=0 tx=0).stop 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- 172.21.15.86:0/2533562594 shutdown_connections 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c08a70 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c06140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 --2- 172.21.15.86:0/2533562594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83bfb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- 172.21.15.86:0/2533562594 >> 172.21.15.86:0/2533562594 conn(0x563b83bf7a30 msgr2=0x563b83bf7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- 172.21.15.86:0/2533562594 shutdown_connections 2024-09-05T07:04:22.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- 172.21.15.86:0/2533562594 wait complete. 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.291+0000 7fc200b63740 1 -- start start 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 Processor -- start 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfbdc0 0x563b83c03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83c06680 0x563b83c03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b83c09250 con 0x563b83bfb4a0 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b83b1c050 con 0x563b83c06680 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563b83c093d0 con 0x563b83bfbdc0 2024-09-05T07:04:22.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc200b63740 10 client.0 _ll_register_callbacks cb 0x563b83b5c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42864/0 (socket says 172.21.15.86:42864) 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 -- 172.21.15.86:0/295051534 learned_addr learned my addr 172.21.15.86:0/295051534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfbdc0 0x563b83c03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 -- 172.21.15.86:0/295051534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfbdc0 msgr2=0x563b83c03200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfbdc0 0x563b83c03200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 -- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83c06680 msgr2=0x563b83c03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83c06680 0x563b83c03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 -- 172.21.15.86:0/295051534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b838d23c0 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1effff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011330 tx=0x7fc1e0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cae14053eb8250 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.292+0000 7fc1d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0019040 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fc1e0005240 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e00110d0 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc1e0005440 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 10 client.20493 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563b83c14b10 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e00110d0 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fc1e000f590 con 0x563b83bfb4a0 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 10 client.20493 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 10 client.20493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:22.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 20 client.20493 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 4 client.20493 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b83b6e850 con 0x563b83c15ec0 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.293+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.294+0000 7fc1d77fe640 10 client.20493 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.294+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c182b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f887fb826d064c4a server_cookie=45ed347298690eec in_seq=0 out_seq=0 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc1d77fe640 1 -- 172.21.15.86:0/295051534 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b83b6e850 con 0x563b83c15ec0 2024-09-05T07:04:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc1d77fe640 10 client.20493 handle_command_reply: tid=0 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 client.20493 shutdown 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 msgr2=0x563b83c05f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011330 tx=0x7fc1e0011f40 comp rx=0 tx=0).stop 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 shutdown_connections 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c182b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83c06680 0x563b83c03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc1d77fe640 0 client.20493 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfbdc0 0x563b83c03200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 --2- 172.21.15.86:0/295051534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c05f50 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 >> 172.21.15.86:0/295051534 conn(0x563b83bf7a30 msgr2=0x563b83bfd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 shutdown_connections 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 1 -- 172.21.15.86:0/295051534 wait complete. 2024-09-05T07:04:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.295+0000 7fc200b63740 20 client.20493 trim_cache size 0 max 16384 2024-09-05T07:04:22.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 Processor -- start 2024-09-05T07:04:22.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 -- start start 2024-09-05T07:04:22.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83c06680 0x563b83c0ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81af170 con 0x563b83bfb4a0 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b0cb0 con 0x563b83bfbdc0 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc200b63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81af840 con 0x563b83c06680 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42878/0 (socket says 172.21.15.86:42878) 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc1effff640 1 -- 172.21.15.86:0/2315264335 learned_addr learned my addr 172.21.15.86:0/2315264335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.312+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 -- 172.21.15.86:0/2315264335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83c06680 msgr2=0x563b83c0ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83c06680 0x563b83c0ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 -- 172.21.15.86:0/2315264335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 msgr2=0x563b83c0ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c0ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 -- 172.21.15.86:0/2315264335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b83bf7860 con 0x563b83bfb4a0 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1effff640 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011b60 tx=0x7fc1e0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a803e38c8ea8ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1edffb640 1 -- 172.21.15.86:0/2315264335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0019040 con 0x563b83bfb4a0 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1edffb640 1 -- 172.21.15.86:0/2315264335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e0023030 con 0x563b83bfb4a0 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.313+0000 7fc1edffb640 1 -- 172.21.15.86:0/2315264335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0028420 con 0x563b83bfb4a0 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- 172.21.15.86:0/2315264335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 msgr2=0x563b83c124b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011b60 tx=0x7fc1e0005e40 comp rx=0 tx=0).stop 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- 172.21.15.86:0/2315264335 shutdown_connections 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83c06680 0x563b83c0ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 --2- 172.21.15.86:0/2315264335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83bfb4a0 0x563b83c124b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- 172.21.15.86:0/2315264335 >> 172.21.15.86:0/2315264335 conn(0x563b83bf7a30 msgr2=0x563b83bfe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- 172.21.15.86:0/2315264335 shutdown_connections 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- 172.21.15.86:0/2315264335 wait complete. 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 -- start start 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.314+0000 7fc200b63740 1 Processor -- start 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83c7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c7c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81b05e0 con 0x563b83c06680 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81aff10 con 0x563b83bfbdc0 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ad170 con 0x563b83bfb4a0 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 10 client.0 _ll_register_callbacks cb 0x7fc1e0013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42888/0 (socket says 172.21.15.86:42888) 2024-09-05T07:04:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 -- 172.21.15.86:0/1127658893 learned_addr learned my addr 172.21.15.86:0/1127658893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83c7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c7c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 -- 172.21.15.86:0/1127658893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 msgr2=0x563b83c7bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83c7bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 -- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 msgr2=0x563b83c7c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c7c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 -- 172.21.15.86:0/1127658893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b83bf86c0 con 0x563b83c06680 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c7c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1effff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x563b83c1cbf0 tx=0x7fc1e0021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c783920bffcc565 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0046060 con 0x563b83c06680 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fc1e000f420 con 0x563b83c06680 2024-09-05T07:04:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.315+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e0023030 con 0x563b83c06680 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc1e0039040 con 0x563b83c06680 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 10 client.20499 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563b83c84740 con 0x563b83c06680 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e0023030 con 0x563b83c06680 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fc1e003f040 con 0x563b83c06680 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 10 client.20499 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 10 client.20499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 20 client.20499 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 4 client.20499 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b83b68d60 con 0x563b83c15ec0 2024-09-05T07:04:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.316+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc1f7fff640 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c86b70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38236c9ddad2fc8e server_cookie=f481978dfe0745fc in_seq=0 out_seq=0 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc1d77fe640 10 client.20499 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc1d77fe640 1 -- 172.21.15.86:0/1127658893 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b83b68d60 con 0x563b83c15ec0 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc1d77fe640 10 client.20499 handle_command_reply: tid=0 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 client.20499 shutdown 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 msgr2=0x563b83c1d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x563b83c1cbf0 tx=0x7fc1e0021040 comp rx=0 tx=0).stop 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 shutdown_connections 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563b83c15ec0 0x563b83c86b70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b83c06680 0x563b83c1d360 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b83bfbdc0 0x563b83c7c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 --2- 172.21.15.86:0/1127658893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563b83bfb4a0 0x563b83c7bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 >> 172.21.15.86:0/1127658893 conn(0x563b83bf7a30 msgr2=0x563b83bf6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc1d77fe640 0 client.20499 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 shutdown_connections 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 1 -- 172.21.15.86:0/1127658893 wait complete. 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.317+0000 7fc200b63740 20 client.20499 trim_cache size 0 max 16384 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:04:22.326 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:04:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 msgr2=0x7fc1e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 0x7fc1e40a51b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc1f009bae0 tx=0x7fc1f0066b70 comp rx=0 tx=0).stop 2024-09-05T07:04:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 msgr2=0x7fc1f81a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000eaa0 tx=0x7fc1f000eea0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 shutdown_connections 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1e40a2cf0 0x7fc1e40a51b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f8106a80 0x7fc1f81a4200 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f8101420 0x7fc1f81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 --2- 172.21.15.86:0/2094064322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1f8100a80 0x7fc1f8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 >> 172.21.15.86:0/2094064322 conn(0x7fc1f80fc7a0 msgr2=0x7fc1f80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 shutdown_connections 2024-09-05T07:04:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:22.318+0000 7fc1fe78a640 1 -- 172.21.15.86:0/2094064322 wait complete. 2024-09-05T07:04:22.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 6.00 MB/sec execute 460 sec latency 23683.767 ms 2024-09-05T07:04:23.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:04:23.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:04:23.362 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:23 smithi115 ceph-mon[34791]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-05T07:04:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:23 smithi086 ceph-mon[30943]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-05T07:04:23.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.99 MB/sec execute 461 sec latency 24683.801 ms 2024-09-05T07:04:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:23 smithi096 ceph-mon[34468]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-09-05T07:04:23.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 -- 172.21.15.86:0/1250649666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc1036c0 msgr2=0x7f47cc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:23.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/1250649666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc1036c0 0x7f47cc10bbb0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009900 tx=0x7f47c002a480 comp rx=0 tx=0).stop 2024-09-05T07:04:23.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 -- 172.21.15.86:0/1250649666 shutdown_connections 2024-09-05T07:04:23.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/1250649666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:23.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/1250649666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc1036c0 0x7f47cc10bbb0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/1250649666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc102da0 0x7f47cc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 -- 172.21.15.86:0/1250649666 >> 172.21.15.86:0/1250649666 conn(0x7f47cc0fc7a0 msgr2=0x7f47cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 -- 172.21.15.86:0/1250649666 shutdown_connections 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 -- 172.21.15.86:0/1250649666 wait complete. 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.859+0000 7f47d1cc2640 1 Processor -- start 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 -- start start 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 0x7f47cc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc111e50 con 0x7f47cc102da0 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc111cd0 con 0x7f47cc10c180 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47d1cc2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc111fd0 con 0x7f47cc1036c0 2024-09-05T07:04:23.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43832/0 (socket says 172.21.15.86:43832) 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 -- 172.21.15.86:0/2786164109 learned_addr learned my addr 172.21.15.86:0/2786164109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47caffd640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 0x7f47cc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cbfff640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 -- 172.21.15.86:0/2786164109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 msgr2=0x7f47cc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 0x7f47cc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 -- 172.21.15.86:0/2786164109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 msgr2=0x7f47cc19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cb7fe640 1 -- 172.21.15.86:0/2786164109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19d2e0 con 0x7f47cc102da0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47cbfff640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.860+0000 7f47caffd640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 0x7f47cc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47cb7fe640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c920 tx=0x7f47bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc007c70 con 0x7f47cc102da0 2024-09-05T07:04:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19d5d0 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47bc007e10 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc018820 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.861+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1a4eb0 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.862+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc103180 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.863+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47bc0189c0 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.864+0000 7f47c8ff9640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 0x7f47b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.864+0000 7f47caffd640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 0x7f47b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.864+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f47bc0cfd40 con 0x7f47cc102da0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.865+0000 7f47caffd640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 0x7f47b40a51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f47c0031440 tx=0x7f47c00095f0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:23.868+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47bc014030 con 0x7f47cc102da0 2024-09-05T07:04:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.069+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47cc199690 con 0x7f47cc102da0 2024-09-05T07:04:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.070+0000 7f47c8ff9640 1 -- 172.21.15.86:0/2786164109 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f47bc09c5b0 con 0x7f47cc102da0 2024-09-05T07:04:24.071 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:04:24.071 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:04:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:04:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 msgr2=0x7f47b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 0x7f47b40a51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f47c0031440 tx=0x7f47c00095f0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 msgr2=0x7f47cc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c920 tx=0x7f47bc00cde0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 shutdown_connections 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f47b40a2cf0 0x7f47b40a51b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47cc10c180 0x7f47cc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc1036c0 0x7f47cc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 --2- 172.21.15.86:0/2786164109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47cc102da0 0x7f47cc198330 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 >> 172.21.15.86:0/2786164109 conn(0x7f47cc0fc7a0 msgr2=0x7f47cc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 shutdown_connections 2024-09-05T07:04:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:24.072+0000 7f47d1cc2640 1 -- 172.21.15.86:0/2786164109 wait complete. 2024-09-05T07:04:24.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:24 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2786164109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:24.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:24 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2786164109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:24.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:04:24.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.98 MB/sec execute 462 sec latency 25683.830 ms 2024-09-05T07:04:24.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:24 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2786164109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:25.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 -- 172.21.15.86:0/3382343105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9244112640 msgr2=0x7f9244112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 --2- 172.21.15.86:0/3382343105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9244112640 0x7f9244112a20 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9240009900 tx=0x7f924002a410 comp rx=0 tx=0).stop 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 -- 172.21.15.86:0/3382343105 shutdown_connections 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 --2- 172.21.15.86:0/3382343105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410cf30 0x7f9244119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 --2- 172.21.15.86:0/3382343105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f924410c590 0x7f924410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 --2- 172.21.15.86:0/3382343105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9244112640 0x7f9244112a20 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 -- 172.21.15.86:0/3382343105 >> 172.21.15.86:0/3382343105 conn(0x7f924406cdc0 msgr2=0x7f924406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 -- 172.21.15.86:0/3382343105 shutdown_connections 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.526+0000 7f924bc62640 1 -- 172.21.15.86:0/3382343105 wait complete. 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 Processor -- start 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 -- start start 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410c590 0x7f92441a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 0x7f92441a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f924411c850 con 0x7f924410cf30 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f924411c6d0 con 0x7f924410c590 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.527+0000 7f924bc62640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f924411c9d0 con 0x7f9244112640 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43856/0 (socket says 172.21.15.86:43856) 2024-09-05T07:04:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 -- 172.21.15.86:0/726787595 learned_addr learned my addr 172.21.15.86:0/726787595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92499fe640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410c590 0x7f92441a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f924a1ff640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 0x7f92441a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 -- 172.21.15.86:0/726787595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 msgr2=0x7f92441a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 0x7f92441a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 -- 172.21.15.86:0/726787595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410c590 msgr2=0x7f92441a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410c590 0x7f92441a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 -- 172.21.15.86:0/726787595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441a7d00 con 0x7f924410cf30 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f924a1ff640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 0x7f92441a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f92491fd640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f923400b4a0 tx=0x7f923400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f923affd640 1 -- 172.21.15.86:0/726787595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234014070 con 0x7f924410cf30 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92441a7ff0 con 0x7f924410cf30 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f923affd640 1 -- 172.21.15.86:0/726787595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9234010ac0 con 0x7f924410cf30 2024-09-05T07:04:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f923affd640 1 -- 172.21.15.86:0/726787595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234017c30 con 0x7f924410cf30 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.528+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441af8d0 con 0x7f924410cf30 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 Processor -- start 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f923affd640 1 -- 172.21.15.86:0/726787595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9234017070 con 0x7f924410cf30 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 -- start start 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 0x55f11fe6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6c5d0 0x55f11fe6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f11fdd22f0 con 0x55f11fe77370 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f11fd79cf0 con 0x55f11fe6db40 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f924e03b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f11fdce2b0 con 0x55f11fe6c5d0 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43858/0 (socket says 172.21.15.86:43858) 2024-09-05T07:04:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.531+0000 7f92491fd640 1 -- 172.21.15.86:0/3735096173 learned_addr learned my addr 172.21.15.86:0/3735096173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92499fe640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6c5d0 0x55f11fe6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f924a1ff640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 0x55f11fe6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f923affd640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 0x7f92300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 -- 172.21.15.86:0/3735096173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6c5d0 msgr2=0x55f11fe6ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6c5d0 0x55f11fe6ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 -- 172.21.15.86:0/3735096173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 msgr2=0x55f11fe6c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 0x55f11fe6c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 -- 172.21.15.86:0/3735096173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11fd5e3a0 con 0x55f11fe77370 2024-09-05T07:04:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f924a1ff640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 0x55f11fe6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f924a1ff640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 0x7f92300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f923affd640 1 -- 172.21.15.86:0/726787595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f92340cee70 con 0x7f924410cf30 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f92491fd640 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f92340cc200 tx=0x7f9234065c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61f34db5d150613 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f924a1ff640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 0x7f92300a51b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f923c00afd0 tx=0x7f923c004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f922ffff640 1 -- 172.21.15.86:0/3735096173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923409aba0 con 0x55f11fe77370 2024-09-05T07:04:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.532+0000 7f922ffff640 1 -- 172.21.15.86:0/3735096173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92340c5e50 con 0x55f11fe77370 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f922ffff640 1 -- 172.21.15.86:0/3735096173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340092b0 con 0x55f11fe77370 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 -- 172.21.15.86:0/3735096173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 msgr2=0x55f11fe79760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f92340cc200 tx=0x7f9234065c90 comp rx=0 tx=0).stop 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 -- 172.21.15.86:0/3735096173 shutdown_connections 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe79760 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6c5d0 0x55f11fe6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 --2- 172.21.15.86:0/3735096173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6db40 0x55f11fe6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 -- 172.21.15.86:0/3735096173 >> 172.21.15.86:0/3735096173 conn(0x55f11fe35310 msgr2=0x55f11fe68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 -- 172.21.15.86:0/3735096173 shutdown_connections 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.533+0000 7f924e03b740 1 -- 172.21.15.86:0/3735096173 wait complete. 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 -- start start 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 Processor -- start 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f11fe7a010 con 0x55f11fe77370 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f11fd8d060 con 0x55f11fe6c5d0 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f11fe7a190 con 0x55f11fe6db40 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924e03b740 10 client.0 _ll_register_callbacks cb 0x55f11fae08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43870/0 (socket says 172.21.15.86:43870) 2024-09-05T07:04:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924a1ff640 1 -- 172.21.15.86:0/2066490453 learned_addr learned my addr 172.21.15.86:0/2066490453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92491fd640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 -- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 msgr2=0x55f11fe74000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe74000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 -- 172.21.15.86:0/2066490453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 msgr2=0x55f11fe745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f92499fe640 1 -- 172.21.15.86:0/2066490453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11fb433c0 con 0x55f11fe6c5d0 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.534+0000 7f924a1ff640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f92499fe640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe76d50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9240005940 tx=0x7f9240031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00b2b003a3ec37a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f922d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240035040 con 0x55f11fe6c5d0 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92340c5e50 con 0x55f11fe6c5d0 2024-09-05T07:04:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f9240004250 con 0x55f11fe6c5d0 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9240004450 con 0x55f11fe6c5d0 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f924e03b740 10 client.29183 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f11fe85ae0 con 0x55f11fe6c5d0 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.535+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92340c5e50 con 0x55f11fe6c5d0 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f9240044070 con 0x55f11fe6c5d0 2024-09-05T07:04:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 10 client.29183 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 10 client.29183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:04:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 20 client.29183 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fe86ef0 0x55f11fe892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 4 client.29183 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11fddf860 con 0x55f11fe86ef0 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f92491fd640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fe86ef0 0x55f11fe892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f922d7fa640 10 client.29183 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.536+0000 7f92491fd640 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fe86ef0 0x55f11fe892e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f1e670fed885261 server_cookie=c298a85ede4c5ee9 in_seq=0 out_seq=0 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f922d7fa640 1 -- 172.21.15.86:0/2066490453 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f11fddf860 con 0x55f11fe86ef0 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f922d7fa640 10 client.29183 handle_command_reply: tid=0 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 client.29183 shutdown 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 msgr2=0x55f11fe76d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe76d50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9240005940 tx=0x7f9240031660 comp rx=0 tx=0).stop 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 shutdown_connections 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fe86ef0 0x55f11fe892e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe77370 0x55f11fe745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe74000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f922d7fa640 0 client.29183 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 --2- 172.21.15.86:0/2066490453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe76d50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 >> 172.21.15.86:0/2066490453 conn(0x55f11fe35310 msgr2=0x55f11fe6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 shutdown_connections 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 1 -- 172.21.15.86:0/2066490453 wait complete. 2024-09-05T07:04:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.537+0000 7f924e03b740 20 client.29183 trim_cache size 0 max 16384 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 Processor -- start 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 -- start start 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe77370 0x55f11fe67c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b9b10 con 0x55f11fe6db40 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92441bb650 con 0x55f11fe6c5d0 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924e03b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92441ba1e0 con 0x55f11fe77370 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92499fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43872/0 (socket says 172.21.15.86:43872) 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 -- 172.21.15.86:0/516183404 learned_addr learned my addr 172.21.15.86:0/516183404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92499fe640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42020/0 (socket says 172.21.15.86:42020) 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f924a1ff640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe77370 0x55f11fe67c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 -- 172.21.15.86:0/516183404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe77370 msgr2=0x55f11fe67c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe77370 0x55f11fe67c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 -- 172.21.15.86:0/516183404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 msgr2=0x55f11fe832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92491fd640 1 -- 172.21.15.86:0/516183404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11fe72fb0 con 0x55f11fe6db40 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.551+0000 7f92499fe640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.552+0000 7f92491fd640 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9234017480 tx=0x7f92340c85d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6744bd4106b8a9f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.552+0000 7f922f7fe640 1 -- 172.21.15.86:0/516183404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923401e070 con 0x55f11fe6db40 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.552+0000 7f922f7fe640 1 -- 172.21.15.86:0/516183404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92340c5e50 con 0x55f11fe6db40 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.552+0000 7f922f7fe640 1 -- 172.21.15.86:0/516183404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340c5a50 con 0x55f11fe6db40 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- 172.21.15.86:0/516183404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 msgr2=0x55f11fe738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9234017480 tx=0x7f92340c85d0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- 172.21.15.86:0/516183404 shutdown_connections 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe77370 0x55f11fe67c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6db40 0x55f11fe738c0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 --2- 172.21.15.86:0/516183404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe6c5d0 0x55f11fe832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- 172.21.15.86:0/516183404 >> 172.21.15.86:0/516183404 conn(0x55f11fe35310 msgr2=0x55f11fe67500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- 172.21.15.86:0/516183404 shutdown_connections 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- 172.21.15.86:0/516183404 wait complete. 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 -- start start 2024-09-05T07:04:25.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.553+0000 7f924e03b740 1 Processor -- start 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 0x55f11fe7bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92441baf80 con 0x55f11fe6c5d0 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92441ba8b0 con 0x55f11fe77370 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b7b10 con 0x55f11fe6db40 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924e03b740 10 client.0 _ll_register_callbacks cb 0x7f9240032ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43880/0 (socket says 172.21.15.86:43880) 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 -- 172.21.15.86:0/1578606178 learned_addr learned my addr 172.21.15.86:0/1578606178 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924a1ff640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 0x55f11fe7bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92491fd640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 -- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 msgr2=0x55f11fe7b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe7b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 -- 172.21.15.86:0/1578606178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 msgr2=0x55f11fe7bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 0x55f11fe7bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 -- 172.21.15.86:0/1578606178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11fef0b60 con 0x55f11fe6c5d0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f924a1ff640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 0x55f11fe7bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92491fd640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe7b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f92499fe640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f92400096c0 tx=0x7f9240032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a187c4492ab2cafb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.554+0000 7f922d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240059060 con 0x55f11fe6c5d0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f92400316d0 con 0x55f11fe6c5d0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92340c5e50 con 0x55f11fe6c5d0 2024-09-05T07:04:25.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9240035030 con 0x55f11fe6c5d0 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f924e03b740 10 client.20523 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f11fef1dc0 con 0x55f11fe6c5d0 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.555+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92340c5e50 con 0x55f11fe6c5d0 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f9240044040 con 0x55f11fe6c5d0 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 10 client.20523 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 10 client.20523 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 20 client.20523 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fef24e0 0x55f11fef4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:25.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 4 client.20523 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.556+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f11fdd9d70 con 0x55f11fef24e0 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f92491fd640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fef24e0 0x55f11fef4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f922d7fa640 10 client.20523 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f92491fd640 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fef24e0 0x55f11fef4900 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21b605e3e3a1c423 server_cookie=86b3d1a674452358 in_seq=0 out_seq=0 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f922d7fa640 1 -- 172.21.15.86:0/1578606178 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f11fdd9d70 con 0x55f11fef24e0 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f922d7fa640 10 client.20523 handle_command_reply: tid=0 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f924e03b740 1 client.20523 shutdown 2024-09-05T07:04:25.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 msgr2=0x55f11fe7b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f92400096c0 tx=0x7f9240032f40 comp rx=0 tx=0).stop 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 shutdown_connections 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.557+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55f11fef24e0 0x55f11fef4900 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f922d7fa640 0 client.20523 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f11fe77370 0x55f11fe7bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f11fe6db40 0x55f11fe7b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 --2- 172.21.15.86:0/1578606178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11fe6c5d0 0x55f11fe7b150 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 >> 172.21.15.86:0/1578606178 conn(0x55f11fe35310 msgr2=0x55f11fe6e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 shutdown_connections 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 1 -- 172.21.15.86:0/1578606178 wait complete. 2024-09-05T07:04:25.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924e03b740 20 client.20523 trim_cache size 0 max 16384 2024-09-05T07:04:25.568 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:04:25.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 msgr2=0x7f92300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 0x7f92300a51b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f923c00afd0 tx=0x7f923c004a40 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 msgr2=0x7f92441a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f923400b4a0 tx=0x7f923400b960 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 shutdown_connections 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92300a2cf0 0x7f92300a51b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.558+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9244112640 0x7f92441a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.559+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924410cf30 0x7f92441a3290 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.559+0000 7f924bc62640 1 --2- 172.21.15.86:0/726787595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f924410c590 0x7f92441a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.559+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 >> 172.21.15.86:0/726787595 conn(0x7f924406cdc0 msgr2=0x7f924410b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.559+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 shutdown_connections 2024-09-05T07:04:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:25.559+0000 7f924bc62640 1 -- 172.21.15.86:0/726787595 wait complete. 2024-09-05T07:04:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:25 smithi115 ceph-mon[34791]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:25.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:25 smithi086 ceph-mon[30943]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:25.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.96 MB/sec execute 463 sec latency 26683.860 ms 2024-09-05T07:04:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:25 smithi096 ceph-mon[34468]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:26.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:04:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:26.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.95 MB/sec execute 464 sec latency 27683.893 ms 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 -- 172.21.15.86:0/2660399432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8106f10 msgr2=0x7f03f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 --2- 172.21.15.86:0/2660399432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8106f10 0x7f03f810ea90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009ca0 tx=0x7f03f002a8c0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 -- 172.21.15.86:0/2660399432 shutdown_connections 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 --2- 172.21.15.86:0/2660399432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8106f10 0x7f03f810ea90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 --2- 172.21.15.86:0/2660399432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 --2- 172.21.15.86:0/2660399432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8101de0 0x7f03f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.827+0000 7f03fea04640 1 -- 172.21.15.86:0/2660399432 >> 172.21.15.86:0/2660399432 conn(0x7f03f80fd8c0 msgr2=0x7f03f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- 172.21.15.86:0/2660399432 shutdown_connections 2024-09-05T07:04:26.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- 172.21.15.86:0/2660399432 wait complete. 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 Processor -- start 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- start start 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8106f10 0x7f03f819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8111ec0 con 0x7f03f8101de0 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8111d40 con 0x7f03f8102790 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03fea04640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8112040 con 0x7f03f8106f10 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43886/0 (socket says 172.21.15.86:43886) 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.828+0000 7f03f7fff640 1 -- 172.21.15.86:0/3639138971 learned_addr learned my addr 172.21.15.86:0/3639138971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03fcfa1640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8106f10 0x7f03f819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f77fe640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 -- 172.21.15.86:0/3639138971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8106f10 msgr2=0x7f03f819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8106f10 0x7f03f819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 -- 172.21.15.86:0/3639138971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 msgr2=0x7f03f81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 -- 172.21.15.86:0/3639138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f819d430 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f77fe640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f7fff640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b4a0 tx=0x7f03e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8014070 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f819d720 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e8010ac0 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8017c30 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.829+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81a5000 con 0x7f03f8101de0 2024-09-05T07:04:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.830+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f81021c0 con 0x7f03f8101de0 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.831+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03e8017070 con 0x7f03f8101de0 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.832+0000 7f03f57fa640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 0x7f03e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.832+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f03e80cefe0 con 0x7f03f8101de0 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.832+0000 7f03f77fe640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 0x7f03e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.833+0000 7f03f77fe640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 0x7f03e00a51b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006fd0 tx=0x7f03ec008040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:26.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:26.835+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03e809b8d0 con 0x7f03f8101de0 2024-09-05T07:04:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.046+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f810a120 con 0x7f03f8101de0 2024-09-05T07:04:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.047+0000 7f03f57fa640 1 -- 172.21.15.86:0/3639138971 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7f03e809b1d0 con 0x7f03f8101de0 2024-09-05T07:04:27.048 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:04:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:04:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:04:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 msgr2=0x7f03e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 0x7f03e00a51b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006fd0 tx=0x7f03ec008040 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 msgr2=0x7f03f8198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b4a0 tx=0x7f03e800b960 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 shutdown_connections 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f03e00a2cf0 0x7f03e00a51b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03f8106f10 0x7f03f819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03f8102790 0x7f03f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 --2- 172.21.15.86:0/3639138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f8101de0 0x7f03f8198480 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 >> 172.21.15.86:0/3639138971 conn(0x7f03f80fd8c0 msgr2=0x7f03f80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 shutdown_connections 2024-09-05T07:04:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.049+0000 7f03fea04640 1 -- 172.21.15.86:0/3639138971 wait complete. 2024-09-05T07:04:27.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 -- 172.21.15.86:0/2004915586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4100a80 msgr2=0x7f46c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/2004915586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4100a80 0x7f46c4100ee0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009930 tx=0x7f46b402a420 comp rx=0 tx=0).stop 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 -- 172.21.15.86:0/2004915586 shutdown_connections 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/2004915586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4101420 0x7f46c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/2004915586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4100a80 0x7f46c4100ee0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/2004915586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4106a80 0x7f46c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 -- 172.21.15.86:0/2004915586 >> 172.21.15.86:0/2004915586 conn(0x7f46c40fc7a0 msgr2=0x7f46c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.543+0000 7f46cc7d7640 1 -- 172.21.15.86:0/2004915586 shutdown_connections 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 -- 172.21.15.86:0/2004915586 wait complete. 2024-09-05T07:04:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 Processor -- start 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 -- start start 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4100a80 0x7f46c4196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4101420 0x7f46c41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4110d50 con 0x7f46c4106a80 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4110bd0 con 0x7f46c4100a80 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cc7d7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4110ed0 con 0x7f46c4101420 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cad74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cad74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43902/0 (socket says 172.21.15.86:43902) 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46cad74640 1 -- 172.21.15.86:0/4293451209 learned_addr learned my addr 172.21.15.86:0/4293451209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.544+0000 7f46ca573640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4100a80 0x7f46c4196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46c9d72640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4101420 0x7f46c41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 -- 172.21.15.86:0/4293451209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4101420 msgr2=0x7f46c41974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4101420 0x7f46c41974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 -- 172.21.15.86:0/4293451209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4100a80 msgr2=0x7f46c4196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4100a80 0x7f46c4196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 -- 172.21.15.86:0/4293451209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c419c540 con 0x7f46c4106a80 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cad74640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b970 tx=0x7f46bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46bb7fe640 1 -- 172.21.15.86:0/4293451209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc019070 con 0x7f46c4106a80 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46bb7fe640 1 -- 172.21.15.86:0/4293451209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46bc014440 con 0x7f46c4106a80 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c419c830 con 0x7f46c4106a80 2024-09-05T07:04:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46bb7fe640 1 -- 172.21.15.86:0/4293451209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0075d0 con 0x7f46c4106a80 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.545+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c419ccf0 con 0x7f46c4106a80 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.547+0000 7f46bb7fe640 1 -- 172.21.15.86:0/4293451209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f46bc01c070 con 0x7f46c4106a80 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.547+0000 7f46cebb0740 1 Processor -- start 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 -- start start 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abb5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abc2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55948ab1b2f0 con 0x55948abb6b40 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55948aac2cf0 con 0x55948abb55d0 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cebb0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55948ab172b0 con 0x55948abc0370 2024-09-05T07:04:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43914/0 (socket says 172.21.15.86:43914) 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 -- 172.21.15.86:0/130860815 learned_addr learned my addr 172.21.15.86:0/130860815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46bb7fe640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 0x7f46b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46ca573640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abb5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abc2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 -- 172.21.15.86:0/130860815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 msgr2=0x55948abc2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abc2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 -- 172.21.15.86:0/130860815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 msgr2=0x55948abb5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46cad74640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 0x7f46b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abb5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 -- 172.21.15.86:0/130860815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948aaa73a0 con 0x55948abb6b40 2024-09-05T07:04:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46ca573640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abb5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46bb7fe640 1 -- 172.21.15.86:0/4293451209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f46bc0ced40 con 0x7f46c4106a80 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.548+0000 7f46c9d72640 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f46b4005ab0 tx=0x7f46b40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb321c53612a257 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46b8ff9640 1 -- 172.21.15.86:0/130860815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4043020 con 0x55948abb6b40 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46b8ff9640 1 -- 172.21.15.86:0/130860815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46b4031840 con 0x55948abb6b40 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46b8ff9640 1 -- 172.21.15.86:0/130860815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4035050 con 0x55948abb6b40 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 -- 172.21.15.86:0/130860815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 msgr2=0x55948abb5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cad74640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 0x7f46b00a51b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cc210 tx=0x7f46bc065b60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f46b4005ab0 tx=0x7f46b40058b0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 -- 172.21.15.86:0/130860815 shutdown_connections 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abc2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abb5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 --2- 172.21.15.86:0/130860815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abb5090 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 -- 172.21.15.86:0/130860815 >> 172.21.15.86:0/130860815 conn(0x55948ab7e310 msgr2=0x55948abb1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 -- 172.21.15.86:0/130860815 shutdown_connections 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.549+0000 7f46cebb0740 1 -- 172.21.15.86:0/130860815 wait complete. 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 -- start start 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 Processor -- start 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abbfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abbd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55948abc3010 con 0x55948abb6b40 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55948aad6060 con 0x55948abb55d0 2024-09-05T07:04:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55948abc3190 con 0x55948abc0370 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46cebb0740 10 client.0 _ll_register_callbacks cb 0x55948a8298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43916/0 (socket says 172.21.15.86:43916) 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 -- 172.21.15.86:0/3178988205 learned_addr learned my addr 172.21.15.86:0/3178988205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abbfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abbd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 -- 172.21.15.86:0/3178988205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 msgr2=0x55948abbd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abbd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 -- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 msgr2=0x55948abbfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abbfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46ca573640 1 -- 172.21.15.86:0/3178988205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55948a88c3c0 con 0x55948abb6b40 2024-09-05T07:04:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.550+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abbfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46ca573640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f46c0012850 tx=0x7f46c0012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46056475fff8ba88 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0010dc0 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f46c00047a0 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b4031840 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f46c0025050 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 10 client.20541 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55948abceaa0 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46b4031840 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f46c000e570 con 0x55948abb6b40 2024-09-05T07:04:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 10 client.20541 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 10 client.20541 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 20 client.20541 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948abd0000 0x55948abd2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 4 client.20541 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.551+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55948ab28860 con 0x55948abd0000 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.552+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948abd0000 0x55948abd2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.552+0000 7f46adffb640 10 client.20541 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.552+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948abd0000 0x55948abd2420 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d42aa60a274e8702 server_cookie=2478549a1e13eb39 in_seq=0 out_seq=0 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46adffb640 1 -- 172.21.15.86:0/3178988205 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55948ab28860 con 0x55948abd0000 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46adffb640 10 client.20541 handle_command_reply: tid=0 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 client.20541 shutdown 2024-09-05T07:04:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 msgr2=0x55948abbcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f46c0012850 tx=0x7f46c0012d10 comp rx=0 tx=0).stop 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 shutdown_connections 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948abd0000 0x55948abd2420 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abc0370 0x55948abbd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abbcfb0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46adffb640 0 client.20541 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3178988205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb55d0 0x55948abbfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 >> 172.21.15.86:0/3178988205 conn(0x55948ab7e310 msgr2=0x55948abb7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 shutdown_connections 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 1 -- 172.21.15.86:0/3178988205 wait complete. 2024-09-05T07:04:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.553+0000 7f46cebb0740 20 client.20541 trim_cache size 0 max 16384 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 Processor -- start 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 -- start start 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abc0370 0x55948abd8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b5630 con 0x55948abb6b40 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b7170 con 0x55948abc0370 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46cebb0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b5d00 con 0x55948abb55d0 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43924/0 (socket says 172.21.15.86:43924) 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 -- 172.21.15.86:0/3444559669 learned_addr learned my addr 172.21.15.86:0/3444559669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 -- 172.21.15.86:0/3444559669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 msgr2=0x55948abcc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46c9d72640 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abc0370 0x55948abd8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abcc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 -- 172.21.15.86:0/3444559669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abc0370 msgr2=0x55948abd8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abc0370 0x55948abd8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 -- 172.21.15.86:0/3444559669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948abb2960 con 0x55948abb6b40 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.566+0000 7f46ca573640 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f46c000eb80 tx=0x7f46c000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e99ba2e02ed207 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46affff640 1 -- 172.21.15.86:0/3444559669 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000ee00 con 0x55948abb6b40 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46affff640 1 -- 172.21.15.86:0/3444559669 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c0010bc0 con 0x55948abb6b40 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46affff640 1 -- 172.21.15.86:0/3444559669 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00116a0 con 0x55948abb6b40 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 -- 172.21.15.86:0/3444559669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 msgr2=0x55948abd7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f46c000eb80 tx=0x7f46c000a510 comp rx=0 tx=0).stop 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 -- 172.21.15.86:0/3444559669 shutdown_connections 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abc0370 0x55948abd8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abb6b40 0x55948abd7cd0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 --2- 172.21.15.86:0/3444559669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abcc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 -- 172.21.15.86:0/3444559669 >> 172.21.15.86:0/3444559669 conn(0x55948ab7e310 msgr2=0x55948abbc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 -- 172.21.15.86:0/3444559669 shutdown_connections 2024-09-05T07:04:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.567+0000 7f46cebb0740 1 -- 172.21.15.86:0/3444559669 wait complete. 2024-09-05T07:04:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 -- start start 2024-09-05T07:04:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 Processor -- start 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abc8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb6b40 0x55948abc8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b6aa0 con 0x55948abc0370 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b63d0 con 0x55948abb6b40 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46c41b3630 con 0x55948abb55d0 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46cebb0740 10 client.0 _ll_register_callbacks cb 0x7f46c000a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43926/0 (socket says 172.21.15.86:43926) 2024-09-05T07:04:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 -- 172.21.15.86:0/2535495662 learned_addr learned my addr 172.21.15.86:0/2535495662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46ca573640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb6b40 0x55948abc8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abc8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 -- 172.21.15.86:0/2535495662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 msgr2=0x55948abc8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abc8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 -- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb6b40 msgr2=0x55948abc8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb6b40 0x55948abc8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.568+0000 7f46c9d72640 1 -- 172.21.15.86:0/2535495662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55948abb8120 con 0x55948abc0370 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46c9d72640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f46b4005610 tx=0x7f46b40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d602f5c760a38e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4043040 con 0x55948abc0370 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f46b4004030 con 0x55948abc0370 2024-09-05T07:04:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c0010bc0 con 0x55948abc0370 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f46b4035040 con 0x55948abc0370 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 10 client.20547 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55948ac3a6a0 con 0x55948abc0370 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c0010bc0 con 0x55948abc0370 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f46b4048040 con 0x55948abc0370 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 10 client.20547 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 10 client.20547 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 20 client.20547 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948ac3a8c0 0x55948ac3ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 4 client.20547 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.569+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55948ab22d70 con 0x55948ac3a8c0 2024-09-05T07:04:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46ca573640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948ac3a8c0 0x55948ac3ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46adffb640 10 client.20547 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46ca573640 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948ac3a8c0 0x55948ac3ccb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72c111e5f7f99c70 server_cookie=3933701e5b3ec24a in_seq=0 out_seq=0 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46adffb640 1 -- 172.21.15.86:0/2535495662 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55948ab22d70 con 0x55948ac3a8c0 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46adffb640 10 client.20547 handle_command_reply: tid=0 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 client.20547 shutdown 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 msgr2=0x55948abc3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f46b4005610 tx=0x7f46b40058b0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 shutdown_connections 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55948ac3a8c0 0x55948ac3ccb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55948abc0370 0x55948abc3b30 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55948abb6b40 0x55948abc8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 --2- 172.21.15.86:0/2535495662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55948abb55d0 0x55948abc8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 >> 172.21.15.86:0/2535495662 conn(0x55948ab7e310 msgr2=0x55948abbe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46adffb640 0 client.20547 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 shutdown_connections 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 1 -- 172.21.15.86:0/2535495662 wait complete. 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.570+0000 7f46cebb0740 20 client.20547 trim_cache size 0 max 16384 2024-09-05T07:04:27.578 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 msgr2=0x7f46b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 0x7f46b00a51b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cc210 tx=0x7f46bc065b60 comp rx=0 tx=0).stop 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 msgr2=0x7f46c419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b970 tx=0x7f46bc00bd70 comp rx=0 tx=0).stop 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 shutdown_connections 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f46b00a2cf0 0x7f46b00a51b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c4106a80 0x7f46c419c000 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46c4101420 0x7f46c41974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 --2- 172.21.15.86:0/4293451209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46c4100a80 0x7f46c4196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 >> 172.21.15.86:0/4293451209 conn(0x7f46c40fc7a0 msgr2=0x7f46c40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 shutdown_connections 2024-09-05T07:04:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:27.571+0000 7f46cc7d7640 1 -- 172.21.15.86:0/4293451209 wait complete. 2024-09-05T07:04:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:27 smithi115 ceph-mon[34791]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:27 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3639138971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:27 smithi086 ceph-mon[30943]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:27.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:27 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3639138971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:27.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.94 MB/sec execute 465 sec latency 28683.964 ms 2024-09-05T07:04:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:27 smithi096 ceph-mon[34468]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T07:04:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:27 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3639138971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:04:28.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:04:28.649 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id e131ad72 --release --await' 2024-09-05T07:04:28.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id e131ad72 --release --await 2024-09-05T07:04:28.777 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:28.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.93 MB/sec execute 466 sec latency 29683.992 ms 2024-09-05T07:04:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 -- 172.21.15.86:0/2344435081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8101420 msgr2=0x7fa2d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2344435081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8101420 0x7fa2d810d950 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009cd0 tx=0x7fa2d002a8e0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 -- 172.21.15.86:0/2344435081 shutdown_connections 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2344435081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8101420 0x7fa2d810d950 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2344435081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8100a80 0x7fa2d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2344435081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8106a80 0x7fa2d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 -- 172.21.15.86:0/2344435081 >> 172.21.15.86:0/2344435081 conn(0x7fa2d80fc7a0 msgr2=0x7fa2d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 -- 172.21.15.86:0/2344435081 shutdown_connections 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.271+0000 7fa2de34d640 1 -- 172.21.15.86:0/2344435081 wait complete. 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 Processor -- start 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 -- start start 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 0x7fa2d81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8106a80 0x7fa2d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8111e50 con 0x7fa2d8100a80 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8111cd0 con 0x7fa2d8101420 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2de34d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8111fd0 con 0x7fa2d8106a80 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43938/0 (socket says 172.21.15.86:43938) 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2551834635 learned_addr learned my addr 172.21.15.86:0/2551834635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 0x7fa2d81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8106a80 0x7fa2d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2551834635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8106a80 msgr2=0x7fa2d819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8106a80 0x7fa2d819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2551834635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 msgr2=0x7fa2d81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 0x7fa2d81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2551834635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d819d320 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 0x7fa2d81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.272+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800d950 tx=0x7fa2c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.273+0000 7fa2d5ffb640 1 -- 172.21.15.86:0/2551834635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8017070 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.273+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d819d610 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.273+0000 7fa2d5ffb640 1 -- 172.21.15.86:0/2551834635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c8009c30 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.273+0000 7fa2d5ffb640 1 -- 172.21.15.86:0/2551834635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800f620 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.273+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a4ef0 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2d5ffb640 1 -- 172.21.15.86:0/2551834635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2c80109a0 con 0x7fa2d8100a80 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 Processor -- start 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 -- start start 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600be37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55600bd9d2f0 con 0x55600be42370 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55600bd44cf0 con 0x55600be375d0 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2e0726740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55600bd992b0 con 0x55600be38b40 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.275+0000 7fa2cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43942/0 (socket says 172.21.15.86:43942) 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 -- 172.21.15.86:0/3268481063 learned_addr learned my addr 172.21.15.86:0/3268481063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600be37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 -- 172.21.15.86:0/3268481063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 msgr2=0x55600be37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 -- 172.21.15.86:0/3268481063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 msgr2=0x55600be37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600be37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 -- 172.21.15.86:0/3268481063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600bd293a0 con 0x55600be42370 2024-09-05T07:04:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600be37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2d5ffb640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009c20 tx=0x7fa2c000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06b2b6696336567 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2d5ffb640 1 -- 172.21.15.86:0/2551834635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa2c80cf520 con 0x7fa2d8100a80 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2ce7fc640 1 -- 172.21.15.86:0/3268481063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0016070 con 0x55600be42370 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2ce7fc640 1 -- 172.21.15.86:0/3268481063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c0011920 con 0x55600be42370 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.276+0000 7fa2ce7fc640 1 -- 172.21.15.86:0/3268481063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0010750 con 0x55600be42370 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- 172.21.15.86:0/3268481063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 msgr2=0x55600be44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005e30 tx=0x7fa2d0005c30 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009c20 tx=0x7fa2c000e6c0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- 172.21.15.86:0/3268481063 shutdown_connections 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600be44760 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600be37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 --2- 172.21.15.86:0/3268481063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- 172.21.15.86:0/3268481063 >> 172.21.15.86:0/3268481063 conn(0x55600be00310 msgr2=0x55600be33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- 172.21.15.86:0/3268481063 shutdown_connections 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- 172.21.15.86:0/3268481063 wait complete. 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 -- start start 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.277+0000 7fa2e0726740 1 Processor -- start 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be3eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55600be45070 con 0x55600be375d0 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55600bd58060 con 0x55600be42370 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55600be451f0 con 0x55600be38b40 2024-09-05T07:04:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 10 client.0 _ll_register_callbacks cb 0x55600baab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43944/0 (socket says 172.21.15.86:43944) 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2872111803 learned_addr learned my addr 172.21.15.86:0/2872111803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be3eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2872111803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 msgr2=0x55600be3eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be3eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 msgr2=0x55600be3f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be3f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 -- 172.21.15.86:0/2872111803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55600bb0e3c0 con 0x55600be375d0 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be3eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c809b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2f029e87f0ef3c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8020070 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa2c8097c70 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c0011920 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa2c8097040 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 10 client.20559 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.278+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55600be50b30 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c0011920 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fa2c80cf120 con 0x55600be375d0 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 10 client.20559 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 10 client.20559 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 20 client.20559 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600be51ff0 0x55600be54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 4 client.20559 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55600bdaa860 con 0x55600be51ff0 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.279+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600be51ff0 0x55600be54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2b77fe640 10 client.20559 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600be51ff0 0x55600be54440 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a357fb55bd0e1bf4 server_cookie=8101f7284c0d396f in_seq=0 out_seq=0 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2872111803 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55600bdaa860 con 0x55600be51ff0 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2b77fe640 10 client.20559 handle_command_reply: tid=0 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 client.20559 shutdown 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 msgr2=0x55600be41d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c809b500 comp rx=0 tx=0).stop 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 shutdown_connections 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600be51ff0 0x55600be54440 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be3eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2872111803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be41d40 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 >> 172.21.15.86:0/2872111803 conn(0x55600be00310 msgr2=0x55600be39990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.280+0000 7fa2b77fe640 0 client.20559 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.281+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 shutdown_connections 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.281+0000 7fa2e0726740 1 -- 172.21.15.86:0/2872111803 wait complete. 2024-09-05T07:04:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.281+0000 7fa2e0726740 20 client.20559 trim_cache size 0 max 16384 2024-09-05T07:04:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.293+0000 7fa2e0726740 1 Processor -- start 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.293+0000 7fa2e0726740 1 -- start start 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81af0c0 con 0x55600be375d0 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b0c00 con 0x55600be42370 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2e0726740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81af790 con 0x55600be38b40 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43956/0 (socket says 172.21.15.86:43956) 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 -- 172.21.15.86:0/1519341020 learned_addr learned my addr 172.21.15.86:0/1519341020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 -- 172.21.15.86:0/1519341020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 msgr2=0x55600be40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be40f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 -- 172.21.15.86:0/1519341020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 msgr2=0x55600be4a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be4a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 -- 172.21.15.86:0/1519341020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600be3dfa0 con 0x55600be375d0 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be4a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80cce30 tx=0x7fa2c8010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6f552c3cc6debb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2cdffb640 1 -- 172.21.15.86:0/1519341020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8020070 con 0x55600be375d0 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2cdffb640 1 -- 172.21.15.86:0/1519341020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c809b600 con 0x55600be375d0 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.294+0000 7fa2cdffb640 1 -- 172.21.15.86:0/1519341020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c809ab40 con 0x55600be375d0 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- 172.21.15.86:0/1519341020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 msgr2=0x55600be3d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80cce30 tx=0x7fa2c8010f70 comp rx=0 tx=0).stop 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- 172.21.15.86:0/1519341020 shutdown_connections 2024-09-05T07:04:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be42370 0x55600be4a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600be40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 --2- 172.21.15.86:0/1519341020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be375d0 0x55600be3d8c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- 172.21.15.86:0/1519341020 >> 172.21.15.86:0/1519341020 conn(0x55600be00310 msgr2=0x55600be40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- 172.21.15.86:0/1519341020 shutdown_connections 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- 172.21.15.86:0/1519341020 wait complete. 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 -- start start 2024-09-05T07:04:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.295+0000 7fa2e0726740 1 Processor -- start 2024-09-05T07:04:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600beb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600beb51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b0530 con 0x55600be42370 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81afe60 con 0x55600be375d0 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ad0c0 con 0x55600be38b40 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2e0726740 10 client.0 _ll_register_callbacks cb 0x7fa2c800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43964/0 (socket says 172.21.15.86:43964) 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 -- 172.21.15.86:0/2926496045 learned_addr learned my addr 172.21.15.86:0/2926496045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2cf7fe640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600beb51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600beb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 -- 172.21.15.86:0/2926496045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 msgr2=0x55600beb51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600beb51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 -- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 msgr2=0x55600beb4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600beb4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 -- 172.21.15.86:0/2926496045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55600bebfe50 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2dc8ea640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009cd0 tx=0x7fa2d00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6df987ea7ad886d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.296+0000 7fa2b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0058060 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c809b600 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa2d0004310 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa2d005f050 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 10 client.20565 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55600bec10c0 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c809b600 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fa2d0030070 con 0x55600be42370 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 10 client.20565 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 10 client.20565 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 20 client.20565 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600bec1870 0x55600bec3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 4 client.20565 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55600bda4d70 con 0x55600bec1870 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.297+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600bec1870 0x55600bec3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2b77fe640 10 client.20565 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2d7fff640 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600bec1870 0x55600bec3c90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b6bea96863e26f7 server_cookie=e4ed644fa6f8f46 in_seq=0 out_seq=0 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2b77fe640 1 -- 172.21.15.86:0/2926496045 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55600bda4d70 con 0x55600bec1870 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2b77fe640 10 client.20565 handle_command_reply: tid=0 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 client.20565 shutdown 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 msgr2=0x55600beb5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009cd0 tx=0x7fa2d00095c0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 shutdown_connections 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55600bec1870 0x55600bec3c90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55600be42370 0x55600beb5720 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55600be38b40 0x55600beb51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 --2- 172.21.15.86:0/2926496045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55600be375d0 0x55600beb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 >> 172.21.15.86:0/2926496045 conn(0x55600be00310 msgr2=0x55600be40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2b77fe640 0 client.20565 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 shutdown_connections 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 1 -- 172.21.15.86:0/2926496045 wait complete. 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.298+0000 7fa2e0726740 20 client.20565 trim_cache size 0 max 16384 2024-09-05T07:04:29.304 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 msgr2=0x7fa2c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005e30 tx=0x7fa2d0005c30 comp rx=0 tx=0).stop 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 msgr2=0x7fa2d8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800d950 tx=0x7fa2c800de10 comp rx=0 tx=0).stop 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 shutdown_connections 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d8106a80 0x7fa2d819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2d8101420 0x7fa2d81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 --2- 172.21.15.86:0/2551834635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d8100a80 0x7fa2d8198370 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 >> 172.21.15.86:0/2551834635 conn(0x7fa2d80fc7a0 msgr2=0x7fa2d80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 shutdown_connections 2024-09-05T07:04:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:29.299+0000 7fa2de34d640 1 -- 172.21.15.86:0/2551834635 wait complete. 2024-09-05T07:04:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:29 smithi086 ceph-mon[30943]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-05T07:04:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:29 smithi115 ceph-mon[34791]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-05T07:04:29.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66466 5.91 MB/sec execute 467 sec latency 30684.067 ms 2024-09-05T07:04:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:29 smithi096 ceph-mon[34468]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-05T07:04:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:30.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:04:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 -- 172.21.15.86:0/2278891830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8102d40 msgr2=0x7fdce8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/2278891830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8102d40 0x7fdce8103120 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ca0 tx=0x7fdcd802a950 comp rx=0 tx=0).stop 2024-09-05T07:04:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 -- 172.21.15.86:0/2278891830 shutdown_connections 2024-09-05T07:04:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/2278891830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 0x7fdce810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/2278891830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8103660 0x7fdce810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/2278891830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8102d40 0x7fdce8103120 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 -- 172.21.15.86:0/2278891830 >> 172.21.15.86:0/2278891830 conn(0x7fdce80fc7a0 msgr2=0x7fdce80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 -- 172.21.15.86:0/2278891830 shutdown_connections 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.487+0000 7fdcedf7a640 1 -- 172.21.15.86:0/2278891830 wait complete. 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.488+0000 7fdcedf7a640 1 Processor -- start 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.488+0000 7fdcedf7a640 1 -- start start 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8102d40 0x7fdce81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 0x7fdce819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdce8111e20 con 0x7fdce8103660 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdce8111ca0 con 0x7fdce810c170 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdce8111fa0 con 0x7fdce8102d40 2024-09-05T07:04:30.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43978/0 (socket says 172.21.15.86:43978) 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4125631677 learned_addr learned my addr 172.21.15.86:0/4125631677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce77fe640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8102d40 0x7fdce81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce7fff640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 0x7fdce819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4125631677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8102d40 msgr2=0x7fdce81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8102d40 0x7fdce81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4125631677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 msgr2=0x7fdce819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 0x7fdce819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4125631677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce819d390 con 0x7fdce8103660 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00d750 tx=0x7fdcdc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce4ff9640 1 -- 172.21.15.86:0/4125631677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc017070 con 0x7fdce8103660 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce4ff9640 1 -- 172.21.15.86:0/4125631677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcdc005020 con 0x7fdce8103660 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdce4ff9640 1 -- 172.21.15.86:0/4125631677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004c60 con 0x7fdce8103660 2024-09-05T07:04:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.489+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce819d680 con 0x7fdce8103660 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.490+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce81a4f60 con 0x7fdce8103660 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdce4ff9640 1 -- 172.21.15.86:0/4125631677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdcdc009a30 con 0x7fdce8103660 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 Processor -- start 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 -- start start 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608df504620 con 0x5608df5aa240 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5608df4abe50 con 0x5608df59f060 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.492+0000 7fdcf0353740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5608df5005e0 con 0x5608df59f980 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42190/0 (socket says 172.21.15.86:42190) 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 -- 172.21.15.86:0/1921085909 learned_addr learned my addr 172.21.15.86:0/1921085909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce4ff9640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 0x7fdcd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce77fe640 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 -- 172.21.15.86:0/1921085909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 msgr2=0x5608df5a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 -- 172.21.15.86:0/1921085909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 msgr2=0x5608df5ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce4ff9640 1 -- 172.21.15.86:0/4125631677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fdcdc0cf160 con 0x7fdce8103660 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 -- 172.21.15.86:0/1921085909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608df49cd40 con 0x5608df59f060 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 0x7fdcd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdce7fff640 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011e90 tx=0x7fdce000a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd6373e7e31f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdccdffb640 1 -- 172.21.15.86:0/1921085909 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce001d750 con 0x5608df59f060 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.493+0000 7fdccdffb640 1 -- 172.21.15.86:0/1921085909 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce001dda0 con 0x5608df59f060 2024-09-05T07:04:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.494+0000 7fdccdffb640 1 -- 172.21.15.86:0/1921085909 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0028c90 con 0x5608df59f060 2024-09-05T07:04:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.494+0000 7fdce7fff640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 0x7fdcd00a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdce0025000 tx=0x7fdce0026000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.494+0000 7fdcf0353740 1 -- 172.21.15.86:0/1921085909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 msgr2=0x5608df59f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.494+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011e90 tx=0x7fdce000a8b0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 -- 172.21.15.86:0/1921085909 shutdown_connections 2024-09-05T07:04:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1921085909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df59f440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 -- 172.21.15.86:0/1921085909 >> 172.21.15.86:0/1921085909 conn(0x5608df59b5f0 msgr2=0x5608df59b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 -- 172.21.15.86:0/1921085909 shutdown_connections 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 -- 172.21.15.86:0/1921085909 wait complete. 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 -- start start 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 Processor -- start 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.495+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcf0353740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608df5ace10 con 0x5608df5aa240 2024-09-05T07:04:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcf0353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5608df48cf10 con 0x5608df59f060 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcf0353740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5608df5acf90 con 0x5608df59f980 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcf0353740 10 client.0 _ll_register_callbacks cb 0x5608df4b0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5a7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44000/0 (socket says 172.21.15.86:44000) 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce7fff640 1 -- 172.21.15.86:0/4176954547 learned_addr learned my addr 172.21.15.86:0/4176954547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 msgr2=0x5608df5a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4176954547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 msgr2=0x5608df5a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5a7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 -- 172.21.15.86:0/4176954547 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608df213580 con 0x5608df59f980 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a76c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cc470 tx=0x7fdcdc09af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad403dc2742f20a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc020040 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.496+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce001dda0 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fdcdc009d20 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdcdc0a3eb0 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 10 client.29269 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5608df2dd960 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce001dda0 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fdcdc0cc6f0 con 0x5608df59f980 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 10 client.29269 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 10 client.29269 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 20 client.29269 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:30.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df5b9e00 0x5608df5bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 4 client.29269 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608df511b90 con 0x5608df5b9e00 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.497+0000 7fdce77fe640 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df5b9e00 0x5608df5bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.498+0000 7fdcc6ffd640 10 client.29269 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.498+0000 7fdce77fe640 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df5b9e00 0x5608df5bc230 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c6797839882166 server_cookie=76e123279213221a in_seq=0 out_seq=0 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.498+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/4176954547 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608df511b90 con 0x5608df5b9e00 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcc6ffd640 10 client.29269 handle_command_reply: tid=0 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 client.29269 shutdown 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 msgr2=0x5608df5a76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a76c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cc470 tx=0x7fdcdc09af30 comp rx=0 tx=0).stop 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 shutdown_connections 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df5b9e00 0x5608df5bc230 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df5a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5a76c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcc6ffd640 0 client.29269 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 --2- 172.21.15.86:0/4176954547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 >> 172.21.15.86:0/4176954547 conn(0x5608df59b5f0 msgr2=0x5608df5a12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 shutdown_connections 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 1 -- 172.21.15.86:0/4176954547 wait complete. 2024-09-05T07:04:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.499+0000 7fdcf0353740 20 client.29269 trim_cache size 0 max 16384 2024-09-05T07:04:30.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.516+0000 7fdcf0353740 1 Processor -- start 2024-09-05T07:04:30.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.516+0000 7fdcf0353740 1 -- start start 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df59a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 0x5608df5c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81acfb0 con 0x5608df59f060 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81aeaf0 con 0x5608df5aa240 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdcf0353740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81ad680 con 0x5608df59f980 2024-09-05T07:04:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44004/0 (socket says 172.21.15.86:44004) 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce77fe640 1 -- 172.21.15.86:0/2079377323 learned_addr learned my addr 172.21.15.86:0/2079377323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 0x5608df5c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df59a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 -- 172.21.15.86:0/2079377323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 msgr2=0x5608df59a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df59a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 -- 172.21.15.86:0/2079377323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 msgr2=0x5608df5a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce7fff640 1 -- 172.21.15.86:0/2079377323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608df599bb0 con 0x5608df5aa240 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.517+0000 7fdce77fe640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.518+0000 7fdce7fff640 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 0x5608df5c1b00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011e90 tx=0x7fdce000a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ef69e3626558b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.518+0000 7fdccd7fa640 1 -- 172.21.15.86:0/2079377323 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0037070 con 0x5608df5aa240 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.518+0000 7fdccd7fa640 1 -- 172.21.15.86:0/2079377323 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce0028c70 con 0x5608df5aa240 2024-09-05T07:04:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.518+0000 7fdccd7fa640 1 -- 172.21.15.86:0/2079377323 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce001da00 con 0x5608df5aa240 2024-09-05T07:04:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- 172.21.15.86:0/2079377323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 msgr2=0x5608df5c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 0x5608df5c1b00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011e90 tx=0x7fdce000a400 comp rx=0 tx=0).stop 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- 172.21.15.86:0/2079377323 shutdown_connections 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df5aa240 0x5608df5c1b00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df59a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 --2- 172.21.15.86:0/2079377323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df59f060 0x5608df5a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- 172.21.15.86:0/2079377323 >> 172.21.15.86:0/2079377323 conn(0x5608df59b5f0 msgr2=0x5608df599f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- 172.21.15.86:0/2079377323 shutdown_connections 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- 172.21.15.86:0/2079377323 wait complete. 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 -- start start 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.519+0000 7fdcf0353740 1 Processor -- start 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdcf0353740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df61db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdce7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df61db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42216/0 (socket says 172.21.15.86:42216) 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.520+0000 7fdce77fe640 1 -- 172.21.15.86:0/1142431591 learned_addr learned my addr 172.21.15.86:0/1142431591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81ae420 con 0x5608df5aa240 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81add50 con 0x5608df59f060 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81aafb0 con 0x5608df59f980 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcf0353740 10 client.0 _ll_register_callbacks cb 0x7fdcdc09b170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 -- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 msgr2=0x5608df5ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 -- 172.21.15.86:0/1142431591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 msgr2=0x5608df61db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df61db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608df61e180 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce77fe640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdce6ffd640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5aed50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0995a0 tx=0x7fdcdc0979d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b436de002b91905 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0b9060 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608df596950 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.521+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fdcdc0cc6f0 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdcdc020070 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 10 client.29275 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5608df625540 con 0x5608df59f980 2024-09-05T07:04:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5608df596950 con 0x5608df59f980 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fdcdc0ccbc0 con 0x5608df59f980 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 10 client.29275 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 10 client.29275 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 20 client.29275 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df625b80 0x5608df627fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:30.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 4 client.29275 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e131ad72", "release": true, "await": true}] 2024-09-05T07:04:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.522+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e131ad72", "release": true, "await": true}) -- 0x5608df50c0a0 con 0x5608df625b80 2024-09-05T07:04:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.523+0000 7fdce77fe640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df625b80 0x5608df627fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.523+0000 7fdce77fe640 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df625b80 0x5608df627fa0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ee6d31a6cfae14 server_cookie=55077623659ef6e in_seq=0 out_seq=0 2024-09-05T07:04:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.523+0000 7fdcc6ffd640 10 client.29275 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.555+0000 7fdcc6ffd640 1 -- 172.21.15.86:0/1142431591 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5608df50c0a0 con 0x5608df625b80 2024-09-05T07:04:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.555+0000 7fdcc6ffd640 10 client.29275 handle_command_reply: tid=0 2024-09-05T07:04:30.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.555+0000 7fdcf0353740 1 client.29275 shutdown 2024-09-05T07:04:30.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.556+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 msgr2=0x5608df5aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.556+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5aed50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0995a0 tx=0x7fdcdc0979d0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 shutdown_connections 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5608df625b80 0x5608df627fa0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcc6ffd640 0 client.29275 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608df5aa240 0x5608df61db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5608df59f980 0x5608df5aed50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 --2- 172.21.15.86:0/1142431591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5608df59f060 0x5608df5ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 >> 172.21.15.86:0/1142431591 conn(0x5608df59b5f0 msgr2=0x5608df5a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 shutdown_connections 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.557+0000 7fdcf0353740 1 -- 172.21.15.86:0/1142431591 wait complete. 2024-09-05T07:04:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.558+0000 7fdcf0353740 20 client.29275 trim_cache size 0 max 16384 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 46, 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 88, 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "e131ad72": { 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 88, 2024-09-05T07:04:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 150.0, 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:04:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:30.561 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 msgr2=0x7fdcd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 0x7fdcd00a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdce0025000 tx=0x7fdce0026000 comp rx=0 tx=0).stop 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 msgr2=0x7fdce8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00d750 tx=0x7fdcdc00dc10 comp rx=0 tx=0).stop 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 shutdown_connections 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fdcd00a2cf0 0x7fdcd00a51b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce810c170 0x7fdce819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce8103660 0x7fdce8198920 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 --2- 172.21.15.86:0/4125631677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdce8102d40 0x7fdce81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.560+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 >> 172.21.15.86:0/4125631677 conn(0x7fdce80fc7a0 msgr2=0x7fdce80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.561+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 shutdown_connections 2024-09-05T07:04:30.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:30.561+0000 7fdcedf7a640 1 -- 172.21.15.86:0/4125631677 wait complete. 2024-09-05T07:04:30.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66499 5.90 MB/sec execute 468 sec latency 31431.146 ms 2024-09-05T07:04:31.514 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e131ad72', total seconds elapsed: 36.0 2024-09-05T07:04:31.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:31 smithi086 ceph-mon[30943]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-05T07:04:31.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:31 smithi115 ceph-mon[34791]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 -- 172.21.15.86:0/3322055504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102dd0 msgr2=0x7fb8241031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/3322055504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102dd0 0x7fb8241031b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb820009930 tx=0x7fb82002a420 comp rx=0 tx=0).stop 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 -- 172.21.15.86:0/3322055504 shutdown_connections 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/3322055504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824108300 0x7fb82410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/3322055504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/3322055504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102dd0 0x7fb8241031b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 -- 172.21.15.86:0/3322055504 >> 172.21.15.86:0/3322055504 conn(0x7fb8240fc7d0 msgr2=0x7fb8240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.742+0000 7fb82b3a4640 1 -- 172.21.15.86:0/3322055504 shutdown_connections 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.743+0000 7fb82b3a4640 1 -- 172.21.15.86:0/3322055504 wait complete. 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 Processor -- start 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 -- start start 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824102dd0 0x7fb824198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111f20 con 0x7fb824102dd0 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111da0 con 0x7fb824108300 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.744+0000 7fb82b3a4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241120a0 con 0x7fb824103780 2024-09-05T07:04:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42244/0 (socket says 172.21.15.86:42244) 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb82893f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 -- 172.21.15.86:0/1871952218 learned_addr learned my addr 172.21.15.86:0/1871952218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 -- 172.21.15.86:0/1871952218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 msgr2=0x7fb824198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 -- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824102dd0 msgr2=0x7fb824198540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824102dd0 0x7fb824198540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 -- 172.21.15.86:0/1871952218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82419d4f0 con 0x7fb824108300 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb82893f640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.745+0000 7fb829941640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00efd0 tx=0x7fb81c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.746+0000 7fb81a7fc640 1 -- 172.21.15.86:0/1871952218 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003b70 con 0x7fb824108300 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.746+0000 7fb81a7fc640 1 -- 172.21.15.86:0/1871952218 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb81c003d10 con 0x7fb824108300 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.746+0000 7fb81a7fc640 1 -- 172.21.15.86:0/1871952218 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c010980 con 0x7fb824108300 2024-09-05T07:04:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.747+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82419d7e0 con 0x7fb824108300 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.747+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb824100960 con 0x7fb824108300 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.750+0000 7fb81a7fc640 1 -- 172.21.15.86:0/1871952218 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb81c010b20 con 0x7fb824108300 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 Processor -- start 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 -- start start 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc43357090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 0x55bc4336cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc432bd2f0 con 0x55bc433575d0 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc43264cf0 con 0x55bc4336ab80 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82d77d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc432b92b0 con 0x55bc43358b40 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb81a7fc640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 0x7fb8100a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb829140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb829140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44024/0 (socket says 172.21.15.86:44024) 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb829140640 1 -- 172.21.15.86:0/3318911113 learned_addr learned my addr 172.21.15.86:0/3318911113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82893f640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc43357090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb829941640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 0x7fb8100a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb82893f640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 0x55bc4336cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.751+0000 7fb829140640 1 -- 172.21.15.86:0/3318911113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 msgr2=0x55bc43357090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb829140640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc43357090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb829140640 1 -- 172.21.15.86:0/3318911113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 msgr2=0x55bc4336cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb829140640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 0x55bc4336cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb829140640 1 -- 172.21.15.86:0/3318911113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc432493a0 con 0x55bc433575d0 2024-09-05T07:04:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb82893f640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 0x55bc4336cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb81a7fc640 1 -- 172.21.15.86:0/1871952218 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb81c0cfcd0 con 0x7fb824108300 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb829140640 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb820005940 tx=0x7fb8200095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52452fd12faced7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb80f7fe640 1 -- 172.21.15.86:0/3318911113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820040020 con 0x55bc433575d0 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb80f7fe640 1 -- 172.21.15.86:0/3318911113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb820032e50 con 0x55bc433575d0 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.752+0000 7fb80f7fe640 1 -- 172.21.15.86:0/3318911113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035040 con 0x55bc433575d0 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 -- 172.21.15.86:0/3318911113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 msgr2=0x55bc43357a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb829941640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 0x7fb8100a52b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0981b0 tx=0x7fb81c097940 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb820005940 tx=0x7fb8200095a0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 -- 172.21.15.86:0/3318911113 shutdown_connections 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc4336ab80 0x55bc4336cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc433575d0 0x55bc43357a10 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3318911113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc43357090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 -- 172.21.15.86:0/3318911113 >> 172.21.15.86:0/3318911113 conn(0x55bc43320310 msgr2=0x55bc43353c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 -- 172.21.15.86:0/3318911113 shutdown_connections 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.753+0000 7fb82d77d740 1 -- 172.21.15.86:0/3318911113 wait complete. 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 -- start start 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 Processor -- start 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc4336a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc433678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 0x55bc43368050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc4336d820 con 0x55bc4336ab80 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc43278060 con 0x55bc433575d0 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc4336d9a0 con 0x55bc43358b40 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.754+0000 7fb82d77d740 10 client.0 _ll_register_callbacks cb 0x55bc42fcb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb82893f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 0x55bc43368050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb82893f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 0x55bc43368050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44036/0 (socket says 172.21.15.86:44036) 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb82893f640 1 -- 172.21.15.86:0/685021753 learned_addr learned my addr 172.21.15.86:0/685021753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc433678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb82893f640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc4336a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 -- 172.21.15.86:0/685021753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 msgr2=0x55bc4336a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc4336a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 -- 172.21.15.86:0/685021753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 msgr2=0x55bc43368050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 0x55bc43368050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 -- 172.21.15.86:0/685021753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4302e3c0 con 0x55bc43358b40 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb829140640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc433678c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb820009a20 tx=0x7fb820005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8470e5c40e69884 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb80cff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.755+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035040 con 0x55bc43358b40 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb8200316a0 con 0x55bc43358b40 2024-09-05T07:04:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb820032e50 con 0x55bc43358b40 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb820057050 con 0x55bc43358b40 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 10 client.29287 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bc4336d820 con 0x55bc43358b40 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb820032e50 con 0x55bc43358b40 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fb8200049a0 con 0x55bc43358b40 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 10 client.29287 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 10 client.29287 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 20 client.29287 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc4337a460 0x55bc4337c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 4 client.29287 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.756+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc432ca860 con 0x55bc4337a460 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.757+0000 7fb82893f640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc4337a460 0x55bc4337c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.757+0000 7fb82893f640 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc4337a460 0x55bc4337c850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5dccb806df95290 server_cookie=8b877f46ab514c6 in_seq=0 out_seq=0 2024-09-05T07:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.757+0000 7fb80cff9640 10 client.29287 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb80cff9640 1 -- 172.21.15.86:0/685021753 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc432ca860 con 0x55bc4337a460 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb80cff9640 10 client.29287 handle_command_reply: tid=0 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 client.29287 shutdown 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 msgr2=0x55bc433678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc433678c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb820009a20 tx=0x7fb820005360 comp rx=0 tx=0).stop 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 shutdown_connections 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc4337a460 0x55bc4337c850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc4336ab80 0x55bc43368050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc43358b40 0x55bc433678c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 --2- 172.21.15.86:0/685021753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc4336a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 >> 172.21.15.86:0/685021753 conn(0x55bc43320310 msgr2=0x55bc43359990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.758+0000 7fb80cff9640 0 client.29287 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.759+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 shutdown_connections 2024-09-05T07:04:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.759+0000 7fb82d77d740 1 -- 172.21.15.86:0/685021753 wait complete. 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.759+0000 7fb82d77d740 20 client.29287 trim_cache size 0 max 16384 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 Processor -- start 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 -- start start 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc4336bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc4336c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ad030 con 0x55bc43358b40 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241aeb70 con 0x55bc433575d0 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82d77d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ad700 con 0x55bc4336ab80 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82893f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82893f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42266/0 (socket says 172.21.15.86:42266) 2024-09-05T07:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82893f640 1 -- 172.21.15.86:0/535185486 learned_addr learned my addr 172.21.15.86:0/535185486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb82893f640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc4336c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.779+0000 7fb829140640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc4336bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 -- 172.21.15.86:0/535185486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 msgr2=0x55bc4336c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc4336c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 -- 172.21.15.86:0/535185486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 msgr2=0x55bc4336bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc4336bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 -- 172.21.15.86:0/535185486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43354960 con 0x55bc433575d0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb829140640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc4336bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb82893f640 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb8140106f0 tx=0x7fb814004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27decdaa89a8ade server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb80effd640 1 -- 172.21.15.86:0/535185486 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814004250 con 0x55bc433575d0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb80effd640 1 -- 172.21.15.86:0/535185486 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8140043f0 con 0x55bc433575d0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.780+0000 7fb80effd640 1 -- 172.21.15.86:0/535185486 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814004bc0 con 0x55bc433575d0 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 -- 172.21.15.86:0/535185486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 msgr2=0x55bc43367890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb8140106f0 tx=0x7fb814004730 comp rx=0 tx=0).stop 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 -- 172.21.15.86:0/535185486 shutdown_connections 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc4336c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc4336bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 --2- 172.21.15.86:0/535185486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43367890 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 -- 172.21.15.86:0/535185486 >> 172.21.15.86:0/535185486 conn(0x55bc43320310 msgr2=0x55bc43367180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.781+0000 7fb82d77d740 1 -- 172.21.15.86:0/535185486 shutdown_connections 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.782+0000 7fb82d77d740 1 -- 172.21.15.86:0/535185486 wait complete. 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.782+0000 7fb82d77d740 1 -- start start 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.782+0000 7fb82d77d740 1 Processor -- start 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc43364b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc43365080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ae4a0 con 0x55bc43358b40 2024-09-05T07:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241addd0 con 0x55bc433575d0 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ab030 con 0x55bc4336ab80 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82893f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc43364b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82893f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:42276/0 (socket says 172.21.15.86:42276) 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82893f640 1 -- 172.21.15.86:0/3050646218 learned_addr learned my addr 172.21.15.86:0/3050646218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82d77d740 10 client.0 _ll_register_callbacks cb 0x7fb814010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82893f640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc43365080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66656 5.91 MB/sec execute 469 sec latency 132.835 ms 2024-09-05T07:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 -- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 msgr2=0x55bc43365080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc43365080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 -- 172.21.15.86:0/3050646218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 msgr2=0x55bc43373110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb829140640 1 -- 172.21.15.86:0/3050646218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc43365670 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.783+0000 7fb82893f640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb829140640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc43364b40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fb820009a50 tx=0x7fb820009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e21a8d9af58355 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035070 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb8200314a0 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8140043f0 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb820064050 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb82d77d740 10 client.20598 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bc43382380 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8140043f0 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.784+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fb82004e470 con 0x55bc43358b40 2024-09-05T07:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 10 client.20598 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 10 client.20598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 20 client.20598 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc433e53d0 0x55bc433e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 4 client.20598 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc432c4d70 con 0x55bc433e53d0 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82893f640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc433e53d0 0x55bc433e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb82893f640 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc433e53d0 0x55bc433e77f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fc137102d07cee8 server_cookie=7bb0282365778be1 in_seq=0 out_seq=0 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.785+0000 7fb80cff9640 10 client.20598 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb80cff9640 1 -- 172.21.15.86:0/3050646218 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc432c4d70 con 0x55bc433e53d0 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb80cff9640 10 client.20598 handle_command_reply: tid=0 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 client.20598 shutdown 2024-09-05T07:04:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 msgr2=0x55bc43364b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc43364b40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fb820009a50 tx=0x7fb820009520 comp rx=0 tx=0).stop 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 shutdown_connections 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55bc433e53d0 0x55bc433e77f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc4336ab80 0x55bc43365080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc43358b40 0x55bc43364b40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb80cff9640 0 client.20598 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 --2- 172.21.15.86:0/3050646218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc433575d0 0x55bc43373110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 >> 172.21.15.86:0/3050646218 conn(0x55bc43320310 msgr2=0x55bc43359110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 shutdown_connections 2024-09-05T07:04:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 1 -- 172.21.15.86:0/3050646218 wait complete. 2024-09-05T07:04:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.786+0000 7fb82d77d740 20 client.20598 trim_cache size 0 max 16384 2024-09-05T07:04:31.803 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:04:31.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 msgr2=0x7fb8100a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 0x7fb8100a52b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0981b0 tx=0x7fb81c097940 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 msgr2=0x7fb82419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00efd0 tx=0x7fb81c00c6a0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 shutdown_connections 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb8100a2df0 0x7fb8100a52b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824108300 0x7fb82419ce10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb824103780 0x7fb824198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 --2- 172.21.15.86:0/1871952218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824102dd0 0x7fb824198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.787+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 >> 172.21.15.86:0/1871952218 conn(0x7fb8240fc7d0 msgr2=0x7fb824105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.788+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 shutdown_connections 2024-09-05T07:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:31.788+0000 7fb82b3a4640 1 -- 172.21.15.86:0/1871952218 wait complete. 2024-09-05T07:04:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:31 smithi096 ceph-mon[34468]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-05T07:04:32.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:04:32.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66818 5.92 MB/sec execute 470 sec latency 68.792 ms 2024-09-05T07:04:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.632+0000 7f60379c8640 1 -- 172.21.15.86:0/2984286754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603010c160 msgr2=0x7f603010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.632+0000 7f60379c8640 1 --2- 172.21.15.86:0/2984286754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603010c160 0x7f603010e600 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6028009ca0 tx=0x7f602802a950 comp rx=0 tx=0).stop 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 -- 172.21.15.86:0/2984286754 shutdown_connections 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 --2- 172.21.15.86:0/2984286754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603010c160 0x7f603010e600 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 --2- 172.21.15.86:0/2984286754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6030103650 0x7f603010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 --2- 172.21.15.86:0/2984286754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030102d30 0x7f6030103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 -- 172.21.15.86:0/2984286754 >> 172.21.15.86:0/2984286754 conn(0x7f60300fc790 msgr2=0x7f60300febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 -- 172.21.15.86:0/2984286754 shutdown_connections 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.633+0000 7f60379c8640 1 -- 172.21.15.86:0/2984286754 wait complete. 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 Processor -- start 2024-09-05T07:04:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 -- start start 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030103650 0x7f60301040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603010c160 0x7f6030104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111e10 con 0x7f6030102d30 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111c90 con 0x7f603010c160 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.634+0000 7f60379c8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111f90 con 0x7f6030103650 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48742/0 (socket says 172.21.15.86:48742) 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6034f63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030103650 0x7f60301040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 -- 172.21.15.86:0/550010462 learned_addr learned my addr 172.21.15.86:0/550010462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 -- 172.21.15.86:0/550010462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030103650 msgr2=0x7f60301040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030103650 0x7f60301040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 -- 172.21.15.86:0/550010462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603010c160 msgr2=0x7f6030104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603010c160 0x7f6030104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 -- 172.21.15.86:0/550010462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301aabf0 con 0x7f6030102d30 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.635+0000 7f6035764640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f602c00ea20 tx=0x7f602c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.636+0000 7f60267fc640 1 -- 172.21.15.86:0/550010462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c00cd80 con 0x7f6030102d30 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.636+0000 7f60267fc640 1 -- 172.21.15.86:0/550010462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f602c0054d0 con 0x7f6030102d30 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.636+0000 7f60267fc640 1 -- 172.21.15.86:0/550010462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c010430 con 0x7f6030102d30 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.636+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301aaee0 con 0x7f6030102d30 2024-09-05T07:04:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.636+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301ab370 con 0x7f6030102d30 2024-09-05T07:04:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.639+0000 7f60267fc640 1 -- 172.21.15.86:0/550010462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f602c005020 con 0x7f6030102d30 2024-09-05T07:04:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f60267fc640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 0x7f601c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 Processor -- start 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 -- start start 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f60267fc640 1 -- 172.21.15.86:0/550010462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f602c0cfb70 con 0x7f6030102d30 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6034f63640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 0x7f601c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23512a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec2351d370 0x55ec2351f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec234782f0 con 0x55ec23513b40 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec2341fcf0 con 0x55ec2351d370 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6039da1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ec234742b0 con 0x55ec235125d0 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6035764640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6035764640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48756/0 (socket says 172.21.15.86:48756) 2024-09-05T07:04:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6035764640 1 -- 172.21.15.86:0/4199684137 learned_addr learned my addr 172.21.15.86:0/4199684137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.640+0000 7f6034f63640 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23512a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6034f63640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 0x7f601c0a5300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6030105370 tx=0x7f6020005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 -- 172.21.15.86:0/4199684137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 msgr2=0x55ec23512a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23512a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 -- 172.21.15.86:0/4199684137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec2351d370 msgr2=0x55ec2351f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec2351d370 0x55ec2351f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 -- 172.21.15.86:0/4199684137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec234043a0 con 0x55ec23513b40 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f6035764640 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f602c098620 tx=0x7f602c005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8140fe77a0591d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f601b7fe640 1 -- 172.21.15.86:0/4199684137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0b1070 con 0x55ec23513b40 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f601b7fe640 1 -- 172.21.15.86:0/4199684137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f602c0b4e50 con 0x55ec23513b40 2024-09-05T07:04:33.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.641+0000 7f601b7fe640 1 -- 172.21.15.86:0/4199684137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0982c0 con 0x55ec23513b40 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 -- 172.21.15.86:0/4199684137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 msgr2=0x55ec23512090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f602c098620 tx=0x7f602c005310 comp rx=0 tx=0).stop 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 -- 172.21.15.86:0/4199684137 shutdown_connections 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec2351d370 0x55ec2351f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23512a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 --2- 172.21.15.86:0/4199684137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23512090 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 -- 172.21.15.86:0/4199684137 >> 172.21.15.86:0/4199684137 conn(0x55ec234db310 msgr2=0x55ec2350ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 -- 172.21.15.86:0/4199684137 shutdown_connections 2024-09-05T07:04:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.642+0000 7f6039da1740 1 -- 172.21.15.86:0/4199684137 wait complete. 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.643+0000 7f6039da1740 1 -- start start 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.643+0000 7f6039da1740 1 Processor -- start 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.643+0000 7f6039da1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec2351ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.643+0000 7f6039da1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.643+0000 7f6039da1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec2351a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6039da1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec23520070 con 0x55ec2351d370 2024-09-05T07:04:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6039da1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec23433060 con 0x55ec235125d0 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6039da1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ec235201f0 con 0x55ec23513b40 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6039da1740 10 client.0 _ll_register_callbacks cb 0x55ec231868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56932/0 (socket says 172.21.15.86:56932) 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 -- 172.21.15.86:0/2504004187 learned_addr learned my addr 172.21.15.86:0/2504004187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6035764640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec2351ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6035f65640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec2351a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 -- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 msgr2=0x55ec2351ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec2351ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 -- 172.21.15.86:0/2504004187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 msgr2=0x55ec2351a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec2351a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6034f63640 1 -- 172.21.15.86:0/2504004187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec231e93c0 con 0x55ec23513b40 2024-09-05T07:04:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.644+0000 7f6035f65640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec2351a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6034f63640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f602001ca40 tx=0x7f6020004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9c95a36fd8bb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602002a060 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c0b4e50 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f602001bba0 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f602001bda0 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6039da1740 10 client.29305 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ec2352bb30 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.645+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f602c0b4e50 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f6020024070 con 0x55ec23513b40 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 10 client.29305 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 10 client.29305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:33.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 20 client.29305 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec2352d380 0x55ec2352f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 4 client.29305 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec23485860 con 0x55ec2352d380 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6035764640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec2352d380 0x55ec2352f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6035764640 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec2352d380 0x55ec2352f7a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93bf5c73a1df3f5 server_cookie=dcd8da394e5df425 in_seq=0 out_seq=0 2024-09-05T07:04:33.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.646+0000 7f6018ff9640 10 client.29305 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.647+0000 7f6018ff9640 1 -- 172.21.15.86:0/2504004187 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec23485860 con 0x55ec2352d380 2024-09-05T07:04:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.647+0000 7f6018ff9640 10 client.29305 handle_command_reply: tid=0 2024-09-05T07:04:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.647+0000 7f6039da1740 1 client.29305 shutdown 2024-09-05T07:04:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.647+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 msgr2=0x55ec23519f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.647+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f602001ca40 tx=0x7f6020004730 comp rx=0 tx=0).stop 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 shutdown_connections 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec2352d380 0x55ec2352f7a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec2351a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec23513b40 0x55ec23519f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 --2- 172.21.15.86:0/2504004187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec2351ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 >> 172.21.15.86:0/2504004187 conn(0x55ec234db310 msgr2=0x55ec23514960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6018ff9640 0 client.29305 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 shutdown_connections 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 1 -- 172.21.15.86:0/2504004187 wait complete. 2024-09-05T07:04:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.648+0000 7f6039da1740 20 client.29305 trim_cache size 0 max 16384 2024-09-05T07:04:33.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 Processor -- start 2024-09-05T07:04:33.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 -- start start 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 0x55ec23519840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec23518850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b5820 con 0x55ec2351d370 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b7360 con 0x55ec23513b40 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6039da1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b5ef0 con 0x55ec235125d0 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6035764640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6034f63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 0x55ec23519840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6034f63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 0x55ec23519840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38054/0 (socket says 172.21.15.86:38054) 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6035764640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56946/0 (socket says 172.21.15.86:56946) 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6034f63640 1 -- 172.21.15.86:0/2018192273 learned_addr learned my addr 172.21.15.86:0/2018192273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6035f65640 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec23518850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.668+0000 7f6035764640 1 -- 172.21.15.86:0/2018192273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 msgr2=0x55ec23519840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f6035764640 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 0x55ec23519840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f6035764640 1 -- 172.21.15.86:0/2018192273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 msgr2=0x55ec23518850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f6035764640 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec23518850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f6035764640 1 -- 172.21.15.86:0/2018192273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2350f960 con 0x55ec235125d0 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f6035764640 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f602c09b950 tx=0x7f602c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8910b5392f370e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f601affd640 1 -- 172.21.15.86:0/2018192273 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0b1040 con 0x55ec235125d0 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f601affd640 1 -- 172.21.15.86:0/2018192273 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f602c0b4e50 con 0x55ec235125d0 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.669+0000 7f601affd640 1 -- 172.21.15.86:0/2018192273 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0982c0 con 0x55ec235125d0 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- 172.21.15.86:0/2018192273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 msgr2=0x55ec23529310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f602c09b950 tx=0x7f602c0097b0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- 172.21.15.86:0/2018192273 shutdown_connections 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec2351d370 0x55ec23518850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec23513b40 0x55ec23519840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 --2- 172.21.15.86:0/2018192273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec235125d0 0x55ec23529310 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- 172.21.15.86:0/2018192273 >> 172.21.15.86:0/2018192273 conn(0x55ec234db310 msgr2=0x55ec23519f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- 172.21.15.86:0/2018192273 shutdown_connections 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- 172.21.15.86:0/2018192273 wait complete. 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.670+0000 7f6039da1740 1 -- start start 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 Processor -- start 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23520980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2351d370 0x55ec23520ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b6c90 con 0x55ec23513b40 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b65c0 con 0x55ec235125d0 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b3820 con 0x55ec2351d370 2024-09-05T07:04:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6039da1740 10 client.0 _ll_register_callbacks cb 0x7f602001c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6035764640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6035764640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38058/0 (socket says 172.21.15.86:38058) 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6035764640 1 -- 172.21.15.86:0/3556647417 learned_addr learned my addr 172.21.15.86:0/3556647417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6034f63640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23520980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.671+0000 7f6035f65640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2351d370 0x55ec23520ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 -- 172.21.15.86:0/3556647417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2351d370 msgr2=0x55ec23520ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2351d370 0x55ec23520ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 -- 172.21.15.86:0/3556647417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 msgr2=0x55ec23520980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23520980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 -- 172.21.15.86:0/3556647417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec23596070 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6035764640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f602c09bbe0 tx=0x7f602c09bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79457b5b9c37813c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0b1070 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c0b4e50 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f602c0cd100 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f602c0a48c0 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6039da1740 10 client.29258 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.672+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ec23597370 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f602c0b4e50 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f602c0a4b10 con 0x55ec235125d0 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 10 client.29258 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:33.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 10 client.29258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 20 client.29258 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec23597600 0x55ec235999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 4 client.29258 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:04:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec2347fd70 con 0x55ec23597600 2024-09-05T07:04:33.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.673+0000 7f6034f63640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec23597600 0x55ec235999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:33.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6018ff9640 10 client.29258 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:33.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6034f63640 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec23597600 0x55ec235999f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3340dd2981ece25e server_cookie=3fc0fccf2f28072d in_seq=0 out_seq=0 2024-09-05T07:04:33.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6018ff9640 1 -- 172.21.15.86:0/3556647417 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec2347fd70 con 0x55ec23597600 2024-09-05T07:04:33.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6018ff9640 10 client.29258 handle_command_reply: tid=0 2024-09-05T07:04:33.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 client.29258 shutdown 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 msgr2=0x55ec23520440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f602c09bbe0 tx=0x7f602c09bcd0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 shutdown_connections 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ec23597600 0x55ec235999f0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ec2351d370 0x55ec23520ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec23513b40 0x55ec23520980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 --2- 172.21.15.86:0/3556647417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec235125d0 0x55ec23520440 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 >> 172.21.15.86:0/3556647417 conn(0x55ec234db310 msgr2=0x55ec23519f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6018ff9640 0 client.29258 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:33.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.674+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 shutdown_connections 2024-09-05T07:04:33.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f6039da1740 1 -- 172.21.15.86:0/3556647417 wait complete. 2024-09-05T07:04:33.683 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f6039da1740 20 client.29258 trim_cache size 0 max 16384 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 msgr2=0x7f601c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 0x7f601c0a5300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6030105370 tx=0x7f6020005f90 comp rx=0 tx=0).stop 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 msgr2=0x7f6030103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f602c00ea20 tx=0x7f602c00ee20 comp rx=0 tx=0).stop 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 shutdown_connections 2024-09-05T07:04:33.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f601c0a2e40 0x7f601c0a5300 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603010c160 0x7f6030104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6030103650 0x7f60301040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 --2- 172.21.15.86:0/550010462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6030102d30 0x7f6030103bb0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.675+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 >> 172.21.15.86:0/550010462 conn(0x7f60300fc790 msgr2=0x7f6030101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.676+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 shutdown_connections 2024-09-05T07:04:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:33.676+0000 7f60379c8640 1 -- 172.21.15.86:0/550010462 wait complete. 2024-09-05T07:04:33.689 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:33 smithi086 ceph-mon[30943]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 482 KiB/s wr, 30 op/s 2024-09-05T07:04:33.783 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66986 5.92 MB/sec execute 471 sec latency 111.816 ms 2024-09-05T07:04:33.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:33 smithi096 ceph-mon[34468]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 482 KiB/s wr, 30 op/s 2024-09-05T07:04:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:33 smithi115 ceph-mon[34791]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 482 KiB/s wr, 30 op/s 2024-09-05T07:04:34.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67136 5.92 MB/sec execute 472 sec latency 80.567 ms 2024-09-05T07:04:35.528 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:04:35.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:04:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:35 smithi086 ceph-mon[30943]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 570 KiB/s wr, 30 op/s 2024-09-05T07:04:35.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67294 5.93 MB/sec execute 473 sec latency 76.400 ms 2024-09-05T07:04:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:35 smithi096 ceph-mon[34468]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 570 KiB/s wr, 30 op/s 2024-09-05T07:04:35.877 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:35.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:35 smithi115 ceph-mon[34791]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 570 KiB/s wr, 30 op/s 2024-09-05T07:04:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.487+0000 7fedfd50d640 1 -- 172.21.15.86:0/3449149403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf80ffe60 msgr2=0x7fedf8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.487+0000 7fedfd50d640 1 --2- 172.21.15.86:0/3449149403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf80ffe60 0x7fedf8100240 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fede8009930 tx=0x7fede802a4d0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 -- 172.21.15.86:0/3449149403 shutdown_connections 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 --2- 172.21.15.86:0/3449149403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf810a2f0 0x7fedf8110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 --2- 172.21.15.86:0/3449149403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf8100780 0x7fedf8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 --2- 172.21.15.86:0/3449149403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf80ffe60 0x7fedf8100240 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 -- 172.21.15.86:0/3449149403 >> 172.21.15.86:0/3449149403 conn(0x7fedf80fba90 msgr2=0x7fedf80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 -- 172.21.15.86:0/3449149403 shutdown_connections 2024-09-05T07:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 -- 172.21.15.86:0/3449149403 wait complete. 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.488+0000 7fedfd50d640 1 Processor -- start 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 -- start start 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf8100780 0x7fedf819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf810a2f0 0x7fedf819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8113fc0 con 0x7fedf810a2f0 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8113e40 con 0x7fedf8100780 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedfd50d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8114140 con 0x7fedf80ffe60 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedf6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf810a2f0 0x7fedf819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedf6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56968/0 (socket says 172.21.15.86:56968) 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedf67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf8100780 0x7fedf819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.489+0000 7fedf6ffd640 1 -- 172.21.15.86:0/4048774702 learned_addr learned my addr 172.21.15.86:0/4048774702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 -- 172.21.15.86:0/4048774702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf8100780 msgr2=0x7fedf819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf8100780 0x7fedf819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 -- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf810a2f0 msgr2=0x7fedf819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf810a2f0 0x7fedf819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 -- 172.21.15.86:0/4048774702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf819f590 con 0x7fedf80ffe60 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fede8005940 tx=0x7fede80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7feddffff640 1 -- 172.21.15.86:0/4048774702 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede803f030 con 0x7fedf80ffe60 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7feddffff640 1 -- 172.21.15.86:0/4048774702 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fede80318a0 con 0x7fedf80ffe60 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf819f820 con 0x7fedf80ffe60 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.490+0000 7feddffff640 1 -- 172.21.15.86:0/4048774702 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8035070 con 0x7fedf80ffe60 2024-09-05T07:04:36.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.491+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf81a7160 con 0x7fedf80ffe60 2024-09-05T07:04:36.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.494+0000 7fedff8e6740 1 Processor -- start 2024-09-05T07:04:36.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.494+0000 7fedff8e6740 1 -- start start 2024-09-05T07:04:36.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.494+0000 7feddffff640 1 -- 172.21.15.86:0/4048774702 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fede8042030 con 0x7fedf80ffe60 2024-09-05T07:04:36.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.494+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.494+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b053846800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:56978/0 (socket says 172.21.15.86:56978) 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053849130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedff8e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0537a1280 con 0x55b05383c480 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedff8e6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b053748b10 con 0x55b053846d40 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedff8e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b05379d240 con 0x55b05383bb60 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedf77fe640 1 -- 172.21.15.86:0/4249913760 learned_addr learned my addr 172.21.15.86:0/4249913760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.495+0000 7fedf67fc640 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053849130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b053846800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 -- 172.21.15.86:0/4249913760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 msgr2=0x55b053849130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053849130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 -- 172.21.15.86:0/4249913760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 msgr2=0x55b053846800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b053846800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 -- 172.21.15.86:0/4249913760 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0537551f0 con 0x55b05383bb60 2024-09-05T07:04:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7fedf77fe640 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009f90 tx=0x7fedf0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4659e525d63fbc19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7feddd7fa640 1 -- 172.21.15.86:0/4249913760 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0011700 con 0x55b05383bb60 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7feddd7fa640 1 -- 172.21.15.86:0/4249913760 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf0011d50 con 0x55b05383bb60 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.496+0000 7feddd7fa640 1 -- 172.21.15.86:0/4249913760 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0019db0 con 0x55b05383bb60 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.497+0000 7feddffff640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 0x7fede00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.497+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 0x7fede00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.497+0000 7fedff8e6740 1 -- 172.21.15.86:0/4249913760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 msgr2=0x55b05383bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.497+0000 7fedff8e6740 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009f90 tx=0x7fedf0010860 comp rx=0 tx=0).stop 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.497+0000 7feddffff640 1 -- 172.21.15.86:0/4048774702 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fede80efbf0 con 0x7fedf80ffe60 2024-09-05T07:04:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 -- 172.21.15.86:0/4249913760 shutdown_connections 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053849130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b053846800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 --2- 172.21.15.86:0/4249913760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383bb60 0x55b05383bf40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 -- 172.21.15.86:0/4249913760 >> 172.21.15.86:0/4249913760 conn(0x55b0538380f0 msgr2=0x55b0538384d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedf6ffd640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 0x7fede00a52b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b053838f50 tx=0x7fede802a4d0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 -- 172.21.15.86:0/4249913760 shutdown_connections 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 -- 172.21.15.86:0/4249913760 wait complete. 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 -- start start 2024-09-05T07:04:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.498+0000 7fedff8e6740 1 Processor -- start 2024-09-05T07:04:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053843b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b0538440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b0538445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b053849910 con 0x55b05383c480 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b05375bf10 con 0x55b05383bb60 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b053849a90 con 0x55b053846d40 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedff8e6740 10 client.0 _ll_register_callbacks cb 0x55b053724a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053843b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053843b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38096/0 (socket says 172.21.15.86:38096) 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 -- 172.21.15.86:0/1843918044 learned_addr learned my addr 172.21.15.86:0/1843918044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b0538445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf67fc640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b0538440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 -- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 msgr2=0x55b053843b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.499+0000 7fedf77fe640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053843b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7fedf77fe640 1 -- 172.21.15.86:0/1843918044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 msgr2=0x55b0538440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7fedf77fe640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b0538440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7fedf77fe640 1 -- 172.21.15.86:0/1843918044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0535126a0 con 0x55b053846d40 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7fedf77fe640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b0538445e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf0005fd0 tx=0x7fedf0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=201531e01ea6aef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7feddaff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0003b70 con 0x55b053846d40 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf0011d50 con 0x55b053846d40 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fedf0011500 con 0x55b053846d40 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.500+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fedf0034050 con 0x55b053846d40 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 10 client.29323 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b05385dca0 con 0x55b053846d40 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf0011d50 con 0x55b053846d40 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fedf001a8f0 con 0x55b053846d40 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 10 client.29323 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 10 client.29323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 20 client.29323 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b05385f0b0 0x55b0538614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 4 client.29323 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.501+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0537ae7f0 con 0x55b05385f0b0 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.502+0000 7fedf67fc640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b05385f0b0 0x55b0538614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.502+0000 7feddaff5640 10 client.29323 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.502+0000 7fedf67fc640 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b05385f0b0 0x55b0538614a0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0be68ed06bf6496 server_cookie=8daa1c557fae4e12 in_seq=0 out_seq=0 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7feddaff5640 1 -- 172.21.15.86:0/1843918044 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0537ae7f0 con 0x55b05385f0b0 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7feddaff5640 10 client.29323 handle_command_reply: tid=0 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 client.29323 shutdown 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 msgr2=0x55b0538445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b0538445e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf0005fd0 tx=0x7fedf0000f40 comp rx=0 tx=0).stop 2024-09-05T07:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 shutdown_connections 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b05385f0b0 0x55b0538614a0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b0538445e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b0538440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 --2- 172.21.15.86:0/1843918044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053843b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.503+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 >> 172.21.15.86:0/1843918044 conn(0x55b0538380f0 msgr2=0x55b05383de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.504+0000 7feddaff5640 0 client.29323 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.504+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 shutdown_connections 2024-09-05T07:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.504+0000 7fedff8e6740 1 -- 172.21.15.86:0/1843918044 wait complete. 2024-09-05T07:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.504+0000 7fedff8e6740 20 client.29323 trim_cache size 0 max 16384 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.528+0000 7fedff8e6740 1 Processor -- start 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.528+0000 7fedff8e6740 1 -- start start 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.528+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053844430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.528+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b05385ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b05385b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedff8e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81ad010 con 0x55b05383c480 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedff8e6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81aeb50 con 0x55b05383bb60 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedff8e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81ad6e0 con 0x55b053846d40 2024-09-05T07:04:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053844430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b05385ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053844430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38100/0 (socket says 172.21.15.86:38100) 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf77fe640 1 -- 172.21.15.86:0/486716025 learned_addr learned my addr 172.21.15.86:0/486716025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf77fe640 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b05385b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 -- 172.21.15.86:0/486716025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 msgr2=0x55b05385b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b05385b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 -- 172.21.15.86:0/486716025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 msgr2=0x55b053844430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053844430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 -- 172.21.15.86:0/486716025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b05383c2b0 con 0x55b05383c480 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf77fe640 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b05385b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.529+0000 7fedf67fc640 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b05385ab30 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x55b053861af0 tx=0x7fedec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8609cd61c5ba02 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.530+0000 7feddcff9640 1 -- 172.21.15.86:0/486716025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec022070 con 0x55b05383c480 2024-09-05T07:04:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.530+0000 7feddcff9640 1 -- 172.21.15.86:0/486716025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec0047a0 con 0x55b05383c480 2024-09-05T07:04:36.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.530+0000 7feddcff9640 1 -- 172.21.15.86:0/486716025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec01f690 con 0x55b05383c480 2024-09-05T07:04:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.530+0000 7fedff8e6740 1 -- 172.21.15.86:0/486716025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 msgr2=0x55b05385ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.530+0000 7fedff8e6740 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b05385ab30 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x55b053861af0 tx=0x7fedec005f90 comp rx=0 tx=0).stop 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 -- 172.21.15.86:0/486716025 shutdown_connections 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b053846d40 0x55b05385b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383c480 0x55b05385ab30 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 --2- 172.21.15.86:0/486716025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b05383bb60 0x55b053844430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 -- 172.21.15.86:0/486716025 >> 172.21.15.86:0/486716025 conn(0x55b0538380f0 msgr2=0x55b05385a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 -- 172.21.15.86:0/486716025 shutdown_connections 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 -- 172.21.15.86:0/486716025 wait complete. 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 -- start start 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.531+0000 7fedff8e6740 1 Processor -- start 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 0x55b053857790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 0x55b053857cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053852560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 0x55b053857790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81ae480 con 0x55b05383bb60 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81addb0 con 0x55b053846d40 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81ab010 con 0x55b05383c480 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 0x55b053857790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48830/0 (socket says 172.21.15.86:48830) 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf77fe640 1 -- 172.21.15.86:0/2430963356 learned_addr learned my addr 172.21.15.86:0/2430963356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 0x55b053857cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedff8e6740 10 client.0 _ll_register_callbacks cb 0x7fedf0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf77fe640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053852560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 -- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 msgr2=0x55b053852560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053852560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 -- 172.21.15.86:0/2430963356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 msgr2=0x55b053857790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 0x55b053857790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 -- 172.21.15.86:0/2430963356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b053852d60 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7feddaff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.532+0000 7fedf67fc640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 0x55b053857cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b053842290 tx=0x7fedec009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275386901c014143 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec035060 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedec0047a0 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fedec004b60 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fedec03c050 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7fedff8e6740 10 client.29329 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b0538ca510 con 0x55b05383c480 2024-09-05T07:04:36.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedec0047a0 con 0x55b05383c480 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.533+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fedec01f880 con 0x55b05383c480 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 10 client.29329 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 10 client.29329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 20 client.29329 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b0538cabc0 0x55b0538ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 4 client.29329 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0537a8d00 con 0x55b0538cabc0 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedf77fe640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b0538cabc0 0x55b0538ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7feddaff5640 10 client.29329 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.534+0000 7fedf77fe640 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b0538cabc0 0x55b0538ccfe0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e128e77104670ac server_cookie=e72869727fa7d1d8 in_seq=0 out_seq=0 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.535+0000 7feddaff5640 1 -- 172.21.15.86:0/2430963356 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0537a8d00 con 0x55b0538cabc0 2024-09-05T07:04:36.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.535+0000 7feddaff5640 10 client.29329 handle_command_reply: tid=0 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.535+0000 7fedff8e6740 1 client.29329 shutdown 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.535+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 msgr2=0x55b053857cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.535+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 0x55b053857cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b053842290 tx=0x7fedec009850 comp rx=0 tx=0).stop 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 shutdown_connections 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b0538cabc0 0x55b0538ccfe0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7feddaff5640 0 client.29329 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b053846d40 0x55b053852560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b05383c480 0x55b053857cd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 --2- 172.21.15.86:0/2430963356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b05383bb60 0x55b053857790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 >> 172.21.15.86:0/2430963356 conn(0x55b0538380f0 msgr2=0x55b05385ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 shutdown_connections 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 1 -- 172.21.15.86:0/2430963356 wait complete. 2024-09-05T07:04:36.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.536+0000 7fedff8e6740 20 client.29329 trim_cache size 0 max 16384 2024-09-05T07:04:36.540 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:04:36.540 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:04:36.540 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "35bcaa01-af53-4e1c-abba-4466c589d7c6", 2024-09-05T07:04:36.540 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:04:36.540 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 msgr2=0x7fede00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 0x7fede00a52b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b053838f50 tx=0x7fede802a4d0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 msgr2=0x7fedf819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fede8005940 tx=0x7fede80042f0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 shutdown_connections 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fede00a2df0 0x7fede00a52b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf810a2f0 0x7fedf819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedf8100780 0x7fedf819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 --2- 172.21.15.86:0/4048774702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedf80ffe60 0x7fedf819a5e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 >> 172.21.15.86:0/4048774702 conn(0x7fedf80fba90 msgr2=0x7fedf8107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 shutdown_connections 2024-09-05T07:04:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:36.537+0000 7fedfd50d640 1 -- 172.21.15.86:0/4048774702 wait complete. 2024-09-05T07:04:36.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67429 5.92 MB/sec execute 474 sec latency 119.280 ms 2024-09-05T07:04:37.740 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 569 KiB/s wr, 29 op/s 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:37.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:37 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67572 5.92 MB/sec execute 475 sec latency 223.677 ms 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 569 KiB/s wr, 29 op/s 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:37.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:37 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 569 KiB/s wr, 29 op/s 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:37.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:37.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:37 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:38 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:04:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:38 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:38 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67739 5.93 MB/sec execute 476 sec latency 60.515 ms 2024-09-05T07:04:38.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:38 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:04:38.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:38 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:38 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:38 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:04:38.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:38 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:38.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:38 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:04:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:39 smithi086 ceph-mon[30943]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-05T07:04:39.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67916 5.93 MB/sec execute 477 sec latency 70.458 ms 2024-09-05T07:04:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:39 smithi096 ceph-mon[34468]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-05T07:04:39.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:39 smithi115 ceph-mon[34791]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-05T07:04:40.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68054 5.94 MB/sec execute 478 sec latency 205.868 ms 2024-09-05T07:04:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:41 smithi086 ceph-mon[30943]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 133 op/s 2024-09-05T07:04:41.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68147 5.94 MB/sec execute 479 sec latency 453.369 ms 2024-09-05T07:04:41.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:41 smithi096 ceph-mon[34468]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 133 op/s 2024-09-05T07:04:41.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:41 smithi115 ceph-mon[34791]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 133 op/s 2024-09-05T07:04:42.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:04:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:04:42.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68334 5.95 MB/sec execute 480 sec latency 454.277 ms 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 -- 172.21.15.86:0/3228289108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c10c180 msgr2=0x7f617c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 --2- 172.21.15.86:0/3228289108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c10c180 0x7f617c10e620 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6174009ca0 tx=0x7f617402a950 comp rx=0 tx=0).stop 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 -- 172.21.15.86:0/3228289108 shutdown_connections 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 --2- 172.21.15.86:0/3228289108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c10c180 0x7f617c10e620 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 --2- 172.21.15.86:0/3228289108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c103670 0x7f617c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 --2- 172.21.15.86:0/3228289108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c102d50 0x7f617c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 -- 172.21.15.86:0/3228289108 >> 172.21.15.86:0/3228289108 conn(0x7f617c0fc7d0 msgr2=0x7f617c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 -- 172.21.15.86:0/3228289108 shutdown_connections 2024-09-05T07:04:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.234+0000 7f6182f1e640 1 -- 172.21.15.86:0/3228289108 wait complete. 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 Processor -- start 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 -- start start 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 0x7f617c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c10c180 0x7f617c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111df0 con 0x7f617c102d50 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111c70 con 0x7f617c103670 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.235+0000 7f6182f1e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111f70 con 0x7f617c10c180 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f6180cba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 0x7f617c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45134/0 (socket says 172.21.15.86:45134) 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f61814bb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c10c180 0x7f617c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 -- 172.21.15.86:0/2791340409 learned_addr learned my addr 172.21.15.86:0/2791340409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 -- 172.21.15.86:0/2791340409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c10c180 msgr2=0x7f617c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c10c180 0x7f617c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 -- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 msgr2=0x7f617c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 0x7f617c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 -- 172.21.15.86:0/2791340409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c19d2c0 con 0x7f617c103670 2024-09-05T07:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f6180cba640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 0x7f617c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f617bfff640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b70 tx=0x7f616800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f6179ffb640 1 -- 172.21.15.86:0/2791340409 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616801e070 con 0x7f617c103670 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f6179ffb640 1 -- 172.21.15.86:0/2791340409 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6168009c60 con 0x7f617c103670 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.236+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c19d5b0 con 0x7f617c103670 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.237+0000 7f6179ffb640 1 -- 172.21.15.86:0/2791340409 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168018670 con 0x7f617c103670 2024-09-05T07:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.237+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1008e0 con 0x7f617c103670 2024-09-05T07:04:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.239+0000 7f6179ffb640 1 -- 172.21.15.86:0/2791340409 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6168021070 con 0x7f617c103670 2024-09-05T07:04:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 Processor -- start 2024-09-05T07:04:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 -- start start 2024-09-05T07:04:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d48d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d4a0e90 0x55b02d4a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b02d3f32e0 con 0x55b02d48d3c0 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b02d39ace0 con 0x55b02d48dce0 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.240+0000 7f61852f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b02d3ef2a0 con 0x55b02d4a0e90 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45150/0 (socket says 172.21.15.86:45150) 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 -- 172.21.15.86:0/124631297 learned_addr learned my addr 172.21.15.86:0/124631297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f61814bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d48d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f617bfff640 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d4a0e90 0x55b02d4a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6179ffb640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 0x7f616c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 -- 172.21.15.86:0/124631297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d4a0e90 msgr2=0x55b02d4a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d4a0e90 0x55b02d4a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 -- 172.21.15.86:0/124631297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 msgr2=0x55b02d48d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d48d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6180cba640 1 -- 172.21.15.86:0/124631297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d37f440 con 0x55b02d48dce0 2024-09-05T07:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f61814bb640 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d48d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f61814bb640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 0x7f616c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.241+0000 7f6179ffb640 1 -- 172.21.15.86:0/2791340409 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f61680d3280 con 0x7f617c103670 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f6180cba640 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6170010480 tx=0x7f6170010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc1c1536bf26a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f6166ffd640 1 -- 172.21.15.86:0/124631297 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170011760 con 0x55b02d48dce0 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f6166ffd640 1 -- 172.21.15.86:0/124631297 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6170011db0 con 0x55b02d48dce0 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f6166ffd640 1 -- 172.21.15.86:0/124631297 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617000e430 con 0x55b02d48dce0 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f61852f7740 1 -- 172.21.15.86:0/124631297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 msgr2=0x55b02d4a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.242+0000 7f61852f7740 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6170010480 tx=0x7f6170010940 comp rx=0 tx=0).stop 2024-09-05T07:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61814bb640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 0x7f616c0a5230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6174005e30 tx=0x7f617403c040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 -- 172.21.15.86:0/124631297 shutdown_connections 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d4a0e90 0x55b02d4a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a0950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 --2- 172.21.15.86:0/124631297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d48d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 -- 172.21.15.86:0/124631297 >> 172.21.15.86:0/124631297 conn(0x55b02d489a30 msgr2=0x55b02d489e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 -- 172.21.15.86:0/124631297 shutdown_connections 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 -- 172.21.15.86:0/124631297 wait complete. 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 -- start start 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.243+0000 7f61852f7740 1 Processor -- start 2024-09-05T07:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d49e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b02d4a3a00 con 0x55b02d48d3c0 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b02d3ae050 con 0x55b02d4a0e90 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b02d4a3b80 con 0x55b02d48dce0 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61852f7740 10 client.0 _ll_register_callbacks cb 0x55b02d3ee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35308/0 (socket says 172.21.15.86:35308) 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 -- 172.21.15.86:0/3226380918 learned_addr learned my addr 172.21.15.86:0/3226380918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f61814bb640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d49e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 -- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 msgr2=0x55b02d49e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d49e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 -- 172.21.15.86:0/3226380918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 msgr2=0x55b02d4a0840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f6180cba640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f617bfff640 1 -- 172.21.15.86:0/3226380918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d1643c0 con 0x55b02d48dce0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.244+0000 7f6180cba640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f617bfff640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b40 tx=0x7f61680d3480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ce44f170eb1e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f615bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616809eeb0 con 0x55b02d48dce0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f61680bf0b0 con 0x55b02d48dce0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6170011db0 con 0x55b02d48dce0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f616809ecd0 con 0x55b02d48dce0 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f61852f7740 10 client.29341 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.245+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b02d4af4b0 con 0x55b02d48dce0 2024-09-05T07:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6170011db0 con 0x55b02d48dce0 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f61680d0ce0 con 0x55b02d48dce0 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 10 client.29341 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 10 client.29341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 20 client.29341 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d4b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 4 client.29341 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02d400850 con 0x55b02d4b0800 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.246+0000 7f6180cba640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d4b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.247+0000 7f615bfff640 10 client.29341 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.247+0000 7f6180cba640 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d4b2c50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688eeab85b6db4ae server_cookie=6b9009bd4ca03e35 in_seq=0 out_seq=0 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f615bfff640 1 -- 172.21.15.86:0/3226380918 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b02d400850 con 0x55b02d4b0800 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f615bfff640 10 client.29341 handle_command_reply: tid=0 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 client.29341 shutdown 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 msgr2=0x55b02d49daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b40 tx=0x7f61680d3480 comp rx=0 tx=0).stop 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 shutdown_connections 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d4b2c50 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d49e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49daf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f615bfff640 0 client.29341 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 --2- 172.21.15.86:0/3226380918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 >> 172.21.15.86:0/3226380918 conn(0x55b02d489a30 msgr2=0x55b02d48f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.248+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 shutdown_connections 2024-09-05T07:04:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.249+0000 7f61852f7740 1 -- 172.21.15.86:0/3226380918 wait complete. 2024-09-05T07:04:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.249+0000 7f61852f7740 20 client.29341 trim_cache size 0 max 16384 2024-09-05T07:04:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 Processor -- start 2024-09-05T07:04:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 -- start start 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1ad090 con 0x55b02d48d3c0 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1aebd0 con 0x55b02d4a0e90 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.273+0000 7f61852f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1ad760 con 0x55b02d48dce0 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:45176/0 (socket says 172.21.15.86:45176) 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f617bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 -- 172.21.15.86:0/1150332141 learned_addr learned my addr 172.21.15.86:0/1150332141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 -- 172.21.15.86:0/1150332141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 msgr2=0x55b02d49d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 -- 172.21.15.86:0/1150332141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 msgr2=0x55b02d4acca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:04:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 -- 172.21.15.86:0/1150332141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d49e400 con 0x55b02d4a0e90 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.274+0000 7f61814bb640 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6174009980 tx=0x7f6174042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9914805fbdcaf45 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.275+0000 7f61667fc640 1 -- 172.21.15.86:0/1150332141 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174056070 con 0x55b02d4a0e90 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.275+0000 7f61667fc640 1 -- 172.21.15.86:0/1150332141 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6174003e20 con 0x55b02d4a0e90 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.275+0000 7f61667fc640 1 -- 172.21.15.86:0/1150332141 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174034040 con 0x55b02d4a0e90 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 -- 172.21.15.86:0/1150332141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 msgr2=0x55b02d499110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6174009980 tx=0x7f6174042000 comp rx=0 tx=0).stop 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 -- 172.21.15.86:0/1150332141 shutdown_connections 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d4a0e90 0x55b02d499110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48dce0 0x55b02d49d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 --2- 172.21.15.86:0/1150332141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d48d3c0 0x55b02d4acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 -- 172.21.15.86:0/1150332141 >> 172.21.15.86:0/1150332141 conn(0x55b02d489a30 msgr2=0x55b02d489e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 -- 172.21.15.86:0/1150332141 shutdown_connections 2024-09-05T07:04:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.276+0000 7f61852f7740 1 -- 172.21.15.86:0/1150332141 wait complete. 2024-09-05T07:04:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.277+0000 7f61852f7740 1 -- start start 2024-09-05T07:04:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.277+0000 7f61852f7740 1 Processor -- start 2024-09-05T07:04:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d4a0e90 0x55b02d4a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1ae500 con 0x55b02d4a0e90 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1ade30 con 0x55b02d48dce0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1ab090 con 0x55b02d48d3c0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61852f7740 10 client.0 _ll_register_callbacks cb 0x7f6168019bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35322/0 (socket says 172.21.15.86:35322) 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 -- 172.21.15.86:0/717750634 learned_addr learned my addr 172.21.15.86:0/717750634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f617bfff640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f61814bb640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d4a0e90 0x55b02d4a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 -- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 msgr2=0x55b02d4a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 -- 172.21.15.86:0/717750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d4a0e90 msgr2=0x55b02d4a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d4a0e90 0x55b02d4a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.278+0000 7f6180cba640 1 -- 172.21.15.86:0/717750634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d51a570 con 0x55b02d48d3c0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f6180cba640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f617000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=615c5cefe84317d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f615bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617001cbe0 con 0x55b02d48d3c0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f617001cd80 con 0x55b02d48d3c0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6174003e20 con 0x55b02d48d3c0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6170005020 con 0x55b02d48d3c0 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f61852f7740 10 client.29347 fetch_fsmap learned FSMap version 46 2024-09-05T07:04:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.279+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b02d51b840 con 0x55b02d48d3c0 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6174003e20 con 0x55b02d48d3c0 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f6170004c30 con 0x55b02d48d3c0 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 10 client.29347 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 10 client.29347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 20 client.29347 populate_metadata read hostname 'smithi086' 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d51dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:04:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 4 client.29347 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b02d3fad60 con 0x55b02d4b0800 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f617bfff640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d51dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.280+0000 7f615bfff640 10 client.29347 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f617bfff640 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d51dd30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218ec0083f884008 server_cookie=4ccfcdcf0a63c039 in_seq=0 out_seq=0 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f615bfff640 1 -- 172.21.15.86:0/717750634 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b02d3fad60 con 0x55b02d4b0800 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f615bfff640 10 client.29347 handle_command_reply: tid=0 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 client.29347 shutdown 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 msgr2=0x55b02d4a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f617000bce0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 shutdown_connections 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55b02d4b0800 0x55b02d51dd30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b02d4a0e90 0x55b02d4a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d48dce0 0x55b02d4a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 --2- 172.21.15.86:0/717750634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b02d48d3c0 0x55b02d4a4bd0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6170000c00 tx=0x7f617000bce0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f615bfff640 0 client.29347 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 >> 172.21.15.86:0/717750634 conn(0x55b02d489a30 msgr2=0x55b02d499110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.281+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 shutdown_connections 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.282+0000 7f61852f7740 1 -- 172.21.15.86:0/717750634 wait complete. 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.282+0000 7f61852f7740 20 client.29347 trim_cache size 0 max 16384 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:04:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: "35bcaa01-af53-4e1c-abba-4466c589d7c6": { 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "35bcaa01-af53-4e1c-abba-4466c589d7c6", 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:04:43.287 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 msgr2=0x7f616c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 0x7f616c0a5230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6174005e30 tx=0x7f617403c040 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 msgr2=0x7f617c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b70 tx=0x7f616800b690 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 shutdown_connections 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f616c0a2d70 0x7f616c0a5230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f617c10c180 0x7f617c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f617c103670 0x7f617c198850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 --2- 172.21.15.86:0/2791340409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c102d50 0x7f617c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 >> 172.21.15.86:0/2791340409 conn(0x7f617c0fc7d0 msgr2=0x7f617c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 shutdown_connections 2024-09-05T07:04:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:04:43.283+0000 7f6182f1e640 1 -- 172.21.15.86:0/2791340409 wait complete. 2024-09-05T07:04:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:43 smithi086 ceph-mon[30943]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 228 op/s 2024-09-05T07:04:43.784 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68536 5.96 MB/sec execute 481 sec latency 92.951 ms 2024-09-05T07:04:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:43 smithi096 ceph-mon[34468]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 228 op/s 2024-09-05T07:04:43.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:43 smithi115 ceph-mon[34791]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 228 op/s 2024-09-05T07:04:44.283 INFO:tasks.cephfs.filesystem:scrub status for tag:35bcaa01-af53-4e1c-abba-4466c589d7c6 - {'path': '/', 'tag': '35bcaa01-af53-4e1c-abba-4466c589d7c6', 'options': 'recursive,force'} 2024-09-05T07:04:44.785 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68706 5.97 MB/sec execute 482 sec latency 60.371 ms 2024-09-05T07:04:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:45 smithi086 ceph-mon[30943]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2024-09-05T07:04:45.786 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68875 5.99 MB/sec execute 483 sec latency 66.238 ms 2024-09-05T07:04:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:45 smithi096 ceph-mon[34468]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2024-09-05T07:04:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:45 smithi115 ceph-mon[34791]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 218 op/s 2024-09-05T07:04:46.786 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69035 5.99 MB/sec execute 484 sec latency 57.071 ms 2024-09-05T07:04:47.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:47 smithi086 ceph-mon[30943]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 214 op/s 2024-09-05T07:04:47.786 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69190 6.00 MB/sec execute 485 sec latency 70.005 ms 2024-09-05T07:04:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:47 smithi096 ceph-mon[34468]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 214 op/s 2024-09-05T07:04:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:47 smithi115 ceph-mon[34791]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 214 op/s 2024-09-05T07:04:48.787 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69351 6.01 MB/sec execute 486 sec latency 63.215 ms 2024-09-05T07:04:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:49 smithi086 ceph-mon[30943]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-09-05T07:04:49.786 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69499 6.01 MB/sec execute 487 sec latency 64.089 ms 2024-09-05T07:04:49.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:49 smithi096 ceph-mon[34468]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-09-05T07:04:49.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:49 smithi115 ceph-mon[34791]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-09-05T07:04:50.788 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69667 6.02 MB/sec execute 488 sec latency 98.709 ms 2024-09-05T07:04:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 209 op/s 2024-09-05T07:04:51.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:51.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:51.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:51.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:51.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:51 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:04:51.790 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69855 6.02 MB/sec execute 489 sec latency 315.907 ms 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 209 op/s 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:51.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:51 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 209 op/s 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:51 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:04:52.790 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69990 6.02 MB/sec execute 490 sec latency 261.785 ms 2024-09-05T07:04:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:53 smithi086 ceph-mon[30943]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 312 op/s 2024-09-05T07:04:53.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70163 6.03 MB/sec execute 491 sec latency 230.367 ms 2024-09-05T07:04:53.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:53 smithi096 ceph-mon[34468]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 312 op/s 2024-09-05T07:04:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:53 smithi115 ceph-mon[34791]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 312 op/s 2024-09-05T07:04:54.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70330 6.04 MB/sec execute 492 sec latency 158.536 ms 2024-09-05T07:04:55.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:55 smithi086 ceph-mon[30943]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 222 op/s 2024-09-05T07:04:55.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70477 6.04 MB/sec execute 493 sec latency 100.781 ms 2024-09-05T07:04:55.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:55 smithi096 ceph-mon[34468]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 222 op/s 2024-09-05T07:04:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:55 smithi115 ceph-mon[34791]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 222 op/s 2024-09-05T07:04:56.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70628 6.05 MB/sec execute 494 sec latency 74.870 ms 2024-09-05T07:04:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:57 smithi086 ceph-mon[30943]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-05T07:04:57.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70798 6.05 MB/sec execute 495 sec latency 67.191 ms 2024-09-05T07:04:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:57 smithi096 ceph-mon[34468]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-05T07:04:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:57 smithi115 ceph-mon[34791]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-05T07:04:58.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70951 6.04 MB/sec execute 496 sec latency 88.563 ms 2024-09-05T07:04:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:04:59 smithi086 ceph-mon[30943]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.1 MiB/s wr, 332 op/s 2024-09-05T07:04:59.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71109 6.04 MB/sec execute 497 sec latency 101.683 ms 2024-09-05T07:04:59.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:04:59 smithi096 ceph-mon[34468]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.1 MiB/s wr, 332 op/s 2024-09-05T07:04:59.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:04:59 smithi115 ceph-mon[34791]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.1 MiB/s wr, 332 op/s 2024-09-05T07:05:00.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71311 6.05 MB/sec execute 498 sec latency 73.293 ms 2024-09-05T07:05:01.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:01 smithi115 ceph-mon[34791]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:05:01.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:01 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:05:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:01 smithi086 ceph-mon[30943]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:05:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:01 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:05:01.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71467 6.05 MB/sec execute 499 sec latency 92.781 ms 2024-09-05T07:05:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:01 smithi096 ceph-mon[34468]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:05:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:01 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:05:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:02 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:05:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:02 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:05:02.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:02 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:05:02.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71621 6.06 MB/sec execute 500 sec latency 137.631 ms 2024-09-05T07:05:02.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:02 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:05:02.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:02 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:05:02.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:02 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:05:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:02 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:05:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:02 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:05:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:02 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:05:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:03 smithi086 ceph-mon[30943]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 345 op/s 2024-09-05T07:05:03.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71820 6.07 MB/sec execute 501 sec latency 61.417 ms 2024-09-05T07:05:03.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:03 smithi096 ceph-mon[34468]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 345 op/s 2024-09-05T07:05:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:03 smithi115 ceph-mon[34791]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 345 op/s 2024-09-05T07:05:04.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72048 6.10 MB/sec execute 502 sec latency 73.215 ms 2024-09-05T07:05:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:05 smithi086 ceph-mon[30943]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 259 op/s 2024-09-05T07:05:05.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72242 6.11 MB/sec execute 503 sec latency 59.682 ms 2024-09-05T07:05:05.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:05 smithi096 ceph-mon[34468]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 259 op/s 2024-09-05T07:05:05.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:05 smithi115 ceph-mon[34791]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 259 op/s 2024-09-05T07:05:06.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72409 6.12 MB/sec execute 504 sec latency 60.485 ms 2024-09-05T07:05:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:07 smithi086 ceph-mon[30943]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-05T07:05:07.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72542 6.12 MB/sec execute 505 sec latency 112.949 ms 2024-09-05T07:05:07.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:07 smithi096 ceph-mon[34468]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-05T07:05:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:07 smithi115 ceph-mon[34791]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-05T07:05:08.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72710 6.12 MB/sec execute 506 sec latency 74.815 ms 2024-09-05T07:05:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:09 smithi086 ceph-mon[30943]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2024-09-05T07:05:09.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72880 6.13 MB/sec execute 507 sec latency 76.521 ms 2024-09-05T07:05:09.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:09 smithi096 ceph-mon[34468]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2024-09-05T07:05:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:09 smithi115 ceph-mon[34791]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2024-09-05T07:05:10.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73035 6.13 MB/sec execute 508 sec latency 62.375 ms 2024-09-05T07:05:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:11 smithi086 ceph-mon[30943]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-09-05T07:05:11.791 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73174 6.13 MB/sec execute 509 sec latency 90.136 ms 2024-09-05T07:05:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:11 smithi096 ceph-mon[34468]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-09-05T07:05:11.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:11 smithi115 ceph-mon[34791]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-09-05T07:05:12.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73331 6.13 MB/sec execute 510 sec latency 77.260 ms 2024-09-05T07:05:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:13 smithi086 ceph-mon[30943]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 336 op/s 2024-09-05T07:05:13.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73523 6.15 MB/sec execute 511 sec latency 106.436 ms 2024-09-05T07:05:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:13 smithi096 ceph-mon[34468]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 336 op/s 2024-09-05T07:05:13.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:13 smithi115 ceph-mon[34791]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.7 MiB/s wr, 336 op/s 2024-09-05T07:05:14.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:05:14.685 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:14.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73686 6.15 MB/sec execute 512 sec latency 154.614 ms 2024-09-05T07:05:15.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 -- 172.21.15.86:0/1311732385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50101410 msgr2=0x7f0b5010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1311732385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50101410 0x7f0b5010d940 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009cd0 tx=0x7f0b4802a8e0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 -- 172.21.15.86:0/1311732385 shutdown_connections 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1311732385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50101410 0x7f0b5010d940 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1311732385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1311732385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50106a70 0x7f0b50106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 -- 172.21.15.86:0/1311732385 >> 172.21.15.86:0/1311732385 conn(0x7f0b500fc790 msgr2=0x7f0b500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 -- 172.21.15.86:0/1311732385 shutdown_connections 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.259+0000 7f0b572a9640 1 -- 172.21.15.86:0/1311732385 wait complete. 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 Processor -- start 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 -- start start 2024-09-05T07:05:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50101410 0x7f0b50198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 0x7f0b5019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50110d70 con 0x7f0b50106a70 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50110bf0 con 0x7f0b50100a70 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b572a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50110ef0 con 0x7f0b50101410 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b55045640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b55045640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38904/0 (socket says 172.21.15.86:38904) 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b55045640 1 -- 172.21.15.86:0/1815764316 learned_addr learned my addr 172.21.15.86:0/1815764316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b55846640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 0x7f0b5019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.260+0000 7f0b54844640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50101410 0x7f0b50198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 -- 172.21.15.86:0/1815764316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50101410 msgr2=0x7f0b50198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50101410 0x7f0b50198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 -- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 msgr2=0x7f0b5019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 0x7f0b5019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 -- 172.21.15.86:0/1815764316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5019d2f0 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55846640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 0x7f0b5019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b55045640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00e9f0 tx=0x7f0b4c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.261+0000 7f0b467fc640 1 -- 172.21.15.86:0/1815764316 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00cd80 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.262+0000 7f0b467fc640 1 -- 172.21.15.86:0/1815764316 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b4c0054d0 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.262+0000 7f0b467fc640 1 -- 172.21.15.86:0/1815764316 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c010490 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.262+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5019d5e0 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.262+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b500fcd90 con 0x7f0b50100a70 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 Processor -- start 2024-09-05T07:05:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 -- start start 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563020ae22e0 con 0x563020b855d0 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563020a89ce0 con 0x563020b8fe90 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b59682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563020ade2a0 con 0x563020b84cb0 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b467fc640 1 -- 172.21.15.86:0/1815764316 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0b4c005020 con 0x7f0b50100a70 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b55846640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b55846640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55010/0 (socket says 172.21.15.86:55010) 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b55846640 1 -- 172.21.15.86:0/909900274 learned_addr learned my addr 172.21.15.86:0/909900274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.266+0000 7f0b54844640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.267+0000 7f0b55846640 1 -- 172.21.15.86:0/909900274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 msgr2=0x563020b92280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.267+0000 7f0b55846640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b92280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.267+0000 7f0b55846640 1 -- 172.21.15.86:0/909900274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 msgr2=0x563020b8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.267+0000 7f0b55045640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b467fc640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 0x7f0b3c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b55846640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b55846640 1 -- 172.21.15.86:0/909900274 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563020a6e440 con 0x563020b84cb0 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b54844640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 0x7f0b3c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b467fc640 1 -- 172.21.15.86:0/1815764316 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f0b4c0cfb80 con 0x7f0b50100a70 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b55045640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b55846640 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009cd0 tx=0x7f0b48005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24efc7e4e09a5ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b3b7fe640 1 -- 172.21.15.86:0/909900274 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48048070 con 0x563020b84cb0 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b3b7fe640 1 -- 172.21.15.86:0/909900274 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b48036070 con 0x563020b84cb0 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.268+0000 7f0b3b7fe640 1 -- 172.21.15.86:0/909900274 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004750 con 0x563020b84cb0 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 -- 172.21.15.86:0/909900274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 msgr2=0x563020b85090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b54844640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 0x7f0b3c0a5230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x563020b825c0 tx=0x7f0b40005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009cd0 tx=0x7f0b48005f80 comp rx=0 tx=0).stop 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 -- 172.21.15.86:0/909900274 shutdown_connections 2024-09-05T07:05:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 --2- 172.21.15.86:0/909900274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b85090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.269+0000 7f0b59682740 1 -- 172.21.15.86:0/909900274 >> 172.21.15.86:0/909900274 conn(0x563020b81240 msgr2=0x563020b81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.270+0000 7f0b59682740 1 -- 172.21.15.86:0/909900274 shutdown_connections 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.270+0000 7f0b59682740 1 -- 172.21.15.86:0/909900274 wait complete. 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.270+0000 7f0b59682740 1 -- start start 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.270+0000 7f0b59682740 1 Processor -- start 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b59682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b59682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b59682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b55045640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b54844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b54844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58736/0 (socket says 172.21.15.86:58736) 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b54844640 1 -- 172.21.15.86:0/311526427 learned_addr learned my addr 172.21.15.86:0/311526427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b55045640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55012/0 (socket says 172.21.15.86:55012) 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b55846640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b59682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563020b92a60 con 0x563020b855d0 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.271+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563020a9d050 con 0x563020b8fe90 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563020b92be0 con 0x563020b84cb0 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b59682740 10 client.0 _ll_register_callbacks cb 0x563020add160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 -- 172.21.15.86:0/311526427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 msgr2=0x563020b8f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 -- 172.21.15.86:0/311526427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 msgr2=0x563020b8cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.585 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:05:15.585 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:05:15.585 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:05:15.585 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630208533c0 con 0x563020b8fe90 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b54844640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55045640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b55846640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b8d050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b4802aed0 tx=0x7f0b48004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4af64f7fb8aec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.272+0000 7f0b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48052020 con 0x563020b8fe90 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b48036070 con 0x563020b8fe90 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f0b48003b70 con 0x563020b8fe90 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0b4805d050 con 0x563020b8fe90 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b59682740 10 client.29306 fetch_fsmap learned FSMap version 46 2024-09-05T07:05:15.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563020b92a60 con 0x563020b8fe90 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.273+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b480098a0 con 0x563020b8fe90 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f0b4804b030 con 0x563020b8fe90 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 10 client.29306 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 10 client.29306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 20 client.29306 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020b9f770 0x563020ba1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 4 client.29306 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0b48036070 con 0x563020b9f770 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.274+0000 7f0b55045640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020b9f770 0x563020ba1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.275+0000 7f0b55045640 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020b9f770 0x563020ba1b90 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3ea4b4e467078d server_cookie=34def5565ffee809 in_seq=0 out_seq=0 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.275+0000 7f0b38ff9640 10 client.29306 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.275+0000 7f0b38ff9640 1 -- 172.21.15.86:0/311526427 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0b48036070 con 0x563020b9f770 2024-09-05T07:05:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.275+0000 7f0b38ff9640 10 client.29306 handle_command_reply: tid=0 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.275+0000 7f0b59682740 1 client.29306 shutdown 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 msgr2=0x563020b8d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b8d050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b4802aed0 tx=0x7f0b48004700 comp rx=0 tx=0).stop 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 shutdown_connections 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020b9f770 0x563020ba1b90 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020b8d050 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020b8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 --2- 172.21.15.86:0/311526427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b84cb0 0x563020b8f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 >> 172.21.15.86:0/311526427 conn(0x563020b81240 msgr2=0x563020b86f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b38ff9640 0 client.29306 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 shutdown_connections 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 1 -- 172.21.15.86:0/311526427 wait complete. 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.276+0000 7f0b59682740 20 client.29306 trim_cache size 0 max 16384 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.308+0000 7f0b59682740 1 Processor -- start 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.308+0000 7f0b59682740 1 -- start start 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 0x563020b9bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020ba74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 0x563020ba79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501acf40 con 0x563020b855d0 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501aea80 con 0x563020b84cb0 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b59682740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501ad610 con 0x563020b8fe90 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b54844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020ba74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 0x563020ba79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b54844640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020ba74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58752/0 (socket says 172.21.15.86:58752) 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b54844640 1 -- 172.21.15.86:0/3103445870 learned_addr learned my addr 172.21.15.86:0/3103445870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55045640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 0x563020b9bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 -- 172.21.15.86:0/3103445870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 msgr2=0x563020b9bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 0x563020b9bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 -- 172.21.15.86:0/3103445870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 msgr2=0x563020ba74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020ba74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 -- 172.21.15.86:0/3103445870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563020b81070 con 0x563020b8fe90 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55045640 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 0x563020b9bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.309+0000 7f0b55846640 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 0x563020ba79e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005e30 tx=0x7f0b48005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94af9baed7ab2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b3affd640 1 -- 172.21.15.86:0/3103445870 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48048040 con 0x563020b8fe90 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b3affd640 1 -- 172.21.15.86:0/3103445870 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b48036070 con 0x563020b8fe90 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b3affd640 1 -- 172.21.15.86:0/3103445870 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480597f0 con 0x563020b8fe90 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 -- 172.21.15.86:0/3103445870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 msgr2=0x563020ba79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 0x563020ba79e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005e30 tx=0x7f0b48005d80 comp rx=0 tx=0).stop 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 -- 172.21.15.86:0/3103445870 shutdown_connections 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b8fe90 0x563020ba79e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b855d0 0x563020ba74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 --2- 172.21.15.86:0/3103445870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b84cb0 0x563020b9bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.310+0000 7f0b59682740 1 -- 172.21.15.86:0/3103445870 >> 172.21.15.86:0/3103445870 conn(0x563020b81240 msgr2=0x563020b8b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.311+0000 7f0b59682740 1 -- 172.21.15.86:0/3103445870 shutdown_connections 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.311+0000 7f0b59682740 1 -- 172.21.15.86:0/3103445870 wait complete. 2024-09-05T07:05:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.311+0000 7f0b59682740 1 -- start start 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.311+0000 7f0b59682740 1 Processor -- start 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b855d0 0x563020b93ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58762/0 (socket says 172.21.15.86:58762) 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 -- 172.21.15.86:0/2158187917 learned_addr learned my addr 172.21.15.86:0/2158187917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b54844640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b855d0 0x563020b93ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020c02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501ae3b0 con 0x563020b84cb0 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501adce0 con 0x563020b8fe90 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501aaf40 con 0x563020b855d0 2024-09-05T07:05:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b59682740 10 client.0 _ll_register_callbacks cb 0x7f0b48046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55846640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020c02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 -- 172.21.15.86:0/2158187917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b855d0 msgr2=0x563020b93ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b855d0 0x563020b93ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 -- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 msgr2=0x563020c02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020c02430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.312+0000 7f0b55045640 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563020c02a20 con 0x563020b84cb0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.313+0000 7f0b55045640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c09b900 tx=0x7f0b4c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5281ac463836ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.313+0000 7f0b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.313+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c09c300 con 0x563020b84cb0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.313+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f0b4c0046c0 con 0x563020b84cb0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.313+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b48036070 con 0x563020b84cb0 2024-09-05T07:05:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0b4c09bc00 con 0x563020b84cb0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 10 client.20658 fetch_fsmap learned FSMap version 46 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563020c09df0 con 0x563020b84cb0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b48036070 con 0x563020b84cb0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f0b4c0a43e0 con 0x563020b84cb0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 10 client.20658 fetch_fsmap finished waiting for FSMap version 46 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 10 client.20658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 20 client.20658 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020c0a5a0 0x563020c0c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.314+0000 7f0b59682740 4 client.20658 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.315+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563020aef850 con 0x563020c0a5a0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.315+0000 7f0b54844640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020c0a5a0 0x563020c0c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b38ff9640 10 client.20658 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b54844640 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020c0a5a0 0x563020c0c9c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991ea0756ef8eb12 server_cookie=b754574fa711f65e in_seq=0 out_seq=0 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b38ff9640 1 -- 172.21.15.86:0/2158187917 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563020aef850 con 0x563020c0a5a0 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b38ff9640 10 client.20658 handle_command_reply: tid=0 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 client.20658 shutdown 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 msgr2=0x563020b93570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c09b900 tx=0x7f0b4c005f10 comp rx=0 tx=0).stop 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 shutdown_connections 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x563020c0a5a0 0x563020c0c9c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563020b8fe90 0x563020c02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563020b855d0 0x563020b93ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 --2- 172.21.15.86:0/2158187917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563020b84cb0 0x563020b93570 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 >> 172.21.15.86:0/2158187917 conn(0x563020b81240 msgr2=0x563020b87810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.316+0000 7f0b38ff9640 0 client.20658 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 shutdown_connections 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b59682740 1 -- 172.21.15.86:0/2158187917 wait complete. 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b59682740 20 client.20658 trim_cache size 0 max 16384 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 msgr2=0x7f0b3c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 0x7f0b3c0a5230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x563020b825c0 tx=0x7f0b40005f90 comp rx=0 tx=0).stop 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 msgr2=0x7f0b50198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00e9f0 tx=0x7f0b4c00edf0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 shutdown_connections 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0b3c0a2d70 0x7f0b3c0a5230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b50106a70 0x7f0b5019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b50101410 0x7f0b50198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.317+0000 7f0b572a9640 1 --2- 172.21.15.86:0/1815764316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b50100a70 0x7f0b50198340 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.318+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 >> 172.21.15.86:0/1815764316 conn(0x7f0b500fc790 msgr2=0x7f0b5006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.318+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 shutdown_connections 2024-09-05T07:05:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:15.318+0000 7f0b572a9640 1 -- 172.21.15.86:0/1815764316 wait complete. 2024-09-05T07:05:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:15 smithi086 ceph-mon[30943]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-05T07:05:15.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73840 6.16 MB/sec execute 513 sec latency 325.272 ms 2024-09-05T07:05:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:15 smithi096 ceph-mon[34468]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-05T07:05:15.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:15 smithi115 ceph-mon[34791]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-05T07:05:16.514 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:05:16.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:05:16.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73974 6.16 MB/sec execute 514 sec latency 205.694 ms 2024-09-05T07:05:16.892 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.463+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3690793458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8100e20 msgr2=0x7fd6c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.463+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3690793458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8100e20 0x7fd6c8101200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009930 tx=0x7fd6b802a4d0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3690793458 shutdown_connections 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3690793458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102ef0 0x7fd6c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3690793458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8102570 0x7fd6c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3690793458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8100e20 0x7fd6c8101200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3690793458 >> 172.21.15.86:0/3690793458 conn(0x7fd6c80fc7d0 msgr2=0x7fd6c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3690793458 shutdown_connections 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.464+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3690793458 wait complete. 2024-09-05T07:05:17.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 Processor -- start 2024-09-05T07:05:17.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 -- start start 2024-09-05T07:05:17.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 0x7fd6c819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 0x7fd6c819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8102ef0 0x7fd6c819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6ccfb9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8111f50 con 0x7fd6c8102570 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 0x7fd6c819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.465+0000 7fd6c5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 0x7fd6c819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6ccfb9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8111dd0 con 0x7fd6c8100e20 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6ccfb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81120d0 con 0x7fd6c8102ef0 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 0x7fd6c819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38934/0 (socket says 172.21.15.86:38934) 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c659c640 1 -- 172.21.15.86:0/3694949347 learned_addr learned my addr 172.21.15.86:0/3694949347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c6d9d640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8102ef0 0x7fd6c819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 -- 172.21.15.86:0/3694949347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8102ef0 msgr2=0x7fd6c819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8102ef0 0x7fd6c819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 -- 172.21.15.86:0/3694949347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 msgr2=0x7fd6c819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 0x7fd6c819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 -- 172.21.15.86:0/3694949347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c819f5f0 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6c5d9b640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 0x7fd6c819ab80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009f60 tx=0x7fd6bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.466+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc022070 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.467+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6bc01b9d0 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.467+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01da70 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.467+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c819f8e0 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.467+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c8107980 con 0x7fd6c8102570 2024-09-05T07:05:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.468+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c81021b0 con 0x7fd6c8102570 2024-09-05T07:05:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.470+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd6bc025070 con 0x7fd6c8102570 2024-09-05T07:05:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.471+0000 7fd6b77fe640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 0x7fd6b00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.471+0000 7fd6c659c640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 0x7fd6b00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.471+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fd6bc0d73f0 con 0x7fd6c8102570 2024-09-05T07:05:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.472+0000 7fd6c659c640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 0x7fd6b00a5300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005c40 tx=0x7fd6b8005b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.477+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6bc0a3580 con 0x7fd6c8102570 2024-09-05T07:05:17.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:17 smithi086 ceph-mon[30943]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2024-09-05T07:05:17.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74134 6.17 MB/sec execute 515 sec latency 83.095 ms 2024-09-05T07:05:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:17 smithi096 ceph-mon[34468]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2024-09-05T07:05:17.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:05:17.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"btime":"2024-09-05T07:03:15:171132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:03:14.624890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:05:17.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.816+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6c8107ca0 con 0x7fd6c8102570 2024-09-05T07:05:17.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.817+0000 7fd6b77fe640 1 -- 172.21.15.86:0/3694949347 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11975 (secure 0 0 0) 0x7fd6bc0a7880 con 0x7fd6c8102570 2024-09-05T07:05:17.820 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-09-05T07:05:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.822+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 msgr2=0x7fd6b00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.822+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 0x7fd6b00a5300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005c40 tx=0x7fd6b8005b50 comp rx=0 tx=0).stop 2024-09-05T07:05:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.822+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 msgr2=0x7fd6c819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.822+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 0x7fd6c819ab80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009f60 tx=0x7fd6bc009f90 comp rx=0 tx=0).stop 2024-09-05T07:05:17.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 shutdown_connections 2024-09-05T07:05:17.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd6b00a2e40 0x7fd6b00a5300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8102ef0 0x7fd6c819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c8102570 0x7fd6c819ab80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 --2- 172.21.15.86:0/3694949347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8100e20 0x7fd6c819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.823+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 >> 172.21.15.86:0/3694949347 conn(0x7fd6c80fc7d0 msgr2=0x7fd6c810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.824+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 shutdown_connections 2024-09-05T07:05:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:17.824+0000 7fd6ccfb9640 1 -- 172.21.15.86:0/3694949347 wait complete. 2024-09-05T07:05:17.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:17 smithi115 ceph-mon[34791]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 224 op/s 2024-09-05T07:05:18.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:05:18.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74289 6.17 MB/sec execute 516 sec latency 73.628 ms 2024-09-05T07:05:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3694949347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:05:18.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3694949347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:05:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:18.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3694949347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:05:19.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74427 6.16 MB/sec execute 517 sec latency 73.696 ms 2024-09-05T07:05:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:19 smithi096 ceph-mon[34468]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 344 op/s 2024-09-05T07:05:19.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:19 smithi115 ceph-mon[34791]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 344 op/s 2024-09-05T07:05:19.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:19 smithi086 ceph-mon[30943]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 344 op/s 2024-09-05T07:05:20.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74551 6.16 MB/sec execute 518 sec latency 148.342 ms 2024-09-05T07:05:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:21 smithi086 ceph-mon[30943]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 227 op/s 2024-09-05T07:05:21.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:21 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:05:21.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74671 6.16 MB/sec execute 519 sec latency 152.474 ms 2024-09-05T07:05:21.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:21 smithi096 ceph-mon[34468]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 227 op/s 2024-09-05T07:05:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:21 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:05:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:21 smithi115 ceph-mon[34791]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 227 op/s 2024-09-05T07:05:21.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:21 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:05:22.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74815 6.15 MB/sec execute 520 sec latency 156.943 ms 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.404+0000 7f99d4a4d640 1 -- 172.21.15.86:0/1447688136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d01027c0 msgr2=0x7f99d0106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.404+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/1447688136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d01027c0 0x7f99d0106810 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009900 tx=0x7f99b802a410 comp rx=0 tx=0).stop 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 -- 172.21.15.86:0/1447688136 shutdown_connections 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/1447688136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/1447688136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d01027c0 0x7f99d0106810 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/1447688136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d0101e10 0x7f99d01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 -- 172.21.15.86:0/1447688136 >> 172.21.15.86:0/1447688136 conn(0x7f99d00fd8d0 msgr2=0x7f99d00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 -- 172.21.15.86:0/1447688136 shutdown_connections 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.405+0000 7f99d4a4d640 1 -- 172.21.15.86:0/1447688136 wait complete. 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.406+0000 7f99d4a4d640 1 Processor -- start 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.406+0000 7f99d4a4d640 1 -- start start 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.406+0000 7f99d4a4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58798/0 (socket says 172.21.15.86:58798) 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 -- 172.21.15.86:0/3064614830 learned_addr learned my addr 172.21.15.86:0/3064614830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d019ce80 0x7f99d019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111ea0 con 0x7f99d0101e10 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111d20 con 0x7f99d0106f40 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0112020 con 0x7f99d019ce80 2024-09-05T07:05:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99c7fff640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ced9d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d019ce80 0x7f99d019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 -- 172.21.15.86:0/3064614830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d019ce80 msgr2=0x7f99d019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d019ce80 0x7f99d019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 -- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 msgr2=0x7f99d0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d019d940 con 0x7f99d0101e10 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99c7fff640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.407+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f99c000e9f0 tx=0x7f99c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.408+0000 7f99c77fe640 1 -- 172.21.15.86:0/3064614830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c000ccc0 con 0x7f99d0101e10 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.408+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d019dc30 con 0x7f99d0101e10 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.408+0000 7f99c77fe640 1 -- 172.21.15.86:0/3064614830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c000ce60 con 0x7f99d0101e10 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.408+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01a5410 con 0x7f99d0101e10 2024-09-05T07:05:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.408+0000 7f99c77fe640 1 -- 172.21.15.86:0/3064614830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0010430 con 0x7f99d0101e10 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.412+0000 7f99c77fe640 1 -- 172.21.15.86:0/3064614830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f99c0005020 con 0x7f99d0101e10 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.413+0000 7f99c77fe640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 0x7f99bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.413+0000 7f99c7fff640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 0x7f99bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.413+0000 7f99c77fe640 1 -- 172.21.15.86:0/3064614830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f99c00cfd10 con 0x7f99d0101e10 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.414+0000 7f99d6e26740 1 Processor -- start 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.414+0000 7f99c7fff640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 0x7f99bc0a5230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b8005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.414+0000 7f99d6e26740 1 -- start start 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 0x55e958c42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c42a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c4d370 0x55e958c4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e958ba82f0 con 0x55e958c43b40 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e958b4fcf0 con 0x55e958c4d370 2024-09-05T07:05:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99d6e26740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e958ba42b0 con 0x55e958c425d0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c42a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 0x55e958c42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 0x55e958c42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58804/0 (socket says 172.21.15.86:58804) 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99ce59c640 1 -- 172.21.15.86:0/1286385738 learned_addr learned my addr 172.21.15.86:0/1286385738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.415+0000 7f99ced9d640 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c4d370 0x55e958c4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.416+0000 7f99c7fff640 1 -- 172.21.15.86:0/1286385738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c4d370 msgr2=0x55e958c4f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.416+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c4d370 0x55e958c4f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.416+0000 7f99c7fff640 1 -- 172.21.15.86:0/1286385738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 msgr2=0x55e958c42090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.416+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 0x55e958c42090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.416+0000 7f99c7fff640 1 -- 172.21.15.86:0/1286385738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e958b343a0 con 0x55e958c425d0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.419+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c42a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99b8008000 tx=0x7f99b803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8dd1c0315798f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.420+0000 7f99c4ff9640 1 -- 172.21.15.86:0/1286385738 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8035040 con 0x55e958c425d0 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.420+0000 7f99c4ff9640 1 -- 172.21.15.86:0/1286385738 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b80054c0 con 0x55e958c425d0 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.421+0000 7f99c4ff9640 1 -- 172.21.15.86:0/1286385738 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003ae0 con 0x55e958c425d0 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.421+0000 7f99d6e26740 1 -- 172.21.15.86:0/1286385738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 msgr2=0x55e958c42a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.421+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c42a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99b8008000 tx=0x7f99b803d000 comp rx=0 tx=0).stop 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 -- 172.21.15.86:0/1286385738 shutdown_connections 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c4d370 0x55e958c4f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c42a10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1286385738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c43b40 0x55e958c42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 -- 172.21.15.86:0/1286385738 >> 172.21.15.86:0/1286385738 conn(0x55e958c0b310 msgr2=0x55e958c3ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 -- 172.21.15.86:0/1286385738 shutdown_connections 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.422+0000 7f99d6e26740 1 -- 172.21.15.86:0/1286385738 wait complete. 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.423+0000 7f99d6e26740 1 -- start start 2024-09-05T07:05:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.423+0000 7f99d6e26740 1 Processor -- start 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.423+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958c4cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.423+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.423+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99d6e26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e958c50060 con 0x55e958c4d370 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99d6e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e958b63060 con 0x55e958c425d0 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99d6e26740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e958c501e0 con 0x55e958c43b40 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958c4cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958c4cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38980/0 (socket says 172.21.15.86:38980) 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99ce59c640 1 -- 172.21.15.86:0/1469031489 learned_addr learned my addr 172.21.15.86:0/1469031489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99c7fff640 1 -- 172.21.15.86:0/1469031489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 msgr2=0x55e958c4cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958c4cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.424+0000 7f99c7fff640 1 -- 172.21.15.86:0/1469031489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 msgr2=0x55e958c4a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99d6e26740 10 client.0 _ll_register_callbacks cb 0x55e9588b68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c4a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99c7fff640 1 -- 172.21.15.86:0/1469031489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9589193c0 con 0x55e958c43b40 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99ced9d640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b80054c0 con 0x55e958c43b40 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c4a0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99b80096c0 tx=0x7f99b80038f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6184e190afd7d18b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.425+0000 7f99b5ffb640 10 client.29383 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.426+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1469031489 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8057020 con 0x55e958c43b40 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.426+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1469031489 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f99b8003cf0 con 0x55e958c43b40 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.426+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1469031489 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f99b8035070 con 0x55e958c43b40 2024-09-05T07:05:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.428+0000 7f99d6e26740 10 client.29383 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.428+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55e958c50060 con 0x55e958c43b40 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.429+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1469031489 <== mon.1 v2:172.21.15.115:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f99b8004770 con 0x55e958c43b40 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.429+0000 7f99d6e26740 10 client.29383 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.429+0000 7f99d6e26740 10 client.29383 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.429+0000 7f99d6e26740 20 client.29383 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.429+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958c702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.430+0000 7f99d6e26740 4 client.29383 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.430+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e958bb5860 con 0x55e958c6dea0 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.430+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958c702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.430+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958c702c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b7b4cdd09fb785 server_cookie=599195576385a9c in_seq=0 out_seq=0 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.430+0000 7f99b5ffb640 10 client.29383 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1469031489 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e958bb5860 con 0x55e958c6dea0 2024-09-05T07:05:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99b5ffb640 10 client.29383 handle_command_reply: tid=0 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 client.29383 shutdown 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 msgr2=0x55e958c4a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c4a0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99b80096c0 tx=0x7f99b80038f0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 shutdown_connections 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958c702c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.431+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99b5ffb640 0 client.29383 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c4a0e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1469031489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958c4cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 >> 172.21.15.86:0/1469031489 conn(0x55e958c0b310 msgr2=0x55e958c44960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 shutdown_connections 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 1 -- 172.21.15.86:0/1469031489 wait complete. 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.432+0000 7f99d6e26740 20 client.29383 trim_cache size 0 max 16384 2024-09-05T07:05:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.449+0000 7f99d6e26740 1 Processor -- start 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.449+0000 7f99d6e26740 1 -- start start 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c75b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01a6e30 con 0x55e958c4d370 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01a8970 con 0x55e958c43b40 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99d6e26740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01a7500 con 0x55e958c425d0 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55118/0 (socket says 172.21.15.86:55118) 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 -- 172.21.15.86:0/3731521751 learned_addr learned my addr 172.21.15.86:0/3731521751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38986/0 (socket says 172.21.15.86:38986) 2024-09-05T07:05:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ced9d640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c75b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 -- 172.21.15.86:0/3731521751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 msgr2=0x55e958c45660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 -- 172.21.15.86:0/3731521751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 msgr2=0x55e958c75b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c75b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ce59c640 1 -- 172.21.15.86:0/3731521751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e958c70630 con 0x55e958c425d0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99c7fff640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.450+0000 7f99ced9d640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c75b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.451+0000 7f99ce59c640 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99c00981f0 tx=0x7f99c0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7161b131c62dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.451+0000 7f99b7fff640 1 -- 172.21.15.86:0/3731521751 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00a4a00 con 0x55e958c425d0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.451+0000 7f99b7fff640 1 -- 172.21.15.86:0/3731521751 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c009a090 con 0x55e958c425d0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.451+0000 7f99b7fff640 1 -- 172.21.15.86:0/3731521751 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0005310 con 0x55e958c425d0 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- 172.21.15.86:0/3731521751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 msgr2=0x55e958c45120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99c00981f0 tx=0x7f99c0009310 comp rx=0 tx=0).stop 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- 172.21.15.86:0/3731521751 shutdown_connections 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c75b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c43b40 0x55e958c45660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 --2- 172.21.15.86:0/3731521751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c425d0 0x55e958c45120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- 172.21.15.86:0/3731521751 >> 172.21.15.86:0/3731521751 conn(0x55e958c0b310 msgr2=0x55e958c49210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- 172.21.15.86:0/3731521751 shutdown_connections 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- 172.21.15.86:0/3731521751 wait complete. 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 -- start start 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.452+0000 7f99d6e26740 1 Processor -- start 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958cd44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01a82a0 con 0x55e958c4d370 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01a7bd0 con 0x55e958c425d0 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0073ba0 con 0x55e958c43b40 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99d6e26740 10 client.0 _ll_register_callbacks cb 0x7f99c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:55120/0 (socket says 172.21.15.86:55120) 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99c7fff640 1 -- 172.21.15.86:0/1450224887 learned_addr learned my addr 172.21.15.86:0/1450224887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958cd44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ced9d640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 -- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 msgr2=0x55e958c55470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 -- 172.21.15.86:0/1450224887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 msgr2=0x55e958c559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.453+0000 7f99ce59c640 1 -- 172.21.15.86:0/1450224887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e958c47030 con 0x55e958c425d0 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99ced9d640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99ce59c640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958cd44f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f99c0098760 tx=0x7f99c0005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d337828875f3de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:05:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00a9060 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f99c0098400 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c009a090 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f99c0005310 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99d6e26740 10 client.29333 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.454+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e958cdd180 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c009a090 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f99c0097ae0 con 0x55e958c425d0 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 10 client.29333 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 10 client.29333 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 20 client.29333 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958cdf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 4 client.29333 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:05:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e958bafd70 con 0x55e958c6dea0 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.455+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958cdf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99b5ffb640 10 client.29333 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99c7fff640 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958cdf750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7dd4cb5643bd6c1 server_cookie=3c811d560f567fde in_seq=0 out_seq=0 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99b5ffb640 1 -- 172.21.15.86:0/1450224887 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e958bafd70 con 0x55e958c6dea0 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99b5ffb640 10 client.29333 handle_command_reply: tid=0 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 client.29333 shutdown 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 msgr2=0x55e958cd44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958cd44f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f99c0098760 tx=0x7f99c0005bf0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 shutdown_connections 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e958c6dea0 0x55e958cdf750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e958c4d370 0x55e958c559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99b5ffb640 0 client.29333 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:05:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e958c43b40 0x55e958c55470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 --2- 172.21.15.86:0/1450224887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e958c425d0 0x55e958cd44f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 >> 172.21.15.86:0/1450224887 conn(0x55e958c0b310 msgr2=0x55e958c4b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 shutdown_connections 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.456+0000 7f99d6e26740 1 -- 172.21.15.86:0/1450224887 wait complete. 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d6e26740 20 client.29333 trim_cache size 0 max 16384 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 msgr2=0x7f99bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 0x7f99bc0a5230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b8005a20 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 msgr2=0x7f99d01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f99c000e9f0 tx=0x7f99c000edf0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 shutdown_connections 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f99bc0a2d70 0x7f99bc0a5230 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99d019ce80 0x7f99d019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0106f40 0x7f99d0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 --2- 172.21.15.86:0/3064614830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d0101e10 0x7f99d01985b0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 >> 172.21.15.86:0/3064614830 conn(0x7f99d00fd8d0 msgr2=0x7f99d00fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 shutdown_connections 2024-09-05T07:05:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:22.457+0000 7f99d4a4d640 1 -- 172.21.15.86:0/3064614830 wait complete. 2024-09-05T07:05:23.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74986 6.16 MB/sec execute 521 sec latency 83.258 ms 2024-09-05T07:05:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:23 smithi096 ceph-mon[34468]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 317 op/s 2024-09-05T07:05:23.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:23 smithi115 ceph-mon[34791]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 317 op/s 2024-09-05T07:05:23.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:23 smithi086 ceph-mon[30943]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 317 op/s 2024-09-05T07:05:24.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75154 6.17 MB/sec execute 522 sec latency 137.758 ms 2024-09-05T07:05:25.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:25 smithi086 ceph-mon[30943]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 243 op/s 2024-09-05T07:05:25.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75333 6.18 MB/sec execute 523 sec latency 149.879 ms 2024-09-05T07:05:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:25 smithi096 ceph-mon[34468]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 243 op/s 2024-09-05T07:05:25.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:25 smithi115 ceph-mon[34791]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 243 op/s 2024-09-05T07:05:26.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75486 6.19 MB/sec execute 524 sec latency 124.544 ms 2024-09-05T07:05:27.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75612 6.20 MB/sec execute 525 sec latency 211.075 ms 2024-09-05T07:05:27.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:27 smithi096 ceph-mon[34468]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-05T07:05:27.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:27 smithi115 ceph-mon[34791]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-05T07:05:27.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:27 smithi086 ceph-mon[30943]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-05T07:05:28.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75764 6.20 MB/sec execute 526 sec latency 109.839 ms 2024-09-05T07:05:29.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:05:29.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:29 smithi086 ceph-mon[30943]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 333 op/s 2024-09-05T07:05:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:29.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75889 6.20 MB/sec execute 527 sec latency 135.453 ms 2024-09-05T07:05:29.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:29 smithi096 ceph-mon[34468]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 333 op/s 2024-09-05T07:05:29.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:29 smithi115 ceph-mon[34791]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 333 op/s 2024-09-05T07:05:30.792 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76054 6.21 MB/sec execute 528 sec latency 60.197 ms 2024-09-05T07:05:31.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76197 6.21 MB/sec execute 529 sec latency 115.981 ms 2024-09-05T07:05:31.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:31 smithi086 ceph-mon[30943]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 214 op/s 2024-09-05T07:05:31.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:31 smithi096 ceph-mon[34468]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 214 op/s 2024-09-05T07:05:31.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:31 smithi115 ceph-mon[34791]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 214 op/s 2024-09-05T07:05:32.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76328 6.21 MB/sec execute 530 sec latency 152.265 ms 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 -- 172.21.15.86:0/246487212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 msgr2=0x7f743c116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 --2- 172.21.15.86:0/246487212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c116ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f742c00fb60 tx=0x7f742c032970 comp rx=0 tx=0).stop 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 -- 172.21.15.86:0/246487212 shutdown_connections 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 --2- 172.21.15.86:0/246487212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c116ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 --2- 172.21.15.86:0/246487212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c109b10 0x7f743c114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 --2- 172.21.15.86:0/246487212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c1091f0 0x7f743c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.879+0000 7f7440f16640 1 -- 172.21.15.86:0/246487212 >> 172.21.15.86:0/246487212 conn(0x7f743c06e140 msgr2=0x7f743c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.880+0000 7f7440f16640 1 -- 172.21.15.86:0/246487212 shutdown_connections 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.880+0000 7f7440f16640 1 -- 172.21.15.86:0/246487212 wait complete. 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 Processor -- start 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 -- start start 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c1091f0 0x7f743c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1184a0 con 0x7f743c1091f0 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f743c118320 con 0x7f743c109b10 2024-09-05T07:05:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7440f16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f743c118620 con 0x7f743c114ab0 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f743a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c1091f0 0x7f743c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52288/0 (socket says 172.21.15.86:52288) 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f7439d9b640 1 -- 172.21.15.86:0/3140026653 learned_addr learned my addr 172.21.15.86:0/3140026653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.881+0000 7f743ad9d640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 -- 172.21.15.86:0/3140026653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 msgr2=0x7f743c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 -- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c1091f0 msgr2=0x7f743c1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c1091f0 0x7f743c1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 -- 172.21.15.86:0/3140026653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c19ef30 con 0x7f743c109b10 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7439d9b640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f743000e9c0 tx=0x7f743000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f742b7fe640 1 -- 172.21.15.86:0/3140026653 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000cde0 con 0x7f743c109b10 2024-09-05T07:05:33.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f742b7fe640 1 -- 172.21.15.86:0/3140026653 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74300035f0 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c19f220 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.882+0000 7f742b7fe640 1 -- 172.21.15.86:0/3140026653 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430010690 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.883+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1ab470 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.885+0000 7f742b7fe640 1 -- 172.21.15.86:0/3140026653 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74300037a0 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.886+0000 7f74432ef740 1 Processor -- start 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.886+0000 7f742b7fe640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 0x7f74240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f742b7fe640 1 -- 172.21.15.86:0/3140026653 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f74300cfb70 con 0x7f743c109b10 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 -- start start 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f743ad9d640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 0x7f74240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa77e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa77ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa789370 0x5597fa78b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597fa6e42f0 con 0x5597fa77fb40 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597fa68bcf0 con 0x5597fa789370 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f74432ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5597fa6e02b0 con 0x5597fa77e5d0 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f743a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa77e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f743a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa77e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43822/0 (socket says 172.21.15.86:43822) 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f743a59c640 1 -- 172.21.15.86:0/1258006448 learned_addr learned my addr 172.21.15.86:0/1258006448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.887+0000 7f7439d9b640 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa77ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f743ad9d640 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa789370 0x5597fa78b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 -- 172.21.15.86:0/1258006448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa789370 msgr2=0x5597fa78b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa789370 0x5597fa78b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 -- 172.21.15.86:0/1258006448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 msgr2=0x5597fa77e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa77e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 -- 172.21.15.86:0/1258006448 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fa6703a0 con 0x5597fa77e5d0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7439d9b640 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa77ea10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74300cce20 tx=0x7f743009b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add9c9c7b33829af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f743ad9d640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 0x7f74240a5230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f742c00f760 tx=0x7f742c039f40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7428ff9640 1 -- 172.21.15.86:0/1258006448 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300b2070 con 0x5597fa77e5d0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7428ff9640 1 -- 172.21.15.86:0/1258006448 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f743009c2f0 con 0x5597fa77e5d0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.888+0000 7f7428ff9640 1 -- 172.21.15.86:0/1258006448 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430098400 con 0x5597fa77e5d0 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 -- 172.21.15.86:0/1258006448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 msgr2=0x5597fa77ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa77ea10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74300cce20 tx=0x7f743009b610 comp rx=0 tx=0).stop 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 -- 172.21.15.86:0/1258006448 shutdown_connections 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa789370 0x5597fa78b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa77ea10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 --2- 172.21.15.86:0/1258006448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa77e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 -- 172.21.15.86:0/1258006448 >> 172.21.15.86:0/1258006448 conn(0x5597fa747310 msgr2=0x5597fa77ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 -- 172.21.15.86:0/1258006448 shutdown_connections 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.889+0000 7f74432ef740 1 -- 172.21.15.86:0/1258006448 wait complete. 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 -- start start 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 Processor -- start 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa7867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597fa78c010 con 0x5597fa789370 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597fa69f060 con 0x5597fa77fb40 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5597fa78c190 con 0x5597fa77e5d0 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f74432ef740 10 client.0 _ll_register_callbacks cb 0x5597fa3f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f743a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52302/0 (socket says 172.21.15.86:52302) 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f743a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35774/0 (socket says 172.21.15.86:35774) 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f7439d9b640 1 -- 172.21.15.86:0/4194591066 learned_addr learned my addr 172.21.15.86:0/4194591066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.890+0000 7f743ad9d640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa7867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 -- 172.21.15.86:0/4194591066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 msgr2=0x5597fa788da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 -- 172.21.15.86:0/4194591066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 msgr2=0x5597fa7867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa7867e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 -- 172.21.15.86:0/4194591066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fa4553c0 con 0x5597fa77fb40 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f743a59c640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7439d9b640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f743009bb80 tx=0x7f7430097f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad2e57bcc65376c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300b2070 con 0x5597fa77fb40 2024-09-05T07:05:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f743009c2f0 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f7430099ef0 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f74300ba050 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f74432ef740 10 client.29342 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5597fa797af0 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.891+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f743009c2f0 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f743009aab0 con 0x5597fa77fb40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 10 client.29342 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 10 client.29342 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 20 client.29342 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa798e40 0x5597fa79b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 4 client.29342 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fa6f1860 con 0x5597fa798e40 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f743a59c640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa798e40 0x5597fa79b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f7421ffb640 10 client.29342 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.892+0000 7f743a59c640 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa798e40 0x5597fa79b230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7ec7ce32e4854b5 server_cookie=43c67fc02ac1799e in_seq=0 out_seq=0 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f7421ffb640 1 -- 172.21.15.86:0/4194591066 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597fa6f1860 con 0x5597fa798e40 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f7421ffb640 10 client.29342 handle_command_reply: tid=0 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 client.29342 shutdown 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 msgr2=0x5597fa786050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f743009bb80 tx=0x7f7430097f00 comp rx=0 tx=0).stop 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 shutdown_connections 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa798e40 0x5597fa79b230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f7421ffb640 0 client.29342 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:05:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa7867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77fb40 0x5597fa786050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 --2- 172.21.15.86:0/4194591066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77e5d0 0x5597fa788da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 >> 172.21.15.86:0/4194591066 conn(0x5597fa747310 msgr2=0x5597fa780960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 shutdown_connections 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 1 -- 172.21.15.86:0/4194591066 wait complete. 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.893+0000 7f74432ef740 20 client.29342 trim_cache size 0 max 16384 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 Processor -- start 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 -- start start 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa785910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 0x5597fa785050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa79e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b57d0 con 0x5597fa789370 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b7310 con 0x5597fa77e5d0 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f74432ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b5ea0 con 0x5597fa77fb40 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 0x5597fa785050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 0x5597fa785050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35780/0 (socket says 172.21.15.86:35780) 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f7439d9b640 1 -- 172.21.15.86:0/1743010974 learned_addr learned my addr 172.21.15.86:0/1743010974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f743ad9d640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa79e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.911+0000 7f743a59c640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa785910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 -- 172.21.15.86:0/1743010974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 msgr2=0x5597fa785050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 0x5597fa785050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 -- 172.21.15.86:0/1743010974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 msgr2=0x5597fa785910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa785910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 -- 172.21.15.86:0/1743010974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fa781120 con 0x5597fa789370 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743a59c640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa785910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f743ad9d640 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa79e250 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f742c032e90 tx=0x7f742c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ce88cd69d766d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f7423fff640 1 -- 172.21.15.86:0/1743010974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c037d80 con 0x5597fa789370 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f7423fff640 1 -- 172.21.15.86:0/1743010974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f742c0054e0 con 0x5597fa789370 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.912+0000 7f7423fff640 1 -- 172.21.15.86:0/1743010974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c004b60 con 0x5597fa789370 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 -- 172.21.15.86:0/1743010974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 msgr2=0x5597fa79e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa79e250 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f742c032e90 tx=0x7f742c04e000 comp rx=0 tx=0).stop 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 -- 172.21.15.86:0/1743010974 shutdown_connections 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa789370 0x5597fa79e250 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa77fb40 0x5597fa785050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.913+0000 7f74432ef740 1 --2- 172.21.15.86:0/1743010974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa785910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 -- 172.21.15.86:0/1743010974 >> 172.21.15.86:0/1743010974 conn(0x5597fa747310 msgr2=0x5597fa784940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 -- 172.21.15.86:0/1743010974 shutdown_connections 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 -- 172.21.15.86:0/1743010974 wait complete. 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 -- start start 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 Processor -- start 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.914+0000 7f74432ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa79bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f74432ef740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa789370 0x5597fa79e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f74432ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b6c40 con 0x5597fa77fb40 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa79bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f74432ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b6570 con 0x5597fa77e5d0 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f74432ef740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b37d0 con 0x5597fa789370 2024-09-05T07:05:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f743a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f74432ef740 10 client.0 _ll_register_callbacks cb 0x7f7434009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f743a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52318/0 (socket says 172.21.15.86:52318) 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f743a59c640 1 -- 172.21.15.86:0/2764005028 learned_addr learned my addr 172.21.15.86:0/2764005028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f743ad9d640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa789370 0x5597fa79e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 -- 172.21.15.86:0/2764005028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa789370 msgr2=0x5597fa79e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa789370 0x5597fa79e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 -- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 msgr2=0x5597fa79ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 -- 172.21.15.86:0/2764005028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fa79e6e0 con 0x5597fa77fb40 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f743a59c640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa79bf90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f743000c6a0 tx=0x7f7430000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1d14682ddc2bce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.915+0000 7f7421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300b2070 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f743009b040 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742c0054e0 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f743009c2f0 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f74432ef740 10 client.20694 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5597fa8032d0 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f742c0054e0 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.916+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f74300a4ab0 con 0x5597fa77fb40 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 10 client.20694 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 10 client.20694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 20 client.20694 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa803760 0x5597fa805b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 4 client.20694 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597fa6ebd70 con 0x5597fa803760 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f743a59c640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa803760 0x5597fa805b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f7421ffb640 10 client.20694 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f743a59c640 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa803760 0x5597fa805b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32df0231d296efef server_cookie=705e5587410acb2d in_seq=0 out_seq=0 2024-09-05T07:05:33.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f7421ffb640 1 -- 172.21.15.86:0/2764005028 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597fa6ebd70 con 0x5597fa803760 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f7421ffb640 10 client.20694 handle_command_reply: tid=0 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.917+0000 7f74432ef740 1 client.20694 shutdown 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 msgr2=0x5597fa79bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa79bf90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f743000c6a0 tx=0x7f7430000bc0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 shutdown_connections 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x5597fa803760 0x5597fa805b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5597fa789370 0x5597fa79e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597fa77fb40 0x5597fa79bf90 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 --2- 172.21.15.86:0/2764005028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597fa77e5d0 0x5597fa79ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 >> 172.21.15.86:0/2764005028 conn(0x5597fa747310 msgr2=0x5597fa794420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f7421ffb640 0 client.20694 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 shutdown_connections 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 1 -- 172.21.15.86:0/2764005028 wait complete. 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.918+0000 7f74432ef740 20 client.20694 trim_cache size 0 max 16384 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 msgr2=0x7f74240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 0x7f74240a5230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f742c00f760 tx=0x7f742c039f40 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 msgr2=0x7f743c1a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f743000e9c0 tx=0x7f743000ee80 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 shutdown_connections 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74240a2d70 0x7f74240a5230 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f743c114ab0 0x7f743c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f743c109b10 0x7f743c1a4b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 --2- 172.21.15.86:0/3140026653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c1091f0 0x7f743c1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 >> 172.21.15.86:0/3140026653 conn(0x7f743c06e140 msgr2=0x7f743c070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 shutdown_connections 2024-09-05T07:05:33.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:32.919+0000 7f7440f16640 1 -- 172.21.15.86:0/3140026653 wait complete. 2024-09-05T07:05:33.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76476 6.21 MB/sec execute 531 sec latency 97.507 ms 2024-09-05T07:05:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:33 smithi096 ceph-mon[34468]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 299 op/s 2024-09-05T07:05:33.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:33 smithi115 ceph-mon[34791]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 299 op/s 2024-09-05T07:05:33.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:33 smithi086 ceph-mon[30943]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.6 MiB/s wr, 299 op/s 2024-09-05T07:05:34.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:05:34.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76619 6.21 MB/sec execute 532 sec latency 225.592 ms 2024-09-05T07:05:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:35 smithi086 ceph-mon[30943]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:05:35.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76761 6.22 MB/sec execute 533 sec latency 176.291 ms 2024-09-05T07:05:35.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:35 smithi096 ceph-mon[34468]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:05:35.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:35 smithi115 ceph-mon[34791]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 -- 172.21.15.86:0/3569473021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102570 msgr2=0x7fb0ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/3569473021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102570 0x7fb0ec1029b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02a480 comp rx=0 tx=0).stop 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 -- 172.21.15.86:0/3569473021 shutdown_connections 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/3569473021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0ec102ef0 0x7fb0ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/3569473021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102570 0x7fb0ec1029b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/3569473021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.756+0000 7fb0f35cb640 1 -- 172.21.15.86:0/3569473021 >> 172.21.15.86:0/3569473021 conn(0x7fb0ec0fc7d0 msgr2=0x7fb0ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.757+0000 7fb0f35cb640 1 -- 172.21.15.86:0/3569473021 shutdown_connections 2024-09-05T07:05:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.757+0000 7fb0f35cb640 1 -- 172.21.15.86:0/3569473021 wait complete. 2024-09-05T07:05:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.758+0000 7fb0f35cb640 1 Processor -- start 2024-09-05T07:05:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.758+0000 7fb0f35cb640 1 -- start start 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.758+0000 7fb0f35cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f35cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0ec102570 0x7fb0ec1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f35cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102ef0 0x7fb0ec19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f35cb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec110e30 con 0x7fb0ec102ef0 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f35cb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec110cb0 con 0x7fb0ec100e20 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f35cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec110fb0 con 0x7fb0ec102570 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52342/0 (socket says 172.21.15.86:52342) 2024-09-05T07:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 -- 172.21.15.86:0/1739346554 learned_addr learned my addr 172.21.15.86:0/1739346554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 -- 172.21.15.86:0/1739346554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0ec102570 msgr2=0x7fb0ec1989a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0ec102570 0x7fb0ec1989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 -- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102ef0 msgr2=0x7fb0ec19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102ef0 0x7fb0ec19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 -- 172.21.15.86:0/1739346554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec19d410 con 0x7fb0ec100e20 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.759+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800d950 tx=0x7fb0e800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.760+0000 7fb0e27fc640 1 -- 172.21.15.86:0/1739346554 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8017070 con 0x7fb0ec100e20 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.760+0000 7fb0e27fc640 1 -- 172.21.15.86:0/1739346554 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0e8009c30 con 0x7fb0ec100e20 2024-09-05T07:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.760+0000 7fb0e27fc640 1 -- 172.21.15.86:0/1739346554 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800f620 con 0x7fb0ec100e20 2024-09-05T07:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.760+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec19d700 con 0x7fb0ec100e20 2024-09-05T07:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.760+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec0fcbb0 con 0x7fb0ec100e20 2024-09-05T07:05:36.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.763+0000 7fb0e27fc640 1 -- 172.21.15.86:0/1739346554 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb0e80109a0 con 0x7fb0ec100e20 2024-09-05T07:05:36.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 Processor -- start 2024-09-05T07:05:36.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 -- start start 2024-09-05T07:05:36.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0e27fc640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 0x7fb0d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad33445d0 0x55dad3344a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad334f370 0x55dad3351760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0e27fc640 1 -- 172.21.15.86:0/1739346554 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb0e80cf510 con 0x7fb0ec100e20 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dad32aa2f0 con 0x55dad334f370 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dad3251cf0 con 0x55dad3345b40 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dad32a62b0 con 0x55dad33445d0 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.764+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 0x7fb0d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52358/0 (socket says 172.21.15.86:52358) 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 -- 172.21.15.86:0/3818216585 learned_addr learned my addr 172.21.15.86:0/3818216585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 -- 172.21.15.86:0/3818216585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad33445d0 msgr2=0x55dad3344a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad33445d0 0x55dad3344a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 -- 172.21.15.86:0/3818216585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad334f370 msgr2=0x55dad3351760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad334f370 0x55dad3351760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 -- 172.21.15.86:0/3818216585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad32363a0 con 0x55dad3345b40 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1367640 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80a3d30 tx=0x7fb0e809bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4b600569ede7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 0x7fb0d80a5230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40092f0 tx=0x7fb0e4009220 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0d77fe640 1 -- 172.21.15.86:0/3818216585 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8097050 con 0x55dad3345b40 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0d77fe640 1 -- 172.21.15.86:0/3818216585 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0e809f9a0 con 0x55dad3345b40 2024-09-05T07:05:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.765+0000 7fb0d77fe640 1 -- 172.21.15.86:0/3818216585 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8020040 con 0x55dad3345b40 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 -- 172.21.15.86:0/3818216585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 msgr2=0x55dad3344090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80a3d30 tx=0x7fb0e809bcf0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 -- 172.21.15.86:0/3818216585 shutdown_connections 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad334f370 0x55dad3351760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad33445d0 0x55dad3344a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/3818216585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad3345b40 0x55dad3344090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.766+0000 7fb0f59a4740 1 -- 172.21.15.86:0/3818216585 >> 172.21.15.86:0/3818216585 conn(0x55dad330d310 msgr2=0x55dad3340c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- 172.21.15.86:0/3818216585 shutdown_connections 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- 172.21.15.86:0/3818216585 wait complete. 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- start start 2024-09-05T07:05:36.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 Processor -- start 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad334c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dad3352070 con 0x55dad33445d0 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dad3265060 con 0x55dad334f370 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.767+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dad33521f0 con 0x55dad3345b40 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f59a4740 10 client.0 _ll_register_callbacks cb 0x55dad2fb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35830/0 (socket says 172.21.15.86:35830) 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 -- 172.21.15.86:0/1256846245 learned_addr learned my addr 172.21.15.86:0/1256846245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad334c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 -- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 msgr2=0x55dad334c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad334c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 -- 172.21.15.86:0/1256846245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 msgr2=0x55dad334edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 -- 172.21.15.86:0/1256846245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad301b3c0 con 0x55dad3345b40 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad334c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0f0b66640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc02a4c0 tx=0x7fb0dc031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc904affd81a8334 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.768+0000 7fb0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:05:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.769+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc03e030 con 0x55dad3345b40 2024-09-05T07:05:36.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.769+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e809f9a0 con 0x55dad3345b40 2024-09-05T07:05:36.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.769+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb0dc004040 con 0x55dad3345b40 2024-09-05T07:05:36.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.770+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc04b050 con 0x55dad3345b40 2024-09-05T07:05:36.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.770+0000 7fb0f59a4740 10 client.29413 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:36.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.770+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dad335dbe0 con 0x55dad3345b40 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0e809f9a0 con 0x55dad3345b40 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fb0dc004730 con 0x55dad3345b40 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0f59a4740 10 client.29413 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0f59a4740 10 client.29413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0f59a4740 20 client.29413 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:36.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad3361420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.772+0000 7fb0f59a4740 4 client.29413 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.773+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dad32b7860 con 0x55dad335eff0 2024-09-05T07:05:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.773+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad3361420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.773+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad3361420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=992d62808088a34f server_cookie=1f487ba52239d015 in_seq=0 out_seq=0 2024-09-05T07:05:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.773+0000 7fb0d4ff9640 10 client.29413 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:05:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.774+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1256846245 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dad32b7860 con 0x55dad335eff0 2024-09-05T07:05:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.774+0000 7fb0d4ff9640 10 client.29413 handle_command_reply: tid=0 2024-09-05T07:05:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.774+0000 7fb0f59a4740 1 client.29413 shutdown 2024-09-05T07:05:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.776+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 msgr2=0x55dad334c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.776+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc02a4c0 tx=0x7fb0dc031660 comp rx=0 tx=0).stop 2024-09-05T07:05:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.776+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 shutdown_connections 2024-09-05T07:05:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.776+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad3361420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.776+0000 7fb0d4ff9640 0 client.29413 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad334c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad334c080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1256846245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 >> 172.21.15.86:0/1256846245 conn(0x55dad330d310 msgr2=0x55dad33469c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 shutdown_connections 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1256846245 wait complete. 2024-09-05T07:05:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.777+0000 7fb0f59a4740 20 client.29413 trim_cache size 0 max 16384 2024-09-05T07:05:36.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76924 6.22 MB/sec execute 534 sec latency 127.873 ms 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 Processor -- start 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 -- start start 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 0x55dad334ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1b1440 con 0x55dad3345b40 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1b2f80 con 0x55dad33445d0 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f59a4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1b1b10 con 0x55dad334f370 2024-09-05T07:05:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f0b66640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f0b66640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43882/0 (socket says 172.21.15.86:43882) 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f0b66640 1 -- 172.21.15.86:0/2988507018 learned_addr learned my addr 172.21.15.86:0/2988507018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.799+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 0x55dad334ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1367640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 -- 172.21.15.86:0/2988507018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad33445d0 msgr2=0x55dad334edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 -- 172.21.15.86:0/2988507018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 msgr2=0x55dad334a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 -- 172.21.15.86:0/2988507018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad334cb80 con 0x55dad334f370 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f0b66640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.800+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 0x55dad334ae90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55dad3345070 tx=0x7fb0e4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79340322a6dd3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.801+0000 7fb0d6ffd640 1 -- 172.21.15.86:0/2988507018 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4022070 con 0x55dad334f370 2024-09-05T07:05:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.801+0000 7fb0d6ffd640 1 -- 172.21.15.86:0/2988507018 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0e4017040 con 0x55dad334f370 2024-09-05T07:05:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.801+0000 7fb0d6ffd640 1 -- 172.21.15.86:0/2988507018 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4010990 con 0x55dad334f370 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 -- 172.21.15.86:0/2988507018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 msgr2=0x55dad334ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 0x55dad334ae90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55dad3345070 tx=0x7fb0e4011000 comp rx=0 tx=0).stop 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 -- 172.21.15.86:0/2988507018 shutdown_connections 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad334f370 0x55dad334ae90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad3345b40 0x55dad334a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/2988507018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad33445d0 0x55dad334edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 -- 172.21.15.86:0/2988507018 >> 172.21.15.86:0/2988507018 conn(0x55dad330d310 msgr2=0x55dad333ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 -- 172.21.15.86:0/2988507018 shutdown_connections 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.802+0000 7fb0f59a4740 1 -- 172.21.15.86:0/2988507018 wait complete. 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.803+0000 7fb0f59a4740 1 -- start start 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.803+0000 7fb0f59a4740 1 Processor -- start 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.803+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad3348b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f1367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43886/0 (socket says 172.21.15.86:43886) 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad3349050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f1367640 1 -- 172.21.15.86:0/1517835701 learned_addr learned my addr 172.21.15.86:0/1517835701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1b28b0 con 0x55dad33445d0 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1b21e0 con 0x55dad334f370 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1af440 con 0x55dad3345b40 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad3349050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f0b66640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad3348b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.804+0000 7fb0f59a4740 10 client.0 _ll_register_callbacks cb 0x7fb0dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 -- 172.21.15.86:0/1517835701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 msgr2=0x55dad3348b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad3348b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 -- 172.21.15.86:0/1517835701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 msgr2=0x55dad3357900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad3349590 con 0x55dad334f370 2024-09-05T07:05:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0f1b68640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad3349050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4000f80 tx=0x7fb0e4010e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa59d7f4d77e9ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.805+0000 7fb0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:05:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.806+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4035060 con 0x55dad334f370 2024-09-05T07:05:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.806+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fb0e401e620 con 0x55dad334f370 2024-09-05T07:05:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.806+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e4017040 con 0x55dad334f370 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.807+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb0e403c050 con 0x55dad334f370 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0f59a4740 10 client.29363 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dad33c9970 con 0x55dad334f370 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0e4017040 con 0x55dad334f370 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fb0e400f040 con 0x55dad334f370 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0f59a4740 10 client.29363 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.808+0000 7fb0f59a4740 10 client.29363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:05:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.809+0000 7fb0f59a4740 20 client.29363 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.809+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad33cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.809+0000 7fb0f59a4740 4 client.29363 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:05:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.809+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dad32b1d70 con 0x55dad335eff0 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad33cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0f1367640 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad33cc050 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66fc1eeec8b6ae52 server_cookie=b2d1984c7187f03a in_seq=0 out_seq=0 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0d4ff9640 10 client.29363 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0d4ff9640 1 -- 172.21.15.86:0/1517835701 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dad32b1d70 con 0x55dad335eff0 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0d4ff9640 10 client.29363 handle_command_reply: tid=0 2024-09-05T07:05:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.811+0000 7fb0f59a4740 1 client.29363 shutdown 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 msgr2=0x55dad3349050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad3349050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4000f80 tx=0x7fb0e4010e80 comp rx=0 tx=0).stop 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 shutdown_connections 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55dad335eff0 0x55dad33cc050 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dad334f370 0x55dad3349050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.812+0000 7fb0d4ff9640 0 client.29363 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dad3345b40 0x55dad3348b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 1 --2- 172.21.15.86:0/1517835701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dad33445d0 0x55dad3357900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 >> 172.21.15.86:0/1517835701 conn(0x55dad330d310 msgr2=0x55dad334d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 shutdown_connections 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 1 -- 172.21.15.86:0/1517835701 wait complete. 2024-09-05T07:05:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.813+0000 7fb0f59a4740 20 client.29363 trim_cache size 0 max 16384 2024-09-05T07:05:36.816 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:05:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 msgr2=0x7fb0d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 0x7fb0d80a5230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40092f0 tx=0x7fb0e4009220 comp rx=0 tx=0).stop 2024-09-05T07:05:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 msgr2=0x7fb0ec198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800d950 tx=0x7fb0e800de10 comp rx=0 tx=0).stop 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 shutdown_connections 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fb0d80a2d70 0x7fb0d80a5230 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec102ef0 0x7fb0ec19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0ec102570 0x7fb0ec1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 --2- 172.21.15.86:0/1739346554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ec100e20 0x7fb0ec198460 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.814+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 >> 172.21.15.86:0/1739346554 conn(0x7fb0ec0fc7d0 msgr2=0x7fb0ec06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.815+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 shutdown_connections 2024-09-05T07:05:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:36.815+0000 7fb0f35cb640 1 -- 172.21.15.86:0/1739346554 wait complete. 2024-09-05T07:05:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:37 smithi086 ceph-mon[30943]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 210 op/s 2024-09-05T07:05:37.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77087 6.23 MB/sec execute 535 sec latency 96.031 ms 2024-09-05T07:05:37.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:05:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:37 smithi096 ceph-mon[34468]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 210 op/s 2024-09-05T07:05:37.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:37 smithi115 ceph-mon[34791]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 210 op/s 2024-09-05T07:05:38.238 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:38.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77234 6.24 MB/sec execute 536 sec latency 129.887 ms 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.804+0000 7fac4168a640 1 -- 172.21.15.86:0/3834788012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c102790 msgr2=0x7fac3c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.804+0000 7fac4168a640 1 --2- 172.21.15.86:0/3834788012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c102790 0x7fac3c1067e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3002a410 comp rx=0 tx=0).stop 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 -- 172.21.15.86:0/3834788012 shutdown_connections 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 --2- 172.21.15.86:0/3834788012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c106f10 0x7fac3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 --2- 172.21.15.86:0/3834788012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c102790 0x7fac3c1067e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 --2- 172.21.15.86:0/3834788012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c101de0 0x7fac3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 -- 172.21.15.86:0/3834788012 >> 172.21.15.86:0/3834788012 conn(0x7fac3c0fd8c0 msgr2=0x7fac3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 -- 172.21.15.86:0/3834788012 shutdown_connections 2024-09-05T07:05:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.805+0000 7fac4168a640 1 -- 172.21.15.86:0/3834788012 wait complete. 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 Processor -- start 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 -- start start 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c106f10 0x7fac3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 0x7fac3c1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111e70 con 0x7fac3c106f10 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111cf0 con 0x7fac3c074f70 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac4168a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111ff0 con 0x7fac3c101de0 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:35852/0 (socket says 172.21.15.86:35852) 2024-09-05T07:05:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac3affd640 1 -- 172.21.15.86:0/49029450 learned_addr learned my addr 172.21.15.86:0/49029450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 0x7fac3c1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.806+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c106f10 0x7fac3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 -- 172.21.15.86:0/49029450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 msgr2=0x7fac3c1a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 0x7fac3c1a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 -- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c106f10 msgr2=0x7fac3c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c106f10 0x7fac3c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 -- 172.21.15.86:0/49029450 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1a6e30 con 0x7fac3c101de0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 0x7fac3c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac3affd640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ea20 tx=0x7fac2c00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac23fff640 1 -- 172.21.15.86:0/49029450 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00cd80 con 0x7fac3c101de0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac23fff640 1 -- 172.21.15.86:0/49029450 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c0108e0 con 0x7fac3c101de0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac23fff640 1 -- 172.21.15.86:0/49029450 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c018840 con 0x7fac3c101de0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c1a7120 con 0x7fac3c101de0 2024-09-05T07:05:38.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.807+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a7520 con 0x7fac3c101de0 2024-09-05T07:05:38.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.810+0000 7fac23fff640 1 -- 172.21.15.86:0/49029450 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fac2c010a90 con 0x7fac3c101de0 2024-09-05T07:05:38.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac23fff640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 0x7fac240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 Processor -- start 2024-09-05T07:05:38.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac23fff640 1 -- 172.21.15.86:0/49029450 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fac2c0cfbf0 con 0x7fac3c101de0 2024-09-05T07:05:38.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 0x7fac240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 -- start start 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c153fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15409f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c1535a2f0 con 0x562c153fe350 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562c15301cf0 con 0x562c153fcde0 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.811+0000 7fac43a63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562c153562b0 con 0x562c15407b80 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15409f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43904/0 (socket says 172.21.15.86:43904) 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 -- 172.21.15.86:0/1801415346 learned_addr learned my addr 172.21.15.86:0/1801415346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 0x7fac240a51b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac30006010 tx=0x7fac30005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c153fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 -- 172.21.15.86:0/1801415346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 msgr2=0x562c15409f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15409f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 -- 172.21.15.86:0/1801415346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 msgr2=0x562c153fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c153fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 -- 172.21.15.86:0/1801415346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c152e63a0 con 0x562c153fe350 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac3affd640 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0986f0 tx=0x7fac2c09bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c2624cba814952 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac217fa640 1 -- 172.21.15.86:0/1801415346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0a48e0 con 0x562c153fe350 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac217fa640 1 -- 172.21.15.86:0/1801415346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c0cd120 con 0x562c153fe350 2024-09-05T07:05:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.812+0000 7fac217fa640 1 -- 172.21.15.86:0/1801415346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c097cd0 con 0x562c153fe350 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 -- 172.21.15.86:0/1801415346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 msgr2=0x562c153fc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0986f0 tx=0x7fac2c09bd20 comp rx=0 tx=0).stop 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 -- 172.21.15.86:0/1801415346 shutdown_connections 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15409f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c153fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 --2- 172.21.15.86:0/1801415346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c153fc8a0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.813+0000 7fac43a63740 1 -- 172.21.15.86:0/1801415346 >> 172.21.15.86:0/1801415346 conn(0x562c153bd310 msgr2=0x562c153f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 -- 172.21.15.86:0/1801415346 shutdown_connections 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 -- 172.21.15.86:0/1801415346 wait complete. 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 -- start start 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 Processor -- start 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c15407570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c15404820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.814+0000 7fac43a63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15404e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac43a63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c1540a820 con 0x562c153fcde0 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac43a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562c15315060 con 0x562c153fe350 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac43a63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562c1540a9a0 con 0x562c15407b80 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c15404820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac43a63740 10 client.0 _ll_register_callbacks cb 0x562c150688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c15407570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c15404820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52416/0 (socket says 172.21.15.86:52416) 2024-09-05T07:05:38.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3a7fc640 1 -- 172.21.15.86:0/1261276060 learned_addr learned my addr 172.21.15.86:0/1261276060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15404e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 -- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 msgr2=0x562c15404e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15404e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 -- 172.21.15.86:0/1261276060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 msgr2=0x562c15404820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c15404820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 -- 172.21.15.86:0/1261276060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c150cb3c0 con 0x562c153fcde0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac3affd640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c15407570 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x562c153fe950 tx=0x7fac2c0978c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def14087e5c5b47c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0b8070 con 0x562c153fcde0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.815+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fac2c010640 con 0x562c153fcde0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac2c0cd120 con 0x562c153fcde0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fac2c0a48e0 con 0x562c153fcde0 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 10 client.20718 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:38.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562c154162c0 con 0x562c153fcde0 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c0cd120 con 0x562c153fcde0 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fac2c09abe0 con 0x562c153fcde0 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 10 client.20718 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 10 client.20718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 20 client.20718 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c15419a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 4 client.20718 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.816+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c15367860 con 0x562c15417610 2024-09-05T07:05:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.817+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c15419a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.817+0000 7fac1eff5640 10 client.20718 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.817+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c15419a00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=226bbae913c9e8f5 server_cookie=83eb2b2e791599cd in_seq=0 out_seq=0 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac1eff5640 1 -- 172.21.15.86:0/1261276060 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c15367860 con 0x562c15417610 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac1eff5640 10 client.20718 handle_command_reply: tid=0 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 client.20718 shutdown 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 msgr2=0x562c15407570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c15407570 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x562c153fe950 tx=0x7fac2c0978c0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 shutdown_connections 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c15419a00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac1eff5640 0 client.20718 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15404e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c15404820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 --2- 172.21.15.86:0/1261276060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c15407570 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 >> 172.21.15.86:0/1261276060 conn(0x562c153bd310 msgr2=0x562c153ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 shutdown_connections 2024-09-05T07:05:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.818+0000 7fac43a63740 1 -- 172.21.15.86:0/1261276060 wait complete. 2024-09-05T07:05:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.819+0000 7fac43a63740 20 client.20718 trim_cache size 0 max 16384 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 Processor -- start 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 -- start start 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c1541cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15401a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1ad600 con 0x562c153fcde0 2024-09-05T07:05:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1af140 con 0x562c153fe350 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac43a63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1adcd0 con 0x562c15407b80 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac3a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c1541cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac3affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43920/0 (socket says 172.21.15.86:43920) 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac3affd640 1 -- 172.21.15.86:0/3525231992 learned_addr learned my addr 172.21.15.86:0/3525231992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.836+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15401a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 -- 172.21.15.86:0/3525231992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 msgr2=0x562c15401a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15401a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 -- 172.21.15.86:0/3525231992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 msgr2=0x562c1541ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 -- 172.21.15.86:0/3525231992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c15407190 con 0x562c153fe350 2024-09-05T07:05:38.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15401a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:38.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3affd640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:38.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c1541cf60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fac30031440 tx=0x7fac30040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d725a726f56eb373 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac20ff9640 1 -- 172.21.15.86:0/3525231992 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30058070 con 0x562c153fe350 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac20ff9640 1 -- 172.21.15.86:0/3525231992 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac30004770 con 0x562c153fe350 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.837+0000 7fac20ff9640 1 -- 172.21.15.86:0/3525231992 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30035040 con 0x562c153fe350 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 -- 172.21.15.86:0/3525231992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 msgr2=0x562c1541cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c1541cf60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fac30031440 tx=0x7fac30040000 comp rx=0 tx=0).stop 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 -- 172.21.15.86:0/3525231992 shutdown_connections 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c15401a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fe350 0x562c1541cf60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 --2- 172.21.15.86:0/3525231992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fcde0 0x562c1541ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.838+0000 7fac43a63740 1 -- 172.21.15.86:0/3525231992 >> 172.21.15.86:0/3525231992 conn(0x562c153bd310 msgr2=0x562c154068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 -- 172.21.15.86:0/3525231992 shutdown_connections 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 -- 172.21.15.86:0/3525231992 wait complete. 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 -- start start 2024-09-05T07:05:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 Processor -- start 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c1541ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c1541b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.839+0000 7fac43a63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1aea70 con 0x562c153fe350 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac43a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1ae3a0 con 0x562c153fcde0 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac43a63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1ab620 con 0x562c15407b80 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac43a63740 10 client.0 _ll_register_callbacks cb 0x7fac34009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:52436/0 (socket says 172.21.15.86:52436) 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 -- 172.21.15.86:0/1848087556 learned_addr learned my addr 172.21.15.86:0/1848087556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c1541ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3b7fe640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c1541b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 -- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 msgr2=0x562c1541b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c1541b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 -- 172.21.15.86:0/1848087556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 msgr2=0x562c1541ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c1541ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 -- 172.21.15.86:0/1848087556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c15481000 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c1541ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac3affd640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00e9f0 tx=0x7fac2c00ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f466f9395776bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.840+0000 7fac1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0b8040 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fac2c0cd120 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac30004770 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fac2c0cd320 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac43a63740 10 client.29381 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562c15481000 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.841+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac30004770 con 0x562c153fcde0 2024-09-05T07:05:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fac2c0a4bc0 con 0x562c153fcde0 2024-09-05T07:05:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 10 client.29381 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 10 client.29381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:05:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 20 client.29381 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c154845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 4 client.29381 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c15361d70 con 0x562c15417610 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c154845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac3a7fc640 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c154845f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce413c040094c936 server_cookie=17c8e222abc9a12b in_seq=0 out_seq=0 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.842+0000 7fac1eff5640 10 client.29381 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac1eff5640 1 -- 172.21.15.86:0/1848087556 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c15361d70 con 0x562c15417610 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac1eff5640 10 client.29381 handle_command_reply: tid=0 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 client.29381 shutdown 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 msgr2=0x562c1541a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00e9f0 tx=0x7fac2c00ef50 comp rx=0 tx=0).stop 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 shutdown_connections 2024-09-05T07:05:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x562c15417610 0x562c154845f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562c15407b80 0x562c1541b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c153fe350 0x562c1541ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 --2- 172.21.15.86:0/1848087556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c153fcde0 0x562c1541a850 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 >> 172.21.15.86:0/1848087556 conn(0x562c153bd310 msgr2=0x562c15406080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac1eff5640 0 client.29381 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 shutdown_connections 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 1 -- 172.21.15.86:0/1848087556 wait complete. 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.843+0000 7fac43a63740 20 client.29381 trim_cache size 0 max 16384 2024-09-05T07:05:38.848 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 msgr2=0x7fac240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 0x7fac240a51b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac30006010 tx=0x7fac30005a20 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 msgr2=0x7fac3c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ea20 tx=0x7fac2c00eee0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 shutdown_connections 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fac240a2cf0 0x7fac240a51b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac3c074f70 0x7fac3c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c106f10 0x7fac3c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 --2- 172.21.15.86:0/49029450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac3c101de0 0x7fac3c0744f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 >> 172.21.15.86:0/49029450 conn(0x7fac3c0fd8c0 msgr2=0x7fac3c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 shutdown_connections 2024-09-05T07:05:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:38.844+0000 7fac4168a640 1 -- 172.21.15.86:0/49029450 wait complete. 2024-09-05T07:05:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:39 smithi086 ceph-mon[30943]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 312 op/s 2024-09-05T07:05:39.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77405 6.24 MB/sec execute 537 sec latency 90.325 ms 2024-09-05T07:05:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:39 smithi096 ceph-mon[34468]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 312 op/s 2024-09-05T07:05:39.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:39 smithi115 ceph-mon[34791]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 312 op/s 2024-09-05T07:05:40.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77521 6.25 MB/sec execute 538 sec latency 351.081 ms 2024-09-05T07:05:41.796 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77648 6.24 MB/sec execute 539 sec latency 119.637 ms 2024-09-05T07:05:41.821 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:41 smithi096 ceph-mon[34468]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 201 op/s 2024-09-05T07:05:41.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:41 smithi115 ceph-mon[34791]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 201 op/s 2024-09-05T07:05:41.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:41 smithi086 ceph-mon[30943]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 201 op/s 2024-09-05T07:05:42.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:05:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:42.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77784 6.24 MB/sec execute 540 sec latency 92.948 ms 2024-09-05T07:05:43.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77920 6.24 MB/sec execute 541 sec latency 105.041 ms 2024-09-05T07:05:43.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:43 smithi096 ceph-mon[34468]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-05T07:05:43.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:43 smithi115 ceph-mon[34791]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-05T07:05:43.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:43 smithi086 ceph-mon[30943]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-05T07:05:44.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78044 6.23 MB/sec execute 542 sec latency 235.219 ms 2024-09-05T07:05:45.450 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 -- 172.21.15.86:0/3003046012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074eb0 msgr2=0x7f5e00112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 --2- 172.21.15.86:0/3003046012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074eb0 0x7f5e00112660 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009900 tx=0x7f5df002a480 comp rx=0 tx=0).stop 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 -- 172.21.15.86:0/3003046012 shutdown_connections 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 --2- 172.21.15.86:0/3003046012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e00114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 --2- 172.21.15.86:0/3003046012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074eb0 0x7f5e00112660 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 --2- 172.21.15.86:0/3003046012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074590 0x7f5e00074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 -- 172.21.15.86:0/3003046012 >> 172.21.15.86:0/3003046012 conn(0x7f5e000fd8b0 msgr2=0x7f5e000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 -- 172.21.15.86:0/3003046012 shutdown_connections 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.884+0000 7f5e0805c640 1 -- 172.21.15.86:0/3003046012 wait complete. 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 Processor -- start 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 -- start start 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074eb0 0x7f5e001a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e0019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00118400 con 0x7f5e00074590 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00118280 con 0x7f5e00112ba0 2024-09-05T07:05:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e0805c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00118580 con 0x7f5e00074eb0 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41142/0 (socket says 172.21.15.86:41142) 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 -- 172.21.15.86:0/2474139696 learned_addr learned my addr 172.21.15.86:0/2474139696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e055f7640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074eb0 0x7f5e001a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e065f9640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e0019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 -- 172.21.15.86:0/2474139696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074eb0 msgr2=0x7f5e001a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074eb0 0x7f5e001a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 -- 172.21.15.86:0/2474139696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 msgr2=0x7f5e0019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e0019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e05df8640 1 -- 172.21.15.86:0/2474139696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0019efb0 con 0x7f5e00074590 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.885+0000 7f5e065f9640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e0019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5e05df8640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d950 tx=0x7f5dfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5df6ffd640 1 -- 172.21.15.86:0/2474139696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc017030 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0019f2a0 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5df6ffd640 1 -- 172.21.15.86:0/2474139696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc004020 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5df6ffd640 1 -- 172.21.15.86:0/2474139696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0047a0 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.886+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001ab4f0 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.888+0000 7f5df6ffd640 1 -- 172.21.15.86:0/2474139696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5dfc0041d0 con 0x7f5e00074590 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.888+0000 7f5e0a435740 1 Processor -- start 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.889+0000 7f5e0a435740 1 -- start start 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.889+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.889+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 0x55ae3b65ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.889+0000 7f5e065f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.889+0000 7f5e065f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41148/0 (socket says 172.21.15.86:41148) 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5df6ffd640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 0x7f5dec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b666370 0x55ae3b668760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e065f9640 1 -- 172.21.15.86:0/3010571597 learned_addr learned my addr 172.21.15.86:0/3010571597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b5c12f0 con 0x55ae3b65cb40 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b568cf0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b5bd2b0 con 0x55ae3b666370 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e05df8640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 0x55ae3b65ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5e055f7640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b666370 0x55ae3b668760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.890+0000 7f5df6ffd640 1 -- 172.21.15.86:0/2474139696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f5dfc0cf2e0 con 0x7f5e00074590 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e05df8640 1 -- 172.21.15.86:0/3010571597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b666370 msgr2=0x55ae3b668760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e05df8640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b666370 0x55ae3b668760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e05df8640 1 -- 172.21.15.86:0/3010571597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 msgr2=0x55ae3b65b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e05df8640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e05df8640 1 -- 172.21.15.86:0/3010571597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3b54d3a0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.891+0000 7f5e065f9640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:45.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.892+0000 7f5e065f9640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 0x7f5dec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.893+0000 7f5e065f9640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 0x7f5dec0a51b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005cc0 tx=0x7f5df8004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.893+0000 7f5e05df8640 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 0x55ae3b65ba10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cc5f0 tx=0x7f5dfc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb961a1f08c53e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.893+0000 7f5debfff640 1 -- 172.21.15.86:0/3010571597 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09b3c0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.893+0000 7f5debfff640 1 -- 172.21.15.86:0/3010571597 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc09f770 con 0x55ae3b65b5d0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.893+0000 7f5debfff640 1 -- 172.21.15.86:0/3010571597 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc010460 con 0x55ae3b65b5d0 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.894+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 msgr2=0x55ae3b65ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.894+0000 7f5e0a435740 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 0x55ae3b65ba10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cc5f0 tx=0x7f5dfc000bc0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.894+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 shutdown_connections 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.894+0000 7f5e0a435740 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b666370 0x55ae3b668760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.895+0000 7f5e0a435740 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b65b5d0 0x55ae3b65ba10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.895+0000 7f5e0a435740 1 --2- 172.21.15.86:0/3010571597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b65b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.895+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 >> 172.21.15.86:0/3010571597 conn(0x55ae3b624310 msgr2=0x55ae3b657c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.895+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 shutdown_connections 2024-09-05T07:05:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.895+0000 7f5e0a435740 1 -- 172.21.15.86:0/3010571597 wait complete. 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.896+0000 7f5e0a435740 1 -- start start 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.896+0000 7f5e0a435740 1 Processor -- start 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.896+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b665e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b6630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b663850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b669060 con 0x55ae3b65cb40 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b57c060 con 0x55ae3b666370 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ae3b6691e0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e0a435740 10 client.0 _ll_register_callbacks cb 0x55ae3b2cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.897+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b665e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b665e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51772/0 (socket says 172.21.15.86:51772) 2024-09-05T07:05:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e065f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b663850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e05df8640 1 -- 172.21.15.86:0/4265648794 learned_addr learned my addr 172.21.15.86:0/4265648794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b6630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 -- 172.21.15.86:0/4265648794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 msgr2=0x55ae3b665e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b665e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 -- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 msgr2=0x55ae3b663850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b663850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.898+0000 7f5e055f7640 1 -- 172.21.15.86:0/4265648794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3b3323c0 con 0x55ae3b65cb40 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.899+0000 7f5e055f7640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b6630c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009230 tx=0x7f5df00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8a9d6fdaa291b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.900+0000 7f5de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.900+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035040 con 0x55ae3b65cb40 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.900+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f5df0031810 con 0x55ae3b65cb40 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.900+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc09f770 con 0x55ae3b65cb40 2024-09-05T07:05:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.902+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5df0004040 con 0x55ae3b65cb40 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.902+0000 7f5e0a435740 10 client.20736 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.902+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ae3b674bb0 con 0x55ae3b65cb40 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc09f770 con 0x55ae3b65cb40 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5df0004850 con 0x55ae3b65cb40 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 10 client.20736 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 10 client.20736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 20 client.20736 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b678460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 4 client.20736 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.905+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae3b5ce860 con 0x55ae3b676070 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.909+0000 7f5e05df8640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b678460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.913+0000 7f5e05df8640 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b678460 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3178d0491f824625 server_cookie=8bcc9d8a1dc6cb2 in_seq=0 out_seq=0 2024-09-05T07:05:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.913+0000 7f5de97fa640 10 client.20736 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.916+0000 7f5de97fa640 1 -- 172.21.15.86:0/4265648794 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae3b5ce860 con 0x55ae3b676070 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.916+0000 7f5de97fa640 10 client.20736 handle_command_reply: tid=0 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 client.20736 shutdown 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 msgr2=0x55ae3b6630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b6630c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009230 tx=0x7f5df00314a0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 shutdown_connections 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b678460 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b663850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5de97fa640 0 client.20736 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b6630c0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 --2- 172.21.15.86:0/4265648794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b665e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.917+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 >> 172.21.15.86:0/4265648794 conn(0x55ae3b624310 msgr2=0x55ae3b65d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.918+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 shutdown_connections 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.918+0000 7f5e0a435740 1 -- 172.21.15.86:0/4265648794 wait complete. 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.919+0000 7f5e0a435740 20 client.20736 trim_cache size 0 max 16384 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.945+0000 7f5e0a435740 1 Processor -- start 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 -- start start 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b6620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b67dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b67e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b59c0 con 0x55ae3b65cb40 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e055f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b67dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b6620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e055f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b67dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41170/0 (socket says 172.21.15.86:41170) 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e055f7640 1 -- 172.21.15.86:0/44153671 learned_addr learned my addr 172.21.15.86:0/44153671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b7500 con 0x55ae3b666370 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b6090 con 0x55ae3b65b5d0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.946+0000 7f5e065f9640 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b67e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.947+0000 7f5e05df8640 1 -- 172.21.15.86:0/44153671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 msgr2=0x55ae3b67e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.947+0000 7f5e05df8640 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b67e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.947+0000 7f5e05df8640 1 -- 172.21.15.86:0/44153671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 msgr2=0x55ae3b67dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.947+0000 7f5e05df8640 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b67dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.947+0000 7f5e05df8640 1 -- 172.21.15.86:0/44153671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3b663bc0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.948+0000 7f5e05df8640 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b6620a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc005170 tx=0x7f5dfc0bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f8f55c1586b069 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.948+0000 7f5deb7fe640 1 -- 172.21.15.86:0/44153671 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09b3c0 con 0x55ae3b65b5d0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.948+0000 7f5deb7fe640 1 -- 172.21.15.86:0/44153671 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc084030 con 0x55ae3b65b5d0 2024-09-05T07:05:45.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.948+0000 7f5deb7fe640 1 -- 172.21.15.86:0/44153671 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc009a30 con 0x55ae3b65b5d0 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 msgr2=0x55ae3b6620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b6620a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc005170 tx=0x7f5dfc0bb040 comp rx=0 tx=0).stop 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 shutdown_connections 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b67e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65cb40 0x55ae3b67dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 --2- 172.21.15.86:0/44153671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65b5d0 0x55ae3b6620a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 >> 172.21.15.86:0/44153671 conn(0x55ae3b624310 msgr2=0x55ae3b661990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 shutdown_connections 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- 172.21.15.86:0/44153671 wait complete. 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.949+0000 7f5e0a435740 1 -- start start 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.950+0000 7f5e0a435740 1 Processor -- start 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.950+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 0x55ae3b66e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 0x55ae3b669c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 0x55ae3b66e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e05df8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 0x55ae3b66e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41184/0 (socket says 172.21.15.86:41184) 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e05df8640 1 -- 172.21.15.86:0/2247219616 learned_addr learned my addr 172.21.15.86:0/2247219616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e055f7640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 0x55ae3b669c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b66a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b6e30 con 0x55ae3b65b5d0 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b6760 con 0x55ae3b666370 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b39c0 con 0x55ae3b65cb40 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e065f9640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b66a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.951+0000 7f5e0a435740 10 client.0 _ll_register_callbacks cb 0x7f5df0005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 -- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 msgr2=0x55ae3b66a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b66a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 -- 172.21.15.86:0/2247219616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 msgr2=0x55ae3b66e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 0x55ae3b66e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3b66a790 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e055f7640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 0x55ae3b669c90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005ec0 tx=0x7f5df0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ef1f8f6245a3747 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035040 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7f5df0003c60 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.952+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc084030 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5df0003e60 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 10 client.29455 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ae3b6e09c0 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dfc084030 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5df0031840 con 0x55ae3b65cb40 2024-09-05T07:05:45.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 10 client.29455 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 10 client.29455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 20 client.29455 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b6e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 4 client.29455 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae3b5c8d70 con 0x55ae3b676070 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.953+0000 7f5e05df8640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b6e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5e05df8640 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b6e3180 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc508c1626fde43 server_cookie=305e060798444994 in_seq=0 out_seq=0 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5de97fa640 10 client.29455 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5de97fa640 1 -- 172.21.15.86:0/2247219616 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae3b5c8d70 con 0x55ae3b676070 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5de97fa640 10 client.29455 handle_command_reply: tid=0 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5e0a435740 1 client.29455 shutdown 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 msgr2=0x55ae3b669c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.954+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 0x55ae3b669c90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005ec0 tx=0x7f5df0000bc0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 shutdown_connections 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55ae3b676070 0x55ae3b6e3180 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae3b666370 0x55ae3b66a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5de97fa640 0 client.29455 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae3b65cb40 0x55ae3b669c90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 --2- 172.21.15.86:0/2247219616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae3b65b5d0 0x55ae3b66e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 >> 172.21.15.86:0/2247219616 conn(0x55ae3b624310 msgr2=0x55ae3b6621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 shutdown_connections 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 1 -- 172.21.15.86:0/2247219616 wait complete. 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.955+0000 7f5e0a435740 20 client.29455 trim_cache size 0 max 16384 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 msgr2=0x7f5dec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 0x7f5dec0a51b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005cc0 tx=0x7f5df8004ab0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 msgr2=0x7f5e001a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d950 tx=0x7f5dfc00de10 comp rx=0 tx=0).stop 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 shutdown_connections 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5dec0a2cf0 0x7f5dec0a51b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00112ba0 0x7f5e0019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00074eb0 0x7f5e001a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 --2- 172.21.15.86:0/2474139696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00074590 0x7f5e001a4690 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 >> 172.21.15.86:0/2474139696 conn(0x7f5e000fd8b0 msgr2=0x7f5e000ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 shutdown_connections 2024-09-05T07:05:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:44.956+0000 7f5e0805c640 1 -- 172.21.15.86:0/2474139696 wait complete. 2024-09-05T07:05:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:45 smithi086 ceph-mon[30943]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 223 op/s 2024-09-05T07:05:45.793 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78165 6.23 MB/sec execute 543 sec latency 352.602 ms 2024-09-05T07:05:45.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:45 smithi096 ceph-mon[34468]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 223 op/s 2024-09-05T07:05:45.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:45 smithi115 ceph-mon[34791]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 223 op/s 2024-09-05T07:05:46.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78241 6.22 MB/sec execute 544 sec latency 472.520 ms 2024-09-05T07:05:47.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78390 6.23 MB/sec execute 545 sec latency 479.174 ms 2024-09-05T07:05:47.806 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:47 smithi096 ceph-mon[34468]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 209 op/s 2024-09-05T07:05:47.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:47 smithi115 ceph-mon[34791]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 209 op/s 2024-09-05T07:05:47.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:47 smithi086 ceph-mon[30943]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 209 op/s 2024-09-05T07:05:48.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:05:48.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:05:48.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78562 6.24 MB/sec execute 546 sec latency 81.259 ms 2024-09-05T07:05:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 -- 172.21.15.86:0/2717905275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064100e20 msgr2=0x7fe064101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 --2- 172.21.15.86:0/2717905275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064100e20 0x7fe064101200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe058009930 tx=0x7fe05802a4d0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 -- 172.21.15.86:0/2717905275 shutdown_connections 2024-09-05T07:05:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 --2- 172.21.15.86:0/2717905275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064102ef0 0x7fe06410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 --2- 172.21.15.86:0/2717905275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 0x7fe0641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 --2- 172.21.15.86:0/2717905275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064100e20 0x7fe064101200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 -- 172.21.15.86:0/2717905275 >> 172.21.15.86:0/2717905275 conn(0x7fe0640fc7d0 msgr2=0x7fe0640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 -- 172.21.15.86:0/2717905275 shutdown_connections 2024-09-05T07:05:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.542+0000 7fe06ad01640 1 -- 172.21.15.86:0/2717905275 wait complete. 2024-09-05T07:05:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.543+0000 7fe06ad01640 1 Processor -- start 2024-09-05T07:05:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.543+0000 7fe06ad01640 1 -- start start 2024-09-05T07:05:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 0x7fe064196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 0x7fe064196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410fd30 con 0x7fe064102570 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410fbb0 con 0x7fe064102ef0 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06ad01640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410feb0 con 0x7fe064100e20 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06929e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06929e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38792/0 (socket says 172.21.15.86:38792) 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe06929e640 1 -- 172.21.15.86:0/3222824796 learned_addr learned my addr 172.21.15.86:0/3222824796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe063fff640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 0x7fe064196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.544+0000 7fe068a9d640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 0x7fe064196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 -- 172.21.15.86:0/3222824796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 msgr2=0x7fe064196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 0x7fe064196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 -- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 msgr2=0x7fe064196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 0x7fe064196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 -- 172.21.15.86:0/3222824796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0641aacd0 con 0x7fe064102ef0 2024-09-05T07:05:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe068a9d640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 0x7fe064196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06929e640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009e80 tx=0x7fe05c00e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe061ffb640 1 -- 172.21.15.86:0/3222824796 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c004b20 con 0x7fe064102ef0 2024-09-05T07:05:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe061ffb640 1 -- 172.21.15.86:0/3222824796 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05c004cc0 con 0x7fe064102ef0 2024-09-05T07:05:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe061ffb640 1 -- 172.21.15.86:0/3222824796 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005880 con 0x7fe064102ef0 2024-09-05T07:05:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.545+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0641aafc0 con 0x7fe064102ef0 2024-09-05T07:05:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.546+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0641ab480 con 0x7fe064102ef0 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe061ffb640 1 -- 172.21.15.86:0/3222824796 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe05c01b020 con 0x7fe064102ef0 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 Processor -- start 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 -- start start 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc72130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be7bbca280 con 0x55be7bc6fd40 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be7bb71b10 con 0x55be7bc64b60 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.549+0000 7fe06d0da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be7bbc6240 con 0x55be7bc65480 2024-09-05T07:05:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38802/0 (socket says 172.21.15.86:38802) 2024-09-05T07:05:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 -- 172.21.15.86:0/973143127 learned_addr learned my addr 172.21.15.86:0/973143127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc72130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe068a9d640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 -- 172.21.15.86:0/973143127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 msgr2=0x55be7bc6f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe061ffb640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 0x7fe0540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 -- 172.21.15.86:0/973143127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 msgr2=0x55be7bc72130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc72130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 -- 172.21.15.86:0/973143127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7bb7e1f0 con 0x55be7bc64b60 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe068a9d640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe061ffb640 1 -- 172.21.15.86:0/3222824796 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fe05c0cf250 con 0x7fe064102ef0 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.550+0000 7fe063fff640 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55be7bc62470 tx=0x7fe05000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a2547baa27bc987 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.551+0000 7fe06929e640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 0x7fe0540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.551+0000 7fe06929e640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 0x7fe0540a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0a3a10 tx=0x7fe05c009f70 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.554+0000 7fe04effd640 1 -- 172.21.15.86:0/973143127 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050012d40 con 0x55be7bc64b60 2024-09-05T07:05:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.554+0000 7fe04effd640 1 -- 172.21.15.86:0/973143127 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe050005020 con 0x55be7bc64b60 2024-09-05T07:05:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.554+0000 7fe04effd640 1 -- 172.21.15.86:0/973143127 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004a00 con 0x55be7bc64b60 2024-09-05T07:05:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 -- 172.21.15.86:0/973143127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 msgr2=0x55be7bc64f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55be7bc62470 tx=0x7fe05000b650 comp rx=0 tx=0).stop 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 -- 172.21.15.86:0/973143127 shutdown_connections 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc72130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 --2- 172.21.15.86:0/973143127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc64f40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 -- 172.21.15.86:0/973143127 >> 172.21.15.86:0/973143127 conn(0x55be7bc610f0 msgr2=0x55be7bc614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 -- 172.21.15.86:0/973143127 shutdown_connections 2024-09-05T07:05:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.555+0000 7fe06d0da740 1 -- 172.21.15.86:0/973143127 wait complete. 2024-09-05T07:05:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.556+0000 7fe06d0da740 1 -- start start 2024-09-05T07:05:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.556+0000 7fe06d0da740 1 Processor -- start 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc6ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc6d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be7bc72910 con 0x55be7bc6fd40 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be7bb84f10 con 0x55be7bc64b60 2024-09-05T07:05:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55be7bc72a90 con 0x55be7bc65480 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe06d0da740 10 client.0 _ll_register_callbacks cb 0x55be7bb4da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51818/0 (socket says 172.21.15.86:51818) 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 -- 172.21.15.86:0/516878985 learned_addr learned my addr 172.21.15.86:0/516878985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe063fff640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc6d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe063fff640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc6ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 -- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 msgr2=0x55be7bc6ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc6ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 -- 172.21.15.86:0/516878985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 msgr2=0x55be7bc6d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc6d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe068a9d640 1 -- 172.21.15.86:0/516878985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7b93b6a0 con 0x55be7bc65480 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.557+0000 7fe063fff640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc6d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe068a9d640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe05802a9f0 tx=0x7fe0580052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d842276e5ea636 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe043fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:05:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058035030 con 0x55be7bc65480 2024-09-05T07:05:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fe0580048c0 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe050005020 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe058005d50 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe06d0da740 10 client.29464 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.558+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55be7bc7e5c0 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0580035f0 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe0580314a0 con 0x55be7bc65480 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 10 client.29464 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 10 client.29464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 20 client.29464 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bc7fa80 0x55be7bc81ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 4 client.29464 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe050005020 con 0x55be7bc7fa80 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.559+0000 7fe063fff640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bc7fa80 0x55be7bc81ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe043fff640 10 client.29464 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe063fff640 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bc7fa80 0x55be7bc81ea0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05dce571bb77ff5 server_cookie=d0603f717c4cd18a in_seq=0 out_seq=0 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe043fff640 1 -- 172.21.15.86:0/516878985 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe050005020 con 0x55be7bc7fa80 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe043fff640 10 client.29464 handle_command_reply: tid=0 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 client.29464 shutdown 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 msgr2=0x55be7bc6d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe05802a9f0 tx=0x7fe0580052f0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 shutdown_connections 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bc7fa80 0x55be7bc81ea0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc6d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6d210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 --2- 172.21.15.86:0/516878985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bc6ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.560+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 >> 172.21.15.86:0/516878985 conn(0x55be7bc610f0 msgr2=0x55be7bc66e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.561+0000 7fe043fff640 0 client.29464 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.561+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 shutdown_connections 2024-09-05T07:05:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.561+0000 7fe06d0da740 1 -- 172.21.15.86:0/516878985 wait complete. 2024-09-05T07:05:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.561+0000 7fe06d0da740 20 client.29464 trim_cache size 0 max 16384 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.589+0000 7fe06d0da740 1 Processor -- start 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.589+0000 7fe06d0da740 1 -- start start 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc6fd40 0x55be7bc87750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b5770 con 0x55be7bc64b60 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b72b0 con 0x55be7bc6fd40 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe06d0da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b5e40 con 0x55be7bc65480 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41236/0 (socket says 172.21.15.86:41236) 2024-09-05T07:05:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 -- 172.21.15.86:0/4271091507 learned_addr learned my addr 172.21.15.86:0/4271091507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 -- 172.21.15.86:0/4271091507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 msgr2=0x55be7bc6eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe068a9d640 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 -- 172.21.15.86:0/4271091507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc6fd40 msgr2=0x55be7bc87750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc6fd40 0x55be7bc87750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe063fff640 1 -- 172.21.15.86:0/4271091507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7bc67b00 con 0x55be7bc64b60 2024-09-05T07:05:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.590+0000 7fe068a9d640 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.591+0000 7fe063fff640 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55be7bc62470 tx=0x7fe05000bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3738c30a03e89d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.591+0000 7fe04e7fc640 1 -- 172.21.15.86:0/4271091507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050005490 con 0x55be7bc64b60 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.591+0000 7fe04e7fc640 1 -- 172.21.15.86:0/4271091507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe050005630 con 0x55be7bc64b60 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.591+0000 7fe04e7fc640 1 -- 172.21.15.86:0/4271091507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050005490 con 0x55be7bc64b60 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 -- 172.21.15.86:0/4271091507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 msgr2=0x55be7bc675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55be7bc62470 tx=0x7fe05000bc10 comp rx=0 tx=0).stop 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 -- 172.21.15.86:0/4271091507 shutdown_connections 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc6fd40 0x55be7bc87750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc6eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 --2- 172.21.15.86:0/4271091507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc64b60 0x55be7bc675c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 -- 172.21.15.86:0/4271091507 >> 172.21.15.86:0/4271091507 conn(0x55be7bc610f0 msgr2=0x55be7bc6e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.592+0000 7fe06d0da740 1 -- 172.21.15.86:0/4271091507 shutdown_connections 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.593+0000 7fe06d0da740 1 -- 172.21.15.86:0/4271091507 wait complete. 2024-09-05T07:05:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.593+0000 7fe06d0da740 1 -- start start 2024-09-05T07:05:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.593+0000 7fe06d0da740 1 Processor -- start 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bce5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc77ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b6be0 con 0x55be7bc6fd40 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b6510 con 0x55be7bc64b60 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b3770 con 0x55be7bc65480 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe06d0da740 10 client.0 _ll_register_callbacks cb 0x7fe05c00b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41248/0 (socket says 172.21.15.86:41248) 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 -- 172.21.15.86:0/2108759718 learned_addr learned my addr 172.21.15.86:0/2108759718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe068a9d640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc77ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bce5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 -- 172.21.15.86:0/2108759718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 msgr2=0x55be7bc77ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc77ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 -- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 msgr2=0x55be7bce5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bce5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.594+0000 7fe063fff640 1 -- 172.21.15.86:0/2108759718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7bc78860 con 0x55be7bc6fd40 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe063fff640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fe05000bbe0 tx=0x7fe050017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dcd9d2e34e42f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe043fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050005490 con 0x55be7bc6fd40 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fe0500109e0 con 0x55be7bc6fd40 2024-09-05T07:05:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be7bbd77f0 con 0x55be7bc6fd40 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe050010be0 con 0x55be7bc6fd40 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe06d0da740 10 client.20760 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.595+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55be7bcfb730 con 0x55be7bc6fd40 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe050010d80 con 0x55be7bc6fd40 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe050004780 con 0x55be7bc6fd40 2024-09-05T07:05:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 10 client.20760 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 10 client.20760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 20 client.20760 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bcfba60 0x55be7bcfde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 4 client.20760 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be7bbd77f0 con 0x55be7bcfba60 2024-09-05T07:05:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.596+0000 7fe068a9d640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bcfba60 0x55be7bcfde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.597+0000 7fe043fff640 10 client.20760 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.597+0000 7fe068a9d640 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bcfba60 0x55be7bcfde50 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efff2ecd74ba7837 server_cookie=37621c0836b53011 in_seq=0 out_seq=0 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.598+0000 7fe043fff640 1 -- 172.21.15.86:0/2108759718 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be7bbd77f0 con 0x55be7bcfba60 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.598+0000 7fe043fff640 10 client.20760 handle_command_reply: tid=0 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.598+0000 7fe06d0da740 1 client.20760 shutdown 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 msgr2=0x55be7bc78220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fe05000bbe0 tx=0x7fe050017040 comp rx=0 tx=0).stop 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 shutdown_connections 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55be7bcfba60 0x55be7bcfde50 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe043fff640 0 client.20760 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be7bc6fd40 0x55be7bc78220 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55be7bc65480 0x55be7bc77ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 --2- 172.21.15.86:0/2108759718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be7bc64b60 0x55be7bce5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 >> 172.21.15.86:0/2108759718 conn(0x55be7bc610f0 msgr2=0x55be7bc60270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 shutdown_connections 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 1 -- 172.21.15.86:0/2108759718 wait complete. 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.599+0000 7fe06d0da740 20 client.20760 trim_cache size 0 max 16384 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c8c88601-df89-43d5-a8de-c1f14ecab151", 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:05:49.603 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 msgr2=0x7fe0540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 0x7fe0540a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0a3a10 tx=0x7fe05c009f70 comp rx=0 tx=0).stop 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 msgr2=0x7fe064196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009e80 tx=0x7fe05c00e490 comp rx=0 tx=0).stop 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 shutdown_connections 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe0540a2cf0 0x7fe0540a51b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe064102ef0 0x7fe064196ad0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe064102570 0x7fe064196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 --2- 172.21.15.86:0/3222824796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe064100e20 0x7fe064196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:49.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 >> 172.21.15.86:0/3222824796 conn(0x7fe0640fc7d0 msgr2=0x7fe0641048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:49.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 shutdown_connections 2024-09-05T07:05:49.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:49.600+0000 7fe06ad01640 1 -- 172.21.15.86:0/3222824796 wait complete. 2024-09-05T07:05:49.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78756 6.26 MB/sec execute 547 sec latency 96.730 ms 2024-09-05T07:05:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:49 smithi096 ceph-mon[34468]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 310 op/s 2024-09-05T07:05:49.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:49 smithi115 ceph-mon[34791]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 310 op/s 2024-09-05T07:05:49.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:49 smithi086 ceph-mon[30943]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 310 op/s 2024-09-05T07:05:50.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:05:50.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78929 6.27 MB/sec execute 548 sec latency 202.148 ms 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:50 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:05:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:05:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:50.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:50 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.058 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:50 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.501+0000 7fa50072d640 1 -- 172.21.15.86:0/1832084141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 msgr2=0x7fa4f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.501+0000 7fa50072d640 1 --2- 172.21.15.86:0/1832084141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f81021c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009900 tx=0x7fa4f402a410 comp rx=0 tx=0).stop 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 -- 172.21.15.86:0/1832084141 shutdown_connections 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 --2- 172.21.15.86:0/1832084141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 0x7fa4f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 --2- 172.21.15.86:0/1832084141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 --2- 172.21.15.86:0/1832084141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f81021c0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 -- 172.21.15.86:0/1832084141 >> 172.21.15.86:0/1832084141 conn(0x7fa4f80fd8e0 msgr2=0x7fa4f80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 -- 172.21.15.86:0/1832084141 shutdown_connections 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 -- 172.21.15.86:0/1832084141 wait complete. 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.502+0000 7fa50072d640 1 Processor -- start 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 -- start start 2024-09-05T07:05:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fe4c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fe4c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41270/0 (socket says 172.21.15.86:41270) 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fdcc8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 0x7fa4f819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fdcc8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f8198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:38836/0 (socket says 172.21.15.86:38836) 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fe4c9640 1 -- 172.21.15.86:0/247140480 learned_addr learned my addr 172.21.15.86:0/247140480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8111e70 con 0x7fa4f8101de0 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8111cf0 con 0x7fa4f8102790 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8111ff0 con 0x7fa4f8106f10 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.503+0000 7fa4fecca640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 0x7fa4f819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 -- 172.21.15.86:0/247140480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 msgr2=0x7fa4f819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 0x7fa4f819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 -- 172.21.15.86:0/247140480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 msgr2=0x7fa4f8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f819d2c0 con 0x7fa4f8101de0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005ab0 tx=0x7fa4f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4ef7fe640 1 -- 172.21.15.86:0/247140480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f403f070 con 0x7fa4f8101de0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f819d550 con 0x7fa4f8101de0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4ef7fe640 1 -- 172.21.15.86:0/247140480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4f403bb20 con 0x7fa4f8101de0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.504+0000 7fa4ef7fe640 1 -- 172.21.15.86:0/247140480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4035030 con 0x7fa4f8101de0 2024-09-05T07:05:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.505+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a4e90 con 0x7fa4f8101de0 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 Processor -- start 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa4ef7fe640 1 -- 172.21.15.86:0/247140480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa4f4042070 con 0x7fa4f8101de0 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 -- start start 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c378050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c378af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e4c2d42e0 con 0x562e4c378670 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e4c27bce0 con 0x562e4c37e000 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.508+0000 7fa502b06740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562e4c2d02a0 con 0x562e4c37d650 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51864/0 (socket says 172.21.15.86:51864) 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 -- 172.21.15.86:0/2236509021 learned_addr learned my addr 172.21.15.86:0/2236509021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c378050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c378af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 -- 172.21.15.86:0/2236509021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 msgr2=0x562e4c378050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c378050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 -- 172.21.15.86:0/2236509021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 msgr2=0x562e4c378af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c378af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 -- 172.21.15.86:0/2236509021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4c260440 con 0x562e4c37d650 2024-09-05T07:05:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4fecca640 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f50 tx=0x7fa4f00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40a82416bb3f14d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.509+0000 7fa4ef7fe640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 0x7fa4e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.510+0000 7fa4ecff9640 1 -- 172.21.15.86:0/2236509021 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004d20 con 0x562e4c37d650 2024-09-05T07:05:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.510+0000 7fa4ecff9640 1 -- 172.21.15.86:0/2236509021 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4f0004050 con 0x562e4c37d650 2024-09-05T07:05:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.510+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 0x7fa4e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.510+0000 7fa4ef7fe640 1 -- 172.21.15.86:0/247140480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa4f40efe40 con 0x7fa4f8101de0 2024-09-05T07:05:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.510+0000 7fa4ecff9640 1 -- 172.21.15.86:0/2236509021 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0005280 con 0x562e4c37d650 2024-09-05T07:05:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- 172.21.15.86:0/2236509021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 msgr2=0x562e4c37da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa4fe4c9640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 0x7fa4e40a51b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ed200 tx=0x7fa4f40f3990 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f50 tx=0x7fa4f00118d0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- 172.21.15.86:0/2236509021 shutdown_connections 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c378af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c378050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- 172.21.15.86:0/2236509021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37d650 0x562e4c37da30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- 172.21.15.86:0/2236509021 >> 172.21.15.86:0/2236509021 conn(0x562e4c36a4e0 msgr2=0x562e4c36a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- 172.21.15.86:0/2236509021 shutdown_connections 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- 172.21.15.86:0/2236509021 wait complete. 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 -- start start 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 Processor -- start 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.511+0000 7fa502b06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c3810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa502b06740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c381620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa502b06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e4c384ac0 con 0x562e4c37d650 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa502b06740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e4c28f050 con 0x562e4c37e000 2024-09-05T07:05:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa502b06740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562e4c384c40 con 0x562e4c378670 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa502b06740 10 client.0 _ll_register_callbacks cb 0x562e4c2cf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:51870/0 (socket says 172.21.15.86:51870) 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 -- 172.21.15.86:0/3970160645 learned_addr learned my addr 172.21.15.86:0/3970160645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c381620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c3810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 -- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 msgr2=0x562e4c381620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c381620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 -- 172.21.15.86:0/3970160645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 msgr2=0x562e4c3810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c3810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 -- 172.21.15.86:0/3970160645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4c0453c0 con 0x562e4c378670 2024-09-05T07:05:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c3810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0003fd0 tx=0x7fa4f00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c33e98757ad5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.512+0000 7fa4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004a00 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa4f0028450 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f0004050 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa4f0004ba0 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 10 client.29479 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562e4c3904c0 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4f0004050 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa4f000f480 con 0x562e4c378670 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 10 client.29479 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 10 client.29479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.513+0000 7fa502b06740 20 client.29479 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c393cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa502b06740 4 client.29479 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e4c2e1850 con 0x562e4c3918c0 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c393cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c393cb0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13de125061c5a67 server_cookie=e1a3b41b7e7fdc09 in_seq=0 out_seq=0 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.514+0000 7fa4e1ffb640 10 client.29479 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3970160645 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e4c2e1850 con 0x562e4c3918c0 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa4e1ffb640 10 client.29479 handle_command_reply: tid=0 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 client.29479 shutdown 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 msgr2=0x562e4c383e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0003fd0 tx=0x7fa4f00047b0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 shutdown_connections 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c393cb0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa4e1ffb640 0 client.29479 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c381620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c3810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 --2- 172.21.15.86:0/3970160645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c383e30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 >> 172.21.15.86:0/3970160645 conn(0x562e4c36a4e0 msgr2=0x562e4c369bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 shutdown_connections 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 1 -- 172.21.15.86:0/3970160645 wait complete. 2024-09-05T07:05:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.515+0000 7fa502b06740 20 client.29479 trim_cache size 0 max 16384 2024-09-05T07:05:51.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 Processor -- start 2024-09-05T07:05:51.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 -- start start 2024-09-05T07:05:51.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c378050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c37ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa4fdcc8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa4fdcc8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41282/0 (socket says 172.21.15.86:41282) 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa4fdcc8640 1 -- 172.21.15.86:0/566940122 learned_addr learned my addr 172.21.15.86:0/566940122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81acf30 con 0x562e4c37d650 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aea70 con 0x562e4c37e000 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ad600 con 0x562e4c378670 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.540+0000 7fa4fecca640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c378050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fecca640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c37ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 -- 172.21.15.86:0/566940122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 msgr2=0x562e4c378050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c378050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 -- 172.21.15.86:0/566940122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 msgr2=0x562e4c37ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c37ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 -- 172.21.15.86:0/566940122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4c37dda0 con 0x562e4c37d650 2024-09-05T07:05:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fecca640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c378050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:05:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.541+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x562e4c380970 tx=0x7fa4e8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33618f41c682e715 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.542+0000 7fa4e3fff640 1 -- 172.21.15.86:0/566940122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8003d20 con 0x562e4c37d650 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.542+0000 7fa4e3fff640 1 -- 172.21.15.86:0/566940122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e80268f0 con 0x562e4c37d650 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.542+0000 7fa4e3fff640 1 -- 172.21.15.86:0/566940122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80107b0 con 0x562e4c37d650 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 msgr2=0x562e4c38ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x562e4c380970 tx=0x7fa4e8004410 comp rx=0 tx=0).stop 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 shutdown_connections 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37e000 0x562e4c37ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c37d650 0x562e4c38ddb0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 --2- 172.21.15.86:0/566940122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c378670 0x562e4c378050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 >> 172.21.15.86:0/566940122 conn(0x562e4c36a4e0 msgr2=0x562e4c37f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 shutdown_connections 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- 172.21.15.86:0/566940122 wait complete. 2024-09-05T07:05:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 -- start start 2024-09-05T07:05:51.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.543+0000 7fa502b06740 1 Processor -- start 2024-09-05T07:05:51.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37d650 0x562e4c399590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa4fecca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41294/0 (socket says 172.21.15.86:41294) 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa4fecca640 1 -- 172.21.15.86:0/3851503600 learned_addr learned my addr 172.21.15.86:0/3851503600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37e000 0x562e4c399ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ae3a0 con 0x562e4c378670 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81adcd0 con 0x562e4c37d650 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aaf30 con 0x562e4c37e000 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa502b06740 10 client.0 _ll_register_callbacks cb 0x7fa4f40f3860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.544+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37e000 0x562e4c399ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37d650 0x562e4c399590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 -- 172.21.15.86:0/3851503600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37e000 msgr2=0x562e4c399ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37e000 0x562e4c399ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 -- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37d650 msgr2=0x562e4c399590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37d650 0x562e4c399590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4c3fb460 con 0x562e4c378670 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4fecca640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0004940 tx=0x7fa4f0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47817ac46d415a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000fc80 con 0x562e4c378670 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.545+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9605+0+0 (secure 0 0 0) 0x7fa4f0028450 con 0x562e4c378670 2024-09-05T07:05:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e80268f0 con 0x562e4c378670 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa4f000f680 con 0x562e4c378670 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 10 client.20775 fetch_fsmap learned FSMap version 47 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562e4c3fc710 con 0x562e4c378670 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e80268f0 con 0x562e4c378670 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa4f003b540 con 0x562e4c378670 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 10 client.20775 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 10 client.20775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 20 client.20775 populate_metadata read hostname 'smithi086' 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c3fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:05:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 4 client.20775 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.546+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e4c2dbd60 con 0x562e4c3918c0 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c3fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa4fdcc8640 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c3fecd0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ab5a13faa3a522 server_cookie=edf6bd0a0c12cad in_seq=0 out_seq=0 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa4e1ffb640 10 client.20775 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa4e1ffb640 1 -- 172.21.15.86:0/3851503600 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562e4c2dbd60 con 0x562e4c3918c0 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa4e1ffb640 10 client.20775 handle_command_reply: tid=0 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.547+0000 7fa502b06740 1 client.20775 shutdown 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 msgr2=0x562e4c399050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0004940 tx=0x7fa4f0011dd0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 shutdown_connections 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x562e4c3918c0 0x562e4c3fecd0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562e4c37e000 0x562e4c399ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa4e1ffb640 0 client.20775 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e4c37d650 0x562e4c399590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 --2- 172.21.15.86:0/3851503600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e4c378670 0x562e4c399050 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.548+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 >> 172.21.15.86:0/3851503600 conn(0x562e4c36a4e0 msgr2=0x562e4c3801d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.549+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 shutdown_connections 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.549+0000 7fa502b06740 1 -- 172.21.15.86:0/3851503600 wait complete. 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.549+0000 7fa502b06740 20 client.20775 trim_cache size 0 max 16384 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:05:51.554 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "c8c88601-df89-43d5-a8de-c1f14ecab151": { 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c8c88601-df89-43d5-a8de-c1f14ecab151", 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:05:51.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 msgr2=0x7fa4e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 0x7fa4e40a51b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ed200 tx=0x7fa4f40f3990 comp rx=0 tx=0).stop 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 msgr2=0x7fa4f8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005ab0 tx=0x7fa4f4003c10 comp rx=0 tx=0).stop 2024-09-05T07:05:51.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 shutdown_connections 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa4e40a2cf0 0x7fa4e40a51b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4f8106f10 0x7fa4f819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f8102790 0x7fa4f8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 --2- 172.21.15.86:0/247140480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4f8101de0 0x7fa4f8198310 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 >> 172.21.15.86:0/247140480 conn(0x7fa4f80fd8e0 msgr2=0x7fa4f80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.550+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 shutdown_connections 2024-09-05T07:05:51.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:05:51.551+0000 7fa50072d640 1 -- 172.21.15.86:0/247140480 wait complete. 2024-09-05T07:05:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:51 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:51 smithi086 ceph-mon[30943]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 207 op/s 2024-09-05T07:05:51.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79076 6.27 MB/sec execute 549 sec latency 267.346 ms 2024-09-05T07:05:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:51 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:51.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:51 smithi115 ceph-mon[34791]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 207 op/s 2024-09-05T07:05:52.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:51 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:05:52.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:51 smithi096 ceph-mon[34468]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 207 op/s 2024-09-05T07:05:52.312 INFO:tasks.cephfs.filesystem:scrub status for tag:c8c88601-df89-43d5-a8de-c1f14ecab151 - {'path': '/', 'tag': 'c8c88601-df89-43d5-a8de-c1f14ecab151', 'options': 'recursive,force'} 2024-09-05T07:05:52.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79247 6.27 MB/sec execute 550 sec latency 154.960 ms 2024-09-05T07:05:53.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79401 6.27 MB/sec execute 551 sec latency 61.803 ms 2024-09-05T07:05:53.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:53 smithi115 ceph-mon[34791]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-05T07:05:53.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:53 smithi086 ceph-mon[30943]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-05T07:05:54.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:53 smithi096 ceph-mon[34468]: pgmap v775: 129 pgs: 129 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-05T07:05:54.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79571 6.28 MB/sec execute 552 sec latency 89.369 ms 2024-09-05T07:05:55.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79725 6.28 MB/sec execute 553 sec latency 80.764 ms 2024-09-05T07:05:55.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:55 smithi115 ceph-mon[34791]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 223 op/s 2024-09-05T07:05:55.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:55 smithi086 ceph-mon[30943]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 223 op/s 2024-09-05T07:05:56.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:55 smithi096 ceph-mon[34468]: pgmap v776: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 223 op/s 2024-09-05T07:05:56.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 79877 6.28 MB/sec execute 554 sec latency 156.811 ms 2024-09-05T07:05:57.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80043 6.29 MB/sec execute 555 sec latency 107.313 ms 2024-09-05T07:05:57.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:57 smithi115 ceph-mon[34791]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 204 op/s 2024-09-05T07:05:57.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:57 smithi086 ceph-mon[30943]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 204 op/s 2024-09-05T07:05:58.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:57 smithi096 ceph-mon[34468]: pgmap v777: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 204 op/s 2024-09-05T07:05:58.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80249 6.30 MB/sec execute 556 sec latency 73.098 ms 2024-09-05T07:05:59.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80441 6.30 MB/sec execute 557 sec latency 131.662 ms 2024-09-05T07:05:59.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:05:59 smithi115 ceph-mon[34791]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 315 op/s 2024-09-05T07:05:59.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:05:59 smithi086 ceph-mon[30943]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 315 op/s 2024-09-05T07:06:00.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:05:59 smithi096 ceph-mon[34468]: pgmap v778: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 315 op/s 2024-09-05T07:06:00.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80570 6.30 MB/sec execute 558 sec latency 78.457 ms 2024-09-05T07:06:00.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:00 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:00.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:00 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:01.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:00 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:01.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80717 6.32 MB/sec execute 559 sec latency 100.688 ms 2024-09-05T07:06:01.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:01 smithi115 ceph-mon[34791]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 215 op/s 2024-09-05T07:06:01.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:01.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:01.833 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:01.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:01 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:01.957 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:01 smithi096 ceph-mon[34468]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 215 op/s 2024-09-05T07:06:01.958 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:01.958 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:01.958 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:01.958 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:01 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:01 smithi086 ceph-mon[30943]: pgmap v779: 129 pgs: 129 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 215 op/s 2024-09-05T07:06:01.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:01.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:01.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:01.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:01 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:02.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80878 6.32 MB/sec execute 560 sec latency 76.590 ms 2024-09-05T07:06:02.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:02 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:06:02.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:02 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:06:03.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:02 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:06:03.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81043 6.33 MB/sec execute 561 sec latency 116.712 ms 2024-09-05T07:06:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:03 smithi115 ceph-mon[34791]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 317 op/s 2024-09-05T07:06:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:06:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:06:03.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:06:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:03 smithi086 ceph-mon[30943]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 317 op/s 2024-09-05T07:06:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:06:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:06:03.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:06:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:03 smithi096 ceph-mon[34468]: pgmap v780: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 317 op/s 2024-09-05T07:06:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:06:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:06:04.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:06:04.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81201 6.33 MB/sec execute 562 sec latency 61.750 ms 2024-09-05T07:06:05.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81342 6.32 MB/sec execute 563 sec latency 138.487 ms 2024-09-05T07:06:05.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:05 smithi115 ceph-mon[34791]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 263 op/s 2024-09-05T07:06:05.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:05 smithi086 ceph-mon[30943]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 263 op/s 2024-09-05T07:06:06.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:05 smithi096 ceph-mon[34468]: pgmap v781: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 263 op/s 2024-09-05T07:06:06.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81479 6.32 MB/sec execute 564 sec latency 132.124 ms 2024-09-05T07:06:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:06 smithi115 ceph-mon[34791]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T07:06:06.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:06 smithi086 ceph-mon[30943]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T07:06:07.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:06 smithi096 ceph-mon[34468]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T07:06:07.794 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81633 6.31 MB/sec execute 565 sec latency 74.457 ms 2024-09-05T07:06:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:07 smithi115 ceph-mon[34791]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 303 op/s 2024-09-05T07:06:07.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:07 smithi086 ceph-mon[30943]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 303 op/s 2024-09-05T07:06:08.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:07 smithi096 ceph-mon[34468]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 303 op/s 2024-09-05T07:06:08.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81822 6.32 MB/sec execute 566 sec latency 134.874 ms 2024-09-05T07:06:09.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82015 6.34 MB/sec execute 567 sec latency 93.780 ms 2024-09-05T07:06:09.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:09 smithi115 ceph-mon[34791]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:09.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:09 smithi086 ceph-mon[30943]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:10.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:09 smithi096 ceph-mon[34468]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:10.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82179 6.34 MB/sec execute 568 sec latency 162.481 ms 2024-09-05T07:06:11.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82379 6.36 MB/sec execute 569 sec latency 66.942 ms 2024-09-05T07:06:11.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:11 smithi115 ceph-mon[34791]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:11.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:11 smithi086 ceph-mon[30943]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:12.057 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:11 smithi096 ceph-mon[34468]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.3 MiB/s wr, 288 op/s 2024-09-05T07:06:12.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82553 6.37 MB/sec execute 570 sec latency 105.077 ms 2024-09-05T07:06:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:13 smithi086 ceph-mon[30943]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 302 op/s 2024-09-05T07:06:13.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:13 smithi086 ceph-mon[30943]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T07:06:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:13 smithi096 ceph-mon[34468]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 302 op/s 2024-09-05T07:06:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:13 smithi096 ceph-mon[34468]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T07:06:13.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:13 smithi115 ceph-mon[34791]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 302 op/s 2024-09-05T07:06:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:13 smithi115 ceph-mon[34791]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T07:06:13.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82700 6.38 MB/sec execute 571 sec latency 65.007 ms 2024-09-05T07:06:14.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82836 6.37 MB/sec execute 572 sec latency 75.133 ms 2024-09-05T07:06:15.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:15 smithi086 ceph-mon[30943]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 324 op/s 2024-09-05T07:06:15.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:15 smithi096 ceph-mon[34468]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 324 op/s 2024-09-05T07:06:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:15 smithi115 ceph-mon[34791]: pgmap v788: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 324 op/s 2024-09-05T07:06:15.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82995 6.37 MB/sec execute 573 sec latency 78.218 ms 2024-09-05T07:06:16.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83141 6.37 MB/sec execute 574 sec latency 170.445 ms 2024-09-05T07:06:17.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:17 smithi086 ceph-mon[30943]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 264 op/s 2024-09-05T07:06:17.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:17 smithi096 ceph-mon[34468]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 264 op/s 2024-09-05T07:06:17.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:17 smithi115 ceph-mon[34791]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.7 MiB/s wr, 264 op/s 2024-09-05T07:06:17.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83279 6.37 MB/sec execute 575 sec latency 76.984 ms 2024-09-05T07:06:18.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83423 6.37 MB/sec execute 576 sec latency 69.791 ms 2024-09-05T07:06:19.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:19 smithi086 ceph-mon[30943]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:19.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:19 smithi096 ceph-mon[34468]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:19 smithi115 ceph-mon[34791]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:19.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83596 6.38 MB/sec execute 577 sec latency 121.935 ms 2024-09-05T07:06:20.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83787 6.38 MB/sec execute 578 sec latency 95.933 ms 2024-09-05T07:06:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:21 smithi086 ceph-mon[30943]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:21.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:21 smithi096 ceph-mon[34468]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:21.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:21 smithi115 ceph-mon[34791]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 274 op/s 2024-09-05T07:06:21.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83971 6.39 MB/sec execute 579 sec latency 115.822 ms 2024-09-05T07:06:22.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:06:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:22.795 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84109 6.39 MB/sec execute 580 sec latency 82.971 ms 2024-09-05T07:06:22.943 INFO:tasks.quiescer.fs.[cephfs]:Total duration (46.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T07:06:22.944 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.05 2024-09-05T07:06:22.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:06:23.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:23 smithi086 ceph-mon[30943]: pgmap v792: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-09-05T07:06:23.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:23 smithi096 ceph-mon[34468]: pgmap v792: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-09-05T07:06:23.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:23 smithi115 ceph-mon[34791]: pgmap v792: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-09-05T07:06:23.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84251 6.40 MB/sec execute 581 sec latency 149.215 ms 2024-09-05T07:06:24.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84389 6.40 MB/sec execute 582 sec latency 92.086 ms 2024-09-05T07:06:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 -- 172.21.15.86:0/3834279408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc811b140 msgr2=0x7f7cc811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3834279408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc811b140 0x7f7cc811d530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009ca0 tx=0x7f7cc002a8e0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 -- 172.21.15.86:0/3834279408 shutdown_connections 2024-09-05T07:06:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3834279408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc811b140 0x7f7cc811d530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3834279408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3834279408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc8109450 0x7f7cc8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.895+0000 7f7ccff48640 1 -- 172.21.15.86:0/3834279408 >> 172.21.15.86:0/3834279408 conn(0x7f7cc806e140 msgr2=0x7f7cc806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.898+0000 7f7ccff48640 1 -- 172.21.15.86:0/3834279408 shutdown_connections 2024-09-05T07:06:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.900+0000 7f7ccff48640 1 -- 172.21.15.86:0/3834279408 wait complete. 2024-09-05T07:06:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.901+0000 7f7ccff48640 1 Processor -- start 2024-09-05T07:06:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 -- start start 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc8109450 0x7f7cc81a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc811b140 0x7f7cc81abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8120c90 con 0x7f7cc8109830 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8120b10 con 0x7f7cc811b140 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccff48640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8120e10 con 0x7f7cc8109450 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.903+0000 7f7ccd4e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.904+0000 7f7ccd4e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45916/0 (socket says 172.21.15.86:45916) 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.904+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/3055393963 learned_addr learned my addr 172.21.15.86:0/3055393963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.904+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc811b140 0x7f7cc81abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.905+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/3055393963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc8109450 msgr2=0x7f7cc81a7230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.905+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc8109450 0x7f7cc81a7230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.905+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/3055393963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc811b140 msgr2=0x7f7cc81abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.905+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc811b140 0x7f7cc81abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.905+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/3055393963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc81ac1e0 con 0x7f7cc8109830 2024-09-05T07:06:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.906+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d950 tx=0x7f7cb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.907+0000 7f7cbeffd640 1 -- 172.21.15.86:0/3055393963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8017070 con 0x7f7cc8109830 2024-09-05T07:06:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.907+0000 7f7cbeffd640 1 -- 172.21.15.86:0/3055393963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cb8009c30 con 0x7f7cc8109830 2024-09-05T07:06:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.907+0000 7f7cbeffd640 1 -- 172.21.15.86:0/3055393963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800f620 con 0x7f7cc8109830 2024-09-05T07:06:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.907+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc81ac4d0 con 0x7f7cc8109830 2024-09-05T07:06:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.907+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc81b3db0 con 0x7f7cc8109830 2024-09-05T07:06:24.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.911+0000 7f7cbeffd640 1 -- 172.21.15.86:0/3055393963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7cb800f7c0 con 0x7f7cc8109830 2024-09-05T07:06:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 Processor -- start 2024-09-05T07:06:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 -- start start 2024-09-05T07:06:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 0x55664f2c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55664f21f2e0 con 0x55664f2b9dc0 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55664f1c6ce0 con 0x55664f2c4680 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cd2321740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55664f21b2a0 con 0x55664f2b94a0 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7cce4e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45928/0 (socket says 172.21.15.86:45928) 2024-09-05T07:06:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.912+0000 7f7ccdce4640 1 -- 172.21.15.86:0/1672714937 learned_addr learned my addr 172.21.15.86:0/1672714937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:24.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.913+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 0x55664f2c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.913+0000 7f7cbeffd640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 0x7f7cb40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.913+0000 7f7cbeffd640 1 -- 172.21.15.86:0/3055393963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f7cb80cf3e0 con 0x7f7cc8109830 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccdce4640 1 -- 172.21.15.86:0/1672714937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 msgr2=0x55664f2b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 0x7f7cb40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccdce4640 1 -- 172.21.15.86:0/1672714937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 msgr2=0x55664f2c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 0x55664f2c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccdce4640 1 -- 172.21.15.86:0/1672714937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664f1ab440 con 0x55664f2b9dc0 2024-09-05T07:06:24.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 0x55664f2c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:24.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.914+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:24.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.915+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 0x7f7cb40a5200 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0004ab0 tx=0x7f7cc002a8e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.916+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009770 tx=0x7f7cc4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3c956aa7fb7139 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.916+0000 7f7cb3fff640 1 -- 172.21.15.86:0/1672714937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4015020 con 0x55664f2b9dc0 2024-09-05T07:06:24.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.916+0000 7f7cb3fff640 1 -- 172.21.15.86:0/1672714937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cc4011420 con 0x55664f2b9dc0 2024-09-05T07:06:24.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.916+0000 7f7cb3fff640 1 -- 172.21.15.86:0/1672714937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40105e0 con 0x55664f2b9dc0 2024-09-05T07:06:24.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 -- 172.21.15.86:0/1672714937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 msgr2=0x55664f2c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009770 tx=0x7f7cc4009c30 comp rx=0 tx=0).stop 2024-09-05T07:06:24.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 -- 172.21.15.86:0/1672714937 shutdown_connections 2024-09-05T07:06:24.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2c4680 0x55664f2c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2b9dc0 0x55664f2c4140 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1672714937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 -- 172.21.15.86:0/1672714937 >> 172.21.15.86:0/1672714937 conn(0x55664f2b5a30 msgr2=0x55664f2b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.917+0000 7f7cd2321740 1 -- 172.21.15.86:0/1672714937 shutdown_connections 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.918+0000 7f7cd2321740 1 -- 172.21.15.86:0/1672714937 wait complete. 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.918+0000 7f7cd2321740 1 -- start start 2024-09-05T07:06:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.918+0000 7f7cd2321740 1 Processor -- start 2024-09-05T07:06:24.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b94a0 0x55664f2c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b9dc0 0x55664f2c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55664f2c7250 con 0x55664f2c4680 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55664f1da050 con 0x55664f2b94a0 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55664f2c73d0 con 0x55664f2b9dc0 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.919+0000 7f7cd2321740 10 client.0 _ll_register_callbacks cb 0x55664f21a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.920+0000 7f7cce4e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.920+0000 7f7cce4e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45942/0 (socket says 172.21.15.86:45942) 2024-09-05T07:06:24.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.920+0000 7f7cce4e5640 1 -- 172.21.15.86:0/2912581575 learned_addr learned my addr 172.21.15.86:0/2912581575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.920+0000 7f7cce4e5640 1 -- 172.21.15.86:0/2912581575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b9dc0 msgr2=0x55664f2c1180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b9dc0 0x55664f2c1180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b94a0 0x55664f2c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cce4e5640 1 -- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b94a0 msgr2=0x55664f2c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b94a0 0x55664f2c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cce4e5640 1 -- 172.21.15.86:0/2912581575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664ef903c0 con 0x55664f2c4680 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0035000 tx=0x7f7cc00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a11294d111a1cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.921+0000 7f7cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc004e070 con 0x55664f2c4680 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f7cc004a8d0 con 0x55664f2c4680 2024-09-05T07:06:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc4011420 con 0x55664f2c4680 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7cc005a050 con 0x55664f2c4680 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cd2321740 10 client.20787 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.922+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55664f2d2af0 con 0x55664f2c4680 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cc4011420 con 0x55664f2c4680 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f7cc0049040 con 0x55664f2c4680 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 10 client.20787 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 10 client.20787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 20 client.20787 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f2d43e0 0x55664f2d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 4 client.20787 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664f22c850 con 0x55664f2d43e0 2024-09-05T07:06:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.923+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f2d43e0 0x55664f2d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.924+0000 7f7cb17fa640 10 client.20787 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.925+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f2d43e0 0x55664f2d68a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85274dfa1a45cc64 server_cookie=7576b883b4bea2e4 in_seq=0 out_seq=0 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.925+0000 7f7cb17fa640 1 -- 172.21.15.86:0/2912581575 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55664f22c850 con 0x55664f2d43e0 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.925+0000 7f7cb17fa640 10 client.20787 handle_command_reply: tid=0 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.925+0000 7f7cd2321740 1 client.20787 shutdown 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 msgr2=0x55664f2c16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0035000 tx=0x7f7cc00095c0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 shutdown_connections 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f2d43e0 0x55664f2d68a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2c16c0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b9dc0 0x55664f2c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2912581575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b94a0 0x55664f2c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 >> 172.21.15.86:0/2912581575 conn(0x55664f2b5a30 msgr2=0x55664f2b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.926+0000 7f7cb17fa640 0 client.20787 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:24.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.927+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 shutdown_connections 2024-09-05T07:06:24.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.927+0000 7f7cd2321740 1 -- 172.21.15.86:0/2912581575 wait complete. 2024-09-05T07:06:24.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.927+0000 7f7cd2321740 20 client.20787 trim_cache size 0 max 16384 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 Processor -- start 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 -- start start 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac0100e0 con 0x55664f2c4680 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac011c20 con 0x55664f2b9dc0 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7cd2321740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac0107b0 con 0x55664f2b94a0 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:40038/0 (socket says 172.21.15.86:40038) 2024-09-05T07:06:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.949+0000 7f7ccdce4640 1 -- 172.21.15.86:0/2952298345 learned_addr learned my addr 172.21.15.86:0/2952298345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/2952298345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 msgr2=0x55664f2d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/2952298345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 msgr2=0x55664f2dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2dc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/2952298345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664f2c2500 con 0x55664f2b9dc0 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2dc1b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80cc690 tx=0x7f7cb809b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a49db9164b7565d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7cb37fe640 1 -- 172.21.15.86:0/2952298345 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8020070 con 0x55664f2b9dc0 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7cb37fe640 1 -- 172.21.15.86:0/2952298345 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cb809b460 con 0x55664f2b9dc0 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.950+0000 7f7cb37fe640 1 -- 172.21.15.86:0/2952298345 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8097c60 con 0x55664f2b9dc0 2024-09-05T07:06:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 -- 172.21.15.86:0/2952298345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 msgr2=0x55664f2dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2dc1b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80cc690 tx=0x7f7cb809b340 comp rx=0 tx=0).stop 2024-09-05T07:06:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 -- 172.21.15.86:0/2952298345 shutdown_connections 2024-09-05T07:06:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f2dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2dc1b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 --2- 172.21.15.86:0/2952298345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.951+0000 7f7cd2321740 1 -- 172.21.15.86:0/2952298345 >> 172.21.15.86:0/2952298345 conn(0x55664f2b5a30 msgr2=0x55664f2c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.952+0000 7f7cd2321740 1 -- 172.21.15.86:0/2952298345 shutdown_connections 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.952+0000 7f7cd2321740 1 -- 172.21.15.86:0/2952298345 wait complete. 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.952+0000 7f7cd2321740 1 -- start start 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 Processor -- start 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f3371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac011550 con 0x55664f2c4680 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac010e80 con 0x55664f2b9dc0 2024-09-05T07:06:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00e0e0 con 0x55664f2b94a0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cd2321740 10 client.0 _ll_register_callbacks cb 0x7f7cc0031d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7ccd4e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7ccdce4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:40048/0 (socket says 172.21.15.86:40048) 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7ccdce4640 1 -- 172.21.15.86:0/1187125986 learned_addr learned my addr 172.21.15.86:0/1187125986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.953+0000 7f7cce4e5640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f3371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/1187125986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 msgr2=0x55664f2c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/1187125986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 msgr2=0x55664f3371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f3371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 -- 172.21.15.86:0/1187125986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664f3377a0 con 0x55664f2b9dc0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7ccd4e3640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2c8300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8095fd0 tx=0x7f7cb8097ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f6d99be2b6c3b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8020070 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f7cb809a8b0 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.954+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb809b460 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7cb80ccb10 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 10 client.29441 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55664f33eb30 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cb809b460 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f7cb809a320 con 0x55664f2b9dc0 2024-09-05T07:06:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 10 client.29441 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 10 client.29441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 20 client.29441 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f33f1d0 0x55664f3415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 4 client.29441 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55664f226d60 con 0x55664f33f1d0 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.955+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f33f1d0 0x55664f3415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cb17fa640 10 client.29441 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7ccdce4640 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f33f1d0 0x55664f3415f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3787891e68f734 server_cookie=c683e0bc5e228675 in_seq=0 out_seq=0 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cb17fa640 1 -- 172.21.15.86:0/1187125986 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55664f226d60 con 0x55664f33f1d0 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cb17fa640 10 client.29441 handle_command_reply: tid=0 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cd2321740 1 client.29441 shutdown 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 msgr2=0x55664f2c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.956+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2c8300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8095fd0 tx=0x7f7cb8097ad0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 shutdown_connections 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55664f33f1d0 0x55664f3415f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55664f2c4680 0x55664f3371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55664f2b9dc0 0x55664f2c8300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 --2- 172.21.15.86:0/1187125986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55664f2b94a0 0x55664f2c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 >> 172.21.15.86:0/1187125986 conn(0x55664f2b5a30 msgr2=0x55664f2c29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cb17fa640 0 client.29441 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 shutdown_connections 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 1 -- 172.21.15.86:0/1187125986 wait complete. 2024-09-05T07:06:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.957+0000 7f7cd2321740 20 client.29441 trim_cache size 0 max 16384 2024-09-05T07:06:24.963 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:24.963 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:06:24.963 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:06:24.963 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 msgr2=0x7f7cb40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 0x7f7cb40a5200 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0004ab0 tx=0x7f7cc002a8e0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 msgr2=0x7f7cc81a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d950 tx=0x7f7cb800de10 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 shutdown_connections 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7cb40a2d40 0x7f7cb40a5200 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cc811b140 0x7f7cc81abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc8109830 0x7f7cc81a7770 secure :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d950 tx=0x7f7cb800de10 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 --2- 172.21.15.86:0/3055393963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc8109450 0x7f7cc81a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 >> 172.21.15.86:0/3055393963 conn(0x7f7cc806e140 msgr2=0x7f7cc811be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 shutdown_connections 2024-09-05T07:06:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:24.958+0000 7f7ccff48640 1 -- 172.21.15.86:0/3055393963 wait complete. 2024-09-05T07:06:25.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:25 smithi086 ceph-mon[30943]: pgmap v793: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:06:25.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:25 smithi096 ceph-mon[34468]: pgmap v793: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:06:25.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:25 smithi115 ceph-mon[34791]: pgmap v793: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 230 op/s 2024-09-05T07:06:25.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84538 6.41 MB/sec execute 583 sec latency 110.041 ms 2024-09-05T07:06:26.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:06:26.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3770058349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b410c160 msgr2=0x7fd1b410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3770058349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b410c160 0x7fd1b410e600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009ca0 tx=0x7fd1ac02a950 comp rx=0 tx=0).stop 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3770058349 shutdown_connections 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3770058349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b410c160 0x7fd1b410e600 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3770058349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b4103650 0x7fd1b410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3770058349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3770058349 >> 172.21.15.86:0/3770058349 conn(0x7fd1b40fc790 msgr2=0x7fd1b40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3770058349 shutdown_connections 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.590+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3770058349 wait complete. 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.591+0000 7fd1b9f68640 1 Processor -- start 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 -- start start 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 0x7fd1b419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:26.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4111e60 con 0x7fd1b4102d30 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4111ce0 con 0x7fd1b4103650 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b9f68640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4111fe0 con 0x7fd1b410c160 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:40056/0 (socket says 172.21.15.86:40056) 2024-09-05T07:06:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1b3fff640 1 -- 172.21.15.86:0/3152073789 learned_addr learned my addr 172.21.15.86:0/3152073789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.592+0000 7fd1abfff640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 0x7fd1b419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b3fff640 1 -- 172.21.15.86:0/3152073789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 msgr2=0x7fd1b419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b3fff640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 0x7fd1b419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b3fff640 1 -- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 msgr2=0x7fd1b419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b37fe640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:26.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b3fff640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b3fff640 1 -- 172.21.15.86:0/3152073789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b410b620 con 0x7fd1b410c160 2024-09-05T07:06:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1b37fe640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.593+0000 7fd1abfff640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 0x7fd1b419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.594+0000 7fd1b3fff640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009dc0 tx=0x7fd1ac003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:26.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.594+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac034040 con 0x7fd1b410c160 2024-09-05T07:06:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.594+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1ac003e50 con 0x7fd1b410c160 2024-09-05T07:06:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.594+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b410b8b0 con 0x7fd1b410c160 2024-09-05T07:06:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.594+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0045b0 con 0x7fd1b410c160 2024-09-05T07:06:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.595+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b419ab80 con 0x7fd1b410c160 2024-09-05T07:06:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.596+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b406b070 con 0x7fd1b410c160 2024-09-05T07:06:26.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.597+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd1ac037020 con 0x7fd1b410c160 2024-09-05T07:06:26.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.598+0000 7fd1b17fa640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 0x7fd1a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:26.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.599+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fd1ac0f05b0 con 0x7fd1b410c160 2024-09-05T07:06:26.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.599+0000 7fd1b37fe640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 0x7fd1a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:26.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.600+0000 7fd1b37fe640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 0x7fd1a00a5230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009ba0 tx=0x7fd1a4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.603+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1ac0bce40 con 0x7fd1b410c160 2024-09-05T07:06:26.790 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:26.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84679 6.41 MB/sec execute 584 sec latency 118.872 ms 2024-09-05T07:06:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.974+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1b4197480 con 0x7fd1b410c160 2024-09-05T07:06:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.975+0000 7fd1b17fa640 1 -- 172.21.15.86:0/3152073789 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11974 (secure 0 0 0) 0x7fd1b4197480 con 0x7fd1b410c160 2024-09-05T07:06:26.977 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:06:26.977 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"btime":"2024-09-05T07:05:20:496062+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:05:19.627701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:06:26.977 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 msgr2=0x7fd1a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 0x7fd1a00a5230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009ba0 tx=0x7fd1a4005f90 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 msgr2=0x7fd1b410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009dc0 tx=0x7fd1ac003ce0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 shutdown_connections 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fd1a00a2d70 0x7fd1a00a5230 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1b410c160 0x7fd1b410ad60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1b4103650 0x7fd1b419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 --2- 172.21.15.86:0/3152073789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4102d30 0x7fd1b419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.978+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 >> 172.21.15.86:0/3152073789 conn(0x7fd1b40fc790 msgr2=0x7fd1b41008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:26.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.979+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 shutdown_connections 2024-09-05T07:06:26.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:26.979+0000 7fd1b9f68640 1 -- 172.21.15.86:0/3152073789 wait complete. 2024-09-05T07:06:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:27 smithi086 ceph-mon[30943]: pgmap v794: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 221 op/s 2024-09-05T07:06:27.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:27 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3152073789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:27 smithi096 ceph-mon[34468]: pgmap v794: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 221 op/s 2024-09-05T07:06:27.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:27 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3152073789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:27 smithi115 ceph-mon[34791]: pgmap v794: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 221 op/s 2024-09-05T07:06:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:27 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3152073789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:27.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 143.05 --await-for 45.0' 2024-09-05T07:06:27.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 143.05 --await-for 45.0 2024-09-05T07:06:27.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84833 6.41 MB/sec execute 585 sec latency 100.211 ms 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 -- 172.21.15.86:0/3998203603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281082a0 msgr2=0x7f292810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 --2- 172.21.15.86:0/3998203603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281082a0 0x7f292810ec40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2920009cd0 tx=0x7f292002a8e0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 -- 172.21.15.86:0/3998203603 shutdown_connections 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 --2- 172.21.15.86:0/3998203603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281082a0 0x7f292810ec40 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 --2- 172.21.15.86:0/3998203603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29281036e0 0x7f2928107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 --2- 172.21.15.86:0/3998203603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928102d30 0x7f2928103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 -- 172.21.15.86:0/3998203603 >> 172.21.15.86:0/3998203603 conn(0x7f29280fc7d0 msgr2=0x7f29280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 -- 172.21.15.86:0/3998203603 shutdown_connections 2024-09-05T07:06:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.007+0000 7f292e28b640 1 -- 172.21.15.86:0/3998203603 wait complete. 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f292e28b640 1 Processor -- start 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f292e28b640 1 -- start start 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f292e28b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 0x7f292819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f292e28b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f292e28b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281082a0 0x7f292819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.008+0000 7f29277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45988/0 (socket says 172.21.15.86:45988) 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f2927fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 0x7f292819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f2927fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 0x7f292819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:57328/0 (socket says 172.21.15.86:57328) 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 -- 172.21.15.86:0/1974726054 learned_addr learned my addr 172.21.15.86:0/1974726054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2928111f50 con 0x7f29281036e0 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2928111dd0 con 0x7f2928102d30 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f29281120d0 con 0x7f29281082a0 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f292c828640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281082a0 0x7f292819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 -- 172.21.15.86:0/1974726054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281082a0 msgr2=0x7f292819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281082a0 0x7f292819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 -- 172.21.15.86:0/1974726054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 msgr2=0x7f292819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 0x7f292819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292819f620 con 0x7f29281036e0 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29277fe640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9f0 tx=0x7f291c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.009+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c00ccc0 con 0x7f29281036e0 2024-09-05T07:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.010+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292819f910 con 0x7f29281036e0 2024-09-05T07:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.010+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f291c0108e0 con 0x7f29281036e0 2024-09-05T07:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.010+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c0187c0 con 0x7f29281036e0 2024-09-05T07:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.010+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281a71f0 con 0x7f29281036e0 2024-09-05T07:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.011+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2928103110 con 0x7f29281036e0 2024-09-05T07:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.012+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f291c010a90 con 0x7f29281036e0 2024-09-05T07:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.013+0000 7f29257fa640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 0x7f29100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.013+0000 7f2927fff640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 0x7f29100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.013+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f291c0cfb50 con 0x7f29281036e0 2024-09-05T07:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.014+0000 7f2927fff640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 0x7f29100a5230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2918009bd0 tx=0x7f2918005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.018+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f291c09c330 con 0x7f29281036e0 2024-09-05T07:06:28.134 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.359+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29281a7480 con 0x7f29281036e0 2024-09-05T07:06:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.360+0000 7f29257fa640 1 -- 172.21.15.86:0/1974726054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11974 (secure 0 0 0) 0x7f291c09bc30 con 0x7f29281036e0 2024-09-05T07:06:28.362 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:06:28.362 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"btime":"2024-09-05T07:05:20:496062+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:05:19.627701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:06:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 msgr2=0x7f29100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 0x7f29100a5230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2918009bd0 tx=0x7f2918005f90 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 msgr2=0x7f292819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9f0 tx=0x7f291c00eeb0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 shutdown_connections 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f29100a2d70 0x7f29100a5230 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281082a0 0x7f292819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29281036e0 0x7f292819abb0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 --2- 172.21.15.86:0/1974726054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2928102d30 0x7f292819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 >> 172.21.15.86:0/1974726054 conn(0x7f29280fc7d0 msgr2=0x7f292810dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 shutdown_connections 2024-09-05T07:06:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:28.363+0000 7f292e28b640 1 -- 172.21.15.86:0/1974726054 wait complete. 2024-09-05T07:06:28.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84968 6.40 MB/sec execute 586 sec latency 166.977 ms 2024-09-05T07:06:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:29 smithi086 ceph-mon[30943]: pgmap v795: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 329 op/s 2024-09-05T07:06:29.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1974726054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:29 smithi096 ceph-mon[34468]: pgmap v795: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 329 op/s 2024-09-05T07:06:29.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1974726054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:29 smithi115 ceph-mon[34791]: pgmap v795: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.6 MiB/s wr, 329 op/s 2024-09-05T07:06:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1974726054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:29.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85102 6.40 MB/sec execute 587 sec latency 265.214 ms 2024-09-05T07:06:30.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85262 6.40 MB/sec execute 588 sec latency 125.848 ms 2024-09-05T07:06:31.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:31 smithi096 ceph-mon[34468]: pgmap v796: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-05T07:06:31.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:31 smithi115 ceph-mon[34791]: pgmap v796: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-05T07:06:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:31 smithi086 ceph-mon[30943]: pgmap v796: 129 pgs: 129 active+clean; 1.7 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-05T07:06:31.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85396 6.41 MB/sec execute 589 sec latency 171.026 ms 2024-09-05T07:06:32.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85555 6.41 MB/sec execute 590 sec latency 172.344 ms 2024-09-05T07:06:33.490 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:33 smithi086 ceph-mon[30943]: pgmap v797: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 316 op/s 2024-09-05T07:06:33.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:06:33.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:33 smithi096 ceph-mon[34468]: pgmap v797: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 316 op/s 2024-09-05T07:06:33.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 -- 172.21.15.86:0/991385781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 msgr2=0x7f5b08100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 --2- 172.21.15.86:0/991385781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b08100d30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009900 tx=0x7f5af802a420 comp rx=0 tx=0).stop 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 -- 172.21.15.86:0/991385781 shutdown_connections 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 --2- 172.21.15.86:0/991385781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 0x7f5b081108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 --2- 172.21.15.86:0/991385781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 0x7f5b0810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 --2- 172.21.15.86:0/991385781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b08100d30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 -- 172.21.15.86:0/991385781 >> 172.21.15.86:0/991385781 conn(0x7f5b080fc7a0 msgr2=0x7f5b080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 -- 172.21.15.86:0/991385781 shutdown_connections 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.482+0000 7f5b0cd65640 1 -- 172.21.15.86:0/991385781 wait complete. 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.484+0000 7f5b0cd65640 1 Processor -- start 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.488+0000 7f5b0cd65640 1 -- start start 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 0x7f5b0819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 0x7f5b0819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08114090 con 0x7f5b08100950 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08113f10 con 0x7f5b08101300 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.489+0000 7f5b0cd65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08114210 con 0x7f5b0810e500 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.491+0000 7f5b0659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.491+0000 7f5b0659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49182/0 (socket says 172.21.15.86:49182) 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.491+0000 7f5b0659c640 1 -- 172.21.15.86:0/352707116 learned_addr learned my addr 172.21.15.86:0/352707116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.491+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 0x7f5b0819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 0x7f5b0819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 -- 172.21.15.86:0/352707116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 msgr2=0x7f5b0819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 0x7f5b0819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 -- 172.21.15.86:0/352707116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 msgr2=0x7f5b0819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 0x7f5b0819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 -- 172.21.15.86:0/352707116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0819f380 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.492+0000 7f5b0659c640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009850 tx=0x7f5af8030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.493+0000 7f5af77fe640 1 -- 172.21.15.86:0/352707116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af803f070 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.493+0000 7f5af77fe640 1 -- 172.21.15.86:0/352707116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5af8004460 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.493+0000 7f5af77fe640 1 -- 172.21.15.86:0/352707116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8035030 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.495+0000 7f5b0cd65640 1 -- 172.21.15.86:0/352707116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0819f610 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.495+0000 7f5b0cd65640 1 -- 172.21.15.86:0/352707116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0819fa70 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.499+0000 7f5b0f13e740 1 Processor -- start 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 -- start start 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5af77fe640 1 -- 172.21.15.86:0/352707116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5af8042070 con 0x7f5b08100950 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9ef273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f9ee76160 con 0x560f9ef25000 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f9ee1d720 con 0x560f9ef1b990 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.501+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f9ee72120 con 0x560f9ef1a880 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.502+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.502+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:35214/0 (socket says 172.21.15.86:35214) 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.502+0000 7f5b06d9d640 1 -- 172.21.15.86:0/3124398139 learned_addr learned my addr 172.21.15.86:0/3124398139 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.502+0000 7f5b0659c640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.502+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9ef273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 -- 172.21.15.86:0/3124398139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 msgr2=0x560f9ef1acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 -- 172.21.15.86:0/3124398139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 msgr2=0x560f9ef273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9ef273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 -- 172.21.15.86:0/3124398139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9ebe7ba0 con 0x560f9ef1b990 2024-09-05T07:06:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9ef273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ec0 tx=0x7f5b000107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae0c9f5ac5454de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5aeffff640 1 -- 172.21.15.86:0/3124398139 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00004d20 con 0x560f9ef1b990 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5aeffff640 1 -- 172.21.15.86:0/3124398139 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b00003e20 con 0x560f9ef1b990 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.503+0000 7f5aeffff640 1 -- 172.21.15.86:0/3124398139 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005020 con 0x560f9ef1b990 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5af77fe640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 0x7f5af00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5af77fe640 1 -- 172.21.15.86:0/352707116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f5af80efe40 con 0x7f5b08100950 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 -- 172.21.15.86:0/3124398139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 msgr2=0x560f9ef1a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ec0 tx=0x7f5b000107a0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 -- 172.21.15.86:0/3124398139 shutdown_connections 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9ef273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/3124398139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1a340 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.504+0000 7f5b0f13e740 1 -- 172.21.15.86:0/3124398139 >> 172.21.15.86:0/3124398139 conn(0x560f9ef168f0 msgr2=0x560f9ef16cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0f13e740 1 -- 172.21.15.86:0/3124398139 shutdown_connections 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0659c640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 0x7f5af00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0f13e740 1 -- 172.21.15.86:0/3124398139 wait complete. 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0f13e740 1 -- start start 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0f13e740 1 Processor -- start 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.505+0000 7f5b0659c640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 0x7f5af00a5380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x560f9ef17750 tx=0x7f5af80f3870 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1b990 0x560f9ef138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f9ef0deb0 con 0x560f9ef1b990 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f9edfe900 con 0x560f9ef1a880 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f9ef0e030 con 0x560f9ef25000 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b0f13e740 10 client.0 _ll_register_callbacks cb 0x560f9edfc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.86:35218/0 (socket says 172.21.15.86:35218) 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1357234086 learned_addr learned my addr 172.21.15.86:0/1357234086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.506+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1b990 0x560f9ef138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1357234086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 msgr2=0x560f9ef13e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef13e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1357234086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1b990 msgr2=0x560f9ef138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1b990 0x560f9ef138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1357234086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9ee3ec50 con 0x560f9ef1a880 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b00004600 tx=0x7f5b00004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5943f3c51363a81b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0001f070 con 0x560f9ef1a880 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f5b00011640 con 0x560f9ef1a880 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b00003e20 con 0x560f9ef1a880 2024-09-05T07:06:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.507+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5b00033050 con 0x560f9ef1a880 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 10 client.29462 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560f9ef3c3f0 con 0x560f9ef1a880 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b00003e20 con 0x560f9ef1a880 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5b0001b870 con 0x560f9ef1a880 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 10 client.29462 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 10 client.29462 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 20 client.29462 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9ef3dbb0 0x560f9ef3ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 4 client.29462 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9ee83890 con 0x560f9ef3dbb0 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.508+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9ef3dbb0 0x560f9ef3ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.509+0000 7f5aedffb640 10 client.29462 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.509+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9ef3dbb0 0x560f9ef3ffd0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa9a727d7972f10 server_cookie=40c1f4c5f8531a9c in_seq=0 out_seq=0 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5aedffb640 1 -- 172.21.15.86:0/1357234086 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f9ee83890 con 0x560f9ef3dbb0 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5aedffb640 10 client.29462 handle_command_reply: tid=0 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 client.29462 shutdown 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 msgr2=0x560f9ef13380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b00004600 tx=0x7f5b00004660 comp rx=0 tx=0).stop 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 shutdown_connections 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9ef3dbb0 0x560f9ef3ffd0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef13e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1b990 0x560f9ef138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1357234086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1a880 0x560f9ef13380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 >> 172.21.15.86:0/1357234086 conn(0x560f9ef168f0 msgr2=0x560f9ef1be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5aedffb640 0 client.29462 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 shutdown_connections 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1357234086 wait complete. 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.510+0000 7f5b0f13e740 20 client.29462 trim_cache size 0 max 16384 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 Processor -- start 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 -- start start 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef45e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081acf60 con 0x560f9ef1a880 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081aeaa0 con 0x560f9ef1b990 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b0f13e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081ad630 con 0x560f9ef25000 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.544+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49210/0 (socket says 172.21.15.86:49210) 2024-09-05T07:06:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1099579509 learned_addr learned my addr 172.21.15.86:0/1099579509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef45e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1099579509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 msgr2=0x560f9ef45e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef45e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1099579509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 msgr2=0x560f9ef458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1099579509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9eee7ac0 con 0x560f9ef1a880 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009690 tx=0x7f5b00004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e38e8943ba0907 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5aef7fe640 1 -- 172.21.15.86:0/1099579509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0003e020 con 0x560f9ef1a880 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5aef7fe640 1 -- 172.21.15.86:0/1099579509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b00030030 con 0x560f9ef1a880 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.545+0000 7f5aef7fe640 1 -- 172.21.15.86:0/1099579509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0001f050 con 0x560f9ef1a880 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.546+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1099579509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 msgr2=0x560f9ef0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.546+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009690 tx=0x7f5b00004560 comp rx=0 tx=0).stop 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.546+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1099579509 shutdown_connections 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.546+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef25000 0x560f9ef45e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.546+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1099579509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef1a880 0x560f9ef0d390 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1099579509 >> 172.21.15.86:0/1099579509 conn(0x560f9ef168f0 msgr2=0x560f9ef0cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1099579509 shutdown_connections 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1099579509 wait complete. 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 -- start start 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.547+0000 7f5b0f13e740 1 Processor -- start 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b0f13e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49626/0 (socket says 172.21.15.86:49626) 2024-09-05T07:06:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b06d9d640 1 -- 172.21.15.86:0/1810160374 learned_addr learned my addr 172.21.15.86:0/1810160374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9efa6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.548+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081ae3d0 con 0x560f9ef25000 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081add00 con 0x560f9ef1b990 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081aafa0 con 0x560f9ef1a880 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9efa6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 -- 172.21.15.86:0/1810160374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 msgr2=0x560f9ef1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 -- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 msgr2=0x560f9efa6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9efa6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9efa6dc0 con 0x560f9ef1b990 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9efa6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b0f13e740 10 client.0 _ll_register_callbacks cb 0x7f5af80f3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b00030030 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5b05d9b640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1f3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x560f9ef40860 tx=0x7f5afc01f770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927f956c2259d708 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.549+0000 7f5aed7fa640 10 client.29468 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5aed7fa640 1 -- 172.21.15.86:0/1810160374 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc004230 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5aed7fa640 1 -- 172.21.15.86:0/1810160374 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f5afc0043d0 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5aed7fa640 1 -- 172.21.15.86:0/1810160374 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5afc0035f0 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 10 client.29468 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=47}) -- 0x560f9efa7cc0 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5aed7fa640 1 -- 172.21.15.86:0/1810160374 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5afc003790 con 0x560f9ef1b990 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 10 client.29468 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 10 client.29468 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 20 client.29468 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9efa83e0 0x560f9efaa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 4 client.29468 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.05, "await_for": 45.0}] 2024-09-05T07:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.550+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.05, "await_for": 45.0}) -- 0x560f9ee7dbe0 con 0x560f9efa83e0 2024-09-05T07:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.551+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9efa83e0 0x560f9efaa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.551+0000 7f5b06d9d640 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9efa83e0 0x560f9efaa870 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217442ac29b9ef55 server_cookie=30696b6e70734e66 in_seq=0 out_seq=0 2024-09-05T07:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.551+0000 7f5aed7fa640 10 client.29468 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:33 smithi115 ceph-mon[34791]: pgmap v797: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 316 op/s 2024-09-05T07:06:33.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85707 6.42 MB/sec execute 591 sec latency 262.612 ms 2024-09-05T07:06:33.859 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:33.952 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:33.952 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 47, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 90, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "760512b": { 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 90, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 143.1, 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:06:33.953 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:33.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:33.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:33.955 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.950+0000 7f5aed7fa640 1 -- 172.21.15.86:0/1810160374 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560f9ee7dbe0 con 0x560f9efa83e0 2024-09-05T07:06:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.950+0000 7f5aed7fa640 10 client.29468 handle_command_reply: tid=0 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.950+0000 7f5b0f13e740 1 client.29468 shutdown 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 msgr2=0x560f9ef1f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1f3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x560f9ef40860 tx=0x7f5afc01f770 comp rx=0 tx=0).stop 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 shutdown_connections 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x560f9efa83e0 0x560f9efaa870 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5aed7fa640 0 client.29468 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f9ef25000 0x560f9efa6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f9ef1b990 0x560f9ef1f3f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 --2- 172.21.15.86:0/1810160374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f9ef1a880 0x560f9ef1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 >> 172.21.15.86:0/1810160374 conn(0x560f9ef168f0 msgr2=0x560f9ef0d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 shutdown_connections 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 1 -- 172.21.15.86:0/1810160374 wait complete. 2024-09-05T07:06:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.951+0000 7f5b0f13e740 20 client.29468 trim_cache size 0 max 16384 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 msgr2=0x7f5af00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 0x7f5af00a5380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x560f9ef17750 tx=0x7f5af80f3870 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 msgr2=0x7f5b0819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009850 tx=0x7f5af8030650 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 shutdown_connections 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5af00a2ec0 0x7f5af00a5380 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b0810e500 0x7f5b0819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b08101300 0x7f5b0819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 --2- 172.21.15.86:0/352707116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08100950 0x7f5b0819a3d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 >> 172.21.15.86:0/352707116 conn(0x7f5b080fc7a0 msgr2=0x7f5b08110670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 shutdown_connections 2024-09-05T07:06:33.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:33.952+0000 7f5af57fa640 1 -- 172.21.15.86:0/352707116 wait complete. 2024-09-05T07:06:34.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.42 MB/sec execute 592 sec latency 1240.514 ms 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 -- 172.21.15.86:0/97550171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 msgr2=0x7f4b8010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/97550171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8010eaa0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009ca0 tx=0x7f4b7802a8c0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 -- 172.21.15.86:0/97550171 shutdown_connections 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/97550171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8010eaa0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/97550171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b801027a0 0x7f4b801067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/97550171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b80101df0 0x7f4b801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 -- 172.21.15.86:0/97550171 >> 172.21.15.86:0/97550171 conn(0x7f4b800fd8f0 msgr2=0x7f4b800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 -- 172.21.15.86:0/97550171 shutdown_connections 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.084+0000 7f4b84ca2640 1 -- 172.21.15.86:0/97550171 wait complete. 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 Processor -- start 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 -- start start 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80101df0 0x7f4b80198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b801027a0 0x7f4b80198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111e30 con 0x7f4b80106f20 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111cb0 con 0x7f4b801027a0 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b84ca2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111fb0 con 0x7f4b80101df0 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b7ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b7ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49248/0 (socket says 172.21.15.86:49248) 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.085+0000 7f4b7ed9d640 1 -- 172.21.15.86:0/4031739366 learned_addr learned my addr 172.21.15.86:0/4031739366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b801027a0 0x7f4b80198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80101df0 0x7f4b80198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 -- 172.21.15.86:0/4031739366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80101df0 msgr2=0x7f4b80198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80101df0 0x7f4b80198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 -- 172.21.15.86:0/4031739366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b801027a0 msgr2=0x7f4b80198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b801027a0 0x7f4b80198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 -- 172.21.15.86:0/4031739366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8019d1e0 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b7ed9d640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009dc0 tx=0x7f4b78003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/4031739366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004040 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/4031739366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b78004690 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8019d470 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/4031739366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78030d90 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.086+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b801a4db0 con 0x7f4b80106f20 2024-09-05T07:06:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.088+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/4031739366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4b780305c0 con 0x7f4b80106f20 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 Processor -- start 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 -- start start 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 0x55e27f0994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e27effb2f0 con 0x55e27f0999e0 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e27efa2cf0 con 0x55e27f09af50 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b8707b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e27eff72b0 con 0x55e27f0a4780 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49250/0 (socket says 172.21.15.86:49250) 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b7e59c640 1 -- 172.21.15.86:0/1726373158 learned_addr learned my addr 172.21.15.86:0/1726373158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b6f7fe640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 0x7f4b680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.089+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 0x55e27f0994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 -- 172.21.15.86:0/1726373158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 msgr2=0x55e27f0a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 -- 172.21.15.86:0/1726373158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 msgr2=0x55e27f0994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 0x55e27f0994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 -- 172.21.15.86:0/1726373158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27ef873a0 con 0x55e27f0999e0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7ed9d640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 0x7f4b680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 0x55e27f0994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b6f7fe640 1 -- 172.21.15.86:0/4031739366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f4b780f1050 con 0x7f4b80106f20 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000cc60 tx=0x7f4b7000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adacfe3dba379c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1726373158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70014070 con 0x55e27f0999e0 2024-09-05T07:06:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1726373158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b700047a0 con 0x55e27f0999e0 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.090+0000 7f4b6cff9640 1 -- 172.21.15.86:0/1726373158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70010690 con 0x55e27f0999e0 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- 172.21.15.86:0/1726373158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 msgr2=0x55e27f099e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000cc60 tx=0x7f4b7000e720 comp rx=0 tx=0).stop 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b7ed9d640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 0x7f4b680a51b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b780ee520 tx=0x7f4b780095c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- 172.21.15.86:0/1726373158 shutdown_connections 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f099e20 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1726373158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f09af50 0x55e27f0994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- 172.21.15.86:0/1726373158 >> 172.21.15.86:0/1726373158 conn(0x55e27f05e310 msgr2=0x55e27f096070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- 172.21.15.86:0/1726373158 shutdown_connections 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- 172.21.15.86:0/1726373158 wait complete. 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 -- start start 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 Processor -- start 2024-09-05T07:06:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.091+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e27f0a7480 con 0x55e27f0999e0 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e27efb6060 con 0x55e27f0a4780 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e27f0a7600 con 0x55e27f09af50 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 10 client.0 _ll_register_callbacks cb 0x55e27ed098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49252/0 (socket says 172.21.15.86:49252) 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/4177644203 learned_addr learned my addr 172.21.15.86:0/4177644203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 msgr2=0x55e27f0a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/4177644203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 msgr2=0x55e27f0a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/4177644203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27ed6c3c0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f4b74011a30 tx=0x7f4b74011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf34515e8ac5f3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74004020 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f4b740041c0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b700047a0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4b740043c0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 10 client.20832 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.092+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e27f0b2ef0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b700047a0 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f4b74004b70 con 0x55e27f0999e0 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 10 client.20832 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 10 client.20832 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 20 client.20832 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f0b43b0 0x55e27f0b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 4 client.20832 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27f008860 con 0x55e27f0b43b0 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.093+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f0b43b0 0x55e27f0b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f0b43b0 0x55e27f0b67c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fcc94b6c13cc284c server_cookie=c1860aaa8837abc8 in_seq=0 out_seq=0 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b65ffb640 10 client.20832 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b65ffb640 1 -- 172.21.15.86:0/4177644203 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e27f008860 con 0x55e27f0b43b0 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b65ffb640 10 client.20832 handle_command_reply: tid=0 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 client.20832 shutdown 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 msgr2=0x55e27f0a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f4b74011a30 tx=0x7f4b74011ef0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 shutdown_connections 2024-09-05T07:06:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f0b43b0 0x55e27f0b67c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b65ffb640 0 client.20832 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4177644203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a4170 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 >> 172.21.15.86:0/4177644203 conn(0x55e27f05e310 msgr2=0x55e27f09bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 shutdown_connections 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.094+0000 7f4b8707b740 1 -- 172.21.15.86:0/4177644203 wait complete. 2024-09-05T07:06:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.095+0000 7f4b8707b740 20 client.20832 trim_cache size 0 max 16384 2024-09-05T07:06:35.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.107+0000 7f4b8707b740 1 Processor -- start 2024-09-05T07:06:35.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.107+0000 7f4b8707b740 1 -- start start 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 0x55e27f096220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801acee0 con 0x55e27f09af50 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801aea20 con 0x55e27f0999e0 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b8707b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ad5b0 con 0x55e27f0a4780 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49254/0 (socket says 172.21.15.86:49254) 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 -- 172.21.15.86:0/4148433554 learned_addr learned my addr 172.21.15.86:0/4148433554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 0x55e27f096220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 -- 172.21.15.86:0/4148433554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 msgr2=0x55e27f0bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0bc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 -- 172.21.15.86:0/4148433554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 msgr2=0x55e27f096220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 0x55e27f096220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 -- 172.21.15.86:0/4148433554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27f09ab70 con 0x55e27f09af50 2024-09-05T07:06:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 0x55e27f096220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000f010 tx=0x7f4b7000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b879711d7803f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b67fff640 1 -- 172.21.15.86:0/4148433554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70014050 con 0x55e27f09af50 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b67fff640 1 -- 172.21.15.86:0/4148433554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b700054d0 con 0x55e27f09af50 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.108+0000 7f4b67fff640 1 -- 172.21.15.86:0/4148433554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70004b20 con 0x55e27f09af50 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- 172.21.15.86:0/4148433554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 msgr2=0x55e27f0b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000f010 tx=0x7f4b7000c550 comp rx=0 tx=0).stop 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- 172.21.15.86:0/4148433554 shutdown_connections 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f0a4780 0x55e27f0bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f09af50 0x55e27f0b0720 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 --2- 172.21.15.86:0/4148433554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0999e0 0x55e27f096220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- 172.21.15.86:0/4148433554 >> 172.21.15.86:0/4148433554 conn(0x55e27f05e310 msgr2=0x55e27f095b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- 172.21.15.86:0/4148433554 shutdown_connections 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- 172.21.15.86:0/4148433554 wait complete. 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 -- start start 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.109+0000 7f4b8707b740 1 Processor -- start 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ae350 con 0x55e27f0999e0 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801adc80 con 0x55e27f0a4780 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801aaee0 con 0x55e27f09af50 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 10 client.0 _ll_register_callbacks cb 0x7f4b74011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49260/0 (socket says 172.21.15.86:49260) 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/1853704084 learned_addr learned my addr 172.21.15.86:0/1853704084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 msgr2=0x55e27f0a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/1853704084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 msgr2=0x55e27f0a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 -- 172.21.15.86:0/1853704084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27f11e090 con 0x55e27f0999e0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b7dd9b640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005e60 tx=0x7f4b74011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7de47f3453c5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74005000 con 0x55e27f0999e0 2024-09-05T07:06:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f4b740051a0 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b700054d0 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4b740053a0 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 10 client.20838 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.110+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e27f11f2c0 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b700054d0 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f4b7402b450 con 0x55e27f0999e0 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 10 client.20838 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 10 client.20838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 20 client.20838 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f11fa10 0x55e27f121e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 4 client.20838 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e27f002d70 con 0x55e27f11fa10 2024-09-05T07:06:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.111+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f11fa10 0x55e27f121e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b7e59c640 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f11fa10 0x55e27f121e30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72762f00103853a7 server_cookie=5f7bb3bce6bd142c in_seq=0 out_seq=0 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b65ffb640 10 client.20838 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b65ffb640 1 -- 172.21.15.86:0/1853704084 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e27f002d70 con 0x55e27f11fa10 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b65ffb640 10 client.20838 handle_command_reply: tid=0 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 client.20838 shutdown 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 msgr2=0x55e27f0a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005e60 tx=0x7f4b74011ef0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 shutdown_connections 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x55e27f11fa10 0x55e27f121e30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b65ffb640 0 client.20838 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e27f0a4780 0x55e27f0a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e27f09af50 0x55e27f0a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 --2- 172.21.15.86:0/1853704084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e27f0999e0 0x55e27f0a8610 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 >> 172.21.15.86:0/1853704084 conn(0x55e27f05e310 msgr2=0x55e27f0904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 shutdown_connections 2024-09-05T07:06:35.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 1 -- 172.21.15.86:0/1853704084 wait complete. 2024-09-05T07:06:35.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.112+0000 7f4b8707b740 20 client.20838 trim_cache size 0 max 16384 2024-09-05T07:06:35.125 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 msgr2=0x7f4b680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 0x7f4b680a51b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b780ee520 tx=0x7f4b780095c0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 msgr2=0x7f4b8019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009dc0 tx=0x7f4b78003ba0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 shutdown_connections 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f4b680a2cf0 0x7f4b680a51b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b80106f20 0x7f4b8019cb00 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b801027a0 0x7f4b80198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 --2- 172.21.15.86:0/4031739366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80101df0 0x7f4b80198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 >> 172.21.15.86:0/4031739366 conn(0x7f4b800fd8f0 msgr2=0x7f4b800fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 shutdown_connections 2024-09-05T07:06:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:35.113+0000 7f4b84ca2640 1 -- 172.21.15.86:0/4031739366 wait complete. 2024-09-05T07:06:35.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:35 smithi086 ceph-mon[30943]: pgmap v798: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 212 op/s 2024-09-05T07:06:35.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:35 smithi096 ceph-mon[34468]: pgmap v798: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 212 op/s 2024-09-05T07:06:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:35 smithi115 ceph-mon[34791]: pgmap v798: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 212 op/s 2024-09-05T07:06:35.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.41 MB/sec execute 593 sec latency 2240.539 ms 2024-09-05T07:06:35.882 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '760512b', quiesce took 12.9 seconds. Will release after: 10.15 2024-09-05T07:06:36.358 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:06:36.606 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:36.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.40 MB/sec execute 594 sec latency 3240.602 ms 2024-09-05T07:06:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 -- 172.21.15.86:0/485833449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228102790 msgr2=0x7f42281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 --2- 172.21.15.86:0/485833449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228102790 0x7f42281067e0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f4224009900 tx=0x7f422402a410 comp rx=0 tx=0).stop 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 -- 172.21.15.86:0/485833449 shutdown_connections 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 --2- 172.21.15.86:0/485833449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 0x7f422810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 --2- 172.21.15.86:0/485833449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228102790 0x7f42281067e0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 --2- 172.21.15.86:0/485833449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228101de0 0x7f42281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 -- 172.21.15.86:0/485833449 >> 172.21.15.86:0/485833449 conn(0x7f42280fd8c0 msgr2=0x7f42280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 -- 172.21.15.86:0/485833449 shutdown_connections 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 -- 172.21.15.86:0/485833449 wait complete. 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.116+0000 7f422f2bd640 1 Processor -- start 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 -- start start 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228102790 0x7f42281989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 0x7f422819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4228111e40 con 0x7f4228101de0 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4228111cc0 con 0x7f4228106f10 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4228111fc0 con 0x7f4228102790 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49282/0 (socket says 172.21.15.86:49282) 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 -- 172.21.15.86:0/2547548550 learned_addr learned my addr 172.21.15.86:0/2547548550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422c858640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228102790 0x7f42281989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d85a640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 0x7f422819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 -- 172.21.15.86:0/2547548550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228102790 msgr2=0x7f42281989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228102790 0x7f42281989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 -- 172.21.15.86:0/2547548550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 msgr2=0x7f422819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 0x7f422819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 -- 172.21.15.86:0/2547548550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819d460 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422d059640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f421c00c890 tx=0x7f421c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f421a7fc640 1 -- 172.21.15.86:0/2547548550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004220 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819d750 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f421a7fc640 1 -- 172.21.15.86:0/2547548550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f421c0043c0 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.117+0000 7f421a7fc640 1 -- 172.21.15.86:0/2547548550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004a00 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.118+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a5030 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.120+0000 7f421a7fc640 1 -- 172.21.15.86:0/2547548550 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f421c004ba0 con 0x7f4228101de0 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.120+0000 7f4231696740 1 Processor -- start 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.120+0000 7f4231696740 1 -- start start 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c76d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f65c6d32f0 con 0x55f65c76d5d0 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f65c67acf0 con 0x55f65c778370 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f4231696740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f65c6cf2b0 con 0x55f65c76eb40 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49288/0 (socket says 172.21.15.86:49288) 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 -- 172.21.15.86:0/4014574496 learned_addr learned my addr 172.21.15.86:0/4014574496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422c858640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f421a7fc640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 0x7f42140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d85a640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c76d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d85a640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 0x7f42140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 -- 172.21.15.86:0/4014574496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 msgr2=0x55f65c76d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c76d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 -- 172.21.15.86:0/4014574496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 msgr2=0x55f65c77a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 -- 172.21.15.86:0/4014574496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65c65f3a0 con 0x55f65c76d5d0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422c858640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f421a7fc640 1 -- 172.21.15.86:0/2547548550 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f421c0cf600 con 0x7f4228101de0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.121+0000 7f422d059640 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f421c09b670 tx=0x7f421c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ce3d33172e99c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f422d85a640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 0x7f42140a51b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4220009fd0 tx=0x7f4220004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f42137fe640 1 -- 172.21.15.86:0/4014574496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004e90 con 0x55f65c76d5d0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f42137fe640 1 -- 172.21.15.86:0/4014574496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f421c09bde0 con 0x55f65c76d5d0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f42137fe640 1 -- 172.21.15.86:0/4014574496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c012110 con 0x55f65c76d5d0 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 -- 172.21.15.86:0/4014574496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 msgr2=0x55f65c76da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f421c09b670 tx=0x7f421c01f000 comp rx=0 tx=0).stop 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 -- 172.21.15.86:0/4014574496 shutdown_connections 2024-09-05T07:06:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c76da10 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 --2- 172.21.15.86:0/4014574496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c76d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 -- 172.21.15.86:0/4014574496 >> 172.21.15.86:0/4014574496 conn(0x55f65c736310 msgr2=0x55f65c769c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 -- 172.21.15.86:0/4014574496 shutdown_connections 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.122+0000 7f4231696740 1 -- 172.21.15.86:0/4014574496 wait complete. 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 -- start start 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 Processor -- start 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c774fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c775570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f65c77b010 con 0x55f65c76d5d0 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f65c68e060 con 0x55f65c778370 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f65c77b190 con 0x55f65c76eb40 2024-09-05T07:06:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f4231696740 10 client.0 _ll_register_callbacks cb 0x55f65c3e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49302/0 (socket says 172.21.15.86:49302) 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.123+0000 7f422d059640 1 -- 172.21.15.86:0/2704169760 learned_addr learned my addr 172.21.15.86:0/2704169760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d85a640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c775570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 -- 172.21.15.86:0/2704169760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 msgr2=0x55f65c774fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422c858640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c774fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c774fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 -- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 msgr2=0x55f65c775570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c775570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 -- 172.21.15.86:0/2704169760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65c4443c0 con 0x55f65c76d5d0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422c858640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c774fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f422d059640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f421c09b3e0 tx=0x7f421c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b8935b50c37664 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004e90 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f421c097d50 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c09bde0 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f421c0a4350 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4231696740 10 client.20850 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f65c77b010 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.124+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f421c09bde0 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f421c0a3ec0 con 0x55f65c76d5d0 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 10 client.20850 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 10 client.20850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 20 client.20850 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c787be0 0x55f65c789fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 4 client.20850 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f65c6e0860 con 0x55f65c787be0 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f422c858640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c787be0 0x55f65c789fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f422c858640 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c787be0 0x55f65c789fd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e9fae77b06fe9cb2 server_cookie=5bfd38baab3b7c03 in_seq=0 out_seq=0 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.125+0000 7f4210ff9640 10 client.20850 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.126+0000 7f4210ff9640 1 -- 172.21.15.86:0/2704169760 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f65c6e0860 con 0x55f65c787be0 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.126+0000 7f4210ff9640 10 client.20850 handle_command_reply: tid=0 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.126+0000 7f4231696740 1 client.20850 shutdown 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.126+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 msgr2=0x55f65c777d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.126+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f421c09b3e0 tx=0x7f421c004730 comp rx=0 tx=0).stop 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 shutdown_connections 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c787be0 0x55f65c789fd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c775570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c774fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 --2- 172.21.15.86:0/2704169760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c777d20 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 >> 172.21.15.86:0/2704169760 conn(0x55f65c736310 msgr2=0x55f65c76f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4210ff9640 0 client.20850 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:06:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 shutdown_connections 2024-09-05T07:06:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 1 -- 172.21.15.86:0/2704169760 wait complete. 2024-09-05T07:06:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.127+0000 7f4231696740 20 client.20850 trim_cache size 0 max 16384 2024-09-05T07:06:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 Processor -- start 2024-09-05T07:06:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 -- start start 2024-09-05T07:06:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 0x55f65c770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 0x55f65c770660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42281af200 con 0x55f65c778370 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42281b0d40 con 0x55f65c76eb40 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f4231696740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42281af8d0 con 0x55f65c76d5d0 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f422d85a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f422d85a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49304/0 (socket says 172.21.15.86:49304) 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.140+0000 7f422d85a640 1 -- 172.21.15.86:0/1153671397 learned_addr learned my addr 172.21.15.86:0/1153671397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d059640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 0x55f65c770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 -- 172.21.15.86:0/1153671397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 msgr2=0x55f65c770120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 0x55f65c770120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422c858640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 0x55f65c770660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 -- 172.21.15.86:0/1153671397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 msgr2=0x55f65c770660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 0x55f65c770660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 -- 172.21.15.86:0/1153671397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65c78a430 con 0x55f65c778370 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d059640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 0x55f65c770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422c858640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 0x55f65c770660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f422d85a640 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f422000a500 tx=0x7f4220009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2459b860eb4955fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f4212ffd640 1 -- 172.21.15.86:0/1153671397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220028070 con 0x55f65c778370 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f4212ffd640 1 -- 172.21.15.86:0/1153671397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f422001aab0 con 0x55f65c778370 2024-09-05T07:06:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.141+0000 7f4212ffd640 1 -- 172.21.15.86:0/1153671397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220023cd0 con 0x55f65c778370 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- 172.21.15.86:0/1153671397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 msgr2=0x55f65c78f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f422000a500 tx=0x7f4220009700 comp rx=0 tx=0).stop 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- 172.21.15.86:0/1153671397 shutdown_connections 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c778370 0x55f65c78f690 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c76eb40 0x55f65c770660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 --2- 172.21.15.86:0/1153671397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76d5d0 0x55f65c770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- 172.21.15.86:0/1153671397 >> 172.21.15.86:0/1153671397 conn(0x55f65c736310 msgr2=0x55f65c768b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- 172.21.15.86:0/1153671397 shutdown_connections 2024-09-05T07:06:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- 172.21.15.86:0/1153671397 wait complete. 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 -- start start 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.142+0000 7f4231696740 1 Processor -- start 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c77c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42281b0670 con 0x55f65c76d5d0 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42281affa0 con 0x55f65c778370 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42281ad200 con 0x55f65c76eb40 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4231696740 10 client.0 _ll_register_callbacks cb 0x7f4224009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49314/0 (socket says 172.21.15.86:49314) 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 -- 172.21.15.86:0/2798813843 learned_addr learned my addr 172.21.15.86:0/2798813843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:37.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422c858640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c77c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d85a640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 -- 172.21.15.86:0/2798813843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 msgr2=0x55f65c77c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c77c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 -- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 msgr2=0x55f65c77cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 -- 172.21.15.86:0/2798813843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65c7f1710 con 0x55f65c76d5d0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f422d059640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f421c0a4510 tx=0x7f421c0a40d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cea60fcab639a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.143+0000 7f4210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c097980 con 0x55f65c76d5d0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f421c004e90 con 0x55f65c76d5d0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422001aab0 con 0x55f65c76d5d0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f421c09b040 con 0x55f65c76d5d0 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 10 client.20856 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:37.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f65c7f2960 con 0x55f65c76d5d0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f422001aab0 con 0x55f65c76d5d0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f421c09a5a0 con 0x55f65c76d5d0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 10 client.20856 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 10 client.20856 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 20 client.20856 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c7f2ee0 0x55f65c7f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 4 client.20856 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f65c6dad70 con 0x55f65c7f2ee0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.144+0000 7f422c858640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c7f2ee0 0x55f65c7f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4210ff9640 10 client.20856 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f422c858640 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c7f2ee0 0x55f65c7f52d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a366b3b349a73f8a server_cookie=1060ff01de880f08 in_seq=0 out_seq=0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4210ff9640 1 -- 172.21.15.86:0/2798813843 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f65c6dad70 con 0x55f65c7f2ee0 2024-09-05T07:06:37.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4210ff9640 10 client.20856 handle_command_reply: tid=0 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 client.20856 shutdown 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 msgr2=0x55f65c77c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f421c0a4510 tx=0x7f421c0a40d0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 shutdown_connections 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55f65c7f2ee0 0x55f65c7f52d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f65c778370 0x55f65c77cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f65c76eb40 0x55f65c77c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4210ff9640 0 client.20856 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 --2- 172.21.15.86:0/2798813843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65c76d5d0 0x55f65c77c220 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 >> 172.21.15.86:0/2798813843 conn(0x55f65c736310 msgr2=0x55f65c776830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 shutdown_connections 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 1 -- 172.21.15.86:0/2798813843 wait complete. 2024-09-05T07:06:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.145+0000 7f4231696740 20 client.20856 trim_cache size 0 max 16384 2024-09-05T07:06:37.152 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:06:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 msgr2=0x7f42140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 0x7f42140a51b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4220009fd0 tx=0x7f4220004a40 comp rx=0 tx=0).stop 2024-09-05T07:06:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 msgr2=0x7f42281984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f421c00c890 tx=0x7f421c00cd50 comp rx=0 tx=0).stop 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 shutdown_connections 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f42140a2cf0 0x7f42140a51b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228106f10 0x7f422819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4228102790 0x7f42281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 --2- 172.21.15.86:0/2547548550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4228101de0 0x7f42281984b0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 >> 172.21.15.86:0/2547548550 conn(0x7f42280fd8c0 msgr2=0x7f42280fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 shutdown_connections 2024-09-05T07:06:37.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:37.146+0000 7f422f2bd640 1 -- 172.21.15.86:0/2547548550 wait complete. 2024-09-05T07:06:37.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:37 smithi086 ceph-mon[30943]: pgmap v799: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 206 op/s 2024-09-05T07:06:37.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:37 smithi096 ceph-mon[34468]: pgmap v799: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 206 op/s 2024-09-05T07:06:37.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:37 smithi115 ceph-mon[34791]: pgmap v799: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 206 op/s 2024-09-05T07:06:37.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.39 MB/sec execute 595 sec latency 4240.630 ms 2024-09-05T07:06:37.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:06:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 -- 172.21.15.86:0/920736662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 msgr2=0x7fa8001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 --2- 172.21.15.86:0/920736662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa8001021c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009900 tx=0x7fa7f002a410 comp rx=0 tx=0).stop 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 -- 172.21.15.86:0/920736662 shutdown_connections 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 --2- 172.21.15.86:0/920736662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 0x7fa80010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 --2- 172.21.15.86:0/920736662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa8001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 --2- 172.21.15.86:0/920736662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa8001021c0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 -- 172.21.15.86:0/920736662 >> 172.21.15.86:0/920736662 conn(0x7fa8000fd880 msgr2=0x7fa8000ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 -- 172.21.15.86:0/920736662 shutdown_connections 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 -- 172.21.15.86:0/920736662 wait complete. 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.634+0000 7fa805c5b640 1 Processor -- start 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 -- start start 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa800198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 0x7fa80019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa800111de0 con 0x7fa800101de0 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa800111c60 con 0x7fa800102790 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa805c5b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa800111f60 con 0x7fa800106f10 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49326/0 (socket says 172.21.15.86:49326) 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1842652969 learned_addr learned my addr 172.21.15.86:0/1842652969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa800198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7fffff640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 0x7fa80019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1842652969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 msgr2=0x7fa80019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 0x7fa80019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1842652969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 msgr2=0x7fa800198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa800198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1842652969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80019d2c0 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa800198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.635+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005940 tx=0x7fa7f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.636+0000 7fa7fcff9640 1 -- 172.21.15.86:0/1842652969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f003f030 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.636+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80019d550 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.636+0000 7fa7fcff9640 1 -- 172.21.15.86:0/1842652969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f0003aa0 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.636+0000 7fa7fcff9640 1 -- 172.21.15.86:0/1842652969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0035070 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.636+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001a4e90 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa7fcff9640 1 -- 172.21.15.86:0/1842652969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa7f0042030 con 0x7fa800101de0 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 Processor -- start 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 -- start start 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0a5d0 0x55bd88a0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bd889702f0 con 0x55bd88a15370 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bd88917cf0 con 0x55bd88a0bb40 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.638+0000 7fa808034740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bd8896c2b0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7fcff9640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 0x7fa7ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49330/0 (socket says 172.21.15.86:49330) 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 -- 172.21.15.86:0/355884361 learned_addr learned my addr 172.21.15.86:0/355884361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7fffff640 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0a5d0 0x55bd88a0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7fffff640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 0x7fa7ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 -- 172.21.15.86:0/355884361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0a5d0 msgr2=0x55bd88a0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0a5d0 0x55bd88a0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 -- 172.21.15.86:0/355884361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 msgr2=0x55bd88a0a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a0a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 -- 172.21.15.86:0/355884361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd888fc3a0 con 0x55bd88a15370 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7fcff9640 1 -- 172.21.15.86:0/1842652969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa7f00f00f0 con 0x7fa800101de0 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7feffd640 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800bd30 tx=0x7fa7e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f913b7f11836e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7fffff640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 0x7fa7ec0a51b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005c40 tx=0x7fa7f8009940 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/355884361 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8015070 con 0x55bd88a15370 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/355884361 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e800be20 con 0x55bd88a15370 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.639+0000 7fa7f5ffb640 1 -- 172.21.15.86:0/355884361 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8011840 con 0x55bd88a15370 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 -- 172.21.15.86:0/355884361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 msgr2=0x55bd88a17760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800bd30 tx=0x7fa7e800f730 comp rx=0 tx=0).stop 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 -- 172.21.15.86:0/355884361 shutdown_connections 2024-09-05T07:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a17760 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0a5d0 0x55bd88a0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 --2- 172.21.15.86:0/355884361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 -- 172.21.15.86:0/355884361 >> 172.21.15.86:0/355884361 conn(0x55bd889d3310 msgr2=0x55bd88a06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 -- 172.21.15.86:0/355884361 shutdown_connections 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.640+0000 7fa808034740 1 -- 172.21.15.86:0/355884361 wait complete. 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 -- start start 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 Processor -- start 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a15370 0x55bd88a125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bd88a18070 con 0x55bd88a0a5d0 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bd8892b060 con 0x55bd88a0bb40 2024-09-05T07:06:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bd88a181f0 con 0x55bd88a15370 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa808034740 10 client.0 _ll_register_callbacks cb 0x55bd8867e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49346/0 (socket says 172.21.15.86:49346) 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1506514107 learned_addr learned my addr 172.21.15.86:0/1506514107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7fffff640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a15370 0x55bd88a125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1506514107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a15370 msgr2=0x55bd88a125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a15370 0x55bd88a125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 msgr2=0x55bd88a11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1506514107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd886e13c0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.641+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00b8630 tx=0x7fa7f0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9933e136554065f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0049070 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa7f00ed680 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e800be20 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa7f003b460 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 10 client.20868 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bd88a23ad0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e800be20 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa7f00b8840 con 0x55bd88a0a5d0 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 10 client.20868 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 10 client.20868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 20 client.20868 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a24e70 0x55bd88a27280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 4 client.20868 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.642+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd8897d860 con 0x55bd88a24e70 2024-09-05T07:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.643+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a24e70 0x55bd88a27280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.643+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a24e70 0x55bd88a27280 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a64873c7243594d server_cookie=3ea97d98d8445ccd in_seq=0 out_seq=0 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.643+0000 7fa7deffd640 10 client.20868 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa7deffd640 1 -- 172.21.15.86:0/1506514107 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd8897d860 con 0x55bd88a24e70 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa7deffd640 10 client.20868 handle_command_reply: tid=0 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 client.20868 shutdown 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 msgr2=0x55bd88a14d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00b8630 tx=0x7fa7f0005b50 comp rx=0 tx=0).stop 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 shutdown_connections 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a24e70 0x55bd88a27280 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a15370 0x55bd88a125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0bb40 0x55bd88a11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa7deffd640 0 client.20868 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 --2- 172.21.15.86:0/1506514107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a14d30 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 >> 172.21.15.86:0/1506514107 conn(0x55bd889d3310 msgr2=0x55bd88a0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 shutdown_connections 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 1 -- 172.21.15.86:0/1506514107 wait complete. 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.644+0000 7fa808034740 20 client.20868 trim_cache size 0 max 16384 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 Processor -- start 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 -- start start 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0a5d0 0x55bd88a21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001aefd0 con 0x55bd88a15370 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b0b10 con 0x55bd88a0a5d0 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa808034740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001af6a0 con 0x55bd88a0bb40 2024-09-05T07:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49362/0 (socket says 172.21.15.86:49362) 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 -- 172.21.15.86:0/4127164014 learned_addr learned my addr 172.21.15.86:0/4127164014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0a5d0 0x55bd88a21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7feffd640 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 -- 172.21.15.86:0/4127164014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 msgr2=0x55bd88a16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 -- 172.21.15.86:0/4127164014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0a5d0 msgr2=0x55bd88a21310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0a5d0 0x55bd88a21310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 -- 172.21.15.86:0/4127164014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd88a064f0 con 0x55bd88a15370 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.657+0000 7fa7fffff640 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8012110 tx=0x7fa7f801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372761ee187fd4ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa7f57fa640 1 -- 172.21.15.86:0/4127164014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8012140 con 0x55bd88a15370 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa7f57fa640 1 -- 172.21.15.86:0/4127164014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f80122e0 con 0x55bd88a15370 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa7f57fa640 1 -- 172.21.15.86:0/4127164014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f802aa30 con 0x55bd88a15370 2024-09-05T07:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 -- 172.21.15.86:0/4127164014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 msgr2=0x55bd88a16ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8012110 tx=0x7fa7f801f000 comp rx=0 tx=0).stop 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 -- 172.21.15.86:0/4127164014 shutdown_connections 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a15370 0x55bd88a16ca0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 --2- 172.21.15.86:0/4127164014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a0a5d0 0x55bd88a21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 -- 172.21.15.86:0/4127164014 >> 172.21.15.86:0/4127164014 conn(0x55bd889d3310 msgr2=0x55bd88a17370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 -- 172.21.15.86:0/4127164014 shutdown_connections 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.658+0000 7fa808034740 1 -- 172.21.15.86:0/4127164014 wait complete. 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 -- start start 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 Processor -- start 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a2ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a15370 0x55bd88a2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b0440 con 0x55bd88a0a5d0 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001afd70 con 0x55bd88a15370 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001acfd0 con 0x55bd88a0bb40 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa808034740 10 client.0 _ll_register_callbacks cb 0x7fa7f0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49368/0 (socket says 172.21.15.86:49368) 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1133431867 learned_addr learned my addr 172.21.15.86:0/1133431867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7fffff640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a15370 0x55bd88a2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a2ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.659+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1133431867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 msgr2=0x55bd88a2ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a2ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a15370 msgr2=0x55bd88a2d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a15370 0x55bd88a2d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7ff7fe640 1 -- 172.21.15.86:0/1133431867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd88a1bdb0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7ff7fe640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00eda00 tx=0x7fa7f00ed8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727a31721ea15e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0049070 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa7f00c49b0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f80122e0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa7f00b8470 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 10 client.20874 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bd88a8ffa0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f80122e0 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa7f00bc430 con 0x55bd88a0a5d0 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 10 client.20874 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 10 client.20874 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:06:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 20 client.20874 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a905b0 0x55bd88a929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 4 client.20874 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.660+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd88977d70 con 0x55bd88a905b0 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a905b0 0x55bd88a929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa7deffd640 10 client.20874 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa7feffd640 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a905b0 0x55bd88a929d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=971e7707c5b9e6de server_cookie=367df8abb6aaa0ea in_seq=0 out_seq=0 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa7deffd640 1 -- 172.21.15.86:0/1133431867 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd88977d70 con 0x55bd88a905b0 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa7deffd640 10 client.20874 handle_command_reply: tid=0 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.661+0000 7fa808034740 1 client.20874 shutdown 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 msgr2=0x55bd88a2c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00eda00 tx=0x7fa7f00ed8b0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 shutdown_connections 2024-09-05T07:06:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x55bd88a905b0 0x55bd88a929d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bd88a15370 0x55bd88a2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bd88a0bb40 0x55bd88a2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 --2- 172.21.15.86:0/1133431867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bd88a0a5d0 0x55bd88a2c8c0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 >> 172.21.15.86:0/1133431867 conn(0x55bd889d3310 msgr2=0x55bd88a054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa7deffd640 0 client.20874 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 shutdown_connections 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 1 -- 172.21.15.86:0/1133431867 wait complete. 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa808034740 20 client.20874 trim_cache size 0 max 16384 2024-09-05T07:06:38.679 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:06:38.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 msgr2=0x7fa7ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 0x7fa7ec0a51b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8005c40 tx=0x7fa7f8009940 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 msgr2=0x7fa800198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005940 tx=0x7fa7f0004270 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 shutdown_connections 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa7ec0a2cf0 0x7fa7ec0a51b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa800106f10 0x7fa80019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa800102790 0x7fa800198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 --2- 172.21.15.86:0/1842652969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa800101de0 0x7fa800198310 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.662+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 >> 172.21.15.86:0/1842652969 conn(0x7fa8000fd880 msgr2=0x7fa8000fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.663+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 shutdown_connections 2024-09-05T07:06:38.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:38.663+0000 7fa805c5b640 1 -- 172.21.15.86:0/1842652969 wait complete. 2024-09-05T07:06:38.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.37 MB/sec execute 596 sec latency 5240.714 ms 2024-09-05T07:06:39.501 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:39 smithi086 ceph-mon[30943]: pgmap v800: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-05T07:06:39.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:06:39.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:39 smithi096 ceph-mon[34468]: pgmap v800: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-05T07:06:39.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:39 smithi115 ceph-mon[34791]: pgmap v800: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-05T07:06:39.774 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:39.797 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.36 MB/sec execute 597 sec latency 6240.754 ms 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 -- 172.21.15.86:0/74011477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102ef0 msgr2=0x7f962c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 --2- 172.21.15.86:0/74011477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102ef0 0x7f962c10da40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f9624009ca0 tx=0x7f962402a950 comp rx=0 tx=0).stop 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 -- 172.21.15.86:0/74011477 shutdown_connections 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 --2- 172.21.15.86:0/74011477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102ef0 0x7f962c10da40 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 --2- 172.21.15.86:0/74011477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c102570 0x7f962c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 --2- 172.21.15.86:0/74011477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c100e20 0x7f962c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 -- 172.21.15.86:0/74011477 >> 172.21.15.86:0/74011477 conn(0x7f962c0fc7d0 msgr2=0x7f962c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 -- 172.21.15.86:0/74011477 shutdown_connections 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.279+0000 7f9631253640 1 -- 172.21.15.86:0/74011477 wait complete. 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.280+0000 7f9631253640 1 Processor -- start 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.280+0000 7f9631253640 1 -- start start 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 0x7f962c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c102ef0 0x7f962c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f962c110ea0 con 0x7f962c102570 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f962c110d20 con 0x7f962c102ef0 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f962c111020 con 0x7f962c100e20 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49390/0 (socket says 172.21.15.86:49390) 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 -- 172.21.15.86:0/2414345849 learned_addr learned my addr 172.21.15.86:0/2414345849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:40.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962ad9d640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 0x7f962c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962b59e640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c102ef0 0x7f962c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 -- 172.21.15.86:0/2414345849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 msgr2=0x7f962c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 0x7f962c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 -- 172.21.15.86:0/2414345849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c102ef0 msgr2=0x7f962c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c102ef0 0x7f962c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 -- 172.21.15.86:0/2414345849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c19c2d0 con 0x7f962c102570 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962ad9d640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 0x7f962c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f962259c640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f961400b4a0 tx=0x7f961400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9628d99640 1 -- 172.21.15.86:0/2414345849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614014070 con 0x7f962c102570 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9628d99640 1 -- 172.21.15.86:0/2414345849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9614010ac0 con 0x7f962c102570 2024-09-05T07:06:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c19c5c0 con 0x7f962c102570 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.281+0000 7f9628d99640 1 -- 172.21.15.86:0/2414345849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614017c30 con 0x7f962c102570 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.282+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1a3ea0 con 0x7f962c102570 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.284+0000 7f9628d99640 1 -- 172.21.15.86:0/2414345849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9614017070 con 0x7f962c102570 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.284+0000 7f963362c740 1 Processor -- start 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.284+0000 7f963362c740 1 -- start start 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c734684090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734684a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f9628d99640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c7345ea2f0 con 0x55c734691580 2024-09-05T07:06:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c734591cf0 con 0x55c734685b40 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f963362c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c7345e62b0 con 0x55c7346845d0 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49406/0 (socket says 172.21.15.86:49406) 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962259c640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 -- 172.21.15.86:0/3291266270 learned_addr learned my addr 172.21.15.86:0/3291266270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962ad9d640 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734684a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c734684090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f9628d99640 1 -- 172.21.15.86:0/2414345849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f96140cef00 con 0x7f962c102570 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 -- 172.21.15.86:0/3291266270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 msgr2=0x55c734684a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734684a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 -- 172.21.15.86:0/3291266270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 msgr2=0x55c734684090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c734684090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 -- 172.21.15.86:0/3291266270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7345763a0 con 0x55c734691580 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962259c640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 0x7f96180a51b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96140cc810 tx=0x7f9614090580 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.285+0000 7f962b59e640 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f9624035000 tx=0x7f9624009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc0ec4a7ed629 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.286+0000 7f962159a640 1 -- 172.21.15.86:0/3291266270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034040 con 0x55c734691580 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.286+0000 7f962159a640 1 -- 172.21.15.86:0/3291266270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f962402fd70 con 0x55c734691580 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.286+0000 7f962159a640 1 -- 172.21.15.86:0/3291266270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624030b80 con 0x55c734691580 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.286+0000 7f963362c740 1 -- 172.21.15.86:0/3291266270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 msgr2=0x55c734693970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.286+0000 7f963362c740 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f9624035000 tx=0x7f9624009800 comp rx=0 tx=0).stop 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- 172.21.15.86:0/3291266270 shutdown_connections 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c734693970 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734684a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- 172.21.15.86:0/3291266270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c734684090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- 172.21.15.86:0/3291266270 >> 172.21.15.86:0/3291266270 conn(0x55c73464d310 msgr2=0x55c734680c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- 172.21.15.86:0/3291266270 shutdown_connections 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- 172.21.15.86:0/3291266270 wait complete. 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- start start 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 Processor -- start 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734690f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c73468e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c734694220 con 0x55c734691580 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c7345a5060 con 0x55c734685b40 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c7346943a0 con 0x55c7346845d0 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f963362c740 10 client.0 _ll_register_callbacks cb 0x55c7342f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49422/0 (socket says 172.21.15.86:49422) 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f962b59e640 1 -- 172.21.15.86:0/1706220238 learned_addr learned my addr 172.21.15.86:0/1706220238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.287+0000 7f962ad9d640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c73468e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734690f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 -- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 msgr2=0x55c734690f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734690f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 -- 172.21.15.86:0/1706220238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 msgr2=0x55c73468e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c73468e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 -- 172.21.15.86:0/1706220238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73435b3c0 con 0x55c734691580 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f962b59e640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f9624009a60 tx=0x7f9624030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6522d43d0eae409e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034030 con 0x55c734691580 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f9624039420 con 0x55c734691580 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962402fd70 con 0x55c734691580 2024-09-05T07:06:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f962403d040 con 0x55c734691580 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f963362c740 10 client.20886 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c734694220 con 0x55c734691580 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f962402fd70 con 0x55c734691580 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.288+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f9624048410 con 0x55c734691580 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 10 client.20886 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 10 client.20886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 20 client.20886 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c7346a0e40 0x55c7346a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 4 client.20886 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:40.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7345f7860 con 0x55c7346a0e40 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f962ad9d640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c7346a0e40 0x55c7346a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f96127fc640 10 client.20886 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.289+0000 7f962ad9d640 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c7346a0e40 0x55c7346a3230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c53c01e03851858d server_cookie=ea90d4fc6ade893b in_seq=0 out_seq=0 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f96127fc640 1 -- 172.21.15.86:0/1706220238 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7345f7860 con 0x55c7346a0e40 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f96127fc640 10 client.20886 handle_command_reply: tid=0 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 client.20886 shutdown 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 msgr2=0x55c73468e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f9624009a60 tx=0x7f9624030800 comp rx=0 tx=0).stop 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 shutdown_connections 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c7346a0e40 0x55c7346a3230 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468e820 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734685b40 0x55c73468e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f96127fc640 0 client.20886 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 --2- 172.21.15.86:0/1706220238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c734690f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 >> 172.21.15.86:0/1706220238 conn(0x55c73464d310 msgr2=0x55c734686960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 shutdown_connections 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 1 -- 172.21.15.86:0/1706220238 wait complete. 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.290+0000 7f963362c740 20 client.20886 trim_cache size 0 max 16384 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 Processor -- start 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 -- start start 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c73469d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734691580 0x55c7346a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1ad030 con 0x55c734685b40 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1aeb70 con 0x55c734691580 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.303+0000 7f963362c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1ad700 con 0x55c7346845d0 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49434/0 (socket says 172.21.15.86:49434) 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 -- 172.21.15.86:0/3012619586 learned_addr learned my addr 172.21.15.86:0/3012619586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962b59e640 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c73469d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 -- 172.21.15.86:0/3012619586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 msgr2=0x55c73469d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c73469d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 -- 172.21.15.86:0/3012619586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734691580 msgr2=0x55c7346a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962b59e640 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734691580 0x55c7346a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734691580 0x55c7346a8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 -- 172.21.15.86:0/3012619586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c734687120 con 0x55c734685b40 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f962ad9d640 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f961c00ccf0 tx=0x7f961c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30c2d397cad4a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f9620d99640 1 -- 172.21.15.86:0/3012619586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c004220 con 0x55c734685b40 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f9620d99640 1 -- 172.21.15.86:0/3012619586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961c0043c0 con 0x55c734685b40 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.304+0000 7f9620d99640 1 -- 172.21.15.86:0/3012619586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c01c730 con 0x55c734685b40 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- 172.21.15.86:0/3012619586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 msgr2=0x55c7346901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f961c00ccf0 tx=0x7f961c010800 comp rx=0 tx=0).stop 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- 172.21.15.86:0/3012619586 shutdown_connections 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c734691580 0x55c7346a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734685b40 0x55c7346901a0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- 172.21.15.86:0/3012619586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c7346845d0 0x55c73469d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- 172.21.15.86:0/3012619586 >> 172.21.15.86:0/3012619586 conn(0x55c73464d310 msgr2=0x55c73468fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- 172.21.15.86:0/3012619586 shutdown_connections 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- 172.21.15.86:0/3012619586 wait complete. 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- start start 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 Processor -- start 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7346845d0 0x55c734699f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c734685b40 0x55c73468b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1ae4a0 con 0x55c734691580 2024-09-05T07:06:40.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1addd0 con 0x55c7346845d0 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1ab030 con 0x55c734685b40 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.305+0000 7f963362c740 10 client.0 _ll_register_callbacks cb 0x7f962402f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49444/0 (socket says 172.21.15.86:49444) 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 -- 172.21.15.86:0/756178616 learned_addr learned my addr 172.21.15.86:0/756178616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962ad9d640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c734685b40 0x55c73468b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 -- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c734685b40 msgr2=0x55c73468b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c734685b40 0x55c73468b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 -- 172.21.15.86:0/756178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7346845d0 msgr2=0x55c734699f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7346845d0 0x55c734699f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 -- 172.21.15.86:0/756178616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73470a830 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f962b59e640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f9624009ca0 tx=0x7f9624005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ab14ebc33bd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f96127fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034040 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f96240483a0 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961c0043c0 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9624067050 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f963362c740 10 client.20892 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.306+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c73470ba10 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961c0043c0 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f962405d030 con 0x55c734691580 2024-09-05T07:06:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 10 client.20892 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 10 client.20892 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 20 client.20892 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c73470c130 0x55c73470e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 4 client.20892 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7345f1d70 con 0x55c73470c130 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f962ad9d640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c73470c130 0x55c73470e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f962ad9d640 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c73470c130 0x55c73470e550 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3e759c463d24607 server_cookie=d98d1e3f19486ada in_seq=0 out_seq=0 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.307+0000 7f96127fc640 10 client.20892 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:06:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f96127fc640 1 -- 172.21.15.86:0/756178616 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7345f1d70 con 0x55c73470c130 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f96127fc640 10 client.20892 handle_command_reply: tid=0 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 client.20892 shutdown 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 msgr2=0x55c73468b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f9624009ca0 tx=0x7f9624005c70 comp rx=0 tx=0).stop 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 shutdown_connections 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55c73470c130 0x55c73470e550 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c734691580 0x55c73468b8d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f96127fc640 0 client.20892 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c734685b40 0x55c73468b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 --2- 172.21.15.86:0/756178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7346845d0 0x55c734699f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 >> 172.21.15.86:0/756178616 conn(0x55c73464d310 msgr2=0x55c7346902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 shutdown_connections 2024-09-05T07:06:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 1 -- 172.21.15.86:0/756178616 wait complete. 2024-09-05T07:06:40.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f963362c740 20 client.20892 trim_cache size 0 max 16384 2024-09-05T07:06:40.344 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:06:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 msgr2=0x7f96180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 0x7f96180a51b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96140cc810 tx=0x7f9614090580 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 msgr2=0x7f962c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f961400b4a0 tx=0x7f961400b960 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 shutdown_connections 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962c102ef0 0x7f962c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f962c102570 0x7f962c197860 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 --2- 172.21.15.86:0/2414345849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c100e20 0x7f962c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.308+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 >> 172.21.15.86:0/2414345849 conn(0x7f962c0fc7d0 msgr2=0x7f962c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.309+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 shutdown_connections 2024-09-05T07:06:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:40.309+0000 7f9631253640 1 -- 172.21.15.86:0/2414345849 wait complete. 2024-09-05T07:06:40.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.35 MB/sec execute 598 sec latency 7240.782 ms 2024-09-05T07:06:41.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:06:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:41.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:41 smithi086 ceph-mon[30943]: pgmap v801: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2024-09-05T07:06:41.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:41 smithi096 ceph-mon[34468]: pgmap v801: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2024-09-05T07:06:41.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:41 smithi115 ceph-mon[34791]: pgmap v801: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 182 op/s 2024-09-05T07:06:41.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85738 6.34 MB/sec execute 599 sec latency 8241.074 ms 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 -- 172.21.15.86:0/1741015387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0101de0 msgr2=0x7fe2b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/1741015387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0101de0 0x7fe2b01021c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02a410 comp rx=0 tx=0).stop 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 -- 172.21.15.86:0/1741015387 shutdown_connections 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/1741015387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/1741015387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0102790 0x7fe2b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/1741015387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0101de0 0x7fe2b01021c0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 -- 172.21.15.86:0/1741015387 >> 172.21.15.86:0/1741015387 conn(0x7fe2b00fd8c0 msgr2=0x7fe2b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 -- 172.21.15.86:0/1741015387 shutdown_connections 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.855+0000 7fe2b73b1640 1 -- 172.21.15.86:0/1741015387 wait complete. 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 Processor -- start 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 -- start start 2024-09-05T07:06:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0101de0 0x7fe2b0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111e20 con 0x7fe2b0102790 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111ca0 con 0x7fe2b0101de0 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b73b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111fa0 con 0x7fe2b0106f10 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b494c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b494c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49462/0 (socket says 172.21.15.86:49462) 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.856+0000 7fe2b494c640 1 -- 172.21.15.86:0/2058436524 learned_addr learned my addr 172.21.15.86:0/2058436524 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b514d640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0101de0 0x7fe2b0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 -- 172.21.15.86:0/2058436524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 msgr2=0x7fe2b019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 -- 172.21.15.86:0/2058436524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0101de0 msgr2=0x7fe2b0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0101de0 0x7fe2b0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 -- 172.21.15.86:0/2058436524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b019d310 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b494c640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000d950 tx=0x7fe2a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2a67fc640 1 -- 172.21.15.86:0/2058436524 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0017030 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2a67fc640 1 -- 172.21.15.86:0/2058436524 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a0004020 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b019d600 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2a67fc640 1 -- 172.21.15.86:0/2058436524 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00047a0 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.857+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a4ee0 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.859+0000 7fe2a67fc640 1 -- 172.21.15.86:0/2058436524 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe2a00041d0 con 0x7fe2b0102790 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 Processor -- start 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 -- start start 2024-09-05T07:06:41.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfde1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfd472f0 con 0x55e6cfde15d0 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfceecf0 con 0x55e6cfdec370 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b978a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfd432b0 con 0x55e6cfde2b40 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2a67fc640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 0x7fe29c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49472/0 (socket says 172.21.15.86:49472) 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b514d640 1 -- 172.21.15.86:0/3905045372 learned_addr learned my addr 172.21.15.86:0/3905045372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.860+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 0x7fe29c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b594e640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfde1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2a67fc640 1 -- 172.21.15.86:0/2058436524 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe2a00cf2a0 con 0x7fe2b0102790 2024-09-05T07:06:41.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b494c640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 -- 172.21.15.86:0/3905045372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 msgr2=0x55e6cfde1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfde1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 -- 172.21.15.86:0/3905045372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 msgr2=0x55e6cfdee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 -- 172.21.15.86:0/3905045372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cfcd33a0 con 0x55e6cfde15d0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b594e640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfde1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b494c640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b514d640 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93991a2602a563ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 0x7fe29c0a5230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009270 tx=0x7fe2a8004ab0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe29b7fe640 1 -- 172.21.15.86:0/3905045372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac035040 con 0x55e6cfde15d0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe29b7fe640 1 -- 172.21.15.86:0/3905045372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2ac004030 con 0x55e6cfde15d0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.861+0000 7fe29b7fe640 1 -- 172.21.15.86:0/3905045372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0055e0 con 0x55e6cfde15d0 2024-09-05T07:06:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- 172.21.15.86:0/3905045372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 msgr2=0x55e6cfde1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac032bd0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- 172.21.15.86:0/3905045372 shutdown_connections 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde1a10 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 --2- 172.21.15.86:0/3905045372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfde1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- 172.21.15.86:0/3905045372 >> 172.21.15.86:0/3905045372 conn(0x55e6cfdaa310 msgr2=0x55e6cfdddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- 172.21.15.86:0/3905045372 shutdown_connections 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- 172.21.15.86:0/3905045372 wait complete. 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.862+0000 7fe2b978a740 1 -- start start 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 Processor -- start 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 0x55e6cfdebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfde9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfdef010 con 0x55e6cfdec370 2024-09-05T07:06:41.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfd02060 con 0x55e6cfde2b40 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e6cfdef190 con 0x55e6cfde15d0 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b978a740 10 client.0 _ll_register_callbacks cb 0x55e6cfa558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49482/0 (socket says 172.21.15.86:49482) 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 -- 172.21.15.86:0/2067278253 learned_addr learned my addr 172.21.15.86:0/2067278253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b514d640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 0x55e6cfdebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b494c640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfde9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 -- 172.21.15.86:0/2067278253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 msgr2=0x55e6cfdebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 0x55e6cfdebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 -- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 msgr2=0x55e6cfde9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfde9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b594e640 1 -- 172.21.15.86:0/2067278253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cfab83c0 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.863+0000 7fe2b514d640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 0x55e6cfdebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b594e640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8010ce0 tx=0x7fe2a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888a4b17bc6912e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8023070 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe2a8004b20 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ac004030 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8004d20 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 10 client.20904 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e6cfdfaac0 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2ac004030 con 0x55e6cfdec370 2024-09-05T07:06:41.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe2a801c040 con 0x55e6cfdec370 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 10 client.20904 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 10 client.20904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 20 client.20904 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfdfe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 4 client.20904 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.864+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6cfd54860 con 0x55e6cfdfbf80 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.865+0000 7fe2b514d640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfdfe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.865+0000 7fe298ff9640 10 client.20904 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.865+0000 7fe2b514d640 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfdfe3d0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8c2ef96693066c server_cookie=1ed4042de76d2092 in_seq=0 out_seq=0 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe298ff9640 1 -- 172.21.15.86:0/2067278253 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6cfd54860 con 0x55e6cfdfbf80 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe298ff9640 10 client.20904 handle_command_reply: tid=0 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 client.20904 shutdown 2024-09-05T07:06:41.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 msgr2=0x55e6cfde97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8010ce0 tx=0x7fe2a8004a40 comp rx=0 tx=0).stop 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 shutdown_connections 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfdfe3d0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe298ff9640 0 client.20904 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfdec370 0x55e6cfde97e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfde9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 --2- 172.21.15.86:0/2067278253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde15d0 0x55e6cfdebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 >> 172.21.15.86:0/2067278253 conn(0x55e6cfdaa310 msgr2=0x55e6cfde3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 shutdown_connections 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 1 -- 172.21.15.86:0/2067278253 wait complete. 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.866+0000 7fe2b978a740 20 client.20904 trim_cache size 0 max 16384 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 Processor -- start 2024-09-05T07:06:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 -- start start 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfdee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfdec370 0x55e6cfddd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01acf50 con 0x55e6cfde15d0 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01aea90 con 0x55e6cfde2b40 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b978a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01ad620 con 0x55e6cfdec370 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49494/0 (socket says 172.21.15.86:49494) 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 -- 172.21.15.86:0/4109510126 learned_addr learned my addr 172.21.15.86:0/4109510126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 -- 172.21.15.86:0/4109510126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfdec370 msgr2=0x55e6cfddd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfdec370 0x55e6cfddd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 -- 172.21.15.86:0/4109510126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 msgr2=0x55e6cfdee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfdee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 -- 172.21.15.86:0/4109510126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6cfde75a0 con 0x55e6cfde15d0 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.879+0000 7fe2b514d640 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x55e6cfddeac0 tx=0x7fe2ac0051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664ad5fa5d22796f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe29affd640 1 -- 172.21.15.86:0/4109510126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac05b070 con 0x55e6cfde15d0 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe29affd640 1 -- 172.21.15.86:0/4109510126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2ac0316a0 con 0x55e6cfde15d0 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe29affd640 1 -- 172.21.15.86:0/4109510126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac035040 con 0x55e6cfde15d0 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 -- 172.21.15.86:0/4109510126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 msgr2=0x55e6cfde9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x55e6cfddeac0 tx=0x7fe2ac0051c0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 -- 172.21.15.86:0/4109510126 shutdown_connections 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfdec370 0x55e6cfddd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfde2b40 0x55e6cfdee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 --2- 172.21.15.86:0/4109510126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfde9760 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 -- 172.21.15.86:0/4109510126 >> 172.21.15.86:0/4109510126 conn(0x55e6cfdaa310 msgr2=0x55e6cfdd8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 -- 172.21.15.86:0/4109510126 shutdown_connections 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.880+0000 7fe2b978a740 1 -- 172.21.15.86:0/4109510126 wait complete. 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 -- start start 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 Processor -- start 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfded640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01ae3c0 con 0x55e6cfde15d0 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01adcf0 con 0x55e6cfdec370 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01aaf50 con 0x55e6cfde2b40 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b978a740 10 client.0 _ll_register_callbacks cb 0x7fe2a80104f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b514d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49504/0 (socket says 172.21.15.86:49504) 2024-09-05T07:06:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.881+0000 7fe2b514d640 1 -- 172.21.15.86:0/1616449012 learned_addr learned my addr 172.21.15.86:0/1616449012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b494c640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfded640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 -- 172.21.15.86:0/1616449012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 msgr2=0x55e6cfded640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfded640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 -- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 msgr2=0x55e6cfdedb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 -- 172.21.15.86:0/1616449012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6cfdee1c0 con 0x55e6cfde15d0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b494c640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfded640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b514d640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac03f8e0 tx=0x7fe2ac005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357019e1cfa6db71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac05b040 con 0x55e6cfde15d0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe2ac031850 con 0x55e6cfde15d0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ac0316a0 con 0x55e6cfde15d0 2024-09-05T07:06:41.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe2ac035040 con 0x55e6cfde15d0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b978a740 10 client.20910 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e6cfdf3300 con 0x55e6cfde15d0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.882+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2ac0316a0 con 0x55e6cfde15d0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe2ac05e070 con 0x55e6cfde15d0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 10 client.20910 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 10 client.20910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 20 client.20910 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfe68c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 4 client.20910 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6cfd4ed70 con 0x55e6cfdfbf80 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b494c640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfe68c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe298ff9640 10 client.20910 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b494c640 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfe68c80 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc01b45437c9328f server_cookie=2dfa78b07629e6fb in_seq=0 out_seq=0 2024-09-05T07:06:41.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe298ff9640 1 -- 172.21.15.86:0/1616449012 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6cfd4ed70 con 0x55e6cfdfbf80 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe298ff9640 10 client.20910 handle_command_reply: tid=0 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 1 client.20910 shutdown 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 msgr2=0x55e6cfdf4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.883+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac03f8e0 tx=0x7fe2ac005250 comp rx=0 tx=0).stop 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 shutdown_connections 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e6cfdfbf80 0x55e6cfe68c80 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6cfdec370 0x55e6cfdedb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e6cfde2b40 0x55e6cfded640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 --2- 172.21.15.86:0/1616449012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6cfde15d0 0x55e6cfdf4920 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 >> 172.21.15.86:0/1616449012 conn(0x55e6cfdaa310 msgr2=0x55e6cfddd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe298ff9640 0 client.20910 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 shutdown_connections 2024-09-05T07:06:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 1 -- 172.21.15.86:0/1616449012 wait complete. 2024-09-05T07:06:41.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b978a740 20 client.20910 trim_cache size 0 max 16384 2024-09-05T07:06:41.902 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:06:41.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 msgr2=0x7fe29c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 0x7fe29c0a5230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009270 tx=0x7fe2a8004ab0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 msgr2=0x7fe2b01988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000d950 tx=0x7fe2a000de10 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 shutdown_connections 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe29c0a2d70 0x7fe29c0a5230 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b0106f10 0x7fe2b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0102790 0x7fe2b01988a0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 --2- 172.21.15.86:0/2058436524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2b0101de0 0x7fe2b0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.884+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 >> 172.21.15.86:0/2058436524 conn(0x7fe2b00fd8c0 msgr2=0x7fe2b00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.885+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 shutdown_connections 2024-09-05T07:06:41.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:41.885+0000 7fe2b73b1640 1 -- 172.21.15.86:0/2058436524 wait complete. 2024-09-05T07:06:42.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 600 sec 2024-09-05T07:06:43.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:43 smithi096 ceph-mon[34468]: pgmap v802: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-09-05T07:06:43.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T07:06:43.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T07:06:43.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:43 smithi086 ceph-mon[30943]: pgmap v802: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-09-05T07:06:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:43 smithi115 ceph-mon[34791]: pgmap v802: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-09-05T07:06:43.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 601 sec 2024-09-05T07:06:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 -- 172.21.15.86:0/1061175768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 msgr2=0x7f9e5810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 --2- 172.21.15.86:0/1061175768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e5810bb50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009900 tx=0x7f9e4c02a480 comp rx=0 tx=0).stop 2024-09-05T07:06:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 -- 172.21.15.86:0/1061175768 shutdown_connections 2024-09-05T07:06:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 --2- 172.21.15.86:0/1061175768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 --2- 172.21.15.86:0/1061175768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e5810bb50 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 --2- 172.21.15.86:0/1061175768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 0x7f9e58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 -- 172.21.15.86:0/1061175768 >> 172.21.15.86:0/1061175768 conn(0x7f9e580fc7a0 msgr2=0x7f9e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.383+0000 7f9e5e935640 1 -- 172.21.15.86:0/1061175768 shutdown_connections 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 -- 172.21.15.86:0/1061175768 wait complete. 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 Processor -- start 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 -- start start 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 0x7f9e581984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.384+0000 7f9e5e935640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111ee0 con 0x7f9e58103660 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e5e935640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58111d60 con 0x7f9e5810c170 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e5e935640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58112060 con 0x7f9e58102d40 2024-09-05T07:06:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43124/0 (socket says 172.21.15.86:43124) 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 -- 172.21.15.86:0/3288209510 learned_addr learned my addr 172.21.15.86:0/3288209510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 0x7f9e581984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 -- 172.21.15.86:0/3288209510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 msgr2=0x7f9e581984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 0x7f9e581984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 -- 172.21.15.86:0/3288209510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 msgr2=0x7f9e5819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 -- 172.21.15.86:0/3288209510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5819d490 con 0x7f9e58103660 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006010 tx=0x7f9e4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e557fa640 1 -- 172.21.15.86:0/3288209510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c03f030 con 0x7f9e58103660 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5819d720 con 0x7f9e58103660 2024-09-05T07:06:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e557fa640 1 -- 172.21.15.86:0/3288209510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c003800 con 0x7f9e58103660 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.385+0000 7f9e557fa640 1 -- 172.21.15.86:0/3288209510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c035070 con 0x7f9e58103660 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.386+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a5060 con 0x7f9e58103660 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e557fa640 1 -- 172.21.15.86:0/3288209510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9e4c0039b0 con 0x7f9e58103660 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 Processor -- start 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 -- start start 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x563136290e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x56313629fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e557fa640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 0x7f9e400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631361f6280 con 0x5631362a0150 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56313619db10 con 0x563136290a20 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631361f2240 con 0x563136291340 2024-09-05T07:06:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e5ced2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e5ced2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43134/0 (socket says 172.21.15.86:43134) 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3643058318 learned_addr learned my addr 172.21.15.86:0/3643058318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e557fa640 1 -- 172.21.15.86:0/3288209510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f9e4c0f01c0 con 0x7f9e58103660 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.388+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 0x7f9e400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x56313629fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x563136290e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3643058318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 msgr2=0x56313629fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x56313629fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3643058318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 msgr2=0x563136290e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x563136290e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3643058318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631361aa1f0 con 0x5631362a0150 2024-09-05T07:06:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 0x7f9e400a51b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009990 tx=0x7f9e48009110 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000ca30 tx=0x7f9e50009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992e16aedcf08441 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e3e7fc640 1 -- 172.21.15.86:0/3643058318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500077b0 con 0x5631362a0150 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e3e7fc640 1 -- 172.21.15.86:0/3643058318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50010030 con 0x5631362a0150 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.389+0000 7f9e3e7fc640 1 -- 172.21.15.86:0/3643058318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000ab80 con 0x5631362a0150 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3643058318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 msgr2=0x5631362a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000ca30 tx=0x7f9e50009a60 comp rx=0 tx=0).stop 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3643058318 shutdown_connections 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x5631362a2540 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x56313629fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3643058318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x563136290e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3643058318 >> 172.21.15.86:0/3643058318 conn(0x56313628d0f0 msgr2=0x56313628d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3643058318 shutdown_connections 2024-09-05T07:06:44.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3643058318 wait complete. 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 -- start start 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.390+0000 7f9e60d0e740 1 Processor -- start 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x56313629d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x56313629db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631362a2d20 con 0x563136291340 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631361b0f10 con 0x563136290a20 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631362a2ea0 con 0x5631362a0150 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 10 client.0 _ll_register_callbacks cb 0x563136179a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43142/0 (socket says 172.21.15.86:43142) 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 -- 172.21.15.86:0/3894689203 learned_addr learned my addr 172.21.15.86:0/3894689203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x56313629d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x56313629db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 -- 172.21.15.86:0/3894689203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 msgr2=0x56313629db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x56313629db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 -- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 msgr2=0x56313629d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x56313629d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 -- 172.21.15.86:0/3894689203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563135f676a0 con 0x563136291340 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x56313629d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c031440 tx=0x7f9e4c0bca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d6662a1e41bc8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c049040 con 0x563136291340 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f9e4c0ed6f0 con 0x563136291340 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.391+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50010030 con 0x563136291340 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c0ed8f0 con 0x563136291340 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 10 client.20922 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5631362aea40 con 0x563136291340 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50010030 con 0x563136291340 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f9e4c0edbc0 con 0x563136291340 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 10 client.20922 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 10 client.20922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 20 client.20922 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631362afde0 0x5631362b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 4 client.20922 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631362037f0 con 0x5631362afde0 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.392+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631362afde0 0x5631362b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.393+0000 7f9e377fe640 10 client.20922 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.393+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631362afde0 0x5631362b2250 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d737b51db7cc21ec server_cookie=3c07e528648cb815 in_seq=0 out_seq=0 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.393+0000 7f9e377fe640 1 -- 172.21.15.86:0/3894689203 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631362037f0 con 0x5631362afde0 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.393+0000 7f9e377fe640 10 client.20922 handle_command_reply: tid=0 2024-09-05T07:06:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.393+0000 7f9e60d0e740 1 client.20922 shutdown 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 msgr2=0x56313629d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c031440 tx=0x7f9e4c0bca80 comp rx=0 tx=0).stop 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 shutdown_connections 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5631362afde0 0x5631362b2250 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x56313629db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x56313629d650 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e377fe640 0 client.20922 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3894689203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x56313629d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 >> 172.21.15.86:0/3894689203 conn(0x56313628d0f0 msgr2=0x563136292e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 shutdown_connections 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3894689203 wait complete. 2024-09-05T07:06:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.394+0000 7f9e60d0e740 20 client.20922 trim_cache size 0 max 16384 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.411+0000 7f9e60d0e740 1 Processor -- start 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.411+0000 7f9e60d0e740 1 -- start start 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.411+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x5631362b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581acfe0 con 0x5631362a0150 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581aeb20 con 0x563136290a20 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581ad6b0 con 0x563136291340 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43144/0 (socket says 172.21.15.86:43144) 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3008352966 learned_addr learned my addr 172.21.15.86:0/3008352966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e577fe640 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x5631362b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e57fff640 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3008352966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 msgr2=0x5631362b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x5631362b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3008352966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 msgr2=0x5631362b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 -- 172.21.15.86:0/3008352966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563136291170 con 0x5631362a0150 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007680 tx=0x7f9e50004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f0563821add43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e3dffb640 1 -- 172.21.15.86:0/3008352966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000c0d0 con 0x5631362a0150 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e3dffb640 1 -- 172.21.15.86:0/3008352966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50010040 con 0x5631362a0150 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.412+0000 7f9e3dffb640 1 -- 172.21.15.86:0/3008352966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001b7b0 con 0x5631362a0150 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3008352966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 msgr2=0x56313629a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007680 tx=0x7f9e50004b50 comp rx=0 tx=0).stop 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3008352966 shutdown_connections 2024-09-05T07:06:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631362a0150 0x56313629a390 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563136291340 0x5631362b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/3008352966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3008352966 >> 172.21.15.86:0/3008352966 conn(0x56313628d0f0 msgr2=0x56313629f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3008352966 shutdown_connections 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- 172.21.15.86:0/3008352966 wait complete. 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.413+0000 7f9e60d0e740 1 -- start start 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 Processor -- start 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x5631362a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581ae450 con 0x563136291340 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581add80 con 0x563136290a20 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581aafe0 con 0x5631362a0150 2024-09-05T07:06:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e60d0e740 10 client.0 _ll_register_callbacks cb 0x7f9e4c0bc010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43152/0 (socket says 172.21.15.86:43152) 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 -- 172.21.15.86:0/1325388056 learned_addr learned my addr 172.21.15.86:0/1325388056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e5ced2640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x5631362a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 -- 172.21.15.86:0/1325388056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 msgr2=0x5631362a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x5631362a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 -- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 msgr2=0x5631362a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e577fe640 1 -- 172.21.15.86:0/1325388056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631362a5640 con 0x563136291340 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.414+0000 7f9e57fff640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e577fe640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0b8c10 tx=0x7f9e4c0b8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84c21c3bb4cdf7c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0bbba0 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f9e4c0c4a20 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50010040 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c0ed8f0 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 10 client.20928 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56313631b230 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50010040 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f9e4c0b8020 con 0x563136291340 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 10 client.20928 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 10 client.20928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 20 client.20928 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56313631b810 0x56313631dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 4 client.20928 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T07:06:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.415+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5631361fdd00 con 0x56313631b810 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.416+0000 7f9e57fff640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56313631b810 0x56313631dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.416+0000 7f9e377fe640 10 client.20928 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.416+0000 7f9e57fff640 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56313631b810 0x56313631dc00 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=924126121d5298c4 server_cookie=23c00c661bdc6f74 in_seq=0 out_seq=0 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e377fe640 1 -- 172.21.15.86:0/1325388056 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5631361fdd00 con 0x56313631b810 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e377fe640 10 client.20928 handle_command_reply: tid=0 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 client.20928 shutdown 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 msgr2=0x5631362a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0b8c10 tx=0x7f9e4c0b8ae0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 shutdown_connections 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x56313631b810 0x56313631dc00 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631362a0150 0x5631362a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563136291340 0x5631362a8670 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 --2- 172.21.15.86:0/1325388056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563136290a20 0x5631362a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.417+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 >> 172.21.15.86:0/1325388056 conn(0x56313628d0f0 msgr2=0x56313628ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e377fe640 0 client.20928 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 shutdown_connections 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e60d0e740 1 -- 172.21.15.86:0/1325388056 wait complete. 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e60d0e740 20 client.20928 trim_cache size 0 max 16384 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "f3acd226-dc2c-494b-a3be-9df7dcad5fb5", 2024-09-05T07:06:44.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 msgr2=0x7f9e400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 0x7f9e400a51b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009990 tx=0x7f9e48009110 comp rx=0 tx=0).stop 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 msgr2=0x7f9e58198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006010 tx=0x7f9e4c0042f0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 shutdown_connections 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9e400a2cf0 0x7f9e400a51b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5810c170 0x7f9e5819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e58103660 0x7f9e58198a20 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 --2- 172.21.15.86:0/3288209510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e58102d40 0x7f9e581984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 >> 172.21.15.86:0/3288209510 conn(0x7f9e580fc7a0 msgr2=0x7f9e580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 shutdown_connections 2024-09-05T07:06:44.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:44.418+0000 7f9e5e935640 1 -- 172.21.15.86:0/3288209510 wait complete. 2024-09-05T07:06:44.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 602 sec 2024-09-05T07:06:45.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: pgmap v803: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub queued for path: / 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:45 smithi086 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: pgmap v803: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub queued for path: / 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: active paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:45 smithi096 ceph-mon[34468]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: pgmap v803: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-09-05T07:06:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub queued for path: / 2024-09-05T07:06:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: active paths [/] 2024-09-05T07:06:45.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:45 smithi115 ceph-mon[34791]: scrub summary: idle+waiting paths [/] 2024-09-05T07:06:45.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 603 sec 2024-09-05T07:06:46.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:06:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1239288805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d4102d30 msgr2=0x7fa1d4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1239288805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d4102d30 0x7fa1d4103110 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009930 tx=0x7fa1c402a420 comp rx=0 tx=0).stop 2024-09-05T07:06:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1239288805 shutdown_connections 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1239288805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d41082a0 0x7fa1d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1239288805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41036e0 0x7fa1d4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1239288805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d4102d30 0x7fa1d4103110 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1239288805 >> 172.21.15.86:0/1239288805 conn(0x7fa1d40fc7d0 msgr2=0x7fa1d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1239288805 shutdown_connections 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.051+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1239288805 wait complete. 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 Processor -- start 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 -- start start 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d4102d30 0x7fa1d4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41082a0 0x7fa1d419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4111f50 con 0x7fa1d41036e0 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4111dd0 con 0x7fa1d4102d30 2024-09-05T07:06:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1da0e4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41120d0 con 0x7fa1d41082a0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43168/0 (socket says 172.21.15.86:43168) 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/1085521747 learned_addr learned my addr 172.21.15.86:0/1085521747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.052+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41082a0 0x7fa1d419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d4102d30 0x7fa1d4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/1085521747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41082a0 msgr2=0x7fa1d419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41082a0 0x7fa1d419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/1085521747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d4102d30 msgr2=0x7fa1d4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d4102d30 0x7fa1d4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/1085521747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d419d400 con 0x7fa1d41036e0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c8c0 tx=0x7fa1c800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d0ff9640 1 -- 172.21.15.86:0/1085521747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8007bb0 con 0x7fa1d41036e0 2024-09-05T07:06:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d0ff9640 1 -- 172.21.15.86:0/1085521747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1c8007d50 con 0x7fa1d41036e0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d419d6f0 con 0x7fa1d41036e0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1d0ff9640 1 -- 172.21.15.86:0/1085521747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80187c0 con 0x7fa1d41036e0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.053+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41a4fd0 con 0x7fa1d41036e0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.055+0000 7fa1d0ff9640 1 -- 172.21.15.86:0/1085521747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa1c800f450 con 0x7fa1d41036e0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 Processor -- start 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 -- start start 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed55fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55baed4b82e0 con 0x55baed5524a0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55baed45fce0 con 0x55baed552dc0 2024-09-05T07:06:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55baed4b42a0 con 0x55baed55d680 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43170/0 (socket says 172.21.15.86:43170) 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 -- 172.21.15.86:0/3988024971 learned_addr learned my addr 172.21.15.86:0/3988024971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d0ff9640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 0x7fa1bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 -- 172.21.15.86:0/3988024971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 msgr2=0x55baed55fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed55fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 -- 172.21.15.86:0/3988024971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 msgr2=0x55baed55d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.056+0000 7fa1d3fff640 1 -- 172.21.15.86:0/3988024971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baed444440 con 0x55baed5524a0 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1d0ff9640 1 -- 172.21.15.86:0/1085521747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa1c80cf940 con 0x7fa1d41036e0 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 0x7fa1bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc010440 tx=0x7fa1cc010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144869ad6b02ab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1b9ffb640 1 -- 172.21.15.86:0/3988024971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01d400 con 0x55baed5524a0 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1b9ffb640 1 -- 172.21.15.86:0/3988024971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1cc01da50 con 0x55baed5524a0 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.057+0000 7fa1b9ffb640 1 -- 172.21.15.86:0/3988024971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01c740 con 0x55baed5524a0 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 0x7fa1bc0a51b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc021000 tx=0x7fa1cc01e000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/3988024971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 msgr2=0x55baed552880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc010440 tx=0x7fa1cc010900 comp rx=0 tx=0).stop 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/3988024971 shutdown_connections 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed55fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/3988024971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed552880 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/3988024971 >> 172.21.15.86:0/3988024971 conn(0x55baed54ea30 msgr2=0x55baed54ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/3988024971 shutdown_connections 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/3988024971 wait complete. 2024-09-05T07:06:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.058+0000 7fa1dc4bd740 1 -- start start 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 Processor -- start 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55baed564660 con 0x55baed55d680 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55baed473050 con 0x55baed552dc0 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55baed5647e0 con 0x55baed5524a0 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1dc4bd740 10 client.0 _ll_register_callbacks cb 0x55baed4b3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43182/0 (socket says 172.21.15.86:43182) 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49900/0 (socket says 172.21.15.86:49900) 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 -- 172.21.15.86:0/1799860245 learned_addr learned my addr 172.21.15.86:0/1799860245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 -- 172.21.15.86:0/1799860245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 msgr2=0x55baed55d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 -- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 msgr2=0x55baed55a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d3fff640 1 -- 172.21.15.86:0/1799860245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baed2293c0 con 0x55baed55d680 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.059+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc01c480 tx=0x7fa1cc01c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5faeac000abe9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01ccb0 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc01da50 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa1cc01d400 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa1cc03d050 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1dc4bd740 10 client.20940 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.060+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55baed570100 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1cc01da50 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa1cc038030 con 0x55baed55d680 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 10 client.20940 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 10 client.20940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 20 client.20940 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed571510 0x55baed573900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 4 client.20940 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baed4c5850 con 0x55baed571510 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.061+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed571510 0x55baed573900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed571510 0x55baed573900 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8278841764bccd server_cookie=3f26d6a8a1134bee in_seq=0 out_seq=0 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1b2ffd640 10 client.20940 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/1799860245 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55baed4c5850 con 0x55baed571510 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1b2ffd640 10 client.20940 handle_command_reply: tid=0 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1dc4bd740 1 client.20940 shutdown 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 msgr2=0x55baed55aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.062+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc01c480 tx=0x7fa1cc01c7d0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 shutdown_connections 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed571510 0x55baed573900 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed55d680 0x55baed55aaf0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed55a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1b2ffd640 0 client.20940 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/1799860245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed55d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 >> 172.21.15.86:0/1799860245 conn(0x55baed54ea30 msgr2=0x55baed554760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 shutdown_connections 2024-09-05T07:06:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/1799860245 wait complete. 2024-09-05T07:06:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.063+0000 7fa1dc4bd740 20 client.20940 trim_cache size 0 max 16384 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 Processor -- start 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 -- start start 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed5790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed579600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41acf80 con 0x55baed5524a0 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41aeac0 con 0x55baed552dc0 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41ad650 con 0x55baed55d680 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43194/0 (socket says 172.21.15.86:43194) 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1d37fe640 1 -- 172.21.15.86:0/4290454867 learned_addr learned my addr 172.21.15.86:0/4290454867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed5790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.080+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed579600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 -- 172.21.15.86:0/4290454867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 msgr2=0x55baed579600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed579600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 -- 172.21.15.86:0/4290454867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 msgr2=0x55baed5790c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed5790c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 -- 172.21.15.86:0/4290454867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baed573c70 con 0x55baed5524a0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed579600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69b2536ace6af66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1b97fa640 1 -- 172.21.15.86:0/4290454867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4049030 con 0x55baed5524a0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1b97fa640 1 -- 172.21.15.86:0/4290454867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1c4031950 con 0x55baed5524a0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.081+0000 7fa1b97fa640 1 -- 172.21.15.86:0/4290454867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4035040 con 0x55baed5524a0 2024-09-05T07:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/4290454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 msgr2=0x55baed559c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4005fd0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/4290454867 shutdown_connections 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed55d680 0x55baed579600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed552dc0 0x55baed5790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/4290454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed5524a0 0x55baed559c20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/4290454867 >> 172.21.15.86:0/4290454867 conn(0x55baed54ea30 msgr2=0x55baed55c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/4290454867 shutdown_connections 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/4290454867 wait complete. 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.082+0000 7fa1dc4bd740 1 -- start start 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 Processor -- start 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed569d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 0x55baed565100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41ae3f0 con 0x55baed552dc0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41add20 con 0x55baed55d680 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d41aaf80 con 0x55baed5524a0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1dc4bd740 10 client.0 _ll_register_callbacks cb 0x7fa1cc020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43200/0 (socket says 172.21.15.86:43200) 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/258475742 learned_addr learned my addr 172.21.15.86:0/258475742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 0x55baed565100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed569d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/258475742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 msgr2=0x55baed569d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed569d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 msgr2=0x55baed565100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 0x55baed565100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d2ffd640 1 -- 172.21.15.86:0/258475742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baed565930 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d3fff640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 0x55baed565100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.083+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed569d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1d2ffd640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ccc50 tx=0x7fa1c809c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483c6fdaceb6755a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80b3070 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fa1c80cce70 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c4031950 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa1c80cd070 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 10 client.20946 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55baed5dbca0 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1c4031950 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa1c80b4bc0 con 0x55baed552dc0 2024-09-05T07:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 10 client.20946 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 10 client.20946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 20 client.20946 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed5dc3d0 0x55baed5de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 4 client.20946 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.084+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55baed4bfd60 con 0x55baed5dc3d0 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed5dc3d0 0x55baed5de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1b2ffd640 10 client.20946 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1d37fe640 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed5dc3d0 0x55baed5de7f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c575e62c6a348 server_cookie=8363ede39da08094 in_seq=0 out_seq=0 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1b2ffd640 1 -- 172.21.15.86:0/258475742 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55baed4bfd60 con 0x55baed5dc3d0 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1b2ffd640 10 client.20946 handle_command_reply: tid=0 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.085+0000 7fa1dc4bd740 1 client.20946 shutdown 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 msgr2=0x55baed56a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80ccc50 tx=0x7fa1c809c0c0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 shutdown_connections 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55baed5dc3d0 0x55baed5de7f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55baed55d680 0x55baed565100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55baed552dc0 0x55baed56a260 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 --2- 172.21.15.86:0/258475742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55baed5524a0 0x55baed569d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 >> 172.21.15.86:0/258475742 conn(0x55baed54ea30 msgr2=0x55baed54d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1b2ffd640 0 client.20946 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 shutdown_connections 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 1 -- 172.21.15.86:0/258475742 wait complete. 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1dc4bd740 20 client.20946 trim_cache size 0 max 16384 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:46.099 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: "f3acd226-dc2c-494b-a3be-9df7dcad5fb5": { 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "f3acd226-dc2c-494b-a3be-9df7dcad5fb5", 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:46.100 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 msgr2=0x7fa1bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.086+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 0x7fa1bc0a51b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc021000 tx=0x7fa1cc01e000 comp rx=0 tx=0).stop 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 msgr2=0x7fa1d4198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c8c0 tx=0x7fa1c800cd80 comp rx=0 tx=0).stop 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 shutdown_connections 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa1bc0a2cf0 0x7fa1bc0a51b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1d41082a0 0x7fa1d419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1d41036e0 0x7fa1d4198990 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 --2- 172.21.15.86:0/1085521747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d4102d30 0x7fa1d4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:46.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 >> 172.21.15.86:0/1085521747 conn(0x7fa1d40fc7d0 msgr2=0x7fa1d4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:46.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 shutdown_connections 2024-09-05T07:06:46.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:46.087+0000 7fa1da0e4640 1 -- 172.21.15.86:0/1085521747 wait complete. 2024-09-05T07:06:46.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 604 sec 2024-09-05T07:06:46.917 INFO:tasks.cephfs.filesystem:scrub status for tag:f3acd226-dc2c-494b-a3be-9df7dcad5fb5 - {'path': '/', 'tag': 'f3acd226-dc2c-494b-a3be-9df7dcad5fb5', 'options': 'recursive,force'} 2024-09-05T07:06:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:47.409 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:47 smithi086 ceph-mon[30943]: pgmap v804: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 -- 172.21.15.86:0/608874004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00103660 msgr2=0x7f0a0010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 --2- 172.21.15.86:0/608874004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00103660 0x7f0a0010bb50 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009900 tx=0x7f09f002a480 comp rx=0 tx=0).stop 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 -- 172.21.15.86:0/608874004 shutdown_connections 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 --2- 172.21.15.86:0/608874004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0010c170 0x7f0a0010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 --2- 172.21.15.86:0/608874004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00103660 0x7f0a0010bb50 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 --2- 172.21.15.86:0/608874004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a00102d40 0x7f0a00103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 -- 172.21.15.86:0/608874004 >> 172.21.15.86:0/608874004 conn(0x7f0a000fc7a0 msgr2=0x7f0a000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 -- 172.21.15.86:0/608874004 shutdown_connections 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.484+0000 7f0a084ee640 1 -- 172.21.15.86:0/608874004 wait complete. 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 Processor -- start 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 -- start start 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 0x7f0a001989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a0010c170 0x7f0a0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00111e70 con 0x7f0a00102d40 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00111cf0 con 0x7f0a00103660 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a084ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00111ff0 con 0x7f0a0010c170 2024-09-05T07:06:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43224/0 (socket says 172.21.15.86:43224) 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 -- 172.21.15.86:0/3558254579 learned_addr learned my addr 172.21.15.86:0/3558254579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a05a89640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 0x7f0a001989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a06a8b640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a0010c170 0x7f0a0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 -- 172.21.15.86:0/3558254579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a0010c170 msgr2=0x7f0a0019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a0010c170 0x7f0a0019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 -- 172.21.15.86:0/3558254579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 msgr2=0x7f0a001989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.485+0000 7f0a0628a640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 0x7f0a001989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f0a0628a640 1 -- 172.21.15.86:0/3558254579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0019d410 con 0x7f0a00102d40 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f0a05a89640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 0x7f0a001989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f0a0628a640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00cd80 con 0x7f0a00102d40 2024-09-05T07:06:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0019d700 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09fc0108e0 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc018920 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.486+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a001a4fe0 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.487+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a00103120 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.488+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09fc010a90 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.489+0000 7f09f77fe640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 0x7f09ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.489+0000 7f0a05a89640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 0x7f09ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.489+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f09fc0cfcc0 con 0x7f0a00102d40 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.489+0000 7f0a05a89640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 0x7f09ec0a51b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09f0000c00 tx=0x7f09f0005a20 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.493+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09fc09c4a0 con 0x7f0a00102d40 2024-09-05T07:06:47.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:47 smithi096 ceph-mon[34468]: pgmap v804: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-05T07:06:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:47 smithi115 ceph-mon[34791]: pgmap v804: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-05T07:06:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.695+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a001071e0 con 0x7f0a00102d40 2024-09-05T07:06:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.696+0000 7f09f77fe640 1 -- 172.21.15.86:0/3558254579 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11974 (secure 0 0 0) 0x7f09fc09bda0 con 0x7f0a00102d40 2024-09-05T07:06:47.696 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:06:47.696 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"btime":"2024-09-05T07:05:20:496062+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:05:19.627701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:06:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-09-05T07:06:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 msgr2=0x7f09ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 0x7f09ec0a51b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09f0000c00 tx=0x7f09f0005a20 comp rx=0 tx=0).stop 2024-09-05T07:06:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 msgr2=0x7f0a00198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 shutdown_connections 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f09ec0a2cf0 0x7f09ec0a51b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a0010c170 0x7f0a0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a00103660 0x7f0a001989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 --2- 172.21.15.86:0/3558254579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00102d40 0x7f0a00198460 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 >> 172.21.15.86:0/3558254579 conn(0x7f0a000fc7a0 msgr2=0x7f0a000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 shutdown_connections 2024-09-05T07:06:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:47.698+0000 7f0a084ee640 1 -- 172.21.15.86:0/3558254579 wait complete. 2024-09-05T07:06:47.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 605 sec 2024-09-05T07:06:48.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 760512b --release --await' 2024-09-05T07:06:48.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 760512b --release --await 2024-09-05T07:06:48.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:48 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3558254579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:48.577 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:48 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3558254579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:48.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:48 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3558254579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:48.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 606 sec 2024-09-05T07:06:48.798 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:49.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 -- 172.21.15.86:0/3671799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16810c1a0 msgr2=0x7fe16810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 --2- 172.21.15.86:0/3671799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16810c1a0 0x7fe16810e610 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe160009ca0 tx=0x7fe16002a950 comp rx=0 tx=0).stop 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 -- 172.21.15.86:0/3671799644 shutdown_connections 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 --2- 172.21.15.86:0/3671799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16810c1a0 0x7fe16810e610 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 --2- 172.21.15.86:0/3671799644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe168103720 0x7fe16810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 --2- 172.21.15.86:0/3671799644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 0x7fe1681031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 -- 172.21.15.86:0/3671799644 >> 172.21.15.86:0/3671799644 conn(0x7fe1680fc7a0 msgr2=0x7fe1680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 -- 172.21.15.86:0/3671799644 shutdown_connections 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 -- 172.21.15.86:0/3671799644 wait complete. 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.309+0000 7fe16d5db640 1 Processor -- start 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 -- start start 2024-09-05T07:06:49.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 0x7fe168100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe16810c1a0 0x7fe1681014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111e20 con 0x7fe168103720 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111ca0 con 0x7fe168102e00 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe16d5db640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111fa0 con 0x7fe16810c1a0 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43236/0 (socket says 172.21.15.86:43236) 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 -- 172.21.15.86:0/1660890345 learned_addr learned my addr 172.21.15.86:0/1660890345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe166ffd640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 0x7fe168100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe1677fe640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe16810c1a0 0x7fe1681014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 -- 172.21.15.86:0/1660890345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe16810c1a0 msgr2=0x7fe1681014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe16810c1a0 0x7fe1681014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 -- 172.21.15.86:0/1660890345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 msgr2=0x7fe168100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 0x7fe168100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 -- 172.21.15.86:0/1660890345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a67c0 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe15000d950 tx=0x7fe15000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.310+0000 7fe164ff9640 1 -- 172.21.15.86:0/1660890345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150017030 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.311+0000 7fe164ff9640 1 -- 172.21.15.86:0/1660890345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe150004020 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.311+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a6ab0 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.311+0000 7fe164ff9640 1 -- 172.21.15.86:0/1660890345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500047a0 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.311+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a7040 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.313+0000 7fe164ff9640 1 -- 172.21.15.86:0/1660890345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe1500041d0 con 0x7fe168103720 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.313+0000 7fe16f9b4740 1 Processor -- start 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.313+0000 7fe16f9b4740 1 -- start start 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.313+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.313+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc980 0x557b883e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b88341620 con 0x557b883e7240 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b882e8e50 con 0x557b883dc980 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b8833d5e0 con 0x557b883dc060 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc980 0x557b883e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.86:49980/0 (socket says 172.21.15.86:49980) 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe1677fe640 1 -- 172.21.15.86:0/877412527 learned_addr learned my addr 172.21.15.86:0/877412527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe15ffff640 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.314+0000 7fe164ff9640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 0x7fe1540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 -- 172.21.15.86:0/877412527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc980 msgr2=0x557b883e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc980 0x557b883e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 -- 172.21.15.86:0/877412527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 msgr2=0x557b883e9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 -- 172.21.15.86:0/877412527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b882d9d40 con 0x557b883dc060 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe15ffff640 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe164ff9640 1 -- 172.21.15.86:0/1660890345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe1500cf190 con 0x7fe168103720 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 0x7fe1540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe160009ca0 tx=0x7fe16002fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e00790c2a47aad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe15d7fa640 1 -- 172.21.15.86:0/877412527 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160034040 con 0x557b883dc060 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe15d7fa640 1 -- 172.21.15.86:0/877412527 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe160004040 con 0x557b883dc060 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe15d7fa640 1 -- 172.21.15.86:0/877412527 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600306a0 con 0x557b883dc060 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.315+0000 7fe1677fe640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 0x7fe1540a5230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe160005fe0 tx=0x7fe16004d000 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- 172.21.15.86:0/877412527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 msgr2=0x557b883dc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe160009ca0 tx=0x7fe16002fd40 comp rx=0 tx=0).stop 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- 172.21.15.86:0/877412527 shutdown_connections 2024-09-05T07:06:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc980 0x557b883e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/877412527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc060 0x557b883dc440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- 172.21.15.86:0/877412527 >> 172.21.15.86:0/877412527 conn(0x557b883d85f0 msgr2=0x557b883d89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- 172.21.15.86:0/877412527 shutdown_connections 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- 172.21.15.86:0/877412527 wait complete. 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.316+0000 7fe16f9b4740 1 -- start start 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 Processor -- start 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b883e9e10 con 0x557b883e7240 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b882c9f10 con 0x557b883dc060 2024-09-05T07:06:49.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b883e9f90 con 0x557b883dc980 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe16f9b4740 10 client.0 _ll_register_callbacks cb 0x557b882eddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43240/0 (socket says 172.21.15.86:43240) 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 -- 172.21.15.86:0/3019765677 learned_addr learned my addr 172.21.15.86:0/3019765677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe166ffd640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe15ffff640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 -- 172.21.15.86:0/3019765677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 msgr2=0x557b883e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 -- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 msgr2=0x557b883e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 -- 172.21.15.86:0/3019765677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b88050580 con 0x557b883e7240 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe15ffff640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:49.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe1677fe640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe16002ae70 tx=0x7fe160003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdfbab9733cbeb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.317+0000 7fe14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160034050 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe160004b40 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe160004040 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe160064050 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 10 client.20964 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557b8811a960 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe160004040 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe16005f030 con 0x557b883e7240 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 10 client.20964 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 10 client.20964 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 20 client.20964 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b883f6c70 0x557b883f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 4 client.20964 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.318+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b8834eb90 con 0x557b883f6c70 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.319+0000 7fe166ffd640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b883f6c70 0x557b883f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.319+0000 7fe166ffd640 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b883f6c70 0x557b883f9060 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da25f87ffd26c541 server_cookie=13060f501996b24a in_seq=0 out_seq=0 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.319+0000 7fe14e7fc640 10 client.20964 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe14e7fc640 1 -- 172.21.15.86:0/3019765677 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b8834eb90 con 0x557b883f6c70 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe14e7fc640 10 client.20964 handle_command_reply: tid=0 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 client.20964 shutdown 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 msgr2=0x557b883e4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe16002ae70 tx=0x7fe160003e30 comp rx=0 tx=0).stop 2024-09-05T07:06:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 shutdown_connections 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b883f6c70 0x557b883f9060 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883e7240 0x557b883e4ba0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/3019765677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 >> 172.21.15.86:0/3019765677 conn(0x557b883d85f0 msgr2=0x557b883de320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe14e7fc640 0 client.20964 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 shutdown_connections 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 1 -- 172.21.15.86:0/3019765677 wait complete. 2024-09-05T07:06:49.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.320+0000 7fe16f9b4740 20 client.20964 trim_cache size 0 max 16384 2024-09-05T07:06:49.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 Processor -- start 2024-09-05T07:06:49.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 -- start start 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883e7240 0x557b883feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b1360 con 0x557b883dc980 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b2ea0 con 0x557b883dc060 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b1a30 con 0x557b883e7240 2024-09-05T07:06:49.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43254/0 (socket says 172.21.15.86:43254) 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.337+0000 7fe15ffff640 1 -- 172.21.15.86:0/1647193881 learned_addr learned my addr 172.21.15.86:0/1647193881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe1677fe640 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883e7240 0x557b883feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 -- 172.21.15.86:0/1647193881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883e7240 msgr2=0x557b883feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe166ffd640 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883e7240 0x557b883feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 -- 172.21.15.86:0/1647193881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 msgr2=0x557b883deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883deb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 -- 172.21.15.86:0/1647193881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b883dc7b0 con 0x557b883dc980 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15ffff640 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fe15009af70 tx=0x7fe1500d2ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a89ddcdf888276 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15cff9640 1 -- 172.21.15.86:0/1647193881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15009b040 con 0x557b883dc980 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15cff9640 1 -- 172.21.15.86:0/1647193881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe15009b970 con 0x557b883dc980 2024-09-05T07:06:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.338+0000 7fe15cff9640 1 -- 172.21.15.86:0/1647193881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15009abd0 con 0x557b883dc980 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- 172.21.15.86:0/1647193881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 msgr2=0x557b883fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fe15009af70 tx=0x7fe1500d2ce0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- 172.21.15.86:0/1647193881 shutdown_connections 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883e7240 0x557b883feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc980 0x557b883fe630 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/1647193881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883dc060 0x557b883deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- 172.21.15.86:0/1647193881 >> 172.21.15.86:0/1647193881 conn(0x557b883d85f0 msgr2=0x557b883e29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- 172.21.15.86:0/1647193881 shutdown_connections 2024-09-05T07:06:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- 172.21.15.86:0/1647193881 wait complete. 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 -- start start 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.339+0000 7fe16f9b4740 1 Processor -- start 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 0x557b8845a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b27d0 con 0x557b883dc060 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b2100 con 0x557b883e7240 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681af360 con 0x557b883dc980 2024-09-05T07:06:49.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 10 client.0 _ll_register_callbacks cb 0x7fe160003d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43258/0 (socket says 172.21.15.86:43258) 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 -- 172.21.15.86:0/2894427922 learned_addr learned my addr 172.21.15.86:0/2894427922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe15ffff640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe1677fe640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 0x557b8845a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 -- 172.21.15.86:0/2894427922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 msgr2=0x557b883ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 -- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 msgr2=0x557b8845a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 0x557b8845a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 -- 172.21.15.86:0/2894427922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8845ac00 con 0x557b883dc060 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe1677fe640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 0x557b8845a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe166ffd640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fe158005bc0 tx=0x7fe15800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122d66e9fcc9d1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158005050 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7fe1580051f0 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.340+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15009b970 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe15802c050 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 10 client.20970 fetch_fsmap learned FSMap version 47 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557b88462050 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe15009b970 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fe15801d430 con 0x557b883dc060 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 10 client.20970 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 10 client.20970 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 20 client.20970 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b88462820 0x557b88464c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 4 client.20970 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "760512b", "release": true, "await": true}] 2024-09-05T07:06:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "set_id": "760512b", "release": true, "await": true}) -- 0x557b883490a0 con 0x557b88462820 2024-09-05T07:06:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.341+0000 7fe15ffff640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b88462820 0x557b88464c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.342+0000 7fe15ffff640 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b88462820 0x557b88464c40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5e5b04e7db41bd server_cookie=a358690df5533cca in_seq=0 out_seq=0 2024-09-05T07:06:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.342+0000 7fe14e7fc640 10 client.20970 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe14e7fc640 1 -- 172.21.15.86:0/2894427922 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557b883490a0 con 0x557b88462820 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe14e7fc640 10 client.20970 handle_command_reply: tid=0 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 client.20970 shutdown 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 msgr2=0x557b883eb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fe158005bc0 tx=0x7fe15800bd60 comp rx=0 tx=0).stop 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 shutdown_connections 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x557b88462820 0x557b88464c40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b883e7240 0x557b8845a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe14e7fc640 0 client.20970 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b883dc980 0x557b883ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 --2- 172.21.15.86:0/2894427922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b883dc060 0x557b883eb820 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 >> 172.21.15.86:0/2894427922 conn(0x557b883d85f0 msgr2=0x557b883e40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 shutdown_connections 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 1 -- 172.21.15.86:0/2894427922 wait complete. 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.374+0000 7fe16f9b4740 20 client.20970 trim_cache size 0 max 16384 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 47, 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 92, 2024-09-05T07:06:49.376 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "760512b": { 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 92, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 143.1, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:06:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:49.378 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 msgr2=0x7fe1540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 0x7fe1540a5230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe160005fe0 tx=0x7fe16004d000 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 msgr2=0x7fe168100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe15000d950 tx=0x7fe15000de10 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 shutdown_connections 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fe1540a2d70 0x7fe1540a5230 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe16810c1a0 0x7fe1681014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe168103720 0x7fe168100fa0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 --2- 172.21.15.86:0/1660890345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe168102e00 0x7fe168100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 >> 172.21.15.86:0/1660890345 conn(0x7fe1680fc7a0 msgr2=0x7fe1680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 shutdown_connections 2024-09-05T07:06:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:49.375+0000 7fe16d5db640 1 -- 172.21.15.86:0/1660890345 wait complete. 2024-09-05T07:06:49.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:49 smithi086 ceph-mon[30943]: pgmap v805: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-05T07:06:49.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:49 smithi096 ceph-mon[34468]: pgmap v805: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-05T07:06:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:49 smithi115 ceph-mon[34791]: pgmap v805: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-05T07:06:49.798 INFO:tasks.workunit.client.0.smithi086.stdout: 10 cleanup 607 sec 2024-09-05T07:06:50.291 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '760512b', total seconds elapsed: 27.3 2024-09-05T07:06:50.291 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.05 2024-09-05T07:06:50.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:06:50.608 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: 0 cleanup 608 sec 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: Operation Count AvgLat MaxLat 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: ---------------------------------------- 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: NTCreateX 121762 16.289 31431.142 2024-09-05T07:06:50.724 INFO:tasks.workunit.client.0.smithi086.stdout: Close 89528 0.196 75.248 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Rename 5161 36.968 19962.470 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Unlink 24541 20.445 31408.758 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Qpathinfo 110482 6.546 601.916 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Qfileinfo 19302 0.368 107.735 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Qfsinfo 20197 1.006 145.501 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Sfileinfo 9955 21.568 31426.611 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Find 42689 44.110 707.968 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: WriteX 60179 7.232 31405.843 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: ReadX 191556 0.019 104.095 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: LockX 398 2.059 31.388 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: UnlockX 398 1.786 18.979 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: Flush 8543 9.507 497.250 2024-09-05T07:06:50.725 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-09-05T07:06:50.726 INFO:tasks.workunit.client.0.smithi086.stdout:Throughput 6.3424 MB/sec 10 clients 10 procs max_latency=31431.146 ms 2024-09-05T07:06:50.726 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-05T07:06:50.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 -- 172.21.15.86:0/1375925307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8102ef0 msgr2=0x7f2fa810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/1375925307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8102ef0 0x7f2fa810da40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009ca0 tx=0x7f2fa002a950 comp rx=0 tx=0).stop 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 -- 172.21.15.86:0/1375925307 shutdown_connections 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/1375925307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8102ef0 0x7f2fa810da40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/1375925307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102570 0x7f2fa81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/1375925307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8100e20 0x7f2fa8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 -- 172.21.15.86:0/1375925307 >> 172.21.15.86:0/1375925307 conn(0x7f2fa80fc7d0 msgr2=0x7f2fa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 -- 172.21.15.86:0/1375925307 shutdown_connections 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.153+0000 7f2faf9cc640 1 -- 172.21.15.86:0/1375925307 wait complete. 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 Processor -- start 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 -- start start 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8102570 0x7f2fa81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 0x7f2fa819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8110e30 con 0x7f2fa8100e20 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8110cb0 con 0x7f2fa8102570 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2faf9cc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8110fb0 con 0x7f2fa8102ef0 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43274/0 (socket says 172.21.15.86:43274) 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 -- 172.21.15.86:0/2863799369 learned_addr learned my addr 172.21.15.86:0/2863799369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fadf69640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 0x7f2fa819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2facf67640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8102570 0x7f2fa81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 -- 172.21.15.86:0/2863799369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 msgr2=0x7f2fa819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 0x7f2fa819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 -- 172.21.15.86:0/2863799369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8102570 msgr2=0x7f2fa81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8102570 0x7f2fa81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fad768640 1 -- 172.21.15.86:0/2863799369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa819c330 con 0x7f2fa8100e20 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.154+0000 7f2fadf69640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 0x7f2fa819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2fad768640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d950 tx=0x7f2fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4017070 con 0x7f2fa8100e20 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa819c620 con 0x7f2fa8100e20 2024-09-05T07:06:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa4009c30 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400f620 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.155+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa8069f70 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.156+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa8101200 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.157+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2fa40109a0 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.158+0000 7f2f9e7fc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 0x7f2f940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.158+0000 7f2facf67640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 0x7f2f940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.158+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f2fa40cf430 con 0x7f2fa8100e20 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.158+0000 7f2facf67640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 0x7f2f940a51b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80ffee0 tx=0x7f2f98005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.162+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fa4009de0 con 0x7f2fa8100e20 2024-09-05T07:06:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.387+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fa806a990 con 0x7f2fa8100e20 2024-09-05T07:06:51.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:51 smithi086 ceph-mon[30943]: pgmap v806: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-09-05T07:06:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.388+0000 7f2f9e7fc640 1 -- 172.21.15.86:0/2863799369 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11974 (secure 0 0 0) 0x7f2fa409b5c0 con 0x7f2fa8100e20 2024-09-05T07:06:51.389 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:06:51.389 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"btime":"2024-09-05T07:05:20:496062+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:05:19.627701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24461,24422,24403]},"id":1}]} 2024-09-05T07:06:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-09-05T07:06:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.390+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 msgr2=0x7f2f940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.390+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 0x7f2f940a51b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80ffee0 tx=0x7f2f98005f90 comp rx=0 tx=0).stop 2024-09-05T07:06:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.390+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 msgr2=0x7f2fa8197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.390+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d950 tx=0x7f2fa400de10 comp rx=0 tx=0).stop 2024-09-05T07:06:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.390+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 shutdown_connections 2024-09-05T07:06:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2f940a2cf0 0x7f2f940a51b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fa8102ef0 0x7f2fa819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa8102570 0x7f2fa81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 --2- 172.21.15.86:0/2863799369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8100e20 0x7f2fa8197380 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 >> 172.21.15.86:0/2863799369 conn(0x7f2fa80fc7d0 msgr2=0x7f2fa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 shutdown_connections 2024-09-05T07:06:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:51.391+0000 7f2faf9cc640 1 -- 172.21.15.86:0/2863799369 wait complete. 2024-09-05T07:06:51.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:51 smithi096 ceph-mon[34468]: pgmap v806: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-09-05T07:06:51.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:51 smithi115 ceph-mon[34791]: pgmap v806: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-09-05T07:06:52.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 143.05 --await-for 45.0' 2024-09-05T07:06:52.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 143.05 --await-for 45.0 2024-09-05T07:06:52.204 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-05T07:06:52.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-05T07:06:52.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:52 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2863799369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:52.538 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:52.556 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:52 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2863799369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:52.675 DEBUG:teuthology.parallel:result is None 2024-09-05T07:06:52.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T07:06:52.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:52 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2863799369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:52.709 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T07:06:52.710 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-05T07:06:52.722 INFO:tasks.quiescer:joining Quiescers 2024-09-05T07:06:52.722 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-05T07:06:52.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:06:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 -- 172.21.15.86:0/2957189101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c106f10 msgr2=0x7f869c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/2957189101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c106f10 0x7f869c10ea90 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8694009ca0 tx=0x7f869402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 -- 172.21.15.86:0/2957189101 shutdown_connections 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/2957189101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c106f10 0x7f869c10ea90 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/2957189101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c102790 0x7f869c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/2957189101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c101de0 0x7f869c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 -- 172.21.15.86:0/2957189101 >> 172.21.15.86:0/2957189101 conn(0x7f869c0fd8c0 msgr2=0x7f869c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 -- 172.21.15.86:0/2957189101 shutdown_connections 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.065+0000 7f86a0cdb640 1 -- 172.21.15.86:0/2957189101 wait complete. 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 Processor -- start 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 -- start start 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 0x7f869c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c106f10 0x7f869c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f869c111e20 con 0x7f869c101de0 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f869c111ca0 con 0x7f869c106f10 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f86a0cdb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f869c111fa0 con 0x7f869c102790 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f869a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f869a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53512/0 (socket says 172.21.15.86:53512) 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.066+0000 7f869a59c640 1 -- 172.21.15.86:0/4169414702 learned_addr learned my addr 172.21.15.86:0/4169414702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f8699d9b640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 0x7f869c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 -- 172.21.15.86:0/4169414702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 msgr2=0x7f869c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 0x7f869c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 -- 172.21.15.86:0/4169414702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c106f10 msgr2=0x7f869c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869ad9d640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c106f10 0x7f869c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c106f10 0x7f869c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 -- 172.21.15.86:0/4169414702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869c19d230 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f8699d9b640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 0x7f869c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f869a59c640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e9f0 tx=0x7f868c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f868b7fe640 1 -- 172.21.15.86:0/4169414702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c00cd80 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869c19d520 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f868b7fe640 1 -- 172.21.15.86:0/4169414702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f868c0054d0 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f868b7fe640 1 -- 172.21.15.86:0/4169414702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c010490 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.067+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869c1a4e00 con 0x7f869c101de0 2024-09-05T07:06:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.069+0000 7f868b7fe640 1 -- 172.21.15.86:0/4169414702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f868c005020 con 0x7f869c101de0 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 Processor -- start 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 -- start start 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f868b7fe640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 0x7f86840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99baae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e99ba04160 con 0x55e99ba9f750 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e99b9ab720 con 0x55e99baa0070 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f86a30b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e99ba00120 con 0x55e99baaee40 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f869a59c640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 0x7f86840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53526/0 (socket says 172.21.15.86:53526) 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f869ad9d640 1 -- 172.21.15.86:0/2922267759 learned_addr learned my addr 172.21.15.86:0/2922267759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f868b7fe640 1 -- 172.21.15.86:0/4169414702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f868c0cfd70 con 0x7f869c101de0 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99baae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.070+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 -- 172.21.15.86:0/2922267759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 msgr2=0x55e99bab1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 -- 172.21.15.86:0/2922267759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 msgr2=0x55e99baae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99baae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 -- 172.21.15.86:0/2922267759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99b775ba0 con 0x55e99ba9f750 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8694005cc0 tx=0x7f8694005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9011ca3ebfdf48c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f869a59c640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 0x7f86840a51b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f868c0cd130 tx=0x7f868c0912e0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f8688ff9640 1 -- 172.21.15.86:0/2922267759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694031450 con 0x55e99ba9f750 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f8688ff9640 1 -- 172.21.15.86:0/2922267759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694045070 con 0x55e99ba9f750 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.071+0000 7f8688ff9640 1 -- 172.21.15.86:0/2922267759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86940043b0 con 0x55e99ba9f750 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- 172.21.15.86:0/2922267759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 msgr2=0x55e99ba9fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8694005cc0 tx=0x7f8694005d80 comp rx=0 tx=0).stop 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- 172.21.15.86:0/2922267759 shutdown_connections 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99baae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2922267759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99ba9fb30 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- 172.21.15.86:0/2922267759 >> 172.21.15.86:0/2922267759 conn(0x55e99ba9bde0 msgr2=0x55e99ba9c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- 172.21.15.86:0/2922267759 shutdown_connections 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- 172.21.15.86:0/2922267759 wait complete. 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- start start 2024-09-05T07:06:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 Processor -- start 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99baabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 0x55e99baac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e99bab1a10 con 0x55e99baaee40 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e99b98c900 con 0x55e99ba9f750 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e99bab1b90 con 0x55e99baa0070 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.072+0000 7f86a30b4740 10 client.0 _ll_register_callbacks cb 0x55e99b98a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53534/0 (socket says 172.21.15.86:53534) 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 -- 172.21.15.86:0/2722352502 learned_addr learned my addr 172.21.15.86:0/2722352502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 0x55e99baac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 -- 172.21.15.86:0/2722352502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 msgr2=0x55e99baac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 0x55e99baac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 -- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 msgr2=0x55e99baabd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99baabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 -- 172.21.15.86:0/2722352502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99b9ccc50 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f8694009620 tx=0x7f8694031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44473fb8325eee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f8681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 0x55e99baac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694050020 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f8694003db0 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.073+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694045070 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8694004130 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 10 client.20988 fetch_fsmap learned FSMap version 48 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e99bab1a10 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694045070 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f8694030ab0 con 0x55e99baaee40 2024-09-05T07:06:53.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 10 client.20988 fetch_fsmap finished waiting for FSMap version 48 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 10 client.20988 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 20 client.20988 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99babe890 0x55e99bac0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 4 client.20988 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.074+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e99ba11890 con 0x55e99babe890 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.075+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99babe890 0x55e99bac0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.075+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99babe890 0x55e99bac0c80 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c6f35e720c08b4 server_cookie=73d3dc06e4d6ee14 in_seq=0 out_seq=0 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.075+0000 7f8681ffb640 10 client.20988 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f8681ffb640 1 -- 172.21.15.86:0/2722352502 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e99ba11890 con 0x55e99babe890 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f8681ffb640 10 client.20988 handle_command_reply: tid=0 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 client.20988 shutdown 2024-09-05T07:06:53.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 msgr2=0x55e99baac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f8694009620 tx=0x7f8694031990 comp rx=0 tx=0).stop 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 shutdown_connections 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99babe890 0x55e99bac0c80 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baaee40 0x55e99baac7c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f8681ffb640 0 client.20988 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baa0070 0x55e99baac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2722352502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99baabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 >> 172.21.15.86:0/2722352502 conn(0x55e99ba9bde0 msgr2=0x55e99baa1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 shutdown_connections 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 1 -- 172.21.15.86:0/2722352502 wait complete. 2024-09-05T07:06:53.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.076+0000 7f86a30b4740 20 client.20988 trim_cache size 0 max 16384 2024-09-05T07:06:53.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 Processor -- start 2024-09-05T07:06:53.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 -- start start 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99bac6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bac6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1acea0 con 0x55e99ba9f750 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1ae9e0 con 0x55e99baa0070 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f86a30b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1ad570 con 0x55e99baaee40 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53540/0 (socket says 172.21.15.86:53540) 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.093+0000 7f869ad9d640 1 -- 172.21.15.86:0/2866678017 learned_addr learned my addr 172.21.15.86:0/2866678017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bac6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f8699d9b640 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99bac6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 -- 172.21.15.86:0/2866678017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 msgr2=0x55e99bac6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bac6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 -- 172.21.15.86:0/2866678017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 msgr2=0x55e99bac6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99bac6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 -- 172.21.15.86:0/2866678017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99ba973f0 con 0x55e99ba9f750 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f869ad9d640 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f869402f3f0 tx=0x7f8694005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef52d8e312f2a23d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f8683fff640 1 -- 172.21.15.86:0/2866678017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694050020 con 0x55e99ba9f750 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f8683fff640 1 -- 172.21.15.86:0/2866678017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694045070 con 0x55e99ba9f750 2024-09-05T07:06:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.094+0000 7f8683fff640 1 -- 172.21.15.86:0/2866678017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869405c030 con 0x55e99ba9f750 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- 172.21.15.86:0/2866678017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 msgr2=0x55e99baac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f869402f3f0 tx=0x7f8694005c60 comp rx=0 tx=0).stop 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- 172.21.15.86:0/2866678017 shutdown_connections 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bac6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99baa0070 0x55e99bac6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 --2- 172.21.15.86:0/2866678017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99ba9f750 0x55e99baac610 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- 172.21.15.86:0/2866678017 >> 172.21.15.86:0/2866678017 conn(0x55e99ba9bde0 msgr2=0x55e99baaddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- 172.21.15.86:0/2866678017 shutdown_connections 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- 172.21.15.86:0/2866678017 wait complete. 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.095+0000 7f86a30b4740 1 -- start start 2024-09-05T07:06:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 Processor -- start 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99bab7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1ae310 con 0x55e99baa0070 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1adc40 con 0x55e99ba9f750 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f869c1aae70 con 0x55e99baaee40 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f86a30b4740 10 client.0 _ll_register_callbacks cb 0x7f8690008310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53556/0 (socket says 172.21.15.86:53556) 2024-09-05T07:06:53.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 -- 172.21.15.86:0/233325652 learned_addr learned my addr 172.21.15.86:0/233325652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f869ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99bab7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f869ad9d640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 -- 172.21.15.86:0/233325652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 msgr2=0x55e99bab2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 -- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 msgr2=0x55e99bab7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99bab7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f8699d9b640 1 -- 172.21.15.86:0/233325652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99bab33c0 con 0x55e99baa0070 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.096+0000 7f869ad9d640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99bab7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:06:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8699d9b640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8690000c00 tx=0x7f86900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacb812581ec41b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86900035f0 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f8690003790 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694045070 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f869002b050 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f86a30b4740 10 client.20994 fetch_fsmap learned FSMap version 48 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e99bb29360 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8694045070 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f869001a8f0 con 0x55e99baa0070 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.097+0000 7f86a30b4740 10 client.20994 fetch_fsmap finished waiting for FSMap version 48 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f86a30b4740 10 client.20994 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f86a30b4740 20 client.20994 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99bb29bd0 0x55e99bb2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f86a30b4740 4 client.20994 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.05, "await_for": 45.0}] 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.05, "await_for": 45.0}) -- 0x55e99ba0bbe0 con 0x55e99bb29bd0 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f869ad9d640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99bb29bd0 0x55e99bb2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f8681ffb640 10 client.20994 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.098+0000 7f869ad9d640 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99bb29bd0 0x55e99bb2bff0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d496306c4e3f730 server_cookie=b8bb805d58e5be35 in_seq=0 out_seq=0 2024-09-05T07:06:53.118 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:53 smithi096 ceph-mon[34468]: pgmap v807: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-09-05T07:06:53.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:53 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:53.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:53 smithi086 ceph-mon[30943]: pgmap v807: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-09-05T07:06:53.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:53 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 -- 172.21.15.86:0/4049397676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910102dd0 msgr2=0x7f69101031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 --2- 172.21.15.86:0/4049397676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910102dd0 0x7f69101031b0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6900009cd0 tx=0x7f690002a8d0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 -- 172.21.15.86:0/4049397676 shutdown_connections 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 --2- 172.21.15.86:0/4049397676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910108300 0x7f691010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 --2- 172.21.15.86:0/4049397676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910103780 0x7f6910107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 --2- 172.21.15.86:0/4049397676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910102dd0 0x7f69101031b0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.639+0000 7f6917a0e640 1 -- 172.21.15.86:0/4049397676 >> 172.21.15.86:0/4049397676 conn(0x7f69100fc7d0 msgr2=0x7f69100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.640+0000 7f6917a0e640 1 -- 172.21.15.86:0/4049397676 shutdown_connections 2024-09-05T07:06:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.640+0000 7f6917a0e640 1 -- 172.21.15.86:0/4049397676 wait complete. 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.640+0000 7f6917a0e640 1 Processor -- start 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.640+0000 7f6917a0e640 1 -- start start 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 0x7f69101984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910108300 0x7f691019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111f60 con 0x7f6910103780 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111de0 con 0x7f6910108300 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69101120e0 con 0x7f6910102dd0 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53580/0 (socket says 172.21.15.86:53580) 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 -- 172.21.15.86:0/1342083206 learned_addr learned my addr 172.21.15.86:0/1342083206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f69157aa640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 0x7f69101984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 -- 172.21.15.86:0/1342083206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 msgr2=0x7f69101984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 0x7f69101984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 -- 172.21.15.86:0/1342083206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910108300 msgr2=0x7f691019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6915fab640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910108300 0x7f691019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910108300 0x7f691019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 -- 172.21.15.86:0/1342083206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691019d490 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f69157aa640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 0x7f69101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6914fa9640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f690400d740 tx=0x7f690400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904017070 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691019d780 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f690400de50 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400f620 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.641+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69101a5060 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.642+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691006a7b0 con 0x7f6910103780 2024-09-05T07:06:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.644+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f690401a070 con 0x7f6910103780 2024-09-05T07:06:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.644+0000 7f690e7fc640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 0x7f68f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.644+0000 7f69157aa640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 0x7f68f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.644+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f69040cf330 con 0x7f6910103780 2024-09-05T07:06:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.645+0000 7f69157aa640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 0x7f68f80a51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6900009a60 tx=0x7f6900000f40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.647+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f690409bb10 con 0x7f6910103780 2024-09-05T07:06:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:53 smithi115 ceph-mon[34791]: pgmap v807: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-09-05T07:06:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:53 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.849+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691010a290 con 0x7f6910103780 2024-09-05T07:06:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.850+0000 7f690e7fc640 1 -- 172.21.15.86:0/1342083206 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11975 (secure 0 0 0) 0x7f690409b410 con 0x7f6910103780 2024-09-05T07:06:53.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:06:53.851 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":48,"btime":"2024-09-05T07:06:52:282918+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:06:52.247247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:06:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 48 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.852+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 msgr2=0x7f68f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.852+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 0x7f68f80a51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6900009a60 tx=0x7f6900000f40 comp rx=0 tx=0).stop 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.852+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 msgr2=0x7f6910198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f690400d740 tx=0x7f690400dc00 comp rx=0 tx=0).stop 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 shutdown_connections 2024-09-05T07:06:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f68f80a2cf0 0x7f68f80a51b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6910108300 0x7f691019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910103780 0x7f6910198a20 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 --2- 172.21.15.86:0/1342083206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6910102dd0 0x7f69101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 >> 172.21.15.86:0/1342083206 conn(0x7f69100fc7d0 msgr2=0x7f6910105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 shutdown_connections 2024-09-05T07:06:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:53.853+0000 7f6917a0e640 1 -- 172.21.15.86:0/1342083206 wait complete. 2024-09-05T07:06:54.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:54 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1342083206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:54.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:54 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1342083206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:54.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f8681ffb640 1 -- 172.21.15.86:0/233325652 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e99ba0bbe0 con 0x55e99bb29bd0 2024-09-05T07:06:54.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f8681ffb640 10 client.20994 handle_command_reply: tid=0 2024-09-05T07:06:54.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 client.20994 shutdown 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 msgr2=0x55e99bab2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8690000c00 tx=0x7f86900118d0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 shutdown_connections 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x55e99bb29bd0 0x55e99bb2bff0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e99baaee40 0x55e99bab2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e99baa0070 0x55e99bab2790 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 --2- 172.21.15.86:0/233325652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e99ba9f750 0x55e99bab7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f8681ffb640 0 client.20994 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.645+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 >> 172.21.15.86:0/233325652 conn(0x55e99ba9bde0 msgr2=0x55e99baa1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 shutdown_connections 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a30b4740 1 -- 172.21.15.86:0/233325652 wait complete. 2024-09-05T07:06:54.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a30b4740 20 client.20994 trim_cache size 0 max 16384 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 48, 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 94, 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "2fd6d14": { 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 94, 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:06:54.648 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 90.0, 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 143.1, 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:54.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 msgr2=0x7f86840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 0x7f86840a51b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f868c0cd130 tx=0x7f868c0912e0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 msgr2=0x7f869c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.646+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e9f0 tx=0x7f868c00edf0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 shutdown_connections 2024-09-05T07:06:54.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f86840a2cf0 0x7f86840a51b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f869c106f10 0x7f869c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f869c102790 0x7f869c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 --2- 172.21.15.86:0/4169414702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f869c101de0 0x7f869c198280 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 >> 172.21.15.86:0/4169414702 conn(0x7f869c0fd8c0 msgr2=0x7f869c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 shutdown_connections 2024-09-05T07:06:54.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:54.647+0000 7f86a0cdb640 1 -- 172.21.15.86:0/4169414702 wait complete. 2024-09-05T07:06:54.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:54 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1342083206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:06:55.297 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --cancel --all' 2024-09-05T07:06:55.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --cancel --all 2024-09-05T07:06:55.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:55 smithi096 ceph-mon[34468]: pgmap v808: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-05T07:06:55.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:55 smithi086 ceph-mon[30943]: pgmap v808: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-05T07:06:55.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:55 smithi115 ceph-mon[34791]: pgmap v808: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-05T07:06:56.077 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2fd6d14', quiesce took 5.8 seconds. Will release after: 17.25 2024-09-05T07:06:56.079 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:56.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:56 smithi096 ceph-mon[34468]: scrub summary: idle 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:56.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:56 smithi086 ceph-mon[30943]: scrub summary: idle 2024-09-05T07:06:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 -- 172.21.15.86:0/54511302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a010c170 msgr2=0x7f84a010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 --2- 172.21.15.86:0/54511302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a010c170 0x7f84a010e610 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f8498009ca0 tx=0x7f849802a950 comp rx=0 tx=0).stop 2024-09-05T07:06:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 -- 172.21.15.86:0/54511302 shutdown_connections 2024-09-05T07:06:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 --2- 172.21.15.86:0/54511302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a010c170 0x7f84a010e610 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 --2- 172.21.15.86:0/54511302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0103660 0x7f84a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 --2- 172.21.15.86:0/54511302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0102d40 0x7f84a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 -- 172.21.15.86:0/54511302 >> 172.21.15.86:0/54511302 conn(0x7f84a00fc7a0 msgr2=0x7f84a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 -- 172.21.15.86:0/54511302 shutdown_connections 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.570+0000 7f84a8736640 1 -- 172.21.15.86:0/54511302 wait complete. 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 Processor -- start 2024-09-05T07:06:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 -- start start 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0103660 0x7f84a0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010c170 0x7f84a019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111e20 con 0x7f84a0102d40 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111ca0 con 0x7f84a010c170 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a8736640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111fa0 con 0x7f84a0103660 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53596/0 (socket says 172.21.15.86:53596) 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 -- 172.21.15.86:0/2481149864 learned_addr learned my addr 172.21.15.86:0/2481149864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010c170 0x7f84a019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0103660 0x7f84a0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 -- 172.21.15.86:0/2481149864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0103660 msgr2=0x7f84a0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0103660 0x7f84a0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 -- 172.21.15.86:0/2481149864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010c170 msgr2=0x7f84a019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010c170 0x7f84a019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.571+0000 7f84a64d2640 1 -- 172.21.15.86:0/2481149864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019d2f0 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84977fe640 1 -- 172.21.15.86:0/2481149864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000cd80 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84977fe640 1 -- 172.21.15.86:0/2481149864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84900054d0 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019d5e0 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84977fe640 1 -- 172.21.15.86:0/2481149864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490010490 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.572+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a4ec0 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 Processor -- start 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 -- start start 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84977fe640 1 -- 172.21.15.86:0/2481149864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8490005020 con 0x7f84a0102d40 2024-09-05T07:06:56.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed20dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed21afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654ed172200 con 0x5654ed20e320 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654ed118dc0 con 0x5654ed218be0 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84aab0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654ed16e1c0 con 0x5654ed20da00 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53608/0 (socket says 172.21.15.86:53608) 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84a64d2640 1 -- 172.21.15.86:0/2076325027 learned_addr learned my addr 172.21.15.86:0/2076325027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:56.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed21afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.575+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed20dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 -- 172.21.15.86:0/2076325027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 msgr2=0x5654ed20dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed20dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 -- 172.21.15.86:0/2076325027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 msgr2=0x5654ed21afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed21afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 -- 172.21.15.86:0/2076325027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ecee3620 con 0x5654ed20e320 2024-09-05T07:06:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84977fe640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84a64d2640 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f84900ccf20 tx=0x7f84900d3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff6943aab34c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f84977fe640 1 -- 172.21.15.86:0/2481149864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f84900cfca0 con 0x7f84a0102d40 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f8494ff9640 1 -- 172.21.15.86:0/2076325027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490098020 con 0x5654ed20e320 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f8494ff9640 1 -- 172.21.15.86:0/2076325027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8490005310 con 0x5654ed20e320 2024-09-05T07:06:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.576+0000 7f8494ff9640 1 -- 172.21.15.86:0/2076325027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849009b6e0 con 0x5654ed20e320 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- 172.21.15.86:0/2076325027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 msgr2=0x5654ed2186a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 0x7f848c0a51b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8498005e30 tx=0x7f8498005c50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f84900ccf20 tx=0x7f84900d3de0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- 172.21.15.86:0/2076325027 shutdown_connections 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed21afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2186a0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 --2- 172.21.15.86:0/2076325027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed20dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- 172.21.15.86:0/2076325027 >> 172.21.15.86:0/2076325027 conn(0x5654ed209f90 msgr2=0x5654ed20a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- 172.21.15.86:0/2076325027 shutdown_connections 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- 172.21.15.86:0/2076325027 wait complete. 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.577+0000 7f84aab0f740 1 -- start start 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 Processor -- start 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed218530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed215d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654ed21b7b0 con 0x5654ed20e320 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654ed10cc50 con 0x5654ed218be0 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654ed21b930 con 0x5654ed20da00 2024-09-05T07:06:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84aab0f740 10 client.0 _ll_register_callbacks cb 0x5654ed11eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53622/0 (socket says 172.21.15.86:53622) 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 -- 172.21.15.86:0/1495266070 learned_addr learned my addr 172.21.15.86:0/1495266070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed218530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed215d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 -- 172.21.15.86:0/1495266070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 msgr2=0x5654ed218530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed218530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 -- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 msgr2=0x5654ed215d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed215d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a5cd1640 1 -- 172.21.15.86:0/1495266070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ed216570 con 0x5654ed20e320 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.578+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed215d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f849c012700 tx=0x7f849c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5a286c40c2a554 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c011c70 con 0x5654ed20e320 2024-09-05T07:06:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f849c011e10 con 0x5654ed20e320 2024-09-05T07:06:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654ed17f770 con 0x5654ed20e320 2024-09-05T07:06:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f849c00f460 con 0x5654ed20e320 2024-09-05T07:06:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 10 client.21012 fetch_fsmap learned FSMap version 49 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5654ed2273c0 con 0x5654ed20e320 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5654ed17f770 con 0x5654ed20e320 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 8386+0+0 (secure 0 0 0) 0x7f849c01c710 con 0x5654ed20e320 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 10 client.21012 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 10 client.21012 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 20 client.21012 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed228920 0x5654ed22ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 4 client.21012 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.579+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ed179c80 con 0x5654ed228920 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.580+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed228920 0x5654ed22ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.580+0000 7f8489ffb640 10 client.21012 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.580+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed228920 0x5654ed22ad40 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1745e2bf7d6d1ff server_cookie=b2798a70c8fc9bed in_seq=0 out_seq=0 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f8489ffb640 1 -- 172.21.15.86:0/1495266070 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654ed179c80 con 0x5654ed228920 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f8489ffb640 10 client.21012 handle_command_reply: tid=0 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 client.21012 shutdown 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 msgr2=0x5654ed2157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f849c012700 tx=0x7f849c012bc0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 shutdown_connections 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed228920 0x5654ed22ad40 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed215d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20e320 0x5654ed2157e0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f8489ffb640 0 client.21012 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1495266070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20da00 0x5654ed218530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 >> 172.21.15.86:0/1495266070 conn(0x5654ed209f90 msgr2=0x5654ed2091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 shutdown_connections 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 1 -- 172.21.15.86:0/1495266070 wait complete. 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.581+0000 7f84aab0f740 20 client.21012 trim_cache size 0 max 16384 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 Processor -- start 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 -- start start 2024-09-05T07:06:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed230610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed230b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01af240 con 0x5654ed20da00 2024-09-05T07:06:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b0de0 con 0x5654ed218be0 2024-09-05T07:06:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84aab0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01af910 con 0x5654ed20e320 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53632/0 (socket says 172.21.15.86:53632) 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.598+0000 7f84a64d2640 1 -- 172.21.15.86:0/120300262 learned_addr learned my addr 172.21.15.86:0/120300262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed230b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed230610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 -- 172.21.15.86:0/120300262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 msgr2=0x5654ed230610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed230610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 -- 172.21.15.86:0/120300262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 msgr2=0x5654ed230b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed230b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 -- 172.21.15.86:0/120300262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ed209dc0 con 0x5654ed20da00 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f84a64d2640 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f84900cd5b0 tx=0x7f8490004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c013288758add server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f848bfff640 1 -- 172.21.15.86:0/120300262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900b8020 con 0x5654ed20da00 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f848bfff640 1 -- 172.21.15.86:0/120300262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84900d35c0 con 0x5654ed20da00 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.599+0000 7f848bfff640 1 -- 172.21.15.86:0/120300262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849009b8c0 con 0x5654ed20da00 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- 172.21.15.86:0/120300262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 msgr2=0x5654ed224a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f84900cd5b0 tx=0x7f8490004990 comp rx=0 tx=0).stop 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- 172.21.15.86:0/120300262 shutdown_connections 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed230b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed230610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 --2- 172.21.15.86:0/120300262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed224a10 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- 172.21.15.86:0/120300262 >> 172.21.15.86:0/120300262 conn(0x5654ed209f90 msgr2=0x5654ed210e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- 172.21.15.86:0/120300262 shutdown_connections 2024-09-05T07:06:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- 172.21.15.86:0/120300262 wait complete. 2024-09-05T07:06:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.600+0000 7f84aab0f740 1 -- start start 2024-09-05T07:06:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 Processor -- start 2024-09-05T07:06:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed28eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed220ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b0710 con 0x5654ed20da00 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b0040 con 0x5654ed218be0 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ad210 con 0x5654ed20e320 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53636/0 (socket says 172.21.15.86:53636) 2024-09-05T07:06:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 -- 172.21.15.86:0/1540704134 learned_addr learned my addr 172.21.15.86:0/1540704134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84aab0f740 10 client.0 _ll_register_callbacks cb 0x7f849c009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed28eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed220ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 -- 172.21.15.86:0/1540704134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 msgr2=0x5654ed28eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed28eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 -- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 msgr2=0x5654ed220ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed220ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a64d2640 1 -- 172.21.15.86:0/1540704134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ed20ab40 con 0x5654ed20da00 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.601+0000 7f84a6cd3640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed220ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84a64d2640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f84900d3270 tx=0x7f849000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da74cdbdeb0f237e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900b8030 con 0x5654ed20da00 2024-09-05T07:06:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f84900cd230 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84900d35c0 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f849009bb50 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 10 client.21018 fetch_fsmap learned FSMap version 49 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5654ed28c950 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f849009c480 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 8386+0+0 (secure 0 0 0) 0x7f84900983f0 con 0x5654ed20da00 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 10 client.21018 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 10 client.21018 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 20 client.21018 populate_metadata read hostname 'smithi086' 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.602+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed2979f0 0x5654ed299de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.603+0000 7f84aab0f740 4 client.21018 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.603+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7f84900d35c0 con 0x5654ed2979f0 2024-09-05T07:06:56.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.603+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed2979f0 0x5654ed299de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.603+0000 7f84a5cd1640 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed2979f0 0x5654ed299de0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d307aeebf28377 server_cookie=d28b3bc51bd87e66 in_seq=0 out_seq=0 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.603+0000 7f8489ffb640 10 client.21018 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f8489ffb640 1 -- 172.21.15.86:0/1540704134 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7f84900d35c0 con 0x5654ed2979f0 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f8489ffb640 10 client.21018 handle_command_reply: tid=0 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 client.21018 shutdown 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 msgr2=0x5654ed28e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f84900d3270 tx=0x7f849000cf20 comp rx=0 tx=0).stop 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 shutdown_connections 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5654ed2979f0 0x5654ed299de0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654ed218be0 0x5654ed220ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654ed20e320 0x5654ed28eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 --2- 172.21.15.86:0/1540704134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654ed20da00 0x5654ed28e960 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 >> 172.21.15.86:0/1540704134 conn(0x5654ed209f90 msgr2=0x5654ed2104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f8489ffb640 0 client.21018 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 shutdown_connections 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 1 -- 172.21.15.86:0/1540704134 wait complete. 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.604+0000 7f84aab0f740 20 client.21018 trim_cache size 0 max 16384 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 49, 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24479, 2024-09-05T07:06:56.628 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 95, 2024-09-05T07:06:56.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": {} 2024-09-05T07:06:56.629 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:06:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 msgr2=0x7f848c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 0x7f848c0a51b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8498005e30 tx=0x7f8498005c50 comp rx=0 tx=0).stop 2024-09-05T07:06:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 msgr2=0x7f84a0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 shutdown_connections 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010c170 0x7f84a019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a0103660 0x7f84a0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 --2- 172.21.15.86:0/2481149864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a0102d40 0x7f84a0198340 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 >> 172.21.15.86:0/2481149864 conn(0x7f84a00fc7a0 msgr2=0x7f84a00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 shutdown_connections 2024-09-05T07:06:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:56.605+0000 7f84a8736640 1 -- 172.21.15.86:0/2481149864 wait complete. 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:56.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:56 smithi115 ceph-mon[34791]: scrub summary: idle 2024-09-05T07:06:57.321 INFO:tasks.quiescer:done joining Quiescers 2024-09-05T07:06:57.321 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-05T07:06:57.331 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-05T07:06:57.331 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T07:06:57.331 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T07:06:57.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-05T07:06:57.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-05T07:06:57.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-05T07:06:57.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:57 smithi115 ceph-mon[34791]: pgmap v809: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-05T07:06:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 -- 172.21.15.86:0/960155050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2454102c80 msgr2=0x7f2454103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 --2- 172.21.15.86:0/960155050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2454102c80 0x7f2454103060 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2444009900 tx=0x7f244402a480 comp rx=0 tx=0).stop 2024-09-05T07:06:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 -- 172.21.15.86:0/960155050 shutdown_connections 2024-09-05T07:06:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 --2- 172.21.15.86:0/960155050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f245410c0d0 0x7f245410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 --2- 172.21.15.86:0/960155050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24541035a0 0x7f245410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 --2- 172.21.15.86:0/960155050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2454102c80 0x7f2454103060 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.712+0000 7f2459a0e640 1 -- 172.21.15.86:0/960155050 >> 172.21.15.86:0/960155050 conn(0x7f2454077990 msgr2=0x7f24540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- 172.21.15.86:0/960155050 shutdown_connections 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- 172.21.15.86:0/960155050 wait complete. 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 Processor -- start 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- start start 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2454102c80 0x7f2454198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24541035a0 0x7f24541987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2454111d60 con 0x7f245410c0d0 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2454111be0 con 0x7f2454102c80 2024-09-05T07:06:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2459a0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2454111ee0 con 0x7f24541035a0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f24537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f24537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53664/0 (socket says 172.21.15.86:53664) 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f24537fe640 1 -- 172.21.15.86:0/2339831727 learned_addr learned my addr 172.21.15.86:0/2339831727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.713+0000 7f2452ffd640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2454102c80 0x7f2454198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f244bfff640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24541035a0 0x7f24541987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 -- 172.21.15.86:0/2339831727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24541035a0 msgr2=0x7f24541987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24541035a0 0x7f24541987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 -- 172.21.15.86:0/2339831727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2454102c80 msgr2=0x7f2454198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2454102c80 0x7f2454198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 -- 172.21.15.86:0/2339831727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245419d270 con 0x7f245410c0d0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f24537fe640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f244c00eb10 tx=0x7f244c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c004b20 con 0x7f245410c0d0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c004cc0 con 0x7f245410c0d0 2024-09-05T07:06:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245419d560 con 0x7f245410c0d0 2024-09-05T07:06:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c010690 con 0x7f245410c0d0 2024-09-05T07:06:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.714+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541a4df0 con 0x7f245410c0d0 2024-09-05T07:06:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.715+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2454103060 con 0x7f245410c0d0 2024-09-05T07:06:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.716+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f244c005020 con 0x7f245410c0d0 2024-09-05T07:06:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.717+0000 7f2458a0c640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 0x7f24400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.717+0000 7f2452ffd640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 0x7f24400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.717+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f244c0cff90 con 0x7f245410c0d0 2024-09-05T07:06:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.717+0000 7f2452ffd640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 0x7f24400a5260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2444031440 tx=0x7f2444005b50 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.721+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f244c09c770 con 0x7f245410c0d0 2024-09-05T07:06:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:57 smithi086 ceph-mon[30943]: pgmap v809: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-05T07:06:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:57 smithi096 ceph-mon[34468]: pgmap v809: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-05T07:06:57.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.916+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2454062630 con 0x7f24400a2da0 2024-09-05T07:06:57.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.926+0000 7f2458a0c640 1 -- 172.21.15.86:0/2339831727 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f2454062630 con 0x7f24400a2da0 2024-09-05T07:06:57.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-05T00:00:00", "created": "2024-09-05T06:41:26", "first": "2024-09-05T06:42:00", "last": "2024-09-05T07:06:00", "last_pruned": "2024-09-05T07:06:01", "created_count": 25, "pruned_count": 18, "active": true} 2024-09-05T07:06:57.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 msgr2=0x7f24400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 0x7f24400a5260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2444031440 tx=0x7f2444005b50 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 msgr2=0x7f245419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f244c00eb10 tx=0x7f244c00c2d0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 shutdown_connections 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24400a2da0 0x7f24400a5260 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f245410c0d0 0x7f245419cb40 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24541035a0 0x7f24541987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 --2- 172.21.15.86:0/2339831727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2454102c80 0x7f2454198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 >> 172.21.15.86:0/2339831727 conn(0x7f2454077990 msgr2=0x7f24540fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 shutdown_connections 2024-09-05T07:06:57.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:57.928+0000 7f2459a0e640 1 -- 172.21.15.86:0/2339831727 wait complete. 2024-09-05T07:06:57.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-05T07:06:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 -- 172.21.15.86:0/3735597396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75001009b0 msgr2=0x7f7500100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 --2- 172.21.15.86:0/3735597396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75001009b0 0x7f7500100e10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009930 tx=0x7f74e802a420 comp rx=0 tx=0).stop 2024-09-05T07:06:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 -- 172.21.15.86:0/3735597396 shutdown_connections 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 --2- 172.21.15.86:0/3735597396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500101350 0x7f750010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 --2- 172.21.15.86:0/3735597396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75001009b0 0x7f7500100e10 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 --2- 172.21.15.86:0/3735597396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001069c0 0x7f7500106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 -- 172.21.15.86:0/3735597396 >> 172.21.15.86:0/3735597396 conn(0x7f7500077990 msgr2=0x7f75000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 -- 172.21.15.86:0/3735597396 shutdown_connections 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.040+0000 7f75049c0640 1 -- 172.21.15.86:0/3735597396 wait complete. 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.041+0000 7f75049c0640 1 Processor -- start 2024-09-05T07:06:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.041+0000 7f75049c0640 1 -- start start 2024-09-05T07:06:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.041+0000 7f75049c0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 0x7f75001991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.041+0000 7f75049c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.041+0000 7f75049c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75001069c0 0x7f75001a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f75049c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7500110cb0 con 0x7f7500101350 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f75049c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7500110b30 con 0x7f75001069c0 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f75049c0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7500110e30 con 0x7f75001009b0 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53676/0 (socket says 172.21.15.86:53676) 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 -- 172.21.15.86:0/1516703015 learned_addr learned my addr 172.21.15.86:0/1516703015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74fe59c640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 0x7f75001991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74fed9d640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75001069c0 0x7f75001a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 -- 172.21.15.86:0/1516703015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 msgr2=0x7f75001991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 0x7f75001991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 -- 172.21.15.86:0/1516703015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75001069c0 msgr2=0x7f75001a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75001069c0 0x7f75001a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 -- 172.21.15.86:0/1516703015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75001a4760 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74fe59c640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 0x7f75001991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f5d9b640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005940 tx=0x7f74e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e803f030 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75001a4930 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74e8003cb0 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8035070 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.042+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75001a4d90 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.043+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7500100e10 con 0x7f7500101350 2024-09-05T07:06:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.044+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74e8042030 con 0x7f7500101350 2024-09-05T07:06:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.045+0000 7f74f7fff640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 0x7f74ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:06:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.045+0000 7f74fe59c640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 0x7f74ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:06:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.045+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9605+0+0 (secure 0 0 0) 0x7f74e80eff00 con 0x7f7500101350 2024-09-05T07:06:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.046+0000 7f74fe59c640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 0x7f74ec0a5260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f74f00058c0 tx=0x7f74f0005d30 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:06:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.049+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74e80bc6e0 con 0x7f7500101350 2024-09-05T07:06:58.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.235+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 --> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7500062630 con 0x7f74ec0a2da0 2024-09-05T07:06:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.244+0000 7f74f7fff640 1 -- 172.21.15.86:0/1516703015 <== mgr.15300 v2:172.21.15.86:6800/388818328 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7500062630 con 0x7f74ec0a2da0 2024-09-05T07:06:58.245 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 msgr2=0x7f74ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 0x7f74ec0a5260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f74f00058c0 tx=0x7f74f0005d30 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 msgr2=0x7f75001996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005940 tx=0x7f74e80042f0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 shutdown_connections 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f74ec0a2da0 0x7f74ec0a5260 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75001069c0 0x7f75001a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7500101350 0x7f75001996e0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 --2- 172.21.15.86:0/1516703015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75001009b0 0x7f75001991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 >> 172.21.15.86:0/1516703015 conn(0x7f7500077990 msgr2=0x7f7500103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 shutdown_connections 2024-09-05T07:06:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:06:58.246+0000 7f75049c0640 1 -- 172.21.15.86:0/1516703015 wait complete. 2024-09-05T07:06:58.258 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-05T07:06:58.268 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-05T07:06:58.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:58 smithi115 ceph-mon[34791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:58.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:58 smithi086 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:58.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:58 smithi096 ceph-mon[34468]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T07:06:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:59 smithi115 ceph-mon[34791]: pgmap v810: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-09-05T07:06:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:59 smithi115 ceph-mon[34791]: from='client.21024 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:06:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:06:59 smithi115 ceph-mon[34791]: from='client.21030 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:06:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:59 smithi086 ceph-mon[30943]: pgmap v810: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-09-05T07:06:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:59 smithi086 ceph-mon[30943]: from='client.21024 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:06:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:06:59 smithi086 ceph-mon[30943]: from='client.21030 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:06:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:59 smithi096 ceph-mon[34468]: pgmap v810: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-09-05T07:06:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:59 smithi096 ceph-mon[34468]: from='client.21024 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:06:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:06:59 smithi096 ceph-mon[34468]: from='client.21030 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T07:07:01.609 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:01 smithi096 ceph-mon[34468]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-05T07:07:01.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:01 smithi115 ceph-mon[34791]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-05T07:07:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:01 smithi086 ceph-mon[30943]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-05T07:07:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:03 smithi115 ceph-mon[34791]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-09-05T07:07:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:07:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:07:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:07:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:07:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:03 smithi086 ceph-mon[30943]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-09-05T07:07:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:07:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:07:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:07:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:07:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:03 smithi096 ceph-mon[34468]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-09-05T07:07:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:07:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:07:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:07:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:07:05.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:05 smithi115 ceph-mon[34791]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 104 op/s 2024-09-05T07:07:05.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:05 smithi086 ceph-mon[30943]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 104 op/s 2024-09-05T07:07:05.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:05 smithi096 ceph-mon[34468]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 104 op/s 2024-09-05T07:07:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:07 smithi115 ceph-mon[34791]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 101 op/s 2024-09-05T07:07:07.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:07 smithi115 ceph-mon[34791]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T07:07:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:07 smithi086 ceph-mon[30943]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 101 op/s 2024-09-05T07:07:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:07 smithi086 ceph-mon[30943]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T07:07:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:07 smithi096 ceph-mon[34468]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 317 KiB/s wr, 101 op/s 2024-09-05T07:07:07.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:07 smithi096 ceph-mon[34468]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T07:07:09.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:09 smithi115 ceph-mon[34791]: pgmap v816: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:09 smithi086 ceph-mon[30943]: pgmap v816: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:09 smithi096 ceph-mon[34468]: pgmap v816: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:11.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:11 smithi115 ceph-mon[34791]: pgmap v817: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:11.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:11 smithi086 ceph-mon[30943]: pgmap v817: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:11.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:11 smithi096 ceph-mon[34468]: pgmap v817: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 40 op/s 2024-09-05T07:07:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:13 smithi115 ceph-mon[34791]: pgmap v818: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-09-05T07:07:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:13 smithi086 ceph-mon[30943]: pgmap v818: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-09-05T07:07:13.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:13 smithi096 ceph-mon[34468]: pgmap v818: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 33 op/s 2024-09-05T07:07:15.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:15 smithi115 ceph-mon[34791]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:15 smithi086 ceph-mon[30943]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:15 smithi096 ceph-mon[34468]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:16.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T07:07:17.173 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:17.595 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:17 smithi086 ceph-mon[30943]: pgmap v820: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:17.595 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:17 smithi086 ceph-mon[30943]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T07:07:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:17 smithi115 ceph-mon[34791]: pgmap v820: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:17.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:17 smithi115 ceph-mon[34791]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T07:07:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 -- 172.21.15.86:0/1448128518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4101de0 msgr2=0x7f92f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 --2- 172.21.15.86:0/1448128518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4101de0 0x7f92f41021c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009900 tx=0x7f92e402a410 comp rx=0 tx=0).stop 2024-09-05T07:07:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 -- 172.21.15.86:0/1448128518 shutdown_connections 2024-09-05T07:07:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 --2- 172.21.15.86:0/1448128518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 0x7f92f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 --2- 172.21.15.86:0/1448128518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4102790 0x7f92f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 --2- 172.21.15.86:0/1448128518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4101de0 0x7f92f41021c0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 -- 172.21.15.86:0/1448128518 >> 172.21.15.86:0/1448128518 conn(0x7f92f40fd8c0 msgr2=0x7f92f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 -- 172.21.15.86:0/1448128518 shutdown_connections 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.677+0000 7f92f896b640 1 -- 172.21.15.86:0/1448128518 wait complete. 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 Processor -- start 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 -- start start 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4101de0 0x7f92f41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 0x7f92f419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4111e20 con 0x7f92f4102790 2024-09-05T07:07:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4111ca0 con 0x7f92f4106f10 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f896b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4111fa0 con 0x7f92f4101de0 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38828/0 (socket says 172.21.15.86:38828) 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92ebfff640 1 -- 172.21.15.86:0/2338880585 learned_addr learned my addr 172.21.15.86:0/2338880585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92f259c640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4101de0 0x7f92f41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.678+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 0x7f92f419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 -- 172.21.15.86:0/2338880585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4101de0 msgr2=0x7f92f41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4101de0 0x7f92f41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 -- 172.21.15.86:0/2338880585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 msgr2=0x7f92f419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 0x7f92f419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 -- 172.21.15.86:0/2338880585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f419d390 con 0x7f92f4102790 2024-09-05T07:07:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92ebfff640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00d950 tx=0x7f92dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92eb7fe640 1 -- 172.21.15.86:0/2338880585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc017070 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f419d680 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92eb7fe640 1 -- 172.21.15.86:0/2338880585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92dc009c30 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92eb7fe640 1 -- 172.21.15.86:0/2338880585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00f620 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.679+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41a4f60 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.681+0000 7f92eb7fe640 1 -- 172.21.15.86:0/2338880585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f92dc0109a0 con 0x7f92f4102790 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.681+0000 7f92fad44740 1 Processor -- start 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 -- start start 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d168e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d1699140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643d15f42e0 con 0x5643d1699680 2024-09-05T07:07:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643d159bce0 con 0x5643d168edc0 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92fad44740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5643d15f02a0 con 0x5643d168e4a0 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92f2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d168e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38832/0 (socket says 172.21.15.86:38832) 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 -- 172.21.15.86:0/1631743563 learned_addr learned my addr 172.21.15.86:0/1631743563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92f259c640 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d1699140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92eb7fe640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 0x7f92e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 -- 172.21.15.86:0/1631743563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 msgr2=0x5643d168e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d168e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 -- 172.21.15.86:0/1631743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 msgr2=0x5643d1699140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d1699140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.682+0000 7f92ebfff640 1 -- 172.21.15.86:0/1631743563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d1580440 con 0x5643d1699680 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 0x7f92e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92eb7fe640 1 -- 172.21.15.86:0/2338880585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f92dc0cf590 con 0x7f92f4102790 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0cc810 tx=0x7f92dc0af7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeb694e86f36366 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92e8ff9640 1 -- 172.21.15.86:0/1631743563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc020030 con 0x5643d1699680 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92e8ff9640 1 -- 172.21.15.86:0/1631743563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92dc09bd70 con 0x5643d1699680 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.683+0000 7f92e8ff9640 1 -- 172.21.15.86:0/1631743563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc09b1b0 con 0x5643d1699680 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- 172.21.15.86:0/1631743563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 msgr2=0x5643d169ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 0x7f92e00a51b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f92ec008e20 tx=0x7f92ec004a40 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0cc810 tx=0x7f92dc0af7d0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- 172.21.15.86:0/1631743563 shutdown_connections 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d169ba70 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d1699140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- 172.21.15.86:0/1631743563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d168e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- 172.21.15.86:0/1631743563 >> 172.21.15.86:0/1631743563 conn(0x5643d168aa30 msgr2=0x5643d168ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- 172.21.15.86:0/1631743563 shutdown_connections 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- 172.21.15.86:0/1631743563 wait complete. 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- start start 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 Processor -- start 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d1698f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643d169c250 con 0x5643d1699680 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643d15af050 con 0x5643d168edc0 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.684+0000 7f92fad44740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5643d169c3d0 con 0x5643d168e4a0 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92fad44740 10 client.0 _ll_register_callbacks cb 0x5643d15ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:17.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38844/0 (socket says 172.21.15.86:38844) 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 -- 172.21.15.86:0/700523482 learned_addr learned my addr 172.21.15.86:0/700523482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f259c640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d1698f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92ebfff640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 -- 172.21.15.86:0/700523482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 msgr2=0x5643d1698f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d1698f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 -- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 msgr2=0x5643d16961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 -- 172.21.15.86:0/700523482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d13653c0 con 0x5643d1699680 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92ebfff640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f92ec012e20 tx=0x7f92ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe27cffdd6f4f042 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:17.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec027670 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f92ec027810 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.685+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92dc09bd70 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f92ec030050 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92fad44740 10 client.21042 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5643d16a7c00 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92dc09bd70 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f92ec01b550 con 0x5643d1699680 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.686+0000 7f92fad44740 10 client.21042 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92fad44740 10 client.21042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92fad44740 20 client.21042 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d16a90c0 0x5643d16ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92fad44740 4 client.21042 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d1601850 con 0x5643d16a90c0 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92f259c640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d16a90c0 0x5643d16ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92d9ffb640 10 client.21042 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.687+0000 7f92f259c640 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d16a90c0 0x5643d16ab4b0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2869c293cd2465e3 server_cookie=d62e4a15194d9541 in_seq=0 out_seq=0 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92d9ffb640 1 -- 172.21.15.86:0/700523482 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643d1601850 con 0x5643d16a90c0 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92d9ffb640 10 client.21042 handle_command_reply: tid=0 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 client.21042 shutdown 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 msgr2=0x5643d1696730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f92ec012e20 tx=0x7f92ec004ab0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 shutdown_connections 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d16a90c0 0x5643d16ab4b0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d1696730 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 --2- 172.21.15.86:0/700523482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d1698f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 >> 172.21.15.86:0/700523482 conn(0x5643d168aa30 msgr2=0x5643d1690760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92d9ffb640 0 client.21042 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 shutdown_connections 2024-09-05T07:07:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 1 -- 172.21.15.86:0/700523482 wait complete. 2024-09-05T07:07:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.688+0000 7f92fad44740 20 client.21042 trim_cache size 0 max 16384 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 Processor -- start 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 -- start start 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41af350 con 0x5643d1699680 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b0f20 con 0x5643d168edc0 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.705+0000 7f92fad44740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41afa20 con 0x5643d168e4a0 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38860/0 (socket says 172.21.15.86:38860) 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 -- 172.21.15.86:0/1534862868 learned_addr learned my addr 172.21.15.86:0/1534862868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f259c640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 -- 172.21.15.86:0/1534862868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 msgr2=0x5643d16a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 -- 172.21.15.86:0/1534862868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 msgr2=0x5643d16917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 -- 172.21.15.86:0/1534862868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d168a860 con 0x5643d1699680 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f259c640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f92ec011ce0 tx=0x7f92ec008fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff8cf82b01c77ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92dbfff640 1 -- 172.21.15.86:0/1534862868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec01bd60 con 0x5643d1699680 2024-09-05T07:07:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92dbfff640 1 -- 172.21.15.86:0/1534862868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec012e50 con 0x5643d1699680 2024-09-05T07:07:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.706+0000 7f92dbfff640 1 -- 172.21.15.86:0/1534862868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec027e30 con 0x5643d1699680 2024-09-05T07:07:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- 172.21.15.86:0/1534862868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 msgr2=0x5643d16b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f92ec011ce0 tx=0x7f92ec008fd0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- 172.21.15.86:0/1534862868 shutdown_connections 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d1699680 0x5643d16b0d80 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d168edc0 0x5643d16917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- 172.21.15.86:0/1534862868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- 172.21.15.86:0/1534862868 >> 172.21.15.86:0/1534862868 conn(0x5643d168aa30 msgr2=0x5643d1691090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- 172.21.15.86:0/1534862868 shutdown_connections 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- 172.21.15.86:0/1534862868 wait complete. 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- start start 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 Processor -- start 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d1699680 0x5643d169d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b0850 con 0x5643d168edc0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.707+0000 7f92fad44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b0150 con 0x5643d1699680 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92fad44740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ad290 con 0x5643d168e4a0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92fad44740 10 client.0 _ll_register_callbacks cb 0x7f92ec008ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38864/0 (socket says 172.21.15.86:38864) 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 -- 172.21.15.86:0/1264168245 learned_addr learned my addr 172.21.15.86:0/1264168245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92f259c640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92f2d9d640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d1699680 0x5643d169d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 -- 172.21.15.86:0/1264168245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 msgr2=0x5643d16a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 -- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d1699680 msgr2=0x5643d169d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d1699680 0x5643d169d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 -- 172.21.15.86:0/1264168245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d169dd60 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92ebfff640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0a44a0 tx=0x7f92dc010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be41c0e8a4d4f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc020040 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f92dc0a0df0 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec012e50 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f92dc0a0bb0 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92fad44740 10 client.21048 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.708+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5643d17180e0 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec012e50 con 0x5643d168edc0 2024-09-05T07:07:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f92dc097ce0 con 0x5643d168edc0 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 10 client.21048 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 10 client.21048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 20 client.21048 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d1718460 0x5643d171a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 4 client.21048 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643d15fbd60 con 0x5643d1718460 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92f259c640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d1718460 0x5643d171a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92f259c640 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d1718460 0x5643d171a850 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0c26ede77071dd server_cookie=1725705ed953b39 in_seq=0 out_seq=0 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.709+0000 7f92d9ffb640 10 client.21048 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92d9ffb640 1 -- 172.21.15.86:0/1264168245 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643d15fbd60 con 0x5643d1718460 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92d9ffb640 10 client.21048 handle_command_reply: tid=0 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 client.21048 shutdown 2024-09-05T07:07:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 msgr2=0x5643d169d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0a44a0 tx=0x7f92dc010f90 comp rx=0 tx=0).stop 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 shutdown_connections 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5643d1718460 0x5643d171a850 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643d1699680 0x5643d169d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643d168edc0 0x5643d169d010 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 --2- 172.21.15.86:0/1264168245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5643d168e4a0 0x5643d16a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 >> 172.21.15.86:0/1264168245 conn(0x5643d168aa30 msgr2=0x5643d1698270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92d9ffb640 0 client.21048 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 shutdown_connections 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 1 -- 172.21.15.86:0/1264168245 wait complete. 2024-09-05T07:07:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92fad44740 20 client.21048 trim_cache size 0 max 16384 2024-09-05T07:07:17.719 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:17.719 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-09-05T07:07:17.719 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-09-05T07:07:17.719 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 msgr2=0x7f92e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 0x7f92e00a51b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f92ec008e20 tx=0x7f92ec004a40 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 msgr2=0x7f92f4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.710+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00d950 tx=0x7f92dc00de10 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 shutdown_connections 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f92e00a2cf0 0x7f92e00a51b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92f4106f10 0x7f92f419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f4102790 0x7f92f4198920 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 --2- 172.21.15.86:0/2338880585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f4101de0 0x7f92f41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 >> 172.21.15.86:0/2338880585 conn(0x7f92f40fd8c0 msgr2=0x7f92f40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 shutdown_connections 2024-09-05T07:07:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:17.711+0000 7f92f896b640 1 -- 172.21.15.86:0/2338880585 wait complete. 2024-09-05T07:07:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:17 smithi096 ceph-mon[34468]: pgmap v820: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-05T07:07:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:17 smithi096 ceph-mon[34468]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T07:07:18.593 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T07:07:18.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:07:18.842 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:19.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 -- 172.21.15.86:0/3526445596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 msgr2=0x7f3398100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:19.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 --2- 172.21.15.86:0/3526445596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398100ee0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f338c009cd0 tx=0x7f338c02a8e0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 -- 172.21.15.86:0/3526445596 shutdown_connections 2024-09-05T07:07:19.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 --2- 172.21.15.86:0/3526445596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398101420 0x7f339810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 --2- 172.21.15.86:0/3526445596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398100ee0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 --2- 172.21.15.86:0/3526445596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398106a80 0x7f3398106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 -- 172.21.15.86:0/3526445596 >> 172.21.15.86:0/3526445596 conn(0x7f33980fc7a0 msgr2=0x7f33980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 -- 172.21.15.86:0/3526445596 shutdown_connections 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 -- 172.21.15.86:0/3526445596 wait complete. 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.317+0000 7f339f42c640 1 Processor -- start 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 -- start start 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398101420 0x7f3398198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 0x7f339819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3398110d30 con 0x7f3398100a80 2024-09-05T07:07:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3398110bb0 con 0x7f3398101420 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339f42c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3398110eb0 con 0x7f3398106a80 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38876/0 (socket says 172.21.15.86:38876) 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 -- 172.21.15.86:0/3750396987 learned_addr learned my addr 172.21.15.86:0/3750396987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339c9c7640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398101420 0x7f3398198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d9c9640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 0x7f339819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 -- 172.21.15.86:0/3750396987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 msgr2=0x7f339819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 0x7f339819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 -- 172.21.15.86:0/3750396987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398101420 msgr2=0x7f3398198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398101420 0x7f3398198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 -- 172.21.15.86:0/3750396987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339819d2c0 con 0x7f3398100a80 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d9c9640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 0x7f339819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.318+0000 7f339d1c8640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f339000d950 tx=0x7f339000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.319+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390017070 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.319+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3390009c30 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.319+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339819d5b0 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.319+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339000f620 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.319+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33981a4e90 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.320+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3398100ee0 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.321+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f33900109a0 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.321+0000 7f33967fc640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.321+0000 7f339c9c7640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.321+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f33900cf460 con 0x7f3398100a80 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.322+0000 7f339c9c7640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f338c006010 tx=0x7f338c0099c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.326+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3390009de0 con 0x7f3398100a80 2024-09-05T07:07:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.529+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3398199670 con 0x7f3398100a80 2024-09-05T07:07:19.530 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:19 smithi086 ceph-mon[30943]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.529+0000 7f33967fc640 1 -- 172.21.15.86:0/3750396987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11986 (secure 0 0 0) 0x7f339009bc40 con 0x7f3398100a80 2024-09-05T07:07:19.531 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:07:19.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"btime":"2024-09-05T07:06:57:311370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:06:57.247303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:07:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-09-05T07:07:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.531+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 msgr2=0x7f33800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:19.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.531+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f338c006010 tx=0x7f338c0099c0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.531+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 msgr2=0x7f3398198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:19.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.531+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f339000d950 tx=0x7f339000de10 comp rx=0 tx=0).stop 2024-09-05T07:07:19.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 shutdown_connections 2024-09-05T07:07:19.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f33800a2cf0 0x7f33800a51b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3398106a80 0x7f339819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398101420 0x7f3398198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 --2- 172.21.15.86:0/3750396987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3398100a80 0x7f3398198310 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 >> 172.21.15.86:0/3750396987 conn(0x7f33980fc7a0 msgr2=0x7f339806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 shutdown_connections 2024-09-05T07:07:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:19.532+0000 7f339f42c640 1 -- 172.21.15.86:0/3750396987 wait complete. 2024-09-05T07:07:19.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:19 smithi115 ceph-mon[34791]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:19 smithi096 ceph-mon[34468]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:20.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-05T07:07:20.524 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:20.645 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:20 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3750396987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:07:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:20 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3750396987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:07:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:20 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3750396987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:07:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 -- 172.21.15.86:0/2894817972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c102790 msgr2=0x7f044c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 --2- 172.21.15.86:0/2894817972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c102790 0x7f044c1067e0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f043802a410 comp rx=0 tx=0).stop 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 -- 172.21.15.86:0/2894817972 shutdown_connections 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 --2- 172.21.15.86:0/2894817972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c106f10 0x7f044c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 --2- 172.21.15.86:0/2894817972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c102790 0x7f044c1067e0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 --2- 172.21.15.86:0/2894817972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c101de0 0x7f044c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 -- 172.21.15.86:0/2894817972 >> 172.21.15.86:0/2894817972 conn(0x7f044c0fd8c0 msgr2=0x7f044c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 -- 172.21.15.86:0/2894817972 shutdown_connections 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.028+0000 7f0452b0d640 1 -- 172.21.15.86:0/2894817972 wait complete. 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 Processor -- start 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 -- start start 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c102790 0x7f044c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c106f10 0x7f044c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f044c111e20 con 0x7f044c101de0 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f044c111ca0 con 0x7f044c102790 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f0452b0d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f044c111fa0 con 0x7f044c106f10 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38904/0 (socket says 172.21.15.86:38904) 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 -- 172.21.15.86:0/348357739 learned_addr learned my addr 172.21.15.86:0/348357739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f044bfff640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c102790 0x7f044c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04510aa640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c106f10 0x7f044c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 -- 172.21.15.86:0/348357739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c106f10 msgr2=0x7f044c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c106f10 0x7f044c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 -- 172.21.15.86:0/348357739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c102790 msgr2=0x7f044c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c102790 0x7f044c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.029+0000 7f04508a9640 1 -- 172.21.15.86:0/348357739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c19d330 con 0x7f044c101de0 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f04508a9640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f0449ffb640 1 -- 172.21.15.86:0/348357739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000cd80 con 0x7f044c101de0 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f0449ffb640 1 -- 172.21.15.86:0/348357739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04400054d0 con 0x7f044c101de0 2024-09-05T07:07:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c19d620 con 0x7f044c101de0 2024-09-05T07:07:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f0449ffb640 1 -- 172.21.15.86:0/348357739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440010490 con 0x7f044c101de0 2024-09-05T07:07:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.030+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c1a4f00 con 0x7f044c101de0 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.032+0000 7f0449ffb640 1 -- 172.21.15.86:0/348357739 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0440005020 con 0x7f044c101de0 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.032+0000 7f0454ee6740 1 Processor -- start 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 -- start start 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0449ffb640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 0x7f043c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e00556090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f04510aa640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 0x7f043c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e00563760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e004bc2f0 con 0x564e005565d0 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e00463cf0 con 0x564e00557b40 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0454ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564e004b82b0 con 0x564e00561370 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f0449ffb640 1 -- 172.21.15.86:0/348357739 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f04400cfca0 con 0x7f044c101de0 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38914/0 (socket says 172.21.15.86:38914) 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 -- 172.21.15.86:0/2134441555 learned_addr learned my addr 172.21.15.86:0/2134441555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f04508a9640 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e00556090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 -- 172.21.15.86:0/2134441555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 msgr2=0x564e00563760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e00563760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 -- 172.21.15.86:0/2134441555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 msgr2=0x564e00556090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e00556090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 -- 172.21.15.86:0/2134441555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e004483a0 con 0x564e005565d0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f04510aa640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 0x7f043c0a5230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f04440092f0 tx=0x7f0444010740 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.033+0000 7f044bfff640 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f0438032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3a9a9e04102f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0436ffd640 1 -- 172.21.15.86:0/2134441555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438040070 con 0x564e005565d0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0436ffd640 1 -- 172.21.15.86:0/2134441555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0438004030 con 0x564e005565d0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0436ffd640 1 -- 172.21.15.86:0/2134441555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438035040 con 0x564e005565d0 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 -- 172.21.15.86:0/2134441555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 msgr2=0x564e00556a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f0438032bb0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 -- 172.21.15.86:0/2134441555 shutdown_connections 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e00563760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e005565d0 0x564e00556a10 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 --2- 172.21.15.86:0/2134441555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e00556090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 -- 172.21.15.86:0/2134441555 >> 172.21.15.86:0/2134441555 conn(0x564e0051f310 msgr2=0x564e00552c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 -- 172.21.15.86:0/2134441555 shutdown_connections 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.034+0000 7f0454ee6740 1 -- 172.21.15.86:0/2134441555 wait complete. 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 -- start start 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 Processor -- start 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e005565d0 0x564e00560d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e0055dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e00568420 con 0x564e00561370 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e00477060 con 0x564e00557b40 2024-09-05T07:07:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564e005685a0 con 0x564e005565d0 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f0454ee6740 10 client.0 _ll_register_callbacks cb 0x564e001ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f04510aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f04510aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38922/0 (socket says 172.21.15.86:38922) 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f04510aa640 1 -- 172.21.15.86:0/3896920551 learned_addr learned my addr 172.21.15.86:0/3896920551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.035+0000 7f044bfff640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e0055dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04508a9640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e005565d0 0x564e00560d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 -- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e005565d0 msgr2=0x564e00560d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e005565d0 0x564e00560d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 -- 172.21.15.86:0/3896920551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 msgr2=0x564e0055dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e0055dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 -- 172.21.15.86:0/3896920551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0022d3c0 con 0x564e00561370 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f044bfff640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e0055dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:07:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f04510aa640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0444012770 tx=0x7f0444004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ef6a9022c03fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f042bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444011a60 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f0444011c00 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0438004030 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f044402e050 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f0454ee6740 10 client.21066 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564e00573f10 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.036+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0438004030 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f044400f040 con 0x564e00561370 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 10 client.21066 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 10 client.21066 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 20 client.21066 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e00575420 0x564e00577850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 4 client.21066 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e004c9860 con 0x564e00575420 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f04508a9640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e00575420 0x564e00577850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f04508a9640 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e00575420 0x564e00577850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b59bb24b2b41358 server_cookie=f24e7b7ab3e32b97 in_seq=0 out_seq=0 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.037+0000 7f042bfff640 10 client.21066 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f042bfff640 1 -- 172.21.15.86:0/3896920551 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e004c9860 con 0x564e00575420 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f042bfff640 10 client.21066 handle_command_reply: tid=0 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 client.21066 shutdown 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 msgr2=0x564e0055e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0444012770 tx=0x7f0444004ab0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 shutdown_connections 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e00575420 0x564e00577850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00561370 0x564e0055e5d0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e00557b40 0x564e0055dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f042bfff640 0 client.21066 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3896920551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e005565d0 0x564e00560d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 >> 172.21.15.86:0/3896920551 conn(0x564e0051f310 msgr2=0x564e00558930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 shutdown_connections 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 1 -- 172.21.15.86:0/3896920551 wait complete. 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.038+0000 7f0454ee6740 20 client.21066 trim_cache size 0 max 16384 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 Processor -- start 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 -- start start 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00552e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0055d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1ad040 con 0x564e00557b40 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1aec10 con 0x564e005565d0 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f0454ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1ad710 con 0x564e00561370 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38930/0 (socket says 172.21.15.86:38930) 2024-09-05T07:07:21.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.051+0000 7f044bfff640 1 -- 172.21.15.86:0/3091383357 learned_addr learned my addr 172.21.15.86:0/3091383357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f04510aa640 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0055d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 -- 172.21.15.86:0/3091383357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 msgr2=0x564e0055d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0055d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 -- 172.21.15.86:0/3091383357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 msgr2=0x564e00552e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00552e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 -- 172.21.15.86:0/3091383357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0055cf90 con 0x564e00557b40 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f04510aa640 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0055d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f044bfff640 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f0438000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd5d43fab98651 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f04367fc640 1 -- 172.21.15.86:0/3091383357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438040070 con 0x564e00557b40 2024-09-05T07:07:21.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f04367fc640 1 -- 172.21.15.86:0/3091383357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0438032e60 con 0x564e00557b40 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f04367fc640 1 -- 172.21.15.86:0/3091383357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438035040 con 0x564e00557b40 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.052+0000 7f0454ee6740 1 -- 172.21.15.86:0/3091383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 msgr2=0x564e005716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0438009900 tx=0x7f0438000bc0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- 172.21.15.86:0/3091383357 shutdown_connections 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0055d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e005716f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3091383357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00552e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- 172.21.15.86:0/3091383357 >> 172.21.15.86:0/3091383357 conn(0x564e0051f310 msgr2=0x564e00552700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- 172.21.15.86:0/3091383357 shutdown_connections 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- 172.21.15.86:0/3091383357 wait complete. 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- start start 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 Processor -- start 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00569000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0056d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1ae540 con 0x564e00557b40 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1ade40 con 0x564e005565d0 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1aaf80 con 0x564e00561370 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f0454ee6740 10 client.0 _ll_register_callbacks cb 0x7f0444009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:21.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.053+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38934/0 (socket says 172.21.15.86:38934) 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 -- 172.21.15.86:0/3016491417 learned_addr learned my addr 172.21.15.86:0/3016491417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f04510aa640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0056d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f04508a9640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00569000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 -- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 msgr2=0x564e0056d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0056d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 -- 172.21.15.86:0/3016491417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 msgr2=0x564e00569000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00569000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 -- 172.21.15.86:0/3016491417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0056dd70 con 0x564e00557b40 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f044bfff640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f043802a930 tx=0x7f0438031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53a30b9a9c4cfcc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f042bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438055070 con 0x564e00557b40 2024-09-05T07:07:21.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f0438005450 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0438032e60 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0438061050 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f0454ee6740 10 client.21072 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564e005dfc00 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.054+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0438032e60 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f043803c4d0 con 0x564e00557b40 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 10 client.21072 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 10 client.21072 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 20 client.21072 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e005dfed0 0x564e005e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:21.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 4 client.21072 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 --> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e004c3d70 con 0x564e005dfed0 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f04508a9640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e005dfed0 0x564e005e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f04508a9640 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e005dfed0 0x564e005e22f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18d31455306bfc55 server_cookie=1f173f89623c1bca in_seq=0 out_seq=0 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.055+0000 7f042bfff640 10 client.21072 ms_handle_connect on v2:172.21.15.115:6832/3498674074 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f042bfff640 1 -- 172.21.15.86:0/3016491417 <== mds.4 v2:172.21.15.115:6832/3498674074 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e004c3d70 con 0x564e005dfed0 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f042bfff640 10 client.21072 handle_command_reply: tid=0 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 client.21072 shutdown 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 msgr2=0x564e00569540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f043802a930 tx=0x7f0438031400 comp rx=0 tx=0).stop 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 shutdown_connections 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:6832/3498674074,v1:172.21.15.115:6833/3498674074] conn(0x564e005dfed0 0x564e005e22f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e00561370 0x564e0056d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e00557b40 0x564e00569540 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 --2- 172.21.15.86:0/3016491417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e005565d0 0x564e00569000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 >> 172.21.15.86:0/3016491417 conn(0x564e0051f310 msgr2=0x564e00558930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f042bfff640 0 client.21072 ms_handle_reset on v2:172.21.15.115:6832/3498674074 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 shutdown_connections 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 1 -- 172.21.15.86:0/3016491417 wait complete. 2024-09-05T07:07:21.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.056+0000 7f0454ee6740 20 client.21072 trim_cache size 0 max 16384 2024-09-05T07:07:21.081 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:07:21.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 msgr2=0x7f043c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 0x7f043c0a5230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f04440092f0 tx=0x7f0444010740 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 msgr2=0x7f044c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000edf0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 shutdown_connections 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f043c0a2d70 0x7f043c0a5230 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f044c106f10 0x7f044c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f044c102790 0x7f044c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 --2- 172.21.15.86:0/348357739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044c101de0 0x7f044c198380 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 >> 172.21.15.86:0/348357739 conn(0x7f044c0fd8c0 msgr2=0x7f044c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 shutdown_connections 2024-09-05T07:07:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:21.057+0000 7f0452b0d640 1 -- 172.21.15.86:0/348357739 wait complete. 2024-09-05T07:07:21.672 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:21 smithi086 ceph-mon[30943]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:21.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:21 smithi115 ceph-mon[34791]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:21.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:21 smithi096 ceph-mon[34468]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-05T07:07:21.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-05T07:07:22.133 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 -- 172.21.15.86:0/2741832136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44100a80 msgr2=0x7fae44100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/2741832136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44100a80 0x7fae44100ee0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fae38009930 tx=0x7fae3802a420 comp rx=0 tx=0).stop 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 -- 172.21.15.86:0/2741832136 shutdown_connections 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/2741832136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44101420 0x7fae4410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/2741832136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44100a80 0x7fae44100ee0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/2741832136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 0x7fae44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 -- 172.21.15.86:0/2741832136 >> 172.21.15.86:0/2741832136 conn(0x7fae440fc7a0 msgr2=0x7fae440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 -- 172.21.15.86:0/2741832136 shutdown_connections 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.638+0000 7fae4b4b1640 1 -- 172.21.15.86:0/2741832136 wait complete. 2024-09-05T07:07:22.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 Processor -- start 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 -- start start 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44100a80 0x7fae441972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 0x7fae4419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae44110d30 con 0x7fae44101420 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae44110bb0 con 0x7fae44100a80 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4b4b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae44110eb0 con 0x7fae44106a80 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57482/0 (socket says 172.21.15.86:57482) 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 -- 172.21.15.86:0/558689779 learned_addr learned my addr 172.21.15.86:0/558689779 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae4924d640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44100a80 0x7fae441972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae49a4e640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 0x7fae4419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 -- 172.21.15.86:0/558689779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 msgr2=0x7fae4419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 0x7fae4419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 -- 172.21.15.86:0/558689779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44100a80 msgr2=0x7fae441972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44100a80 0x7fae441972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.639+0000 7fae48a4c640 1 -- 172.21.15.86:0/558689779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4419c260 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae48a4c640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fae38004770 tx=0x7fae380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae367fc640 1 -- 172.21.15.86:0/558689779 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3803f070 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4419c4f0 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae367fc640 1 -- 172.21.15.86:0/558689779 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae3803b460 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae367fc640 1 -- 172.21.15.86:0/558689779 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38035070 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.640+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a3e30 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.642+0000 7fae367fc640 1 -- 172.21.15.86:0/558689779 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae3803b610 con 0x7fae44101420 2024-09-05T07:07:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 Processor -- start 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 -- start start 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec7352c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aec72b62f0 con 0x55aec735d580 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aec725dcf0 con 0x55aec73527e0 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4d88a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aec72b22b0 con 0x55aec7353d50 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae367fc640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 0x7fae300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57496/0 (socket says 172.21.15.86:57496) 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae48a4c640 1 -- 172.21.15.86:0/1982660755 learned_addr learned my addr 172.21.15.86:0/1982660755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae49a4e640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 0x7fae300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae4924d640 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec7352c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.643+0000 7fae49a4e640 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae367fc640 1 -- 172.21.15.86:0/558689779 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7fae380efc50 con 0x7fae44101420 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 -- 172.21.15.86:0/1982660755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 msgr2=0x55aec73522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 -- 172.21.15.86:0/1982660755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 msgr2=0x55aec7352c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec7352c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 -- 172.21.15.86:0/1982660755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec72423a0 con 0x55aec735d580 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae49a4e640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 0x7fae300a51b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005fd0 tx=0x7fae3c0096a0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae48a4c640 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fae380bc390 tx=0x7fae3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f109485eecbb51d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae2f7fe640 1 -- 172.21.15.86:0/1982660755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38048070 con 0x55aec735d580 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae2f7fe640 1 -- 172.21.15.86:0/1982660755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae380bbd30 con 0x55aec735d580 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.644+0000 7fae2f7fe640 1 -- 172.21.15.86:0/1982660755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380ed1e0 con 0x55aec735d580 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- 172.21.15.86:0/1982660755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 msgr2=0x55aec735f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fae380bc390 tx=0x7fae3802a420 comp rx=0 tx=0).stop 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- 172.21.15.86:0/1982660755 shutdown_connections 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec735f970 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec7352c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 --2- 172.21.15.86:0/1982660755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- 172.21.15.86:0/1982660755 >> 172.21.15.86:0/1982660755 conn(0x55aec7319310 msgr2=0x55aec734ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- 172.21.15.86:0/1982660755 shutdown_connections 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- 172.21.15.86:0/1982660755 wait complete. 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 -- start start 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.645+0000 7fae4d88a740 1 Processor -- start 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec7358da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aec72b62f0 con 0x55aec7353d50 2024-09-05T07:07:22.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aec7271060 con 0x55aec735d580 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aec734c630 con 0x55aec73527e0 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 10 client.0 _ll_register_callbacks cb 0x55aec6fc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57502/0 (socket says 172.21.15.86:57502) 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 -- 172.21.15.86:0/2831257206 learned_addr learned my addr 172.21.15.86:0/2831257206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4924d640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae49a4e640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec7358da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 -- 172.21.15.86:0/2831257206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 msgr2=0x55aec7358320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 -- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 msgr2=0x55aec7358da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec7358da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 -- 172.21.15.86:0/2831257206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec70273c0 con 0x55aec7353d50 2024-09-05T07:07:22.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae49a4e640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec7358da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fae380bba30 tx=0x7fae380329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0813c33c742603 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380b5020 con 0x55aec7353d50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7fae380b83a0 con 0x55aec7353d50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.646+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aec72c3860 con 0x55aec7353d50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fae38048040 con 0x55aec7353d50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 10 client.21084 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aec73697a0 con 0x55aec7353d50 2024-09-05T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55aec72c3860 con 0x55aec7353d50 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fae380d2020 con 0x55aec7353d50 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 10 client.21084 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 10 client.21084 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 20 client.21084 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec736a920 0x55aec736cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 4 client.21084 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aec72bdd70 con 0x55aec736a920 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.647+0000 7fae4924d640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec736a920 0x55aec736cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.648+0000 7fae2cff9640 10 client.21084 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.648+0000 7fae4924d640 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec736a920 0x55aec736cd40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=476237bc64eae1cf server_cookie=a9c0acd62a26332c in_seq=0 out_seq=0 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae2cff9640 1 -- 172.21.15.86:0/2831257206 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aec72bdd70 con 0x55aec736a920 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae2cff9640 10 client.21084 handle_command_reply: tid=0 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 client.21084 shutdown 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 msgr2=0x55aec7358860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fae380bba30 tx=0x7fae380329f0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 shutdown_connections 2024-09-05T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec736a920 0x55aec736cd40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec7358da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec7358860 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae2cff9640 0 client.21084 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2831257206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 >> 172.21.15.86:0/2831257206 conn(0x55aec7319310 msgr2=0x55aec7354ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 shutdown_connections 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 1 -- 172.21.15.86:0/2831257206 wait complete. 2024-09-05T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.649+0000 7fae4d88a740 20 client.21084 trim_cache size 0 max 16384 2024-09-05T07:07:22.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 Processor -- start 2024-09-05T07:07:22.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 -- start start 2024-09-05T07:07:22.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec735dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae441acfb0 con 0x55aec7353d50 2024-09-05T07:07:22.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae441aeb80 con 0x55aec735d580 2024-09-05T07:07:22.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae4d88a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ad680 con 0x55aec73527e0 2024-09-05T07:07:22.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae48a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57510/0 (socket says 172.21.15.86:57510) 2024-09-05T07:07:22.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.667+0000 7fae48a4c640 1 -- 172.21.15.86:0/2195930299 learned_addr learned my addr 172.21.15.86:0/2195930299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:22.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae4924d640 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 -- 172.21.15.86:0/2195930299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 msgr2=0x55aec7358bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 -- 172.21.15.86:0/2195930299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 msgr2=0x55aec735dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec735dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 -- 172.21.15.86:0/2195930299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec7355330 con 0x55aec7353d50 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae48a4c640 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fae380ed560 tx=0x7fae380092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebed5346a43aea04 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae2effd640 1 -- 172.21.15.86:0/2195930299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380b5050 con 0x55aec7353d50 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae2effd640 1 -- 172.21.15.86:0/2195930299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae380bc430 con 0x55aec7353d50 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.668+0000 7fae2effd640 1 -- 172.21.15.86:0/2195930299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380bb180 con 0x55aec7353d50 2024-09-05T07:07:22.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- 172.21.15.86:0/2195930299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 msgr2=0x55aec735d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fae380ed560 tx=0x7fae380092d0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- 172.21.15.86:0/2195930299 shutdown_connections 2024-09-05T07:07:22.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec735d580 0x55aec735dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec7353d50 0x55aec735d960 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 --2- 172.21.15.86:0/2195930299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec73527e0 0x55aec7358bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- 172.21.15.86:0/2195930299 >> 172.21.15.86:0/2195930299 conn(0x55aec7319310 msgr2=0x55aec7355d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- 172.21.15.86:0/2195930299 shutdown_connections 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- 172.21.15.86:0/2195930299 wait complete. 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 -- start start 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.669+0000 7fae4d88a740 1 Processor -- start 2024-09-05T07:07:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec73611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ae4b0 con 0x55aec735d580 2024-09-05T07:07:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae441addb0 con 0x55aec73527e0 2024-09-05T07:07:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fae441aaef0 con 0x55aec7353d50 2024-09-05T07:07:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4d88a740 10 client.0 _ll_register_callbacks cb 0x7fae4000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57524/0 (socket says 172.21.15.86:57524) 2024-09-05T07:07:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 -- 172.21.15.86:0/3923269411 learned_addr learned my addr 172.21.15.86:0/3923269411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4924d640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec73611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae48a4c640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 -- 172.21.15.86:0/3923269411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 msgr2=0x55aec73616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 -- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 msgr2=0x55aec73611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec73611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae49a4e640 1 -- 172.21.15.86:0/3923269411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec73d3880 con 0x55aec735d580 2024-09-05T07:07:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.670+0000 7fae4924d640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec73611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:07:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae49a4e640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004310 tx=0x7fae3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d497d139241d0aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c003e20 con 0x55aec735d580 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7fae3c00e960 con 0x55aec735d580 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae380bc430 con 0x55aec735d580 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fae3c037050 con 0x55aec735d580 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae4d88a740 10 client.21090 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aec73d4b80 con 0x55aec735d580 2024-09-05T07:07:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae380bc430 con 0x55aec735d580 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.671+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7fae3c018070 con 0x55aec735d580 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 10 client.21090 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 10 client.21090 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 20 client.21090 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec73d5390 0x55aec73d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 4 client.21090 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:07:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 --> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aec723a170 con 0x55aec73d5390 2024-09-05T07:07:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4924d640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec73d5390 0x55aec73d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae2cff9640 10 client.21090 ms_handle_connect on v2:172.21.15.96:6832/1668283475 2024-09-05T07:07:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.672+0000 7fae4924d640 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec73d5390 0x55aec73d77b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14473a40cf66703e server_cookie=e88f1518f12ba4e9 in_seq=0 out_seq=0 2024-09-05T07:07:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae2cff9640 1 -- 172.21.15.86:0/3923269411 <== mds.3 v2:172.21.15.96:6832/1668283475 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aec723a170 con 0x55aec73d5390 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae2cff9640 10 client.21090 handle_command_reply: tid=0 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 client.21090 shutdown 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 msgr2=0x55aec7361c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004310 tx=0x7fae3c004ab0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 shutdown_connections 2024-09-05T07:07:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:6832/1668283475,v1:172.21.15.96:6833/1668283475] conn(0x55aec73d5390 0x55aec73d77b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae2cff9640 0 client.21090 ms_handle_reset on v2:172.21.15.96:6832/1668283475 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aec735d580 0x55aec7361c20 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aec7353d50 0x55aec73616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 --2- 172.21.15.86:0/3923269411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aec73527e0 0x55aec73611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 >> 172.21.15.86:0/3923269411 conn(0x55aec7319310 msgr2=0x55aec735d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 shutdown_connections 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 1 -- 172.21.15.86:0/3923269411 wait complete. 2024-09-05T07:07:22.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.673+0000 7fae4d88a740 20 client.21090 trim_cache size 0 max 16384 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 msgr2=0x7fae300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 0x7fae300a51b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005fd0 tx=0x7fae3c0096a0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 msgr2=0x7fae441977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fae38004770 tx=0x7fae380042f0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 shutdown_connections 2024-09-05T07:07:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fae300a2cf0 0x7fae300a51b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae44106a80 0x7fae4419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44101420 0x7fae441977f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 --2- 172.21.15.86:0/558689779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae44100a80 0x7fae441972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 >> 172.21.15.86:0/558689779 conn(0x7fae440fc7a0 msgr2=0x7fae440fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 shutdown_connections 2024-09-05T07:07:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:22.674+0000 7fae4b4b1640 1 -- 172.21.15.86:0/558689779 wait complete. 2024-09-05T07:07:23.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-05T07:07:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:23.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:23 smithi115 ceph-mon[34791]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:23.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:23 smithi086 ceph-mon[30943]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:23.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:23 smithi096 ceph-mon[34468]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:24.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 -- 172.21.15.86:0/1818813756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 msgr2=0x7f2a3c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 --2- 172.21.15.86:0/1818813756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c10da40 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009ca0 tx=0x7f2a3402a950 comp rx=0 tx=0).stop 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 -- 172.21.15.86:0/1818813756 shutdown_connections 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 --2- 172.21.15.86:0/1818813756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c10da40 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 --2- 172.21.15.86:0/1818813756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 0x7f2a3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 --2- 172.21.15.86:0/1818813756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 0x7f2a3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 -- 172.21.15.86:0/1818813756 >> 172.21.15.86:0/1818813756 conn(0x7f2a3c0fc7d0 msgr2=0x7f2a3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 -- 172.21.15.86:0/1818813756 shutdown_connections 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 -- 172.21.15.86:0/1818813756 wait complete. 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.132+0000 7f2a444a7640 1 Processor -- start 2024-09-05T07:07:24.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 -- start start 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 0x7f2a3c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 0x7f2a3c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c110e30 con 0x7f2a3c102ef0 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c110cb0 con 0x7f2a3c102570 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a444a7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c110fb0 con 0x7f2a3c100e20 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57554/0 (socket says 172.21.15.86:57554) 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 -- 172.21.15.86:0/3444408956 learned_addr learned my addr 172.21.15.86:0/3444408956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42243640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 0x7f2a3c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 0x7f2a3c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 -- 172.21.15.86:0/3444408956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 msgr2=0x7f2a3c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 0x7f2a3c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 -- 172.21.15.86:0/3444408956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 msgr2=0x7f2a3c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 0x7f2a3c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.133+0000 7f2a42a44640 1 -- 172.21.15.86:0/3444408956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c19c2c0 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a42a44640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2a3402ae70 tx=0x7f2a3402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a2f7fe640 1 -- 172.21.15.86:0/3444408956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34034040 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a2f7fe640 1 -- 172.21.15.86:0/3444408956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a340308f0 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a2f7fe640 1 -- 172.21.15.86:0/3444408956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004750 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c19c550 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.134+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c19c9e0 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.136+0000 7f2a2f7fe640 1 -- 172.21.15.86:0/3444408956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a34037020 con 0x7f2a3c102ef0 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.136+0000 7f2a46880740 1 Processor -- start 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.136+0000 7f2a46880740 1 -- start start 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef44d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d8ee992f0 con 0x557d8ef335d0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d8ee40cf0 con 0x557d8ef42970 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a46880740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d8ee952b0 con 0x557d8ef34a20 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a2f7fe640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 0x7f2a280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 -- 172.21.15.86:0/3888750238 learned_addr learned my addr 172.21.15.86:0/3888750238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42a44640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 0x7f2a280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a2f7fe640 1 -- 172.21.15.86:0/3444408956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2a340f0710 con 0x7f2a3c102ef0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef44d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 -- 172.21.15.86:0/3888750238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 msgr2=0x557d8ef33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 -- 172.21.15.86:0/3888750238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 msgr2=0x557d8ef44d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef44d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 -- 172.21.15.86:0/3888750238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8ee253a0 con 0x557d8ef335d0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef44d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42a44640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 0x7f2a280a51b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a340edbe0 tx=0x7f2a3408e8c0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.137+0000 7f2a42243640 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b760 tx=0x7f2a3800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33d892a4cfd2ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a2cff9640 1 -- 172.21.15.86:0/3888750238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003dd0 con 0x557d8ef335d0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a2cff9640 1 -- 172.21.15.86:0/3888750238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a38015070 con 0x557d8ef335d0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a2cff9640 1 -- 172.21.15.86:0/3888750238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004d90 con 0x557d8ef335d0 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 -- 172.21.15.86:0/3888750238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 msgr2=0x557d8ef33a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b760 tx=0x7f2a3800bc20 comp rx=0 tx=0).stop 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 -- 172.21.15.86:0/3888750238 shutdown_connections 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef44d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef33a10 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 --2- 172.21.15.86:0/3888750238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 -- 172.21.15.86:0/3888750238 >> 172.21.15.86:0/3888750238 conn(0x557d8eefc310 msgr2=0x557d8ef2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 -- 172.21.15.86:0/3888750238 shutdown_connections 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.138+0000 7f2a46880740 1 -- 172.21.15.86:0/3888750238 wait complete. 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 -- start start 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 Processor -- start 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef335d0 0x557d8ef42340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d8ef45540 con 0x557d8ef42970 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d8ee54060 con 0x557d8ef335d0 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d8ef456c0 con 0x557d8ef34a20 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a46880740 10 client.0 _ll_register_callbacks cb 0x557d8eba78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57564/0 (socket says 172.21.15.86:57564) 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a41a42640 1 -- 172.21.15.86:0/3071582349 learned_addr learned my addr 172.21.15.86:0/3071582349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a42243640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.139+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef335d0 0x557d8ef42340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 -- 172.21.15.86:0/3071582349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 msgr2=0x557d8ef3f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 -- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef335d0 msgr2=0x557d8ef42340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef335d0 0x557d8ef42340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 -- 172.21.15.86:0/3071582349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8ec0a3c0 con 0x557d8ef42970 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000ae70 tx=0x7f2a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3167efc877bf9f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005220 con 0x557d8ef42970 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2a300053c0 con 0x557d8ef42970 2024-09-05T07:07:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a38015070 con 0x557d8ef42970 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2a300055c0 con 0x557d8ef42970 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 10 client.21102 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557d8ef51060 con 0x557d8ef42970 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a38015070 con 0x557d8ef42970 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f2a3001d030 con 0x557d8ef42970 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 10 client.21102 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 10 client.21102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.140+0000 7f2a46880740 20 client.21102 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8ef52520 0x557d8ef54910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a46880740 4 client.21102 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d8eea6860 con 0x557d8ef52520 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a42243640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8ef52520 0x557d8ef54910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a42243640 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8ef52520 0x557d8ef54910 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d02750ddc9c2602d server_cookie=60b3b2d36875556d in_seq=0 out_seq=0 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.141+0000 7f2a25ffb640 10 client.21102 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a25ffb640 1 -- 172.21.15.86:0/3071582349 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d8eea6860 con 0x557d8ef52520 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a25ffb640 10 client.21102 handle_command_reply: tid=0 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 client.21102 shutdown 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 msgr2=0x557d8ef3fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000ae70 tx=0x7f2a300046c0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 shutdown_connections 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8ef52520 0x557d8ef54910 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef3fbe0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a25ffb640 0 client.21102 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 --2- 172.21.15.86:0/3071582349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef335d0 0x557d8ef42340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 >> 172.21.15.86:0/3071582349 conn(0x557d8eefc310 msgr2=0x557d8ef359c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 shutdown_connections 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 1 -- 172.21.15.86:0/3071582349 wait complete. 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.142+0000 7f2a46880740 20 client.21102 trim_cache size 0 max 16384 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 Processor -- start 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 -- start start 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef42d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1ace10 con 0x557d8ef335d0 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1ae9e0 con 0x557d8ef42970 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a46880740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1ad4e0 con 0x557d8ef34a20 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57576/0 (socket says 172.21.15.86:57576) 2024-09-05T07:07:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.155+0000 7f2a41a42640 1 -- 172.21.15.86:0/3070133829 learned_addr learned my addr 172.21.15.86:0/3070133829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a42243640 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 -- 172.21.15.86:0/3070133829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 msgr2=0x557d8ef3d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 -- 172.21.15.86:0/3070133829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 msgr2=0x557d8ef42d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef42d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 -- 172.21.15.86:0/3070133829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8ef4ece0 con 0x557d8ef335d0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a41a42640 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005940 tx=0x7f2a3000dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef4e2ac62ffe2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a27fff640 1 -- 172.21.15.86:0/3070133829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002f070 con 0x557d8ef335d0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a27fff640 1 -- 172.21.15.86:0/3070133829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3000fdf0 con 0x557d8ef335d0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.156+0000 7f2a27fff640 1 -- 172.21.15.86:0/3070133829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002bb60 con 0x557d8ef335d0 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- 172.21.15.86:0/3070133829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 msgr2=0x557d8ef4e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005940 tx=0x7f2a3000dc20 comp rx=0 tx=0).stop 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- 172.21.15.86:0/3070133829 shutdown_connections 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef42970 0x557d8ef42d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef34a20 0x557d8ef3d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- 172.21.15.86:0/3070133829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef335d0 0x557d8ef4e7a0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- 172.21.15.86:0/3070133829 >> 172.21.15.86:0/3070133829 conn(0x557d8eefc310 msgr2=0x557d8ef3c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- 172.21.15.86:0/3070133829 shutdown_connections 2024-09-05T07:07:24.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- 172.21.15.86:0/3070133829 wait complete. 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- start start 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 Processor -- start 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef335d0 0x557d8ef5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef34a20 0x557d8ef4a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1ae310 con 0x557d8ef42970 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1adc10 con 0x557d8ef34a20 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1aad50 con 0x557d8ef335d0 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a46880740 10 client.0 _ll_register_callbacks cb 0x7f2a30005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a41a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57584/0 (socket says 172.21.15.86:57584) 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.157+0000 7f2a41a42640 1 -- 172.21.15.86:0/4243754498 learned_addr learned my addr 172.21.15.86:0/4243754498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef335d0 0x557d8ef5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a42243640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef34a20 0x557d8ef4a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 -- 172.21.15.86:0/4243754498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef335d0 msgr2=0x557d8ef5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef335d0 0x557d8ef5a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 -- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef34a20 msgr2=0x557d8ef4a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef34a20 0x557d8ef4a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 -- 172.21.15.86:0/4243754498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8ef4b4f0 con 0x557d8ef42970 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a41a42640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000a3f0 tx=0x7f2a3002b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abff103d3d0ca63f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002f040 con 0x557d8ef42970 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2a3002b7e0 con 0x557d8ef42970 2024-09-05T07:07:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3000fdf0 con 0x557d8ef42970 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2a3000f4b0 con 0x557d8ef42970 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a46880740 10 client.21108 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557d8efbd250 con 0x557d8ef42970 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.158+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3000fdf0 con 0x557d8ef42970 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f2a3001d050 con 0x557d8ef42970 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 10 client.21108 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 10 client.21108 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 20 client.21108 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8efbd920 0x557d8efbfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 4 client.21108 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 --> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d8eea0d70 con 0x557d8efbd920 2024-09-05T07:07:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a42243640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8efbd920 0x557d8efbfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a25ffb640 10 client.21108 ms_handle_connect on v2:172.21.15.96:6836/1007288434 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a42243640 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8efbd920 0x557d8efbfd40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e3ea9be8f1ba7 server_cookie=a42a1293fbadd05d in_seq=0 out_seq=0 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a25ffb640 1 -- 172.21.15.86:0/4243754498 <== mds.2 v2:172.21.15.96:6836/1007288434 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d8eea0d70 con 0x557d8efbd920 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a25ffb640 10 client.21108 handle_command_reply: tid=0 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 1 client.21108 shutdown 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 msgr2=0x557d8ef4ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.159+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000a3f0 tx=0x7f2a3002b740 comp rx=0 tx=0).stop 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 shutdown_connections 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:6836/1007288434,v1:172.21.15.96:6837/1007288434] conn(0x557d8efbd920 0x557d8efbfd40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d8ef42970 0x557d8ef4ae50 secure :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000a3f0 tx=0x7f2a3002b740 comp rx=0 tx=0).stop 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d8ef34a20 0x557d8ef4a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 --2- 172.21.15.86:0/4243754498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d8ef335d0 0x557d8ef5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 >> 172.21.15.86:0/4243754498 conn(0x557d8eefc310 msgr2=0x557d8ef3d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a25ffb640 0 client.21108 ms_handle_reset on v2:172.21.15.96:6836/1007288434 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 shutdown_connections 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 1 -- 172.21.15.86:0/4243754498 wait complete. 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.160+0000 7f2a46880740 20 client.21108 trim_cache size 0 max 16384 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 msgr2=0x7f2a280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 0x7f2a280a51b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a340edbe0 tx=0x7f2a3408e8c0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 msgr2=0x7f2a3c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2a3402ae70 tx=0x7f2a3402f5d0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 shutdown_connections 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2a280a2cf0 0x7f2a280a51b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a3c102ef0 0x7f2a3c19bbe0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c102570 0x7f2a3c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 --2- 172.21.15.86:0/3444408956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c100e20 0x7f2a3c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 >> 172.21.15.86:0/3444408956 conn(0x7f2a3c0fc7d0 msgr2=0x7f2a3c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 shutdown_connections 2024-09-05T07:07:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:24.161+0000 7f2a444a7640 1 -- 172.21.15.86:0/3444408956 wait complete. 2024-09-05T07:07:24.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-05T07:07:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:25.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:25 smithi086 ceph-mon[30943]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:25.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:25 smithi115 ceph-mon[34791]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 -- 172.21.15.86:0/88394318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 msgr2=0x7f0ea4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 --2- 172.21.15.86:0/88394318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea4106f90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e9402a410 comp rx=0 tx=0).stop 2024-09-05T07:07:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 -- 172.21.15.86:0/88394318 shutdown_connections 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 --2- 172.21.15.86:0/88394318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea4100970 0x7f0ea410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 --2- 172.21.15.86:0/88394318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40fffd0 0x7f0ea4100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 --2- 172.21.15.86:0/88394318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea4106f90 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 -- 172.21.15.86:0/88394318 >> 172.21.15.86:0/88394318 conn(0x7f0ea40fbee0 msgr2=0x7f0ea40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 -- 172.21.15.86:0/88394318 shutdown_connections 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.693+0000 7f0eaa582640 1 -- 172.21.15.86:0/88394318 wait complete. 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 Processor -- start 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 -- start start 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 0x7f0ea41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 0x7f0ea41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4111f30 con 0x7f0ea4106bb0 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4111db0 con 0x7f0ea40fffd0 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.694+0000 7f0eaa582640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41120b0 con 0x7f0ea4100970 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57604/0 (socket says 172.21.15.86:57604) 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 -- 172.21.15.86:0/713291688 learned_addr learned my addr 172.21.15.86:0/713291688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 0x7f0ea41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 0x7f0ea41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 -- 172.21.15.86:0/713291688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 msgr2=0x7f0ea41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 0x7f0ea41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 -- 172.21.15.86:0/713291688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 msgr2=0x7f0ea41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 0x7f0ea41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 -- 172.21.15.86:0/713291688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea419d360 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 0x7f0ea41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 0x7f0ea41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea8b1f640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9d0 tx=0x7f0e9c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea17fa640 1 -- 172.21.15.86:0/713291688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00cd80 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea419d650 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea17fa640 1 -- 172.21.15.86:0/713291688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e9c005020 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0ea17fa640 1 -- 172.21.15.86:0/713291688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0116d0 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.695+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41a4f30 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.697+0000 7f0ea17fa640 1 -- 172.21.15.86:0/713291688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0e9c0051d0 con 0x7f0ea4106bb0 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 Processor -- start 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 -- start start 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8665090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e8665a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2e85cb2f0 con 0x55a2e8670370 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2e8572cf0 con 0x55a2e8666b40 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0eac95b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a2e85c72b0 con 0x55a2e86655d0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea17fa640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 0x7f0e8c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57614/0 (socket says 172.21.15.86:57614) 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3622712520 learned_addr learned my addr 172.21.15.86:0/3622712520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea8b1f640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 0x7f0e8c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e8665a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8665090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea17fa640 1 -- 172.21.15.86:0/713291688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f0e9c0cfae0 con 0x7f0ea4106bb0 2024-09-05T07:07:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3622712520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 msgr2=0x55a2e8665a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e8665a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3622712520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 msgr2=0x55a2e8665090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8665090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3622712520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e85573a0 con 0x55a2e8670370 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.698+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e8665a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0e980125f0 tx=0x7f0e9800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c105db174ad17021 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0ea8b1f640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 0x7f0e8c0a5230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cd3f0 tx=0x7f0e9c080040 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0e8a7fc640 1 -- 172.21.15.86:0/3622712520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98018030 con 0x55a2e8670370 2024-09-05T07:07:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0e8a7fc640 1 -- 172.21.15.86:0/3622712520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e980054d0 con 0x55a2e8670370 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0e8a7fc640 1 -- 172.21.15.86:0/3622712520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98010920 con 0x55a2e8670370 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0eac95b740 1 -- 172.21.15.86:0/3622712520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 msgr2=0x55a2e8672760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.699+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0e980125f0 tx=0x7f0e9800b6e0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 -- 172.21.15.86:0/3622712520 shutdown_connections 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8670370 0x55a2e8672760 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e8665a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3622712520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8665090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 -- 172.21.15.86:0/3622712520 >> 172.21.15.86:0/3622712520 conn(0x55a2e862e310 msgr2=0x55a2e8661c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 -- 172.21.15.86:0/3622712520 shutdown_connections 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 -- 172.21.15.86:0/3622712520 wait complete. 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 -- start start 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 Processor -- start 2024-09-05T07:07:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e866fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.700+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8670370 0x55a2e866d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2e8673070 con 0x55a2e8666b40 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2e8586060 con 0x55a2e8670370 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a2e86731f0 con 0x55a2e86655d0 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 10 client.0 _ll_register_callbacks cb 0x55a2e82d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57620/0 (socket says 172.21.15.86:57620) 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 -- 172.21.15.86:0/1203666573 learned_addr learned my addr 172.21.15.86:0/1203666573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e866fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8670370 0x55a2e866d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 -- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 msgr2=0x55a2e866fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e866fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 -- 172.21.15.86:0/1203666573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8670370 msgr2=0x55a2e866d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8670370 0x55a2e866d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 -- 172.21.15.86:0/1203666573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e833c3c0 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0e940096c0 tx=0x7f0e94031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7616a69bba101d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94035070 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f0e94004300 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e980054d0 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0e9404d050 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 10 client.21120 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.701+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a2e867eae0 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e980054d0 con 0x55a2e8666b40 2024-09-05T07:07:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f0e94004a30 con 0x55a2e8666b40 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 10 client.21120 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 10 client.21120 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 20 client.21120 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e867fff0 0x55a2e8682410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 4 client.21120 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e85d8860 con 0x55a2e867fff0 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.702+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e867fff0 0x55a2e8682410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e867fff0 0x55a2e8682410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=400aea1618f234a6 server_cookie=fff25bd801122ec2 in_seq=0 out_seq=0 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0e837fe640 10 client.21120 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0e837fe640 1 -- 172.21.15.86:0/1203666573 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2e85d8860 con 0x55a2e867fff0 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0e837fe640 10 client.21120 handle_command_reply: tid=0 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 client.21120 shutdown 2024-09-05T07:07:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 msgr2=0x55a2e866cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0e940096c0 tx=0x7f0e94031820 comp rx=0 tx=0).stop 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 shutdown_connections 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e867fff0 0x55a2e8682410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8670370 0x55a2e866d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e866cff0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0e837fe640 0 client.21120 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 --2- 172.21.15.86:0/1203666573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e86655d0 0x55a2e866fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.703+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 >> 172.21.15.86:0/1203666573 conn(0x55a2e862e310 msgr2=0x55a2e8667960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.704+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 shutdown_connections 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.704+0000 7f0eac95b740 1 -- 172.21.15.86:0/1203666573 wait complete. 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.704+0000 7f0eac95b740 20 client.21120 trim_cache size 0 max 16384 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.716+0000 7f0eac95b740 1 Processor -- start 2024-09-05T07:07:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.716+0000 7f0eac95b740 1 -- start start 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e86655d0 0x55a2e8661d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e866bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41af120 con 0x55a2e8666b40 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b0cf0 con 0x55a2e86655d0 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0eac95b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41af7f0 con 0x55a2e8670370 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57628/0 (socket says 172.21.15.86:57628) 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 -- 172.21.15.86:0/2152086790 learned_addr learned my addr 172.21.15.86:0/2152086790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e86655d0 0x55a2e8661d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e866bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 -- 172.21.15.86:0/2152086790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 msgr2=0x55a2e866bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e866bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 -- 172.21.15.86:0/2152086790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e86655d0 msgr2=0x55a2e8661d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e86655d0 0x55a2e8661d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 -- 172.21.15.86:0/2152086790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e8682a70 con 0x55a2e8666b40 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e866bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009850 tx=0x7f0e940035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672a904c9a268dae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.717+0000 7f0e89ffb640 1 -- 172.21.15.86:0/2152086790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94056070 con 0x55a2e8666b40 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0e89ffb640 1 -- 172.21.15.86:0/2152086790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e94003770 con 0x55a2e8666b40 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0e89ffb640 1 -- 172.21.15.86:0/2152086790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94035030 con 0x55a2e8666b40 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 -- 172.21.15.86:0/2152086790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 msgr2=0x55a2e867c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009850 tx=0x7f0e940035f0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 -- 172.21.15.86:0/2152086790 shutdown_connections 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e866bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e8666b40 0x55a2e867c310 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 --2- 172.21.15.86:0/2152086790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e86655d0 0x55a2e8661d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 -- 172.21.15.86:0/2152086790 >> 172.21.15.86:0/2152086790 conn(0x55a2e862e310 msgr2=0x55a2e866b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 -- 172.21.15.86:0/2152086790 shutdown_connections 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.718+0000 7f0eac95b740 1 -- 172.21.15.86:0/2152086790 wait complete. 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 -- start start 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 Processor -- start 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8678c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e8669860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b0620 con 0x55a2e86655d0 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41aff20 con 0x55a2e8666b40 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41ad060 con 0x55a2e8670370 2024-09-05T07:07:25.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0eac95b740 10 client.0 _ll_register_callbacks cb 0x7f0e94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57640/0 (socket says 172.21.15.86:57640) 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3899815022 learned_addr learned my addr 172.21.15.86:0/3899815022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8678c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 msgr2=0x55a2e8669860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e8669860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3899815022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 msgr2=0x55a2e8678c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8678c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea37fe640 1 -- 172.21.15.86:0/3899815022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e866a060 con 0x55a2e86655d0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.719+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8678c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0ea37fe640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f0e980125f0 tx=0x7f0e9800b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f120e8078a4b6a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98018040 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f0e980046f0 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e94003770 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0e98039050 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 10 client.21126 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a2e86ea700 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e94003770 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f0e98010720 con 0x55a2e86655d0 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 10 client.21126 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 10 client.21126 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 20 client.21126 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e86eac40 0x55a2e86ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 4 client.21126 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:07:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.720+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 --> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2e85d2d70 con 0x55a2e86eac40 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e86eac40 0x55a2e86ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0ea3fff640 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e86eac40 0x55a2e86ed060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2765d5807fdb746 server_cookie=f740122d41d65968 in_seq=0 out_seq=0 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0e837fe640 10 client.21126 ms_handle_connect on v2:172.21.15.115:6838/4249007973 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0e837fe640 1 -- 172.21.15.86:0/3899815022 <== mds.1 v2:172.21.15.115:6838/4249007973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2e85d2d70 con 0x55a2e86eac40 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0e837fe640 10 client.21126 handle_command_reply: tid=0 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 client.21126 shutdown 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 msgr2=0x55a2e86786d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f0e980125f0 tx=0x7f0e9800b750 comp rx=0 tx=0).stop 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 shutdown_connections 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:6838/4249007973,v1:172.21.15.115:6839/4249007973] conn(0x55a2e86eac40 0x55a2e86ed060 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0e837fe640 0 client.21126 ms_handle_reset on v2:172.21.15.115:6838/4249007973 2024-09-05T07:07:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a2e8670370 0x55a2e8669860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2e8666b40 0x55a2e8678c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 --2- 172.21.15.86:0/3899815022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2e86655d0 0x55a2e86786d0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.721+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 >> 172.21.15.86:0/3899815022 conn(0x55a2e862e310 msgr2=0x55a2e8667110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 shutdown_connections 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eac95b740 1 -- 172.21.15.86:0/3899815022 wait complete. 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eac95b740 20 client.21126 trim_cache size 0 max 16384 2024-09-05T07:07:25.740 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 msgr2=0x7f0e8c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 0x7f0e8c0a5230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cd3f0 tx=0x7f0e9c080040 comp rx=0 tx=0).stop 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 msgr2=0x7f0ea419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9d0 tx=0x7f0e9c00ee90 comp rx=0 tx=0).stop 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 shutdown_connections 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f0e8c0a2d70 0x7f0e8c0a5230 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea4106bb0 0x7f0ea419cc80 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea4100970 0x7f0ea41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 --2- 172.21.15.86:0/713291688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea40fffd0 0x7f0ea41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 >> 172.21.15.86:0/713291688 conn(0x7f0ea40fbee0 msgr2=0x7f0ea40fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 shutdown_connections 2024-09-05T07:07:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:25.722+0000 7f0eaa582640 1 -- 172.21.15.86:0/713291688 wait complete. 2024-09-05T07:07:25.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:25 smithi096 ceph-mon[34468]: pgmap v825: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:26.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T07:07:26.886 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 -- 172.21.15.86:0/4105788733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 msgr2=0x7f2c101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 --2- 172.21.15.86:0/4105788733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c101029b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009900 tx=0x7f2bf802a480 comp rx=0 tx=0).stop 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 -- 172.21.15.86:0/4105788733 shutdown_connections 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 --2- 172.21.15.86:0/4105788733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10102ef0 0x7f2c1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 --2- 172.21.15.86:0/4105788733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c101029b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 --2- 172.21.15.86:0/4105788733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10100e20 0x7f2c10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 -- 172.21.15.86:0/4105788733 >> 172.21.15.86:0/4105788733 conn(0x7f2c100fc7d0 msgr2=0x7f2c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 -- 172.21.15.86:0/4105788733 shutdown_connections 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.356+0000 7f2c14af3640 1 -- 172.21.15.86:0/4105788733 wait complete. 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 Processor -- start 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 -- start start 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 0x7f2c10198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10102ef0 0x7f2c1019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10110e30 con 0x7f2c10102570 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10110cb0 con 0x7f2c10100e20 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.357+0000 7f2c14af3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10110fb0 con 0x7f2c10102ef0 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57664/0 (socket says 172.21.15.86:57664) 2024-09-05T07:07:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 -- 172.21.15.86:0/277288290 learned_addr learned my addr 172.21.15.86:0/277288290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 0x7f2c10198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10102ef0 0x7f2c1019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 -- 172.21.15.86:0/277288290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10102ef0 msgr2=0x7f2c1019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10102ef0 0x7f2c1019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 -- 172.21.15.86:0/277288290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 msgr2=0x7f2c10198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 0x7f2c10198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 -- 172.21.15.86:0/277288290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019d4c0 con 0x7f2c10102570 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 0x7f2c10198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c07fff640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006010 tx=0x7f2bf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c077fe640 1 -- 172.21.15.86:0/277288290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf803f030 con 0x7f2c10102570 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c077fe640 1 -- 172.21.15.86:0/277288290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bf8003cb0 con 0x7f2c10102570 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019d750 con 0x7f2c10102570 2024-09-05T07:07:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.358+0000 7f2c077fe640 1 -- 172.21.15.86:0/277288290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8035070 con 0x7f2c10102570 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.359+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101a5090 con 0x7f2c10102570 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 Processor -- start 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 -- start start 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c077fe640 1 -- 172.21.15.86:0/277288290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2bf8042030 con 0x7f2c10102570 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1503090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1510760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614b14692f0 con 0x5614b15035d0 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5614b1410cf0 con 0x5614b1504b40 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.361+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5614b14652b0 con 0x5614b150e370 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57672/0 (socket says 172.21.15.86:57672) 2024-09-05T07:07:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 -- 172.21.15.86:0/2366847356 learned_addr learned my addr 172.21.15.86:0/2366847356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1503090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c07fff640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1510760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 -- 172.21.15.86:0/2366847356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 msgr2=0x5614b1510760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1510760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 -- 172.21.15.86:0/2366847356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 msgr2=0x5614b1503090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1503090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 -- 172.21.15.86:0/2366847356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b13f53a0 con 0x5614b15035d0 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1503090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2c000104d0 tx=0x7f2c00010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5d4bd7bcca92db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c077fe640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 0x7f2bfc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c04ff9640 1 -- 172.21.15.86:0/2366847356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00011760 con 0x5614b15035d0 2024-09-05T07:07:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 0x7f2bfc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c04ff9640 1 -- 172.21.15.86:0/2366847356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00011db0 con 0x5614b15035d0 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c04ff9640 1 -- 172.21.15.86:0/2366847356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0001ce10 con 0x5614b15035d0 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.362+0000 7f2c077fe640 1 -- 172.21.15.86:0/277288290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2bf80efcc0 con 0x7f2c10102570 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 -- 172.21.15.86:0/2366847356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 msgr2=0x5614b1503a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2c000104d0 tx=0x7f2c00010990 comp rx=0 tx=0).stop 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 0x7f2bfc0a51b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2c08005fd0 tx=0x7f2c08009600 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 -- 172.21.15.86:0/2366847356 shutdown_connections 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1510760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b1503a10 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/2366847356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1503090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 -- 172.21.15.86:0/2366847356 >> 172.21.15.86:0/2366847356 conn(0x5614b14cc310 msgr2=0x5614b14ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 -- 172.21.15.86:0/2366847356 shutdown_connections 2024-09-05T07:07:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.363+0000 7f2c16ecc740 1 -- 172.21.15.86:0/2366847356 wait complete. 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 -- start start 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 Processor -- start 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b150aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b150b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614b1511070 con 0x5614b15035d0 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5614b1424060 con 0x5614b1504b40 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5614b15111f0 con 0x5614b150e370 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c16ecc740 10 client.0 _ll_register_callbacks cb 0x5614b11778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57682/0 (socket says 172.21.15.86:57682) 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c0e59c640 1 -- 172.21.15.86:0/3233319812 learned_addr learned my addr 172.21.15.86:0/3233319812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b150aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.364+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b150b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 -- 172.21.15.86:0/3233319812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 msgr2=0x5614b150b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b150b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 -- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 msgr2=0x5614b150aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b150aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 -- 172.21.15.86:0/3233319812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b11da3c0 con 0x5614b15035d0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000be50 tx=0x7f2c0001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8db4eefc8ccbe3f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0002a060 con 0x5614b15035d0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2c0001cda0 con 0x5614b15035d0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c00011db0 con 0x5614b15035d0 2024-09-05T07:07:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2c00031050 con 0x5614b15035d0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c16ecc740 10 client.21138 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5614b151cae0 con 0x5614b15035d0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.365+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00011db0 con 0x5614b15035d0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f2c00011560 con 0x5614b15035d0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 10 client.21138 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 10 client.21138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 20 client.21138 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b151dee0 0x5614b15202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 4 client.21138 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b1476860 con 0x5614b151dee0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b151dee0 0x5614b15202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2bf5ffb640 10 client.21138 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.366+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b151dee0 0x5614b15202d0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dccd0b04bfec316c server_cookie=82c5a0c2ee665167 in_seq=0 out_seq=0 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/3233319812 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614b1476860 con 0x5614b151dee0 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2bf5ffb640 10 client.21138 handle_command_reply: tid=0 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 client.21138 shutdown 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 msgr2=0x5614b150dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000be50 tx=0x7f2c0001c480 comp rx=0 tx=0).stop 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 shutdown_connections 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b151dee0 0x5614b15202d0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b150b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b150aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2bf5ffb640 0 client.21138 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3233319812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b150dd40 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 >> 172.21.15.86:0/3233319812 conn(0x5614b14cc310 msgr2=0x5614b1505960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 shutdown_connections 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3233319812 wait complete. 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.367+0000 7f2c16ecc740 20 client.21138 trim_cache size 0 max 16384 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.380+0000 7f2c16ecc740 1 Processor -- start 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.380+0000 7f2c16ecc740 1 -- start start 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b15035d0 0x5614b151a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b150e370 0x5614b1506660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101af1e0 con 0x5614b1504b40 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101b0db0 con 0x5614b150e370 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101af8b0 con 0x5614b15035d0 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57694/0 (socket says 172.21.15.86:57694) 2024-09-05T07:07:27.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 -- 172.21.15.86:0/3853946353 learned_addr learned my addr 172.21.15.86:0/3853946353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b15035d0 0x5614b151a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b150e370 0x5614b1506660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 -- 172.21.15.86:0/3853946353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b15035d0 msgr2=0x5614b151a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b15035d0 0x5614b151a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 -- 172.21.15.86:0/3853946353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b150e370 msgr2=0x5614b1506660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b150e370 0x5614b1506660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 -- 172.21.15.86:0/3853946353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b1500960 con 0x5614b1504b40 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.381+0000 7f2c07fff640 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80ed5d0 tx=0x7f2bf8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62bd135f4bcd9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.382+0000 7f2bf7fff640 1 -- 172.21.15.86:0/3853946353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80bf070 con 0x5614b1504b40 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.383+0000 7f2bf7fff640 1 -- 172.21.15.86:0/3853946353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bf80bbe50 con 0x5614b1504b40 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.383+0000 7f2bf7fff640 1 -- 172.21.15.86:0/3853946353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8049040 con 0x5614b1504b40 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.383+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3853946353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 msgr2=0x5614b1506120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.383+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80ed5d0 tx=0x7f2bf8005e00 comp rx=0 tx=0).stop 2024-09-05T07:07:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3853946353 shutdown_connections 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b150e370 0x5614b1506660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b1504b40 0x5614b1506120 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/3853946353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b15035d0 0x5614b151a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3853946353 >> 172.21.15.86:0/3853946353 conn(0x5614b14cc310 msgr2=0x5614b150a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3853946353 shutdown_connections 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 -- 172.21.15.86:0/3853946353 wait complete. 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 -- start start 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 Processor -- start 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1507b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.384+0000 7f2c16ecc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1508070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101b06e0 con 0x5614b15035d0 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101affe0 con 0x5614b1504b40 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101ad120 con 0x5614b150e370 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 10 client.0 _ll_register_callbacks cb 0x7f2c0000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57708/0 (socket says 172.21.15.86:57708) 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 -- 172.21.15.86:0/1580753453 learned_addr learned my addr 172.21.15.86:0/1580753453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c07fff640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1507b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1508070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 -- 172.21.15.86:0/1580753453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 msgr2=0x5614b1508070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1508070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 -- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 msgr2=0x5614b1507b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1507b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 -- 172.21.15.86:0/1580753453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b1508660 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0ed9d640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1508070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c07fff640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1507b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c0e59c640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2c000109d0 tx=0x7f2c00011790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba276879f9f3fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0002a070 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2c00011d60 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf80bbe50 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2c0001c4f0 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 10 client.21144 fetch_fsmap learned FSMap version 50 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.385+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5614b1588640 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bf80bbe50 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7f2c000035f0 con 0x5614b15035d0 2024-09-05T07:07:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 10 client.21144 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T07:07:27.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 10 client.21144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T07:07:27.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 20 client.21144 populate_metadata read hostname 'smithi086' 2024-09-05T07:07:27.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b1588cf0 0x5614b158b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:27.415 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-09-05T07:07:27.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 4 client.21144 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 --> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614b1470d70 con 0x5614b1588cf0 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.386+0000 7f2c07fff640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b1588cf0 0x5614b158b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2bf5ffb640 10 client.21144 ms_handle_connect on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c07fff640 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b1588cf0 0x5614b158b110 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2bfbcedc94218c server_cookie=bb04a94a71fcdc0f in_seq=0 out_seq=0 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2bf5ffb640 1 -- 172.21.15.86:0/1580753453 <== mds.0 v2:172.21.15.96:6838/2996609527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614b1470d70 con 0x5614b1588cf0 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2bf5ffb640 10 client.21144 handle_command_reply: tid=0 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 client.21144 shutdown 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 msgr2=0x5614b15169d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2c000109d0 tx=0x7f2c00011790 comp rx=0 tx=0).stop 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 shutdown_connections 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:6838/2996609527,v1:172.21.15.96:6839/2996609527] conn(0x5614b1588cf0 0x5614b158b110 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2bf5ffb640 0 client.21144 ms_handle_reset on v2:172.21.15.96:6838/2996609527 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614b150e370 0x5614b1508070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614b1504b40 0x5614b1507b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 --2- 172.21.15.86:0/1580753453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614b15035d0 0x5614b15169d0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 >> 172.21.15.86:0/1580753453 conn(0x5614b14cc310 msgr2=0x5614b14fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 shutdown_connections 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 1 -- 172.21.15.86:0/1580753453 wait complete. 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.387+0000 7f2c16ecc740 20 client.21144 trim_cache size 0 max 16384 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 msgr2=0x7f2bfc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 0x7f2bfc0a51b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2c08005fd0 tx=0x7f2c08009600 comp rx=0 tx=0).stop 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 msgr2=0x7f2c10198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006010 tx=0x7f2bf80042f0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 shutdown_connections 2024-09-05T07:07:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bfc0a2cf0 0x7f2bfc0a51b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c10102ef0 0x7f2c1019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c10102570 0x7f2c10198a50 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 --2- 172.21.15.86:0/277288290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c10100e20 0x7f2c10198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 >> 172.21.15.86:0/277288290 conn(0x7f2c100fc7d0 msgr2=0x7f2c1006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 shutdown_connections 2024-09-05T07:07:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:27.388+0000 7f2c14af3640 1 -- 172.21.15.86:0/277288290 wait complete. 2024-09-05T07:07:27.553 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:27 smithi086 ceph-mon[30943]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:27.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:27 smithi115 ceph-mon[34791]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:27.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:27 smithi096 ceph-mon[34468]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-05T07:07:28.124 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-05T07:07:28.125 INFO:tasks.fwd_scrub:done joining 2024-09-05T07:07:28.125 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-05T07:07:28.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T07:07:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- 172.21.15.86:0/2479271777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac101320 msgr2=0x7f24ac10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 --2- 172.21.15.86:0/2479271777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac101320 0x7f24ac10d850 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009cd0 tx=0x7f24a402a8e0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- 172.21.15.86:0/2479271777 shutdown_connections 2024-09-05T07:07:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 --2- 172.21.15.86:0/2479271777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac101320 0x7f24ac10d850 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 --2- 172.21.15.86:0/2479271777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac100980 0x7f24ac100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 --2- 172.21.15.86:0/2479271777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac106990 0x7f24ac106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- 172.21.15.86:0/2479271777 >> 172.21.15.86:0/2479271777 conn(0x7f24ac077a10 msgr2=0x7f24ac0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- 172.21.15.86:0/2479271777 shutdown_connections 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- 172.21.15.86:0/2479271777 wait complete. 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 Processor -- start 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.250+0000 7f24b0f42640 1 -- start start 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 0x7f24ac1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac101320 0x7f24ac198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac110c80 con 0x7f24ac106990 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac110b00 con 0x7f24ac100980 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac110e00 con 0x7f24ac101320 2024-09-05T07:07:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57726/0 (socket says 172.21.15.86:57726) 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 -- 172.21.15.86:0/3081083062 learned_addr learned my addr 172.21.15.86:0/3081083062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24a9d9b640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac101320 0x7f24ac198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aa59c640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 0x7f24ac1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 -- 172.21.15.86:0/3081083062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac101320 msgr2=0x7f24ac198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac101320 0x7f24ac198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 -- 172.21.15.86:0/3081083062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 msgr2=0x7f24ac1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 0x7f24ac1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 -- 172.21.15.86:0/3081083062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19d2d0 con 0x7f24ac106990 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aa59c640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 0x7f24ac1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24aad9d640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f24a4033040 tx=0x7f24a4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4004040 con 0x7f24ac106990 2024-09-05T07:07:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19d560 con 0x7f24ac106990 2024-09-05T07:07:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24a4004690 con 0x7f24ac106990 2024-09-05T07:07:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4030e10 con 0x7f24ac106990 2024-09-05T07:07:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.251+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1a4e50 con 0x7f24ac106990 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.252+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac100de0 con 0x7f24ac106990 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.254+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f24a40305a0 con 0x7f24ac106990 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.255+0000 7f249b7fe640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 0x7f24940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.255+0000 7f24aa59c640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 0x7f24940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.255+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f24a402e030 con 0x7f24ac106990 2024-09-05T07:07:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.256+0000 7f24aa59c640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 0x7f24940a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f249c009ab0 tx=0x7f249c0094b0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.257+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24a404b060 con 0x7f24ac106990 2024-09-05T07:07:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.428+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ac1a50e0 con 0x7f24ac106990 2024-09-05T07:07:28.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.429+0000 7f249b7fe640 1 -- 172.21.15.86:0/3081083062 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+25533 (secure 0 0 0) 0x7f24a40bd690 con 0x7f24ac106990 2024-09-05T07:07:28.430 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:07:28.430 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":128,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T07:07:17.231327+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4062552821":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/2959599766":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/3708924769":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4040819651":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3625555544":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4279196951":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6800/3708924769":"2024-09-06T06:41:15.328251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":46,"length":1}]},{"pool":4,"snaps":[{"begin":34,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T07:07:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.431+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 msgr2=0x7f24940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.431+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 0x7f24940a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f249c009ab0 tx=0x7f249c0094b0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.431+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 msgr2=0x7f24ac19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.431+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f24a4033040 tx=0x7f24a4003970 comp rx=0 tx=0).stop 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 shutdown_connections 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f24940a2d00 0x7f24940a51c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24ac106990 0x7f24ac19cba0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24ac101320 0x7f24ac198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 --2- 172.21.15.86:0/3081083062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac100980 0x7f24ac1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 >> 172.21.15.86:0/3081083062 conn(0x7f24ac077a10 msgr2=0x7f24ac0694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 shutdown_connections 2024-09-05T07:07:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.432+0000 7f24b0f42640 1 -- 172.21.15.86:0/3081083062 wait complete. 2024-09-05T07:07:28.443 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T06:34:21.377500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T06:38:51.679225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T06:38:54.212638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T06:39:01.086734+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T07:07:28.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 -- 172.21.15.86:0/1120742107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30101350 msgr2=0x7f5b3010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 --2- 172.21.15.86:0/1120742107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30101350 0x7f5b3010d880 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009cd0 tx=0x7f5b2802a8e0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 -- 172.21.15.86:0/1120742107 shutdown_connections 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 --2- 172.21.15.86:0/1120742107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b30101350 0x7f5b3010d880 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 --2- 172.21.15.86:0/1120742107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 0x7f5b30100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 --2- 172.21.15.86:0/1120742107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b301069c0 0x7f5b30106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 -- 172.21.15.86:0/1120742107 >> 172.21.15.86:0/1120742107 conn(0x7f5b300fc690 msgr2=0x7f5b300feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 -- 172.21.15.86:0/1120742107 shutdown_connections 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.552+0000 7f5b36d91640 1 -- 172.21.15.86:0/1120742107 wait complete. 2024-09-05T07:07:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 Processor -- start 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 -- start start 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 0x7f5b30198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101350 0x7f5b301988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30111da0 con 0x7f5b301069c0 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30111c20 con 0x7f5b301009b0 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b36d91640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30111f20 con 0x7f5b30101350 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57754/0 (socket says 172.21.15.86:57754) 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 -- 172.21.15.86:0/2150233161 learned_addr learned my addr 172.21.15.86:0/2150233161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b34b2d640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 0x7f5b30198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b27fff640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101350 0x7f5b301988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 -- 172.21.15.86:0/2150233161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101350 msgr2=0x7f5b301988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101350 0x7f5b301988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 -- 172.21.15.86:0/2150233161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 msgr2=0x7f5b30198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.553+0000 7f5b3532e640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 0x7f5b30198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b3532e640 1 -- 172.21.15.86:0/2150233161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3019d380 con 0x7f5b301069c0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b3532e640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005cc0 tx=0x7f5b28003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004450 con 0x7f5b301069c0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b28031c90 con 0x7f5b301069c0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3019d610 con 0x7f5b301069c0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2803b400 con 0x7f5b301069c0 2024-09-05T07:07:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.554+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a4f00 con 0x7f5b301069c0 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.555+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b30100e10 con 0x7f5b301069c0 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.556+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b28004680 con 0x7f5b301069c0 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.557+0000 7f5b25ffb640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 0x7f5b180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.557+0000 7f5b34b2d640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 0x7f5b180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.557+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f5b2802e030 con 0x7f5b301069c0 2024-09-05T07:07:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.557+0000 7f5b34b2d640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 0x7f5b180a51c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c00b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.560+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b280bd700 con 0x7f5b301069c0 2024-09-05T07:07:28.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.727+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b30108fa0 con 0x7f5b301069c0 2024-09-05T07:07:28.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.727+0000 7f5b25ffb640 1 -- 172.21.15.86:0/2150233161 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f5b280bd000 con 0x7f5b301069c0 2024-09-05T07:07:28.728 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 msgr2=0x7f5b180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 0x7f5b180a51c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c00b390 comp rx=0 tx=0).stop 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 msgr2=0x7f5b3019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005cc0 tx=0x7f5b28003c20 comp rx=0 tx=0).stop 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 shutdown_connections 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5b180a2d00 0x7f5b180a51c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b301069c0 0x7f5b3019cc50 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101350 0x7f5b301988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 --2- 172.21.15.86:0/2150233161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b301009b0 0x7f5b30198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 >> 172.21.15.86:0/2150233161 conn(0x7f5b300fc690 msgr2=0x7f5b300fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:28.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.729+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 shutdown_connections 2024-09-05T07:07:28.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.730+0000 7f5b36d91640 1 -- 172.21.15.86:0/2150233161 wait complete. 2024-09-05T07:07:28.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T07:07:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 -- 172.21.15.86:0/2210082096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff01069a0 msgr2=0x7feff0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 --2- 172.21.15.86:0/2210082096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff01069a0 0x7feff0106d80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009930 tx=0x7fefe002a420 comp rx=0 tx=0).stop 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 -- 172.21.15.86:0/2210082096 shutdown_connections 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 --2- 172.21.15.86:0/2210082096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feff0101330 0x7feff010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 --2- 172.21.15.86:0/2210082096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0100990 0x7feff0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 --2- 172.21.15.86:0/2210082096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff01069a0 0x7feff0106d80 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 -- 172.21.15.86:0/2210082096 >> 172.21.15.86:0/2210082096 conn(0x7feff00fc6d0 msgr2=0x7feff00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 -- 172.21.15.86:0/2210082096 shutdown_connections 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.843+0000 7feff5e7b640 1 -- 172.21.15.86:0/2210082096 wait complete. 2024-09-05T07:07:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 Processor -- start 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 -- start start 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0101330 0x7feff01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feff01069a0 0x7feff019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feff0110c90 con 0x7feff0100990 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feff0110b10 con 0x7feff01069a0 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7feff5e7b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feff0110e10 con 0x7feff0101330 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57770/0 (socket says 172.21.15.86:57770) 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 -- 172.21.15.86:0/1389593400 learned_addr learned my addr 172.21.15.86:0/1389593400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefeeffd640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0101330 0x7feff01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 -- 172.21.15.86:0/1389593400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0101330 msgr2=0x7feff01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0101330 0x7feff01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 -- 172.21.15.86:0/1389593400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feff01069a0 msgr2=0x7feff019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feff01069a0 0x7feff019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.844+0000 7fefef7fe640 1 -- 172.21.15.86:0/1389593400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff019d380 con 0x7feff0100990 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7fefef7fe640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fefe0004770 tx=0x7fefe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe003f070 con 0x7feff0100990 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff019d610 con 0x7feff0100990 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefe003b460 con 0x7feff0100990 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0035070 con 0x7feff0100990 2024-09-05T07:07:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.845+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01a4f00 con 0x7feff0100990 2024-09-05T07:07:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.846+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0100df0 con 0x7feff0100990 2024-09-05T07:07:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.847+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fefe003b610 con 0x7feff0100990 2024-09-05T07:07:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.848+0000 7fefecff9640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 0x7fefd80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.848+0000 7fefeeffd640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 0x7fefd80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.848+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7fefe00eff90 con 0x7feff0100990 2024-09-05T07:07:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.848+0000 7fefeeffd640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 0x7fefd80a51c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fefe4005fd0 tx=0x7fefe4005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:28.851+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefe00bc770 con 0x7feff0100990 2024-09-05T07:07:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.019+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feff01996e0 con 0x7feff0100990 2024-09-05T07:07:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.019+0000 7fefecff9640 1 -- 172.21.15.86:0/1389593400 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7fefe00bc070 con 0x7feff0100990 2024-09-05T07:07:29.020 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 msgr2=0x7fefd80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 0x7fefd80a51c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fefe4005fd0 tx=0x7fefe4005850 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 msgr2=0x7feff0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fefe0004770 tx=0x7fefe00042f0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 shutdown_connections 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fefd80a2d00 0x7fefd80a51c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feff01069a0 0x7feff019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff0101330 0x7feff01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 --2- 172.21.15.86:0/1389593400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff0100990 0x7feff0198390 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 >> 172.21.15.86:0/1389593400 conn(0x7feff00fc6d0 msgr2=0x7feff006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 shutdown_connections 2024-09-05T07:07:29.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.021+0000 7feff5e7b640 1 -- 172.21.15.86:0/1389593400 wait complete. 2024-09-05T07:07:29.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T07:07:29.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 -- 172.21.15.86:0/2202737796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc100990 msgr2=0x7f2bdc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 --2- 172.21.15.86:0/2202737796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc100990 0x7f2bdc100df0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009930 tx=0x7f2bd002a420 comp rx=0 tx=0).stop 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 -- 172.21.15.86:0/2202737796 shutdown_connections 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 --2- 172.21.15.86:0/2202737796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc101330 0x7f2bdc10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 --2- 172.21.15.86:0/2202737796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc100990 0x7f2bdc100df0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 --2- 172.21.15.86:0/2202737796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.134+0000 7f2be1b33640 1 -- 172.21.15.86:0/2202737796 >> 172.21.15.86:0/2202737796 conn(0x7f2bdc0fc6b0 msgr2=0x7f2bdc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- 172.21.15.86:0/2202737796 shutdown_connections 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- 172.21.15.86:0/2202737796 wait complete. 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 Processor -- start 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- start start 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc100990 0x7f2bdc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 0x7f2bdc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc110c90 con 0x7f2bdc1069a0 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc110b10 con 0x7f2bdc100990 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.135+0000 7f2be1b33640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc110e10 con 0x7f2bdc101330 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57790/0 (socket says 172.21.15.86:57790) 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 -- 172.21.15.86:0/3701569337 learned_addr learned my addr 172.21.15.86:0/3701569337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdaffd640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 0x7f2bdc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdb7fe640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc100990 0x7f2bdc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 -- 172.21.15.86:0/3701569337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 msgr2=0x7f2bdc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 0x7f2bdc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 -- 172.21.15.86:0/3701569337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc100990 msgr2=0x7f2bdc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc100990 0x7f2bdc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 -- 172.21.15.86:0/3701569337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc19d360 con 0x7f2bdc1069a0 2024-09-05T07:07:29.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdaffd640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 0x7f2bdc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bdbfff640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400cad0 tx=0x7f2bd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400ae20 con 0x7f2bdc1069a0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bd4005020 con 0x7f2bdc1069a0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400f6b0 con 0x7f2bdc1069a0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc19d650 con 0x7f2bdc1069a0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.136+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a4ee0 con 0x7f2bdc1069a0 2024-09-05T07:07:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.137+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc100df0 con 0x7f2bdc1069a0 2024-09-05T07:07:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.138+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2bd40051d0 con 0x7f2bdc1069a0 2024-09-05T07:07:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.139+0000 7f2bd8ff9640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 0x7f2bc40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.139+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f2bd40cfd30 con 0x7f2bdc1069a0 2024-09-05T07:07:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.139+0000 7f2bdb7fe640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 0x7f2bc40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.140+0000 7f2bdb7fe640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 0x7f2bc40a51c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009a20 tx=0x7f2bcc005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.142+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bd4014030 con 0x7f2bdc1069a0 2024-09-05T07:07:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.305+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bdc02d7c0 con 0x7f2bdc1069a0 2024-09-05T07:07:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.305+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/3701569337 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f2bd409c510 con 0x7f2bdc1069a0 2024-09-05T07:07:29.306 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 msgr2=0x7f2bc40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 0x7f2bc40a51c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009a20 tx=0x7f2bcc005f90 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 msgr2=0x7f2bdc19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400cad0 tx=0x7f2bd400cf90 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 shutdown_connections 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f2bc40a2d00 0x7f2bc40a51c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bdc1069a0 0x7f2bdc19cc30 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bdc101330 0x7f2bdc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 --2- 172.21.15.86:0/3701569337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bdc100990 0x7f2bdc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 >> 172.21.15.86:0/3701569337 conn(0x7f2bdc0fc6b0 msgr2=0x7f2bdc06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.307+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 shutdown_connections 2024-09-05T07:07:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.308+0000 7f2be1b33640 1 -- 172.21.15.86:0/3701569337 wait complete. 2024-09-05T07:07:29.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T07:07:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 -- 172.21.15.86:0/2283647717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e4101350 msgr2=0x7f14e410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 --2- 172.21.15.86:0/2283647717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e4101350 0x7f14e410d880 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009cd0 tx=0x7f14dc02a8e0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 -- 172.21.15.86:0/2283647717 shutdown_connections 2024-09-05T07:07:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 --2- 172.21.15.86:0/2283647717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e4101350 0x7f14e410d880 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 --2- 172.21.15.86:0/2283647717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e41009b0 0x7f14e4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 --2- 172.21.15.86:0/2283647717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e41069c0 0x7f14e4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 -- 172.21.15.86:0/2283647717 >> 172.21.15.86:0/2283647717 conn(0x7f14e40fc690 msgr2=0x7f14e40feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 -- 172.21.15.86:0/2283647717 shutdown_connections 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.421+0000 7f14eb217640 1 -- 172.21.15.86:0/2283647717 wait complete. 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 Processor -- start 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 -- start start 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4101350 0x7f14e41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e41069c0 0x7f14e419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4110d00 con 0x7f14e41009b0 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4110b80 con 0x7f14e4101350 2024-09-05T07:07:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14eb217640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4110e80 con 0x7f14e41069c0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57796/0 (socket says 172.21.15.86:57796) 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 -- 172.21.15.86:0/1014742748 learned_addr learned my addr 172.21.15.86:0/1014742748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 -- 172.21.15.86:0/1014742748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e41069c0 msgr2=0x7f14e419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e41069c0 0x7f14e419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14dbfff640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4101350 0x7f14e41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 -- 172.21.15.86:0/1014742748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4101350 msgr2=0x7f14e41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4101350 0x7f14e41977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.422+0000 7f14e8fb3640 1 -- 172.21.15.86:0/1014742748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e419c2a0 con 0x7f14e41009b0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14e8fb3640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c920 tx=0x7f14e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007c70 con 0x7f14e41009b0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14e0007e10 con 0x7f14e41009b0 2024-09-05T07:07:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e419c590 con 0x7f14e41009b0 2024-09-05T07:07:29.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0018820 con 0x7f14e41009b0 2024-09-05T07:07:29.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.423+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e41a3e20 con 0x7f14e41009b0 2024-09-05T07:07:29.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.424+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e4100e10 con 0x7f14e41009b0 2024-09-05T07:07:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.425+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14e00189c0 con 0x7f14e41009b0 2024-09-05T07:07:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.425+0000 7f14d9ffb640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 0x7f14cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.426+0000 7f14dbfff640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 0x7f14cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.426+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f14e00cf9f0 con 0x7f14e41009b0 2024-09-05T07:07:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.426+0000 7f14dbfff640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 0x7f14cc0a51c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f14e40fd5e0 tx=0x7f14d4005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.429+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14e0014030 con 0x7f14e41009b0 2024-09-05T07:07:29.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.595+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f14e4108f40 con 0x7f14e41009b0 2024-09-05T07:07:29.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.595+0000 7f14d9ffb640 1 -- 172.21.15.86:0/1014742748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v128) ==== 103+0+11 (secure 0 0 0) 0x7f14e000de20 con 0x7f14e41009b0 2024-09-05T07:07:29.596 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 msgr2=0x7f14cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 0x7f14cc0a51c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f14e40fd5e0 tx=0x7f14d4005f90 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 msgr2=0x7f14e41972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c920 tx=0x7f14e000cde0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 shutdown_connections 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f14cc0a2d00 0x7f14cc0a51c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e41069c0 0x7f14e419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4101350 0x7f14e41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 --2- 172.21.15.86:0/1014742748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e41009b0 0x7f14e41972b0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 >> 172.21.15.86:0/1014742748 conn(0x7f14e40fc690 msgr2=0x7f14e40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 shutdown_connections 2024-09-05T07:07:29.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.597+0000 7f14eb217640 1 -- 172.21.15.86:0/1014742748 wait complete. 2024-09-05T07:07:29.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:29 smithi086 ceph-mon[30943]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-09-05T07:07:29.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3081083062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:07:29.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2150233161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1389593400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:29 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3701569337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-05T07:07:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:29 smithi115 ceph-mon[34791]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-09-05T07:07:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3081083062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:07:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2150233161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1389593400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:29 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3701569337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 -- 172.21.15.86:0/3486538124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c106e60 msgr2=0x7fda1c10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 --2- 172.21.15.86:0/3486538124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c106e60 0x7fda1c10e9e0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fda14009ca0 tx=0x7fda1402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 -- 172.21.15.86:0/3486538124 shutdown_connections 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 --2- 172.21.15.86:0/3486538124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c106e60 0x7fda1c10e9e0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 --2- 172.21.15.86:0/3486538124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c100c70 0x7fda1c106730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 --2- 172.21.15.86:0/3486538124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c1002c0 0x7fda1c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 -- 172.21.15.86:0/3486538124 >> 172.21.15.86:0/3486538124 conn(0x7fda1c0771a0 msgr2=0x7fda1c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 -- 172.21.15.86:0/3486538124 shutdown_connections 2024-09-05T07:07:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.707+0000 7fda22959640 1 -- 172.21.15.86:0/3486538124 wait complete. 2024-09-05T07:07:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 Processor -- start 2024-09-05T07:07:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 -- start start 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 0x7fda1c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c19cc20 0x7fda1c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c111dd0 con 0x7fda1c1002c0 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c111c50 con 0x7fda1c106e60 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda22959640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c111f50 con 0x7fda1c19cc20 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57820/0 (socket says 172.21.15.86:57820) 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda1bfff640 1 -- 172.21.15.86:0/2568834722 learned_addr learned my addr 172.21.15.86:0/2568834722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda1b7fe640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 0x7fda1c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda1bfff640 1 -- 172.21.15.86:0/2568834722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c19cc20 msgr2=0x7fda1c19d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.708+0000 7fda20ef6640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c19cc20 0x7fda1c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1bfff640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c19cc20 0x7fda1c19d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1bfff640 1 -- 172.21.15.86:0/2568834722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 msgr2=0x7fda1c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1bfff640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 0x7fda1c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1bfff640 1 -- 172.21.15.86:0/2568834722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c19d710 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1b7fe640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 0x7fda1c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda1bfff640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fda0400b4a0 tx=0x7fda0400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04014070 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c19da00 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda04010ac0 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04017c30 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.709+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a51e0 con 0x7fda1c1002c0 2024-09-05T07:07:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.710+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c1006a0 con 0x7fda1c1002c0 2024-09-05T07:07:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.711+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda04017070 con 0x7fda1c1002c0 2024-09-05T07:07:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.712+0000 7fda197fa640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 0x7fda080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.712+0000 7fda1b7fe640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 0x7fda080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.712+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7fda040ceef0 con 0x7fda1c1002c0 2024-09-05T07:07:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.712+0000 7fda1b7fe640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 0x7fda080a51c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c005ea0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.716+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda0409b7e0 con 0x7fda1c1002c0 2024-09-05T07:07:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:29 smithi096 ceph-mon[34468]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-09-05T07:07:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3081083062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:07:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2150233161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1389593400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:29 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3701569337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T07:07:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.928+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fda1c100c70 con 0x7fda1c1002c0 2024-09-05T07:07:29.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.936+0000 7fda197fa640 1 -- 172.21.15.86:0/2568834722 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740898 (secure 0 0 0) 0x7fda0409b0e0 con 0x7fda1c1002c0 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 25, 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 15300, 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 388818328 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 388818328 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/388818328", 2024-09-05T07:07:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-09-05T06:41:15.328490+0000", 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540701547738038271, 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24853, 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:29.948 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:29.950 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:29.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:29.953 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-09-05T07:07:29.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-09-05T07:07:29.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-09-05T07:07:29.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-09-05T07:07:29.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.967 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:29.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:29.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.975 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:29.976 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.979 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-09-05T07:07:29.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:29.985 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T07:07:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T07:07:29.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T07:07:29.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-09-05T07:07:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-09-05T07:07:29.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-09-05T07:07:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-09-05T07:07:29.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-09-05T07:07:30.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-09-05T07:07:30.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-09-05T07:07:30.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-09-05T07:07:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T07:07:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T07:07:30.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T07:07:30.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-09-05T07:07:30.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.018 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-09-05T07:07:30.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.026 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.027 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-09-05T07:07:30.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T07:07:30.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-09-05T07:07:30.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-09-05T07:07:30.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.044 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-09-05T07:07:30.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-09-05T07:07:30.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T07:07:30.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-09-05T07:07:30.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T07:07:30.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-09-05T07:07:30.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.061 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T07:07:30.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-09-05T07:07:30.066 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.067 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.074 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T07:07:30.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T07:07:30.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T07:07:30.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.099 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T07:07:30.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T07:07:30.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.109 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T07:07:30.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T07:07:30.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T07:07:30.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T07:07:30.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.121 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T07:07:30.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.135 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.138 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-09-05T07:07:30.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.141 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.155 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-09-05T07:07:30.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-09-05T07:07:30.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.160 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-09-05T07:07:30.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.163 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-09-05T07:07:30.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.175 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T07:07:30.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.179 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.182 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:30.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-09-05T07:07:30.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.188 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.189 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.194 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-09-05T07:07:30.196 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.197 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-09-05T07:07:30.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-09-05T07:07:30.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.205 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.210 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.212 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-09-05T07:07:30.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-09-05T07:07:30.222 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.223 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.231 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.234 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:30.239 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-09-05T07:07:30.247 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.248 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.254 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.255 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.261 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.268 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T07:07:30.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.277 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.286 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.292 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.296 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.297 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.302 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.314 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.322 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-09-05T07:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-09-05T07:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T07:07:30.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-09-05T07:07:30.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.361 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.366 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.367 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.368 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.370 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.371 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.373 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.376 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.377 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.379 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-09-05T07:07:30.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.385 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-09-05T07:07:30.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.390 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.396 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-09-05T07:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.417 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-09-05T07:07:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.429 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.431 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-09-05T07:07:30.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-09-05T07:07:30.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.435 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24896, 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-09-05T07:07:30.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.444 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-09-05T07:07:30.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-09-05T07:07:30.449 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.453 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T07:07:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:30.456 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T07:07:30.458 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-09-05T07:07:30.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.466 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-09-05T07:07:30.469 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-09-05T07:07:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-09-05T07:07:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-09-05T07:07:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-09-05T07:07:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-09-05T07:07:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-09-05T07:07:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-09-05T07:07:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T07:07:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-09-05T07:07:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-09-05T07:07:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-09-05T07:07:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-09-05T07:07:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-09-05T07:07:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-09-05T07:07:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-09-05T07:07:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-09-05T07:07:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-09-05T07:07:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T07:07:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T07:07:30.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.568 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.571 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T07:07:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T07:07:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T07:07:30.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.590 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T07:07:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-09-05T07:07:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-09-05T07:07:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-09-05T07:07:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-09-05T07:07:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-09-05T07:07:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-09-05T07:07:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-09-05T07:07:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-09-05T07:07:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.657 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-09-05T07:07:30.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.677 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-09-05T07:07:30.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.680 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.681 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-09-05T07:07:30.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.694 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-09-05T07:07:30.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T07:07:30.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.706 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.712 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.714 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-09-05T07:07:30.724 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.725 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.726 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.729 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-09-05T07:07:30.737 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.738 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.740 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-09-05T07:07:30.745 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.751 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-09-05T07:07:30.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.761 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-09-05T07:07:30.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.769 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.773 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.783 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.784 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.795 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-09-05T07:07:30.797 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.798 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.800 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.801 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-09-05T07:07:30.804 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.805 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-09-05T07:07:30.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-09-05T07:07:30.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-09-05T07:07:30.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-09-05T07:07:30.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-09-05T07:07:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-09-05T07:07:30.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.826 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.832 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.840 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.848 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-09-05T07:07:30.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-09-05T07:07:30.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T07:07:30.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-09-05T07:07:30.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.868 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.869 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-09-05T07:07:30.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-09-05T07:07:30.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.878 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.883 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.884 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-09-05T07:07:30.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T07:07:30.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-09-05T07:07:30.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.898 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.901 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.906 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "restful", 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_schedule" 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.910 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-09-05T07:07:30.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-09-05T07:07:30.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-09-05T07:07:30.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T07:07:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-09-05T07:07:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T07:07:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-09-05T07:07:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-09-05T07:07:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-09-05T07:07:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-09-05T07:07:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-09-05T07:07:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-09-05T07:07:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-09-05T07:07:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-09-05T07:07:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-09-05T07:07:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T07:07:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T07:07:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T07:07:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T07:07:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T07:07:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T07:07:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T07:07:31.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.051 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.059 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T07:07:31.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T07:07:31.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.063 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:31.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.071 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-09-05T07:07:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.074 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.077 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.080 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-09-05T07:07:31.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-09-05T07:07:31.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.084 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.085 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-09-05T07:07:31.089 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:31.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.092 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.094 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.096 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.097 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-09-05T07:07:31.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-09-05T07:07:31.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.101 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.108 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.110 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-09-05T07:07:31.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-09-05T07:07:31.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:31.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.121 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T07:07:31.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.133 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.139 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.143 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-09-05T07:07:31.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.149 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.154 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.156 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-09-05T07:07:31.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-09-05T07:07:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-09-05T07:07:31.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.164 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.167 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-09-05T07:07:31.169 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.170 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.173 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.175 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.176 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-09-05T07:07:31.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.188 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.190 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.193 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-09-05T07:07:31.195 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.196 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.199 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.201 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.205 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.206 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:31.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-09-05T07:07:31.209 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T07:07:31.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.212 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.213 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.214 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.215 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.225 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-09-05T07:07:31.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-09-05T07:07:31.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-09-05T07:07:31.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.232 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.236 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.237 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.238 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.239 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-09-05T07:07:31.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-09-05T07:07:31.244 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.247 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.255 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-09-05T07:07:31.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.273 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.274 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.285 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.288 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.294 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.298 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.303 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.305 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.307 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.313 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-09-05T07:07:31.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.322 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.333 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-09-05T07:07:31.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.336 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.342 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.348 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-09-05T07:07:31.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-09-05T07:07:31.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.359 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-09-05T07:07:31.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-09-05T07:07:31.367 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-09-05T07:07:31.368 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-09-05T07:07:31.369 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-09-05T07:07:31.370 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "squid": [ 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-09-05T07:07:31.371 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 90, 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 292655976 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3037773313 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-09-05T07:07:31.373 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 444077638 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.374 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3469038062 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3476778252 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.375 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1385174100 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-09-05T07:07:31.376 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:31.378 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:30 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1014742748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T07:07:31.378 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:30 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2568834722' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T07:07:31.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:30 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1014742748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T07:07:31.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:30 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2568834722' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.940+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 msgr2=0x7fda080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.940+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 0x7fda080a51c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c005ea0 comp rx=0 tx=0).stop 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.940+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 msgr2=0x7fda1c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.940+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fda0400b4a0 tx=0x7fda0400b960 comp rx=0 tx=0).stop 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 shutdown_connections 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fda080a2d00 0x7fda080a51c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda1c19cc20 0x7fda1c19d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda1c106e60 0x7fda1c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:31.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 --2- 172.21.15.86:0/2568834722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1c1002c0 0x7fda1c198360 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:31.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 >> 172.21.15.86:0/2568834722 conn(0x7fda1c0771a0 msgr2=0x7fda1c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:31.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 shutdown_connections 2024-09-05T07:07:31.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:29.941+0000 7fda22959640 1 -- 172.21.15.86:0/2568834722 wait complete. 2024-09-05T07:07:31.387 DEBUG:tasks.check_counter:Getting stats from a 2024-09-05T07:07:31.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-05T07:07:31.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:30 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1014742748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T07:07:31.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:30 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2568834722' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T07:07:31.647 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:31.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:31 smithi115 ceph-mon[34791]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:31.708 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:31 smithi096 ceph-mon[34468]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:31.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:31 smithi086 ceph-mon[30943]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:32.224 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:32.224 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3112, 2024-09-05T07:07:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 3114, 2024-09-05T07:07:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 56444275, 2024-09-05T07:07:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 989617, 2024-09-05T07:07:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.290450212, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.081732677, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.105256959, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.081424692, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3112, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.093210758, 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000029952 2024-09-05T07:07:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2346, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2311, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 98650987, 2024-09-05T07:07:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 866346, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.274374549, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.069673091, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.155751977, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.050667058, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2311, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.072340204, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031302 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:32.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1724656, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 292288 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 619, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 968, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 58884180, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 974942, 2024-09-05T07:07:32.229 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.108367837, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.031885398, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.054407246, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.015051342, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 965, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.041148240, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000042640 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 778736 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-09-05T07:07:32.231 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9182, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.210401843, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000022914 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-09-05T07:07:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-09-05T07:07:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 8186, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 8186, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 8186, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-09-05T07:07:32.235 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-09-05T07:07:32.236 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-09-05T07:07:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:32.239 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-09-05T07:07:32.240 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 45555, 2024-09-05T07:07:32.241 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 72, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 5, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 214887804, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 214880986, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.242 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 45555 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1399, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 8194, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 6795, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 59, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 86, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 27, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 8186, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 8543, 2024-09-05T07:07:32.243 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 357, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 63924, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215328 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-09-05T07:07:32.244 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.246 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.248 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-09-05T07:07:32.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:32.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.255 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.256 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:32.257 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-09-05T07:07:32.258 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2667, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:32.259 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:32.260 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-09-05T07:07:32.261 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 240987, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 83, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:32.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 200, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 9067326, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 41095, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:32.263 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 5534, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 5534, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5534, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4.212909188, 2024-09-05T07:07:32.264 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000761277 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5534, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5535 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 5534, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 5534, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 3264, 2024-09-05T07:07:32.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 2268, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-09-05T07:07:32.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-09-05T07:07:32.267 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-09-05T07:07:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 128, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 41, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 13, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 2, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-09-05T07:07:32.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.270 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:32.271 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 6073, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 213402507, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 6073, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 6073, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 213402507, 2024-09-05T07:07:32.272 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:32.273 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5534, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 210709500, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5534, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 210709500, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:32.274 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5534, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 5534, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5534, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 5534, 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:32.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.276 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.276 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-09-05T07:07:32.277 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x561ed47eab40": { 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:32.278 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:32.279 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:33.228 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-05T07:07:33.228 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-05T07:07:33.228 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-05T07:07:33.229 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:33.229 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:33.229 DEBUG:tasks.check_counter:Getting stats from d 2024-09-05T07:07:33.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-05T07:07:33.474 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:33.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:33 smithi086 ceph-mon[30943]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:33.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:33 smithi115 ceph-mon[34791]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:33.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:33 smithi096 ceph-mon[34468]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001158187, 2024-09-05T07:07:33.975 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000224028, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.976 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 473, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 463, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1514944, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 286992, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.060845815, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.027215504, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.045694240, 2024-09-05T07:07:33.977 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 462, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.018332227, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000039680 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1514944, 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 286992 2024-09-05T07:07:33.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 7, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 358, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3424, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 298928, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.025704209, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.017924268, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.004100032, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 356, 2024-09-05T07:07:33.979 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.024942008, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000070061 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 298928 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.980 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:33.981 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:33.982 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:33.983 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148714, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:33.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:33.985 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 476, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1448121, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 476, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 476, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1448121, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:33.986 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:33.987 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:33.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:33.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:34.650 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-05T07:07:34.650 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-05T07:07:34.650 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-05T07:07:34.650 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-05T07:07:34.650 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-05T07:07:34.651 DEBUG:tasks.check_counter:Getting stats from g 2024-09-05T07:07:34.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-05T07:07:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:35.428 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2683, 2024-09-05T07:07:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2648, 2024-09-05T07:07:35.428 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 52737768, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 974088, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.288034230, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.106391245, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.122655216, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.056328109, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2648, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.098268180, 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000037110 2024-09-05T07:07:35.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1725200, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 293568 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2539, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2889, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 80337382, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1587351, 2024-09-05T07:07:35.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.239051384, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.057868948, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.110014592, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.056522383, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2886, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.080614992, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000027933 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:35.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 780528 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 893, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 895, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 87964349, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 283986, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:35.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.126592181, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.018335337, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.081881345, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.020993274, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 893, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.026625526, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000029815 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9380, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.205682203, 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000021927 2024-09-05T07:07:35.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 8477, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 8477, 2024-09-05T07:07:35.436 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 8477, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-09-05T07:07:35.437 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-09-05T07:07:35.438 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-09-05T07:07:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-09-05T07:07:35.440 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-09-05T07:07:35.441 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-09-05T07:07:35.442 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 49208, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 74, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 6, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 221953027, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 221945924, 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-09-05T07:07:35.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 49208 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1430, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 8632, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 7202, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 59, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 96, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 37, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 8477, 2024-09-05T07:07:35.444 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 8986, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 509, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 63880, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215328 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-09-05T07:07:35.445 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.447 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:35.448 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.450 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.451 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:35.452 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.453 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.455 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.456 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:35.457 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 3, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 3, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-09-05T07:07:35.458 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 3222, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:35.459 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:35.460 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:35.461 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 240935, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 80, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 200, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:35.462 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 9347798, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 42408, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 5570, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-09-05T07:07:35.463 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 5570, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5570, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4.270908126, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000766769 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5570, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5571 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 5570, 2024-09-05T07:07:35.464 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 5570, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 3287, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 2281, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-09-05T07:07:35.465 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-09-05T07:07:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-09-05T07:07:35.467 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 128, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 41, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-09-05T07:07:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 14, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 3, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:35.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 6111, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 220459040, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 6111, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 6111, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 220459040, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5570, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 217759391, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5570, 2024-09-05T07:07:35.472 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 217759391, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:35.473 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5570, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 5570, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5570, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 5570, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:35.474 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5608cda3eb40": { 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-09-05T07:07:35.476 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:35.477 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:35.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:35 smithi115 ceph-mon[34791]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:35.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:35 smithi086 ceph-mon[30943]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:35.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:35 smithi096 ceph-mon[34468]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:36.418 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-05T07:07:36.418 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-05T07:07:36.418 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-05T07:07:36.418 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:36.418 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:36.418 DEBUG:tasks.check_counter:Getting stats from j 2024-09-05T07:07:36.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-05T07:07:36.666 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-09-05T07:07:37.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001180491, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000196568, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:37.170 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 358, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 480, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 298832, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.024102007, 2024-09-05T07:07:37.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.018525367, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002189500, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 355, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.022440964, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000063213 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:37.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 298832 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 479, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 467, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1518432, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 288720, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.068147026, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.028529967, 2024-09-05T07:07:37.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.051641001, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 467, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.026140778, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000055975 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1518432, 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 288720 2024-09-05T07:07:37.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:37.175 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:37.176 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:37.177 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148714, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:37.178 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-09-05T07:07:37.179 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 478, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1448381, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 478, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 478, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1448381, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T07:07:37.180 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T07:07:37.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:37 smithi115 ceph-mon[34791]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:37.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:37 smithi086 ceph-mon[30943]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:37.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:37 smithi096 ceph-mon[34468]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:37.895 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-05T07:07:37.896 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-05T07:07:37.896 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-05T07:07:37.896 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-05T07:07:37.896 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-05T07:07:37.896 DEBUG:tasks.check_counter:Getting stats from b 2024-09-05T07:07:37.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-05T07:07:38.226 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 37038, 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 43385, 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 10954202, 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 165082094, 2024-09-05T07:07:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 342, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 9, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 2.921997570, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 1.003682646, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 1.203028101, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.708317719, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 43376, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.556015490, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000035872 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 46606, 2024-09-05T07:07:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008818762, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000189 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 932544, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 310464 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 919679, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 957207, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 267502233, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 710891366, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 25, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 39.762646043, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 16.291841841, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 28.437968533, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.693548702, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 957207, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 15.957260721, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000016670 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1821261, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.240241489, 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000131 2024-09-05T07:07:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 135792 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 121283, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 111733, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 50894112, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 115924150, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 11, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 6.269678876, 2024-09-05T07:07:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 2.624517414, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 4.181637488, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.329027193, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 111736, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.037120419, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000027181 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 207821, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.031024270, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000149 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1788912, 2024-09-05T07:07:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 1008800 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 61544, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 33.160301246, 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000538806 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 10929, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.906102225, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000082908 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 793313, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 757264, 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-05T07:07:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 757264, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 691.143797489, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000912685 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 36049, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 6, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 21, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 1, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 2074, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 267, 2024-09-05T07:07:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 1740, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 67, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 1693, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 2167, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 3, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 3, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 6, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 1531205, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 1520143, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 9063, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 148, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 1, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 1, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 31, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 1867, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 65, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 1907, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 1897, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 234, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 7, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 1897, 2024-09-05T07:07:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 850, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 30, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 820, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 1047, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 13, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 35285, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 20132, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 42329, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 5503, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 488, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 17639 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 38, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 63, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 42, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 170, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 139, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 33, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 4672, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 4639, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 24, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 1825, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 30, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 39, 2024-09-05T07:07:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 1, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 50829, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 50829, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-05T07:07:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 101, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 101, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 6, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 379200326, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 30761, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 90.767309875, 2024-09-05T07:07:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002950726 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 1685, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 8030, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 6345, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 93, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 320, 2024-09-05T07:07:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 227, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 2074, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 9241, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 7167, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 3, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 5517, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 5514, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 76780, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231712 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 825354, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 20571, 2024-09-05T07:07:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 793313, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 635, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 41633, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5311, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 141.524381665, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.026647407 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 713489, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 237.898656354, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000333430 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 22176, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 132.678074643, 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005982957 2024-09-05T07:07:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 26, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.007284812, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000280185 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 119, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.340941978, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002865058 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 26, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.491509924, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.057365766 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2351, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.467818151, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001049688 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7906, 2024-09-05T07:07:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 34.424029690, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004354165 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 946, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 98.379108786, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.103994829 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 44, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.348170520, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007912966 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 19, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.160167746, 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.008429881 2024-09-05T07:07:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 32.245032598, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 8.061258149 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 200, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.071101626, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000355508 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.018945401, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002368175 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4639, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 9.088573595, 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001959166 2024-09-05T07:07:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-09-05T07:07:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 37271, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 18635, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1558, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-05T07:07:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 576009, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 178, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 200, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 5672467, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 24769, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 45544, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 376039427, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 36, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 45544, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 45544, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 165.565108137, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003635278 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 45544, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 46648 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 45544, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 2802, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 42742, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 11, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 864, 2024-09-05T07:07:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 31595, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 553, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 1, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 9879, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1927, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 22, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-05T07:07:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 128, 2024-09-05T07:07:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 49, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 1248, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 17, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 531 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 1501 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 5, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 9278, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 4639, 2024-09-05T07:07:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 1077996, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 232196227, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 1077996, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1077996, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 232196227, 2024-09-05T07:07:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 45544, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 376549545, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 45544, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 376549545, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 45544, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 45544, 2024-09-05T07:07:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 45544, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 45544, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 4639, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 510363, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 4639, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 834, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 510363, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x5653e1088960": { 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 50829, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 375006022, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 50829, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 30761, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 375006022, 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:38.893 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-05T07:07:39.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:39 smithi115 ceph-mon[34791]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:39.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:39 smithi086 ceph-mon[30943]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:39.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:39 smithi096 ceph-mon[34468]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:39.888 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-09-05T07:07:39.888 INFO:tasks.check_counter:Daemon mds.b mds.imported=65 2024-09-05T07:07:39.888 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-05T07:07:39.888 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-09-05T07:07:39.888 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-09-05T07:07:39.889 DEBUG:tasks.check_counter:Getting stats from e 2024-09-05T07:07:39.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-05T07:07:40.143 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T07:07:40.736 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-05T07:07:40.737 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:40.737 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 3738, 2024-09-05T07:07:40.737 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 3740, 2024-09-05T07:07:40.737 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 152827667, 2024-09-05T07:07:40.737 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1188691, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 8, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.358993305, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.093006415, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.156982567, 2024-09-05T07:07:40.738 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.085079992, 2024-09-05T07:07:40.739 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:40.739 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3738, 2024-09-05T07:07:40.739 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.105395805, 2024-09-05T07:07:40.739 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028195 2024-09-05T07:07:40.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:40.741 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:40.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.741 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:40.741 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1758, 2024-09-05T07:07:40.741 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1720, 2024-09-05T07:07:40.742 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 98533832, 2024-09-05T07:07:40.742 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 680522, 2024-09-05T07:07:40.742 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-05T07:07:40.742 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-09-05T07:07:40.742 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.204259999, 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.054119801, 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.132185368, 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.027077777, 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1719, 2024-09-05T07:07:40.743 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.050426572, 2024-09-05T07:07:40.744 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000029334 2024-09-05T07:07:40.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.744 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:40.744 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.744 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1748768, 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 295680 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.745 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1217, 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1568, 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 121946424, 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1168218, 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 9, 2024-09-05T07:07:40.746 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:40.747 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.169717665, 2024-09-05T07:07:40.747 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.035978307, 2024-09-05T07:07:40.747 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.097320510, 2024-09-05T07:07:40.747 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.026675080, 2024-09-05T07:07:40.747 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:40.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1566, 2024-09-05T07:07:40.748 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.054201411, 2024-09-05T07:07:40.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000034611 2024-09-05T07:07:40.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.748 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-05T07:07:40.749 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 783120 2024-09-05T07:07:40.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.750 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-05T07:07:40.750 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-05T07:07:40.750 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-05T07:07:40.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 10385, 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.208332703, 2024-09-05T07:07:40.751 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000020060 2024-09-05T07:07:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-05T07:07:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-05T07:07:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-05T07:07:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-05T07:07:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-05T07:07:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-05T07:07:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-05T07:07:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 12204, 2024-09-05T07:07:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 173, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 12031, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 11956, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-05T07:07:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-05T07:07:40.758 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:40.759 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.760 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:40.761 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-05T07:07:40.762 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 80606, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 131, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 8, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194352, 2024-09-05T07:07:40.763 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 374035500, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 374028882, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 80606 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 2330, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 12435, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 10105, 2024-09-05T07:07:40.764 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 96, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 164, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 68, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 12204, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 12787, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 583, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 69804, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215328 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-05T07:07:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-05T07:07:40.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:40.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.775 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 3, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 5742, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:40.778 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:40.779 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:40.780 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 241771, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 85, 2024-09-05T07:07:40.781 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:40.782 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:40.782 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:40.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 200, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 14043983, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 60676, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:40.784 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 6159, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-05T07:07:40.785 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 6159, 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6159, 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 4.831886318, 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000784524 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-05T07:07:40.786 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6159, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 6160 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 6159, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 6159, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-05T07:07:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 3423, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 2734, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-05T07:07:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-05T07:07:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-05T07:07:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-05T07:07:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-05T07:07:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-05T07:07:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 128, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 41, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 11, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 1, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-05T07:07:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-05T07:07:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:40.796 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 6709, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 372673209, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 6709, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 6709, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 372673209, 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:40.797 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:40.798 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 6159, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 369850797, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 6159, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 369850797, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 6159, 2024-09-05T07:07:40.800 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 6159, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 6159, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 6159, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:40.801 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x558e1ee2eb40": { 2024-09-05T07:07:40.802 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:40.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:40.804 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-05T07:07:41.659 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:41 smithi096 ceph-mon[34468]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:41.661 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-05T07:07:41.661 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-05T07:07:41.661 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-05T07:07:41.661 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:41.661 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:41.661 DEBUG:tasks.check_counter:Getting stats from h 2024-09-05T07:07:41.662 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-05T07:07:41.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:41 smithi115 ceph-mon[34791]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:41.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:41 smithi086 ceph-mon[30943]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:41.908 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T07:07:42.516 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 16617, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 16632, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 6002884, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 65716308, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 69, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 8, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 1.462599683, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.491158615, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.360014282, 2024-09-05T07:07:42.517 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.555903572, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 16631, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.674556518, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000040560 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2079, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000521364, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000250 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.518 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1767392, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 345344 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 60505, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 50327, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 20867747, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 166350798, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 19, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 3.362612929, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 1.142808231, 2024-09-05T07:07:42.519 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 1.571684527, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.807507141, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 50324, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.695328651, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000033688 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 60700, 2024-09-05T07:07:42.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009332431, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000153 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3888, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 878528 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 454216, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 509349, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 139536109, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 378552129, 2024-09-05T07:07:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 14, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 23.930643269, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 9.921205316, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 16.518116910, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.723121008, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 509348, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 10.942573388, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000021483 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 912199, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.148290089, 2024-09-05T07:07:42.522 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000162 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-05T07:07:42.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 53297, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 31.146739673, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000584399 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:42.524 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 23436, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.576538407, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000067269 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 311347, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 310801, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 310801, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 636.252682124, 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002047138 2024-09-05T07:07:42.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 546, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 2, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 56, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 5, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 1862, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 235, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 1526, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 101, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 1524, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 981, 2024-09-05T07:07:42.526 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 1, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 1, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 6, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 608445, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 574102, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 540, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 77, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 1, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 18, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 702, 2024-09-05T07:07:42.527 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 11, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 372, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 2771, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 685, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 216093760, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 7, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 2771, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 260, 2024-09-05T07:07:42.528 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 260, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 2511, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 2, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 32148, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 45743, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 51325, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 15868, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 883, 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 39999 2024-09-05T07:07:42.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 4, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 41, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 27, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 97, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 57, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 20, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 10544, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 10524, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-05T07:07:42.530 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 24, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 3039, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 18, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-05T07:07:42.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 5, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 80854, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 80854, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 111, 2024-09-05T07:07:42.532 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 111, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 8, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 292851313, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 25861, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 82.235270000, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003179895 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.533 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 1571, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 12883, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 11312, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 81, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 159, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 78, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 1862, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 15485, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 13623, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 1, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 10614, 2024-09-05T07:07:42.534 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 10613, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 78772, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223520 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 362040, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 327, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 311347, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 1015, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 11243, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 10332, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 249.810112574, 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.024178291 2024-09-05T07:07:42.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 250042, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 79.416293981, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000317611 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7308, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 6.660471527, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000911394 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.537 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:42.538 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.539 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.540 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.540 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.541 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-05T07:07:42.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6751, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 4.123578606, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000610810 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 22531, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 61.498027817, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002729485 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2863, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 32.208619301, 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.011249954 2024-09-05T07:07:42.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-05T07:07:42.544 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 20, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.125254040, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006262702 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.545 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 34.083210324, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 17.041605162 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:42.546 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 428, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.132946997, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000310623 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 10524, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 168.194166957, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.015981961 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:42.548 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 4, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-09-05T07:07:42.549 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 34652, 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 17326, 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1562, 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:42.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:42.551 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:42.552 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:42.552 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:42.552 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:42.552 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:42.552 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:42.553 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 473299, 2024-09-05T07:07:42.554 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 157, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 200, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:42.555 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 5191891, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 22800, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 55012, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 291246187, 2024-09-05T07:07:42.556 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 28, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 55012, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 55012, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 186.558614257, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003391234 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 55012, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 56860 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 55012, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 4390, 2024-09-05T07:07:42.557 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 50622, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1609, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 27470, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 938, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 5, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 21128, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-05T07:07:42.558 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 2771, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:42.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-05T07:07:42.560 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-05T07:07:42.561 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 128, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 41, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-05T07:07:42.562 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 1969, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 12, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 928 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-05T07:07:42.563 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 2507 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 5, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 21048, 2024-09-05T07:07:42.564 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 10524, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 531334, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 118550908, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 531334, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 531334, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 118550908, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:42.566 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 55012, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 292405521, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 55012, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 292405521, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.567 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 55012, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 55012, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 55012, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 55012, 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:42.568 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 10524, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1159596, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 10524, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:42.569 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1609, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1159596, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x56083dbb8960": { 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:42.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 80854, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 288657009, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 80854, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 25861, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 288657009, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:42.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:42.572 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-05T07:07:43.405 INFO:tasks.check_counter:Daemon mds.h mds.exported=18 2024-09-05T07:07:43.406 INFO:tasks.check_counter:Daemon mds.h mds.imported=11 2024-09-05T07:07:43.406 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-05T07:07:43.406 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:43.406 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:43.406 DEBUG:tasks.check_counter:Getting stats from k 2024-09-05T07:07:43.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-05T07:07:43.655 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T07:07:43.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:43 smithi115 ceph-mon[34791]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:43.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:43 smithi086 ceph-mon[30943]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:43.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:43 smithi096 ceph-mon[34468]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 340072, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 384370, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 102443868, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 317718619, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 17, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-05T07:07:44.209 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 17.678962513, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 7.012572028, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 12.323741167, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.680202787, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 384369, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 8.561606840, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000022274 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 689039, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.106697112, 2024-09-05T07:07:44.210 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000154 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 27491, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 20755, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 16081663, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 99159479, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 50, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:44.211 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 1.741490699, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.611732367, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.677214315, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.497298804, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 20752, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.810625302, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000039062 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 23266, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003222866, 2024-09-05T07:07:44.212 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000138 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 874192 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 29999, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 22287, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 13280319, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 57664694, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 38, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 9, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 1.892631256, 2024-09-05T07:07:44.213 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.808577393, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 1.072637667, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.215965413, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 22286, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.887894474, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000039840 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 35190, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006208065, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000176 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.214 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1754784, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 314528 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 24572, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 21.444219822, 2024-09-05T07:07:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000872709 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 14360, 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.117231723, 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000077801 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-05T07:07:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 292234, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 292221, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 292221, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 521.125713939, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001783327 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 13, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 56, 2024-09-05T07:07:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 5, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 1654, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 214, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 1340, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 100, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 1346, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 437, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 1, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 1, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 6, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 550518, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 546514, 2024-09-05T07:07:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 12, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 17, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 7, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 10, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 341, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 3, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 264, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 2341, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 685, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 216093760, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 7, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 2341, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 260, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 260, 2024-09-05T07:07:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 2081, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 24147, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 33404, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 35188, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 8671, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 510, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 23221 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 8, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 41, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 27, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 39, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 16, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 20, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 6512, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 6492, 2024-09-05T07:07:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 24, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 2506, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 10, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-05T07:07:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 5, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 45555, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 45555, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-05T07:07:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 72, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 72, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 5, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 214887804, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11812, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 60.856472300, 2024-09-05T07:07:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005152088 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 1363, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 8217, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 6854, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 74, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 115, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 41, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 1654, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 9331, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 7677, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 1, 2024-09-05T07:07:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 6520, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 6519, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 74552, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223520 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 310317, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 14, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 292234, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 689, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 777, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6340, 2024-09-05T07:07:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 221.736581047, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.034974224 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 253276, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 78.212832923, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000308804 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1301, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.757256823, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002119336 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-05T07:07:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4442, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.374676125, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000759719 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 18867, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 62.048274020, 2024-09-05T07:07:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003288719 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1285, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 22.021221526, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.017137137 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 20, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.066828268, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003341413 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 51.230872075, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 25.615436037 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 196, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.068598920, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000349994 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6492, 2024-09-05T07:07:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 79.608572212, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012262565 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-05T07:07:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 1, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 34541, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 34541, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1588, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-05T07:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 481529, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 160, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 200, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 4476283, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 20119, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-05T07:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 30221, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 212555046, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 33, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 30221, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 30221, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 149.472498282, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004945981 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 30221, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 31330 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 30221, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3278, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 26943, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 937, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 12749, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 530, 2024-09-05T07:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 5, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 13046, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 2341, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-05T07:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-05T07:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 128, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 41, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 18, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 6, 2024-09-05T07:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 1168, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 524 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-05T07:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 2060 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 5, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 12984, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 6492, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 397558, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 95991964, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 397558, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 397558, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 95991964, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 30221, 2024-09-05T07:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 213271882, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 30221, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 213271882, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 30221, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 30221, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 30221, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 30221, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 6492, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 716836, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 6492, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 937, 2024-09-05T07:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 716836, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55ea5076a960": { 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 45555, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 210693500, 2024-09-05T07:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 45555, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 11812, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 210693500, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-05T07:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-05T07:07:45.076 INFO:tasks.check_counter:Daemon mds.k mds.exported=10 2024-09-05T07:07:45.076 INFO:tasks.check_counter:Daemon mds.k mds.imported=3 2024-09-05T07:07:45.077 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=7 2024-09-05T07:07:45.077 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:45.077 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:45.077 DEBUG:tasks.check_counter:Getting stats from c 2024-09-05T07:07:45.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-05T07:07:45.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:45 smithi115 ceph-mon[34791]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:45.704 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T07:07:45.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:45 smithi086 ceph-mon[30943]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:45.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:45 smithi096 ceph-mon[34468]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1201, 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1203, 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 105163775, 2024-09-05T07:07:46.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 381924, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.142072983, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.027213177, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.080710390, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.025459722, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1201, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.035563191, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000029611 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:46.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3976, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 4328, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 189894786, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 2049049, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-05T07:07:46.583 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.421106647, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.105805392, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.184076066, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.100851101, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4326, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.143631434, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000033201 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 786528 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1771, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1733, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 83462539, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 684446, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.193131431, 2024-09-05T07:07:46.585 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.061013130, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.110430356, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.027974697, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1732, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.061434681, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000035470 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.586 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1748928, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 295840 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10816, 2024-09-05T07:07:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.229651039, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000021232 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-05T07:07:46.588 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-05T07:07:46.589 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 6703, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 198, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 6505, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 6420, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 2, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:46.590 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 234, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 7, 2024-09-05T07:07:46.591 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:46.592 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:46.593 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-05T07:07:46.594 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 50829, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-05T07:07:46.595 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 101, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 6, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 379200326, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 379181194, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 50829 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1671, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 7847, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 6176, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 79, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 179, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 100, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 6703, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 8570, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 1867, 2024-09-05T07:07:46.597 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 63376, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215328 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.598 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:46.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.603 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.604 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.605 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.607 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.609 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.610 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-05T07:07:46.611 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 4672, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:46.612 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:46.613 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:46.614 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 240203, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 81, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-05T07:07:46.615 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 8767902, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 37604, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-05T07:07:46.616 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 6393, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 26, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 4, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 6393, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6393, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.673909853, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000731098 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6393, 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6394 2024-09-05T07:07:46.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 6393, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 6393, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 3423, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 2967, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-05T07:07:46.618 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-05T07:07:46.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-05T07:07:46.620 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 128, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 41, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 11, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 11, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-05T07:07:46.621 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.622 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:46.623 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 6944, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 377865190, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 6944, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6944, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 377865190, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.624 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 6393, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 375006032, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6393, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 375006032, 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:46.625 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:46.626 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 6393, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 6393, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6393, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 6393, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:46.627 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x559c50ae0b40": { 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-05T07:07:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:46.630 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-05T07:07:47.614 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-05T07:07:47.614 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-05T07:07:47.614 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-05T07:07:47.614 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:47.614 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:47.614 DEBUG:tasks.check_counter:Getting stats from f 2024-09-05T07:07:47.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-05T07:07:47.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:47 smithi115 ceph-mon[34791]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:47.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:47 smithi086 ceph-mon[30943]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:47.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:47 smithi096 ceph-mon[34468]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1664, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1624, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 106900829, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 650072, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.205418003, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.052582689, 2024-09-05T07:07:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.136845346, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.023717320, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1624, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.056945415, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000035064 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1751984, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 296768 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1853, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2208, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 113723142, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1374879, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.198991613, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.056285002, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.087401075, 2024-09-05T07:07:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.040884070, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2205, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.068046057, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000030859 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 786256 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3120, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 3122, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 71486272, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 992218, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.297313322, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.076670234, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.111812705, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.084515651, 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3120, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.106154348, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000034023 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-05T07:07:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10295, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.222291677, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000021592 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-05T07:07:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 12424, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 12424, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 12424, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 1, 2024-09-05T07:07:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-05T07:07:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 80854, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-05T07:07:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 111, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 8, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 292851313, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 292850714, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 80854 2024-09-05T07:07:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1605, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 12772, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 11167, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 59, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 99, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 40, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 12424, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 13157, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 733, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-09-05T07:07:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 68348, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215328 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-05T07:07:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-05T07:07:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-05T07:07:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 4670, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 233730, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 82, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-05T07:07:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 12540838, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 59038, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 6082, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 6082, 2024-09-05T07:07:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6082, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.564912234, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000750561 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6082, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6083 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 6082, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 6082, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-05T07:07:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 3359, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 2721, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-05T07:07:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 128, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 41, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 10, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 11, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 1, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-09-05T07:07:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 6633, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 291482317, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 6633, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6633, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 291482317, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 6082, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 288672939, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6082, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 288672939, 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 6082, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 6082, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 6082, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 6082, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x5614b33deb40": { 2024-09-05T07:07:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:48.421 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-05T07:07:49.180 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-05T07:07:49.180 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-05T07:07:49.180 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-05T07:07:49.181 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:49.181 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:49.181 DEBUG:tasks.check_counter:Getting stats from i 2024-09-05T07:07:49.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-05T07:07:49.426 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T07:07:49.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:49 smithi115 ceph-mon[34791]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:49.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:49 smithi086 ceph-mon[30943]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:49.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:49 smithi096 ceph-mon[34468]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:49.932 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-05T07:07:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 119598, 2024-09-05T07:07:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 147614, 2024-09-05T07:07:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 34239210, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 150601936, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 14, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 6.237607789, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 2.558128880, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 3.821405686, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.368854535, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 147614, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.038513932, 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000020584 2024-09-05T07:07:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 232604, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.033200543, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000142 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1771744, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 330528 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 20877, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 25475, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 5030749, 2024-09-05T07:07:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 158257456, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 17, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.640415621, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.693521568, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.503160887, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.390455106, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 25470, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.778058582, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000030548 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14655, 2024-09-05T07:07:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002532814, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000172 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3904, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 880320 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 390369, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 449157, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 117163734, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 438899802, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 69, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2024-09-05T07:07:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 20.447913429, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 8.125563728, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 14.550521046, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.576715302, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 449157, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 8.462215266, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000018840 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 783687, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.116761524, 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000148 2024-09-05T07:07:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 47473, 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 25.846688343, 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000544450 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 22185, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.242092764, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000055987 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 321242, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 320495, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 320495, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 572.614171774, 2024-09-05T07:07:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001786655 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 747, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 10, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 95, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 9, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 5, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 2632, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 605, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 1993, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 34, 2024-09-05T07:07:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 1765, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 696, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 2, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 2, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 6, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 607265, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 568642, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 80, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 71, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 85, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 27, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 534, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 21, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 1125, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 2502, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 685, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 216093760, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 7, 2024-09-05T07:07:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 2502, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 260, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 260, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 2242, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 1683, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 25164, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 36367, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-05T07:07:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 39589, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 10246, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 876, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 32988 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 20, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 29, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 19, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 82, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 183, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 20, 2024-09-05T07:07:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 9599, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 9579, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 24, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 2506, 2024-09-05T07:07:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 37, 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 14, 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-05T07:07:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 80606, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 3073, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 1, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 80605, 2024-09-05T07:07:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 3072, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 132, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 4, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 1, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 131, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 7, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 3, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194352, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 374035500, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-05T07:07:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 22882, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 75.470745571, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003298258 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2341, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 12500, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 10159, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 120, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 171, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 51, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 2632, 2024-09-05T07:07:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 17561, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 14929, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 2, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 9655, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 9653, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 77536, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223520 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 368771, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 2200, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 321242, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 609, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 11653, 2024-09-05T07:07:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9422, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 169.116081968, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.017949064 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 245806, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 69.323540990, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000282025 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9269, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 7.158844686, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000772342 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-05T07:07:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6633, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.093875253, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000617198 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 38072, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 59.936966534, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001574305 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1438, 2024-09-05T07:07:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 27.718284054, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.019275580 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.140995254, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.007049762 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 68, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 67.720094479, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.995883742 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 188, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.127280946, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000677026 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9579, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 167.278207610, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.017463013 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 1, 2024-09-05T07:07:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 34078, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 34078, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 1593, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 473195, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 156, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-05T07:07:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-05T07:07:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 7453339, 2024-09-05T07:07:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 30925, 2024-09-05T07:07:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-05T07:07:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 49942, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 372562190, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 30, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 49942, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49942, 2024-09-05T07:07:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 172.565680636, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003455321 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49942, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 51842 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 49942, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 4111, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 45831, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-05T07:07:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 14, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1594, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 24476, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 941, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 7, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 19270, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2502, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 28, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-05T07:07:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 128, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 41, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 2040, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 25, 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 945 2024-09-05T07:07:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 2111 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-05T07:07:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 19158, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 9579, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 530840, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 108622832, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 530840, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 530840, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 108622832, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 49942, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 373617022, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 49942, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 373617022, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 49942, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 49942, 2024-09-05T07:07:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 49942, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 49942, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 9579, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1055135, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 9579, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 1594, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1055135, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x56547c0a2960": { 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 80606, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 369841196, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 80606, 2024-09-05T07:07:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 22882, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 369841196, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:49.986 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-05T07:07:50.767 INFO:tasks.check_counter:Daemon mds.i mds.exported=27 2024-09-05T07:07:50.767 INFO:tasks.check_counter:Daemon mds.i mds.imported=21 2024-09-05T07:07:50.767 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-05T07:07:50.768 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:50.768 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:50.768 DEBUG:tasks.check_counter:Getting stats from l 2024-09-05T07:07:50.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-05T07:07:51.018 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T07:07:51.533 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-05T07:07:51.533 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T07:07:51.533 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 35592, 2024-09-05T07:07:51.533 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 27727, 2024-09-05T07:07:51.533 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 14357305, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 60322250, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 42, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.868126468, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.674088948, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 1.109292074, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.264144264, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 27726, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.807032498, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000029107 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:51.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 45414, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.006883780, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000151 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 8080, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 30800 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 21020, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 16748, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 10296831, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 85325251, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 49, 2024-09-05T07:07:51.535 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.422953287, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.562686777, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.582529138, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.323120344, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 16747, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.659154531, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000039359 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14689, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001925601, 2024-09-05T07:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000131 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1746496, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 321024 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 345532, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 391499, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 103397691, 2024-09-05T07:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 342823501, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 14, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 17.715587767, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 7.149465948, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 12.527459140, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.460282822, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 391496, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 7.571373502, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000019339 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-05T07:07:51.538 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 697993, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.106636113, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000152 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 789568 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:51.539 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 25950, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 20.457386465, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000788338 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14824, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.035433664, 2024-09-05T07:07:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000069848 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 294891, 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 294883, 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 294883, 2024-09-05T07:07:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 525.712093472, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001782781 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 8, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 65, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 6, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-05T07:07:51.542 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 1675, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 212, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 1361, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 102, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 1369, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 620, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 1, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 1, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 6, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 555925, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 549939, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 6, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 22, 2024-09-05T07:07:51.543 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 3, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 18, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 490, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 4, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 266, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 2478, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 685, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 216093760, 2024-09-05T07:07:51.544 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 7, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 2478, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 260, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 260, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 2218, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 24653, 2024-09-05T07:07:51.545 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 34360, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 36113, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 8897, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 510, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 24206 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 4, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 43, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 28, 2024-09-05T07:07:51.546 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 41, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 34, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 20, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 6772, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 6752, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-05T07:07:51.547 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 24, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 2663, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 18, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 6, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 49208, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-05T07:07:51.548 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 49208, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 74, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 74, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 6, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 221953027, 2024-09-05T07:07:51.549 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 12547, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 59.089017436, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004709413 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1384, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 8659, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 7275, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 79, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 127, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 48, 2024-09-05T07:07:51.550 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 1675, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 9935, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 8260, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 1, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 6797, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 6796, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 74260, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223520 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 315147, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 206, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 294891, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 708, 2024-09-05T07:07:51.551 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 1566, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6585, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 263.208880012, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.039970976 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 253171, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 70.017330255, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000276561 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.552 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1949, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.741571104, 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001406655 2024-09-05T07:07:51.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.554 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-05T07:07:51.555 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4831, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.127610461, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000647404 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20016, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 61.199710519, 2024-09-05T07:07:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003057539 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1363, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.846102555, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002821792 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.558 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.132119627, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006605981 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-05T07:07:51.559 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 196, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.061721366, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000314904 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.561 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6752, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 121.377047573, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.017976458 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-05T07:07:51.562 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 1, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-05T07:07:51.563 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 34666, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 17333, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 1571, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T07:07:51.564 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T07:07:51.565 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T07:07:51.566 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 457347, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 151, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9872, 2024-09-05T07:07:51.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 4585139, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 20455, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-05T07:07:51.568 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 31592, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 219798058, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 33, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 31592, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 31592, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 149.154143805, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004721263 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 31592, 2024-09-05T07:07:51.569 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 32709 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 31592, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3404, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 28188, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 926, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 13473, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 527, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-05T07:07:51.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 6, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 13567, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2478, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-05T07:07:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-05T07:07:51.572 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-05T07:07:51.573 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 128, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 41, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 18, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 6, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 1179, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 523 2024-09-05T07:07:51.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 2190 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-05T07:07:51.575 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 13504, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 6752, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:51.576 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 402140, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 91824334, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 402140, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 402140, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 91824334, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 31592, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 220544194, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 31592, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 220544194, 2024-09-05T07:07:51.578 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 31592, 2024-09-05T07:07:51.579 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 31592, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 31592, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 31592, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 6752, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 746136, 2024-09-05T07:07:51.580 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 6752, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 926, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 746136, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55e7e9c08960": { 2024-09-05T07:07:51.581 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 49208, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 217758723, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 49208, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 12547, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 217758723, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-05T07:07:51.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-05T07:07:51.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-05T07:07:51.583 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-05T07:07:51.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:51 smithi115 ceph-mon[34791]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:51.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:51 smithi086 ceph-mon[30943]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:51.746 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:51 smithi096 ceph-mon[34468]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:52.326 INFO:tasks.check_counter:Daemon mds.l mds.exported=18 2024-09-05T07:07:52.326 INFO:tasks.check_counter:Daemon mds.l mds.imported=4 2024-09-05T07:07:52.326 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2024-09-05T07:07:52.326 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-05T07:07:52.326 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T07:07:52.327 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-05T07:07:52.339 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-05T07:07:52.339 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T07:07:52.339 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T07:07:52.368 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T07:07:52.368 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T07:07:52.428 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T07:07:52.730 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:53.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- 172.21.15.86:0/3699149304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 msgr2=0x7f660c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:53.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 --2- 172.21.15.86:0/3699149304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c10ea90 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f6604009ca0 tx=0x7f660402a8c0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- 172.21.15.86:0/3699149304 shutdown_connections 2024-09-05T07:07:53.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 --2- 172.21.15.86:0/3699149304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c10ea90 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 --2- 172.21.15.86:0/3699149304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c102790 0x7f660c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 --2- 172.21.15.86:0/3699149304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c101de0 0x7f660c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- 172.21.15.86:0/3699149304 >> 172.21.15.86:0/3699149304 conn(0x7f660c0fd8c0 msgr2=0x7f660c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- 172.21.15.86:0/3699149304 shutdown_connections 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- 172.21.15.86:0/3699149304 wait complete. 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 Processor -- start 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.235+0000 7f66124c9640 1 -- start start 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c101de0 0x7f660c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c102790 0x7f660c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:53.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f660c111e70 con 0x7f660c106f10 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f660c111cf0 con 0x7f660c102790 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f660c111ff0 con 0x7f660c101de0 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40144/0 (socket says 172.21.15.86:40144) 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 -- 172.21.15.86:0/1303023851 learned_addr learned my addr 172.21.15.86:0/1303023851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6603fff640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c102790 0x7f660c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f660bfff640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c101de0 0x7f660c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 -- 172.21.15.86:0/1303023851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c101de0 msgr2=0x7f660c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c101de0 0x7f660c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 -- 172.21.15.86:0/1303023851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c102790 msgr2=0x7f660c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c102790 0x7f660c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 -- 172.21.15.86:0/1303023851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c19d2c0 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6610a66640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6604031040 tx=0x7f6604003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604003ce0 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c19d550 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66040044f0 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.236+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604030c90 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.237+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c1a4e90 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.238+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c1021c0 con 0x7f660c106f10 2024-09-05T07:07:53.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.239+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f66040046a0 con 0x7f660c106f10 2024-09-05T07:07:53.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.239+0000 7f6609ffb640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 0x7f65f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:53.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.239+0000 7f660bfff640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 0x7f65f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:53.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.239+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f66040f0d20 con 0x7f660c106f10 2024-09-05T07:07:53.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.240+0000 7f660bfff640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 0x7f65f80a51b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00bbe0 tx=0x7f65fc00b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:53.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.243+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66040bd610 con 0x7f660c106f10 2024-09-05T07:07:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.421+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f660c10a180 con 0x7f660c106f10 2024-09-05T07:07:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.422+0000 7f6609ffb640 1 -- 172.21.15.86:0/1303023851 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v128) ==== 81+0+1001 (secure 0 0 0) 0x7f66040098b0 con 0x7f660c106f10 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4062552821 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2959599766 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/3708924769 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/740829125 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4040819651 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4008919610 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3625555544 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1227677145 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2886866246 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:53.423 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2555553887 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3421533074 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4279196951 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/3708924769 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:listed 17 entries 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 msgr2=0x7f65f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 0x7f65f80a51b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00bbe0 tx=0x7f65fc00b390 comp rx=0 tx=0).stop 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 msgr2=0x7f660c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6604031040 tx=0x7f6604003970 comp rx=0 tx=0).stop 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 shutdown_connections 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f65f80a2cf0 0x7f65f80a51b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f660c106f10 0x7f660c19cbe0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660c102790 0x7f660c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.424+0000 7f66124c9640 1 --2- 172.21.15.86:0/1303023851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f660c101de0 0x7f660c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.425+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 >> 172.21.15.86:0/1303023851 conn(0x7f660c0fd8c0 msgr2=0x7f660c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.425+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 shutdown_connections 2024-09-05T07:07:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:53.425+0000 7f66124c9640 1 -- 172.21.15.86:0/1303023851 wait complete. 2024-09-05T07:07:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:53 smithi115 ceph-mon[34791]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:53.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:53 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1303023851' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:53 smithi086 ceph-mon[30943]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:53.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:53 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1303023851' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:53 smithi096 ceph-mon[34468]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-05T07:07:53.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:53 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1303023851' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:54.285 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T07:07:54.285 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T07:07:54.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T07:07:54.593 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:07:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- 172.21.15.86:0/754484311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 msgr2=0x7f50281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 --2- 172.21.15.86:0/754484311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50281029b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5024009900 tx=0x7f502402a480 comp rx=0 tx=0).stop 2024-09-05T07:07:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- 172.21.15.86:0/754484311 shutdown_connections 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 --2- 172.21.15.86:0/754484311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028102ef0 0x7f502810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 --2- 172.21.15.86:0/754484311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50281029b0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 --2- 172.21.15.86:0/754484311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028100e20 0x7f5028101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- 172.21.15.86:0/754484311 >> 172.21.15.86:0/754484311 conn(0x7f50280fc7d0 msgr2=0x7f50280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- 172.21.15.86:0/754484311 shutdown_connections 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- 172.21.15.86:0/754484311 wait complete. 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 Processor -- start 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.071+0000 7f502f5af640 1 -- start start 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028100e20 0x7f50280fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 0x7f50280fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5028110e30 con 0x7f5028102570 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5028110cb0 con 0x7f5028100e20 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502f5af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5028110fb0 con 0x7f5028102ef0 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40170/0 (socket says 172.21.15.86:40170) 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 -- 172.21.15.86:0/2943385596 learned_addr learned my addr 172.21.15.86:0/2943385596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502d34b640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028100e20 0x7f50280fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502db4c640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 0x7f50280fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 -- 172.21.15.86:0/2943385596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 msgr2=0x7f50280fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 0x7f50280fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 -- 172.21.15.86:0/2943385596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028100e20 msgr2=0x7f50280fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028100e20 0x7f50280fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 -- 172.21.15.86:0/2943385596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50281a6930 con 0x7f5028102570 2024-09-05T07:07:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502db4c640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 0x7f50280fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f502cb4a640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5024006010 tx=0x7f50240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.072+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502403f070 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.073+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50281a6bc0 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.073+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5024003aa0 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.073+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024035030 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.073+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50281a70f0 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.073+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5028101200 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.075+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5024042070 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.075+0000 7f501a7fc640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 0x7f50140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.076+0000 7f502d34b640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 0x7f50140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.076+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7f50240f0190 con 0x7f5028102570 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.076+0000 7f502d34b640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 0x7f50140a51b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f501c0058c0 tx=0x7f501c00a250 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:07:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.080+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50240bc970 con 0x7f5028102570 2024-09-05T07:07:55.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.257+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f502810a0a0 con 0x7f5028102570 2024-09-05T07:07:55.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.258+0000 7f501a7fc640 1 -- 172.21.15.86:0/2943385596 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v128) ==== 81+0+1001 (secure 0 0 0) 0x7f50240bc270 con 0x7f5028102570 2024-09-05T07:07:55.258 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4062552821 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.258 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2959599766 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.258 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/3708924769 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/740829125 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4040819651 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4008919610 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3625555544 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1227677145 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2886866246 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1793567579 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2555553887 2024-09-06T06:32:11.079228+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3421533074 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/164214999 2024-09-06T06:31:48.412507+0000 2024-09-05T07:07:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4279196951 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.260 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/3708924769 2024-09-06T06:41:15.328251+0000 2024-09-05T07:07:55.260 INFO:teuthology.orchestra.run.smithi086.stderr:listed 17 entries 2024-09-05T07:07:55.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 msgr2=0x7f50140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 0x7f50140a51b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f501c0058c0 tx=0x7f501c00a250 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 msgr2=0x7f50280fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5024006010 tx=0x7f50240042f0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 shutdown_connections 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f50140a2cf0 0x7f50140a51b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5028102ef0 0x7f50280fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5028102570 0x7f50280fd1c0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 --2- 172.21.15.86:0/2943385596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5028100e20 0x7f50280fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 >> 172.21.15.86:0/2943385596 conn(0x7f50280fc7d0 msgr2=0x7f502806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 shutdown_connections 2024-09-05T07:07:55.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:07:55.260+0000 7f502f5af640 1 -- 172.21.15.86:0/2943385596 wait complete. 2024-09-05T07:07:55.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:55 smithi086 ceph-mon[30943]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:55.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:55 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2943385596' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:55.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:55 smithi115 ceph-mon[34791]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:55.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:55 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2943385596' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:55 smithi096 ceph-mon[34468]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T07:07:55.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:55 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2943385596' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T07:07:56.094 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-09-05T07:07:56.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T07:07:56.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-05T07:07:56.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83137]: fuse finished with error 0 and tester_r 0 2024-09-05T07:07:56.135 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T07:07:57.682 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:57 smithi115 ceph-mon[34791]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:57.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:57 smithi086 ceph-mon[30943]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:57.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:57 smithi096 ceph-mon[34468]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:07:59.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:07:59 smithi115 ceph-mon[34791]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:59.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:07:59 smithi086 ceph-mon[30943]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:07:59.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:07:59 smithi096 ceph-mon[34468]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-09-05T07:08:01.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:01 smithi086 ceph-mon[30943]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:08:01.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:01 smithi096 ceph-mon[34468]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:08:01.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:01 smithi115 ceph-mon[34791]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-05T07:08:02.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T07:08:02.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T07:08:02.237 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-05T07:08:02.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T07:08:02.314 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-09-05T07:08:02.376 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-05T07:08:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T07:08:02.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T07:08:02.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T07:08:02.408 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-09-05T07:08:02.505 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-05T07:08:02.515 INFO:tasks.cephadm:Teardown begin 2024-09-05T07:08:02.516 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-05T07:08:02.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:08:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- 172.21.15.86:0/2598583936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 msgr2=0x7ff6dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 --2- 172.21.15.86:0/2598583936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc10ea90 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009900 tx=0x7ff6d402a410 comp rx=0 tx=0).stop 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- 172.21.15.86:0/2598583936 shutdown_connections 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 --2- 172.21.15.86:0/2598583936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc10ea90 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 --2- 172.21.15.86:0/2598583936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 --2- 172.21.15.86:0/2598583936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- 172.21.15.86:0/2598583936 >> 172.21.15.86:0/2598583936 conn(0x7ff6dc0fd8c0 msgr2=0x7ff6dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- 172.21.15.86:0/2598583936 shutdown_connections 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- 172.21.15.86:0/2598583936 wait complete. 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 Processor -- start 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.266+0000 7ff6e401f640 1 -- start start 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc111e20 con 0x7ff6dc106f10 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc111ca0 con 0x7ff6dc102790 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc111fa0 con 0x7ff6dc101de0 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51598/0 (socket says 172.21.15.86:51598) 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 -- 172.21.15.86:0/1360738189 learned_addr learned my addr 172.21.15.86:0/1360738189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e1dbb640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e15ba640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 -- 172.21.15.86:0/1360738189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 msgr2=0x7ff6dc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 -- 172.21.15.86:0/1360738189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 msgr2=0x7ff6dc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 -- 172.21.15.86:0/1360738189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc19d2d0 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e1dbb640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e15ba640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e25bc640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4004770 tx=0x7ff6d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d403f030 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.267+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc19d560 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.268+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6d4004440 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.268+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc1a4ea0 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.268+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4035070 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.269+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc1021c0 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.270+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff6d4042030 con 0x7ff6dc106f10 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.270+0000 7ff6daffd640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 0x7ff6c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.270+0000 7ff6e1dbb640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 0x7ff6c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.270+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9605+0+0 (secure 0 0 0) 0x7ff6d40ef890 con 0x7ff6dc106f10 2024-09-05T07:08:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.271+0000 7ff6e1dbb640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 0x7ff6c40a51b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009fc0 tx=0x7ff6cc005f90 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.275+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6d40bba20 con 0x7ff6dc106f10 2024-09-05T07:08:03.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.483+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6dc10a150 con 0x7ff6dc106f10 2024-09-05T07:08:03.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.484+0000 7ff6daffd640 1 -- 172.21.15.86:0/1360738189 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11986 (secure 0 0 0) 0x7ff6d40bfd20 con 0x7ff6dc106f10 2024-09-05T07:08:03.486 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:08:03.486 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"btime":"2024-09-05T07:06:57:311370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:06:57.247303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:08:03.486 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-09-05T07:08:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 msgr2=0x7ff6c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 0x7ff6c40a51b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009fc0 tx=0x7ff6cc005f90 comp rx=0 tx=0).stop 2024-09-05T07:08:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 msgr2=0x7ff6dc19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4004770 tx=0x7ff6d4004040 comp rx=0 tx=0).stop 2024-09-05T07:08:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 shutdown_connections 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7ff6c40a2cf0 0x7ff6c40a51b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6dc106f10 0x7ff6dc19cbf0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc102790 0x7ff6dc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 --2- 172.21.15.86:0/1360738189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6dc101de0 0x7ff6dc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.486+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 >> 172.21.15.86:0/1360738189 conn(0x7ff6dc0fd8c0 msgr2=0x7ff6dc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.487+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 shutdown_connections 2024-09-05T07:08:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:03.487+0000 7ff6e401f640 1 -- 172.21.15.86:0/1360738189 wait complete. 2024-09-05T07:08:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:03 smithi115 ceph-mon[34791]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-05T07:08:03.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:03 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:08:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:03 smithi086 ceph-mon[30943]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-05T07:08:03.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:03 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:08:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:03 smithi096 ceph-mon[34468]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-05T07:08:03.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:03 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T07:08:04.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T07:08:04.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:04 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1360738189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:04.509 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:04 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1360738189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:04.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:04 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1360738189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:05.629 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:05 smithi115 ceph-mon[34791]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-09-05T07:08:05.655 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:05 smithi096 ceph-mon[34468]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-09-05T07:08:05.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:05 smithi086 ceph-mon[30943]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-09-05T07:08:06.533 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:06.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:06 smithi086 ceph-mon[30943]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T07:08:06.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:06 smithi096 ceph-mon[34468]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T07:08:06.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:06 smithi115 ceph-mon[34791]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T07:08:07.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 -- 172.21.15.86:0/1781739177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 msgr2=0x7f799810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 --2- 172.21.15.86:0/1781739177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799810da40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7990009ca0 tx=0x7f799002a950 comp rx=0 tx=0).stop 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 -- 172.21.15.86:0/1781739177 shutdown_connections 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 --2- 172.21.15.86:0/1781739177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799810da40 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 --2- 172.21.15.86:0/1781739177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 0x7f79981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 --2- 172.21.15.86:0/1781739177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 0x7f7998101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 -- 172.21.15.86:0/1781739177 >> 172.21.15.86:0/1781739177 conn(0x7f79980fc7d0 msgr2=0x7f79980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 -- 172.21.15.86:0/1781739177 shutdown_connections 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.263+0000 7f799d238640 1 -- 172.21.15.86:0/1781739177 wait complete. 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.264+0000 7f799d238640 1 Processor -- start 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.264+0000 7f799d238640 1 -- start start 2024-09-05T07:08:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 0x7f79981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 0x7f79981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110e30 con 0x7f7998102ef0 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110cb0 con 0x7f7998102570 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110fb0 con 0x7f7998100e20 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51616/0 (socket says 172.21.15.86:51616) 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 -- 172.21.15.86:0/796644285 learned_addr learned my addr 172.21.15.86:0/796644285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"btime":"2024-09-05T07:06:57:311370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3431663557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3431663557},{"type":"v1","addr":"172.21.15.86:6835","nonce":3431663557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/78876235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":78876235},{"type":"v1","addr":"172.21.15.86:6837","nonce":78876235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T06:38:55.927385+0000","modified":"2024-09-05T07:06:57.247303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24461,"mds_3":24422,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3075373024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3075373024},{"type":"v1","addr":"172.21.15.86:6839","nonce":3075373024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/1507644375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":1507644375},{"type":"v1","addr":"172.21.15.86:6841","nonce":1507644375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.115:6833/3498674074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3498674074},{"type":"v1","addr":"172.21.15.115:6833","nonce":3498674074}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":20,"addr":"172.21.15.96:6833/1668283475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1668283475},{"type":"v1","addr":"172.21.15.96:6833","nonce":1668283475}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1272008759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1272008759},{"type":"v1","addr":"172.21.15.115:6835","nonce":1272008759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/3823127248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3823127248},{"type":"v1","addr":"172.21.15.96:6835","nonce":3823127248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6837/2087354694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2087354694},{"type":"v1","addr":"172.21.15.115:6837","nonce":2087354694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6837/1007288434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":1007288434},{"type":"v1","addr":"172.21.15.96:6837","nonce":1007288434}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6839/4249007973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4249007973},{"type":"v1","addr":"172.21.15.115:6839","nonce":4249007973}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/2996609527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2996609527},{"type":"v1","addr":"172.21.15.96:6839","nonce":2996609527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24403,24422,24461,24466,24479]},"id":1}]} 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799659c640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 0x7f79981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f7996d9d640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 0x7f79981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 -- 172.21.15.86:0/796644285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 msgr2=0x7f79981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 0x7f79981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 -- 172.21.15.86:0/796644285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 msgr2=0x7f79981988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 0x7f79981988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 -- 172.21.15.86:0/796644285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799819d360 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799759e640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f799002ae70 tx=0x7f799002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990034040 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799819d5f0 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79900308f0 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990004750 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.265+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981a4f30 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.266+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79981021b0 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.268+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7990037020 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.268+0000 7f797ffff640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 0x7f79800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.268+0000 7f7996d9d640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 0x7f79800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.268+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9693+0+0 (secure 0 0 0) 0x7f79900f0860 con 0x7f7998102ef0 2024-09-05T07:08:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.269+0000 7f7996d9d640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 0x7f79800a51b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005850 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.271+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79900bd100 con 0x7f7998102ef0 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.476+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7998101200 con 0x7f7998102ef0 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.476+0000 7f797ffff640 1 -- 172.21.15.86:0/796644285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11986 (secure 0 0 0) 0x7f7998101200 con 0x7f7998102ef0 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 msgr2=0x7f79800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 0x7f79800a51b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005850 comp rx=0 tx=0).stop 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 msgr2=0x7f799819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f799002ae70 tx=0x7f799002f5d0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 shutdown_connections 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f79800a2cf0 0x7f79800a51b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7998102ef0 0x7f799819cc80 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7998102570 0x7f79981988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 --2- 172.21.15.86:0/796644285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7998100e20 0x7f79981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.478+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 >> 172.21.15.86:0/796644285 conn(0x7f79980fc7d0 msgr2=0x7f799806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.479+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 shutdown_connections 2024-09-05T07:08:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:07.479+0000 7f799d238640 1 -- 172.21.15.86:0/796644285 wait complete. 2024-09-05T07:08:07.702 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:07 smithi096 ceph-mon[34468]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-05T07:08:07.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:07 smithi086 ceph-mon[30943]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-05T07:08:07.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:07 smithi115 ceph-mon[34791]: pgmap v847: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-05T07:08:08.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T07:08:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:08 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/796644285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:08 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:08 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:08 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:08 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:08 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/796644285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:08 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:08 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:08 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:08 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:08 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/796644285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T07:08:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:08 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:08 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:08 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:08.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:08 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 -- 172.21.15.86:0/455441310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 msgr2=0x7f5d201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 --2- 172.21.15.86:0/455441310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d201029b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009900 tx=0x7f5d0802a480 comp rx=0 tx=0).stop 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 -- 172.21.15.86:0/455441310 shutdown_connections 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 --2- 172.21.15.86:0/455441310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 0x7f5d2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 --2- 172.21.15.86:0/455441310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d201029b0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 --2- 172.21.15.86:0/455441310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 -- 172.21.15.86:0/455441310 >> 172.21.15.86:0/455441310 conn(0x7f5d200fc7d0 msgr2=0x7f5d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.589+0000 7f5d24f24640 1 -- 172.21.15.86:0/455441310 shutdown_connections 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.590+0000 7f5d24f24640 1 -- 172.21.15.86:0/455441310 wait complete. 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.590+0000 7f5d24f24640 1 Processor -- start 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.590+0000 7f5d24f24640 1 -- start start 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d201991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 0x7f5d201a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20110e30 con 0x7f5d20102570 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20110cb0 con 0x7f5d20102ef0 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d24f24640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20110fb0 con 0x7f5d20100e20 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d1e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d201991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51640/0 (socket says 172.21.15.86:51640) 2024-09-05T07:08:09.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 -- 172.21.15.86:0/2587725313 learned_addr learned my addr 172.21.15.86:0/2587725313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d1ed9d640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 0x7f5d201a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 -- 172.21.15.86:0/2587725313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 msgr2=0x7f5d201991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d201991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 -- 172.21.15.86:0/2587725313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 msgr2=0x7f5d201a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 0x7f5d201a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 -- 172.21.15.86:0/2587725313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201a47a0 con 0x7f5d20102570 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d1e59c640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d201991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.591+0000 7f5d15d9b640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5d08006010 tx=0x7f5d080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.592+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0803f030 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.592+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a4970 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.592+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d08003cb0 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.592+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08035070 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.592+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a4e20 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.593+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20101200 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.594+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5d08042030 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.595+0000 7f5d17fff640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 0x7f5d0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.595+0000 7f5d1e59c640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 0x7f5d0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.595+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9693+0+0 (secure 0 0 0) 0x7f5d080effe0 con 0x7f5d20102570 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.595+0000 7f5d1e59c640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 0x7f5d0c0a51b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5d10005d00 tx=0x7f5d1000b410 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.598+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d080bc120 con 0x7f5d20102570 2024-09-05T07:08:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:09.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:08:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:08:09.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:09 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.769+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d20109090 con 0x7f5d20102570 2024-09-05T07:08:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.770+0000 7f5d17fff640 1 -- 172.21.15.86:0/2587725313 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+25670 (secure 0 0 0) 0x7f5d080c0420 con 0x7f5d20102570 2024-09-05T07:08:09.772 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T07:08:09.772 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":129,"fsid":"5a055634-6b50-11ef-bcd6-c7b262605968","created":"2024-09-05T06:31:26.636788+0000","modified":"2024-09-05T07:08:05.445265+0000","last_up_change":"2024-09-05T06:38:04.894358+0000","last_in_change":"2024-09-05T06:37:41.863921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T06:34:21.377500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T06:38:51.679225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T06:38:54.212638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T06:39:01.086734+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"964c1a8b-7ecd-4cf8-80bb-1498f3f4b29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6803","nonce":2287131107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6805","nonce":2287131107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6809","nonce":2287131107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2287131107},{"type":"v1","addr":"172.21.15.86:6807","nonce":2287131107}]},"public_addr":"172.21.15.86:6803/2287131107","cluster_addr":"172.21.15.86:6805/2287131107","heartbeat_back_addr":"172.21.15.86:6809/2287131107","heartbeat_front_addr":"172.21.15.86:6807/2287131107","state":["exists","up"]},{"osd":1,"uuid":"e2892f95-df5c-4d96-955a-d3d741302bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6801","nonce":3435233109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6803","nonce":3435233109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6807","nonce":3435233109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3435233109},{"type":"v1","addr":"172.21.15.96:6805","nonce":3435233109}]},"public_addr":"172.21.15.96:6801/3435233109","cluster_addr":"172.21.15.96:6803/3435233109","heartbeat_back_addr":"172.21.15.96:6807/3435233109","heartbeat_front_addr":"172.21.15.96:6805/3435233109","state":["exists","up"]},{"osd":2,"uuid":"6ec111fa-9256-44e0-b091-da1f005c82d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6801","nonce":1161034740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6803","nonce":1161034740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6807","nonce":1161034740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1161034740},{"type":"v1","addr":"172.21.15.115:6805","nonce":1161034740}]},"public_addr":"172.21.15.115:6801/1161034740","cluster_addr":"172.21.15.115:6803/1161034740","heartbeat_back_addr":"172.21.15.115:6807/1161034740","heartbeat_front_addr":"172.21.15.115:6805/1161034740","state":["exists","up"]},{"osd":3,"uuid":"d41eff29-18b3-48ac-8d2b-5ef2945f4144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6811","nonce":3414369107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6813","nonce":3414369107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6817","nonce":3414369107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3414369107},{"type":"v1","addr":"172.21.15.86:6815","nonce":3414369107}]},"public_addr":"172.21.15.86:6811/3414369107","cluster_addr":"172.21.15.86:6813/3414369107","heartbeat_back_addr":"172.21.15.86:6817/3414369107","heartbeat_front_addr":"172.21.15.86:6815/3414369107","state":["exists","up"]},{"osd":4,"uuid":"83c0b669-bfcd-4de2-8349-2eb03e0f28fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6809","nonce":208598279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6811","nonce":208598279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6815","nonce":208598279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":208598279},{"type":"v1","addr":"172.21.15.96:6813","nonce":208598279}]},"public_addr":"172.21.15.96:6809/208598279","cluster_addr":"172.21.15.96:6811/208598279","heartbeat_back_addr":"172.21.15.96:6815/208598279","heartbeat_front_addr":"172.21.15.96:6813/208598279","state":["exists","up"]},{"osd":5,"uuid":"6110ec1a-05cb-42e3-9e2f-8442e1e2abf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6809","nonce":464616540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6811","nonce":464616540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6815","nonce":464616540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":464616540},{"type":"v1","addr":"172.21.15.115:6813","nonce":464616540}]},"public_addr":"172.21.15.115:6809/464616540","cluster_addr":"172.21.15.115:6811/464616540","heartbeat_back_addr":"172.21.15.115:6815/464616540","heartbeat_front_addr":"172.21.15.115:6813/464616540","state":["exists","up"]},{"osd":6,"uuid":"637d7ec0-a913-4e18-92a2-9503fedd4a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6819","nonce":1960864620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6821","nonce":1960864620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6825","nonce":1960864620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1960864620},{"type":"v1","addr":"172.21.15.86:6823","nonce":1960864620}]},"public_addr":"172.21.15.86:6819/1960864620","cluster_addr":"172.21.15.86:6821/1960864620","heartbeat_back_addr":"172.21.15.86:6825/1960864620","heartbeat_front_addr":"172.21.15.86:6823/1960864620","state":["exists","up"]},{"osd":7,"uuid":"76945c8d-4539-4690-a020-b9af497ee44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6817","nonce":2748292566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6819","nonce":2748292566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6823","nonce":2748292566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2748292566},{"type":"v1","addr":"172.21.15.96:6821","nonce":2748292566}]},"public_addr":"172.21.15.96:6817/2748292566","cluster_addr":"172.21.15.96:6819/2748292566","heartbeat_back_addr":"172.21.15.96:6823/2748292566","heartbeat_front_addr":"172.21.15.96:6821/2748292566","state":["exists","up"]},{"osd":8,"uuid":"608c50ef-1ed5-407d-8375-a0eff5fa26f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6817","nonce":3761965299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6819","nonce":3761965299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6823","nonce":3761965299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3761965299},{"type":"v1","addr":"172.21.15.115:6821","nonce":3761965299}]},"public_addr":"172.21.15.115:6817/3761965299","cluster_addr":"172.21.15.115:6819/3761965299","heartbeat_back_addr":"172.21.15.115:6823/3761965299","heartbeat_front_addr":"172.21.15.115:6821/3761965299","state":["exists","up"]},{"osd":9,"uuid":"311dbd75-2e56-4242-b1b4-536719722c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6827","nonce":174097683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6829","nonce":174097683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6833","nonce":174097683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":174097683},{"type":"v1","addr":"172.21.15.86:6831","nonce":174097683}]},"public_addr":"172.21.15.86:6827/174097683","cluster_addr":"172.21.15.86:6829/174097683","heartbeat_back_addr":"172.21.15.86:6833/174097683","heartbeat_front_addr":"172.21.15.86:6831/174097683","state":["exists","up"]},{"osd":10,"uuid":"d9e00a49-77d5-4256-8ec1-9f43c99a1326","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6825","nonce":1196038229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6827","nonce":1196038229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6831","nonce":1196038229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1196038229},{"type":"v1","addr":"172.21.15.96:6829","nonce":1196038229}]},"public_addr":"172.21.15.96:6825/1196038229","cluster_addr":"172.21.15.96:6827/1196038229","heartbeat_back_addr":"172.21.15.96:6831/1196038229","heartbeat_front_addr":"172.21.15.96:6829/1196038229","state":["exists","up"]},{"osd":11,"uuid":"dfda802c-c8b3-4230-81b0-48a3a331b1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6825","nonce":3733177277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6827","nonce":3733177277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6831","nonce":3733177277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":3733177277},{"type":"v1","addr":"172.21.15.115:6829","nonce":3733177277}]},"public_addr":"172.21.15.115:6825/3733177277","cluster_addr":"172.21.15.115:6827/3733177277","heartbeat_back_addr":"172.21.15.115:6831/3733177277","heartbeat_front_addr":"172.21.15.115:6829/3733177277","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:32.175222+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:33:55.121402+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:18.815307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:34:44.290677+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:08.463044+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:32.933990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:35:56.780311+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:22.310614+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:36:46.714140+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:12.955878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:37:37.642275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T06:38:02.829660+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4062552821":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/2959599766":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/3708924769":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/740829125":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4040819651":"2024-09-06T06:41:15.328251+0000","172.21.15.86:0/4008919610":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3625555544":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6801/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/1227677145":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/2886866246":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6800/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:6801/1793567579":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/2555553887":"2024-09-06T06:32:11.079228+0000","172.21.15.86:0/3421533074":"2024-09-06T06:31:48.412507+0000","172.21.15.86:6800/164214999":"2024-09-06T06:31:48.412507+0000","172.21.15.86:0/4279196951":"2024-09-06T06:41:15.328251+0000","172.21.15.86:6800/3708924769":"2024-09-06T06:41:15.328251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":36,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":36,"length":1},{"begin":48,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":36,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":36,"length":1},{"begin":48,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T07:08:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 msgr2=0x7f5d0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 0x7f5d0c0a51b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5d10005d00 tx=0x7f5d1000b410 comp rx=0 tx=0).stop 2024-09-05T07:08:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 msgr2=0x7f5d20199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5d08006010 tx=0x7f5d080042f0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 shutdown_connections 2024-09-05T07:08:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f5d0c0a2cf0 0x7f5d0c0a51b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d20102ef0 0x7f5d201a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d20102570 0x7f5d20199720 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 --2- 172.21.15.86:0/2587725313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d20100e20 0x7f5d201991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 >> 172.21.15.86:0/2587725313 conn(0x7f5d200fc7d0 msgr2=0x7f5d200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 shutdown_connections 2024-09-05T07:08:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:09.773+0000 7f5d24f24640 1 -- 172.21.15.86:0/2587725313 wait complete. 2024-09-05T07:08:09.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:09.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:08:09.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:08:09.808 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:09 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: pgmap v848: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T07:08:09.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:09 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' 2024-09-05T07:08:10.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:10 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2587725313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:08:10.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs fail cephfs 2024-09-05T07:08:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:10.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:10 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2587725313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:08:10.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:10 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2587725313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T07:08:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 -- 172.21.15.86:0/3284449279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 msgr2=0x7f9a68100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/3284449279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a68100d30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009930 tx=0x7f9a5c02a420 comp rx=0 tx=0).stop 2024-09-05T07:08:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 -- 172.21.15.86:0/3284449279 shutdown_connections 2024-09-05T07:08:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/3284449279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a68101270 0x7f9a6810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/3284449279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a68100d30 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/3284449279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a681068d0 0x7f9a68106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 -- 172.21.15.86:0/3284449279 >> 172.21.15.86:0/3284449279 conn(0x7f9a680fc610 msgr2=0x7f9a680fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 -- 172.21.15.86:0/3284449279 shutdown_connections 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.209+0000 7f9a6d79f640 1 -- 172.21.15.86:0/3284449279 wait complete. 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 Processor -- start 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 -- start start 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 0x7f9a68197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 0x7f9a6819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68110b80 con 0x7f9a681008d0 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68110a00 con 0x7f9a68101270 2024-09-05T07:08:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6d79f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68110d00 con 0x7f9a681068d0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51660/0 (socket says 172.21.15.86:51660) 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a67fff640 1 -- 172.21.15.86:0/1389844983 learned_addr learned my addr 172.21.15.86:0/1389844983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a677fe640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 0x7f9a68197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a67fff640 1 -- 172.21.15.86:0/1389844983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 msgr2=0x7f9a6819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a6cf9e640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 0x7f9a6819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.210+0000 7f9a67fff640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 0x7f9a6819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a67fff640 1 -- 172.21.15.86:0/1389844983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 msgr2=0x7f9a68197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a67fff640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 0x7f9a68197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a67fff640 1 -- 172.21.15.86:0/1389844983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6819c090 con 0x7f9a681008d0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a677fe640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 0x7f9a68197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a6cf9e640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 0x7f9a6819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a67fff640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c890 tx=0x7f9a5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004220 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a580043c0 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6819c380 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004a00 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.211+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a3c60 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.212+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a68100d30 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.213+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9a58004ba0 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.214+0000 7f9a657fa640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 0x7f9a500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.214+0000 7f9a677fe640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 0x7f9a500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.214+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9693+0+0 (secure 0 0 0) 0x7f9a580cf580 con 0x7f9a681008d0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.215+0000 7f9a677fe640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 0x7f9a500a51b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c000c00 tx=0x7f9a5c031400 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.218+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a58014030 con 0x7f9a681008d0 2024-09-05T07:08:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.419+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9a68198440 con 0x7f9a681008d0 2024-09-05T07:08:11.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.422+0000 7f9a657fa640 1 -- 172.21.15.86:0/1389844983 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f9a5809bdb0 con 0x7f9a681008d0 2024-09-05T07:08:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-05T07:08:11.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 msgr2=0x7f9a500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 0x7f9a500a51b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c000c00 tx=0x7f9a5c031400 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 msgr2=0x7f9a681970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c890 tx=0x7f9a5800cd50 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 shutdown_connections 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f9a500a2cf0 0x7f9a500a51b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a681068d0 0x7f9a6819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a68101270 0x7f9a68197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 --2- 172.21.15.86:0/1389844983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a681008d0 0x7f9a681970e0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 >> 172.21.15.86:0/1389844983 conn(0x7f9a680fc610 msgr2=0x7f9a680fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 shutdown_connections 2024-09-05T07:08:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:11.427+0000 7f9a6d79f640 1 -- 172.21.15.86:0/1389844983 wait complete. 2024-09-05T07:08:11.738 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.738 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.738 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mds[69714]: -- 172.21.15.86:0/2686647651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fe36c1c20 con 0x555fe369f000 2024-09-05T07:08:11.738 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: starting mds.a at 2024-09-05T07:08:11.739 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.739 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.739 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mds[69341]: -- 172.21.15.86:0/2358818709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d2dfafc20 con 0x561d2df8d000 2024-09-05T07:08:11.739 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: starting mds.g at 2024-09-05T07:08:11.740 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a[30939]: 2024-09-05T07:08:11.420+0000 7f4e91144640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T07:08:11.740 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: pgmap v849: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:11.740 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T07:08:11.740 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:11.741 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:11.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:11.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:11.742 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:11 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:11.807 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.807 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.807 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mds[63772]: -- 172.21.15.96:0/2040388439 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3a1a2b860 con 0x55a3a1a09400 2024-09-05T07:08:11.808 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: starting mds.h at 2024-09-05T07:08:11.808 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.808 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.808 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mds[63471]: -- 172.21.15.96:0/1251137709 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a78a43b860 con 0x55a78a418800 2024-09-05T07:08:11.808 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: starting mds.e at 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mds[63162]: -- 172.21.15.96:0/3017395000 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643539ff860 con 0x5643539dc800 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: starting mds.k at 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.809 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mds[64072]: -- 172.21.15.96:0/1536315192 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639d6efd860 con 0x5639d6edb000 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: starting mds.b at 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: pgmap v849: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:11.810 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:11.811 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:11.811 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:11.811 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:11 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:11.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mds[64607]: -- 172.21.15.115:0/2542745803 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f916c9d860 con 0x55f916c7b000 2024-09-05T07:08:11.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: starting mds.c at 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mds[64906]: -- 172.21.15.115:0/1069586365 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b95d595860 con 0x55b95d572800 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: starting mds.i at 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.934 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mds[64002]: -- 172.21.15.115:0/2831611906 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640e41bb860 con 0x5640e4198800 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: starting mds.l at 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: ignoring --setuser ceph since I am not root 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: ignoring --setgroup ceph since I am not root 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mds[64306]: -- 172.21.15.115:0/910888800 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56314696d860 con 0x56314694b000 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: starting mds.f at 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: pgmap v849: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-05T07:08:11.935 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1389844983' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:11 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:12.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-05T07:08:12.394 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1604208857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac102d30 msgr2=0x7fc1ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1604208857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac102d30 0x7fc1ac103110 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009930 tx=0x7fc19c02a440 comp rx=0 tx=0).stop 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1604208857 shutdown_connections 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1604208857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1604208857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1604208857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac102d30 0x7fc1ac103110 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1604208857 >> 172.21.15.86:0/1604208857 conn(0x7fc1ac0fc7d0 msgr2=0x7fc1ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1604208857 shutdown_connections 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.834+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1604208857 wait complete. 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 Processor -- start 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 -- start start 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 0x7fc1ac1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac112010 con 0x7fc1ac1036e0 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac111e90 con 0x7fc1ac1082a0 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.835+0000 7fc1b15e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac112190 con 0x7fc1ac102d30 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57972/0 (socket says 172.21.15.86:57972) 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 -- 172.21.15.86:0/1611472469 learned_addr learned my addr 172.21.15.86:0/1611472469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aaffd640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 0x7fc1ac1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1ab7fe640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 -- 172.21.15.86:0/1611472469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 msgr2=0x7fc1ac1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 0x7fc1ac1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 -- 172.21.15.86:0/1611472469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 msgr2=0x7fc1ac19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 -- 172.21.15.86:0/1611472469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac19aee0 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aaffd640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 0x7fc1ac1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1ab7fe640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1aa7fc640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0006fd0 tx=0x7fc1a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0011830 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a0016070 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0010df0 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac19b1d0 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.836+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a7350 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.837+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac06a850 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.838+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1a00119d0 con 0x7fc1ac1036e0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.839+0000 7fc193fff640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 0x7fc1940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.839+0000 7fc1aaffd640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 0x7fc1940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:12.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.840+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 10035+0+0 (secure 0 0 0) 0x7fc1a00cf020 con 0x7fc1ac1036e0 2024-09-05T07:08:12.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.840+0000 7fc1aaffd640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 0x7fc1940a5230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c031cd0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:12.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:12.844+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1a009b760 con 0x7fc1ac1036e0 2024-09-05T07:08:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.051+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc1ac1a7580 con 0x7fc1ac1036e0 2024-09-05T07:08:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.479+0000 7fc193fff640 1 -- 172.21.15.86:0/1611472469 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7fc1a009b060 con 0x7fc1ac1036e0 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.481+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 msgr2=0x7fc1940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.481+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 0x7fc1940a5230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c031cd0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 msgr2=0x7fc1ac1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0006fd0 tx=0x7fc1a000b8e0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 shutdown_connections 2024-09-05T07:08:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fc1940a2d70 0x7fc1940a5230 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1ac1082a0 0x7fc1ac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ac1036e0 0x7fc1ac1a0ae0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 --2- 172.21.15.86:0/1611472469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1ac102d30 0x7fc1ac1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 >> 172.21.15.86:0/1611472469 conn(0x7fc1ac0fc7d0 msgr2=0x7fc1ac1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 shutdown_connections 2024-09-05T07:08:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:13.482+0000 7fc1b15e3640 1 -- 172.21.15.86:0/1611472469 wait complete. 2024-09-05T07:08:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T07:08:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6832/943997132,v1:172.21.15.96:6833/943997132] up:boot 2024-09-05T07:08:13.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6835/2974739219,v1:172.21.15.96:6838/2974739219] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6834/1163844570,v1:172.21.15.96:6836/1163844570] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.96:6837/1928153236,v1:172.21.15.96:6839/1928153236] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6832/3595349876,v1:172.21.15.115:6833/3595349876] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6834/1019489923,v1:172.21.15.115:6835/1019489923] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6836/1522768726,v1:172.21.15.115:6838/1522768726] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.115:6837/2926515339,v1:172.21.15.115:6839/2926515339] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6838/863999171,v1:172.21.15.86:6840/863999171] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: mds.? [v2:172.21.15.86:6839/744750249,v1:172.21.15.86:6841/744750249] up:boot 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T07:08:13.558 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T07:08:13.559 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:13 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T07:08:13.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6832/943997132,v1:172.21.15.96:6833/943997132] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6835/2974739219,v1:172.21.15.96:6838/2974739219] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6834/1163844570,v1:172.21.15.96:6836/1163844570] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.96:6837/1928153236,v1:172.21.15.96:6839/1928153236] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6832/3595349876,v1:172.21.15.115:6833/3595349876] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6834/1019489923,v1:172.21.15.115:6835/1019489923] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6836/1522768726,v1:172.21.15.115:6838/1522768726] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.115:6837/2926515339,v1:172.21.15.115:6839/2926515339] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6838/863999171,v1:172.21.15.86:6840/863999171] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: mds.? [v2:172.21.15.86:6839/744750249,v1:172.21.15.86:6841/744750249] up:boot 2024-09-05T07:08:13.684 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T07:08:13.685 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:13 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T07:08:13.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: pgmap v851: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6832/943997132,v1:172.21.15.96:6833/943997132] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6835/2974739219,v1:172.21.15.96:6838/2974739219] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6834/1163844570,v1:172.21.15.96:6836/1163844570] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.96:6837/1928153236,v1:172.21.15.96:6839/1928153236] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6832/3595349876,v1:172.21.15.115:6833/3595349876] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6834/1019489923,v1:172.21.15.115:6835/1019489923] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6836/1522768726,v1:172.21.15.115:6838/1522768726] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.115:6837/2926515339,v1:172.21.15.115:6839/2926515339] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6838/863999171,v1:172.21.15.86:6840/863999171] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: mds.? [v2:172.21.15.86:6839/744750249,v1:172.21.15.86:6841/744750249] up:boot 2024-09-05T07:08:13.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='mgr.15300 172.21.15.86:0/4023701215' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T07:08:13.739 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:13 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T07:08:14.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-05T07:08:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:14 smithi086 ceph-mon[30943]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T07:08:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:14 smithi086 ceph-mon[30943]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T07:08:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:14 smithi086 ceph-mon[30943]: Cluster is now healthy 2024-09-05T07:08:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:14 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T07:08:14.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:14 smithi086 ceph-mon[30943]: fsmap 12 up:standby 2024-09-05T07:08:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:14 smithi096 ceph-mon[34468]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T07:08:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:14 smithi096 ceph-mon[34468]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T07:08:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:14 smithi096 ceph-mon[34468]: Cluster is now healthy 2024-09-05T07:08:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:14 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T07:08:14.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:14 smithi096 ceph-mon[34468]: fsmap 12 up:standby 2024-09-05T07:08:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:14 smithi115 ceph-mon[34791]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T07:08:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:14 smithi115 ceph-mon[34791]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T07:08:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:14 smithi115 ceph-mon[34791]: Cluster is now healthy 2024-09-05T07:08:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:14 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/1611472469' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T07:08:14.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:14 smithi115 ceph-mon[34791]: fsmap 12 up:standby 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/1553694751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc102dd0 msgr2=0x7f7dcc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/1553694751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc1031b0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009930 tx=0x7f7dc002a420 comp rx=0 tx=0).stop 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/1553694751 shutdown_connections 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/1553694751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc108300 0x7f7dcc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/1553694751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 0x7f7dcc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/1553694751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc1031b0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/1553694751 >> 172.21.15.86:0/1553694751 conn(0x7f7dcc0fc7d0 msgr2=0x7f7dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/1553694751 shutdown_connections 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.939+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/1553694751 wait complete. 2024-09-05T07:08:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 Processor -- start 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 -- start start 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 0x7f7dcc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc111f40 con 0x7f7dcc108300 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc111dc0 con 0x7f7dcc103780 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd2c2f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1120c0 con 0x7f7dcc102dd0 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd09cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57992/0 (socket says 172.21.15.86:57992) 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 -- 172.21.15.86:0/2885385699 learned_addr learned my addr 172.21.15.86:0/2885385699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dcbfff640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 0x7f7dcc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 -- 172.21.15.86:0/2885385699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 msgr2=0x7f7dcc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 -- 172.21.15.86:0/2885385699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 msgr2=0x7f7dcc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.940+0000 7f7dd11cc640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 0x7f7dcc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dd11cc640 1 -- 172.21.15.86:0/2885385699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc19d4b0 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dd09cb640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dd11cc640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009fd0 tx=0x7f7dc400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4017070 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dc4003e20 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc19d7a0 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4010460 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.941+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1a5080 con 0x7f7dcc108300 2024-09-05T07:08:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.942+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc06a7b0 con 0x7f7dcc108300 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.943+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7dc4003970 con 0x7f7dcc108300 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.944+0000 7f7dc9ffb640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 0x7f7dbc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.944+0000 7f7dd09cb640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 0x7f7dbc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.944+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 10035+0+0 (secure 0 0 0) 0x7f7dc40cf7c0 con 0x7f7dcc108300 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.944+0000 7f7dd09cb640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 0x7f7dbc0a5230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009900 tx=0x7f7dc0000bc0 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:14.948+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7dc409bdf0 con 0x7f7dcc108300 2024-09-05T07:08:15.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.113+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7dcc10a290 con 0x7f7dcc108300 2024-09-05T07:08:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.489+0000 7f7dc9ffb640 1 -- 172.21.15.86:0/2885385699 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) ==== 181+0+0 (secure 0 0 0) 0x7f7dc409b6f0 con 0x7f7dcc108300 2024-09-05T07:08:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-09-05T07:08:15.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 msgr2=0x7f7dbc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:15.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 0x7f7dbc0a5230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009900 tx=0x7f7dc0000bc0 comp rx=0 tx=0).stop 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 msgr2=0x7f7dcc19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009fd0 tx=0x7f7dc400b260 comp rx=0 tx=0).stop 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 shutdown_connections 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f7dbc0a2d70 0x7f7dbc0a5230 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dcc108300 0x7f7dcc19cdd0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dcc103780 0x7f7dcc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:15.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 --2- 172.21.15.86:0/2885385699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dcc102dd0 0x7f7dcc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 >> 172.21.15.86:0/2885385699 conn(0x7f7dcc0fc7d0 msgr2=0x7f7dcc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 shutdown_connections 2024-09-05T07:08:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:15.492+0000 7f7dd2c2f640 1 -- 172.21.15.86:0/2885385699 wait complete. 2024-09-05T07:08:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:15 smithi086 ceph-mon[30943]: pgmap v853: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:15.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:15 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:15 smithi096 ceph-mon[34468]: pgmap v853: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:15.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:15 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:15 smithi115 ceph-mon[34791]: pgmap v853: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T07:08:15.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:15 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:16.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-05T07:08:16.494 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:16 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:16.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:16 smithi086 ceph-mon[30943]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T07:08:16.561 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:16 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:16.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:16 smithi096 ceph-mon[34468]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T07:08:16.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:16 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/2885385699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:16.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:16 smithi115 ceph-mon[34791]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T07:08:17.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.057+0000 7f6d02c27640 1 -- 172.21.15.86:0/2105474009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102ef0 msgr2=0x7f6cfc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:17.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.057+0000 7f6d02c27640 1 --2- 172.21.15.86:0/2105474009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc10da40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009ca0 tx=0x7f6cf402a950 comp rx=0 tx=0).stop 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 -- 172.21.15.86:0/2105474009 shutdown_connections 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 --2- 172.21.15.86:0/2105474009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc10da40 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 --2- 172.21.15.86:0/2105474009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc102570 0x7f6cfc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 --2- 172.21.15.86:0/2105474009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc100e20 0x7f6cfc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 -- 172.21.15.86:0/2105474009 >> 172.21.15.86:0/2105474009 conn(0x7f6cfc0fc7d0 msgr2=0x7f6cfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 -- 172.21.15.86:0/2105474009 shutdown_connections 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 -- 172.21.15.86:0/2105474009 wait complete. 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 Processor -- start 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.058+0000 7f6d02c27640 1 -- start start 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc100e20 0x7f6cfc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc110e30 con 0x7f6cfc102570 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc110cb0 con 0x7f6cfc102ef0 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc110fb0 con 0x7f6cfc100e20 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58002/0 (socket says 172.21.15.86:58002) 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 -- 172.21.15.86:0/3028126667 learned_addr learned my addr 172.21.15.86:0/3028126667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d009c3640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc100e20 0x7f6cfc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d011c4640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 -- 172.21.15.86:0/3028126667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc100e20 msgr2=0x7f6cfc1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc100e20 0x7f6cfc1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 -- 172.21.15.86:0/3028126667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc102ef0 msgr2=0x7f6cfc19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 -- 172.21.15.86:0/3028126667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc19c2a0 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cfbfff640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c890 tx=0x7f6ce800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8004220 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ce80043c0 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.059+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc19c590 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.060+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8004a00 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.060+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1a3e70 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.061+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc101200 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.062+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ce8004ba0 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.062+0000 7f6cf9ffb640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 0x7f6cec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.062+0000 7f6d009c3640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 0x7f6cec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.062+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9393+0+0 (secure 0 0 0) 0x7f6ce80cf3a0 con 0x7f6cfc102570 2024-09-05T07:08:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.063+0000 7f6d009c3640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 0x7f6cec0a51b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000bbe0 tx=0x7f6cf000b390 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:17.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.066+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ce8014030 con 0x7f6cfc102570 2024-09-05T07:08:17.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.235+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6cfc198650 con 0x7f6cfc102570 2024-09-05T07:08:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.501+0000 7f6cf9ffb640 1 -- 172.21.15.86:0/3028126667 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) ==== 169+0+0 (secure 0 0 0) 0x7f6cfc198650 con 0x7f6cfc102570 2024-09-05T07:08:17.501 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-09-05T07:08:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 msgr2=0x7f6cec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:17.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 0x7f6cec0a51b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000bbe0 tx=0x7f6cf000b390 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 msgr2=0x7f6cfc197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c890 tx=0x7f6ce800cd50 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 shutdown_connections 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7f6cec0a2cf0 0x7f6cec0a51b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cfc102ef0 0x7f6cfc19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cfc102570 0x7f6cfc197830 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 --2- 172.21.15.86:0/3028126667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cfc100e20 0x7f6cfc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 >> 172.21.15.86:0/3028126667 conn(0x7f6cfc0fc7d0 msgr2=0x7f6cfc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 shutdown_connections 2024-09-05T07:08:17.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:17.503+0000 7f6d02c27640 1 -- 172.21.15.86:0/3028126667 wait complete. 2024-09-05T07:08:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:17 smithi096 ceph-mon[34468]: pgmap v855: 97 pgs: 97 active+clean; 284 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:17.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:17 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:17.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:17 smithi086 ceph-mon[30943]: pgmap v855: 97 pgs: 97 active+clean; 284 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:17.878 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:17 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:17.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:17 smithi115 ceph-mon[34791]: pgmap v855: 97 pgs: 97 active+clean; 284 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:17.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:17 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:18.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-05T07:08:18.503 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T07:08:18.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:18 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:18.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:18 smithi086 ceph-mon[30943]: osdmap e133: 12 total, 12 up, 12 in 2024-09-05T07:08:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:18 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:18.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:18 smithi096 ceph-mon[34468]: osdmap e133: 12 total, 12 up, 12 in 2024-09-05T07:08:18.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:18 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3028126667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:18.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:18 smithi115 ceph-mon[34791]: osdmap e133: 12 total, 12 up, 12 in 2024-09-05T07:08:19.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.990+0000 7fa990c9a640 1 -- 172.21.15.86:0/419268856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c100e20 msgr2=0x7fa98c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:19.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.990+0000 7fa990c9a640 1 --2- 172.21.15.86:0/419268856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c100e20 0x7fa98c101200 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009ca0 tx=0x7fa97c02a950 comp rx=0 tx=0).stop 2024-09-05T07:08:19.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 -- 172.21.15.86:0/419268856 shutdown_connections 2024-09-05T07:08:19.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 --2- 172.21.15.86:0/419268856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c102ef0 0x7fa98c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 --2- 172.21.15.86:0/419268856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 0x7fa98c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 --2- 172.21.15.86:0/419268856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c100e20 0x7fa98c101200 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 -- 172.21.15.86:0/419268856 >> 172.21.15.86:0/419268856 conn(0x7fa98c0fc7d0 msgr2=0x7fa98c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 -- 172.21.15.86:0/419268856 shutdown_connections 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 -- 172.21.15.86:0/419268856 wait complete. 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 Processor -- start 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.991+0000 7fa990c9a640 1 -- start start 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c100e20 0x7fa98c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 0x7fa98c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c110e30 con 0x7fa98c102ef0 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c110cb0 con 0x7fa98c102570 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c110fb0 con 0x7fa98c100e20 2024-09-05T07:08:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58026/0 (socket says 172.21.15.86:58026) 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 -- 172.21.15.86:0/3152870579 learned_addr learned my addr 172.21.15.86:0/3152870579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98a59c640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c100e20 0x7fa98c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa989d9b640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 0x7fa98c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 -- 172.21.15.86:0/3152870579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c100e20 msgr2=0x7fa98c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c100e20 0x7fa98c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 -- 172.21.15.86:0/3152870579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 msgr2=0x7fa98c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 0x7fa98c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 -- 172.21.15.86:0/3152870579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c19c2e0 con 0x7fa98c102ef0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa98ad9d640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c970 tx=0x7fa98400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984007bb0 con 0x7fa98c102ef0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c19c5d0 con 0x7fa98c102ef0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa984007d50 con 0x7fa98c102ef0 2024-09-05T07:08:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984004e10 con 0x7fa98c102ef0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.992+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a3eb0 con 0x7fa98c102ef0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.993+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c101200 con 0x7fa98c102ef0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.994+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa984005020 con 0x7fa98c102ef0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.995+0000 7fa9837fe640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 0x7fa9780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.995+0000 7fa98a59c640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 0x7fa9780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.995+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 8797+0+0 (secure 0 0 0) 0x7fa9840cf1a0 con 0x7fa98c102ef0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.996+0000 7fa98a59c640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 0x7fa9780a51b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa97c004ad0 tx=0x7fa97c004a60 comp rx=0 tx=0).ready entity=mgr.15300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T07:08:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:18.999+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa984014030 con 0x7fa98c102ef0 2024-09-05T07:08:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.167+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa98c109090 con 0x7fa98c102ef0 2024-09-05T07:08:19.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.507+0000 7fa9837fe640 1 -- 172.21.15.86:0/3152870579 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v134) ==== 178+0+0 (secure 0 0 0) 0x7fa98409b660 con 0x7fa98c102ef0 2024-09-05T07:08:19.508 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' removed 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 msgr2=0x7fa9780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 0x7fa9780a51b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa97c004ad0 tx=0x7fa97c004a60 comp rx=0 tx=0).stop 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 msgr2=0x7fa98c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c970 tx=0x7fa98400ce30 comp rx=0 tx=0).stop 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 shutdown_connections 2024-09-05T07:08:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:6800/388818328,v1:172.21.15.86:6801/388818328] conn(0x7fa9780a2cf0 0x7fa9780a51b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c102ef0 0x7fa98c19bc00 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa98c102570 0x7fa98c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 --2- 172.21.15.86:0/3152870579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa98c100e20 0x7fa98c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T07:08:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.511+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 >> 172.21.15.86:0/3152870579 conn(0x7fa98c0fc7d0 msgr2=0x7fa98c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T07:08:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.512+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 shutdown_connections 2024-09-05T07:08:19.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T07:08:19.512+0000 7fa990c9a640 1 -- 172.21.15.86:0/3152870579 wait complete. 2024-09-05T07:08:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:19 smithi096 ceph-mon[34468]: pgmap v857: 33 pgs: 33 active+clean; 284 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:19.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:19 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:19.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:19 smithi086 ceph-mon[30943]: pgmap v857: 33 pgs: 33 active+clean; 284 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:19.929 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:19 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:19.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:19 smithi115 ceph-mon[34791]: pgmap v857: 33 pgs: 33 active+clean; 284 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T07:08:19.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:19 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T07:08:20.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:08:20.293 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:08:20.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:08:20.362 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-05T07:08:20.362 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T07:08:20.378 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T07:08:20.494 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T07:08:20.512 INFO:tasks.cephadm:Stopping all daemons... 2024-09-05T07:08:20.512 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-05T07:08:20.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a 2024-09-05T07:08:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:20 smithi115 ceph-mon[34791]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:20.683 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:20 smithi115 ceph-mon[34791]: osdmap e134: 12 total, 12 up, 12 in 2024-09-05T07:08:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:20 smithi096 ceph-mon[34468]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:20.807 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:20 smithi096 ceph-mon[34468]: osdmap e134: 12 total, 12 up, 12 in 2024-09-05T07:08:20.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 ceph-mon[30943]: from='client.? 172.21.15.86:0/3152870579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T07:08:20.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 ceph-mon[30943]: osdmap e134: 12 total, 12 up, 12 in 2024-09-05T07:08:20.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 systemd[1]: Stopping Ceph mon.a for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:21.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a[30939]: 2024-09-05T07:08:20.851+0000 7f4e9694f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:21.203 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a[30939]: 2024-09-05T07:08:20.851+0000 7f4e9694f640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-05T07:08:21.203 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:20 smithi086 podman[149105]: 2024-09-05 07:08:20.892521381 +0000 UTC m=+0.169391336 container died 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T07:08:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:21 smithi086 podman[149105]: 2024-09-05 07:08:21.310398252 +0000 UTC m=+0.587268201 container cleanup 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-05T07:08:21.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:21 smithi086 bash[149105]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a 2024-09-05T07:08:21.873 INFO:journalctl@ceph.mon.a.smithi086.stdout:Sep 05 07:08:21 smithi086 podman[149116]: 2024-09-05 07:08:21.578789911 +0000 UTC m=+0.689285084 container remove 07527dcb92f7be4b11b2cf6c53ba8a36a9037d147ef35a350fb3b276e02c545a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-05T07:08:21.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.a.service' 2024-09-05T07:08:21.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:21.942 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-05T07:08:21.942 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-05T07:08:21.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.b 2024-09-05T07:08:22.288 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:21 smithi096 systemd[1]: Stopping Ceph mon.b for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-b[34464]: 2024-09-05T07:08:22.287+0000 7f2d40ec3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:22.557 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-b[34464]: 2024-09-05T07:08:22.287+0000 7f2d40ec3640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-05T07:08:22.856 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:22 smithi096 podman[75928]: 2024-09-05 07:08:22.574005703 +0000 UTC m=+0.410093565 container died e06d482d9f347067f8c139b3ec15c4cbff370b2d12ad7fccbd09dbf8ccfbddbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:23.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:22 smithi096 podman[75928]: 2024-09-05 07:08:22.955374809 +0000 UTC m=+0.791462665 container cleanup e06d482d9f347067f8c139b3ec15c4cbff370b2d12ad7fccbd09dbf8ccfbddbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T07:08:23.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 05 07:08:22 smithi096 bash[75928]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-b 2024-09-05T07:08:23.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.b.service' 2024-09-05T07:08:23.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:23.443 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-05T07:08:23.443 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-05T07:08:23.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.c 2024-09-05T07:08:23.837 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:23 smithi115 systemd[1]: Stopping Ceph mon.c for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:24.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c[34787]: 2024-09-05T07:08:23.836+0000 7f53f2e77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:24.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c[34787]: 2024-09-05T07:08:23.836+0000 7f53f2e77640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-05T07:08:24.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:23 smithi115 podman[77872]: 2024-09-05 07:08:23.908070512 +0000 UTC m=+0.187355283 container died 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T07:08:24.555 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 podman[77872]: 2024-09-05 07:08:24.29537969 +0000 UTC m=+0.574664460 container cleanup 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-05T07:08:24.555 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 bash[77872]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c 2024-09-05T07:08:24.792 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.c.service' 2024-09-05T07:08:24.814 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 podman[77884]: 2024-09-05 07:08:24.55569079 +0000 UTC m=+0.650172980 container remove 10e8b9151908dfd1e6b5e00cf885ef9eb981e498424194cedcc69e1adba34f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T07:08:24.814 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-05T07:08:24.814 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 systemd[1]: Stopped Ceph mon.c for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T07:08:24.815 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 05 07:08:24 smithi115 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mon.c.service: Consumed 19.664s CPU time. 2024-09-05T07:08:24.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:24.838 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-05T07:08:24.838 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-05T07:08:24.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T07:08:25.172 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:24 smithi086 systemd[1]: Stopping Ceph mgr.x for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:25.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T07:08:25.172+0000 7f909c734640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:25.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x[31172]: 2024-09-05T07:08:25.172+0000 7f909c734640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-05T07:08:25.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:25 smithi086 podman[149226]: 2024-09-05 07:08:25.203393973 +0000 UTC m=+0.176325765 container died 058d60dfd4a848a82efbb25c9cd107b5ec4feb052f40dcf70a17327368c4909f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T07:08:25.975 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:25 smithi086 podman[149226]: 2024-09-05 07:08:25.715598716 +0000 UTC m=+0.688530506 container cleanup 058d60dfd4a848a82efbb25c9cd107b5ec4feb052f40dcf70a17327368c4909f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T07:08:25.975 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Sep 05 07:08:25 smithi086 bash[149226]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-x 2024-09-05T07:08:26.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.x.service' 2024-09-05T07:08:26.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:26.175 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-05T07:08:26.175 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-05T07:08:26.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.y 2024-09-05T07:08:26.505 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:26 smithi096 systemd[1]: Stopping Ceph mgr.y for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:26.807 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:26 smithi096 podman[76050]: 2024-09-05 07:08:26.506172139 +0000 UTC m=+0.139934063 container died fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:27.809 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:27 smithi096 podman[76050]: 2024-09-05 07:08:27.428099276 +0000 UTC m=+1.061861200 container cleanup fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T07:08:27.809 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:27 smithi096 bash[76050]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y 2024-09-05T07:08:27.809 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:27 smithi096 podman[76061]: 2024-09-05 07:08:27.638162513 +0000 UTC m=+1.126604934 container remove fb5b4d804b903e0481fee07b88853859731d39f0647f0a86837428bb1e1bd1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T07:08:27.809 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 05 07:08:27 smithi096 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-05T07:08:27.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.y.service' 2024-09-05T07:08:27.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:27.885 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-05T07:08:27.885 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-05T07:08:27.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.z 2024-09-05T07:08:28.183 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 07:08:27 smithi115 systemd[1]: Stopping Ceph mgr.z for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:28.683 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 07:08:28 smithi115 podman[77995]: 2024-09-05 07:08:28.431398288 +0000 UTC m=+0.346630450 container died dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e) 2024-09-05T07:08:29.085 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 07:08:28 smithi115 podman[77995]: 2024-09-05 07:08:28.834199504 +0000 UTC m=+0.749431681 container cleanup dbbbd764a2c6ec7930870fb81fb27d3641a05f76e182561b4e05f002607b20bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-05T07:08:29.085 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Sep 05 07:08:28 smithi115 bash[77995]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mgr-z 2024-09-05T07:08:29.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mgr.z.service' 2024-09-05T07:08:29.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:29.287 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-05T07:08:29.287 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-05T07:08:29.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.0 2024-09-05T07:08:29.604 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:29 smithi086 systemd[1]: Stopping Ceph osd.0 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:29.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T07:08:29.604+0000 7f64aaef8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:29.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T07:08:29.604+0000 7f64aaef8640 -1 osd.0 134 *** Got signal Terminated *** 2024-09-05T07:08:29.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0[42074]: 2024-09-05T07:08:29.604+0000 7f64aaef8640 -1 osd.0 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:08:35.302 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:35 smithi086 podman[149345]: 2024-09-05 07:08:35.042871605 +0000 UTC m=+5.551379879 container died 0fc10a7af3cae11a8b90ef32db8a009b55ba6d85b5ef51c6c01cf365383dd4ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:35.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:35 smithi086 podman[149345]: 2024-09-05 07:08:35.580830614 +0000 UTC m=+6.089338886 container cleanup 0fc10a7af3cae11a8b90ef32db8a009b55ba6d85b5ef51c6c01cf365383dd4ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731) 2024-09-05T07:08:35.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:35 smithi086 bash[149345]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0 2024-09-05T07:08:35.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:35 smithi086 podman[149363]: 2024-09-05 07:08:35.807638197 +0000 UTC m=+0.748859296 container remove 0fc10a7af3cae11a8b90ef32db8a009b55ba6d85b5ef51c6c01cf365383dd4ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:08:36.520 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.145142247 +0000 UTC m=+0.016019158 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:08:36.520 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.266957589 +0000 UTC m=+0.137834489 container create ab03f199c81781fb9ed6a6df49adc0ddce54f2cf1e6a5f5e9a05ea1743786927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T07:08:36.520 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.520078592 +0000 UTC m=+0.390955501 container init ab03f199c81781fb9ed6a6df49adc0ddce54f2cf1e6a5f5e9a05ea1743786927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:08:36.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.523777217 +0000 UTC m=+0.394654123 container start ab03f199c81781fb9ed6a6df49adc0ddce54f2cf1e6a5f5e9a05ea1743786927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T07:08:36.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.620024284 +0000 UTC m=+0.490901181 container attach ab03f199c81781fb9ed6a6df49adc0ddce54f2cf1e6a5f5e9a05ea1743786927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:08:36.987 INFO:journalctl@ceph.osd.0.smithi086.stdout:Sep 05 07:08:36 smithi086 podman[149453]: 2024-09-05 07:08:36.725671385 +0000 UTC m=+0.596548301 container died ab03f199c81781fb9ed6a6df49adc0ddce54f2cf1e6a5f5e9a05ea1743786927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e) 2024-09-05T07:08:37.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.0.service' 2024-09-05T07:08:37.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:37.591 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-05T07:08:37.591 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-05T07:08:37.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.1 2024-09-05T07:08:37.908 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:37 smithi096 systemd[1]: Stopping Ceph osd.1 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:38.307 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T07:08:37.907+0000 7f904d170640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:38.307 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T07:08:37.907+0000 7f904d170640 -1 osd.1 134 *** Got signal Terminated *** 2024-09-05T07:08:38.307 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1[38887]: 2024-09-05T07:08:37.907+0000 7f904d170640 -1 osd.1 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:08:43.751 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:43 smithi096 podman[76188]: 2024-09-05 07:08:43.445583163 +0000 UTC m=+5.660073277 container stop a155f923dcdf6eb8e5954a7611dbec3ffd336c183166403e604aad1d6c5704d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T07:08:43.751 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:43 smithi096 podman[76188]: 2024-09-05 07:08:43.446058638 +0000 UTC m=+5.660548750 container died a155f923dcdf6eb8e5954a7611dbec3ffd336c183166403e604aad1d6c5704d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T07:08:44.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:43 smithi096 podman[76188]: 2024-09-05 07:08:43.849383551 +0000 UTC m=+6.063873662 container cleanup a155f923dcdf6eb8e5954a7611dbec3ffd336c183166403e604aad1d6c5704d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1, RELEASE=squid-91c8a7e, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-05T07:08:44.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:43 smithi096 bash[76188]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1 2024-09-05T07:08:44.460 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76204]: 2024-09-05 07:08:44.13428128 +0000 UTC m=+0.681121912 container remove a155f923dcdf6eb8e5954a7611dbec3ffd336c183166403e604aad1d6c5704d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e) 2024-09-05T07:08:44.729 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.363823018 +0000 UTC m=+0.015987706 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:08:44.730 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.493480488 +0000 UTC m=+0.145645166 container create dfc2b2f5e1c744491a38b7d9754eaacd63048f496c7f56d10eb3a1fc11c592a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-05T07:08:45.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.730520059 +0000 UTC m=+0.382684745 container init dfc2b2f5e1c744491a38b7d9754eaacd63048f496c7f56d10eb3a1fc11c592a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T07:08:45.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.734156559 +0000 UTC m=+0.386321245 container start dfc2b2f5e1c744491a38b7d9754eaacd63048f496c7f56d10eb3a1fc11c592a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-05T07:08:45.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.830139046 +0000 UTC m=+0.482303739 container attach dfc2b2f5e1c744491a38b7d9754eaacd63048f496c7f56d10eb3a1fc11c592a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T07:08:45.057 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 05 07:08:44 smithi096 podman[76295]: 2024-09-05 07:08:44.920923054 +0000 UTC m=+0.573087748 container died dfc2b2f5e1c744491a38b7d9754eaacd63048f496c7f56d10eb3a1fc11c592a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:45.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.1.service' 2024-09-05T07:08:45.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:45.641 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-05T07:08:45.642 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-05T07:08:45.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.2 2024-09-05T07:08:45.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:45 smithi115 systemd[1]: Stopping Ceph osd.2 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:46.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T07:08:45.970+0000 7fad6a981640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:46.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T07:08:45.970+0000 7fad6a981640 -1 osd.2 134 *** Got signal Terminated *** 2024-09-05T07:08:46.433 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2[39709]: 2024-09-05T07:08:45.970+0000 7fad6a981640 -1 osd.2 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:08:51.656 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:51 smithi115 podman[78131]: 2024-09-05 07:08:51.354465394 +0000 UTC m=+5.514570372 container died 1038e3565477b66fbe5683c37d0d8701288ed60783d97e47327c69b8ed2cbeb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T07:08:51.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:51 smithi115 podman[78131]: 2024-09-05 07:08:51.721220335 +0000 UTC m=+5.881325313 container cleanup 1038e3565477b66fbe5683c37d0d8701288ed60783d97e47327c69b8ed2cbeb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:51.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:51 smithi115 bash[78131]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2 2024-09-05T07:08:52.382 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78147]: 2024-09-05 07:08:52.039169569 +0000 UTC m=+0.680912652 container remove 1038e3565477b66fbe5683c37d0d8701288ed60783d97e47327c69b8ed2cbeb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-05T07:08:52.676 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.285331879 +0000 UTC m=+0.016068473 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:08:52.676 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.406796651 +0000 UTC m=+0.137533245 container create 5ddbed55d4160b0d6e151c5623bebdd519f719ab0a3bc68f770cc696da973124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:08:52.676 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.676694105 +0000 UTC m=+0.407430702 container init 5ddbed55d4160b0d6e151c5623bebdd519f719ab0a3bc68f770cc696da973124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-05T07:08:52.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.680368722 +0000 UTC m=+0.411105317 container start 5ddbed55d4160b0d6e151c5623bebdd519f719ab0a3bc68f770cc696da973124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:52.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.784848047 +0000 UTC m=+0.515584642 container attach 5ddbed55d4160b0d6e151c5623bebdd519f719ab0a3bc68f770cc696da973124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:08:52.933 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 05 07:08:52 smithi115 podman[78237]: 2024-09-05 07:08:52.875829967 +0000 UTC m=+0.606566564 container died 5ddbed55d4160b0d6e151c5623bebdd519f719ab0a3bc68f770cc696da973124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:08:53.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.2.service' 2024-09-05T07:08:53.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:08:53.530 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-05T07:08:53.530 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-05T07:08:53.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.3 2024-09-05T07:08:53.861 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:53 smithi086 systemd[1]: Stopping Ceph osd.3 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:08:54.237 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T07:08:53.861+0000 7ffabd284640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:08:54.237 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T07:08:53.861+0000 7ffabd284640 -1 osd.3 134 *** Got signal Terminated *** 2024-09-05T07:08:54.237 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:53 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T07:08:53.861+0000 7ffabd284640 -1 osd.3 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:08:58.237 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:57 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3[48546]: 2024-09-05T07:08:57.981+0000 7ffab5892640 -1 osd.3 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.999430+0000 front 2024-09-05T07:08:32.999461+0000 (oldest deadline 2024-09-05T07:08:57.699564+0000) 2024-09-05T07:08:58.346 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:08:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:08:58.040+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:08:58.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:08:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:08:58.346+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:08:59.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:08:58 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:08:58.973+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:08:59.433 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:08:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:08:59.309+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:08:59.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:08:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:08:59.064+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:08:59.737 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:59 smithi086 podman[149566]: 2024-09-05 07:08:59.467173952 +0000 UTC m=+5.742660228 container died 0071b51cd474512431b6bc13bdddc0509d07457544066922c19d750ae1e8ff76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T07:09:00.221 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:59 smithi086 podman[149566]: 2024-09-05 07:08:59.953399724 +0000 UTC m=+6.228885999 container cleanup 0071b51cd474512431b6bc13bdddc0509d07457544066922c19d750ae1e8ff76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-05T07:09:00.221 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:08:59 smithi086 bash[149566]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3 2024-09-05T07:09:00.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:08:59 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:08:59.943+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:00.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:00.108+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:00.433 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:00.349+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:00.487 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149583]: 2024-09-05 07:09:00.221672865 +0000 UTC m=+0.753140739 container remove 0071b51cd474512431b6bc13bdddc0509d07457544066922c19d750ae1e8ff76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T07:09:00.866 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:00 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:00.644+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:00.867 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149673]: 2024-09-05 07:09:00.469656105 +0000 UTC m=+0.015972549 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:00.867 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149673]: 2024-09-05 07:09:00.605923233 +0000 UTC m=+0.152239676 container create 90cfe41dd9d8aa24ad0702ccf26c86b8f91c749cef2d97369a5554f4d00a5b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:01.237 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149673]: 2024-09-05 07:09:00.867024271 +0000 UTC m=+0.413340716 container init 90cfe41dd9d8aa24ad0702ccf26c86b8f91c749cef2d97369a5554f4d00a5b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0) 2024-09-05T07:09:01.238 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149673]: 2024-09-05 07:09:00.870353568 +0000 UTC m=+0.416670014 container start 90cfe41dd9d8aa24ad0702ccf26c86b8f91c749cef2d97369a5554f4d00a5b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3-deactivate, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T07:09:01.238 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:00 smithi086 podman[149673]: 2024-09-05 07:09:00.950309248 +0000 UTC m=+0.496625714 container attach 90cfe41dd9d8aa24ad0702ccf26c86b8f91c749cef2d97369a5554f4d00a5b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-05T07:09:01.238 INFO:journalctl@ceph.osd.3.smithi086.stdout:Sep 05 07:09:01 smithi086 podman[149673]: 2024-09-05 07:09:01.053275837 +0000 UTC m=+0.599592285 container died 90cfe41dd9d8aa24ad0702ccf26c86b8f91c749cef2d97369a5554f4d00a5b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-05T07:09:01.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:00 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:00.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:01.382 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:01.150+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:01.382 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:01.071+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:01.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:01.381+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:01.704 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:01 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:01.634+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:01.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.3.service' 2024-09-05T07:09:01.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:01.776 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-05T07:09:01.777 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-05T07:09:01.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.4 2024-09-05T07:09:02.057 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:01 smithi096 systemd[1]: Stopping Ceph osd.4 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:02.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:01 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:01.917+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:02.381 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:02.186+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:02.382 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:02.039+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:02.557 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:02 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T07:09:02.094+0000 7f26c159d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:02.557 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:02 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T07:09:02.094+0000 7f26c159d640 -1 osd.4 134 *** Got signal Terminated *** 2024-09-05T07:09:02.557 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:02 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T07:09:02.094+0000 7f26c159d640 -1 osd.4 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:02.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:02.382+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:02.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:02 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:02.604+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:03.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:02 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:02.954+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:03.431 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:03 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:03.058+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:03.432 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:03 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:03.149+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:03.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:03 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:03.430+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:03.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:03 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:03.616+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:04.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:03 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:03.998+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:04.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:04 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:04.102+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:04.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:04 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:04.122+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:04.758 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:04 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:04.458+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:04.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:04 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:04.648+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:04.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:04 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:04.648+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:05.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:05 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:05.040+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:05.382 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:05 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:05.150+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:05.382 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:05 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:05.111+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:05.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:05 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:05.504+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:05.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:05 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:05.504+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:05.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:05 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:05.411+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:05.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:05 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:05.619+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:05.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:05 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:05.619+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:06.307 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:06 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T07:09:06.001+0000 7f26b9bab640 -1 osd.4 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.342015+0000 front 2024-09-05T07:08:40.342071+0000 (oldest deadline 2024-09-05T07:09:05.042269+0000) 2024-09-05T07:09:06.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:05 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:05.997+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:06.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:06.100+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:06.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:06.117+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:06.687 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:06 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:06.428+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:06.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:06.513+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:06.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:06 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:06.513+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:06.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:06.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:06.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:06 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:06.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:07.307 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4[45326]: 2024-09-05T07:09:07.012+0000 7f26b9bab640 -1 osd.4 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.342015+0000 front 2024-09-05T07:08:40.342071+0000 (oldest deadline 2024-09-05T07:09:05.042269+0000) 2024-09-05T07:09:07.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:07.016+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:07.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:07.100+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:07.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:07.112+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:07.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:07.421+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:07.807 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:07 smithi096 podman[76398]: 2024-09-05 07:09:07.549019589 +0000 UTC m=+5.574351664 container died 7d258aeaeda7c7797405563f77ccb26220eba2575edfab351256a76f76d77a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:07.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:07.556+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:07.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:07.556+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:07.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:07.527+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:07.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:07 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:07.527+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:08.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:07 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:07.820+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:08.237 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:07 smithi096 podman[76398]: 2024-09-05 07:09:07.968938615 +0000 UTC m=+5.994270690 container cleanup 7d258aeaeda7c7797405563f77ccb26220eba2575edfab351256a76f76d77a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T07:09:08.238 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:07 smithi096 bash[76398]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4 2024-09-05T07:09:08.238 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:07.973+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:08.238 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:07 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:07.973+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:08.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:08.130+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:08.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:08.130+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:08.436 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:08.152+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:08.436 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:08.152+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:08.557 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:08 smithi096 podman[76425]: 2024-09-05 07:09:08.238083507 +0000 UTC m=+0.681990918 container remove 7d258aeaeda7c7797405563f77ccb26220eba2575edfab351256a76f76d77a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e) 2024-09-05T07:09:08.557 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:08 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:08.400+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:08.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:08.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:08.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:08.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:08.925 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:08 smithi096 podman[76512]: 2024-09-05 07:09:08.533348266 +0000 UTC m=+0.015435022 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:08.925 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:08 smithi096 podman[76512]: 2024-09-05 07:09:08.66770408 +0000 UTC m=+0.149790866 container create a7282568ae1da20daee747db0661d06e7c2063ddb63e77ed0dc6067368f02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-05T07:09:08.932 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:08.561+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:08.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:08 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:08.561+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:09.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:08.829+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:09.307 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:08 smithi096 podman[76512]: 2024-09-05 07:09:08.925490303 +0000 UTC m=+0.407577047 container init a7282568ae1da20daee747db0661d06e7c2063ddb63e77ed0dc6067368f02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:09.307 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:08 smithi096 podman[76512]: 2024-09-05 07:09:08.929145529 +0000 UTC m=+0.411232281 container start a7282568ae1da20daee747db0661d06e7c2063ddb63e77ed0dc6067368f02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T07:09:09.307 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:09 smithi096 podman[76512]: 2024-09-05 07:09:09.025473563 +0000 UTC m=+0.507560316 container attach a7282568ae1da20daee747db0661d06e7c2063ddb63e77ed0dc6067368f02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:09.308 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 05 07:09:09 smithi096 podman[76512]: 2024-09-05 07:09:09.117872492 +0000 UTC m=+0.599959249 container died a7282568ae1da20daee747db0661d06e7c2063ddb63e77ed0dc6067368f02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T07:09:09.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:08 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:08.975+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:09.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:08 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:08.975+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:09.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:09.173+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:09.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:09.173+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:09.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:09.119+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:09.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:09.119+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:09.762 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:09.442+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:09.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.4.service' 2024-09-05T07:09:09.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:09.832 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-05T07:09:09.832 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-05T07:09:09.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.5 2024-09-05T07:09:09.860 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:09.598+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:09.860 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:09 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:09.598+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:09.874 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:09.489+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:09.875 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:09.489+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:10.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:09.930+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:10.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:09 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:09.930+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:10.146 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:09 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:09.874+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:09 smithi115 systemd[1]: Stopping Ceph osd.5 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:10.094+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:10.094+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:10.157+0000 7f4592b23640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:10.157+0000 7f4592b23640 -1 osd.5 134 *** Got signal Terminated *** 2024-09-05T07:09:10.158 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:10.157+0000 7f4592b23640 -1 osd.5 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:10.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:10.223+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:10.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:10.223+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:10.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:10.492+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:10.851 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:10.504+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:10.851 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:10.504+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:10.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:10.612+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:10.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:10 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:10.612+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:11.101 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:10 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:10.851+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:10.939+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:11.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:10 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:10.939+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:11.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:11.268+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:11.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:11.268+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:11.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:11.055+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:11.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:11.055+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:11.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:11.456+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:11.892 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:11.492+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:11.893 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:11.492+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:11.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:11.619+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:11.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:11 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:11.619+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:12.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:11 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:11.892+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:11.950+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:12.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:11 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:11.950+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:12.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:12.298+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:12.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:12.298+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:12.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:12.090+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:12.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:12.090+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:12.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:12.433+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:12.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:12.433+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:12.896 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:12.531+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:12.897 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:12.531+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:12.897 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:12.531+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:12.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:12.627+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:12.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:12 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:12.627+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:13.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:12.896+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:13.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:12 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:12.896+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:12.973+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:13.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:12 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:12.973+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:13.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:13.114+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:13.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:13.115+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:13.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:13.278+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:13.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:13.278+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:13.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:13.278+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:13.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:13.398+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:13.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:13.398+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:13.926 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:13.510+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:13.926 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:13.510+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:13.926 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:13.510+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:13.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:13.623+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:13.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:13 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:13.623+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:14.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:13.926+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:14.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:13.926+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:13.966+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:14.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:13 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:13.966+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:14.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:14.101+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:14.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:14.101+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:14.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:14.101+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:49.037633+0000 front 2024-09-05T07:08:49.037610+0000 (oldest deadline 2024-09-05T07:09:13.137711+0000) 2024-09-05T07:09:14.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:14.283+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:14.434 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:14.283+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:14.434 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:14.283+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:14.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:14.435+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:14.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:14 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:14.435+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:14.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:14.616+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:14.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:14 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:14.616+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:14.948 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:14.536+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:14.948 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:14.536+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:14.948 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:14.536+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:15.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:14.948+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:15.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:14 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:14.948+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:15.014+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:15.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:15.014+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:15.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:15.148+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:34.435251+0000 front 2024-09-05T07:08:34.435331+0000 (oldest deadline 2024-09-05T07:09:00.335205+0000) 2024-09-05T07:09:15.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:15.148+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.436542+0000 front 2024-09-05T07:08:41.436558+0000 (oldest deadline 2024-09-05T07:09:07.336494+0000) 2024-09-05T07:09:15.433 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5[46437]: 2024-09-05T07:09:15.148+0000 7f458f139640 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:49.037633+0000 front 2024-09-05T07:08:49.037610+0000 (oldest deadline 2024-09-05T07:09:13.137711+0000) 2024-09-05T07:09:15.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:15.301+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:15.434 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:15.301+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:15.434 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:15.301+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:15.810 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:15.474+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:15.810 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:15.474+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:15.846 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:15 smithi115 podman[78365]: 2024-09-05 07:09:15.658777917 +0000 UTC m=+5.629893483 container died b90fb8878d047882839b82f5fa9df5181a435df6a95ad0dd0df5cb7b9d5918f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T07:09:15.847 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:15.578+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:15.847 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:15.578+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:15.847 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:15 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:15.578+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:15.956 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:15.496+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:15.957 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:15.496+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:15.957 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:15.496+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:16.183 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78365]: 2024-09-05 07:09:16.057702633 +0000 UTC m=+6.028818206 container cleanup b90fb8878d047882839b82f5fa9df5181a435df6a95ad0dd0df5cb7b9d5918f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T07:09:16.183 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 bash[78365]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5 2024-09-05T07:09:16.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:15.956+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:16.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:15 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:15.956+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:15.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:16.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:15 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:15.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:16.617 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78383]: 2024-09-05 07:09:16.359106984 +0000 UTC m=+0.701837356 container remove b90fb8878d047882839b82f5fa9df5181a435df6a95ad0dd0df5cb7b9d5918f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T07:09:16.617 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:16.256+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:16.617 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:16.256+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:16.618 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:16.256+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:16.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:16 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:16.464+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:16.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:16 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:16.464+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:16.881 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78471]: 2024-09-05 07:09:16.575853299 +0000 UTC m=+0.015690564 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:16.881 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78471]: 2024-09-05 07:09:16.728755653 +0000 UTC m=+0.168592958 container create b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-05T07:09:16.881 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:16.616+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:16.881 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:16.616+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:16.881 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:16 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:16.616+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:16.973 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:16.513+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:16.973 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:16.513+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:16.973 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:16.513+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:17.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:16.973+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:17.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:16.973+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:17.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78471]: 2024-09-05 07:09:16.996205132 +0000 UTC m=+0.436042401 container init b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:17.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:16 smithi115 podman[78471]: 2024-09-05 07:09:16.999583045 +0000 UTC m=+0.439420312 container start b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-05T07:09:17.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 podman[78471]: 2024-09-05 07:09:17.079616709 +0000 UTC m=+0.519453994 container attach b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T07:09:17.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 podman[78471]: 2024-09-05 07:09:17.189673741 +0000 UTC m=+0.629511014 container died b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:09:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:16 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:16.960+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:16 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:16.960+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:17.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:16 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:16.960+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:17.653 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:17.304+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:17.653 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:17.304+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:17.653 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:17.304+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:17.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:17.446+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:17.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:17.446+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:17.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.5.service' 2024-09-05T07:09:17.905 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 podman[78494]: 2024-09-05 07:09:17.808419963 +0000 UTC m=+0.612819470 container remove b04663b865b2cdc05159c5128d4e012e38182763c5a279812e355ff6a8c67dde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:17.905 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-05T07:09:17.905 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 systemd[1]: Stopped Ceph osd.5 for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T07:09:17.905 INFO:journalctl@ceph.osd.5.smithi115.stdout:Sep 05 07:09:17 smithi115 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.5.service: Consumed 3min 52.211s CPU time. 2024-09-05T07:09:17.906 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:17.652+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:17.906 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:17.652+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:17.906 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:17 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:17.652+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:17.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:17.924 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-05T07:09:17.924 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-05T07:09:17.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.6 2024-09-05T07:09:17.934 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:17 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:17.543+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:17.935 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:17 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:17.543+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:17.935 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:17 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:17.543+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:18.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:17 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:17.934+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:18.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:17 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:17.934+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:18.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:17 smithi086 systemd[1]: Stopping Ceph osd.6 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:17.970+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:17.970+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:18.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:17 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:17.970+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:18.583 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:18.253+0000 7f2fca20c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:18.583 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:18.253+0000 7f2fca20c640 -1 osd.6 134 *** Got signal Terminated *** 2024-09-05T07:09:18.583 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:18.253+0000 7f2fca20c640 -1 osd.6 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:18.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:18.287+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:18.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:18.287+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:18.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:18.287+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:18.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:18 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:18.486+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:18.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:18 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:18.486+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:18.943 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:18.583+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:18.943 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:18.583+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:18.943 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:18.583+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:19.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:18.693+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:19.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:18.693+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:19.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:18 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:18.693+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:19.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:18.943+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:19.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:18 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:18.943+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:18 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:18.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:18 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:18.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:19.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:18 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:18.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:19.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:19.298+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:19.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:19.298+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:19.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:19.298+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:19.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:19 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:19.514+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:19.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:19 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:19.514+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:19.936 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:19 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:19.568+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:19.936 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:19 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:19.568+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:19.937 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:19 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:19.568+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:20.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:19.732+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:20.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:19.732+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:20.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:19 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:19.732+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:20.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:19 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:19.936+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:20.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:19 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:19.936+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:20.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:19 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:19.938+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:20.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:19 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:19.938+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:20.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:19 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:19.938+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:20.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:20.297+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:20.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:20.297+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:20.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:20.297+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:20.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:20 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:20.467+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:20.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:20 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:20.467+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:20.893 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:20.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:20.893 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:20.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:20.893 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:20.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:20.893 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:20.526+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:21.165 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:20.892+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:21.165 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:20 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:20.892+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:21.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:20.702+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:21.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:20.702+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:21.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:20 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:20.702+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:20 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:20.936+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:20 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:20.936+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:21.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:20 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:20.936+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:21.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:21.313+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:21.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:21.313+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:21.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:21.313+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:21.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:21.471+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:21.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:21.471+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:21.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:21.471+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:21.939 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:21.524+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:21.939 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:21.524+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:21.939 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:21.524+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:21.940 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:21.524+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:22.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:21.694+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:22.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:21.694+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:22.185 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:21 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:21.694+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:22.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:21.939+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:22.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:21 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:21.939+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:21.984+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:22.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:21.984+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:22.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:21.984+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:22.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:22.302+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:22.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:22.303+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:22.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:22.303+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:22.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:22.480+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:22.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:22.480+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:22.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:22.480+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:22.890 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:22.565+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:22.890 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:22.565+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:22.890 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:22.565+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:22.891 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:22.565+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:23.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:22.708+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:23.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:22.708+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:23.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:22.708+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:23.183 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:22 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:22.708+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:23.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:22.890+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.768017+0000 front 2024-09-05T07:08:41.768042+0000 (oldest deadline 2024-09-05T07:09:07.068230+0000) 2024-09-05T07:09:23.237 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:22 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6[55449]: 2024-09-05T07:09:22.890+0000 7f2fc281a640 -1 osd.6 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:48.669799+0000 front 2024-09-05T07:08:48.669926+0000 (oldest deadline 2024-09-05T07:09:12.769982+0000) 2024-09-05T07:09:23.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:22.958+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:23.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:22.958+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:23.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:22 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:22.958+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:23.670 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:23.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:23.670 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:23.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:23.670 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:23.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:23.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:23.472+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:23.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:23.472+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:23.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:23.472+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:23.860 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:23 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:23.582+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:23.860 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:23 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:23.582+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:23.860 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:23 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:23.582+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:23.860 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:23 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:23.582+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:23.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:23.669+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:23.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:23.669+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:23.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:23.669+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:23.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:23 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:23.669+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:24.174 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:23 smithi086 podman[149787]: 2024-09-05 07:09:23.860021051 +0000 UTC m=+5.738225277 container died ab9a4b41cfdb92ae84df9de9ee4d2a924878275c0bfc5a89a04dba631564ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:24.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:23.998+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:24.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:23.998+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:24.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:23 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:23.998+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:24.487 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:24 smithi086 podman[149787]: 2024-09-05 07:09:24.374227496 +0000 UTC m=+6.252431768 container cleanup ab9a4b41cfdb92ae84df9de9ee4d2a924878275c0bfc5a89a04dba631564ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:24.487 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:24 smithi086 bash[149787]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6 2024-09-05T07:09:24.634 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:24.329+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:24.634 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:24.329+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:24.634 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:24.329+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:24.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:24.447+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:24.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:24.447+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:24.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:24.447+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:24.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:24.633+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:24.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:24.633+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:24.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:24.633+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:24.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:24 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:24.633+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:24.975 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:24 smithi086 podman[149803]: 2024-09-05 07:09:24.667547636 +0000 UTC m=+0.803720380 container remove ab9a4b41cfdb92ae84df9de9ee4d2a924878275c0bfc5a89a04dba631564ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e) 2024-09-05T07:09:24.975 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:24 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:24.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:24.975 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:24 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:24.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:24.975 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:24 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:24.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:24.975 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:24 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:24.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:25.229 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:24 smithi086 podman[149891]: 2024-09-05 07:09:24.877920781 +0000 UTC m=+0.016060988 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:25.229 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:24 smithi086 podman[149891]: 2024-09-05 07:09:24.984992113 +0000 UTC m=+0.123132359 container create 8c3d4fa43c9735905b25e75c45867d1b972413325b199041527564aa14c9dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:25.229 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:25 smithi086 podman[149891]: 2024-09-05 07:09:25.229231957 +0000 UTC m=+0.367372164 container init 8c3d4fa43c9735905b25e75c45867d1b972413325b199041527564aa14c9dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T07:09:25.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:24.975+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:25.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:24.975+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:25.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:24.975+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:25.487 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:25 smithi086 podman[149891]: 2024-09-05 07:09:25.232749425 +0000 UTC m=+0.370889630 container start 8c3d4fa43c9735905b25e75c45867d1b972413325b199041527564aa14c9dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-05T07:09:25.487 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:25 smithi086 podman[149891]: 2024-09-05 07:09:25.320737203 +0000 UTC m=+0.458877437 container attach 8c3d4fa43c9735905b25e75c45867d1b972413325b199041527564aa14c9dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T07:09:25.487 INFO:journalctl@ceph.osd.6.smithi086.stdout:Sep 05 07:09:25 smithi086 podman[149891]: 2024-09-05 07:09:25.418025019 +0000 UTC m=+0.556165228 container died 8c3d4fa43c9735905b25e75c45867d1b972413325b199041527564aa14c9dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T07:09:25.638 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:25.332+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:25.638 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:25.332+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:25.638 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:25.332+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:25.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:25.419+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:25.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:25.419+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:25.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:25.419+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:25.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:25.638+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:25.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:25.638+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:25.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:25.638+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:25.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:25 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:25.638+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:25.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:25.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:25.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:25.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:25.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:25.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:25.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:25 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:25.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:26.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.6.service' 2024-09-05T07:09:26.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:26.293 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-05T07:09:26.293 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-05T07:09:26.293 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.7 2024-09-05T07:09:26.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:25.946+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:26.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:25.946+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:26.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:25.946+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:26.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:25 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:25.946+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:26.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:26.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:26.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:26.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:26.590 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:26.330+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:26.606 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 systemd[1]: Stopping Ceph osd.7 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:26.606 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.422+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:26.606 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.422+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:26.607 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.422+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:26.607 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.605+0000 7fd5dc00d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:26.607 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.605+0000 7fd5dc00d640 -1 osd.7 134 *** Got signal Terminated *** 2024-09-05T07:09:26.607 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:26.605+0000 7fd5dc00d640 -1 osd.7 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:26.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:26 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:26.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:26.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:26 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:26.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:26.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:26 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:26.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:26.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:26 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:26.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:26.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:26.589+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:26.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:26.589+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:26.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:26.589+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:26.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:26 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:26.589+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:27.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:26.961+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:27.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:26.961+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:27.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:26.961+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:27.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:26 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:26.961+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:27.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:27.588+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:27.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:27.588+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:27.685 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:27.588+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:27.685 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:27.588+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:27.685 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:27.360+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:27.685 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:27.360+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:27.685 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:27 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:27.360+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:27.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:27 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:27.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:27.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:27 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:27.577+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:27.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:27 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:27.577+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:27.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:27 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:27.577+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:27.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:27.390+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:27.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:27.390+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:27.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:27.390+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:28.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:27.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:28.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:27.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:28.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:27.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:28.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:27 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:27.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:28.630 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:28.355+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:28.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:28.355+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:28.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:28.355+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:28.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:28.355+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:28.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:28.351+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:28.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:28.351+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:28.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:28.351+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:28.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:28.630+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:28.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:28.630+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:28.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:28.630+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:28.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:28 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:28.630+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:28.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:28 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:28.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:28.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:28 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:28.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:28.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:28 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:28.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:28.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:28 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:28.593+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:29.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:28.991+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:29.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:28.991+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:29.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:28.991+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:29.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:28.991+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:29.622 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:29.308+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:29.623 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:29.308+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:29.623 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:29.308+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:29.623 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:29.308+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:29.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:29.331+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:29.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:29.331+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:29.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:29.331+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:29.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:29.622+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:29.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:29.622+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:29.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:29.622+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:29.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:29 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:29.622+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:29.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:29.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:29.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:29.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:29.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:29.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:29.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:29.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:29.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:29 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:29.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:30.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:29.978+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:30.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:29.978+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:30.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:29.978+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:30.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:29 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:29.978+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:30.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:30.351+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:30.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:30.351+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:30.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:30.351+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:30.631 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:30.351+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:30.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:30.340+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:30.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:30.340+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:30.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:30.340+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:30.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:30.340+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.919333+0000 front 2024-09-05T07:09:03.919356+0000 (oldest deadline 2024-09-05T07:09:29.819520+0000) 2024-09-05T07:09:30.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:30.631+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:30.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:30.631+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:30.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:30.631+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:30.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:30 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:30.631+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:30.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:30.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:30.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:30.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:30.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:30.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:30.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:30.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:30.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:30 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:30.576+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:31.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:30.982+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:31.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:30.982+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:31.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:30.982+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:31.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:30 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:30.982+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:31.620+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:31.620+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:31.620+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:31.324+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:31.324+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:31.324+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:31.621 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:31.324+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:31.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:31 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:31.387+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:39.414700+0000 front 2024-09-05T07:08:39.414727+0000 (oldest deadline 2024-09-05T07:09:05.314895+0000) 2024-09-05T07:09:31.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:31 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:31.387+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:46.416424+0000 front 2024-09-05T07:08:46.416134+0000 (oldest deadline 2024-09-05T07:09:12.316178+0000) 2024-09-05T07:09:31.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:31 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:31.387+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.617612+0000 front 2024-09-05T07:08:57.617699+0000 (oldest deadline 2024-09-05T07:09:21.018181+0000) 2024-09-05T07:09:31.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:31 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7[51918]: 2024-09-05T07:09:31.387+0000 7fd5d461b640 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.919333+0000 front 2024-09-05T07:09:03.919356+0000 (oldest deadline 2024-09-05T07:09:29.819520+0000) 2024-09-05T07:09:31.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:31 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:31.620+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:31.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:31.625+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:31.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:31.625+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:31.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:31.625+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:31.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:31.625+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:31.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:31 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:31.625+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:32.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:32 smithi096 podman[76615]: 2024-09-05 07:09:32.03205851 +0000 UTC m=+5.538912156 container died 9ee75ac65ee97e1a715dfe78469b6248815edbbc0f7b4b07966b6b1f4ce0a411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T07:09:32.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:32.028+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:32.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:32.028+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:32.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:32.028+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:32.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:32.028+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:32.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:32.028+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:32.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:32.370+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:32.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:32.370+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:32.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:32.370+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:32.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:32.370+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:32.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:32.580+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:32.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:32.580+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:32.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:32.580+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:32.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:32 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:32.580+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:32.698 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:32 smithi096 podman[76615]: 2024-09-05 07:09:32.40584354 +0000 UTC m=+5.912697186 container cleanup 9ee75ac65ee97e1a715dfe78469b6248815edbbc0f7b4b07966b6b1f4ce0a411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T07:09:32.698 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:32 smithi096 bash[76615]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7 2024-09-05T07:09:32.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:32.614+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:32.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:32.614+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:32.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:32.614+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:32.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:32.614+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:32.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:32 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:32.614+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:33.005 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:32 smithi096 podman[76629]: 2024-09-05 07:09:32.698928901 +0000 UTC m=+0.664060872 container remove 9ee75ac65ee97e1a715dfe78469b6248815edbbc0f7b4b07966b6b1f4ce0a411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:32.90862226 +0000 UTC m=+0.015505901 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:33.058548947 +0000 UTC m=+0.165432569 container create 3a3faee9e5fa006cbdf2f4da0993f647c35c37c841048e38813294caba2ec9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:33 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:33.076+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:33 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:33.076+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:33 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:33.076+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:33 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:33.076+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:33.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:33 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:33.076+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:33.413+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:33.413+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:33.413+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:33.413+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:33.617+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:33.617+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:33.683 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:33.617+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:33.684 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:33 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:33.617+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:33.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:33.311005691 +0000 UTC m=+0.417889328 container init 3a3faee9e5fa006cbdf2f4da0993f647c35c37c841048e38813294caba2ec9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7-deactivate, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-05T07:09:33.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:33.314821866 +0000 UTC m=+0.421705489 container start 3a3faee9e5fa006cbdf2f4da0993f647c35c37c841048e38813294caba2ec9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:33.807 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:33.394207414 +0000 UTC m=+0.501091051 container attach 3a3faee9e5fa006cbdf2f4da0993f647c35c37c841048e38813294caba2ec9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-05T07:09:33.808 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 05 07:09:33 smithi096 podman[76721]: 2024-09-05 07:09:33.503780677 +0000 UTC m=+0.610664302 container died 3a3faee9e5fa006cbdf2f4da0993f647c35c37c841048e38813294caba2ec9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T07:09:33.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:33 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:33.620+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:33.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:33 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:33.620+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:33.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:33 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:33.620+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:33.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:33 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:33.620+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:33.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:33 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:33.620+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:34.131 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.033+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:34.131 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.033+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:34.131 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.033+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:34.131 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.033+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:34.131 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.033+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:34.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.7.service' 2024-09-05T07:09:34.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:34.231 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-05T07:09:34.231 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-05T07:09:34.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.8 2024-09-05T07:09:34.526 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 systemd[1]: Stopping Ceph osd.8 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:34.526 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.526+0000 7f0aa4fd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:34.526 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.526+0000 7f0aa4fd7640 -1 osd.8 134 *** Got signal Terminated *** 2024-09-05T07:09:34.527 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.526+0000 7f0aa4fd7640 -1 osd.8 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:34.527 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:34.449+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:34.527 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:34.449+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:34.527 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:34.449+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:34.527 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:34.449+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:34.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.584+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:34.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.584+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:34.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.584+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:34.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:34 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:34.584+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:34.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:34.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:34.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:34.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:34.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:34.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:34.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:34.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:34.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:34 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:34.586+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:35.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:35.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:35.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:35.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:35.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:34 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:34.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:35.834 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:35.485+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:35.834 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:35.485+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:35.485+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:35.485+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:35.585+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:35.585+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:35.585+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:35.835 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:35 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:35.585+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:35.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:35.566+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:35.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:35.566+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:35.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:35.566+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:35.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:35.566+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:35.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:35 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:35.566+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:36.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:35.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:36.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:35.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:36.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:35.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:36.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:35.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:36.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:35 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:35.995+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:36.850 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:36.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:36.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:36.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:36.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:36.482+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:36.482+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:36.482+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:36.851 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:36.482+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:36.931 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:36.560+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:36.931 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:36.560+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:36.931 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:36.560+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:36.931 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:36.560+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:36.931 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:36 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:36.560+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:37.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.011+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:37.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.011+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:37.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.011+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:37.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.011+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:37.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.011+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:37.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:37.610+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:37.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:37.610+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:37.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:37.610+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:37.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:37.610+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:37.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:37 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:37.610+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:37.526+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:37.526+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:37.526+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:37.526+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:37.567+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:37.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:37.567+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:37.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:37.567+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:37.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:37.567+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:37.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:37 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:37.567+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.092367+0000 front 2024-09-05T07:09:14.092386+0000 (oldest deadline 2024-09-05T07:09:36.992563+0000) 2024-09-05T07:09:38.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:38.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:38.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:38.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:38.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:37 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:37.986+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:38.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:38.554+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:38.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:38.554+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:38.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:38.554+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:38.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:38.554+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:38.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:38.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.785281+0000 front 2024-09-05T07:08:31.785141+0000 (oldest deadline 2024-09-05T07:08:57.585545+0000) 2024-09-05T07:09:38.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:38.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.787420+0000 front 2024-09-05T07:08:42.787450+0000 (oldest deadline 2024-09-05T07:09:04.987696+0000) 2024-09-05T07:09:38.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:38.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.288507+0000 front 2024-09-05T07:08:50.288605+0000 (oldest deadline 2024-09-05T07:09:14.988634+0000) 2024-09-05T07:09:38.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:38.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:58.390359+0000 front 2024-09-05T07:08:58.390212+0000 (oldest deadline 2024-09-05T07:09:21.890010+0000) 2024-09-05T07:09:38.934 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:38 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8[53209]: 2024-09-05T07:09:38.565+0000 7f0a9d5e5640 -1 osd.8 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.092367+0000 front 2024-09-05T07:09:14.092386+0000 (oldest deadline 2024-09-05T07:09:36.992563+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:38.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:38 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:38.639+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:39.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:38 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:38.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:39.906 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.642+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:39.906 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.643+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:39.906 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.643+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:39.907 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.643+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:39.907 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.643+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:39.907 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:39 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:39.643+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:39.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:39 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:39.580+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:39.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:39 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:39.580+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:39.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:39 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:39.580+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:39.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:39 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:39.580+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:39.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:39 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:39.580+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:40.252 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:39 smithi115 podman[78572]: 2024-09-05 07:09:39.962548883 +0000 UTC m=+5.540453443 container died 01724e5f53b305f9a0fdf5cd3b31048a56fbab23e265a7feddaa6d20a8e03947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.004+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.004+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.004+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.004+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.004+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:40.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.005+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:40.583 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:40 smithi115 podman[78572]: 2024-09-05 07:09:40.334589166 +0000 UTC m=+5.912493726 container cleanup 01724e5f53b305f9a0fdf5cd3b31048a56fbab23e265a7feddaa6d20a8e03947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:09:40.584 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:40 smithi115 bash[78572]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:40 smithi115 podman[78601]: 2024-09-05 07:09:40.660799213 +0000 UTC m=+0.685609009 container remove 01724e5f53b305f9a0fdf5cd3b31048a56fbab23e265a7feddaa6d20a8e03947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:40.583+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:40.583+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:40.583+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:40.583+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:40.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:40.583+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:40.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:40.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:40.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:40.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:40.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:40.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:40 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:40.621+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:41.281 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:40.908735016 +0000 UTC m=+0.015750201 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:41.281 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:41.028480286 +0000 UTC m=+0.135495471 container create 2fd79c256781abf50fc5285e4ad1574007c2f33bb40c988609b1fd67b8620654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8-deactivate, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T07:09:41.281 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:41.281209551 +0000 UTC m=+0.388224741 container init 2fd79c256781abf50fc5285e4ad1574007c2f33bb40c988609b1fd67b8620654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:41.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:40 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:40.974+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:41.587 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:41.284683893 +0000 UTC m=+0.391699080 container start 2fd79c256781abf50fc5285e4ad1574007c2f33bb40c988609b1fd67b8620654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:09:41.587 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:41.372671557 +0000 UTC m=+0.479686744 container attach 2fd79c256781abf50fc5285e4ad1574007c2f33bb40c988609b1fd67b8620654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:41.587 INFO:journalctl@ceph.osd.8.smithi115.stdout:Sep 05 07:09:41 smithi115 podman[78690]: 2024-09-05 07:09:41.471295374 +0000 UTC m=+0.578310559 container died 2fd79c256781abf50fc5285e4ad1574007c2f33bb40c988609b1fd67b8620654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-05T07:09:41.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:41 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:41.587+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:41.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:41 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:41.587+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:41.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:41 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:41.587+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:41.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:41 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:41.587+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:41.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:41 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:41.587+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:41.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:41.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:41.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:41.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:41.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:41.988 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:41 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:41.655+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:42.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.8.service' 2024-09-05T07:09:42.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:42.112 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-05T07:09:42.113 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-05T07:09:42.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.9 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:42.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:41 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:41.948+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:42.467 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 systemd[1]: Stopping Ceph osd.9 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:42.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.466+0000 7fc9fb6f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:42.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.466+0000 7fc9fb6f2640 -1 osd.9 134 *** Got signal Terminated *** 2024-09-05T07:09:42.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.466+0000 7fc9fb6f2640 -1 osd.9 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:42.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:42.737 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:42.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:42.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:42.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:42.738 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:42 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:42.623+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:42.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:42 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:42.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:42.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:42 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:42.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:42.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:42 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:42.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:42.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:42 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:42.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:42.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:42 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:42.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:43.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:42 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:42.969+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:43.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:43 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:43.657+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:43.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:43 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:43.657+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:43.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:43 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:43.657+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:43.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:43 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:43.657+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:43.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:43 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:43.657+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:43.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:43 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:43.606+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:44.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:44.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:44.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:44.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:44.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:44.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:43 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:43.963+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:44.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:44.664+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:44.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:44.664+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:44.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:44.664+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:44.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:44.664+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:44.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:44.664+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:44.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:44 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:44.611+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:45.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:45.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:45.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:45.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:45.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:45.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:44 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:44.952+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:45.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:45.615+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:45.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:45.615+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:45.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:45.615+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:45.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:45.615+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:45.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:45 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:45.615+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:45.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:45 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:45.631+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:46.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:45 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:45.918+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:46.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:46 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:46.609+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:31.408134+0000 front 2024-09-05T07:08:31.408213+0000 (oldest deadline 2024-09-05T07:09:00.208935+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:40.209555+0000 front 2024-09-05T07:08:40.209613+0000 (oldest deadline 2024-09-05T07:09:03.709445+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.111144+0000 front 2024-09-05T07:08:50.111180+0000 (oldest deadline 2024-09-05T07:09:11.711767+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:57.513386+0000 front 2024-09-05T07:08:57.513346+0000 (oldest deadline 2024-09-05T07:09:19.813495+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.714472+0000 front 2024-09-05T07:09:05.714286+0000 (oldest deadline 2024-09-05T07:09:28.614313+0000) 2024-09-05T07:09:46.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:46 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9[62413]: 2024-09-05T07:09:46.597+0000 7fc9f7d08640 -1 osd.9 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.816559+0000 front 2024-09-05T07:09:14.816652+0000 (oldest deadline 2024-09-05T07:09:38.316412+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:47.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:46 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:46.933+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:47.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:47 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:47.589+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:48.162 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:47 smithi086 podman[150020]: 2024-09-05 07:09:47.894856706 +0000 UTC m=+5.580504299 container died bf8e969ffc5387aee8500f91a24fdbe7f8b29f3418b0d887db44bdf1611a9e92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-05T07:09:48.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:48.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:48.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:48.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:48.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:48.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:48.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:47 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:47.901+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:48.487 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:48 smithi086 podman[150020]: 2024-09-05 07:09:48.336515733 +0000 UTC m=+6.022163313 container cleanup bf8e969ffc5387aee8500f91a24fdbe7f8b29f3418b0d887db44bdf1611a9e92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T07:09:48.487 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:48 smithi086 bash[150020]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:48.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:48.639+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:48.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:48 smithi086 podman[150038]: 2024-09-05 07:09:48.646219285 +0000 UTC m=+0.753495115 container remove bf8e969ffc5387aee8500f91a24fdbe7f8b29f3418b0d887db44bdf1611a9e92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:49.248 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:48 smithi086 podman[150125]: 2024-09-05 07:09:48.893591691 +0000 UTC m=+0.015743117 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:49.249 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150125]: 2024-09-05 07:09:49.030289126 +0000 UTC m=+0.152440552 container create b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-05T07:09:49.249 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150125]: 2024-09-05 07:09:49.249091168 +0000 UTC m=+0.371242593 container init b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:49.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:48 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:48.865+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:49.736 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150125]: 2024-09-05 07:09:49.252628337 +0000 UTC m=+0.374779768 container start b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T07:09:49.736 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150125]: 2024-09-05 07:09:49.340872689 +0000 UTC m=+0.463024123 container attach b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:49.736 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150125]: 2024-09-05 07:09:49.446950579 +0000 UTC m=+0.569102018 container died b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:49.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:49 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:49.636+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:49.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.9.service' 2024-09-05T07:09:49.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 podman[150149]: 2024-09-05 07:09:49.952457947 +0000 UTC m=+0.499146209 container remove b2f1eddde39de39094f971b1fdf3d4805a715b1af911aa88cfd73b3bd62c1396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-05T07:09:49.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.9.service: Deactivated successfully. 2024-09-05T07:09:49.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 systemd[1]: Stopped Ceph osd.9 for 5a055634-6b50-11ef-bcd6-c7b262605968. 2024-09-05T07:09:49.987 INFO:journalctl@ceph.osd.9.smithi086.stdout:Sep 05 07:09:49 smithi086 systemd[1]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.9.service: Consumed 3min 34.188s CPU time. 2024-09-05T07:09:50.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:50.009 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-05T07:09:50.009 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-05T07:09:50.009 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.10 2024-09-05T07:09:50.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:50.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:49 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:49.909+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:50.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 systemd[1]: Stopping Ceph osd.10 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:50.807 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.307+0000 7fe606fa2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:50.807 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.307+0000 7fe606fa2640 -1 osd.10 134 *** Got signal Terminated *** 2024-09-05T07:09:50.807 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.307+0000 7fe606fa2640 -1 osd.10 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:50.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:50 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:50.638+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:51.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:51.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:51.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:51.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:51.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:51.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:51.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:50 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:50.953+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:51.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:51 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:51.631+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:52.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:51 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:51.965+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:52.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:52.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:52.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:52.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:52.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:52.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:52 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:52.628+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:53.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:53.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:53.308 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:53.309 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:53.309 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:53.309 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:53.309 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:52 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:52.935+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:53.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:53 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:53.624+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:54.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:53 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:53.971+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:54.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:54 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:54.629+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:32.332078+0000 front 2024-09-05T07:08:32.331991+0000 (oldest deadline 2024-09-05T07:08:58.232054+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:41.733040+0000 front 2024-09-05T07:08:41.733105+0000 (oldest deadline 2024-09-05T07:09:07.633276+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:50.534275+0000 front 2024-09-05T07:08:50.534383+0000 (oldest deadline 2024-09-05T07:09:16.434251+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-09-05T07:08:56.434831+0000 front 2024-09-05T07:08:56.434904+0000 (oldest deadline 2024-09-05T07:09:25.235180+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:05.235560+0000 front 2024-09-05T07:09:05.235532+0000 (oldest deadline 2024-09-05T07:09:31.135534+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.636548+0000 front 2024-09-05T07:09:14.636568+0000 (oldest deadline 2024-09-05T07:09:38.136425+0000) 2024-09-05T07:09:55.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:54 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10[58464]: 2024-09-05T07:09:54.990+0000 7fe6035b8640 -1 osd.10 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:23.238183+0000 front 2024-09-05T07:09:23.238097+0000 (oldest deadline 2024-09-05T07:09:46.738012+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:55.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:55.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:55 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:55.608+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:09:56.057 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:55 smithi096 podman[76835]: 2024-09-05 07:09:55.804789983 +0000 UTC m=+5.594494684 container died deb301d36b671e8f19b001df6f7ab3017f9457b294d0c5c983170056dcbfa72e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T07:09:56.557 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76835]: 2024-09-05 07:09:56.148839934 +0000 UTC m=+5.938544624 container cleanup deb301d36b671e8f19b001df6f7ab3017f9457b294d0c5c983170056dcbfa72e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-05T07:09:56.557 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 bash[76835]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10 2024-09-05T07:09:56.557 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76850]: 2024-09-05 07:09:56.333583386 +0000 UTC m=+0.524552583 container remove deb301d36b671e8f19b001df6f7ab3017f9457b294d0c5c983170056dcbfa72e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:56.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:56 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:56.613+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:09:57.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76938]: 2024-09-05 07:09:56.72603104 +0000 UTC m=+0.106854905 container create d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:09:57.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76938]: 2024-09-05 07:09:56.634905908 +0000 UTC m=+0.015729784 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:09:57.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76938]: 2024-09-05 07:09:56.928300834 +0000 UTC m=+0.309124699 container init d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T07:09:57.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:56 smithi096 podman[76938]: 2024-09-05 07:09:56.931683085 +0000 UTC m=+0.312506959 container start d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, RELEASE=squid-91c8a7e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T07:09:57.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:57 smithi096 podman[76938]: 2024-09-05 07:09:57.011499562 +0000 UTC m=+0.392323432 container attach d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T07:09:57.307 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:57 smithi096 podman[76938]: 2024-09-05 07:09:57.118749508 +0000 UTC m=+0.499573375 container died d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-05T07:09:57.670 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 05 07:09:57 smithi096 podman[76961]: 2024-09-05 07:09:57.656532338 +0000 UTC m=+0.531567204 container remove d968b49be9e735161cd25361ea94bfd1165cc557a609e633c40733042e3aa783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-10-deactivate, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-05T07:09:57.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.10.service' 2024-09-05T07:09:57.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:09:57.714 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-05T07:09:57.714 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-05T07:09:57.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.11 2024-09-05T07:09:57.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:57.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:57.606+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:09:57.935 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:57 smithi115 systemd[1]: Stopping Ceph osd.11 for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:09:58.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.051+0000 7f3f2add3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:09:58.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.051+0000 7f3f2add3640 -1 osd.11 134 *** Got signal Terminated *** 2024-09-05T07:09:58.433 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.051+0000 7f3f2add3640 -1 osd.11 134 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:58.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:58 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:58.561+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:09:59.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:09:59 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:09:59.605+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:10:00.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:10:00.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:00 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:00.612+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6820 osd.8 since back 2024-09-05T07:09:34.676566+0000 front 2024-09-05T07:09:34.676593+0000 (oldest deadline 2024-09-05T07:09:59.976608+0000) 2024-09-05T07:10:01.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:10:01.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:10:01.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:10:01.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:10:01.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:10:01.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:10:01.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:10:01.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:01 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:01.635+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6820 osd.8 since back 2024-09-05T07:09:34.676566+0000 front 2024-09-05T07:09:34.676593+0000 (oldest deadline 2024-09-05T07:09:59.976608+0000) 2024-09-05T07:10:02.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-09-05T07:08:33.467671+0000 front 2024-09-05T07:08:33.467638+0000 (oldest deadline 2024-09-05T07:08:57.567741+0000) 2024-09-05T07:10:02.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-05T07:08:42.869057+0000 front 2024-09-05T07:08:42.868976+0000 (oldest deadline 2024-09-05T07:09:07.568986+0000) 2024-09-05T07:10:02.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2024-09-05T07:08:47.569415+0000 front 2024-09-05T07:08:47.569442+0000 (oldest deadline 2024-09-05T07:09:12.869580+0000) 2024-09-05T07:10:02.933 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-05T07:09:03.371854+0000 front 2024-09-05T07:09:03.371868+0000 (oldest deadline 2024-09-05T07:09:28.071937+0000) 2024-09-05T07:10:02.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2024-09-05T07:09:14.373859+0000 front 2024-09-05T07:09:14.373893+0000 (oldest deadline 2024-09-05T07:09:39.073999+0000) 2024-09-05T07:10:02.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-09-05T07:09:22.375551+0000 front 2024-09-05T07:09:22.375634+0000 (oldest deadline 2024-09-05T07:09:45.875510+0000) 2024-09-05T07:10:02.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-05T07:09:29.376466+0000 front 2024-09-05T07:09:29.376387+0000 (oldest deadline 2024-09-05T07:09:54.676231+0000) 2024-09-05T07:10:02.934 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:02 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11[59760]: 2024-09-05T07:10:02.676+0000 7f3f233e1640 -1 osd.11 134 heartbeat_check: no reply from 172.21.15.115:6820 osd.8 since back 2024-09-05T07:09:34.676566+0000 front 2024-09-05T07:09:34.676593+0000 (oldest deadline 2024-09-05T07:09:59.976608+0000) 2024-09-05T07:10:03.820 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:03 smithi115 podman[78793]: 2024-09-05 07:10:03.565096048 +0000 UTC m=+5.644164505 container died e8670243ec6534628be6a3f14b17e30b3fac0eb2b08fef6e4717692b7b77d994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e) 2024-09-05T07:10:04.183 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:03 smithi115 podman[78793]: 2024-09-05 07:10:03.918318979 +0000 UTC m=+5.997387478 container cleanup e8670243ec6534628be6a3f14b17e30b3fac0eb2b08fef6e4717692b7b77d994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T07:10:04.183 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:03 smithi115 bash[78793]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11 2024-09-05T07:10:04.525 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78808]: 2024-09-05 07:10:04.211130376 +0000 UTC m=+0.645186673 container remove e8670243ec6534628be6a3f14b17e30b3fac0eb2b08fef6e4717692b7b77d994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T07:10:04.839 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.428606879 +0000 UTC m=+0.015893893 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T07:10:04.839 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.537052282 +0000 UTC m=+0.124339294 container create 8bb5597958b03f99519f832d3860faa6c8aaffe00ab8b567154af81afb9deeaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T07:10:04.839 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.764413074 +0000 UTC m=+0.351700096 container init 8bb5597958b03f99519f832d3860faa6c8aaffe00ab8b567154af81afb9deeaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T07:10:04.840 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.767830759 +0000 UTC m=+0.355117771 container start 8bb5597958b03f99519f832d3860faa6c8aaffe00ab8b567154af81afb9deeaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T07:10:04.840 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.839218821 +0000 UTC m=+0.426505835 container attach 8bb5597958b03f99519f832d3860faa6c8aaffe00ab8b567154af81afb9deeaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T07:10:05.183 INFO:journalctl@ceph.osd.11.smithi115.stdout:Sep 05 07:10:04 smithi115 podman[78899]: 2024-09-05 07:10:04.953945606 +0000 UTC m=+0.541232618 container died 8bb5597958b03f99519f832d3860faa6c8aaffe00ab8b567154af81afb9deeaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T07:10:05.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@osd.11.service' 2024-09-05T07:10:05.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:05.537 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-05T07:10:05.537 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-05T07:10:05.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.a 2024-09-05T07:10:05.830 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:05 smithi086 systemd[1]: Stopping Ceph mds.a for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:06.237 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:05 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: 2024-09-05T07:10:05.829+0000 7fe2fc092640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:06.237 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:05 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a[69710]: 2024-09-05T07:10:05.829+0000 7fe2fc092640 -1 mds.a *** got signal Terminated *** 2024-09-05T07:10:07.895 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:07 smithi086 podman[150230]: 2024-09-05 07:10:07.566295119 +0000 UTC m=+1.834712685 container died 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:10:07.895 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:07 smithi086 podman[150230]: 2024-09-05 07:10:07.895853806 +0000 UTC m=+2.164271370 container cleanup 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:08.242 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:07 smithi086 bash[150230]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a 2024-09-05T07:10:08.242 INFO:journalctl@ceph.mds.a.smithi086.stdout:Sep 05 07:10:08 smithi086 podman[150242]: 2024-09-05 07:10:08.105650521 +0000 UTC m=+0.537823192 container remove 113a8c7c1239677bc184e78f69819000df32f269519b33c4523ad09345bae92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-05T07:10:08.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.a.service' 2024-09-05T07:10:08.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:08.360 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-05T07:10:08.360 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-05T07:10:08.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.d 2024-09-05T07:10:08.683 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:08 smithi086 systemd[1]: Stopping Ceph mds.d for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:08.987 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d[68521]: 2024-09-05T07:10:08.682+0000 7fba7657c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:08.987 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:08 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d[68521]: 2024-09-05T07:10:08.682+0000 7fba7657c640 -1 mds.d *** got signal Terminated *** 2024-09-05T07:10:12.142 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:11 smithi086 podman[150354]: 2024-09-05 07:10:11.876074744 +0000 UTC m=+3.291099140 container died f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T07:10:12.487 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:12 smithi086 podman[150354]: 2024-09-05 07:10:12.232999891 +0000 UTC m=+3.648024287 container cleanup f72d40b29bccc1e87b2d6d4be8965aec45ec5e186300964ee15d0a963869596b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-05T07:10:12.487 INFO:journalctl@ceph.mds.d.smithi086.stdout:Sep 05 07:10:12 smithi086 bash[150354]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-d 2024-09-05T07:10:12.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.d.service' 2024-09-05T07:10:12.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:12.699 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-05T07:10:12.699 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-05T07:10:12.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.g 2024-09-05T07:10:13.237 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:12 smithi086 systemd[1]: Stopping Ceph mds.g for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:13.237 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: 2024-09-05T07:10:13.027+0000 7f6e80b8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:13.237 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:13 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g[69335]: 2024-09-05T07:10:13.027+0000 7f6e80b8e640 -1 mds.g *** got signal Terminated *** 2024-09-05T07:10:15.831 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:15 smithi086 podman[150475]: 2024-09-05 07:10:15.563150221 +0000 UTC m=+2.641119460 container died 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:16.237 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:16 smithi086 podman[150475]: 2024-09-05 07:10:16.022958502 +0000 UTC m=+3.100927742 container cleanup 032f3ad3e747dee0c6119c17c7d7574881c1194f3daa9b6ea333ec6c26ee03e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_CLEAN=True) 2024-09-05T07:10:16.237 INFO:journalctl@ceph.mds.g.smithi086.stdout:Sep 05 07:10:16 smithi086 bash[150475]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-g 2024-09-05T07:10:16.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.g.service' 2024-09-05T07:10:16.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:16.473 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-05T07:10:16.474 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-05T07:10:16.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.j 2024-09-05T07:10:16.803 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:16 smithi086 systemd[1]: Stopping Ceph mds.j for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:17.237 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j[68127]: 2024-09-05T07:10:16.802+0000 7fb816d37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:17.237 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:16 smithi086 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j[68127]: 2024-09-05T07:10:16.802+0000 7fb816d37640 -1 mds.j *** got signal Terminated *** 2024-09-05T07:10:20.567 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:20 smithi086 podman[150610]: 2024-09-05 07:10:20.296444422 +0000 UTC m=+3.603749597 container died 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T07:10:20.987 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:20 smithi086 podman[150610]: 2024-09-05 07:10:20.75232477 +0000 UTC m=+4.059629933 container cleanup 4ddc3b180a387e80044e2eac5185dbb472c1b91345e60d9deca0b056590abab4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T07:10:20.987 INFO:journalctl@ceph.mds.j.smithi086.stdout:Sep 05 07:10:20 smithi086 bash[150610]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-j 2024-09-05T07:10:21.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.j.service' 2024-09-05T07:10:21.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:21.209 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-05T07:10:21.209 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-05T07:10:21.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.b 2024-09-05T07:10:21.489 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:21 smithi096 systemd[1]: Stopping Ceph mds.b for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:21.743 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T07:10:21.489+0000 7f42409fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:21.743 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:21 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b[64068]: 2024-09-05T07:10:21.489+0000 7f42409fd640 -1 mds.b *** got signal Terminated *** 2024-09-05T07:10:23.780 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:23 smithi096 podman[77040]: 2024-09-05 07:10:23.497960011 +0000 UTC m=+2.100160945 container died 22f4f023cf608e374640ebd05e485ed06647adbdf020c40f5bdc24687b165ca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:24.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:23 smithi096 podman[77040]: 2024-09-05 07:10:23.888566639 +0000 UTC m=+2.490767572 container cleanup 22f4f023cf608e374640ebd05e485ed06647adbdf020c40f5bdc24687b165ca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:24.057 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 05 07:10:23 smithi096 bash[77040]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-b 2024-09-05T07:10:24.311 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.b.service' 2024-09-05T07:10:24.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:24.349 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-05T07:10:24.349 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-05T07:10:24.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.e 2024-09-05T07:10:24.685 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:24 smithi096 systemd[1]: Stopping Ceph mds.e for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:25.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: 2024-09-05T07:10:24.685+0000 7fbb8d8f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:25.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:24 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e[63467]: 2024-09-05T07:10:24.685+0000 7fbb8d8f1640 -1 mds.e *** got signal Terminated *** 2024-09-05T07:10:27.776 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:27 smithi096 podman[77172]: 2024-09-05 07:10:27.500069664 +0000 UTC m=+2.932401886 container died 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T07:10:28.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:27 smithi096 podman[77172]: 2024-09-05 07:10:27.851024955 +0000 UTC m=+3.283357183 container cleanup 8d73361f83ac09884dc712192d35b5a15fab662555225ccc3bfa3f8f1f34b607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T07:10:28.057 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 05 07:10:27 smithi096 bash[77172]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-e 2024-09-05T07:10:28.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.e.service' 2024-09-05T07:10:28.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:28.313 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-05T07:10:28.313 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-05T07:10:28.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.h 2024-09-05T07:10:28.557 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:28 smithi096 systemd[1]: Stopping Ceph mds.h for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:29.057 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: 2024-09-05T07:10:28.654+0000 7fc709c1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:29.057 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:28 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h[63768]: 2024-09-05T07:10:28.654+0000 7fc709c1a640 -1 mds.h *** got signal Terminated *** 2024-09-05T07:10:31.771 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:31 smithi096 podman[77294]: 2024-09-05 07:10:31.502841215 +0000 UTC m=+2.977772660 container died 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T07:10:32.057 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:31 smithi096 podman[77294]: 2024-09-05 07:10:31.863033268 +0000 UTC m=+3.337964758 container cleanup 136a750f00993090e0a5d66100fa81ae9758311fa9e0e5e46fcc44a15d27da8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:10:32.057 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 05 07:10:31 smithi096 bash[77294]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-h 2024-09-05T07:10:32.285 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.h.service' 2024-09-05T07:10:32.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:32.319 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-05T07:10:32.319 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-05T07:10:32.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.k 2024-09-05T07:10:32.557 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:32 smithi096 systemd[1]: Stopping Ceph mds.k for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:33.057 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: 2024-09-05T07:10:32.649+0000 7fb632755640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:33.057 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:32 smithi096 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k[63158]: 2024-09-05T07:10:32.649+0000 7fb632755640 -1 mds.k *** got signal Terminated *** 2024-09-05T07:10:35.807 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:35 smithi096 podman[77413]: 2024-09-05 07:10:35.498662936 +0000 UTC m=+2.962485371 container died 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0) 2024-09-05T07:10:36.100 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:35 smithi096 podman[77413]: 2024-09-05 07:10:35.883469776 +0000 UTC m=+3.347292209 container cleanup 440a1bb0b310f48971311da2a9e2c5b048f5380148f19f00441162e9679a688f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-05T07:10:36.100 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 05 07:10:35 smithi096 bash[77413]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-k 2024-09-05T07:10:36.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.k.service' 2024-09-05T07:10:36.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:36.244 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-05T07:10:36.244 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-05T07:10:36.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.c 2024-09-05T07:10:36.624 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:36 smithi115 systemd[1]: Stopping Ceph mds.c for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:36.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: 2024-09-05T07:10:36.623+0000 7f5d9d00d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:36.933 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:36 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c[64603]: 2024-09-05T07:10:36.623+0000 7f5d9d00d640 -1 mds.c *** got signal Terminated *** 2024-09-05T07:10:39.865 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:39 smithi115 podman[79013]: 2024-09-05 07:10:39.522567264 +0000 UTC m=+3.083319250 container died 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-05T07:10:39.865 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:39 smithi115 podman[79013]: 2024-09-05 07:10:39.865279023 +0000 UTC m=+3.426031007 container cleanup 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T07:10:40.182 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:39 smithi115 bash[79013]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c 2024-09-05T07:10:40.183 INFO:journalctl@ceph.mds.c.smithi115.stdout:Sep 05 07:10:40 smithi115 podman[79037]: 2024-09-05 07:10:40.058051341 +0000 UTC m=+0.531818654 container remove 534a97c1e314f728afdb3e27cd89f2f4d2021668b7bc5853cf0b763011b8cb8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e) 2024-09-05T07:10:40.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.c.service' 2024-09-05T07:10:40.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:40.322 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-05T07:10:40.322 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-05T07:10:40.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.f 2024-09-05T07:10:40.683 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:40 smithi115 systemd[1]: Stopping Ceph mds.f for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:41.183 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: 2024-09-05T07:10:40.693+0000 7f5804e52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:41.183 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:40 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f[64302]: 2024-09-05T07:10:40.693+0000 7f5804e52640 -1 mds.f *** got signal Terminated *** 2024-09-05T07:10:43.926 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:43 smithi115 podman[79146]: 2024-09-05 07:10:43.585705348 +0000 UTC m=+3.050527330 container died ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T07:10:43.926 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:43 smithi115 podman[79146]: 2024-09-05 07:10:43.92704977 +0000 UTC m=+3.391871746 container cleanup ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:44.183 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:43 smithi115 bash[79146]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f 2024-09-05T07:10:44.183 INFO:journalctl@ceph.mds.f.smithi115.stdout:Sep 05 07:10:44 smithi115 podman[79158]: 2024-09-05 07:10:44.128361386 +0000 UTC m=+0.535978501 container remove ff33ea77cdd1e815662a07f13c22c4e7204f8b5151170aff4a6af61aa8efde2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:44.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.f.service' 2024-09-05T07:10:44.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:44.389 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-05T07:10:44.389 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-05T07:10:44.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.i 2024-09-05T07:10:44.683 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:44 smithi115 systemd[1]: Stopping Ceph mds.i for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:45.183 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: 2024-09-05T07:10:44.709+0000 7f1ef899c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:45.183 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:44 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i[64902]: 2024-09-05T07:10:44.709+0000 7f1ef899c640 -1 mds.i *** got signal Terminated *** 2024-09-05T07:10:47.789 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:47 smithi115 podman[79266]: 2024-09-05 07:10:47.510956851 +0000 UTC m=+2.908519714 container died 2bce40969aea963b4bbc5fd6b57f68f5f926d5f8aa24432c77cb3372cd750ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:48.131 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:47 smithi115 podman[79266]: 2024-09-05 07:10:47.888626743 +0000 UTC m=+3.286189594 container cleanup 2bce40969aea963b4bbc5fd6b57f68f5f926d5f8aa24432c77cb3372cd750ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T07:10:48.131 INFO:journalctl@ceph.mds.i.smithi115.stdout:Sep 05 07:10:47 smithi115 bash[79266]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-i 2024-09-05T07:10:48.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.i.service' 2024-09-05T07:10:48.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:48.297 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-05T07:10:48.297 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-05T07:10:48.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.l 2024-09-05T07:10:48.433 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:48 smithi115 systemd[1]: Stopping Ceph mds.l for 5a055634-6b50-11ef-bcd6-c7b262605968... 2024-09-05T07:10:48.909 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: 2024-09-05T07:10:48.608+0000 7f7d892ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T07:10:48.909 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:48 smithi115 ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l[63998]: 2024-09-05T07:10:48.608+0000 7f7d892ee640 -1 mds.l *** got signal Terminated *** 2024-09-05T07:10:51.784 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:51 smithi115 podman[79385]: 2024-09-05 07:10:51.516354621 +0000 UTC m=+3.003451449 container died 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-05T07:10:52.142 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:51 smithi115 podman[79385]: 2024-09-05 07:10:51.866822507 +0000 UTC m=+3.353919334 container cleanup 05e4d7cfd4c7b9fa468c869f3cc85f5521755c3ea984cad9f2e505ae7c273388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-05T07:10:52.142 INFO:journalctl@ceph.mds.l.smithi115.stdout:Sep 05 07:10:51 smithi115 bash[79385]: ceph-5a055634-6b50-11ef-bcd6-c7b262605968-mds-l 2024-09-05T07:10:52.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a055634-6b50-11ef-bcd6-c7b262605968@mds.l.service' 2024-09-05T07:10:52.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T07:10:52.304 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-05T07:10:52.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T07:10:52.506 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:10:55.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T07:10:55.353 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:10:57.989 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T07:10:58.191 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:11:00.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:11:00.638 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:11:00.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T07:11:00.700 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-05T07:11:00.703 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi086/crash 2024-09-05T07:11:00.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T07:11:00.744 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T07:11:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-09-05T07:11:00.747 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi096/crash 2024-09-05T07:11:00.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T07:11:00.839 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T07:11:00.840 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-09-05T07:11:00.842 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi115/crash 2024-09-05T07:11:00.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T07:11:00.876 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T07:11:00.877 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-09-05T07:11:00.877 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-05T07:11:00.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-05T07:11:00.925 INFO:tasks.cephadm:Compressing logs... 2024-09-05T07:11:00.925 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T07:11:00.927 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T07:11:00.930 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T07:11:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T07:11:00.960 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-05T07:11:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.0.log 2024-09-05T07:11:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.9.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.a.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.a.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.g.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.6.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr: 84.7% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.d.log 2024-09-05T07:11:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.x.log 2024-09-05T07:11:00.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.j.log 2024-09-05T07:11:00.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.3.log 2024-09-05T07:11:00.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-05T07:11:00.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.3.log: 90.3% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T07:11:00.972 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T07:11:00.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 87.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T07:11:00.972 INFO:teuthology.orchestra.run.smithi086.stderr: 94.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.d.log.gz 2024-09-05T07:11:00.972 INFO:teuthology.orchestra.run.smithi086.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.10.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.1.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.7.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.b.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.k.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.4.log 2024-09-05T07:11:00.978 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.y.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.h.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.b.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.e.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T07:11:00.979 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.i.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.c.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.i.log: /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.11.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.2.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.5.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.l.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.8.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.z.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.c.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.f.log 2024-09-05T07:11:00.980 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T07:11:01.084 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T07:11:01.102 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.5% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T07:11:01.112 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.8% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.z.log.gz 2024-09-05T07:11:01.119 INFO:teuthology.orchestra.run.smithi115.stderr: 82.5% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T07:11:01.124 INFO:teuthology.orchestra.run.smithi096.stderr: 92.7% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.y.log.gz 2024-09-05T07:11:01.172 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T07:11:01.179 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T07:11:01.196 INFO:teuthology.orchestra.run.smithi115.stderr: 87.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T07:11:01.231 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T07:11:01.231 INFO:teuthology.orchestra.run.smithi086.stderr: 94.7% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.j.log.gz 2024-09-05T07:11:01.417 INFO:teuthology.orchestra.run.smithi115.stderr: 90.5% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T07:11:01.417 INFO:teuthology.orchestra.run.smithi115.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T07:11:01.627 INFO:teuthology.orchestra.run.smithi086.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T07:11:02.191 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T07:11:02.363 INFO:teuthology.orchestra.run.smithi096.stderr: 87.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T07:11:02.839 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.a.log.gz 2024-09-05T07:11:03.214 INFO:teuthology.orchestra.run.smithi086.stderr: 94.9% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.g.log.gz 2024-09-05T07:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr: 95.1% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.c.log.gz 2024-09-05T07:11:07.323 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mgr.x.log.gz 2024-09-05T07:11:13.936 INFO:teuthology.orchestra.run.smithi115.stderr: 91.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.c.log.gz 2024-09-05T07:11:20.591 INFO:teuthology.orchestra.run.smithi115.stderr: 95.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.f.log.gz 2024-09-05T07:11:23.936 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mon.b.log.gz 2024-09-05T07:12:14.266 INFO:teuthology.orchestra.run.smithi086.stderr: 94.8% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.a.log.gz 2024-09-05T07:12:52.019 INFO:teuthology.orchestra.run.smithi115.stderr: 94.2% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.i.log.gz 2024-09-05T07:13:12.663 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.8.log.gz 2024-09-05T07:13:23.938 INFO:teuthology.orchestra.run.smithi096.stderr: 95.1% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.e.log.gz 2024-09-05T07:14:03.429 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.2.log.gz 2024-09-05T07:15:00.860 INFO:teuthology.orchestra.run.smithi115.stderr: 91.4% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.11.log.gz 2024-09-05T07:15:40.386 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.0.log.gz 2024-09-05T07:15:47.133 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.6.log.gz 2024-09-05T07:15:48.328 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.5.log.gz 2024-09-05T07:16:07.309 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.9.log.gz 2024-09-05T07:16:37.603 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.l.log.gz 2024-09-05T07:16:37.604 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-05T07:16:37.605 INFO:teuthology.orchestra.run.smithi115.stderr:real 5m36.660s 2024-09-05T07:16:37.605 INFO:teuthology.orchestra.run.smithi115.stderr:user 5m5.752s 2024-09-05T07:16:37.605 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m12.425s 2024-09-05T07:16:55.921 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.10.log.gz 2024-09-05T07:17:16.385 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.1.log.gz 2024-09-05T07:17:18.602 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.3.log.gz 2024-09-05T07:18:50.626 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.7.log.gz 2024-09-05T07:19:13.509 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-osd.4.log.gz 2024-09-05T07:22:05.931 INFO:teuthology.orchestra.run.smithi086.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-05T07:22:05.932 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T07:22:05.933 INFO:teuthology.orchestra.run.smithi086.stderr:real 11m4.991s 2024-09-05T07:22:05.933 INFO:teuthology.orchestra.run.smithi086.stderr:user 6m35.386s 2024-09-05T07:22:05.933 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m17.814s 2024-09-05T07:30:47.856 INFO:teuthology.orchestra.run.smithi096.stderr: 94.4% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.k.log.gz 2024-09-05T07:33:52.389 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.h.log.gz 2024-09-05T07:35:36.823 INFO:teuthology.orchestra.run.smithi096.stderr: 95.5% -- replaced with /var/log/ceph/5a055634-6b50-11ef-bcd6-c7b262605968/ceph-mds.b.log.gz 2024-09-05T07:35:36.825 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-05T07:35:36.825 INFO:teuthology.orchestra.run.smithi096.stderr:real 24m35.879s 2024-09-05T07:35:36.825 INFO:teuthology.orchestra.run.smithi096.stderr:user 8m5.253s 2024-09-05T07:35:36.826 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m25.056s 2024-09-05T07:35:36.826 INFO:tasks.cephadm:Archiving logs... 2024-09-05T07:35:36.827 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi086/log 2024-09-05T07:35:36.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T07:36:44.217 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi096/log 2024-09-05T07:36:44.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T07:37:52.137 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi115/log 2024-09-05T07:37:52.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T07:38:39.135 INFO:tasks.cephadm:Removing cluster... 2024-09-05T07:38:39.136 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force 2024-09-05T07:38:39.447 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:38:39.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force 2024-09-05T07:38:40.325 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:38:41.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a055634-6b50-11ef-bcd6-c7b262605968 --force 2024-09-05T07:38:41.228 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5a055634-6b50-11ef-bcd6-c7b262605968 2024-09-05T07:38:41.854 INFO:tasks.cephadm:Removing cephadm ... 2024-09-05T07:38:41.854 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T07:38:41.874 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T07:38:41.892 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T07:38:41.908 INFO:tasks.cephadm:Teardown complete 2024-09-05T07:38:41.908 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T07:38:41.919 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T07:38:41.919 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T07:38:41.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T07:38:41.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T07:38:42.111 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T07:38:42.122 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T07:38:42.122 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T07:38:42.125 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T07:38:42.128 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T07:38:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-05T07:38:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-05T07:38:42.142 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-05T07:38:42.153 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T07:38:42.153 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-05T07:38:42.153 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +820us[ +852us] +/- 54ms 2024-09-05T07:38:42.153 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1757us[-1757us] +/- 73ms 2024-09-05T07:38:42.153 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +551us[ +583us] +/- 64ms 2024-09-05T07:38:42.154 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +825us[ +827us] +/- 54ms 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1751us[-1749us] +/- 72ms 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +537us[ +537us] +/- 64ms 2024-09-05T07:38:42.175 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +794us[ +791us] +/- 54ms 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1788us[-1791us] +/- 73ms 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +518us[ +515us] +/- 64ms 2024-09-05T07:38:42.180 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T07:38:42.181 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T07:38:42.192 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T07:38:42.192 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T07:38:42.202 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T07:38:42.369 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T07:38:42.501 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T07:38:42.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T07:38:42.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:42.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T07:38:42.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:42.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T07:38:42.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:42.729 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T07:38:42.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T07:38:42.753 INFO:teuthology.task.internal:Duration was 4420.717309 seconds 2024-09-05T07:38:42.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T07:38:42.764 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T07:38:42.764 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T07:38:42.767 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T07:38:42.769 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T07:38:42.837 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T07:38:42.853 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T07:38:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T07:38:43.229 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T07:38:43.229 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T07:38:43.229 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T07:38:43.256 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-05T07:38:43.256 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T07:38:43.292 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-05T07:38:43.293 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T07:38:43.316 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T07:38:43.317 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T07:38:43.319 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T07:38:43.335 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T07:38:43.384 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T07:38:43.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T07:38:43.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T07:38:43.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T07:38:45.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T07:38:45.605 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T07:38:45.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T07:38:45.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T07:38:45.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T07:38:45.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T07:38:45.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.714 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-09-05T07:38:45.754 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-05T07:38:45.776 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-05T07:38:45.800 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:45.820 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:45.851 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T07:38:45.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T07:38:45.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T07:38:45.879 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T07:38:45.880 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi086 2024-09-05T07:38:45.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T07:38:46.147 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi096 2024-09-05T07:38:46.147 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T07:38:46.742 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889968/remote/smithi115 2024-09-05T07:38:46.742 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T07:38:46.787 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T07:38:46.787 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T07:38:46.790 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T07:38:46.793 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T07:38:46.836 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T07:38:46.849 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T07:38:46.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T07:38:46.861 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T07:38:46.861 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T07:38:46.864 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T07:38:46.866 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T07:38:46.879 INFO:teuthology.orchestra.run.smithi086.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 07:38 /home/ubuntu/cephtest 2024-09-05T07:38:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 07:38 /home/ubuntu/cephtest 2024-09-05T07:38:46.893 INFO:teuthology.orchestra.run.smithi115.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 07:38 /home/ubuntu/cephtest 2024-09-05T07:38:46.894 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T07:38:46.906 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T07:38:46.933 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4420.717308759689 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-05T07:38:46.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T07:38:47.014 INFO:teuthology.run:pass